File size: 901 Bytes
1d15ee3 c25b3dd 1c1f77f 1d15ee3 1c1f77f 1d15ee3 1c1f77f 1d15ee3 1c1f77f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
---
library_name: peft
base_model: ai21labs/Jamba-v0.1
license: apache-2.0
datasets:
- mhenrichsen/alpaca_2k_test
tags:
- axolotl
---
# Jambalpaca-v0.1
This is a test run to fine-tune [ai21labs/Jamba-v0.1](https://huggingface.co/ai21labs/Jamba-v0.1) on an A100 80GB GPU using [Axolotl](https://github.com/OpenAccess-AI-Collective/axolotl) and a custom version of [LazyAxolotl](https://colab.research.google.com/drive/1TsDKNo2riwVmU55gjuBgB1AXVtRRfRHW).
I used [mhenrichsen/alpaca_2k_test](https://huggingface.co/datasets/mhenrichsen/alpaca_2k_test) as a dataset.
I had to quantize the base model in 8-bit precision due to how merging models with adapters work. Weirdly enough, I didn't have to do it in a notebook version I created.
I also pushed the adapter so I or someone else could do a better merge.
Let me know if you're interested, I can give you access to Jamba's version of LazyAxolotl. |