Commit
•
a7bdd41
1
Parent(s):
0ac3b98
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,12 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
+
Mamba-2.8b-slimpj is a model using the [Mamba](https://arxiv.org/abs/2312.00752) architecture, with 2.8B parameters, trained for 600B tokens on the SlimPajama dataset.
|
| 5 |
+
|
| 6 |
+
Model code: https://github.com/state-spaces/mamba/tree/main
|
| 7 |
+
|
| 8 |
+
To load the model, follow the installation instruction in the code repo, and then:
|
| 9 |
+
```
|
| 10 |
+
from mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel
|
| 11 |
+
model = MambaLMHeadModel.from_pretrained("state-spaces/mamba-2.8b-slimpj")
|
| 12 |
+
```
|