Update README.md
Browse files
README.md
CHANGED
|
@@ -25,6 +25,8 @@ This Model is a test to combine [Jamba](https://huggingface.co/ai21labs/Jamba-v0
|
|
| 25 |
|
| 26 |
The goal is to developpe and test if this kind of architectures have not too much quality loss for a fast inference.
|
| 27 |
|
|
|
|
|
|
|
| 28 |
|
| 29 |
- **Model type:** Mixture of attention head mixture of depth and mixture of expert 1.58bit linear layers **excepted for attention layer**
|
| 30 |
- **License:** Apache licence 2.0
|
|
|
|
| 25 |
|
| 26 |
The goal is to developpe and test if this kind of architectures have not too much quality loss for a fast inference.
|
| 27 |
|
| 28 |
+
Only 17.8M parameter over 1000 is in bf16 precision
|
| 29 |
+
|
| 30 |
|
| 31 |
- **Model type:** Mixture of attention head mixture of depth and mixture of expert 1.58bit linear layers **excepted for attention layer**
|
| 32 |
- **License:** Apache licence 2.0
|