Update README.md
#1
by
Ashmal
- opened
README.md
CHANGED
@@ -2,11 +2,13 @@
|
|
2 |
license: mit
|
3 |
license_link: https://huggingface.co/microsoft/phi-2/resolve/main/LICENSE
|
4 |
language:
|
5 |
-
|
6 |
pipeline_tag: text-generation
|
7 |
tags:
|
8 |
-
|
9 |
-
|
|
|
|
|
10 |
---
|
11 |
# MobiLlama-05B
|
12 |
|
@@ -57,7 +59,20 @@ print(tokenizer.batch_decode(outputs[:, input_ids.shape[1]:-1])[0].strip())
|
|
57 |
|
58 |
```
|
59 |
|
60 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
61 |
|
62 |
-
|
63 |
|
|
|
|
2 |
license: mit
|
3 |
license_link: https://huggingface.co/microsoft/phi-2/resolve/main/LICENSE
|
4 |
language:
|
5 |
+
- en
|
6 |
pipeline_tag: text-generation
|
7 |
tags:
|
8 |
+
- nlp
|
9 |
+
- code
|
10 |
+
datasets:
|
11 |
+
- LLM360/AmberDatasets
|
12 |
---
|
13 |
# MobiLlama-05B
|
14 |
|
|
|
59 |
|
60 |
```
|
61 |
|
62 |
+
## Evaluation
|
63 |
+
| Evaluation Benchmark | MobiLlama-0.5B | MobiLlama-0.8B | MobiLlama-1.2B |
|
64 |
+
| ----------- | ----------- | ----------- |
|
65 |
+
| HellaSwag | 0.5252 | 0.5409 | 0.6299 |
|
66 |
+
| MMLU | 0.2645 | 0.2692 | 0.2423 |
|
67 |
+
| Arc Challenge | 0.2952 | 0.3020 | 0.3455 |
|
68 |
+
| TruthfulQA | 0.3805 | 0.3848 | 0.3557 |
|
69 |
+
| CrowsPairs | 0.6403 | 0.6482 | 0.6812 |
|
70 |
+
| PIQA | 0.7203 | 0.7317 | 0.7529 |
|
71 |
+
| Race | 0.3368 | 0.3337 | 0.3531 |
|
72 |
+
| SIQA | 0.4022 | 0.4160 | 0.4196 |
|
73 |
+
| Winogrande | 0.5753 | 0.5745 | 0.6108 |
|
74 |
+
|
75 |
|
76 |
+
## Intended Uses
|
77 |
|
78 |
+
Given the nature of the training data, the MobiLlama-05B model is best suited for prompts using the QA format, the chat format, and the code format.
|