sambanovasystems
/

SambaLingo-Japanese-Base

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

zolicsaki commited on Feb 23

Commit

27008c7

•

1 Parent(s): 3ad0882

Update README.md

Files changed (1) hide show

README.md +8 -8

README.md CHANGED Viewed

@@ -54,15 +54,15 @@ We extended the vocabulary of the base llama model from 32,000 tokens to 57,000
 ## Evaluation
 || SambaLingo-Japanese-Base     | ELYZA-japanese-Llama-2-7b-7b | bloom-7b1 | xglm-7.5B | mGPT-13B |
 |------------------------------|------------------------------|-----------|-----------|----------|--------|
-| Perplexity (Lower Is Better) | 1.559                        | 1.754     | 2.216     | 1.775    | 2.349  |
-| FLORES en->ja (8 shot, CHRF) | 0.281                        | 0.250     | 0.056     | 0.156    | 0.111  |
-| FLORES ja->en (8 shot, CHRF) | 0.495                        | 0.436     | 0.262     | 0.369    | 0.297  |
 | FLORES en->ja (8 shot, BLEU) | 0.000                        | 0.000     | 0.000     | 0.000    | 0.000  |
-| FLORES ja->en (8 shot, BLEU) | 0.184                        | 0.144     | 0.043     | 0.084    | 0.036  |
-| Belebele (3 shot)            | 36.56%                       | 53.67%    | 26.67%    | 24.00%   | 22.89% |
-| SIB-200 (3 shot)             | 68.63%                       | 74.02%    | 60.29%    | 60.78%   | 41.18% |
-| PAWS-X                       | 46.80%                       | 50.50%    | 45.40%    | 51.95%   | 45.20% |
-| XWinograd (0 shot)           | 76.64%                       | 77.58%    | 58.92%    | 64.96%   | 57.77% |
 ## Uses
 <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

 ## Evaluation
 || SambaLingo-Japanese-Base     | ELYZA-japanese-Llama-2-7b-7b | bloom-7b1 | xglm-7.5B | mGPT-13B |
 |------------------------------|------------------------------|-----------|-----------|----------|--------|
+| Perplexity (Lower Is Better) | **1.559**                        | 1.754     | 2.216     | 1.775    | 2.349  |
+| FLORES en->ja (8 shot, CHRF) | **0.281**                        | 0.250     | 0.056     | 0.156    | 0.111  |
+| FLORES ja->en (8 shot, CHRF) | **0.495**                        | 0.436     | 0.262     | 0.369    | 0.297  |
 | FLORES en->ja (8 shot, BLEU) | 0.000                        | 0.000     | 0.000     | 0.000    | 0.000  |
+| FLORES ja->en (8 shot, BLEU) | **0.184**                        | 0.144     | 0.043     | 0.084    | 0.036  |
+| Belebele (3 shot)            | 36.56%                       | **53.67%**    | 26.67%    | 24.00%   | 22.89% |
+| SIB-200 (3 shot)             | 68.63%                       | **74.02%**    | 60.29%    | 60.78%   | 41.18% |
+| PAWS-X                       | 46.80%                       | 50.50%   | 45.40%    | **51.95%**    | 45.20% |
+| XWinograd (0 shot)           | 76.64%                       | **77.58%**    | 58.92%    | 64.96%   | 57.77% |
 ## Uses
 <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->