|
--- |
|
inference: false |
|
language: |
|
- ja |
|
- en |
|
- de |
|
- is |
|
- zh |
|
- cs |
|
--- |
|
# webbigdata/ALMA-7B-Ja-V2 |
|
|
|
ALMA-7B-Ja-V2は日本語から英語、英語から日本語の翻訳が可能な機械翻訳モデルです。 |
|
The ALMA-7B-Ja-V2 is a machine translation model capable of translating from Japanese to English and English to Japanese. |
|
|
|
ALMA-7B-Ja-V2は以前のモデル(ALMA-7B-Ja)に更に学習を追加し、性能を向上しています。 |
|
The ALMA-7B-Ja-V2 adds further learning to the previous model (ALMA-7B-Ja) and improves performance. |
|
|
|
日本語と英語間に加えて、このモデルは以下の言語間の翻訳能力も持っています。 |
|
In addition to translation between Japanese and English, the model also has the ability to translate the following four languages. |
|
|
|
- ドイツ語 German(de) and 英語 English(en) |
|
- 中国語 Chinese(zh) and 英語 English(en) |
|
- アイスランド語 Icelandic(is) and 英語 English(en) |
|
- チェコ語 Czech(cs) and 英語 English(en) |
|
|
|
# ベンチマーク結果 |
|
|
|
Meta社の200言語以上の翻訳に対応した超多言語対応機械翻訳モデルNLLB-200シリーズと比較したベンチマーク結果は以下です。 |
|
Benchmark results compared to Meta's NLLB-200 series of super multilingual machine translation models, which support translations in over 200 languages, are shown below. |
|
|
|
|
|
## NLLB-200 |
|
| Model Name | file size |E->J chrf++/F2|E->J comet|J->E chrf++/F2|J->E comet | |
|
|------------------------------|-----------|--------------|----------|--------------|-----------| |
|
| NLLB-200-Distilled | 2.46GB | 23.6/- | - | 50.2/- | - | |
|
| NLLB-200-Distilled | 5.48GB | 25.4/- | - | 54.2/- | - | |
|
| NLLB-200 | 5.48GB | 24.2/- | - | 53.6/- | - | |
|
| NLLB-200 | 17.58GB | 25.2/- | - | 55.1/- | - | |
|
| NLLB-200 | 220.18GB | 27.9/33.2 | 0.8908 | 55.8/59.8 | 0.8792 | |
|
|
|
## previous our model(ALMA-7B-Ja) |
|
| Model Name | file size |E->J chrf++/F2|E->J comet|J->E chrf++/F2|J->E comet | |
|
|------------------------------|-----------|--------------|----------|--------------|-----------| |
|
| webbigdata-ALMA-7B-Ja-q4_K_S | 3.6GB | -/24.2 | 0.8210 | -/54.2 | 0.8559 | |
|
| ALMA-7B-Ja-GPTQ-Ja-En | 3.9GB | -/30.8 | 0.8743 | -/60.9 | 0.8743 | |
|
| ALMA-Ja(Ours) | 13.48GB | -/31.8 | 0.8811 | -/61.6 | 0.8773 | |
|
|
|
## ALMA-7B-Ja-V2 |
|
| Model Name | file size |E->J chrf++/F2|E->J comet|J->E chrf++/F2|J->E comet | |
|
|------------------------------|-----------|--------------|----------|--------------|-----------| |
|
| ALMA-7B-Ja-V2-GPTQ-Ja-En | 3.9GB | -/33.0 | 0.8818 | -/62.0 | 0.8774 | |
|
| ALMA-Ja-V2(Ours) | 13.48GB | -/33.9 | 0.8820 | -/63.1 | 0.8873 | |
|
| ALMA-Ja-V2-Lora(Ours) | 13.48GB | -/33.7 | 0.8843 | -/61.1 | 0.8775 | |
|
|
|
|
|
|
|
様々なジャンルの文章を実際のアプリケーションと比較した結果は以下です。 |
|
Here are the results of a comparison of various genres of writing with the actual application. |
|
|
|
## 政府の公式文章 Government Official Announcements |
|
| |e->j chrF2++|e->j BLEU|e->j comet|j->e chrF2++|j->e BLEU|j->e comet| |
|
|--------------------------|------------|---------|----------|------------|---------|----------| |
|
| ALMA-7B-Ja-V2-GPTQ-Ja-En | 25.3 | 15.00 | 0.8848 | 60.3 | 26.82 | 0.6189 | |
|
| ALMA-Ja-V2 | 27.2 | 15.60 | 0.8868 | 58.5 | 29.27 | 0.6155 | |
|
| ALMA-7B-Ja-V2-Lora | 24.5 | 13.58 | 0.8670 | 50.7 | 21.85 | 0.6196 | |
|
| gpt-3.5 | 34.6 | 28.33 | 0.8895 | 74.5 | 49.20 | 0.6382 | |
|
| gpt-4.0 | 36.5 | 28.07 | 0.9255 | 62.5 | 33.63 | 0.6320 | |
|
| google-translate | 43.5 | 35.37 | 0.9181 | 62.7 | 29.22 | 0.6446 | |
|
| deepl | 43.5 | 35.74 | 0.9301 | 60.1 | 27.40 | 0.6389 | |
|
|
|
## 二次創作 Fanfiction |
|
| |e->j chrF2++|e->j BLEU|e->j comet|j->e chrF2++|j->e BLEU|j->e comet| |
|
|--------------------------|------------|---------|----------|------------|---------|----------| |
|
| ALMA-7B-Ja-V2-GPTQ-Ja-En | 27.6 | 18.28 | 0.8643 | 52.1 | 24.58 | 0.6106 | |
|
| ALMA-Ja-V2 | 20.4 | 8.45 | 0.7870 | 48.7 | 23.06 | 0.6050 | |
|
| ALMA-7B-Ja-V2-Lora | 23.9 | 18.55 | 0.8634 | 55.6 | 29.91 | 0.6093 | |
|
| gpt-3.5 | 31.2 | 23.37 | 0.9001 | - | - | 0.5948 | |
|
| gpt-4.0 | 30.7 | 24.31 | 0.8848 | 53.9 | 24.89 | 0.6163 | |
|
| google-translate | 32.4 | 25.36 | 0.8968 | 58.5 | 29.88 | 0.6022 | |
|
| deepl | 33.5 | 28.38 | 0.9094 | 60.0 | 31.14 | 0.6124 | |
|
|
|
|
|
[Sample Code For Free Colab](https://github.com/webbigdata-jp/python_sample/blob/main/ALMA_7B_Ja_V2_Free_Colab_sample.ipynb) |
|
|
|
|
|
|
|
## Other Version |
|
|
|
### ALMA-7B-Ja-V2-GPTQ-Ja-En |
|
GPTQ is quantized(reduce the size of the model) method and ALMA-7B-Ja-V2-GPTQ has GPTQ quantized version that reduces model size(3.9GB) and memory usage. |
|
But the performance is probably lower. And translation ability for languages other than Japanese and English has deteriorated significantly. |
|
|
|
[Sample Code For Free Colab webbigdata/ALMA-7B-Ja-V2-GPTQ-Ja-En](https://github.com/webbigdata-jp/ALMA/blob/master/ALMA_7B_Ja_V2_GPTQ_Ja_En_Free_Colab_sample.ipynb) |
|
|
|
If you want to translate the entire file at once, try Colab below. |
|
[ALMA_7B_Ja_GPTQ_Ja_En_batch_translation_sample](https://github.com/webbigdata-jp/ALMA/blob/master/ALMA_7B_Ja_V2_GPTQ_Ja_En_batch_translation_sample.ipynb) |
|
|
|
|
|
**ALMA** (**A**dvanced **L**anguage **M**odel-based tr**A**nslator) is an LLM-based translation model, which adopts a new translation model paradigm: it begins with fine-tuning on monolingual data and is further optimized using high-quality parallel data. This two-step fine-tuning process ensures strong translation performance. |
|
Please find more details in their [paper](https://arxiv.org/abs/2309.11674). |
|
``` |
|
@misc{xu2023paradigm, |
|
title={A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models}, |
|
author={Haoran Xu and Young Jin Kim and Amr Sharaf and Hany Hassan Awadalla}, |
|
year={2023}, |
|
eprint={2309.11674}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL} |
|
} |
|
``` |
|
|
|
|
|
Original Model [ALMA-7B](https://huggingface.co/haoranxu/ALMA-7B). (26.95GB) |
|
Prevous Model [ALMA-7B-Ja](https://huggingface.co/webbigdata/ALMA-7B-Ja). (13.3 GB) |
|
|
|
|
|
## about this work |
|
- **This work was done by :** [webbigdata](https://webbigdata.jp/). |
