MaLA-LM

/

emma-500-llama2-7b

like 11

Follow

MaLA-LM 19

Hi,

We did evaluations on En-X and X-En translation (See EMMA-500 paper). Other directions are theoretically supported, but we have not done detailed testing. I did a quick test on Zh-De and it works.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "MaLA-LM/emma-500-llama2-7b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)

src_lang = "Chinese"
tgt_lang = "Germany"
src_sent = "格陵兰岛位于北大西洋和北冰洋之间。"
input_text = f"Translate the following sentence from {src_lang} to {tgt_lang}\n[{src_lang}]: {src_sent}\n[{tgt_lang}]:"
inputs = tokenizer(input_text, return_tensors="pt").to(device)

with torch.no_grad():
    outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Translate the following sentence from Chinese to Germany
[Chinese]: 格陵兰岛位于北大西洋和北冰洋之间。
[Germany]: Grönland liegt zwischen dem Nordatlantik und dem Nordpolarmeer.

Hi,
Thanks for your prompt reply. This is great, of course I'm aware that "Your Mileage May Vary" wrt the language pair chosen, as with all translation models. I was about to ask you a question, but I found it on Table 2. Another question I have is, do you have any ball park figure as to how much accuracy would drop for a 4-bit quantization, any rough estimates?
Cheers!

Sorry, we haven't done any x-bit quantization testing, so I cannot tell you what the accuracy drop would be.

No worries, thanks again for the great work, and your insightful answer.