Bueorm/HMTM

Model Details

Model Description

Our model leverages the power of multiple pre-trained translation models from Helsinki-NLP, including translations between English, French, Spanish, German, and Italian. Each translation direction utilizes state-of-the-art models such as Helsinki-NLP/opus-mt-en-fr, Helsinki-NLP/opus-mt-fr-en, and others. This model integrates a Mixture of Experts (MoE) approach to dynamically select the most appropriate model for translation based on input language pairs, ensuring high accuracy and versatility. Developed by Bueorm, this model aims to provide robust translation capabilities across multiple languages, thanks to the pioneering work of Helsinki-NLP in machine translation.

• Developed by: BueormAI
• Funded by: Gerson Buenahora
• Shared by: Bueorm
• Model type: Translation
• Language(s) (NLP): English, Espanish, German, French, Italian
• License: MiT

How to Use

import torch
from transformers import MarianMTModel, MarianTokenizer

def load_model(file_path):
    return torch.load(file_path)

loaded_model_dict = load_model('MoE_translation_model.pth')
loaded_models = loaded_model_dict['models']
loaded_tokenizers = loaded_model_dict['tokenizers']

def translate(text, src_lang, tgt_lang):
    pair = f'{src_lang}-{tgt_lang}'
    if pair in loaded_models:
        model = loaded_models[pair]
        tokenizer = loaded_tokenizers[pair]
        inputs = tokenizer(text, return_tensors='pt')
        translated_tokens = model.generate(**inputs)
        translated_text = tokenizer.decode(translated_tokens[0], skip_special_tokens=True)
        return translated_text
    else:
        return f"No model found to translate from {src_lang} to {tgt_lang}"

text = "Hello, how are you?"
translated_text = translate(text, 'en', 'fr')
print(translated_text)

Languages

• English to French (en, fr)
• French to English (fr, en)
• English to Spanish (en, es)
• Spanish to English (es, en)
• English to German (en, de)
• German to English (de, en)
• Spanish to German (es, de)
• German to Spanish (de, es)
• Spanish to French (es, fr)
• French to Spanish (fr, es)
• German to French (de, fr)
• French to German (fr, de)
• Spanish to Italian (es, it)
• Italian to Spanish (it, es)
• English to Italian (en, it)
• Italian to English (it, en)