license: mit
datasets:
- VishnuPJ/Malayalam_CultureX_IndicCorp_SMC
library_name: transformers
language:
- ml
tags:
- mamba
- ssm
- s6
- jamba
- llm
- state space models
- malayalam
- indic
Ma-layala-mba
Welcome to the Ma-layala-mba model, an advanced Indic language model designed to push the boundaries of NLP for Indian languages. This model leverages a combination of Attention mechanisms, Multi-Layer Perceptrons (MLPs), and State Space Models (SSMs) to deliver cutting-edge performance in text generation tasks.
Model Description
Ma-layala-mba is a state-of-the-art S6 SSM model specifically crafted for Indic languages. It integrates traditional Attention mechanisms with innovative approaches such as MLPs and SSMs to handle complex linguistic features and achieve high accuracy in language understanding and generation.
- Model Type: Mamba model with Attention, MLP, and SSMs components.
- Language(s): Malayalam
- License: GNU General Public License v3.0
- Training Precision: bfloat16
Example Usage
Here's a quick example to get you started with the Ma-layala-mba model:
from transformers import MaLayalaMbaForCausalLM, AutoTokenizer, pipeline
model = MaLayalaMbaForCausalLM.from_pretrained(
"aoxo/Ma-layala-mba_Tiny_128M",
# load_in_8bit=True, # Set this depending on the GPU you have
torch_dtype=torch.bfloat16,
device_map={"": 0}, # Set this depending on the number of GPUs you have
local_files_only=False # Optional
)
model.eval()
tokenizer = AutoTokenizer.from_pretrained("aoxo/Ma-layala-mba_Tiny_128M")
input_ids = tokenizer("മലയാളം പര്യായപദങ്ങളിൽ ഒരു പരീക്ഷ പേപ്പർ ഉണ്ടാക്കുക", return_tensors='pt').to(model.device)["input_ids"]
outputs = model.generate(input_ids, max_new_tokens=100)
print(tokenizer.batch_decode(outputs))
Example Output:
മലയാളം പര്യായപദങ്ങളിൽ ഒരു പരീക്ഷ പേപ്പർ ഉണ്ടാക്കുക
a. വലിയ - __________
b. രസം - __________
c. സുഖം - __________
d. പ്രകാശം - __________
e. വേഗം - __________
Usage Note
Please be aware that this model has not undergone comprehensive detoxification or censorship. While it exhibits strong linguistic capabilities, there is a possibility of generating content that may be deemed harmful or offensive. We advise users to apply discretion and closely monitor the model's outputs, especially in public or sensitive settings.
Meet the Developers
- Alosh Denny