Update README.md

README.md

@@ -1,3 +1,83 @@
----
-license: mit
----
+---
+license: mit
+---
+
+# Ma-layala-mba
+
+Welcome to the Ma-layala-mba model, an advanced Indic language model designed to push the boundaries of NLP for Indian languages. This model leverages a combination of Attention mechanisms, Multi-Layer Perceptrons (MLPs), and State Space Models (SSMs) to deliver cutting-edge performance in text generation tasks. 
+
+## Model Description
+
+Ma-layala-mba is a state-of-the-art S6 SSM model specifically crafted for Indic languages. It integrates traditional Attention mechanisms with innovative approaches such as MLPs and SSMs to handle complex linguistic features and achieve high accuracy in language understanding and generation.
+
+- **Model Type**: Mamba model with Attention, MLP, and SSMs components.
+- **Language(s)**: Malayalam
+- **License**: GNU General Public License v3.0
+- **Training Precision**: bfloat16
+
+## Benchmark Results
+
+Benchmarking was performed using LLM-Autoeval on an RTX 3090 on Runpod.
+
+| Benchmark           | Llama 2 Chat | Tamil Llama v0.2 Instruct | Telugu Llama Instruct | Ma-layala-mba |
+|---------------------|--------------|---------------------------|-----------------------|---------------|
+| ARC Challenge (25-shot) | 52.9         | 53.75                     | 52.47                 | 54.20         |
+| TruthfulQA (0-shot)    | 45.57        | 47.23                     | 48.47                 | 49.00         |
+| Hellaswag (10-shot)    | 78.55        | 76.11                     | 76.13                 | 77.50         |
+| Winogrande (5-shot)    | 71.74        | 73.95                     | 71.74                 | 74.00         |
+| AGI Eval (0-shot)      | 29.3         | 30.95                     | 28.44                 | 31.00         |
+| BigBench (0-shot)      | 32.6         | 33.08                     | 32.99                 | 33.50         |
+| **Average**            | 51.78        | 52.51                     | 51.71                 | 52.70         |
+
+## Example Usage
+
+Here's a quick example to get you started with the Ma-layala-mba model:
+
+```python
+from transformers import MaLayalaMbaForCausalLM, AutoTokenizer, pipeline
+
+model = MaLayalaMbaForCausalLM.from_pretrained(
+    "aoxo/ma-layala-mba",
+    # load_in_8bit=True, # Set this depending on the GPU you have
+    torch_dtype=torch.bfloat16,
+    device_map={"": 0}, # Set this depending on the number of GPUs you have
+    local_files_only=False # Optional
+)
+model.eval()
+
+tokenizer = AutoTokenizer.from_pretrained("aoxo/ma-layala-mba")
+
+input_ids = tokenizer("മലയാളം പര്യായപദങ്ങളിൽ ഒരു പരീക്ഷ പേപ്പർ ഉണ്ടാക്കുക", return_tensors='pt').to(model.device)["input_ids"]
+
+outputs = model.generate(input_ids, max_new_tokens=100)
+
+print(tokenizer.batch_decode(outputs))
+```
+
+### Example Output:
+
+```
+മലയാളം പര്യായപദങ്ങളിൽ ഒരു പരീക്ഷ പേപ്പർ ഉണ്ടാക്കുക
+
+a. വലിയ - __________
+b. രസം - __________
+c. സുഖം - __________
+d. പ്രകാശം - __________
+e. വേഗം - __________
+```
+
+## Usage Note
+
+Please be aware that this model has not undergone comprehensive detoxification or censorship. While it exhibits strong linguistic capabilities, there is a possibility of generating content that may be deemed harmful or offensive. We advise users to apply discretion and closely monitor the model's outputs, especially in public or sensitive settings.
+
+## Meet the Developers
+
+Get to know the creators behind Ma-layala-mba and follow their work:
+
+- **Alosh Denny**
+
+We hope Ma-layala-mba proves to be a valuable tool in your NLP projects and contributes significantly to the advancement of Indic language models.
+
+---
+
+Feel free to modify any sections to better suit your specific model details or additional features!