Create README.md

README.md

@@ -0,0 +1,132 @@
+---
+language:
+- en
+- hi
+tags:
+- Llama3
+- Pytorch
+- Bilingual
+pipeline_tag: text-generation
+---
+# SandLogic Technologies - Quantized LLama3-Gaja-Hindi-8B Models
+
+## Model Description
+
+We have quantized the LLama3-Gaja-Hindi-8B model into two variants:
+
+1. Q5_KM
+2. Q4_KM
+
+These quantized models offer improved efficiency while maintaining performance.
+
+Discover our full range of quantized language models by visiting our [SandLogic Lexicon](https://github.com/sandlogic/SandLogic-Lexicon) GitHub. To learn more about our company and services, check out our website at [SandLogic](https://www.sandlogic.com).
+
+## Original Model Information
+
+- **Name**: [LLama3-Gaja-Hindi-8B-v0.1](https://huggingface.co/Cognitive-Lab/LLama3-Gaja-Hindi-8B-v0.1)
+- **Developer**: Cognitivelab.in
+- **Base Model**: Llama3-8B
+- **Model Type**: Bilingual English/Hindi language model
+- **Parameters**: 8 billion
+- **Specialization**: Natural language understanding, particularly for instructional pairs
+
+## Model Capabilities
+
+LLama3-Gaja-Hindi-8B is designed for bilingual (English/Hindi) natural language understanding tasks, with a focus on:
+
+- Responding appropriately in either English or Hindi based on user prompts
+- Understanding and generating instructional content in both languages
+- Handling a variety of natural language processing tasks across both languages
+
+## Training Approach
+
+The model underwent supervised fine-tuning with low-rank adaptation, focusing on bilingual instruct fine-tuning. The training data consisted of a curated dataset of translated instructional pairs.
+
+## Use Cases
+
+1. **Bilingual Conversational AI**: Chatbots and virtual assistants with English/Hindi capabilities
+2. **Language Learning Tools**: Interactive platforms for English and Hindi learners
+3. **Content Translation**: Assistance in translating between English and Hindi, especially for instructional materials
+4. **Cross-lingual Information Retrieval**: Enabling queries in one language with responses in either English or Hindi
+5. **Cultural Context Understanding**: Helping users grasp cultural nuances in both languages
+6. **Multilingual Customer Support**: Powering customer service applications for diverse user bases
+
+## Model Variants
+
+We offer two quantized versions of the LLama3-Gaja-Hindi-8B model:
+
+1. **Q5_KM**: 5-bit quantization using the KM method
+2. **Q4_KM**: 4-bit quantization using the KM method
+
+These quantized models aim to reduce model size and improve inference speed while maintaining performance as close to the original model as possible.
+
+## Input and Output
+
+- **Input**: Text prompts or instructions in either English or Hindi
+- **Output**: Generated text responses in the same language as the input, or as specified in the prompt
+
+## Usage
+
+```bash
+pip install llama-cpp-python 
+```
+Please refer to the llama-cpp-python [documentation](https://llama-cpp-python.readthedocs.io/en/latest/) to install with GPU support.
+
+### Basic Text Completion
+Here's an example demonstrating how to use the high-level API for basic text completion:
+
+```bash
+from llama_cpp import Llama
+
+llm = Llama(
+    model_path="./models/7B/LLama3-Gaja-Hindi-8B-v0.1.gguf",
+    verbose=False,
+    # n_gpu_layers=-1, # Uncomment to use GPU acceleration
+    # n_ctx=2048, # Uncomment to increase the context window
+)
+
+output = llm.create_chat_completion(
+    messages =[
+    {
+        "role": "system",
+        "content": """ You are an AI assistant trained on top of Llama 3 Large language model (LLM), proficient in English and Hindi. You can respond in both languages based on the user's request."""
+            
+        ,
+    },
+    {"role": "user", "content": "Write an poem in hindi"},
+]
+)
+
+print(output["choices"][0]['message']['content'])
+```
+
+## Download
+You can download `Llama` models in `gguf` format directly from Hugging Face using the `from_pretrained` method. This feature requires the `huggingface-hub` package.
+
+To install it, run: `pip install huggingface-hub`
+
+```bash
+from llama_cpp import Llama
+
+llm = Llama.from_pretrained(
+    repo_id="SandLogicTechnologies/LLama3-Gaja-Hindi-8B-GGUF",
+    filename="*llama3-gaja-hindi-8b-v0.1.Q5_K_M.gguf",
+    verbose=False
+)
+```
+By default, from_pretrained will download the model to the Hugging Face cache directory. You can manage installed model files using the huggingface-cli tool.
+
+
+## Ethical Considerations
+
+Users should be aware of potential biases in the model's outputs, especially when dealing with cultural or linguistic nuances. Always review and validate the model's responses for sensitive applications.
+
+
+
+## Acknowledgements
+
+We thank Cognitivelab.in for developing the original LLama3-Gaja-Hindi-8B model and the creators of Llama3 for their foundational work.Special thanks to Georgi Gerganov and the entire llama.cpp development team for their outstanding contributions.
+
+## Contact
+
+For any inquiries or support, please contact us at support@sandlogic.com or visit our [support page](https://www.sandlogic.com/LingoForge/support).