license: cc-by-nc-4.0
language:
- or
Model Card for Model ID
Model description
odia_llama2_512 is based on Llama2-7b and finetuned with 180k Odia instruction set. The instruction set is translated data from open-source resources and a prepared domain knowledge instruction set, resulting in good Odia instruction understanding and response generation capabilities.
The code of Odia data generation and other detailed information can be found in our Github project repository: https://github.com/OdiaGenAI/GenerativeAI_and_LLM_Odia.
Training procedure
The following bitsandbytes quantization config was used during training:
- quant_method: bitsandbytes
- load_in_8bit: False
- load_in_4bit: True
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: nf4
- bnb_4bit_use_double_quant: False
- bnb_4bit_compute_dtype: float16
Framework versions
- PEFT 0.5.0
Instructions for running it can be found at https://github.com/OdiaGenAI/GenerativeAI_and_LLM_Odia.
Licensing Information
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Citation Information
If you find this helpful repository, please consider giving 👏 and citing:
@misc{OdiaGenAI-Odia_Llama2_512,
author = {Shantipriya Parida and Sambit Sekhar and Guneet Singh Kohli},
title = {OdiaGenAI-Odia_Llama2_512},
year = {2023},
publisher = {Hugging Face},
journal = {Hugging Face repository},
howpublished = {\url{https://huggingface.co/OdiaGenAI}},
}
Contributors:
- Guneet Singh Kohli
- Sambit Shekhar
- Shantipriya Parida