File size: 1,654 Bytes
fec90af 1fae59e ce59ec4 1fae59e a2fa6ef 8defbbd 1fae59e a2fa6ef d63f2a7 1fae59e 7032039 d63f2a7 ce59ec4 d63f2a7 1fae59e ce59ec4 c64dfc8 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
---
title: README
emoji: 📊
colorFrom: purple
colorTo: gray
sdk: static
pinned: false
---
Multilingual language models are typically large, requiring significant computational resources.
Can we create multilingual models that maintain performance comparable to their larger models while reducing size, latency and inference speeds?
# Techniques:
- Pruning
- SparseGPT | [GitHub](https://github.com/VishnuVardhanSaiLanka/sparsegpt/tree/aya)
- ShortGPT | [KLDBasedPruning & Perplexity Sensivities](https://github.com/rsk2327/DistAya/tree/main)
- Knowledge Distillation
- DistillKit | [GitHub](https://github.com/ShayekhBinIslam/DistillKit)
- Distil-Whisper based method
- On policy distillation of language models
- Minitron: Compact Language models via Pruning & Knowledge Distillation
- DistiLLM: Towards Streamlined Distillation for Large Language Models
- Quantization
- KV Cache Compression
- Fine-Tuning | [GitHub](https://github.com/rsk2327/DistAya/tree/track/fine-tuning)
# Datasets:
Initial 7 datasets unified, having 6.62M rows which includes the following:
- Bangla_Alpaca_Orca : Bangle
- Urdu_Instruct_News_Article_Generation: Urdu
- Urdu_Instruct_News_Headline_Generation: Urdu
- Urdu_Instruct_News_Category_Classification: Urdu
- cidar: Arabic
- Six_Millions_Instruction_Dataset_For_Arabic_Llm_Ft: Arabic
- instructv3: English
## Get in touch with the team:
- Mayank Bhaskar -> mayankbhaskar007@gmail.com
- Ahmad Anis -> ahmadanis5050@gmail.com
- Drishti Sharma -> drishtisharma96505@gmail.com
- Vishnu Vardhan -> vardhanvishnu691@gmail.com
- Yaya -> yayasysco@gmail.com
- Shayekh Bin Islam -> shayekh.bin.islam@gmail.com |