File size: 3,013 Bytes
7b674cb ca4e94b 7b674cb ca4e94b 7b674cb ca4e94b 7b674cb ca4e94b 7b674cb ca4e94b 7b674cb ca4e94b 7b674cb ca4e94b 7b674cb ca4e94b 7b674cb ca4e94b 7b674cb ca4e94b 7b674cb ca4e94b 7b674cb ca4e94b 7b674cb ca4e94b 226d68b ca4e94b 7b674cb ca4e94b 7b674cb ca4e94b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 |
---
library_name: transformers
tags:
- audio
- automatic-speech-recognition
license: mit
language:
- ar
---
# ArTST-V2 (ASR task)
ArTST model finetuned for automatic speech recognition (speech-to-text) on QASR to improve dialectal generalization.
### Model Description
<!-- Provide a longer summary of what this model is. -->
- **Developed by:** Speech Lab, MBZUAI
- **Model type:** SpeechT5
- **Language:** Arabic
- **Finetuned from:** [ArTST-v2 pretrained](https://github.com/mbzuai-nlp/ArTST)
## How to Get Started with the Model
```python
import soundfile as sf
from transformers import (
SpeechT5Config,
SpeechT5FeatureExtractor,
SpeechT5ForSpeechToText,
SpeechT5Processor,
SpeechT5Tokenizer,
)
device = "cuda" if torch.cuda.is_available() else "CPU"
model_id = "mbzuai/artst-v2-asr"
tokenizer = SpeechT5Tokenizer.from_pretrained(model_id)
processor = SpeechT5Processor.from_pretrained(model_id , tokenizer=tokenizer)
model = SpeechT5ForSpeechToText.from_pretrained(model_id).to(device)
audio, sr = sf.read("audio.wav")
inputs = processor(audio=audio, sampling_rate=sr, return_tensors="pt")
predicted_ids = model.generate(**inputs.to(device), max_length=150)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
print(transcription[0])
```
### Model Sources [optional]
- **Repository:** [github](https://github.com/mbzuai-nlp/ArTST)
- **Paper :** [pre-print](/)
<!-- - **Demo [optional]:** [More Information Needed] -->
## Citation [optional]
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
**BibTeX:**
```
@misc{djanibekov2024dialectalcoveragegeneralizationarabic,
title={Dialectal Coverage And Generalization in Arabic Speech Recognition},
author={Amirbek Djanibekov and Hawau Olamide Toyin and Raghad Alshalan and Abdullah Alitr and Hanan Aldarmaki},
year={2024},
eprint={2411.05872},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2411.05872},
}
@inproceedings{toyin-etal-2023-artst,
title = "{A}r{TST}: {A}rabic Text and Speech Transformer",
author = "Toyin, Hawau and
Djanibekov, Amirbek and
Kulkarni, Ajinkya and
Aldarmaki, Hanan",
booktitle = "Proceedings of ArabicNLP 2023",
month = dec,
year = "2023",
address = "Singapore (Hybrid)",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.arabicnlp-1.5",
doi = "10.18653/v1/2023.arabicnlp-1.5",
pages = "41--51",
}
```
<!-- **APA:**
[More Information Needed]
## Glossary [optional]
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
<!-- [More Information Needed]
## More Information [optional]
[More Information Needed]
## Model Card Authors [optional]
[More Information Needed]
## Model Card Contact
[More Information Needed] --> |