Indri
Collection
Multimodal audio LMs for TTS, ASR, and voice cloning
•
4 items
•
Updated
•
2
Platform | Link |
---|---|
🌎 Live Demo | indrivoice.ai |
𝕏 Twitter | @11mlabs_in |
🐱 GitHub | Indri Repository |
🤗 Hugging Face (Collection) | Indri collection |
🤗 Hugging Face (Spaces) | Live Server |
📝 Release Blog | Release Blog |
Indri is a series of audio models that can do TTS, ASR, and audio continuation. This is the smallest model (124M) in our series and supports TTS tasks in 2 languages:
indri-0.1-124m-tts
is a novel, ultra-small, and lightweight TTS model based on the transformer architecture.
It models audio as tokens and can generate high-quality audio with consistent style cloning of the speaker.
Text | Sample |
---|---|
मित्रों, हम आज एक नया छोटा और शक्तिशाली मॉडल रिलीज कर रहे हैं। | |
भाइयों और बहनों, ये हमारा सौभाग्य है कि हम सब मिलकर इस महान देश को नई ऊंचाइयों पर ले जाने का सपना देख रहे हैं। | |
Hello दोस्तों, future of speech technology mein अपका स्वागत है | |
In this model zoo, a new model called Indri has appeared. |
Here's a brief of how the model works:
Please read our blog here for more technical details on how it was built.
Use the code below to get started with the model. Pipelines are the best way to get started with the model.
import torch
import torchaudio
from transformers import pipeline
model_id = '11mlabs/indri-0.1-124m-tts'
task = 'indri-tts'
pipe = pipeline(
task,
model=model_id,
device=torch.device('cuda:0'), # Update this based on your hardware,
trust_remote_code=True
)
output = pipe(['Hi, my name is Indri and I like to talk.'], speaker = '[spkr_63]')
torchaudio.save('output.wav', output[0]['audio'][0], sample_rate=24000)
Available speakers
Speaker ID | Speaker name |
---|---|
[spkr_63] |
🇬🇧 👨 book reader |
[spkr_67] |
🇺🇸 👨 influencer |
[spkr_68] |
🇮🇳 👨 book reader |
[spkr_69] |
🇮🇳 👨 book reader |
[spkr_70] |
🇮🇳 👨 motivational speaker |
[spkr_62] |
🇮🇳 👨 book reader heavy |
[spkr_53] |
🇮🇳 👩 recipe reciter |
[spkr_60] |
🇮🇳 👩 book reader |
[spkr_74] |
🇺🇸 👨 book reader |
[spkr_75] |
🇮🇳 👨 entrepreneur |
[spkr_76] |
🇬🇧 👨 nature lover |
[spkr_77] |
🇮🇳 👨 influencer |
[spkr_66] |
🇮🇳 👨 politician |
git clone https://github.com/cmeraki/indri.git
cd indri
pip install -r requirements.txt
# Install ffmpeg (for Mac/Windows, refer here: https://www.ffmpeg.org/download.html)
sudo apt update -y
sudo apt upgrade -y
sudo apt install ffmpeg -y
python -m inference --model_path 11mlabs/indri-0.1-124m-tts --device cuda:0 --port 8000
If you use this model in your research, please cite:
@misc{indri-multimodal-alm,
author = {11mlabs},
title = {Indri: Multimodal audio language model},
year = {2024},
publisher = {GitHub},
journal = {GitHub Repository},
howpublished = {\url{https://github.com/cmeraki/indri}},
email = {compute@merakilabs.com}
}
@techreport{kyutai2024moshi,
title={Moshi: a speech-text foundation model for real-time dialogue},
author={Alexandre D\'efossez and Laurent Mazar\'e and Manu Orsini and
Am\'elie Royer and Patrick P\'erez and Herv\'e J\'egou and Edouard Grave and Neil Zeghidour},
year={2024},
eprint={2410.00037},
archivePrefix={arXiv},
primaryClass={eess.AS},
url={https://arxiv.org/abs/2410.00037},
}
@misc{radford2022whisper,
doi = {10.48550/ARXIV.2212.04356},
url = {https://arxiv.org/abs/2212.04356},
author = {Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
title = {Robust Speech Recognition via Large-Scale Weak Supervision},
publisher = {arXiv},
year = {2022},
copyright = {arXiv.org perpetual, non-exclusive license}
}
@misc{Silero VAD,
author = {Silero Team},
title = {Silero VAD: pre-trained enterprise-grade Voice Activity Detector (VAD), Number Detector and Language Classifier},
year = {2024},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/snakers4/silero-vad}},
commit = {insert_some_commit_here},
email = {hello@silero.ai}
}