Hugging Face's logo
bitsoko's picture
1 1

bitsoko

bitsoko
JohnRoger's profile picture sid7's profile picture razvanab's profile picture
·
https://gumzo.bitsoko.org

AI & ML interests

None yet

Recent Activity

liked a model 30 days ago
Goekdeniz-Guelmez/J.O.S.I.E.v4o
Reacted to reach-vb's post with 🧠 about 2 months ago
Less than two days ago Kyutai Labs open sourced Moshi - an ~7.6B on-device Speech to Speech foundation model and Mimi - SoTA streaming speech codec! 🔥 The release includes: 1. Moshiko & Moshika - Moshi finetuned on synthetic data (CC-BY license) (https://huggingface.co/collections/kyutai/moshi-v01-release-66eaeaf3302bef6bd9ad7acd) 2. Mimi - Streaiming Audio Codec, processes 24 kHz audio, down to a 12.5 Hz representation with a bandwidth of 1.1 kbps (CC-BY license) (https://huggingface.co/kyutai/mimi) 3. Model checkpoints & Inference codebase written in Rust (Candle), PyTorch & MLX (Apache license) (https://github.com/kyutai-labs/moshi) How does Moshi work? 1. Moshi processes two audio streams: one for itself and one for the user, with the user's stream coming from audio input and Moshi's stream generated by the model. 2. Along with these audio streams, Moshi predicts text tokens for its speech, enhancing its generation quality. 3. The model uses a small Depth Transformer for codebook dependencies and a large 7B parameter Temporal Transformer for temporal dependencies. 4. The theoretical latency is 160ms, with a practical latency of around 200ms on an L4 GPU. Model size & inference: Moshiko/ka are 7.69B param models bf16 ~16GB VRAM 8-bit ~8GB VRAM 4-bit ~4GB VRAM You can run inference via Candle 🦀, PyTorch and MLX - based on your hardware. The Kyutai team, @adefossez @lmz and team are cracked AF, they're bringing some serious firepower to the open source/ science AI scene, looking forward to what's next! 🐐
Reacted to reach-vb's post with 🔥 about 2 months ago
Less than two days ago Kyutai Labs open sourced Moshi - an ~7.6B on-device Speech to Speech foundation model and Mimi - SoTA streaming speech codec! 🔥 The release includes: 1. Moshiko & Moshika - Moshi finetuned on synthetic data (CC-BY license) (https://huggingface.co/collections/kyutai/moshi-v01-release-66eaeaf3302bef6bd9ad7acd) 2. Mimi - Streaiming Audio Codec, processes 24 kHz audio, down to a 12.5 Hz representation with a bandwidth of 1.1 kbps (CC-BY license) (https://huggingface.co/kyutai/mimi) 3. Model checkpoints & Inference codebase written in Rust (Candle), PyTorch & MLX (Apache license) (https://github.com/kyutai-labs/moshi) How does Moshi work? 1. Moshi processes two audio streams: one for itself and one for the user, with the user's stream coming from audio input and Moshi's stream generated by the model. 2. Along with these audio streams, Moshi predicts text tokens for its speech, enhancing its generation quality. 3. The model uses a small Depth Transformer for codebook dependencies and a large 7B parameter Temporal Transformer for temporal dependencies. 4. The theoretical latency is 160ms, with a practical latency of around 200ms on an L4 GPU. Model size & inference: Moshiko/ka are 7.69B param models bf16 ~16GB VRAM 8-bit ~8GB VRAM 4-bit ~4GB VRAM You can run inference via Candle 🦀, PyTorch and MLX - based on your hardware. The Kyutai team, @adefossez @lmz and team are cracked AF, they're bringing some serious firepower to the open source/ science AI scene, looking forward to what's next! 🐐
View all activity

Organizations

None yet

bitsoko's activity

upvoted a collection 2 months ago