| # Model Card for Diva Llama 3 | |
| <!-- Provide a quick summary of what the model is/does. [Optional] --> | |
| This is an end-to-end Voice Assistant Model which can handle speech and text as inputs. It is trained using distillation loss. More details will be in a paper [COMING SOON]! | |
| See also [value-nlp.github.io/DiVA-Demo](value-nlp.github.io/DiVA-Demo). | |
| ## Citation | |
| No Publication As of Yet, But If You Use Please Cite the Below | |
| **BibTeX:** | |
| ``` | |
| @InProceedings{hewitt2023backpack, | |
| author = "Held, Will and Zhang, Yanzhe and Ryan, Michael and Shi, Weiyan and Li, Ella and Yang, Diyi", | |
| title = "Distilling an End-to-End Voice Assistant from Speech Recognition Data", | |
| year = "2024", | |
| publisher = "HuggingFace", | |
| } | |
| ``` | |
| ## Table of Contents | |
| - [Model Card for DiVA Llama 3](#model-card-for-DiVA-Llama-3) | |
| - [Citation](#citation) | |
| - [Table of Contents](#table-of-contents) | |
| - [Training Details](#training-details) | |
| - [Training Data](#training-data) | |
| - [Training Procedure](#training-procedure) | |
| - [Environmental Impact](#environmental-impact) | |
| - [Technical Specifications [optional]](#technical-specifications-optional) | |
| - [Model Architecture and Objective](#model-architecture-and-objective) | |
| - [Compute Infrastructure](#compute-infrastructure) | |
| - [Hardware](#hardware) | |
| - [Software](#software) | |
| - [Model Card Contact](#model-card-contact) | |
| ## Training Details | |
| ### Training Data | |
| <!-- This should link to a Data Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. --> | |
| This model was trained on the [CommonVoice](https://huggingface.co/datasets/mozilla-foundation/common_voice_16_1) corpus. | |
| ### Training Procedure | |
| This model was trained for 7k gradient steps with a batch size of 512 Recordings and a linearly decaying learning rate from 5e-5 to zero, with a linear warmup of 70 steps. | |
| ### Environmental Impact | |
| - **Hardware Type:** V4-32 TPU | |
| - **Hours used:** 8 Hours | |
| - **Cloud Provider:** Google Cloud. | |
| - **Compute Region:** US Central C | |
| ### Hardware | |
| This model was trained on at V4 TPU on Google Cloud. | |
| ### Software | |
| This model was trained with [Levanter](https://github.com/stanford-crfm/levanter) | |
| ## Model Card Authors [optional] | |
| <!-- This section provides another layer of transparency and accountability. Whose views is this model card representing? How many voices were included in its construction? Etc. --> | |
| Will Held | |
| ## Model Card Contact | |
| held@stanford.edu | |