BabelBox

non-profit

AI & ML interests

Automatic Speech Recognition (ASR) and Natural Language Processing (NLP)

Whisper Fine-Tuning Event 🤗


Whisper Fine-Tuning Event

https://github.com/huggingface/community-events/blob/main/whisper-fine-tuning-event/README.md


Sign in to Lambdalabs

https://cloud.lambdalabs.com/instances

and create a new instance with a A100 GPU without disk storage


Log in to Huggingface

pip install git+https://github.com/huggingface/transformers
git config --global credential.helper store
huggingface-cli login

Provide Huggingface access token for your account

https://huggingface.co/settings/tokens


Clone Whisper train project for run scripts

git clone https://huggingface.co/bjelkenhed/whisper-train-ts

Install requirements and python libraries

sudo apt-get install git-lfs
pip install --pre --force-reinstall torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/nightly/cu117

pip install -r whisper-medium/requirements.txt

python -c "import torch; print(torch.cuda.is_available())"

Create and clone Huggingface model repository

Create a new model named whisper-small-sv (change to your selected name in following sections)

huggingface-cli repo create whisper-medium-ts-sv
git clone https://huggingface.co/bjelkenhed/whisper-medium-ts-sv

cd whisper-medium-ts-sv
git lfs install

Copy runscripts to model

cp ~/whisper-small/run.sh ~/whisper-small/run_speech_recognition_seq2seq_streaming.py .

or

cp ~/whisper-medium/run.sh ~/whisper-medium/run_speech_recognition_seq2seq_streaming.py .

or

cp ~/whisper-large/run.sh ~/whisper-large/run_speech_recognition_seq2seq_streaming.py .

Start training

tmux new -s mysession
bash run.sh

To resume session

tmux a -t mysession