| Tevatron | |
| ``` | |
| bs=512 | |
| epoch=40 | |
| save_steps=4000 | |
| backbone=bert-base-multilingual-cased | |
| output_dir=mlm.bs-$bs.epoch-$epoch.$backbone | |
| WANDB_PROJECT=mlm-mrtydi-DDR \ | |
| python examples/dense-adapter/dense-adapter-train.py \ | |
| --output_dir $output_dir \ | |
| --model_name_or_path $backbone \ | |
| --tokenizer_name bert-base-multilingual-cased \ | |
| --save_steps $save_steps \ | |
| --dataset_name Tevatron/msmarco-passage \ | |
| --fp16 \ | |
| --per_device_train_batch_size $bs \ | |
| --train_n_passages 2 \ | |
| --learning_rate 1e-5 \ | |
| --q_max_len 32 \ | |
| --p_max_len 128 \ | |
| --num_train_epochs $epoch \ | |
| --logging_steps 100 \ | |
| --overwrite_output_dir \ | |
| --dataloader_num_workers 4 \ | |
| ``` |