README.md

wangchanberta-base-att-spm-uncased-finetune-qa

Finetuning wangchanberta-base-att-spm-uncased with the training set of iapp_wiki_qa_squad and thaiqa (removed examples which have cosine similarity with validation and test examples over 0.8). Benchmarks shared on wandb using validation and test sets of iapp_wiki_qa_squad.

Trained with

export WANDB_PROJECT=wangchanberta-qa

export MODEL_NAME=wangchanberta-base-att-spm-uncased
python train_question_answering_lm_finetuning.py \
  --model_name $MODEL_NAME \
  --dataset_name iapp_thaiqa \
  --output_dir $MODEL_NAME-finetune-iapp_thaiqa-model \
  --log_dir $MODEL_NAME-finetune-iapp_thaiqa-log \
  --lowercase \
  --pad_on_right \
  --fp16

export MODEL_NAME=xlm-roberta-base
python train_question_answering_lm_finetuning.py \
  --model_name $MODEL_NAME \
  --dataset_name iapp_thaiqa \
  --output_dir $MODEL_NAME-finetune-iapp_thaiqa-model \
  --log_dir $MODEL_NAME-finetune-iapp_thaiqa-log \
  --model_max_length 416 \
  --pad_on_right \
  --fp16

export MODEL_NAME=bert-base-multilingual-cased
python train_question_answering_lm_finetuning.py \
  --model_name $MODEL_NAME \
  --dataset_name iapp_thaiqa \
  --output_dir $MODEL_NAME-finetune-iapp_thaiqa-model \
  --log_dir $MODEL_NAME-finetune-iapp_thaiqa-log \
  --pad_on_right \
  --fp16

export MODEL_NAME=wangchanberta-base-wiki-spm
python train_question_answering_lm_finetuning.py \
  --model_name $MODEL_NAME \
  --dataset_name iapp_thaiqa \
  --output_dir $MODEL_NAME-finetune-iapp_thaiqa-model \
  --log_dir $MODEL_NAME-finetune-iapp_thaiqa-log \
  --pad_on_right \
  --fp16