Create README.md

README.md

@@ -0,0 +1,46 @@
+# wangchanberta-base-att-spm-uncased-finetune-qa
+
+Finetuning `wangchanberta-base-att-spm-uncased` with the training set of `iapp_wiki_qa_squad` and `thaiqa` (removed examples which have cosine similarity with validation and test examples over 0.8). Benchmarks shared on [wandb](https://wandb.ai/cstorm125/wangchanberta-qa) using validation and test sets of `iapp_wiki_qa_squad`.
+
+Trained with
+```
+export WANDB_PROJECT=wangchanberta-qa
+
+export MODEL_NAME=wangchanberta-base-att-spm-uncased
+python train_question_answering_lm_finetuning.py \
+  --model_name $MODEL_NAME \
+  --dataset_name iapp_thaiqa \
+  --output_dir $MODEL_NAME-finetune-iapp_thaiqa-model \
+  --log_dir $MODEL_NAME-finetune-iapp_thaiqa-log \
+  --lowercase \
+  --pad_on_right \
+  --fp16
+
+export MODEL_NAME=xlm-roberta-base
+python train_question_answering_lm_finetuning.py \
+  --model_name $MODEL_NAME \
+  --dataset_name iapp_thaiqa \
+  --output_dir $MODEL_NAME-finetune-iapp_thaiqa-model \
+  --log_dir $MODEL_NAME-finetune-iapp_thaiqa-log \
+  --model_max_length 416 \
+  --pad_on_right \
+  --fp16
+
+export MODEL_NAME=bert-base-multilingual-cased
+python train_question_answering_lm_finetuning.py \
+  --model_name $MODEL_NAME \
+  --dataset_name iapp_thaiqa \
+  --output_dir $MODEL_NAME-finetune-iapp_thaiqa-model \
+  --log_dir $MODEL_NAME-finetune-iapp_thaiqa-log \
+  --pad_on_right \
+  --fp16
+
+export MODEL_NAME=wangchanberta-base-wiki-spm
+python train_question_answering_lm_finetuning.py \
+  --model_name $MODEL_NAME \
+  --dataset_name iapp_thaiqa \
+  --output_dir $MODEL_NAME-finetune-iapp_thaiqa-model \
+  --log_dir $MODEL_NAME-finetune-iapp_thaiqa-log \
+  --pad_on_right \
+  --fp16
+```