README.md

metadata

library_name: transformers
language:
  - ms

Finetune MLM Malaysian Mistral 191M on MNLI

Original model https://huggingface.co/mesolitica/malaysian-mistral-191M-MLM-512, done by https://github.com/aisyahrzk https://twitter.com/aisyahhhrzk

You must use model from here https://github.com/mesolitica/malaya/blob/master/session/llm2vec/classifier.py

dataset

1. source code at https://github.com/mesolitica/malaya/tree/master/session/similarity/hf-t5
2. prepared dataset at https://huggingface.co/datasets/mesolitica/semisupervised-corpus/tree/main/similarity

Accuracy

              precision    recall  f1-score   support

           0    0.84488   0.90914   0.87583      7165
           1    0.92182   0.86519   0.89261      8872

    accuracy                        0.88483     16037
   macro avg    0.88335   0.88717   0.88422     16037
weighted avg    0.88744   0.88483   0.88511     16037