updated README.md

README.md

@@ -1,98 +1,39 @@
----
-language: 
-- ru
-- en
-thumbnail: https://raw.githubusercontent.com/JetRunner/BERT-of-Theseus/master/bert-of-theseus.png
-tags:
-- translation
-- fsmt
-license: Apache 2.0
-datasets:
-- wmt19
-metrics:
-- bleu
-- sacrebleu
----
-
-# MyModel
+# Albumin-15s
 
 ## Model description
 
-This is a ported version of [fairseq wmt19 transformer](https://github.com/pytorch/fairseq/blob/master/examples/wmt19/README.md) for {src_lang}-{tgt_lang}.
-
-For more details, please see, [Facebook FAIR's WMT19 News Translation Task Submission](https://arxiv.org/abs/1907.06616).
-
-The abbreviation FSMT stands for FairSeqMachineTranslation
+This is a version of [Albert-base-v2](https://huggingface.co/albert-base-v2) for 15's long aptamers comparison to determine which one is more affine to target protein Albumin.
 
-All four models are available:
+The Albert model was pretrained in the English language, it has many similarities with language or proteins and aptamers which is why we had to fine-tune it to help the model learn embedded positioning for aptamers to be able to distinguish better sequences.
 
-* [wmt19-en-ru](https://huggingface.co/facebook/wmt19-en-ru)
-* [wmt19-ru-en](https://huggingface.co/facebook/wmt19-ru-en)
-* [wmt19-en-de](https://huggingface.co/facebook/wmt19-en-de)
-* [wmt19-de-en](https://huggingface.co/facebook/wmt19-de-en)
+More information can be found in our [github]() and our iGEMs [wiki]().
 
 ## Intended uses & limitations
 
+You can use the fine-tuned model for either masked aptamer pair sequence classification, which one is more affine for target protein Albumin, prediction, but it's mostly intended to be fine-tuned again on a different length aptamer or simply expanded datasets.
+
 #### How to use
 
-```python
-from transformers.tokenization_fsmt import FSMTTokenizer
-from transformers.modeling_fsmt import FSMTForConditionalGeneration
-mname = "facebook/wmt19-ru-en"
-tokenizer = FSMTTokenizer.from_pretrained(mname)
-model = FSMTForConditionalGeneration.from_pretrained(mname)
+This model can be used to predict compared affinity with dataset preprocessing function which encodes the specific type of data (Sequence1, Sequence2, Label) where Label indicates binary if Sequence1 is more affine to target protein Albumin.
 
-input = "Машинное обучение - это здорово, не так ли?"
-input_ids = tokenizer.encode(input, return_tensors="pt")
-outputs = model.generate(input_ids)
-decoded = tokenizer.decode(outputs[0], skip_special_tokens=True)
-print(decoded) # Machine learning is great, isn't it?
+```python
+from transformers import AutoTokenizer, BertModel
+mname = "Vilnius-Lithuania-iGEM/Albumin"
+model = BertModel.from_pretrained(mname)
 ```
 
-#### Limitations and bias
-
-- The original (and this ported model) doesn't seem to handle well inputs with repeated sub-phrases, [content gets truncated](https://discuss.huggingface.co/t/issues-with-translating-inputs-containing-repeated-phrases/981)
-
-
-## Training data
+To predict batches of sequences you have to employ custom functions shown in [git/prediction.ipynb]()
 
-Pretrained weights were left identical to the original model released by fairseq. For more details, please, see the [paper](https://arxiv.org/abs/1907.06616).
-
-
-## Training procedure
+#### Limitations and bias
 
+- It seems that fine-tuned Albert model for this kind of task has limition of 90 % accuracy predicting which aptamer is more suitable for a target protein, also Albert-large or immense dataset of 15s aptamer could increase accuracy few %, however extrapolation case is not studied and we cannot confirm this model is state-of-The-art when one of aptamers is SUPER good (has almost maximum entropy to the Albumin).
 
 ## Eval results
 
-pair   | fairseq | transformers
--------|---------|----------
-ru-en  | [41.3](http://matrix.statmt.org/matrix/output/1907?run_id=6937) | 39.20
-
-
-The score was calculated using this code:
+accuracy :  0.8601
+precision:  0.8515
+recall   :  0.8725
+f1       :  0.8618
+roc_auc  :  0.9388
 
-```bash
-git clone https://github.com/huggingface/transformers
-cd transformers
-export PAIR=ru-en
-export DATA_DIR=data/$PAIR
-export SAVE_DIR=data/$PAIR
-export BS=8
-export NUM_BEAMS=15
-mkdir -p $DATA_DIR
-sacrebleu -t wmt19 -l $PAIR --echo src > $DATA_DIR/val.source
-sacrebleu -t wmt19 -l $PAIR --echo ref > $DATA_DIR/val.target
-echo $PAIR
-PYTHONPATH="src:examples/seq2seq" python examples/seq2seq/run_eval.py facebook/wmt19-$PAIR $DATA_DIR/val.source $SAVE_DIR/test_translations.txt --reference_path $DATA_DIR/val.target --score_path $SAVE_DIR/test_bleu.json --bs $BS --task translation --num_beams $NUM_BEAMS
-```
-
-### BibTeX entry and citation info
-
-```bibtex
-@inproceedings{...,
-  year={2020},
-  title={Facebook FAIR's WMT19 News Translation Task Submission},
-  author={Ng, Nathan and Yee, Kyra and Baevski, Alexei and Ott, Myle and Auli, Michael and Edunov, Sergey},
-  booktitle={Proc. of WMT},
-}
-```
+The score was calculated using sklearn.metrics.