This repo offers a set of fairseq roberta based models fine tuned to specific NLP tasks. These models are:
- Small
- Easy to host on T4 or V100
- 100x faster than using LLMs for similar tasks
- Easy to fine tune
All the models below were trained at Nlmatics Corp. from 2019-2023 with base model from: https://github.com/facebookresearch/fairseq/blob/main/examples/roberta/README.md
To run the models:
Use https://github.com/nlmatics/nlm-model-service
To acccess the models
Use https://github.com/nlmatics/nlm-utils
To train the models
TBD
List of Models
Click on each model to see details:
roberta.large.boolq
Location: roberta.large.boolq
Trained with MNLI + Boolq
Trained by: Evan Li
Application: Given a passage and a question, answer the question with yes, no or unsure.
Training Process: https://blogs.nlmatics.com/2020/03/12/Boolean-Question-Answering-with-Neutral-Labels.html
roberta.large.qa
See folder: roberta.large.qa
Trained with SQuAD 2.0 + Custom Dataset preferring shorter spans better suited for data extraction
Trained by: Ambika Sukla
Application: Given a passage and a question, pick the shortest span from the passage that answers the question
Training Process: start, end location head on the top of Roberta Base
roberta.large.stsb
See folder: roberta.large.stsb
Trained with STSB dataset
Trained by: Meta/Fairseq
Application: Given two passages, return a score beteen 0 and 1 to evaluate their similarity
Training Process: regression head on top of Roberta Base
roberta.large.phraseqa
See folder: roberta.large.phraseqa
Trained with Roberta 2.0 with the question words removed from the question
Trained By: Batya Stein
Application: Given a passage and phrase (key), extract a value from the passage
Training Process: https://blogs.nlmatics.com/2020/08/25/Optimizing-Transformer-Q&A-Models-for-Naturalistic-Search.html
roberta.large.qasrl
See folder: roberta.large.qasrl
Trained with QASRL dataset
Application: Given a passage, get back values for who, what, when, where etc.
Trained By: Nima Sheikholeslami
roberta.large.qatype.lower.RothWithQ
See folder: roberta.large.qatype.lower.RothWithQ
Trained with the Roth Question Type dataset.
Application: Given a question, return one of the answer types e.g. number, location. See the Roth dataset for full list.
Trained By: Evan Li
roberta.large.io_qa
See folder: roberta.large.io_qa Trained with SQuAD 2.0 dataset
Trained By: Nima Sheikholeslami
Training Process: Use io head to support multiple spans.