roberta / README.md
ansukla's picture
Update README.md
c81e4a6 verified
metadata
license: apache-2.0

This repo offers a set of fairseq roberta based models fine tuned to specific NLP tasks. These models are:

  • Small
  • Easy to host on T4 or V100
  • 100x faster than using LLMs for similar tasks
  • Easy to fine tune

All the models below were trained at Nlmatics Corp. from 2019-2023 with base model from: https://github.com/facebookresearch/fairseq/blob/main/examples/roberta/README.md

To run the models:

Use https://github.com/nlmatics/nlm-model-service

To acccess the models

Use https://github.com/nlmatics/nlm-utils

To train the models

TBD

List of Models

Click on each model to see details:

roberta.large.boolq

Location: roberta.large.boolq

Trained with MNLI + Boolq

Trained by: Evan Li

Application: Given a passage and a question, answer the question with yes, no or unsure.

Training Process: https://blogs.nlmatics.com/2020/03/12/Boolean-Question-Answering-with-Neutral-Labels.html

roberta.large.qa

See folder: roberta.large.qa

Trained with SQuAD 2.0 + Custom Dataset preferring shorter spans better suited for data extraction

Trained by: Ambika Sukla

Application: Given a passage and a question, pick the shortest span from the passage that answers the question

Training Process: start, end location head on the top of Roberta Base

roberta.large.stsb

See folder: roberta.large.stsb

Trained with STSB dataset

Trained by: Meta/Fairseq

Application: Given two passages, return a score beteen 0 and 1 to evaluate their similarity

Training Process: regression head on top of Roberta Base

roberta.large.phraseqa

See folder: roberta.large.phraseqa

Trained with Roberta 2.0 with the question words removed from the question

Trained By: Batya Stein

Application: Given a passage and phrase (key), extract a value from the passage

Training Process: https://blogs.nlmatics.com/2020/08/25/Optimizing-Transformer-Q&A-Models-for-Naturalistic-Search.html

roberta.large.qasrl

See folder: roberta.large.qasrl

Trained with QASRL dataset

Application: Given a passage, get back values for who, what, when, where etc.

Trained By: Nima Sheikholeslami

roberta.large.qatype.lower.RothWithQ

See folder: roberta.large.qatype.lower.RothWithQ

Trained with the Roth Question Type dataset.

Application: Given a question, return one of the answer types e.g. number, location. See the Roth dataset for full list.

Trained By: Evan Li

roberta.large.io_qa

See folder: roberta.large.io_qa Trained with SQuAD 2.0 dataset

Trained By: Nima Sheikholeslami

Training Process: Use io head to support multiple spans.