metadata

license: mit
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: bert-paper-classifier
    results: []

bert-paper-classifier

This model is a fine-tuned version of microsoft/BiomedNLP-PubMedBERT-base-uncased-abstract on the dataset with titles and abstracts of papers from arXiv.

Intended uses & limitations

This model is intended to predict the arXiv taxonomy terms given the paper title (and optionally its abstract) — this is relevant for papers in mathematics, computer science, physics, etc. For example, it is likely to predict cs.CL (computation and language) as the mostl likely category for the original transformers paper.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 128
eval_batch_size: 32
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 3

Framework versions

Transformers 4.28.1
Pytorch 2.0.0+cu117
Datasets 2.11.0
Tokenizers 0.13.3