Model Card for Model ID

This model was finetuned on RACE for multiple choice (text classification). The initial model used was distilbert-uncased-base https://huggingface.co/distilbert-uncased-base

The model was trained using the code from https://github.com/zphang/lrqa. Please refer to and cite the authors.

Model Details

  • Initial model: distilbert-uncased-base
  • LR: 1e-5
  • Epochs: 3
  • Warmup Ratio: 0.1 (10%)
  • Batch Size: 16
  • Max Seq Len: 512

Model Description

  • Model type: [DistilBERT]
  • Language(s) (NLP): [English]
  • License: [Apache-2.0]
  • Finetuned from model [optional]: [distilbert-uncased-base]

Model Sources [optional]

Bias, Risks, and Limitations

[More Information Needed]

How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

Training Details

Training Data

[More Information Needed]

Model Examination [optional]

[More Information Needed]

Environmental Impact

  • Hardware Type: A100 - 40GB
  • Hours used: 4
  • Cloud Provider: Private
  • Compute Region: Portugal
  • Carbon Emitted: 0.18 kgCO2

Experiments were conducted using a private infrastructure, which has a carbon efficiency of 0.178 kgCO$_2$eq/kWh. A cumulative of 4 hours of computation was performed on hardware of type A100 PCIe 40/80GB (TDP of 250W). Total emissions are estimated to be 0.18 kgCO$_2$eq of which 0 percent were directly offset. Estimations were conducted using the \href{https://mlco2.github.io/impact#compute}{MachineLearning Impact calculator} presented in \cite{lacoste2019quantifying}.

Downloads last month
9
Inference API
Inference API (serverless) does not yet support transformers models for this pipeline type.

Dataset used to train gsgoncalves/distilbert-base-uncased-race