siddharthl1293's picture
Update README.md
6604d9f verified
metadata
license: apache-2.0
base_model: albert/albert-large-v2
tags:
  - generated_from_trainer
model-index:
  - name: albert-albert-large-v2
    results: []

albert-albert-large-v2

This model is a fine-tuned version of albert/albert-large-v2 on the raw version of the dataset https://huggingface.co/datasets/siddharthl1293/engineering_design_facts. It achieves the following results on the evaluation set:

  • Loss: 0.0032

Model Intent

The model was trained to identify relationship tokens in a sentence when a pair of entities are marked. For more info, please go through the dataset description: https://huggingface.co/datasets/siddharthl1293/engineering_design_facts

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
0.0032 1.0 9378 0.0032

Testing Results

Testing accuracy was calculated on a test set wherein, all relationship tokens need to be identified in an example for the accuracy to be 1. The average testing accuracy across 37,509 testing examples is 0.997.

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.3.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1