distilbert-base-uncased-finetuned-bib-lang-grammar

This model is a fine-tuned version of distilbert/distilbert-base-uncased on the dataset RickBrannan/categorize_bib_lang_grammar.

Intended uses & limitations

This model is intended to be used to locate discussions that utilize grammatical terminology within resources like biblical commentaries or study notes.

Training and evaluation data

This model is trained on RickBrannan/categorize_bib_lang_grammar, which is a collection of 2,700+ sentences categorized as NOT-GRAMMAR or GRAMMAR.

For details on the dataset and its sources, see the Dataset Card.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 2

Framework versions

• Transformers 4.46.3
• Pytorch 2.5.1+cpu
• Datasets 3.1.0
• Tokenizers 0.20.3