---
tags:
- generated_from_trainer
datasets:
- uonlp/CulturaX
metrics:
- accuracy
model-index:
- name: gpt2+morf_u0-30-50-x_cx-en_00000-00009_50k
  results:
  - task:
      name: Causal Language Modeling
      type: text-generation
    dataset:
      name: uonlp/CulturaX en
      type: uonlp/CulturaX
      args: en
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.39500633343599556
license: mit
language:
- en
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# gpt2+morf_u0-30-50-x_cx-en_00000-00009_50k

This model is a fine-tuned version of [](https://huggingface.co/) on the uonlp/CulturaX en dataset.
It achieves the following results on the evaluation set:

- Loss: 3.3667
- Accuracy: 0.3950

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:

- learning_rate: 5e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1.0

### Training results

| Training Loss | Epoch |  Step  | Validation Loss | Accuracy |
| :-----------: | :---: | :----: | :-------------: | :------: |
|    4.3227     | 0.04  | 10000  |     4.2268      |  0.3161  |
|    4.0305     | 0.07  | 20000  |     3.9455      |  0.3393  |
|    3.8916     | 0.11  | 30000  |     3.8194      |  0.3502  |
|    3.8104     | 0.15  | 40000  |     3.7340      |  0.3580  |
|    3.7491     | 0.19  | 50000  |     3.6770      |  0.3633  |
|    3.7062     | 0.22  | 60000  |     3.6288      |  0.3679  |
|    3.6724     | 0.26  | 70000  |     3.5938      |  0.3714  |
|    3.6399     |  0.3  | 80000  |     3.5652      |  0.3743  |
|    3.6147     | 0.34  | 90000  |     3.5396      |  0.3768  |
|    3.5946     | 0.37  | 100000 |     3.5158      |  0.3791  |
|    3.5726     | 0.41  | 110000 |     3.4986      |  0.3809  |
|    3.5631     | 0.45  | 120000 |     3.4819      |  0.3826  |
|    3.5459     | 0.49  | 130000 |     3.4678      |  0.3842  |
|    3.5304     | 0.52  | 140000 |     3.4535      |  0.3857  |
|    3.5245     | 0.56  | 150000 |     3.4430      |  0.3867  |
|    3.5124     |  0.6  | 160000 |     3.4329      |  0.3877  |
|     3.501     | 0.63  | 170000 |     3.4223      |  0.3890  |
|    3.4934     | 0.67  | 180000 |     3.4130      |  0.3901  |
|    3.4863     | 0.71  | 190000 |     3.4042      |  0.3909  |
|    3.4799     | 0.75  | 200000 |     3.3991      |  0.3914  |
|    3.4682     | 0.78  | 210000 |     3.3909      |  0.3924  |
|    3.4667     | 0.82  | 220000 |     3.3852      |  0.3930  |
|    3.4564     | 0.86  | 230000 |     3.3790      |  0.3936  |
|    3.4581     |  0.9  | 240000 |     3.3753      |  0.3941  |
|    3.4553     | 0.93  | 250000 |     3.3710      |  0.3945  |
|    3.4508     | 0.97  | 260000 |     3.3680      |  0.3949  |

### Framework versions

- Transformers 4.37.1
- Pytorch 2.1.2+cu121
- Datasets 2.16.1
- Tokenizers 0.15.1