---
tags:
- generated_from_trainer
datasets:
- uonlp/CulturaX
metrics:
- accuracy
model-index:
- name: gpt2_cx-cs_00000-00019_50k
  results:
  - task:
      name: Causal Language Modeling
      type: text-generation
    dataset:
      name: uonlp/CulturaX cs
      type: uonlp/CulturaX
      args: cs
    metrics:
    - name: Accuracy
      type: accuracy
      value: 0.38830943632679016
license: mit
language:
- cs
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# gpt2_cx-cs_00000-00019_50k

This model is a fine-tuned version of [](https://huggingface.co/) on the uonlp/CulturaX cs dataset.
It achieves the following results on the evaluation set:
- Loss: 3.5060
- Accuracy: 0.3883

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1.0

### Training results

| Training Loss | Epoch | Step   | Validation Loss | Accuracy |
|:-------------:|:-----:|:------:|:---------------:|:--------:|
| 4.729         | 0.04  | 10000  | 4.6077          | 0.2836   |
| 4.3383        | 0.07  | 20000  | 4.2318          | 0.3162   |
| 4.1706        | 0.11  | 30000  | 4.0651          | 0.3316   |
| 4.0594        | 0.15  | 40000  | 3.9599          | 0.3416   |
| 3.9842        | 0.19  | 50000  | 3.8825          | 0.3487   |
| 3.9298        | 0.22  | 60000  | 3.8244          | 0.3545   |
| 3.8777        | 0.26  | 70000  | 3.7791          | 0.3592   |
| 3.8455        | 0.3   | 80000  | 3.7436          | 0.3629   |
| 3.8104        | 0.33  | 90000  | 3.7120          | 0.3660   |
| 3.7908        | 0.37  | 100000 | 3.6862          | 0.3687   |
| 3.7613        | 0.41  | 110000 | 3.6628          | 0.3712   |
| 3.7492        | 0.45  | 120000 | 3.6434          | 0.3731   |
| 3.7228        | 0.48  | 130000 | 3.6246          | 0.3751   |
| 3.7127        | 0.52  | 140000 | 3.6090          | 0.3767   |
| 3.694         | 0.56  | 150000 | 3.5962          | 0.3783   |
| 3.6871        | 0.59  | 160000 | 3.5831          | 0.3797   |
| 3.6784        | 0.63  | 170000 | 3.5708          | 0.3810   |
| 3.6606        | 0.67  | 180000 | 3.5593          | 0.3823   |
| 3.646         | 0.71  | 190000 | 3.5491          | 0.3835   |
| 3.6453        | 0.74  | 200000 | 3.5410          | 0.3843   |
| 3.6393        | 0.78  | 210000 | 3.5342          | 0.3851   |
| 3.6207        | 0.82  | 220000 | 3.5280          | 0.3857   |
| 3.6288        | 0.86  | 230000 | 3.5218          | 0.3865   |
| 3.6176        | 0.89  | 240000 | 3.5151          | 0.3872   |
| 3.6099        | 0.93  | 250000 | 3.5108          | 0.3878   |
| 3.6093        | 0.97  | 260000 | 3.5079          | 0.3881   |


### Framework versions

- Transformers 4.37.1
- Pytorch 2.1.2+cu121
- Datasets 2.16.1
- Tokenizers 0.15.1