---
tags:
- generated_from_trainer
metrics:
- spearmanr
model-index:
- name: thermo-predictor-thermo-evotuning-prot_bert
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# thermo-predictor-thermo-evotuning-prot_bert

This model is a fine-tuned version of [thundaa/thermo-evotuning-prot_bert](https://huggingface.co/thundaa/thermo-evotuning-prot_bert) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.1617
- Spearmanr: 0.6914

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 4e-05
- train_batch_size: 256
- eval_batch_size: 256
- seed: 42
- gradient_accumulation_steps: 64
- total_train_batch_size: 16384
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Spearmanr |
|:-------------:|:-----:|:----:|:---------------:|:---------:|
| 0.4734        | 0.68  | 2    | 0.3146          | 0.3359    |
| 0.4392        | 1.68  | 4    | 0.2936          | 0.3407    |
| 0.4034        | 2.68  | 6    | 0.2633          | 0.3696    |
| 0.3669        | 3.68  | 8    | 0.2437          | 0.3903    |
| 0.3496        | 4.68  | 10   | 0.2377          | 0.4102    |
| 0.3351        | 5.68  | 12   | 0.2285          | 0.4204    |
| 0.3289        | 6.68  | 14   | 0.2267          | 0.4180    |
| 0.3267        | 7.68  | 16   | 0.2258          | 0.4242    |
| 0.3177        | 8.68  | 18   | 0.2206          | 0.4295    |
| 0.3116        | 9.68  | 20   | 0.2150          | 0.4365    |
| 0.3039        | 10.68 | 22   | 0.2115          | 0.4365    |
| 0.2985        | 11.68 | 24   | 0.2062          | 0.4469    |
| 0.2927        | 12.68 | 26   | 0.2045          | 0.4531    |
| 0.2885        | 13.68 | 28   | 0.2005          | 0.4603    |
| 0.2838        | 14.68 | 30   | 0.1987          | 0.4690    |
| 0.2806        | 15.68 | 32   | 0.1975          | 0.4744    |
| 0.2772        | 16.68 | 34   | 0.1970          | 0.4765    |
| 0.2728        | 17.68 | 36   | 0.1939          | 0.4845    |
| 0.2684        | 18.68 | 38   | 0.1931          | 0.4858    |
| 0.2641        | 19.68 | 40   | 0.1925          | 0.4936    |
| 0.2608        | 20.68 | 42   | 0.1905          | 0.4929    |
| 0.2566        | 21.68 | 44   | 0.1886          | 0.5049    |
| 0.2518        | 22.68 | 46   | 0.1875          | 0.5095    |
| 0.2467        | 23.68 | 48   | 0.1869          | 0.5141    |
| 0.2424        | 24.68 | 50   | 0.1859          | 0.5161    |
| 0.2375        | 25.68 | 52   | 0.1850          | 0.5223    |
| 0.2329        | 26.68 | 54   | 0.1851          | 0.5210    |
| 0.2279        | 27.68 | 56   | 0.1850          | 0.5294    |
| 0.2226        | 28.68 | 58   | 0.1837          | 0.5310    |


### Framework versions

- Transformers 4.18.0
- Pytorch 1.11.0
- Datasets 2.1.0
- Tokenizers 0.12.1