File size: 953 Bytes
67a526d 900c716 a0c273b 67a526d 900c716 67a526d f39bc62 ee2a78a a0c273b 67a526d a0c273b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 |
---
license: apache-2.0
base_model: Qwen/Qwen2-1.5B
metrics:
- accuracy
datasets:
- BEE-spoke-data/stepbasin-books
language:
- en
---
[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/pszemraj/long-generation-tests/runs/ethp25f9)
# Qwen2-1.5B-stepbasin-books
> [!IMPORTANT]
> this was finetuned at 16384 context length
This is an experiment on long context text **generation** (i.e. 6k+ tokens generated) to evaluate if/when generation breaks down, etc. As such, all the data on which this model has been fine-tuned are full-length books.
## Details
This model is a fine-tuned version of [Qwen/Qwen2-1.5B](https://huggingface.co/Qwen/Qwen2-1.5B) on https://github.com/stepbasin/books/tree/master/books
It achieves the following results on the evaluation set:
- Loss: 2.8110
- Accuracy: 0.4298
- Num Input Tokens Seen: 44040192 |