---
license: mit
language:
- en
library_name: transformers
pipeline_tag: text-generation
tags:
- gpt2
---
# FineNeo: A simple way to finetune your very own GPT-Neo model.
> Created by Tekkonetes with debugging help from ChatGPT.
> @Tekkonetes (HuggingFace) / @pxlmastrXD (Replit)

So, you want to fine-tune a GPT-Neo model? Well, here's the simplest script you will get. It uses a text dataset to fine-tune, and it also will tune the model fairly quickly. Normally, it takes about 5 seconds for an epoch to finish, so here are some estimated times:
|Epochs|Time (Seconds)| Adjusted time
|--|--|--|
| 1 | 5 | |
| 10 | 50 | 
| 50 | 250 | 4m 10s
| 100 | 500 | 8m 20s

Yes, it's fairly fast. However, it depends on which GPT-Neo model you're fine-tuning. For example, the chart above is the `EleutherAI/GPT-Neo-125M` model. If you use the `Gpt-Neo-1.3B` model, it will probably take longer.

## Using the script.
First, download the `tune.py` file to your computer. Then (optional) set up a virtual environment:
```bash
python -m venv venv
source venv/bin/activate
```
Now, install the needed packages:
```bash
pip install transformers torch
```
Finally, create your dataset, modify the file to use the dataset, and run the python script.
```bash
python tune.py
```
Your model and tokenizer will appear in the `fine-tuned-gpt-neo` directory. You can then use transformers to run the model, or upload the files to the HuggingFace hub.

Best of luck!
    - Tekkonetes
    - @pxlmastrXD (Replit)