Bad GPT
Based on the Let's build GPT video from Andrej Karpathy.
This is just an attempt to recreate the transformer Andrej made in his video with the goal of learning more about torch, transformers, and neural networks in general.
To run, make sure python
3.10
and poetry
are installed. You can then run poetry install
to get the dependencies (it's just torch and numpy).
Finally, you can run the code with poetry run python ./main.py
Note that the first run will train the model and then save the trained weights to model.pth
. Subsequent runs will load these weights.
Inference Providers
NEW
This model is not currently available via any of the supported third-party Inference Providers, and
HF Inference API was unable to determine this model's library.