readme update
Browse files
README.md
CHANGED
|
@@ -42,3 +42,7 @@ The Bigram Language Model is based on the Transformer architecture, which has be
|
|
| 42 |
|
| 43 |
This architecture allows the Bigram Language Model to capture complex dependencies in the input data and generate text that follows the patterns and style of the training data. It is a fundamental building block for a wide range of natural language processing tasks and can be further extended and fine-tuned for specific applications.
|
| 44 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
|
| 43 |
This architecture allows the Bigram Language Model to capture complex dependencies in the input data and generate text that follows the patterns and style of the training data. It is a fundamental building block for a wide range of natural language processing tasks and can be further extended and fine-tuned for specific applications.
|
| 44 |
|
| 45 |
+
## Training Data
|
| 46 |
+
The data used for training the GPT models can be found here:
|
| 47 |
+
- [Shakespeare dataset](https://github.com/karpathy/char-rnn/blob/6f9487a6fe5b420b7ca9afb0d7c078e37c1d1b4e/data/tinyshakespeare/input.txt)
|
| 48 |
+
- [Wikiedpia dataset](https://www.kaggle.com/datasets/mikeortman/wikipedia-sentences)
|