Locutusque
commited on
Commit
·
7356321
1
Parent(s):
5d18c0a
Update README.md
Browse files
README.md
CHANGED
@@ -51,12 +51,18 @@ The model is trained on a large dataset of conversational data, consisting of in
|
|
51 |
The model architecture used in this model is GPT-2, a transformer-based language model that is capable of generating high-quality text with a wide range of styles and tones. The GPT-2 architecture consists of a multi-layered decoder-only transformer, with self-attention mechanisms that allow the model to capture long-term dependencies and generate coherent text.
|
52 |
|
53 |
## Evaluation Metrics
|
54 |
-
The model is evaluated based on several metrics, including loss, reward, penalty, BLEU score, and perplexity. The loss metric is calculated during training and reflects the difference between the predicted output and the actual output. The reward metric is based on the number of correct words generated by the model, while the penalty metric penalizes the model for repeating words consecutively. The BLEU score measures the similarity between the generated text and the ground truth text, while the perplexity metric measures how well the model is able to predict the next word in a sequence. During
|
55 |
|
56 |
- BLEU score: 52
|
57 |
- Accuracy: 53
|
58 |
- perplexity: 4.3
|
59 |
|
|
|
|
|
|
|
|
|
|
|
|
|
60 |
## Limitations and Bias
|
61 |
This model is not suitable for all use cases due to its limited training time on a weak computer. As a result, it may produce irrelevant or nonsensical responses. For optimal performance, I recommend using a GPU with at least 16 GB of VRAM and downloading the model manually instead of using the Transformers library. Here's how you should deploy the model:
|
62 |
|
|
|
51 |
The model architecture used in this model is GPT-2, a transformer-based language model that is capable of generating high-quality text with a wide range of styles and tones. The GPT-2 architecture consists of a multi-layered decoder-only transformer, with self-attention mechanisms that allow the model to capture long-term dependencies and generate coherent text.
|
52 |
|
53 |
## Evaluation Metrics
|
54 |
+
The model is evaluated based on several metrics, including loss, reward, penalty, BLEU score, and perplexity. The loss metric is calculated during training and reflects the difference between the predicted output and the actual output. The reward metric is based on the number of correct words generated by the model, while the penalty metric penalizes the model for repeating words consecutively. The BLEU score measures the similarity between the generated text and the ground truth text, while the perplexity metric measures how well the model is able to predict the next word in a sequence. During training, the model achieved the following metrics:
|
55 |
|
56 |
- BLEU score: 52
|
57 |
- Accuracy: 53
|
58 |
- perplexity: 4.3
|
59 |
|
60 |
+
- Evaluation metrics:
|
61 |
+
- | Task |Version|Metric|Value| |Stderr|
|
62 |
+
|--------|------:|------|----:|---|-----:|
|
63 |
+
|pubmedqa| 0|acc |0.536|± |0.0223|
|
64 |
+
|
65 |
+
|
66 |
## Limitations and Bias
|
67 |
This model is not suitable for all use cases due to its limited training time on a weak computer. As a result, it may produce irrelevant or nonsensical responses. For optimal performance, I recommend using a GPU with at least 16 GB of VRAM and downloading the model manually instead of using the Transformers library. Here's how you should deploy the model:
|
68 |
|