Update README.md
Browse files
README.md
CHANGED
@@ -4,33 +4,28 @@ tags:
|
|
4 |
metrics:
|
5 |
- accuracy
|
6 |
model-index:
|
7 |
-
- name: pygmalion-
|
8 |
results: []
|
9 |
---
|
10 |
|
11 |
-
|
12 |
-
should probably proofread and complete it, then remove this comment. -->
|
13 |
|
14 |
-
|
15 |
-
|
16 |
-
This model is a fine-tuned version of [/notebooks/pygmalion/pygmalion-350m/](https://huggingface.co//notebooks/pygmalion/pygmalion-350m/) on an unknown dataset.
|
17 |
It achieves the following results on the evaluation set:
|
18 |
- Loss: 2.2731
|
19 |
- Accuracy: 0.5187
|
20 |
|
21 |
## Model description
|
22 |
|
23 |
-
|
24 |
-
|
25 |
-
## Intended uses & limitations
|
26 |
|
27 |
-
|
28 |
|
29 |
-
##
|
30 |
|
31 |
-
|
32 |
|
33 |
-
## Training
|
34 |
|
35 |
### Training hyperparameters
|
36 |
|
@@ -43,10 +38,6 @@ The following hyperparameters were used during training:
|
|
43 |
- lr_scheduler_type: linear
|
44 |
- num_epochs: 3.0
|
45 |
|
46 |
-
### Training results
|
47 |
-
|
48 |
-
|
49 |
-
|
50 |
### Framework versions
|
51 |
|
52 |
- Transformers 4.27.0.dev0
|
|
|
4 |
metrics:
|
5 |
- accuracy
|
6 |
model-index:
|
7 |
+
- name: pygmalion-350m
|
8 |
results: []
|
9 |
---
|
10 |
|
11 |
+
# pygmalion-350m
|
|
|
12 |
|
13 |
+
This model is a fine-tuned version of [PygmalionAI/pygmalion-350m](https://huggingface.co/PygmalionAI/pygmalion-350m/) on a 2.4MB dataset.
|
|
|
|
|
14 |
It achieves the following results on the evaluation set:
|
15 |
- Loss: 2.2731
|
16 |
- Accuracy: 0.5187
|
17 |
|
18 |
## Model description
|
19 |
|
20 |
+
A proof-of-concept model based on PygmalionAI/pygmalion-350m, which was in turn based on OPT-350m.
|
|
|
|
|
21 |
|
22 |
+
This model was fine-tuned purely for testing purposes.
|
23 |
|
24 |
+
## Fine-tuning process
|
25 |
|
26 |
+
Fine-tuned on an A100-80GB with HF's `run_clm.py` script. It was run through 3 epochs with 8 batch size using 2.4MB dataset (split 75/25 between training and validation sets).
|
27 |
|
28 |
+
## Training and evaluation data
|
29 |
|
30 |
### Training hyperparameters
|
31 |
|
|
|
38 |
- lr_scheduler_type: linear
|
39 |
- num_epochs: 3.0
|
40 |
|
|
|
|
|
|
|
|
|
41 |
### Framework versions
|
42 |
|
43 |
- Transformers 4.27.0.dev0
|