Hugging Face's logo

Dev-SriramB
/
XPFaculty-ft

PEFT
TensorBoard
Safetensors
Generated from Trainer
Model card Files Metrics Community
Dev-SriramB commited on
Commit
41f03df
·
verified ·
1 Parent(s): 6165c5d

Dev-SriramB/qa_final01

Browse files
Files changed (5) hide show
  1. README.md +13 -10
  2. adapter_config.json +3 -3
  3. adapter_model.safetensors +2 -2
  4. runs/Feb07_15-29-35_4f00189539be/events.out.tfevents.1738942178.4f00189539be.446.0 +3 -0
  5. training_args.bin +1 -1
README.md CHANGED
@@ -16,7 +16,7 @@ should probably proofread and complete it, then remove this comment. -->
16
 
17
  This model is a fine-tuned version of [TheBloke/Mistral-7B-Instruct-v0.2-GPTQ](https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GPTQ) on the None dataset.
18
  It achieves the following results on the evaluation set:
19
- - Loss: 1.2620
20
 
21
  ## Model description
22
 
@@ -44,26 +44,29 @@ The following hyperparameters were used during training:
44
  - optimizer: Use OptimizerNames.PAGED_ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
45
  - lr_scheduler_type: linear
46
  - lr_scheduler_warmup_steps: 2
47
- - num_epochs: 7
48
  - mixed_precision_training: Native AMP
49
 
50
  ### Training results
51
 
52
  | Training Loss | Epoch | Step | Validation Loss |
53
  |:-------------:|:-----:|:----:|:---------------:|
54
- | 10.0354 | 1.0 | 25 | 2.1524 |
55
- | 8.0146 | 2.0 | 50 | 1.7660 |
56
- | 6.243 | 3.0 | 75 | 1.3734 |
57
- | 5.268 | 4.0 | 100 | 1.2929 |
58
- | 5.0111 | 5.0 | 125 | 1.2709 |
59
- | 4.8748 | 6.0 | 150 | 1.2624 |
60
- | 4.8125 | 7.0 | 175 | 1.2620 |
 
 
 
61
 
62
 
63
  ### Framework versions
64
 
65
  - PEFT 0.14.0
66
- - Transformers 4.47.1
67
  - Pytorch 2.5.1+cu124
68
  - Datasets 3.2.0
69
  - Tokenizers 0.21.0
 
16
 
17
  This model is a fine-tuned version of [TheBloke/Mistral-7B-Instruct-v0.2-GPTQ](https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GPTQ) on the None dataset.
18
  It achieves the following results on the evaluation set:
19
+ - Loss: 1.2424
20
 
21
  ## Model description
22
 
 
44
  - optimizer: Use OptimizerNames.PAGED_ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
45
  - lr_scheduler_type: linear
46
  - lr_scheduler_warmup_steps: 2
47
+ - num_epochs: 10
48
  - mixed_precision_training: Native AMP
49
 
50
  ### Training results
51
 
52
  | Training Loss | Epoch | Step | Validation Loss |
53
  |:-------------:|:-----:|:----:|:---------------:|
54
+ | 2.4997 | 1.0 | 25 | 2.1442 |
55
+ | 1.982 | 2.0 | 50 | 1.7214 |
56
+ | 1.5052 | 3.0 | 75 | 1.3375 |
57
+ | 1.2921 | 4.0 | 100 | 1.2731 |
58
+ | 1.2246 | 5.0 | 125 | 1.2574 |
59
+ | 1.18 | 6.0 | 150 | 1.2512 |
60
+ | 1.1403 | 7.0 | 175 | 1.2428 |
61
+ | 1.1051 | 8.0 | 200 | 1.2458 |
62
+ | 1.0811 | 9.0 | 225 | 1.2420 |
63
+ | 1.0686 | 10.0 | 250 | 1.2424 |
64
 
65
 
66
  ### Framework versions
67
 
68
  - PEFT 0.14.0
69
+ - Transformers 4.48.2
70
  - Pytorch 2.5.1+cu124
71
  - Datasets 3.2.0
72
  - Tokenizers 0.21.0
adapter_config.json CHANGED
@@ -19,12 +19,12 @@
19
  "megatron_core": "megatron.core",
20
  "modules_to_save": null,
21
  "peft_type": "LORA",
22
- "r": 16,
23
  "rank_pattern": {},
24
  "revision": null,
25
  "target_modules": [
26
- "v_proj",
27
- "q_proj"
28
  ],
29
  "task_type": "CAUSAL_LM",
30
  "use_dora": false,
 
19
  "megatron_core": "megatron.core",
20
  "modules_to_save": null,
21
  "peft_type": "LORA",
22
+ "r": 8,
23
  "rank_pattern": {},
24
  "revision": null,
25
  "target_modules": [
26
+ "q_proj",
27
+ "v_proj"
28
  ],
29
  "task_type": "CAUSAL_LM",
30
  "use_dora": false,
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e06b90827d092941ed0a4801a5ff3d4c3a30127ab67382d97a2e5918f13af9c2
3
- size 27280152
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0af397b7982754cda5a356966cc848e69c764c39688e81a0ec3915060e569a93
3
+ size 13648432
runs/Feb07_15-29-35_4f00189539be/events.out.tfevents.1738942178.4f00189539be.446.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:561b7e355a76f9b4c97b0e62788119e4ec45476a338551c4d0aa0a6440dc74cb
3
+ size 10855
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8f57697774a3dec1c12cb3b3f60dc28af49a1d010e9b4799c210c018b1fa5616
3
  size 5304
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:08470a030982ef2f9b1879f96c8aa61a8fe1c05160157935e64ff2c6b68c957f
3
  size 5304