srihariEmids commited on
Commit
a7de9b6
1 Parent(s): a794de4

End of training

Browse files
README.md ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ library_name: peft
4
+ tags:
5
+ - generated_from_trainer
6
+ base_model: microsoft/phi-1_5
7
+ model-index:
8
+ - name: phi-1_5-finetuned-emids-text
9
+ results: []
10
+ ---
11
+
12
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
+ should probably proofread and complete it, then remove this comment. -->
14
+
15
+ # phi-1_5-finetuned-emids-text
16
+
17
+ This model is a fine-tuned version of [microsoft/phi-1_5](https://huggingface.co/microsoft/phi-1_5) on the None dataset.
18
+
19
+ ## Model description
20
+
21
+ More information needed
22
+
23
+ ## Intended uses & limitations
24
+
25
+ More information needed
26
+
27
+ ## Training and evaluation data
28
+
29
+ More information needed
30
+
31
+ ## Training procedure
32
+
33
+
34
+ The following `bitsandbytes` quantization config was used during training:
35
+ - quant_method: bitsandbytes
36
+ - load_in_8bit: False
37
+ - load_in_4bit: True
38
+ - llm_int8_threshold: 6.0
39
+ - llm_int8_skip_modules: None
40
+ - llm_int8_enable_fp32_cpu_offload: False
41
+ - llm_int8_has_fp16_weight: False
42
+ - bnb_4bit_quant_type: nf4
43
+ - bnb_4bit_use_double_quant: True
44
+ - bnb_4bit_compute_dtype: float16
45
+
46
+ ### Training hyperparameters
47
+
48
+ The following hyperparameters were used during training:
49
+ - learning_rate: 0.0002
50
+ - train_batch_size: 4
51
+ - eval_batch_size: 8
52
+ - seed: 42
53
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
54
+ - lr_scheduler_type: cosine
55
+ - training_steps: 1000
56
+
57
+ ### Training results
58
+
59
+
60
+
61
+ ### Framework versions
62
+
63
+ - PEFT 0.7.0
64
+ - Transformers 4.36.0
65
+ - Pytorch 2.1.0+cu118
66
+ - Datasets 2.15.0
67
+ - Tokenizers 0.15.0
adapter_config.json ADDED
@@ -0,0 +1,26 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "microsoft/phi-1_5",
5
+ "bias": "none",
6
+ "fan_in_fan_out": false,
7
+ "inference_mode": true,
8
+ "init_lora_weights": true,
9
+ "layers_pattern": null,
10
+ "layers_to_transform": null,
11
+ "loftq_config": {},
12
+ "lora_alpha": 16,
13
+ "lora_dropout": 0.05,
14
+ "megatron_config": null,
15
+ "megatron_core": "megatron.core",
16
+ "modules_to_save": null,
17
+ "peft_type": "LORA",
18
+ "r": 16,
19
+ "rank_pattern": {},
20
+ "revision": null,
21
+ "target_modules": [
22
+ "out_proj",
23
+ "Wqkv"
24
+ ],
25
+ "task_type": "CAUSAL_LM"
26
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e6916b49b9ea7cd6c1cdabc6c06567661709e53898f425f46fa69d93de77a97
3
+ size 18886928
runs/Dec12_12-21-27_84381c7146fb/events.out.tfevents.1702383690.84381c7146fb.277.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a6d1aa39e5d8e51323e4f74bf40194267bc65fb52ce18ab3a22e24c6d305cb63
3
+ size 6815
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d7f98495648da8ac12ba63aa85f462329221bd40d4e44383488fc97bbeefb3fe
3
+ size 4728