safe049 commited on
Commit
88db4ad
·
verified ·
1 Parent(s): 7f41a25

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +45 -3
README.md CHANGED
@@ -1,22 +1,64 @@
1
  ---
2
  base_model: NeverSleep/Lumimaid-v0.2-8B
 
3
  tags:
4
  - text-generation-inference
5
  - transformers
6
  - unsloth
7
  - llama
8
- - gguf
 
9
  license: apache-2.0
10
  language:
11
  - en
 
 
 
12
  ---
13
 
14
- # Uploaded model
15
-
 
 
 
 
 
 
16
  - **Developed by:** safe049
17
  - **License:** apache-2.0
18
  - **Finetuned from model :** NeverSleep/Lumimaid-v0.2-8B
 
19
 
20
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
21
 
22
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  base_model: NeverSleep/Lumimaid-v0.2-8B
3
+ quantized_model: safe049/SmolLumi-8B-Instruct-GGUF
4
  tags:
5
  - text-generation-inference
6
  - transformers
7
  - unsloth
8
  - llama
9
+ - trl
10
+ - sft
11
  license: apache-2.0
12
  language:
13
  - en
14
+ datasets:
15
+ - HuggingFaceTB/smol-smoltalk
16
+ library_name: transformers
17
  ---
18
 
19
+ # SmolLumi-8B-Instruct
20
+ ```
21
+ ____ _ _ _
22
+ / ___| _ __ ___ ___ | | | _ _ _ __ ___ (_)
23
+ \___ \| '_ ` _ \ / _ \| | | | | | | '_ ` _ \| |
24
+ ___) | | | | | | (_) | | |__| |_| | | | | | | |
25
+ |____/|_| |_| |_|\___/|_|_____\__,_|_| |_| |_|_|
26
+ ```
27
  - **Developed by:** safe049
28
  - **License:** apache-2.0
29
  - **Finetuned from model :** NeverSleep/Lumimaid-v0.2-8B
30
+ - **Original[Non-Quantized] :** [safe049/SmolLumi-8B-Instruct](https://huggingface.co/safe049/SmolLumi-8B-Instruct)
31
 
32
  This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
33
 
34
  [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
35
+
36
+ --------------------------------------------------
37
+ # Description
38
+ ## Arguments:
39
+ - per_device_train_batch_size = 2,
40
+ - gradient_accumulation_steps = 4,
41
+ - warmup_steps = 5,
42
+ - max_steps = 60,
43
+ - learning_rate = 2e-4,
44
+ - fp16 = not is_bfloat16_supported(),
45
+ - bf16 = is_bfloat16_supported(),
46
+ - logging_steps = 1,
47
+ - optim = "adamw_8bit",
48
+ - weight_decay = 0.01,
49
+ - lr_scheduler_type = "linear",
50
+ - seed = 3407
51
+ -
52
+ ## Used Dataset
53
+ - [HuggingFaceTB/smol-smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk/tree/main/data)
54
+
55
+ ## Used Library
56
+ - transformers
57
+ - unsloth
58
+ - trl
59
+ - sft
60
+ ---------------------------------------------
61
+ # More
62
+
63
+ Yet another model created cuz of boring
64
+ **This Model is Uncensored**, it might generate illegal,non-moral contents,and I am not reponsable for that.