Update README.md
Browse files
README.md
CHANGED
@@ -1,22 +1,64 @@
|
|
1 |
---
|
2 |
base_model: NeverSleep/Lumimaid-v0.2-8B
|
|
|
3 |
tags:
|
4 |
- text-generation-inference
|
5 |
- transformers
|
6 |
- unsloth
|
7 |
- llama
|
8 |
-
-
|
|
|
9 |
license: apache-2.0
|
10 |
language:
|
11 |
- en
|
|
|
|
|
|
|
12 |
---
|
13 |
|
14 |
-
#
|
15 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
16 |
- **Developed by:** safe049
|
17 |
- **License:** apache-2.0
|
18 |
- **Finetuned from model :** NeverSleep/Lumimaid-v0.2-8B
|
|
|
19 |
|
20 |
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
21 |
|
22 |
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
base_model: NeverSleep/Lumimaid-v0.2-8B
|
3 |
+
quantized_model: safe049/SmolLumi-8B-Instruct-GGUF
|
4 |
tags:
|
5 |
- text-generation-inference
|
6 |
- transformers
|
7 |
- unsloth
|
8 |
- llama
|
9 |
+
- trl
|
10 |
+
- sft
|
11 |
license: apache-2.0
|
12 |
language:
|
13 |
- en
|
14 |
+
datasets:
|
15 |
+
- HuggingFaceTB/smol-smoltalk
|
16 |
+
library_name: transformers
|
17 |
---
|
18 |
|
19 |
+
# SmolLumi-8B-Instruct
|
20 |
+
```
|
21 |
+
____ _ _ _
|
22 |
+
/ ___| _ __ ___ ___ | | | _ _ _ __ ___ (_)
|
23 |
+
\___ \| '_ ` _ \ / _ \| | | | | | | '_ ` _ \| |
|
24 |
+
___) | | | | | | (_) | | |__| |_| | | | | | | |
|
25 |
+
|____/|_| |_| |_|\___/|_|_____\__,_|_| |_| |_|_|
|
26 |
+
```
|
27 |
- **Developed by:** safe049
|
28 |
- **License:** apache-2.0
|
29 |
- **Finetuned from model :** NeverSleep/Lumimaid-v0.2-8B
|
30 |
+
- **Original[Non-Quantized] :** [safe049/SmolLumi-8B-Instruct](https://huggingface.co/safe049/SmolLumi-8B-Instruct)
|
31 |
|
32 |
This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
|
33 |
|
34 |
[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
|
35 |
+
|
36 |
+
--------------------------------------------------
|
37 |
+
# Description
|
38 |
+
## Arguments:
|
39 |
+
- per_device_train_batch_size = 2,
|
40 |
+
- gradient_accumulation_steps = 4,
|
41 |
+
- warmup_steps = 5,
|
42 |
+
- max_steps = 60,
|
43 |
+
- learning_rate = 2e-4,
|
44 |
+
- fp16 = not is_bfloat16_supported(),
|
45 |
+
- bf16 = is_bfloat16_supported(),
|
46 |
+
- logging_steps = 1,
|
47 |
+
- optim = "adamw_8bit",
|
48 |
+
- weight_decay = 0.01,
|
49 |
+
- lr_scheduler_type = "linear",
|
50 |
+
- seed = 3407
|
51 |
+
-
|
52 |
+
## Used Dataset
|
53 |
+
- [HuggingFaceTB/smol-smoltalk](https://huggingface.co/datasets/HuggingFaceTB/smol-smoltalk/tree/main/data)
|
54 |
+
|
55 |
+
## Used Library
|
56 |
+
- transformers
|
57 |
+
- unsloth
|
58 |
+
- trl
|
59 |
+
- sft
|
60 |
+
---------------------------------------------
|
61 |
+
# More
|
62 |
+
|
63 |
+
Yet another model created cuz of boring
|
64 |
+
**This Model is Uncensored**, it might generate illegal,non-moral contents,and I am not reponsable for that.
|