concedo commited on
Commit
699446a
1 Parent(s): beb92dd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +34 -0
README.md CHANGED
@@ -1,3 +1,37 @@
1
  ---
2
  license: apache-2.0
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ language:
4
+ - en
5
  ---
6
+ <div align="center">
7
+
8
+ # KobbleTiny-1.1B
9
+ </div>
10
+
11
+ This is a finetune of https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T trained on a small 50mb subset of the Kobble Dataset.
12
+ Training was done in under 2 hours on a single Nvidia RTX 2060 Mobile GPU with qLora (LR 1.5e-4, rank 8, alpha 16, batch size 1, gradient acc. 4, 1024 ctx).
13
+
14
+ ## Dataset and Objectives
15
+
16
+ The Kobble Dataset is a semi-private aggregated dataset made from multiple online sources and web scrapes.
17
+ It contains content chosen and formatted specifically to work with KoboldAI software and Kobold Lite.
18
+
19
+ #### Dataset Categories:
20
+ - Instruct: Single turn instruct examples presented in the Alpaca format, with an emphasis on uncensored and unrestricted responses.
21
+ - Chat: Two participant roleplay conversation logs in a multi-turn raw chat format that KoboldAI uses.
22
+ - Story: Unstructured fiction excerpts, including literature containing various erotic and provocative content.
23
+
24
+ <!-- prompt-template start -->
25
+ ## Prompt template: Alpaca
26
+
27
+ ```
28
+ ### Instruction:
29
+ {prompt}
30
+
31
+ ### Response:
32
+ ```
33
+
34
+ <!-- prompt-template end -->
35
+
36
+ **Note:** *No assurances will be provided about the **origins, safety, or copyright status** of this model, or of **any content** within the Kobble dataset.*
37
+ *If you belong to a country or organization that has strict AI laws or restrictions against unlabelled or unrestricted content, you are advised not to use this model.*