danielpark commited on
Commit
3dcdd6b
1 Parent(s): 5b52374

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +29 -22
README.md CHANGED
@@ -8,6 +8,14 @@ pipeline_tag: text-generation
8
  ---
9
 
10
 
 
 
 
 
 
 
 
 
11
  # The project is currently in progress. Please refrain from using weights and datasets.
12
  KORANI is derived from GORANI, a project within llama2 that experiments with the distribution of appropriate datasets to transfer or distill knowledge based on English datasets. Officially, it's called Grid Of Ranvier Node In llama2 (GORANI), based on the biological term Ranvier Node, and aims to explore the optimal dataset for transferring knowledge in various languages and specific domains. Due to strict licensing issues with English datasets, gorani is primarily for research purposes. Therefore, we are refining and training a commercially usable Korean dataset on top of llama2, based on the experimental results of the GORANI project, and this project is named KORANI (Korean GORANI).
13
  - We are currently conducting experiments using various techniques such as max sequence length, rope scaling, attention sinks, and flash attention 2.
@@ -16,27 +24,7 @@ The most stringent non-commercial use license (CC-BY-NC-4.0) among the licenses
16
  - Once the training is complete, we will provide information about the datasets used along with the official release.
17
  - For `GORANI`, it is intended for research purposes, and for the Korean language model, `KORANI`, it can be used under a commercial use license.
18
 
19
- ## Update
20
- - Since we cannot control resources, we will record the schedule retrospectively.
21
-
22
- | Update Schedule | Task Description | Status |
23
- |-----------------|----------------------------|--------|
24
- | 23-10-05 | Completed training - 19.7k 13b weight (specific data)| Done |
25
- | 23-10-06 | Submitted hf model weights (REV 01) | Done |
26
- | 23-10-20 | Q.C | On Process |
27
- | 23-10- | Completed training - 50k 13b weight | |
28
- | 23-10- | Q.C | |
29
- | 23-10- | Submitted hf model weights | |
30
- | 23-10- | Completed training - 100k 13b weight | |
31
- | 23-10- | Q.C | |
32
- | 23-10- | Q.A | |
33
- | 23-11- | Official weight release | |
34
-
35
- # GORANI 100k
36
-
37
- - Model: [danielpark/gorani-100k-llama2-13b-instruct](https://huggingface.co/danielpark/gorani-100k-llama2-13b-instruct)
38
- - Dataset: [danielpark/gorani-100k](https://huggingface.co/danielpark/gorani-100k)
39
- - **License**: This model is licensed under the Meta's [LLaMA2 license](https://github.com/facebookresearch/llama/blob/main/LICENSE). You may not use it commercially, and you must adhere to the licenses of the included datasets. Therefore, we currently adopt the strictest and most restrictive license. Please refrain from using it for commercial purposes under any circumstances until an official license is issued.
40
 
41
  ## Template
42
  I use llama2-13b with LFM, but I have used it without a default system message. If a system message is specified in some datasets, I use that content.
@@ -54,10 +42,29 @@ I use llama2-13b with LFM, but I have used it without a default system message.
54
  {New_Assistant_Answer}
55
  ```
56
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57
  ## Caution
58
  The model weights and dataset have not been properly curated yet and are strictly prohibited for use under any license. In relation to this, the developers do not assume any responsibility, either implicitly or explicitly.
59
 
60
- ## Updates
 
61
  | Revision | Commit Hash | Updated | Train Process | Status |
62
  | ---------------|------------------------------------------------------------|------------|------------------|---------------|
63
  | Revision 01 | [6d30494fa8da84128499d55075eef57094336d03](https://huggingface.co/danielpark/gorani-100k-llama2-13b-instruct/commit/6d30494fa8da84128499d55075eef57094336d03) | 23.10.04 | 19,740/100,000 | On Training |
 
8
  ---
9
 
10
 
11
+ # GORANI 100k
12
+ - LFM: [llama2-13b-chat](https://huggingface.co/meta-llama/Llama-2-13b-chat-hf)
13
+ - Model: [danielpark/gorani-100k-llama2-13b-instruct](https://huggingface.co/danielpark/gorani-100k-llama2-13b-instruct)
14
+ - Dataset: [danielpark/gorani-100k](https://huggingface.co/danielpark/gorani-100k)
15
+ - **License**: This model is licensed under the Meta's [LLaMA2 license](https://github.com/facebookresearch/llama/blob/main/LICENSE). You may not use it commercially, and you must adhere to the licenses of the included datasets. Therefore, we currently adopt the strictest and most restrictive license. Please refrain from using it for commercial purposes under any circumstances until an official license is issued.
16
+
17
+ <br>
18
+
19
  # The project is currently in progress. Please refrain from using weights and datasets.
20
  KORANI is derived from GORANI, a project within llama2 that experiments with the distribution of appropriate datasets to transfer or distill knowledge based on English datasets. Officially, it's called Grid Of Ranvier Node In llama2 (GORANI), based on the biological term Ranvier Node, and aims to explore the optimal dataset for transferring knowledge in various languages and specific domains. Due to strict licensing issues with English datasets, gorani is primarily for research purposes. Therefore, we are refining and training a commercially usable Korean dataset on top of llama2, based on the experimental results of the GORANI project, and this project is named KORANI (Korean GORANI).
21
  - We are currently conducting experiments using various techniques such as max sequence length, rope scaling, attention sinks, and flash attention 2.
 
24
  - Once the training is complete, we will provide information about the datasets used along with the official release.
25
  - For `GORANI`, it is intended for research purposes, and for the Korean language model, `KORANI`, it can be used under a commercial use license.
26
 
27
+ <br>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
 
29
  ## Template
30
  I use llama2-13b with LFM, but I have used it without a default system message. If a system message is specified in some datasets, I use that content.
 
42
  {New_Assistant_Answer}
43
  ```
44
 
45
+
46
+ ## Update
47
+ - Since we cannot control resources, we will record the schedule retrospectively.
48
+
49
+ | Update Schedule | Task Description | Status |
50
+ |-----------------|----------------------------|--------|
51
+ | 23-10-05 | Completed training - 19.7k 13b weight (specific data)| Done |
52
+ | 23-10-06 | Submitted hf model weights (REV 01) | Done |
53
+ | 23-10-20 | Q.C | On Process |
54
+ | 23-10- | Completed training - 50k 13b weight | |
55
+ | 23-10- | Q.C | |
56
+ | 23-10- | Submitted hf model weights | |
57
+ | 23-10- | Completed training - 100k 13b weight | |
58
+ | 23-10- | Q.C | |
59
+ | 23-10- | Q.A | |
60
+ | 23-11- | Official weight release | |
61
+
62
+
63
  ## Caution
64
  The model weights and dataset have not been properly curated yet and are strictly prohibited for use under any license. In relation to this, the developers do not assume any responsibility, either implicitly or explicitly.
65
 
66
+
67
+ ## Revisions
68
  | Revision | Commit Hash | Updated | Train Process | Status |
69
  | ---------------|------------------------------------------------------------|------------|------------------|---------------|
70
  | Revision 01 | [6d30494fa8da84128499d55075eef57094336d03](https://huggingface.co/danielpark/gorani-100k-llama2-13b-instruct/commit/6d30494fa8da84128499d55075eef57094336d03) | 23.10.04 | 19,740/100,000 | On Training |