victormiller commited on
Commit
1c1fbe8
1 Parent(s): 36c5dd8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +58 -0
README.md CHANGED
@@ -1,3 +1,61 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ language:
4
+ - en
5
+ pipeline_tag: text-generation
6
+ library_name: transformers
7
+ tags:
8
+ - nlp
9
+ - llm
10
  ---
11
+ # K2 - Deciphering Llama 2 70B
12
+
13
+ K2 is a fully transparent large language model on par with Llama 2 - 70B.
14
+
15
+ [insert table with comparison to llama 2 on open llm leaderboard metrics]
16
+
17
+ [insert about the data mix]
18
+
19
+
20
+ The first 10 checkpoints are available for download now.
21
+
22
+ Checkpoint 360[link]
23
+ Checkpoint 359[link]
24
+ Checkpoint 358[link]
25
+ Checkpoint 357[link]
26
+ Checkpoint 356[link]
27
+ Checkpoint 355[link]
28
+ Checkpoint 354[link]
29
+ Checkpoint 353[link]
30
+ Checkpoint 352[link]
31
+ Checkpoint 351[link]
32
+
33
+
34
+
35
+ We are working on release caliber artifacts for the dataset, code, and analysis which will be released over the next few weeks.
36
+
37
+
38
+ ## 🟠 Model Description
39
+
40
+ - **Model type:** Language model with the same architecture as LLaMA-7B
41
+ - **Language(s) (NLP):** English
42
+ - **License:** Apache 2.0
43
+ - **Resources for more information:**
44
+ - [Training Code]
45
+ - [Data Preparation]
46
+ - [Metrics]
47
+ - [Fully processed Amber pretraining data]
48
+
49
+
50
+ ## About LLM360
51
+ LLM360 is an initiative for comprehensive and fully open-sourced LLMs,
52
+ where all training details, model checkpoints, intermediate results, and
53
+ additional analyses are made available to the community. Our goal is to advance
54
+ the field by inviting the community to deepen the understanding of LLMs
55
+ together. As the first step of the project LLM360, we release all intermediate
56
+ model checkpoints, our fully-prepared pre-training dataset, all source code and
57
+ configurations, and training details. We are
58
+ committed to continually pushing the boundaries of LLMs through this open-source
59
+ effort.
60
+
61
+ [Visit us](https://www.llm360.ai/)