wangxh07 commited on
Commit
9d1dff5
1 Parent(s): 353b4d8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -5
README.md CHANGED
@@ -33,7 +33,18 @@ The project's `initial phase` introduced a knowledge extraction LLM based on LLa
33
  - The **full-scale pre-training code** (providing conversion, construction, and loading of large corpora) and **LoRA instruction fine-tuning code** are open-sourced (support multi-machine multi-GPU).
34
 
35
 
36
- All weights have been uploaded to Hugging Face. The ZhiXi differential weights can be found [here](https://huggingface.co/zjunlp/zhixi-13B-Diff), and the LoRA weights can be found [here](https://huggingface.co/zjunlp/zhixi-13B-LoRA).
 
 
 
 
 
 
 
 
 
 
 
37
 
38
  ## Contents
39
 
@@ -606,6 +617,9 @@ Due to time constraints, hardware limitations, and technical reasons, our model
606
 
607
  - Instruction tuning using full tuning instead of LoRA version is being trained and will be released soon.
608
  - New instruction tuning weights using LoRA will be updated shortly.
 
 
 
609
  - ......
610
 
611
 
@@ -626,7 +640,7 @@ Due to time constraints, hardware limitations, and technical reasons, our model
626
 
627
  <h2 id="7">7. Others</h2>
628
 
629
- <h3 id="7-1">7.1 Contributors(in random order)</h3>
630
 
631
  Pretraining:Xiang Chen, Jintian Zhang, Xiaozhuan Liang
632
 
@@ -638,7 +652,7 @@ Tool learning and Multimodal:Shuofei Qiao, Yixin Ou, Lei Li
638
 
639
  Model Editing and Safety:Yunzhi Yao, Peng Wang, Siyuan Cheng, Bozhong Tian, Mengru Wang, Zhoubo Li
640
 
641
- Model Testing and Deployment:Yinuo Jiang, Yuqi Zhu, Hongbin Ye, Zekun Xi
642
 
643
 
644
  <h3 id="7-2">7.2 Citation</h3>
@@ -647,7 +661,7 @@ If you use our repository, please cite the following related papers:
647
 
648
  ```bibtex
649
  @article{cama,
650
- author = {Jintian Zhang, Xiaohan Wang, Honghao Gui, Xiang Chen, Yinuo Jiang, Zhen Bi, Jing Chen, Shengyu Mao, Shuofei Qiao, Xiaozhuan Liang, Yixin Ou, Ruinan Fang, Zekun Xi, Shumin Deng, Huajun Chen, Ningyu Zhang},
651
  title = {DeepKE-LLM: A Large Language Model Based Knowledge Extraction Toolkit},
652
  year = {2023},
653
  publisher = {GitHub},
@@ -657,7 +671,6 @@ If you use our repository, please cite the following related papers:
657
  ```
658
 
659
 
660
-
661
  <h3 id="7-3">7.3 Acknowledgment</h3>
662
 
663
  We are very grateful to the following open source projects for their help:
 
33
  - The **full-scale pre-training code** (providing conversion, construction, and loading of large corpora) and **LoRA instruction fine-tuning code** are open-sourced (support multi-machine multi-GPU).
34
 
35
 
36
+ All weights have been uploaded to HuggingFace🤗. It should be noted that all the following effects are based on `ZhiXi-13B-Diff`. If you have downloaded `ZhiXi-13B-Diff-fp16`, there may be some variations in the effects.
37
+ | Model Name | Train Method | Weight Type | Size | Download Link | Notes |
38
+ | -------------- | ------------ | --------------------- | -------- | ---------------------------------- | ------------------------------------------------------------ |
39
+ | ZhiXi-13B-Diff | Full Pretraining | Differential Weights | 48GB | [HuggingFace](https://huggingface.co/zjunlp/zhixi-13b-diff) <br/> [GoogleDrive](https://drive.google.com/drive/folders/1PZDqZNaBJYQYeON1-9aFBtagktEWAtUK?usp=drive_link)| Restoring the pre-trained weights (i.e. **ZhiXi-13B**) needs to match the weights of `LLaMA-13B`, please refer to [here](#2-2) for specific instructions. |
40
+ | ZhiXi-13B-Diff-fp16 | Full Pretraining | Differential Weights(fp16) | 24GB | [HuggingFace](https://huggingface.co/zjunlp/zhixi-13b-diff-fp16) <br/> [Google Drive](https://drive.google.com/drive/folders/1LYm-HUSSQ5Rl8nqZcswdiSpcP9xYTXaO?usp=sharing) | The main difference with `ZhiXi-13B-Diff` is the adoption of the `fp16` format for storage, which reduces memory usage. However, it may result in slight differences in the weights obtained from our actual training, which can slightly impact performance. For specific usage instructions, please refer to [here](#2-2) for specific instructions. |
41
+ | ZhiXi-13B-LoRA | LoRA Instruction-tuning | LoRA Weights | 251MB | [HuggingFace](https://huggingface.co/zjunlp/zhixi-13b-lora) <br/> [GoogleDrive](https://drive.google.com/drive/folders/1GLyaWIyDIayudrQhb_tJYoNPAUk1xByS?usp=drive_link) | It needs to be used with **ZhiXi-13B**. For specific instructions, please refer to [here](#2-4). |
42
+ | ZhiXi-7B Series | Coming soon | Coming soon | Coming soon | Coming soon | Coming soon |
43
+
44
+
45
+ ## NEWS
46
+ - \[**June 2023**\] The project name has been changed from CaMA to KnowLM.
47
+ - \[**June 2023**\] Release the first version of pre-trained weights and the LoRA weights.
48
 
49
  ## Contents
50
 
 
617
 
618
  - Instruction tuning using full tuning instead of LoRA version is being trained and will be released soon.
619
  - New instruction tuning weights using LoRA will be updated shortly.
620
+ - New models (Llama-7b, Falcon-7b) are being trained (We have limited GPUs!).
621
+ - New abilities such as molecule and protein generation with [Mol-Instructions](https://github.com/zjunlp/Mol-Instructions), a large-scale biomolecules instruction dataset for large language models.
622
+ - supporting llama.cpp
623
  - ......
624
 
625
 
 
640
 
641
  <h2 id="7">7. Others</h2>
642
 
643
+ <h3 id="7-1">7.1 Contributors(In Random Order)</h3>
644
 
645
  Pretraining:Xiang Chen, Jintian Zhang, Xiaozhuan Liang
646
 
 
652
 
653
  Model Editing and Safety:Yunzhi Yao, Peng Wang, Siyuan Cheng, Bozhong Tian, Mengru Wang, Zhoubo Li
654
 
655
+ Model Testing and Deployment:Yinuo Jiang, Yuqi Zhu, Hongbin Ye, Zekun Xi, Xinrong Li
656
 
657
 
658
  <h3 id="7-2">7.2 Citation</h3>
 
661
 
662
  ```bibtex
663
  @article{cama,
664
+ author = {Jintian Zhang, Ningyu Zhang, Xiaohan Wang, Honghao Gui, Xiang Chen, Yinuo Jiang, Zhen Bi, Jing Chen, Shengyu Mao, Shuofei Qiao, Xiaozhuan Liang, Yixin Ou, Runnan Fang, Zekun Xi, Xin Xu, Huajun Chen},
665
  title = {DeepKE-LLM: A Large Language Model Based Knowledge Extraction Toolkit},
666
  year = {2023},
667
  publisher = {GitHub},
 
671
  ```
672
 
673
 
 
674
  <h3 id="7-3">7.3 Acknowledgment</h3>
675
 
676
  We are very grateful to the following open source projects for their help: