yi-01-ai
commited on
Commit
•
3ff16a1
1
Parent(s):
4d9b983
Auto Sync from git://github.com/01-ai/Yi.git/commit/f7523ae4443a4d627c61bbf09fc8fa8a1adca47b
Browse files
README.md
CHANGED
@@ -119,29 +119,29 @@ pipeline_tag: text-generation
|
|
119 |
|
120 |
- 🙌 Targeted as a bilingual language model and trained on 3T multilingual corpus, the Yi series models become one of the strongest LLM worldwide, showing promise in language understanding, commonsense reasoning, reading comprehension, and more. For example,
|
121 |
|
122 |
-
- For English language capability, the Yi series models ranked 2nd (just behind GPT-4), outperforming other LLMs (such as
|
123 |
|
124 |
- For Chinese language capability, the Yi series models landed in 2nd place (following GPT-4), surpassing other LLMs (such as Baidu ERNIE, Qwen, and Baichuan) on the [SuperCLUE](https://www.superclueai.com/) in Oct 2023.
|
125 |
|
126 |
-
- 🙏 (Credits to
|
127 |
|
128 |
-
<details style="display: inline;"><summary> If you're interested in Yi's adoption of
|
129 |
|
130 |
> 💡 TL;DR
|
131 |
>
|
132 |
-
> The Yi series models adopt the same model architecture as
|
133 |
|
134 |
-
- Both Yi and
|
135 |
|
136 |
-
- Grounded in the Transformer architecture,
|
137 |
|
138 |
-
- Thanks to the Transformer and
|
139 |
|
140 |
-
- However, the Yi series models are NOT derivatives of
|
141 |
|
142 |
-
- As
|
143 |
|
144 |
-
- Developing in a unique and proprietary way, Yi has independently created its own high-quality training datasets, efficient training pipelines, and robust training infrastructure entirely from the ground up. This effort has led to excellent performance with Yi series models ranking just behind GPT4 and surpassing
|
145 |
</ul>
|
146 |
</details>
|
147 |
|
@@ -302,7 +302,7 @@ If you prefer to deploy Yi models locally,
|
|
302 |
- [Docker](#quick-start---docker)
|
303 |
- [conda-lock](#quick-start---conda-lock)
|
304 |
|
305 |
-
-
|
306 |
|
307 |
#### 🎯 Not to deploy Yi locally
|
308 |
|
@@ -336,6 +336,10 @@ If you want to chat with Yi with more customizable options (e.g., system prompt,
|
|
336 |
- [Yi-34B-Chat](https://platform.lingyiwanwu.com/) (Yi official beta)
|
337 |
- Access is available through a whitelist. Welcome to apply (fill out a form in [English](https://cn.mikecrm.com/l91ODJf) or [Chinese](https://cn.mikecrm.com/gnEZjiQ)).
|
338 |
|
|
|
|
|
|
|
|
|
339 |
### Quick start - pip
|
340 |
|
341 |
This tutorial guides you through every step of running **Yi-34B-Chat locally on an A800 (80G)** and then performing inference.
|
@@ -344,7 +348,7 @@ This tutorial guides you through every step of running **Yi-34B-Chat locally on
|
|
344 |
|
345 |
- Make sure Python 3.10 or a later version is installed.
|
346 |
|
347 |
-
- If you want to run other Yi models, see [software and hardware requirements](#deployment)
|
348 |
|
349 |
#### Step 1: Prepare your environment
|
350 |
|
@@ -435,6 +439,10 @@ Then you can see an output similar to the one below. 🥳
|
|
435 |
|
436 |
</details>
|
437 |
|
|
|
|
|
|
|
|
|
438 |
### Quick start - Docker
|
439 |
<details>
|
440 |
<summary> Run Yi-34B-chat locally with Docker: a step-by-step guide. ⬇️</summary>
|
@@ -474,6 +482,7 @@ To install the dependencies, follow these steps:
|
|
474 |
2. Execute <code>micromamba install -y -n yi -f conda-lock.yml</code> to create a conda environment named <code>yi</code> and install the necessary dependencies.
|
475 |
</details>
|
476 |
|
|
|
477 |
### Quick start - llama.cpp
|
478 |
<details>
|
479 |
<summary> Run Yi-chat-6B-2bits locally with llama.cpp: a step-by-step guide. ⬇️</summary>
|
@@ -610,6 +619,10 @@ Now you have successfully asked a question to the Yi model and got an answer!
|
|
610 |
</ul>
|
611 |
</details>
|
612 |
|
|
|
|
|
|
|
|
|
613 |
### Web demo
|
614 |
|
615 |
You can build a web UI demo for Yi **chat** models (note that Yi base models are not supported in this senario).
|
@@ -628,6 +641,10 @@ You can access the web UI by entering the address provided in the console into y
|
|
628 |
|
629 |
![Quick start - web demo](https://github.com/01-ai/Yi/blob/main/assets/img/yi_34b_chat_web_demo.gif?raw=true)
|
630 |
|
|
|
|
|
|
|
|
|
631 |
### Fine-tuning
|
632 |
|
633 |
```bash
|
@@ -749,10 +766,14 @@ cd finetune/scripts
|
|
749 |
bash run_eval.sh
|
750 |
```
|
751 |
|
752 |
-
Then you'll see the answer from both the base model and the finetuned model
|
753 |
</ul>
|
754 |
</details>
|
755 |
|
|
|
|
|
|
|
|
|
756 |
### Quantization
|
757 |
|
758 |
#### GPT-Q
|
@@ -906,6 +927,10 @@ Below are detailed minimum VRAM requirements under different batch use cases.
|
|
906 |
| Yi-34B | 72 GB | 4 x RTX 4090 <br> A800 (80 GB) |
|
907 |
| Yi-34B-200K | 200 GB | 4 x A800 (80 GB) |
|
908 |
|
|
|
|
|
|
|
|
|
909 |
### Learning hub
|
910 |
|
911 |
<details>
|
@@ -923,18 +948,22 @@ At the same time, we also warmly invite you to join our collaborative effort by
|
|
923 |
With all these resources at your fingertips, you're ready to start your exciting journey with Yi. Happy learning! 🥳
|
924 |
|
925 |
#### Tutorials
|
926 |
-
|
927 |
| Type | Deliverable | Date | Author |
|
928 |
|-------------|--------------------------------------------------------|----------------|----------------|
|
929 |
| Blog | [实测零一万物Yi-VL多模态语言模型:能准确“识图吃瓜”](https://mp.weixin.qq.com/s/fu4O9XvJ03JhimsEyI-SsQ) | 2024-02-02 | [苏洋](https://github.com/soulteary) |
|
930 |
| Blog | [本地运行零一万物 34B 大模型,使用 Llama.cpp & 21G 显存](https://zhuanlan.zhihu.com/p/668921042) | 2023-11-26 | [苏洋](https://github.com/soulteary) |
|
931 |
-
| Blog | [Running Yi-34B-Chat locally using LlamaEdge](https://www.secondstate.io/articles/yi-34b/) | 2023-11-30 | [Second State](https://github.com/second-state) |
|
932 |
| Blog | [零一万物模型折腾笔记:官方 Yi-34B 模型基础使用](https://zhuanlan.zhihu.com/p/671387298) | 2023-12-10 | [苏洋](https://github.com/soulteary) |
|
933 |
| Blog | [CPU 混合推理,非常见大模型量化方案:“二三五六” 位量化方案](https://zhuanlan.zhihu.com/p/671698216) | 2023-12-12 | [苏洋](https://github.com/soulteary) |
|
934 |
-
| Blog | [单卡 3 小时训练 Yi-6B 大模型 Agent:基于
|
935 |
| Blog | [零一万物开源Yi-VL多模态大模型,魔搭社区推理&微调最佳实践来啦!](https://zhuanlan.zhihu.com/p/680098411) | 2024-01-26 | [ModelScope](https://github.com/modelscope) |
|
936 |
-
| Video | [只需 24G 显存,用 vllm 跑起来 Yi-34B 中英双语大模型](https://www.bilibili.com/video/BV17t4y1f7Ee/) | 2023-12-28 | 漆妮妮 |
|
937 |
-
|
|
|
|
|
|
|
|
|
|
|
938 |
</details>
|
939 |
|
940 |
|
@@ -964,9 +993,9 @@ Yi has a comprehensive ecosystem, offering a range of tools, services, and model
|
|
964 |
|
965 |
### 💦 Upstream
|
966 |
|
967 |
-
The Yi series models follow the same model architecture as
|
968 |
|
969 |
-
For example, the Yi series models are saved in the format of the
|
970 |
|
971 |
```python
|
972 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
@@ -976,6 +1005,10 @@ tokenizer = AutoTokenizer.from_pretrained("01-ai/Yi-34b", use_fast=False)
|
|
976 |
model = AutoModelForCausalLM.from_pretrained("01-ai/Yi-34b", device_map="auto")
|
977 |
```
|
978 |
|
|
|
|
|
|
|
|
|
979 |
### 🌊 Downstream
|
980 |
|
981 |
> 💡 Tip
|
@@ -1072,6 +1105,10 @@ The Yi-34B and Yi-34B-200K models stand out as the top performers among open-sou
|
|
1072 |
- **Falcon-180B caveat**: Falcon-180B was not tested on QuAC and OBQA due to technical constraints. Its performance score is an average from other tasks, and considering the generally lower scores of these two tasks, Falcon-180B's capabilities are likely not underestimated.
|
1073 |
</details>
|
1074 |
|
|
|
|
|
|
|
|
|
1075 |
# 🟢 Who can use Yi?
|
1076 |
|
1077 |
Everyone! 🙌 ✅
|
|
|
119 |
|
120 |
- 🙌 Targeted as a bilingual language model and trained on 3T multilingual corpus, the Yi series models become one of the strongest LLM worldwide, showing promise in language understanding, commonsense reasoning, reading comprehension, and more. For example,
|
121 |
|
122 |
+
- For English language capability, the Yi series models ranked 2nd (just behind GPT-4), outperforming other LLMs (such as Llama2-chat-70B, Claude 2, and ChatGPT) on the [AlpacaEval Leaderboard](https://tatsu-lab.github.io/alpaca_eval/) in Dec 2023.
|
123 |
|
124 |
- For Chinese language capability, the Yi series models landed in 2nd place (following GPT-4), surpassing other LLMs (such as Baidu ERNIE, Qwen, and Baichuan) on the [SuperCLUE](https://www.superclueai.com/) in Oct 2023.
|
125 |
|
126 |
+
- 🙏 (Credits to Llama) Thanks to the Transformer and Llama open-source communities, as they reducing the efforts required to build from scratch and enabling the utilization of the same tools within the AI ecosystem.
|
127 |
|
128 |
+
<details style="display: inline;"><summary> If you're interested in Yi's adoption of Llama architecture and license usage policy, see <span style="color: green;">Yi's relation with Llama.</span> ⬇️</summary> <ul> <br>
|
129 |
|
130 |
> 💡 TL;DR
|
131 |
>
|
132 |
+
> The Yi series models adopt the same model architecture as Llama but are **NOT** derivatives of Llama.
|
133 |
|
134 |
+
- Both Yi and Llama are all based on the Transformer structure, which has been the standard architecture for large language models since 2018.
|
135 |
|
136 |
+
- Grounded in the Transformer architecture, Llama has become a new cornerstone for the majority of state-of-the-art open-source models due to its excellent stability, reliable convergence, and robust compatibility. This positions Llama as the recognized foundational framework for models including Yi.
|
137 |
|
138 |
+
- Thanks to the Transformer and Llama architectures, other models can leverage their power, reducing the effort required to build from scratch and enabling the utilization of the same tools within their ecosystems.
|
139 |
|
140 |
+
- However, the Yi series models are NOT derivatives of Llama, as they do not use Llama's weights.
|
141 |
|
142 |
+
- As Llama's structure is employed by the majority of open-source models, the key factors of determining model performance are training datasets, training pipelines, and training infrastructure.
|
143 |
|
144 |
+
- Developing in a unique and proprietary way, Yi has independently created its own high-quality training datasets, efficient training pipelines, and robust training infrastructure entirely from the ground up. This effort has led to excellent performance with Yi series models ranking just behind GPT4 and surpassing Llama on the [Alpaca Leaderboard in Dec 2023](https://tatsu-lab.github.io/alpaca_eval/).
|
145 |
</ul>
|
146 |
</details>
|
147 |
|
|
|
302 |
- [Docker](#quick-start---docker)
|
303 |
- [conda-lock](#quick-start---conda-lock)
|
304 |
|
305 |
+
- 🙋♀️ and you have **limited** resources (for example, a MacBook Pro), you can use [llama.cpp](#quick-start---llamacpp).
|
306 |
|
307 |
#### 🎯 Not to deploy Yi locally
|
308 |
|
|
|
336 |
- [Yi-34B-Chat](https://platform.lingyiwanwu.com/) (Yi official beta)
|
337 |
- Access is available through a whitelist. Welcome to apply (fill out a form in [English](https://cn.mikecrm.com/l91ODJf) or [Chinese](https://cn.mikecrm.com/gnEZjiQ)).
|
338 |
|
339 |
+
<p align="right"> [
|
340 |
+
<a href="#top">Back to top ⬆️ </a> ]
|
341 |
+
</p>
|
342 |
+
|
343 |
### Quick start - pip
|
344 |
|
345 |
This tutorial guides you through every step of running **Yi-34B-Chat locally on an A800 (80G)** and then performing inference.
|
|
|
348 |
|
349 |
- Make sure Python 3.10 or a later version is installed.
|
350 |
|
351 |
+
- If you want to run other Yi models, see [software and hardware requirements](#deployment).
|
352 |
|
353 |
#### Step 1: Prepare your environment
|
354 |
|
|
|
439 |
|
440 |
</details>
|
441 |
|
442 |
+
<p align="right"> [
|
443 |
+
<a href="#top">Back to top ⬆️ </a> ]
|
444 |
+
</p>
|
445 |
+
|
446 |
### Quick start - Docker
|
447 |
<details>
|
448 |
<summary> Run Yi-34B-chat locally with Docker: a step-by-step guide. ⬇️</summary>
|
|
|
482 |
2. Execute <code>micromamba install -y -n yi -f conda-lock.yml</code> to create a conda environment named <code>yi</code> and install the necessary dependencies.
|
483 |
</details>
|
484 |
|
485 |
+
|
486 |
### Quick start - llama.cpp
|
487 |
<details>
|
488 |
<summary> Run Yi-chat-6B-2bits locally with llama.cpp: a step-by-step guide. ⬇️</summary>
|
|
|
619 |
</ul>
|
620 |
</details>
|
621 |
|
622 |
+
<p align="right"> [
|
623 |
+
<a href="#top">Back to top ⬆️ </a> ]
|
624 |
+
</p>
|
625 |
+
|
626 |
### Web demo
|
627 |
|
628 |
You can build a web UI demo for Yi **chat** models (note that Yi base models are not supported in this senario).
|
|
|
641 |
|
642 |
![Quick start - web demo](https://github.com/01-ai/Yi/blob/main/assets/img/yi_34b_chat_web_demo.gif?raw=true)
|
643 |
|
644 |
+
<p align="right"> [
|
645 |
+
<a href="#top">Back to top ⬆️ </a> ]
|
646 |
+
</p>
|
647 |
+
|
648 |
### Fine-tuning
|
649 |
|
650 |
```bash
|
|
|
766 |
bash run_eval.sh
|
767 |
```
|
768 |
|
769 |
+
Then you'll see the answer from both the base model and the finetuned model.
|
770 |
</ul>
|
771 |
</details>
|
772 |
|
773 |
+
<p align="right"> [
|
774 |
+
<a href="#top">Back to top ⬆️ </a> ]
|
775 |
+
</p>
|
776 |
+
|
777 |
### Quantization
|
778 |
|
779 |
#### GPT-Q
|
|
|
927 |
| Yi-34B | 72 GB | 4 x RTX 4090 <br> A800 (80 GB) |
|
928 |
| Yi-34B-200K | 200 GB | 4 x A800 (80 GB) |
|
929 |
|
930 |
+
<p align="right"> [
|
931 |
+
<a href="#top">Back to top ⬆️ </a> ]
|
932 |
+
</p>
|
933 |
+
|
934 |
### Learning hub
|
935 |
|
936 |
<details>
|
|
|
948 |
With all these resources at your fingertips, you're ready to start your exciting journey with Yi. Happy learning! 🥳
|
949 |
|
950 |
#### Tutorials
|
951 |
+
##### Chinese tutorials
|
952 |
| Type | Deliverable | Date | Author |
|
953 |
|-------------|--------------------------------------------------------|----------------|----------------|
|
954 |
| Blog | [实测零一万物Yi-VL多模态语言模型:能准确“识图吃瓜”](https://mp.weixin.qq.com/s/fu4O9XvJ03JhimsEyI-SsQ) | 2024-02-02 | [苏洋](https://github.com/soulteary) |
|
955 |
| Blog | [本地运行零一万物 34B 大模型,使用 Llama.cpp & 21G 显存](https://zhuanlan.zhihu.com/p/668921042) | 2023-11-26 | [苏洋](https://github.com/soulteary) |
|
|
|
956 |
| Blog | [零一万物模型折腾笔记:官方 Yi-34B 模型基础使用](https://zhuanlan.zhihu.com/p/671387298) | 2023-12-10 | [苏洋](https://github.com/soulteary) |
|
957 |
| Blog | [CPU 混合推理,非常见大模型量化方案:“二三五六” 位量化方案](https://zhuanlan.zhihu.com/p/671698216) | 2023-12-12 | [苏洋](https://github.com/soulteary) |
|
958 |
+
| Blog | [单卡 3 小时训练 Yi-6B 大模型 Agent:基于 Llama Factory 实战](https://zhuanlan.zhihu.com/p/678989191) | 2024-01-22 | [郑耀威](https://github.com/hiyouga) |
|
959 |
| Blog | [零一万物开源Yi-VL多模态大模型,魔搭社区推理&微调最佳实践来啦!](https://zhuanlan.zhihu.com/p/680098411) | 2024-01-26 | [ModelScope](https://github.com/modelscope) |
|
960 |
+
| Video | [只需 24G 显存,用 vllm 跑起来 Yi-34B 中英双语大模型](https://www.bilibili.com/video/BV17t4y1f7Ee/) | 2023-12-28 | [漆妮妮](https://space.bilibili.com/1262370256) |
|
961 |
+
|
962 |
+
##### English tutorials
|
963 |
+
| Type | Deliverable | Date | Author |
|
964 |
+
|-------------|--------------------------------------------------------|----------------|----------------|
|
965 |
+
| Blog | [Running Yi-34B-Chat locally using LlamaEdge](https://www.secondstate.io/articles/yi-34b/) | 2023-11-30 | [Second State](https://github.com/second-state) |
|
966 |
+
| Video | [Install Yi 34B Locally - Chinese English Bilingual LLM](https://www.youtube.com/watch?v=CVQvj4Wrh4w&t=476s) | 2023-11-05 | [Fahd Mirza](https://www.youtube.com/watch?v=CVQvj4Wrh4w&t=476s) |
|
967 |
</details>
|
968 |
|
969 |
|
|
|
993 |
|
994 |
### 💦 Upstream
|
995 |
|
996 |
+
The Yi series models follow the same model architecture as Llama. By choosing Yi, you can leverage existing tools, libraries, and resources within the Llama ecosystem, eliminating the need to create new tools and enhancing development efficiency.
|
997 |
|
998 |
+
For example, the Yi series models are saved in the format of the Llama model. You can directly use `LlamaForCausalLM` and `LlamaTokenizer` to load the model. For more information, see [Use the chat model](#31-use-the-chat-model).
|
999 |
|
1000 |
```python
|
1001 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
1005 |
model = AutoModelForCausalLM.from_pretrained("01-ai/Yi-34b", device_map="auto")
|
1006 |
```
|
1007 |
|
1008 |
+
<p align="right"> [
|
1009 |
+
<a href="#top">Back to top ⬆️ </a> ]
|
1010 |
+
</p>
|
1011 |
+
|
1012 |
### 🌊 Downstream
|
1013 |
|
1014 |
> 💡 Tip
|
|
|
1105 |
- **Falcon-180B caveat**: Falcon-180B was not tested on QuAC and OBQA due to technical constraints. Its performance score is an average from other tasks, and considering the generally lower scores of these two tasks, Falcon-180B's capabilities are likely not underestimated.
|
1106 |
</details>
|
1107 |
|
1108 |
+
<p align="right"> [
|
1109 |
+
<a href="#top">Back to top ⬆️ </a> ]
|
1110 |
+
</p>
|
1111 |
+
|
1112 |
# 🟢 Who can use Yi?
|
1113 |
|
1114 |
Everyone! 🙌 ✅
|