yi-01-ai
commited on
Commit
•
1fa5680
1
Parent(s):
72ba73b
Auto Sync from git://github.com/01-ai/Yi.git/commit/a7cc760cb87e7c21a330ddca0d14c14565e72b14
Browse files
README.md
CHANGED
@@ -81,7 +81,9 @@ pipeline_tag: text-generation
|
|
81 |
- [🟢 How to use Yi?](#-how-to-use-yi)
|
82 |
- [Quick start](#quick-start)
|
83 |
- [Choose your path](#choose-your-parth)
|
84 |
-
- [
|
|
|
|
|
85 |
- [Fine tune](#fine-tune)
|
86 |
- [Quantization](#quantization)
|
87 |
- [Deployment](https://github.com/01-ai/Yi/blob/main/docs/deployment.md)
|
@@ -231,7 +233,9 @@ sequence length and can be extended to 32K during inference time.
|
|
231 |
|
232 |
- [Quick start](#quick-start)
|
233 |
- [Choose your path](#choose-your-parth)
|
234 |
-
- [
|
|
|
|
|
235 |
- [Fine tune](#fine-tune)
|
236 |
- [Quantization](#quantization)
|
237 |
- [Deployment](https://github.com/01-ai/Yi/blob/main/docs/deployment.md)
|
@@ -252,7 +256,7 @@ Select one of the following paths to begin your journey with Yi!
|
|
252 |
If you prefer to deploy Yi models locally,
|
253 |
|
254 |
- 🙋♀️ and you have **sufficient** resources (for example, NVIDIA A800 80GB), you can choose one of the following methods:
|
255 |
-
- [pip](#
|
256 |
- [Docker](https://github.com/01-ai/Yi/blob/main/docs/README_legacy.md#11-docker)
|
257 |
- [conda-lock](https://github.com/01-ai/Yi/blob/main/docs/README_legacy.md#12-local-development-environment)
|
258 |
|
@@ -290,18 +294,18 @@ If you want to chat with Yi with more customizable options (e.g., system prompt,
|
|
290 |
- [Yi-34B-Chat](https://platform.lingyiwanwu.com/) (Yi official beta)
|
291 |
- Access is available through a whitelist. Welcome to apply (fill out a form in [English](https://cn.mikecrm.com/l91ODJf) or [Chinese](https://cn.mikecrm.com/gnEZjiQ)).
|
292 |
|
293 |
-
|
294 |
|
295 |
This tutorial guides you through every step of running Yi (Yi-34B-Chat) locally and then performing inference.
|
296 |
|
297 |
-
|
298 |
|
299 |
- This tutorial assumes you are running the **Yi-34B-Chat** with an **A800 (80G)** GPU.
|
300 |
- For detailed deployment requirements to run Yi models, see [hardware requirements]( https://github.com/01-ai/Yi/blob/main/docs/deployment.md).
|
301 |
|
302 |
- Make sure Python 3.10 or later version is installed.
|
303 |
|
304 |
-
|
305 |
|
306 |
To set up the environment and install the required packages, execute the following command.
|
307 |
|
@@ -311,7 +315,7 @@ cd yi
|
|
311 |
pip install -r requirements.txt
|
312 |
```
|
313 |
|
314 |
-
|
315 |
|
316 |
You can download the weights and tokenizer of Yi models from the following sources:
|
317 |
|
@@ -319,11 +323,11 @@ You can download the weights and tokenizer of Yi models from the following sourc
|
|
319 |
- [ModelScope](https://www.modelscope.cn/organization/01ai/)
|
320 |
- [WiseModel](https://wisemodel.cn/organization/01.AI)
|
321 |
|
322 |
-
|
323 |
|
324 |
You can perform inference with Yi chat or base models as below.
|
325 |
|
326 |
-
|
327 |
|
328 |
1. Create a file named `quick_start.py` and copy the following content to it.
|
329 |
|
@@ -366,7 +370,7 @@ You can perform inference with Yi chat or base models as below.
|
|
366 |
Hello! How can I assist you today?
|
367 |
```
|
368 |
|
369 |
-
|
370 |
|
371 |
The steps are similar to [Run Yi chat model](#run-yi-chat-model).
|
372 |
|
@@ -390,6 +394,30 @@ Then you can see an output similar to the one below. 🥳
|
|
390 |
|
391 |
</details>
|
392 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
393 |
### Finetuning
|
394 |
|
395 |
```bash
|
@@ -685,4 +713,4 @@ are fully open for academic research and free commercial usage with permission
|
|
685 |
via applications. All usage must adhere to the [Yi Series Models Community License Agreement 2.1](https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMENT.txt).
|
686 |
For free commercial use, you only need to send an email to [get official commercial permission](https://www.lingyiwanwu.com/yi-license).
|
687 |
|
688 |
-
<div align="right"> [ <a href="#building-the-next-generation-of-open-source-and-bilingual-llms">Back to top ⬆️ </a> ] </div>
|
|
|
81 |
- [🟢 How to use Yi?](#-how-to-use-yi)
|
82 |
- [Quick start](#quick-start)
|
83 |
- [Choose your path](#choose-your-parth)
|
84 |
+
- [pip](#pip)
|
85 |
+
- [llama.cpp](https://github.com/01-ai/Yi/blob/main/docs/yi_llama.cpp.md)
|
86 |
+
- [Web demo](#web-demo)
|
87 |
- [Fine tune](#fine-tune)
|
88 |
- [Quantization](#quantization)
|
89 |
- [Deployment](https://github.com/01-ai/Yi/blob/main/docs/deployment.md)
|
|
|
233 |
|
234 |
- [Quick start](#quick-start)
|
235 |
- [Choose your path](#choose-your-parth)
|
236 |
+
- [pip](#pip)
|
237 |
+
- [llama.cpp](https://github.com/01-ai/Yi/blob/main/docs/yi_llama.cpp.md)
|
238 |
+
- [Web demo](#web-demo)
|
239 |
- [Fine tune](#fine-tune)
|
240 |
- [Quantization](#quantization)
|
241 |
- [Deployment](https://github.com/01-ai/Yi/blob/main/docs/deployment.md)
|
|
|
256 |
If you prefer to deploy Yi models locally,
|
257 |
|
258 |
- 🙋♀️ and you have **sufficient** resources (for example, NVIDIA A800 80GB), you can choose one of the following methods:
|
259 |
+
- [pip](#pip)
|
260 |
- [Docker](https://github.com/01-ai/Yi/blob/main/docs/README_legacy.md#11-docker)
|
261 |
- [conda-lock](https://github.com/01-ai/Yi/blob/main/docs/README_legacy.md#12-local-development-environment)
|
262 |
|
|
|
294 |
- [Yi-34B-Chat](https://platform.lingyiwanwu.com/) (Yi official beta)
|
295 |
- Access is available through a whitelist. Welcome to apply (fill out a form in [English](https://cn.mikecrm.com/l91ODJf) or [Chinese](https://cn.mikecrm.com/gnEZjiQ)).
|
296 |
|
297 |
+
### pip
|
298 |
|
299 |
This tutorial guides you through every step of running Yi (Yi-34B-Chat) locally and then performing inference.
|
300 |
|
301 |
+
#### Step 0: Prerequistes
|
302 |
|
303 |
- This tutorial assumes you are running the **Yi-34B-Chat** with an **A800 (80G)** GPU.
|
304 |
- For detailed deployment requirements to run Yi models, see [hardware requirements]( https://github.com/01-ai/Yi/blob/main/docs/deployment.md).
|
305 |
|
306 |
- Make sure Python 3.10 or later version is installed.
|
307 |
|
308 |
+
#### Step 1: Prepare your environment
|
309 |
|
310 |
To set up the environment and install the required packages, execute the following command.
|
311 |
|
|
|
315 |
pip install -r requirements.txt
|
316 |
```
|
317 |
|
318 |
+
#### Step 2: Download the Yi model
|
319 |
|
320 |
You can download the weights and tokenizer of Yi models from the following sources:
|
321 |
|
|
|
323 |
- [ModelScope](https://www.modelscope.cn/organization/01ai/)
|
324 |
- [WiseModel](https://wisemodel.cn/organization/01.AI)
|
325 |
|
326 |
+
#### Step 3: Perform inference
|
327 |
|
328 |
You can perform inference with Yi chat or base models as below.
|
329 |
|
330 |
+
##### Perform inference with Yi chat model
|
331 |
|
332 |
1. Create a file named `quick_start.py` and copy the following content to it.
|
333 |
|
|
|
370 |
Hello! How can I assist you today?
|
371 |
```
|
372 |
|
373 |
+
##### Perform inference with Yi base model
|
374 |
|
375 |
The steps are similar to [Run Yi chat model](#run-yi-chat-model).
|
376 |
|
|
|
394 |
|
395 |
</details>
|
396 |
|
397 |
+
### Run Yi with llama.cpp
|
398 |
+
|
399 |
+
If you have limited resources, you can try [llama.cpp](https://github.com/ggerganov/llama.cpp) or [ollama.cpp](https://ollama.ai/) (especially for Chinese users) to run Yi models in a few minutes locally.
|
400 |
+
|
401 |
+
For a step-by-step tutorial,, see [Run Yi with llama.cpp](https://github.com/01-ai/Yi/edit/main/docs/yi_llama.cpp.md).
|
402 |
+
|
403 |
+
### Web demo
|
404 |
+
|
405 |
+
You can build a web UI demo for Yi **chat** models (note that Yi base models are not supported in this senario).
|
406 |
+
|
407 |
+
[Step 1: Prepare your environment](#step-1-prepare-your-environment).
|
408 |
+
|
409 |
+
[Step 2: Download the Yi model](#step-2-download-the-yi-model).
|
410 |
+
|
411 |
+
Step 3. To start a web service locally, run the following command.
|
412 |
+
|
413 |
+
```bash
|
414 |
+
python demo/web_demo.py --checkpoint-path <your-model-path>
|
415 |
+
```
|
416 |
+
|
417 |
+
You can access the web UI by entering the address provided in the console into your browser.
|
418 |
+
|
419 |
+
![Quick start - web demo](./assets/img/yi_34b_chat_web_demo.gif)
|
420 |
+
|
421 |
### Finetuning
|
422 |
|
423 |
```bash
|
|
|
713 |
via applications. All usage must adhere to the [Yi Series Models Community License Agreement 2.1](https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMENT.txt).
|
714 |
For free commercial use, you only need to send an email to [get official commercial permission](https://www.lingyiwanwu.com/yi-license).
|
715 |
|
716 |
+
<div align="right"> [ <a href="#building-the-next-generation-of-open-source-and-bilingual-llms">Back to top ⬆️ </a> ] </div>
|