👋 join us on Github
## Introduction XiXiLM（GouMang LLM） has open-sourced a 7 billion parameter base model and a chat model tailored for agricultural scenarios. The model has the following characteristics: 1. **High Professionalism**: XiXiLM focuses on the agricultural field, providing professional and accurate answers especially in areas such as tuber crop cultivation, pest and disease control, and soil management. 2. **Academic Support**: The model is based on the latest agricultural research findings, capable of providing academic-level answers to help researchers and agricultural practitioners gain a deeper understanding of agricultural issues. 3. **Multilingual Support**: Supports both Chinese and English languages, making it convenient for users both domestically and internationally. 4. **Free Commercial Use**: The model weights are fully open, supporting not only academic research but also allowing **free** commercial usage. Users can use the model in commercial projects for free, lowering the usage threshold. 5. **Efficient Training**: Employs advanced training algorithms and techniques, enabling the model to respond quickly to user inquiries and provide efficient Q&A services. 6. **Continuous Optimization**: The model will be continuously optimized based on user feedback and the latest research findings, constantly improving the quality and coverage of its answers. ## XiXiLM-Qwen-14B **Limitations:** Although we have made efforts to ensure the safety of the model during the training process and to encourage the model to generate text that complies with ethical and legal requirements, the model may still produce unexpected outputs due to its size and probabilistic generation paradigm. For example, the generated responses may contain biases, discrimination, or other harmful content. Please do not propagate such content. We are not responsible for any consequences resulting from the dissemination of harmful information. ### Import from Transformers To load the XiXiLM model using Transformers, use the following code: ```python import torch from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("AI4Bread/XiXi_Qwen_base_14b", trust_remote_code=True) # Set `torch_dtype=torch.float16` to load model in float16, otherwise it will be loaded as float32 and cause OOM Error. model = AutoModelForCausalLM.from_pretrained("AI4Bread/XiXi_Qwen_base_14b", torch_dtype=torch.float16, trust_remote_code=True).cuda() model = model.eval() response, history = model.chat(tokenizer, "你好", history=[]) print(response) # Hello! How can I help you today? response, history = model.chat(tokenizer, "马铃薯育种有什么注意事项？需要注意什么呢？", history=history) print(response) ``` The responses can be streamed using `stream_chat`: ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer model_path = "AI4Bread/XiXi_Qwen_base_14b" model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.float16, trust_remote_code=True).cuda() tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True) model = model.eval() length = 0 for response, history in model.stream_chat(tokenizer, "Hello", history=[]): print(response[length:], flush=True, end="") length = len(response) ``` ## Deployment ### LMDeploy LMDeploy is a toolkit for compressing, deploying, and serving LLM, developed by the MMRazor and MMDeploy teams. ```bash pip install lmdeploy ``` Or you can launch an OpenAI compatible server with the following command: ```bash lmdeploy serve api_server internlm/internlm2-chat-7b --model-name internlm2-chat-7b --server-port 23333 ``` Then you can send a chat request to the server: ```bash curl http://localhost:23333/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "internlm2-chat-7b", "messages": [ {"role": "system", "content": "你是一个专业的农业专家"}, {"role": "user", "content": "马铃薯种植的时候有哪些注意事项？"} ] }' ``` The output be like:  Find more details in the [LMDeploy documentation](https://lmdeploy.readthedocs.io/en/latest/) ### vLLM Launch OpenAI compatible server with `vLLM>=0.3.2`: ```bash pip install vllm ``` ```bash python -m vllm.entrypoints.openai.api_server --model internlm/internlm2-chat-7b --served-model-name internlm2-chat-7b --trust-remote-code ``` Then you can send a chat request to the server: ```bash curl http://localhost:8000/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "internlm2-chat-7b", "messages": [ {"role": "system", "content": "You are a professional agriculture expert."}, {"role": "user", "content": "Introduce potato farming to me."} ] }' ``` Find more details in the [vLLM documentation](https://docs.vllm.ai/en/latest/index.html) ## Used local trained model ### First: Convert lmdeploy TurboMind Here, we will use our pre-trained model file and execute the conversion in the user's root directory, as shown below. ```bash # Converting Model to TurboMind (FastTransformer Format) lmdeploy convert internlm2-chat-7b /root/autodl-tmp/agri_intern/XiXiLM --tokenizer-path ./GouMang/tokenizer.json ``` After execution, a workspace folder will be generated in the current directory. This folder contains the necessary files for TurboMind and Triton "Model Inference." as shown below:  ### Second: Chat Locally ```bash lmdeploy chat turbomind ./workspace ``` ### Third(Optional): TurboMind Inference + API Service In the previous section, we tried starting the Client directly using the command line. Now, we will attempt to use lmdeploy for service deployment. The "Model Inference/Service" currently offers two service deployment methods: TurboMind and TritonServer. In this case, the Server is either TurboMind or TritonServer, and the API Server can provide external API services. We recommend using TurboMind. First, start the service with the following command: ```bash # ApiServer+Turbomind api_server => AsyncEngine => TurboMind lmdeploy serve api_server ./workspace \ --server-name 0.0.0.0 \ --server-port 23333 \ --tp 1 ``` In the above parameters, `server_name` and `server_port` indicate the service address and port, respectively. The `tp` parameter, as mentioned earlier, stands for Tensor Parallelism. After this, users can start the Web Service as described in [TurboMind Service as the Backend](#--turbomind-service-as-the-backend). ## Web Service Startup Method 1: ### Starting the Service with Gradio This section demonstrates using Gradio as a front-end demo. > Since Gradio requires local access to display the interface, > you also need to forward the data to your local machine via SSH. The command is as follows: > > ssh -CNg -L 6006:127.0.0.1:6006 root@ssh.intern-ai.org.cn -p