diff --git a/.gitignore b/.gitignore
index 2a64e5feabfb20fa8b11bb2b19469ece489a8b11..f72ae70f70cd10883e64e2a99f9953490205173b 100644
--- a/.gitignore
+++ b/.gitignore
@@ -150,3 +150,4 @@ dmypy.json
/huggingface_tokenizers_cache
/llama-factory/huggingface_tokenizers_cache
**/Icon?
+llama-factory/data/mgtv_train.json
diff --git a/llama-factory/README.md b/llama-factory/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..89c1ccb9dfc443455e61d27663c0f16cafbe1193
--- /dev/null
+++ b/llama-factory/README.md
@@ -0,0 +1,645 @@
+
+
+[](https://github.com/hiyouga/LLaMA-Factory/stargazers)
+[](LICENSE)
+[](https://github.com/hiyouga/LLaMA-Factory/commits/main)
+[](https://pypi.org/project/llamafactory/)
+[](#projects-using-llama-factory)
+[](https://github.com/hiyouga/LLaMA-Factory/pulls)
+[](https://discord.gg/rKfvV9r9FK)
+[](https://twitter.com/llamafactory_ai)
+[](https://colab.research.google.com/drive/1eRTPn37ltBbYsISy9Aw2NuI2Aq5CQrD9?usp=sharing)
+[](https://gallery.pai-ml.com/#/preview/deepLearning/nlp/llama_factory)
+[](https://huggingface.co/spaces/hiyouga/LLaMA-Board)
+[](https://modelscope.cn/studios/hiyouga/LLaMA-Board)
+
+[](https://trendshift.io/repositories/4535)
+
+👋 Join our [WeChat](assets/wechat.jpg) or [NPU user group](assets/wechat_npu.jpg).
+
+\[ English | [中文](README_zh.md) \]
+
+**Fine-tuning a large language model can be easy as...**
+
+https://github.com/hiyouga/LLaMA-Factory/assets/16256802/9840a653-7e9c-41c8-ae89-7ace5698baf6
+
+Choose your path:
+
+- **Colab**: https://colab.research.google.com/drive/1eRTPn37ltBbYsISy9Aw2NuI2Aq5CQrD9?usp=sharing
+- **PAI-DSW**: https://gallery.pai-ml.com/#/preview/deepLearning/nlp/llama_factory
+- **Local machine**: Please refer to [usage](#getting-started)
+
+## Table of Contents
+
+- [Features](#features)
+- [Benchmark](#benchmark)
+- [Changelog](#changelog)
+- [Supported Models](#supported-models)
+- [Supported Training Approaches](#supported-training-approaches)
+- [Provided Datasets](#provided-datasets)
+- [Requirement](#requirement)
+- [Getting Started](#getting-started)
+- [Projects using LLaMA Factory](#projects-using-llama-factory)
+- [License](#license)
+- [Citation](#citation)
+- [Acknowledgement](#acknowledgement)
+
+## Features
+
+- **Various models**: LLaMA, LLaVA, Mistral, Mixtral-MoE, Qwen, Yi, Gemma, Baichuan, ChatGLM, Phi, etc.
+- **Integrated methods**: (Continuous) pre-training, (multimodal) supervised fine-tuning, reward modeling, PPO, DPO, KTO, ORPO, etc.
+- **Scalable resources**: 16-bit full-tuning, freeze-tuning, LoRA and 2/3/4/5/6/8-bit QLoRA via AQLM/AWQ/GPTQ/LLM.int8/HQQ/EETQ.
+- **Advanced algorithms**: GaLore, BAdam, DoRA, LongLoRA, LLaMA Pro, Mixture-of-Depths, LoRA+, LoftQ, PiSSA and Agent tuning.
+- **Practical tricks**: FlashAttention-2, Unsloth, RoPE scaling, NEFTune and rsLoRA.
+- **Experiment monitors**: LlamaBoard, TensorBoard, Wandb, MLflow, etc.
+- **Faster inference**: OpenAI-style API, Gradio UI and CLI with vLLM worker.
+
+## Benchmark
+
+Compared to ChatGLM's [P-Tuning](https://github.com/THUDM/ChatGLM2-6B/tree/main/ptuning), LLaMA Factory's LoRA tuning offers up to **3.7 times faster** training speed with a better Rouge score on the advertising text generation task. By leveraging 4-bit quantization technique, LLaMA Factory's QLoRA further improves the efficiency regarding the GPU memory.
+
+
+
+Definitions
+
+- **Training Speed**: the number of training samples processed per second during the training. (bs=4, cutoff_len=1024)
+- **Rouge Score**: Rouge-2 score on the development set of the [advertising text generation](https://aclanthology.org/D19-1321.pdf) task. (bs=4, cutoff_len=1024)
+- **GPU Memory**: Peak GPU memory usage in 4-bit quantized training. (bs=1, cutoff_len=1024)
+- We adopt `pre_seq_len=128` for ChatGLM's P-Tuning and `lora_rank=32` for LLaMA Factory's LoRA tuning.
+
+Full Changelog
+
+[24/05/20] We supported fine-tuning the **PaliGemma** series models. Note that the PaliGemma models are pre-trained models, you need to fine-tune them with `gemma` template for chat completion.
+
+[24/05/18] We supported **[KTO](https://arxiv.org/abs/2402.01306)** algorithm for preference learning. See [examples](examples/README.md) for usage.
+
+[24/05/14] We supported training and inference on the Ascend NPU devices. Check [installation](#installation) section for details.
+
+[24/04/26] We supported fine-tuning the **LLaVA-1.5** multimodal LLMs. See [examples](examples/README.md) for usage.
+
+[24/04/22] We provided a **[Colab notebook](https://colab.research.google.com/drive/1eRTPn37ltBbYsISy9Aw2NuI2Aq5CQrD9?usp=sharing)** for fine-tuning the Llama-3 model on a free T4 GPU. Two Llama-3-derived models fine-tuned using LLaMA Factory are available at Hugging Face, check [Llama3-8B-Chinese-Chat](https://huggingface.co/shenzhi-wang/Llama3-8B-Chinese-Chat) and [Llama3-Chinese](https://huggingface.co/zhichen/Llama3-Chinese) for details.
+
+[24/04/21] We supported **[Mixture-of-Depths](https://arxiv.org/abs/2404.02258)** according to [AstraMindAI's implementation](https://github.com/astramind-ai/Mixture-of-depths). See [examples](examples/README.md) for usage.
+
+[24/04/16] We supported **[BAdam](https://arxiv.org/abs/2404.02827)**. See [examples](examples/README.md) for usage.
+
+[24/04/16] We supported **[unsloth](https://github.com/unslothai/unsloth)**'s long-sequence training (Llama-2-7B-56k within 24GB). It achieves **117%** speed and **50%** memory compared with FlashAttention-2, more benchmarks can be found in [this page](https://github.com/hiyouga/LLaMA-Factory/wiki/Performance-comparison).
+
+[24/03/31] We supported **[ORPO](https://arxiv.org/abs/2403.07691)**. See [examples](examples/README.md) for usage.
+
+[24/03/21] Our paper "[LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://arxiv.org/abs/2403.13372)" is available at arXiv!
+
+[24/03/20] We supported **FSDP+QLoRA** that fine-tunes a 70B model on 2x24GB GPUs. See [examples](examples/README.md) for usage.
+
+[24/03/13] We supported **[LoRA+](https://arxiv.org/abs/2402.12354)**. See [examples](examples/README.md) for usage.
+
+[24/03/07] We supported gradient low-rank projection (**[GaLore](https://arxiv.org/abs/2403.03507)**) algorithm. See [examples](examples/README.md) for usage.
+
+[24/03/07] We integrated **[vLLM](https://github.com/vllm-project/vllm)** for faster and concurrent inference. Try `infer_backend: vllm` to enjoy **270%** inference speed.
+
+[24/02/28] We supported weight-decomposed LoRA (**[DoRA](https://arxiv.org/abs/2402.09353)**). Try `use_dora: true` to activate DoRA training.
+
+[24/02/15] We supported **block expansion** proposed by [LLaMA Pro](https://github.com/TencentARC/LLaMA-Pro). See [examples](examples/README.md) for usage.
+
+[24/02/05] Qwen1.5 (Qwen2 beta version) series models are supported in LLaMA-Factory. Check this [blog post](https://qwenlm.github.io/blog/qwen1.5/) for details.
+
+[24/01/18] We supported **agent tuning** for most models, equipping model with tool using abilities by fine-tuning with `dataset: glaive_toolcall_en`.
+
+[23/12/23] We supported **[unsloth](https://github.com/unslothai/unsloth)**'s implementation to boost LoRA tuning for the LLaMA, Mistral and Yi models. Try `use_unsloth: true` argument to activate unsloth patch. It achieves **170%** speed in our benchmark, check [this page](https://github.com/hiyouga/LLaMA-Factory/wiki/Performance-comparison) for details.
+
+[23/12/12] We supported fine-tuning the latest MoE model **[Mixtral 8x7B](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1)** in our framework. See hardware requirement [here](#hardware-requirement).
+
+[23/12/01] We supported downloading pre-trained models and datasets from the **[ModelScope Hub](https://modelscope.cn/models)** for Chinese mainland users. See [this tutorial](#download-from-modelscope-hub) for usage.
+
+[23/10/21] We supported **[NEFTune](https://arxiv.org/abs/2310.05914)** trick for fine-tuning. Try `neftune_noise_alpha: 5` argument to activate NEFTune.
+
+[23/09/27] We supported **$S^2$-Attn** proposed by [LongLoRA](https://github.com/dvlab-research/LongLoRA) for the LLaMA models. Try `shift_attn: true` argument to enable shift short attention.
+
+[23/09/23] We integrated MMLU, C-Eval and CMMLU benchmarks in this repo. See [examples](examples/README.md) for usage.
+
+[23/09/10] We supported **[FlashAttention-2](https://github.com/Dao-AILab/flash-attention)**. Try `flash_attn: fa2` argument to enable FlashAttention-2 if you are using RTX4090, A100 or H100 GPUs.
+
+[23/08/12] We supported **RoPE scaling** to extend the context length of the LLaMA models. Try `rope_scaling: linear` argument in training and `rope_scaling: dynamic` argument at inference to extrapolate the position embeddings.
+
+[23/08/11] We supported **[DPO training](https://arxiv.org/abs/2305.18290)** for instruction-tuned models. See [examples](examples/README.md) for usage.
+
+[23/07/31] We supported **dataset streaming**. Try `streaming: true` and `max_steps: 10000` arguments to load your dataset in streaming mode.
+
+[23/07/29] We released two instruction-tuned 13B models at Hugging Face. See these Hugging Face Repos ([LLaMA-2](https://huggingface.co/hiyouga/Llama-2-Chinese-13b-chat) / [Baichuan](https://huggingface.co/hiyouga/Baichuan-13B-sft)) for details.
+
+[23/07/18] We developed an **all-in-one Web UI** for training, evaluation and inference. Try `train_web.py` to fine-tune models in your Web browser. Thank [@KanadeSiina](https://github.com/KanadeSiina) and [@codemayq](https://github.com/codemayq) for their efforts in the development.
+
+[23/07/09] We released **[FastEdit](https://github.com/hiyouga/FastEdit)** ⚡🩹, an easy-to-use package for editing the factual knowledge of large language models efficiently. Please follow [FastEdit](https://github.com/hiyouga/FastEdit) if you are interested.
+
+[23/06/29] We provided a **reproducible example** of training a chat model using instruction-following datasets, see [Baichuan-7B-sft](https://huggingface.co/hiyouga/Baichuan-7B-sft) for details.
+
+[23/06/22] We aligned the [demo API](src/api_demo.py) with the [OpenAI's](https://platform.openai.com/docs/api-reference/chat) format where you can insert the fine-tuned model in **arbitrary ChatGPT-based applications**.
+
+[23/06/03] We supported quantized training and inference (aka **[QLoRA](https://github.com/artidoro/qlora)**). See [examples](examples/README.md) for usage.
+
+Pre-training datasets
+
+- [Wiki Demo (en)](data/wiki_demo.txt)
+- [RefinedWeb (en)](https://huggingface.co/datasets/tiiuae/falcon-refinedweb)
+- [RedPajama V2 (en)](https://huggingface.co/datasets/togethercomputer/RedPajama-Data-V2)
+- [Wikipedia (en)](https://huggingface.co/datasets/olm/olm-wikipedia-20221220)
+- [Wikipedia (zh)](https://huggingface.co/datasets/pleisto/wikipedia-cn-20230720-filtered)
+- [Pile (en)](https://huggingface.co/datasets/EleutherAI/pile)
+- [SkyPile (zh)](https://huggingface.co/datasets/Skywork/SkyPile-150B)
+- [FineWeb (en)](https://huggingface.co/datasets/HuggingFaceFW/fineweb)
+- [FineWeb-Edu (en)](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu)
+- [The Stack (en)](https://huggingface.co/datasets/bigcode/the-stack)
+- [StarCoder (en)](https://huggingface.co/datasets/bigcode/starcoderdata)
+
+Supervised fine-tuning datasets
+
+- [Identity (en&zh)](data/identity.json)
+- [Stanford Alpaca (en)](https://github.com/tatsu-lab/stanford_alpaca)
+- [Stanford Alpaca (zh)](https://github.com/ymcui/Chinese-LLaMA-Alpaca-3)
+- [Alpaca GPT4 (en&zh)](https://github.com/Instruction-Tuning-with-GPT-4/GPT-4-LLM)
+- [Glaive Function Calling V2 (en&zh)](https://huggingface.co/datasets/glaiveai/glaive-function-calling-v2)
+- [LIMA (en)](https://huggingface.co/datasets/GAIR/lima)
+- [Guanaco Dataset (multilingual)](https://huggingface.co/datasets/JosephusCheung/GuanacoDataset)
+- [BELLE 2M (zh)](https://huggingface.co/datasets/BelleGroup/train_2M_CN)
+- [BELLE 1M (zh)](https://huggingface.co/datasets/BelleGroup/train_1M_CN)
+- [BELLE 0.5M (zh)](https://huggingface.co/datasets/BelleGroup/train_0.5M_CN)
+- [BELLE Dialogue 0.4M (zh)](https://huggingface.co/datasets/BelleGroup/generated_chat_0.4M)
+- [BELLE School Math 0.25M (zh)](https://huggingface.co/datasets/BelleGroup/school_math_0.25M)
+- [BELLE Multiturn Chat 0.8M (zh)](https://huggingface.co/datasets/BelleGroup/multiturn_chat_0.8M)
+- [UltraChat (en)](https://github.com/thunlp/UltraChat)
+- [OpenPlatypus (en)](https://huggingface.co/datasets/garage-bAInd/Open-Platypus)
+- [CodeAlpaca 20k (en)](https://huggingface.co/datasets/sahil2801/CodeAlpaca-20k)
+- [Alpaca CoT (multilingual)](https://huggingface.co/datasets/QingyiSi/Alpaca-CoT)
+- [OpenOrca (en)](https://huggingface.co/datasets/Open-Orca/OpenOrca)
+- [SlimOrca (en)](https://huggingface.co/datasets/Open-Orca/SlimOrca)
+- [MathInstruct (en)](https://huggingface.co/datasets/TIGER-Lab/MathInstruct)
+- [Firefly 1.1M (zh)](https://huggingface.co/datasets/YeungNLP/firefly-train-1.1M)
+- [Wiki QA (en)](https://huggingface.co/datasets/wiki_qa)
+- [Web QA (zh)](https://huggingface.co/datasets/suolyer/webqa)
+- [WebNovel (zh)](https://huggingface.co/datasets/zxbsmk/webnovel_cn)
+- [Nectar (en)](https://huggingface.co/datasets/berkeley-nest/Nectar)
+- [deepctrl (en&zh)](https://www.modelscope.cn/datasets/deepctrl/deepctrl-sft-data)
+- [Advertise Generating (zh)](https://huggingface.co/datasets/HasturOfficial/adgen)
+- [ShareGPT Hyperfiltered (en)](https://huggingface.co/datasets/totally-not-an-llm/sharegpt-hyperfiltered-3k)
+- [ShareGPT4 (en&zh)](https://huggingface.co/datasets/shibing624/sharegpt_gpt4)
+- [UltraChat 200k (en)](https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k)
+- [AgentInstruct (en)](https://huggingface.co/datasets/THUDM/AgentInstruct)
+- [LMSYS Chat 1M (en)](https://huggingface.co/datasets/lmsys/lmsys-chat-1m)
+- [Evol Instruct V2 (en)](https://huggingface.co/datasets/WizardLM/WizardLM_evol_instruct_V2_196k)
+- [Cosmopedia (en)](https://huggingface.co/datasets/HuggingFaceTB/cosmopedia)
+- [STEM (zh)](https://huggingface.co/datasets/hfl/stem_zh_instruction)
+- [Ruozhiba (zh)](https://huggingface.co/datasets/hfl/ruozhiba_gpt4_turbo)
+- [Neo-sft (zh)](https://huggingface.co/datasets/m-a-p/neo_sft_phase2)
+- [WebInstructSub (en)](https://huggingface.co/datasets/TIGER-Lab/WebInstructSub)
+- [Magpie-Pro-300K-Filtered (en)](https://huggingface.co/datasets/Magpie-Align/Magpie-Pro-300K-Filtered)
+- [LLaVA mixed (en&zh)](https://huggingface.co/datasets/BUAADreamer/llava-en-zh-300k)
+- [Open Assistant (de)](https://huggingface.co/datasets/mayflowergmbh/oasst_de)
+- [Dolly 15k (de)](https://huggingface.co/datasets/mayflowergmbh/dolly-15k_de)
+- [Alpaca GPT4 (de)](https://huggingface.co/datasets/mayflowergmbh/alpaca-gpt4_de)
+- [OpenSchnabeltier (de)](https://huggingface.co/datasets/mayflowergmbh/openschnabeltier_de)
+- [Evol Instruct (de)](https://huggingface.co/datasets/mayflowergmbh/evol-instruct_de)
+- [Dolphin (de)](https://huggingface.co/datasets/mayflowergmbh/dolphin_de)
+- [Booksum (de)](https://huggingface.co/datasets/mayflowergmbh/booksum_de)
+- [Airoboros (de)](https://huggingface.co/datasets/mayflowergmbh/airoboros-3.0_de)
+- [Ultrachat (de)](https://huggingface.co/datasets/mayflowergmbh/ultra-chat_de)
+
+Preference datasets
+
+- [DPO mixed (en&zh)](https://huggingface.co/datasets/hiyouga/DPO-En-Zh-20k)
+- [UltraFeedback (en)](https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized)
+- [Orca DPO Pairs (en)](https://huggingface.co/datasets/Intel/orca_dpo_pairs)
+- [HH-RLHF (en)](https://huggingface.co/datasets/Anthropic/hh-rlhf)
+- [Nectar (en)](https://huggingface.co/datasets/berkeley-nest/Nectar)
+- [Orca DPO (de)](https://huggingface.co/datasets/mayflowergmbh/intel_orca_dpo_pairs_de)
+- [KTO mixed (en)](https://huggingface.co/datasets/argilla/kto-mix-15k)
+
+For Windows users
+
+If you want to enable the quantized LoRA (QLoRA) on the Windows platform, you need to install a pre-built version of `bitsandbytes` library, which supports CUDA 11.1 to 12.2, please select the appropriate [release version](https://github.com/jllllll/bitsandbytes-windows-webui/releases/tag/wheels) based on your CUDA version.
+
+```bash
+pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.2.post2-py3-none-win_amd64.whl
+```
+
+To enable FlashAttention-2 on the Windows platform, you need to install the precompiled `flash-attn` library, which supports CUDA 12.1 to 12.2. Please download the corresponding version from [flash-attention](https://github.com/bdashore3/flash-attention/releases) based on your requirements.
+
+For Ascend NPU users
+
+To install LLaMA Factory on Ascend NPU devices, please specify extra dependencies: `pip install -e ".[torch-npu,metrics]"`. Additionally, you need to install the **[Ascend CANN Toolkit and Kernels](https://www.hiascend.com/developer/download/community/result?module=cann)**. Please follow the [installation tutorial](https://www.hiascend.com/document/detail/en/CANNCommunityEdition/600alphaX/softwareinstall/instg/atlasdeploy_03_0031.html) or use the following commands:
+
+```bash
+# replace the url according to your CANN version and devices
+# install CANN Toolkit
+wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/Milan-ASL/Milan-ASL%20V100R001C17SPC701/Ascend-cann-toolkit_8.0.RC1.alpha001_linux-"$(uname -i)".run
+bash Ascend-cann-toolkit_8.0.RC1.alpha001_linux-"$(uname -i)".run --install
+
+# install CANN Kernels
+wget https://ascend-repo.obs.cn-east-2.myhuaweicloud.com/Milan-ASL/Milan-ASL%20V100R001C17SPC701/Ascend-cann-kernels-910b_8.0.RC1.alpha001_linux.run
+bash Ascend-cann-kernels-910b_8.0.RC1.alpha001_linux.run --install
+
+# set env variables
+source /usr/local/Ascend/ascend-toolkit/set_env.sh
+```
+
+| Requirement | Minimum | Recommend |
+| ------------ | ------- | ----------- |
+| CANN | 8.0.RC1 | 8.0.RC1 |
+| torch | 2.1.0 | 2.1.0 |
+| torch-npu | 2.1.0 | 2.1.0.post3 |
+| deepspeed | 0.13.2 | 0.13.2 |
+
+Remember to use `ASCEND_RT_VISIBLE_DEVICES` instead of `CUDA_VISIBLE_DEVICES` to specify the device to use.
+
+If you cannot infer model on NPU devices, try setting `do_sample: false` in the configurations.
+
+Download the pre-built Docker images: [32GB](http://mirrors.cn-central-221.ovaijisuan.com/detail/130.html) | [64GB](http://mirrors.cn-central-221.ovaijisuan.com/detail/131.html)
+
+Build without Docker Compose
+
+For CUDA users:
+
+```bash
+docker build -f ./docker/docker-cuda/Dockerfile \
+ --build-arg INSTALL_BNB=false \
+ --build-arg INSTALL_VLLM=false \
+ --build-arg INSTALL_DEEPSPEED=false \
+ --build-arg INSTALL_FLASHATTN=false \
+ --build-arg PIP_INDEX=https://pypi.org/simple \
+ -t llamafactory:latest .
+
+docker run -dit --gpus=all \
+ -v ./hf_cache:/root/.cache/huggingface \
+ -v ./ms_cache:/root/.cache/modelscope \
+ -v ./data:/app/data \
+ -v ./output:/app/output \
+ -p 7860:7860 \
+ -p 8000:8000 \
+ --shm-size 16G \
+ --name llamafactory \
+ llamafactory:latest
+
+docker exec -it llamafactory bash
+```
+
+For Ascend NPU users:
+
+```bash
+# Choose docker image upon your environment
+docker build -f ./docker/docker-npu/Dockerfile \
+ --build-arg INSTALL_DEEPSPEED=false \
+ --build-arg PIP_INDEX=https://pypi.org/simple \
+ -t llamafactory:latest .
+
+# Change `device` upon your resources
+docker run -dit \
+ -v ./hf_cache:/root/.cache/huggingface \
+ -v ./ms_cache:/root/.cache/modelscope \
+ -v ./data:/app/data \
+ -v ./output:/app/output \
+ -v /usr/local/dcmi:/usr/local/dcmi \
+ -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
+ -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
+ -v /etc/ascend_install.info:/etc/ascend_install.info \
+ -p 7860:7860 \
+ -p 8000:8000 \
+ --device /dev/davinci0 \
+ --device /dev/davinci_manager \
+ --device /dev/devmm_svm \
+ --device /dev/hisi_hdc \
+ --shm-size 16G \
+ --name llamafactory \
+ llamafactory:latest
+
+docker exec -it llamafactory bash
+```
+
+Details about volume
+
+- hf_cache: Utilize Hugging Face cache on the host machine. Reassignable if a cache already exists in a different directory.
+- data: Place datasets on this dir of the host machine so that they can be selected on LLaMA Board GUI.
+- output: Set export dir to this location so that the merged result can be accessed directly on the host machine.
+
+Click to show
+
+1. Wang et al. ESRL: Efficient Sampling-based Reinforcement Learning for Sequence Generation. 2023. [[arxiv]](https://arxiv.org/abs/2308.02223)
+1. Yu et al. Open, Closed, or Small Language Models for Text Classification? 2023. [[arxiv]](https://arxiv.org/abs/2308.10092)
+1. Wang et al. UbiPhysio: Support Daily Functioning, Fitness, and Rehabilitation with Action Understanding and Feedback in Natural Language. 2023. [[arxiv]](https://arxiv.org/abs/2308.10526)
+1. Luceri et al. Leveraging Large Language Models to Detect Influence Campaigns in Social Media. 2023. [[arxiv]](https://arxiv.org/abs/2311.07816)
+1. Zhang et al. Alleviating Hallucinations of Large Language Models through Induced Hallucinations. 2023. [[arxiv]](https://arxiv.org/abs/2312.15710)
+1. Wang et al. Know Your Needs Better: Towards Structured Understanding of Marketer Demands with Analogical Reasoning Augmented LLMs. KDD 2024. [[arxiv]](https://arxiv.org/abs/2401.04319)
+1. Wang et al. CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning. ACL 2024. [[arxiv]](https://arxiv.org/abs/2401.07286)
+1. Choi et al. FACT-GPT: Fact-Checking Augmentation via Claim Matching with LLMs. 2024. [[arxiv]](https://arxiv.org/abs/2402.05904)
+1. Zhang et al. AutoMathText: Autonomous Data Selection with Language Models for Mathematical Texts. 2024. [[arxiv]](https://arxiv.org/abs/2402.07625)
+1. Lyu et al. KnowTuning: Knowledge-aware Fine-tuning for Large Language Models. 2024. [[arxiv]](https://arxiv.org/abs/2402.11176)
+1. Yang et al. LaCo: Large Language Model Pruning via Layer Collaps. 2024. [[arxiv]](https://arxiv.org/abs/2402.11187)
+1. Bhardwaj et al. Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic. 2024. [[arxiv]](https://arxiv.org/abs/2402.11746)
+1. Yang et al. Enhancing Empathetic Response Generation by Augmenting LLMs with Small-scale Empathetic Models. 2024. [[arxiv]](https://arxiv.org/abs/2402.11801)
+1. Yi et al. Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding. ACL 2024 Findings. [[arxiv]](https://arxiv.org/abs/2402.11809)
+1. Cao et al. Head-wise Shareable Attention for Large Language Models. 2024. [[arxiv]](https://arxiv.org/abs/2402.11819)
+1. Zhang et al. Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languages. 2024. [[arxiv]](https://arxiv.org/abs/2402.12204)
+1. Kim et al. Efficient and Effective Vocabulary Expansion Towards Multilingual Large Language Models. 2024. [[arxiv]](https://arxiv.org/abs/2402.14714)
+1. Yu et al. KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models. ACL 2024. [[arxiv]](https://arxiv.org/abs/2402.15043)
+1. Huang et al. Key-Point-Driven Data Synthesis with its Enhancement on Mathematical Reasoning. 2024. [[arxiv]](https://arxiv.org/abs/2403.02333)
+1. Duan et al. Negating Negatives: Alignment without Human Positive Samples via Distributional Dispreference Optimization. 2024. [[arxiv]](https://arxiv.org/abs/2403.03419)
+1. Xie and Schwertfeger. Empowering Robotics with Large Language Models: osmAG Map Comprehension with LLMs. 2024. [[arxiv]](https://arxiv.org/abs/2403.08228)
+1. Wu et al. Large Language Models are Parallel Multilingual Learners. 2024. [[arxiv]](https://arxiv.org/abs/2403.09073)
+1. Zhang et al. EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling. 2024. [[arxiv]](https://arxiv.org/abs/2403.14541)
+1. Weller et al. FollowIR: Evaluating and Teaching Information Retrieval Models to Follow Instructions. 2024. [[arxiv]](https://arxiv.org/abs/2403.15246)
+1. Hongbin Na. CBT-LLM: A Chinese Large Language Model for Cognitive Behavioral Therapy-based Mental Health Question Answering. COLING 2024. [[arxiv]](https://arxiv.org/abs/2403.16008)
+1. Zan et al. CodeS: Natural Language to Code Repository via Multi-Layer Sketch. 2024. [[arxiv]](https://arxiv.org/abs/2403.16443)
+1. Liu et al. Extensive Self-Contrast Enables Feedback-Free Language Model Alignment. 2024. [[arxiv]](https://arxiv.org/abs/2404.00604)
+1. Luo et al. BAdam: A Memory Efficient Full Parameter Training Method for Large Language Models. 2024. [[arxiv]](https://arxiv.org/abs/2404.02827)
+1. Du et al. Chinese Tiny LLM: Pretraining a Chinese-Centric Large Language Model. 2024. [[arxiv]](https://arxiv.org/abs/2404.04167)
+1. Ma et al. Parameter Efficient Quasi-Orthogonal Fine-Tuning via Givens Rotation. ICML 2024. [[arxiv]](https://arxiv.org/abs/2404.04316)
+1. Liu et al. Dynamic Generation of Personalities with Large Language Models. 2024. [[arxiv]](https://arxiv.org/abs/2404.07084)
+1. Shang et al. How Far Have We Gone in Stripped Binary Code Understanding Using Large Language Models. 2024. [[arxiv]](https://arxiv.org/abs/2404.09836)
+1. Huang et al. LLMTune: Accelerate Database Knob Tuning with Large Language Models. 2024. [[arxiv]](https://arxiv.org/abs/2404.11581)
+1. Deng et al. Text-Tuple-Table: Towards Information Integration in Text-to-Table Generation via Global Tuple Extraction. 2024. [[arxiv]](https://arxiv.org/abs/2404.14215)
+1. Acikgoz et al. Hippocrates: An Open-Source Framework for Advancing Large Language Models in Healthcare. 2024. [[arxiv]](https://arxiv.org/abs/2404.16621)
+1. Zhang et al. Small Language Models Need Strong Verifiers to Self-Correct Reasoning. ACL 2024 Findings. [[arxiv]](https://arxiv.org/abs/2404.17140)
+1. Zhou et al. FREB-TQA: A Fine-Grained Robustness Evaluation Benchmark for Table Question Answering. NAACL 2024. [[arxiv]](https://arxiv.org/abs/2404.18585)
+1. Xu et al. Large Language Models for Cyber Security: A Systematic Literature Review. 2024. [[arxiv]](https://arxiv.org/abs/2405.04760)
+1. Dammu et al. "They are uncultured": Unveiling Covert Harms and Social Threats in LLM Generated Conversations. 2024. [[arxiv]](https://arxiv.org/abs/2405.05378)
+1. Yi et al. A safety realignment framework via subspace-oriented model fusion for large language models. 2024. [[arxiv]](https://arxiv.org/abs/2405.09055)
+1. Lou et al. SPO: Multi-Dimensional Preference Sequential Alignment With Implicit Reward Modeling. 2024. [[arxiv]](https://arxiv.org/abs/2405.12739)
+1. Zhang et al. Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners. 2024. [[arxiv]](https://arxiv.org/abs/2405.13816)
+1. Zhang et al. TS-Align: A Teacher-Student Collaborative Framework for Scalable Iterative Finetuning of Large Language Models. 2024. [[arxiv]](https://arxiv.org/abs/2405.20215)
+1. Zihong Chen. Sentence Segmentation and Sentence Punctuation Based on XunziALLM. 2024. [[paper]](https://aclanthology.org/2024.lt4hala-1.30)
+1. Gao et al. The Best of Both Worlds: Toward an Honest and Helpful Large Language Model. 2024. [[arxiv]](https://arxiv.org/abs/2406.00380)
+1. Wang and Song. MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset. 2024. [[arxiv]](https://arxiv.org/abs/2406.02106)
+1. Hu et al. Computational Limits of Low-Rank Adaptation (LoRA) for Transformer-Based Models. 2024. [[arxiv]](https://arxiv.org/abs/2406.03136)
+1. Ge et al. Time Sensitive Knowledge Editing through Efficient Finetuning. ACL 2024. [[arxiv]](https://arxiv.org/abs/2406.04496)
+1. Tan et al. Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions. 2024. [[arxiv]](https://arxiv.org/abs/2406.05688)
+1. Song et al. Turbo Sparse: Achieving LLM SOTA Performance with Minimal Activated Parameters. 2024. [[arxiv]](https://arxiv.org/abs/2406.05955)
+1. Gu et al. RWKV-CLIP: A Robust Vision-Language Representation Learner. 2024. [[arxiv]](https://arxiv.org/abs/2406.06973)
+1. Chen et al. Advancing Tool-Augmented Large Language Models: Integrating Insights from Errors in Inference Trees. 2024. [[arxiv]](https://arxiv.org/abs/2406.07115)
+1. Zhu et al. Are Large Language Models Good Statisticians?. 2024. [[arxiv]](https://arxiv.org/abs/2406.07815)
+1. Li et al. Know the Unknown: An Uncertainty-Sensitive Method for LLM Instruction Tuning. 2024. [[arxiv]](https://arxiv.org/abs/2406.10099)
+1. Ding et al. IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commerce. 2024. [[arxiv]](https://arxiv.org/abs/2406.10173)
+1. He et al. COMMUNITY-CROSS-INSTRUCT: Unsupervised Instruction Generation for Aligning Large Language Models to Online Communities. 2024. [[arxiv]](https://arxiv.org/abs/2406.12074)
+1. Lin et al. FVEL: Interactive Formal Verification Environment with Large Language Models via Theorem Proving. 2024. [[arxiv]](https://arxiv.org/abs/2406.14408)
+1. Treutlein et al. Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data. 2024. [[arxiv]](https://arxiv.org/abs/2406.14546)
+1. Feng et al. SS-Bench: A Benchmark for Social Story Generation and Evaluation. 2024. [[arxiv]](https://arxiv.org/abs/2406.15695)
+1. Feng et al. Self-Constructed Context Decompilation with Fined-grained Alignment Enhancement. 2024. [[arxiv]](https://arxiv.org/abs/2406.17233)
+1. Liu et al. Large Language Models for Cuffless Blood Pressure Measurement From Wearable Biosignals. 2024. [[arxiv]](https://arxiv.org/abs/2406.18069)
+1. Iyer et al. Exploring Very Low-Resource Translation with LLMs: The University of Edinburgh’s Submission to AmericasNLP 2024 Translation Task. AmericasNLP 2024. [[paper]](https://aclanthology.org/2024.americasnlp-1.25)
+1. **[StarWhisper](https://github.com/Yu-Yang-Li/StarWhisper)**: A large language model for Astronomy, based on ChatGLM2-6B and Qwen-14B.
+1. **[DISC-LawLLM](https://github.com/FudanDISC/DISC-LawLLM)**: A large language model specialized in Chinese legal domain, based on Baichuan-13B, is capable of retrieving and reasoning on legal knowledge.
+1. **[Sunsimiao](https://github.com/X-D-Lab/Sunsimiao)**: A large language model specialized in Chinese medical domain, based on Baichuan-7B and ChatGLM-6B.
+1. **[CareGPT](https://github.com/WangRongsheng/CareGPT)**: A series of large language models for Chinese medical domain, based on LLaMA2-7B and Baichuan-13B.
+1. **[MachineMindset](https://github.com/PKU-YuanGroup/Machine-Mindset/)**: A series of MBTI Personality large language models, capable of giving any LLM 16 different personality types based on different datasets and training methods.
+1. **[Luminia-13B-v3](https://huggingface.co/Nekochu/Luminia-13B-v3)**: A large language model specialized in generate metadata for stable diffusion. [[🤗Demo]](https://huggingface.co/spaces/Nekochu/Luminia-13B_SD_Prompt)
+1. **[Chinese-LLaVA-Med](https://github.com/BUAADreamer/Chinese-LLaVA-Med)**: A multimodal large language model specialized in Chinese medical domain, based on LLaVA-1.5-7B.
+1. **[AutoRE](https://github.com/THUDM/AutoRE)**: A document-level relation extraction system based on large language models.
+1. **[NVIDIA RTX AI Toolkit](https://github.com/NVIDIA/RTX-AI-Toolkit)**: SDKs for fine-tuning LLMs on Windows PC for NVIDIA RTX.
+1. **[LazyLLM](https://github.com/LazyAGI/LazyLLM)**: An easy and lazy way for building multi-agent LLMs applications and supports model fine-tuning via LLaMA Factory.
+
+LLaMA Board: A One-stop Web UI for Getting Started with LLaMA Factory
")
+ gr.HTML(
+ '"
+ )
+ gr.DuplicateButton(value="Duplicate Space for private use", elem_classes="duplicate-button")
+
+ engine.manager.add_elems("top", create_top())
+ lang: "gr.Dropdown" = engine.manager.get_elem_by_id("top.lang")
+
+ with gr.Tab("Train"):
+ engine.manager.add_elems("train", create_train_tab(engine))
+
+ with gr.Tab("Evaluate & Predict"):
+ engine.manager.add_elems("eval", create_eval_tab(engine))
+
+ with gr.Tab("Chat"):
+ engine.manager.add_elems("infer", create_infer_tab(engine))
+
+ if not demo_mode:
+ with gr.Tab("Export"):
+ engine.manager.add_elems("export", create_export_tab(engine))
+
+ demo.load(engine.resume, outputs=engine.manager.get_elem_list(), concurrency_limit=None)
+ lang.change(engine.change_lang, [lang], engine.manager.get_elem_list(), queue=False)
+ lang.input(save_config, inputs=[lang], queue=False)
+
+ return demo
+
+
+def create_web_demo() -> "gr.Blocks":
+ engine = Engine(pure_chat=True)
+
+ with gr.Blocks(title="Web Demo", css=CSS) as demo:
+ lang = gr.Dropdown(choices=["en", "zh"])
+ engine.manager.add_elems("top", dict(lang=lang))
+
+ _, _, chat_elems = create_chat_box(engine, visible=True)
+ engine.manager.add_elems("infer", chat_elems)
+
+ demo.load(engine.resume, outputs=engine.manager.get_elem_list(), concurrency_limit=None)
+ lang.change(engine.change_lang, [lang], engine.manager.get_elem_list(), queue=False)
+ lang.input(save_config, inputs=[lang], queue=False)
+
+ return demo
+
+
+def run_web_ui() -> None:
+ gradio_share = os.environ.get("GRADIO_SHARE", "0").lower() in ["true", "1"]
+ server_name = os.environ.get("GRADIO_SERVER_NAME", "0.0.0.0")
+ create_ui().queue().launch(share=gradio_share, server_name=server_name, inbrowser=True)
+
+
+def run_web_demo() -> None:
+ gradio_share = os.environ.get("GRADIO_SHARE", "0").lower() in ["true", "1"]
+ server_name = os.environ.get("GRADIO_SERVER_NAME", "0.0.0.0")
+ create_web_demo().queue().launch(share=gradio_share, server_name=server_name, inbrowser=True)
diff --git a/llama-factory/src/llamafactory/webui/locales.py b/llama-factory/src/llamafactory/webui/locales.py
new file mode 100644
index 0000000000000000000000000000000000000000..b1f2a80294aa59222ee77cf481712ae737a2c1ae
--- /dev/null
+++ b/llama-factory/src/llamafactory/webui/locales.py
@@ -0,0 +1,1669 @@
+# Copyright 2024 the LlamaFactory team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+LOCALES = {
+ "lang": {
+ "en": {
+ "label": "Lang",
+ },
+ "ru": {
+ "label": "Русский",
+ },
+ "zh": {
+ "label": "语言",
+ },
+ },
+ "model_name": {
+ "en": {
+ "label": "Model name",
+ },
+ "ru": {
+ "label": "Название модели",
+ },
+ "zh": {
+ "label": "模型名称",
+ },
+ },
+ "model_path": {
+ "en": {
+ "label": "Model path",
+ "info": "Path to pretrained model or model identifier from Hugging Face.",
+ },
+ "ru": {
+ "label": "Путь к модели",
+ "info": "Путь к предварительно обученной модели или идентификатор модели от Hugging Face.",
+ },
+ "zh": {
+ "label": "模型路径",
+ "info": "本地模型的文件路径或 Hugging Face 的模型标识符。",
+ },
+ },
+ "finetuning_type": {
+ "en": {
+ "label": "Finetuning method",
+ },
+ "ru": {
+ "label": "Метод дообучения",
+ },
+ "zh": {
+ "label": "微调方法",
+ },
+ },
+ "checkpoint_path": {
+ "en": {
+ "label": "Checkpoint path",
+ },
+ "ru": {
+ "label": "Путь контрольной точки",
+ },
+ "zh": {
+ "label": "检查点路径",
+ },
+ },
+ "advanced_tab": {
+ "en": {
+ "label": "Advanced configurations",
+ },
+ "ru": {
+ "label": "Расширенные конфигурации",
+ },
+ "zh": {
+ "label": "高级设置",
+ },
+ },
+ "quantization_bit": {
+ "en": {
+ "label": "Quantization bit",
+ "info": "Enable quantization (QLoRA).",
+ },
+ "ru": {
+ "label": "Уровень квантования",
+ "info": "Включить квантование (QLoRA).",
+ },
+ "zh": {
+ "label": "量化等级",
+ "info": "启用量化（QLoRA）。",
+ },
+ },
+ "quantization_method": {
+ "en": {
+ "label": "Quantization method",
+ "info": "Quantization algorithm to use.",
+ },
+ "ru": {
+ "label": "Метод квантования",
+ "info": "Алгоритм квантования, который следует использовать.",
+ },
+ "zh": {
+ "label": "量化方法",
+ "info": "使用的量化算法。",
+ },
+ },
+ "template": {
+ "en": {
+ "label": "Prompt template",
+ "info": "The template used in constructing prompts.",
+ },
+ "ru": {
+ "label": "Шаблон запроса",
+ "info": "Шаблон, используемый при формировании запросов.",
+ },
+ "zh": {
+ "label": "提示模板",
+ "info": "构建提示词时使用的模板",
+ },
+ },
+ "rope_scaling": {
+ "en": {
+ "label": "RoPE scaling",
+ },
+ "ru": {
+ "label": "Масштабирование RoPE",
+ },
+ "zh": {
+ "label": "RoPE 插值方法",
+ },
+ },
+ "booster": {
+ "en": {
+ "label": "Booster",
+ },
+ "ru": {
+ "label": "Ускоритель",
+ },
+ "zh": {
+ "label": "加速方式",
+ },
+ },
+ "visual_inputs": {
+ "en": {
+ "label": "Visual inputs",
+ },
+ "ru": {
+ "label": "визуальные входы",
+ },
+ "zh": {
+ "label": "图像输入",
+ },
+ },
+ "training_stage": {
+ "en": {
+ "label": "Stage",
+ "info": "The stage to perform in training.",
+ },
+ "ru": {
+ "label": "Этап",
+ "info": "Этап выполнения обучения.",
+ },
+ "zh": {
+ "label": "训练阶段",
+ "info": "目前采用的训练方式。",
+ },
+ },
+ "dataset_dir": {
+ "en": {
+ "label": "Data dir",
+ "info": "Path to the data directory.",
+ },
+ "ru": {
+ "label": "Директория данных",
+ "info": "Путь к директории данных.",
+ },
+ "zh": {
+ "label": "数据路径",
+ "info": "数据文件夹的路径。",
+ },
+ },
+ "dataset": {
+ "en": {
+ "label": "Dataset",
+ },
+ "ru": {
+ "label": "Набор данных",
+ },
+ "zh": {
+ "label": "数据集",
+ },
+ },
+ "data_preview_btn": {
+ "en": {
+ "value": "Preview dataset",
+ },
+ "ru": {
+ "value": "Просмотреть набор данных",
+ },
+ "zh": {
+ "value": "预览数据集",
+ },
+ },
+ "preview_count": {
+ "en": {
+ "label": "Count",
+ },
+ "ru": {
+ "label": "Количество",
+ },
+ "zh": {
+ "label": "数量",
+ },
+ },
+ "page_index": {
+ "en": {
+ "label": "Page",
+ },
+ "ru": {
+ "label": "Страница",
+ },
+ "zh": {
+ "label": "页数",
+ },
+ },
+ "prev_btn": {
+ "en": {
+ "value": "Prev",
+ },
+ "ru": {
+ "value": "Предыдущая",
+ },
+ "zh": {
+ "value": "上一页",
+ },
+ },
+ "next_btn": {
+ "en": {
+ "value": "Next",
+ },
+ "ru": {
+ "value": "Следующая",
+ },
+ "zh": {
+ "value": "下一页",
+ },
+ },
+ "close_btn": {
+ "en": {
+ "value": "Close",
+ },
+ "ru": {
+ "value": "Закрыть",
+ },
+ "zh": {
+ "value": "关闭",
+ },
+ },
+ "preview_samples": {
+ "en": {
+ "label": "Samples",
+ },
+ "ru": {
+ "label": "Примеры",
+ },
+ "zh": {
+ "label": "样例",
+ },
+ },
+ "learning_rate": {
+ "en": {
+ "label": "Learning rate",
+ "info": "Initial learning rate for AdamW.",
+ },
+ "ru": {
+ "label": "Скорость обучения",
+ "info": "Начальная скорость обучения для AdamW.",
+ },
+ "zh": {
+ "label": "学习率",
+ "info": "AdamW 优化器的初始学习率。",
+ },
+ },
+ "num_train_epochs": {
+ "en": {
+ "label": "Epochs",
+ "info": "Total number of training epochs to perform.",
+ },
+ "ru": {
+ "label": "Эпохи",
+ "info": "Общее количество эпох обучения.",
+ },
+ "zh": {
+ "label": "训练轮数",
+ "info": "需要执行的训练总轮数。",
+ },
+ },
+ "max_grad_norm": {
+ "en": {
+ "label": "Maximum gradient norm",
+ "info": "Norm for gradient clipping.",
+ },
+ "ru": {
+ "label": "Максимальная норма градиента",
+ "info": "Норма для обрезки градиента.",
+ },
+ "zh": {
+ "label": "最大梯度范数",
+ "info": "用于梯度裁剪的范数。",
+ },
+ },
+ "max_samples": {
+ "en": {
+ "label": "Max samples",
+ "info": "Maximum samples per dataset.",
+ },
+ "ru": {
+ "label": "Максимальное количество образцов",
+ "info": "Максимальное количество образцов на набор данных.",
+ },
+ "zh": {
+ "label": "最大样本数",
+ "info": "每个数据集的最大样本数。",
+ },
+ },
+ "compute_type": {
+ "en": {
+ "label": "Compute type",
+ "info": "Whether to use mixed precision training.",
+ },
+ "ru": {
+ "label": "Тип вычислений",
+ "info": "Использовать ли обучение смешанной точности.",
+ },
+ "zh": {
+ "label": "计算类型",
+ "info": "是否使用混合精度训练。",
+ },
+ },
+ "cutoff_len": {
+ "en": {
+ "label": "Cutoff length",
+ "info": "Max tokens in input sequence.",
+ },
+ "ru": {
+ "label": "Длина обрезки",
+ "info": "Максимальное количество токенов во входной последовательности.",
+ },
+ "zh": {
+ "label": "截断长度",
+ "info": "输入序列分词后的最大长度。",
+ },
+ },
+ "batch_size": {
+ "en": {
+ "label": "Batch size",
+ "info": "Number of samples processed on each GPU.",
+ },
+ "ru": {
+ "label": "Размер пакета",
+ "info": "Количество образцов для обработки на каждом GPU.",
+ },
+ "zh": {
+ "label": "批处理大小",
+ "info": "每个 GPU 处理的样本数量。",
+ },
+ },
+ "gradient_accumulation_steps": {
+ "en": {
+ "label": "Gradient accumulation",
+ "info": "Number of steps for gradient accumulation.",
+ },
+ "ru": {
+ "label": "Накопление градиента",
+ "info": "Количество шагов накопления градиента.",
+ },
+ "zh": {
+ "label": "梯度累积",
+ "info": "梯度累积的步数。",
+ },
+ },
+ "val_size": {
+ "en": {
+ "label": "Val size",
+ "info": "Proportion of data in the dev set.",
+ },
+ "ru": {
+ "label": "Размер валидации",
+ "info": "Пропорция данных в наборе для разработки.",
+ },
+ "zh": {
+ "label": "验证集比例",
+ "info": "验证集占全部样本的百分比。",
+ },
+ },
+ "lr_scheduler_type": {
+ "en": {
+ "label": "LR scheduler",
+ "info": "Name of the learning rate scheduler.",
+ },
+ "ru": {
+ "label": "Планировщик скорости обучения",
+ "info": "Название планировщика скорости обучения.",
+ },
+ "zh": {
+ "label": "学习率调节器",
+ "info": "学习率调度器的名称。",
+ },
+ },
+ "extra_tab": {
+ "en": {
+ "label": "Extra configurations",
+ },
+ "ru": {
+ "label": "Дополнительные конфигурации",
+ },
+ "zh": {
+ "label": "其它参数设置",
+ },
+ },
+ "logging_steps": {
+ "en": {
+ "label": "Logging steps",
+ "info": "Number of steps between two logs.",
+ },
+ "ru": {
+ "label": "Шаги логирования",
+ "info": "Количество шагов между двумя записями в журнале.",
+ },
+ "zh": {
+ "label": "日志间隔",
+ "info": "每两次日志输出间的更新步数。",
+ },
+ },
+ "save_steps": {
+ "en": {
+ "label": "Save steps",
+ "info": "Number of steps between two checkpoints.",
+ },
+ "ru": {
+ "label": "Шаги сохранения",
+ "info": "Количество шагов между двумя контрольными точками.",
+ },
+ "zh": {
+ "label": "保存间隔",
+ "info": "每两次断点保存间的更新步数。",
+ },
+ },
+ "warmup_steps": {
+ "en": {
+ "label": "Warmup steps",
+ "info": "Number of steps used for warmup.",
+ },
+ "ru": {
+ "label": "Шаги прогрева",
+ "info": "Количество шагов, используемых для прогрева.",
+ },
+ "zh": {
+ "label": "预热步数",
+ "info": "学习率预热采用的步数。",
+ },
+ },
+ "neftune_alpha": {
+ "en": {
+ "label": "NEFTune Alpha",
+ "info": "Magnitude of noise adding to embedding vectors.",
+ },
+ "ru": {
+ "label": "NEFTune Alpha",
+ "info": "Величина шума, добавляемого к векторам вложений.",
+ },
+ "zh": {
+ "label": "NEFTune 噪声参数",
+ "info": "嵌入向量所添加的噪声大小。",
+ },
+ },
+ "optim": {
+ "en": {
+ "label": "Optimizer",
+ "info": "The optimizer to use: adamw_torch, adamw_8bit or adafactor.",
+ },
+ "ru": {
+ "label": "Оптимизатор",
+ "info": "Оптимизатор для использования: adamw_torch, adamw_8bit или adafactor.",
+ },
+ "zh": {
+ "label": "优化器",
+ "info": "使用的优化器：adamw_torch、adamw_8bit 或 adafactor。",
+ },
+ },
+ "packing": {
+ "en": {
+ "label": "Pack sequences",
+ "info": "Pack sequences into samples of fixed length.",
+ },
+ "ru": {
+ "label": "Упаковка последовательностей",
+ "info": "Упаковка последовательностей в образцы фиксированной длины.",
+ },
+ "zh": {
+ "label": "序列打包",
+ "info": "将序列打包为等长样本。",
+ },
+ },
+ "neat_packing": {
+ "en": {
+ "label": "Use neat packing",
+ "info": "Avoid cross-attention between packed sequences.",
+ },
+ "ru": {
+ "label": "Используйте аккуратную упаковку",
+ "info": "избегайте перекрестного внимания между упакованными последовательностями.",
+ },
+ "zh": {
+ "label": "使用无污染打包",
+ "info": "避免打包后的序列产生交叉注意力。",
+ },
+ },
+ "train_on_prompt": {
+ "en": {
+ "label": "Train on prompt",
+ "info": "Disable the label mask on the prompt (only for SFT).",
+ },
+ "ru": {
+ "label": "Тренировка на подсказке",
+ "info": "Отключить маску меток на подсказке (только для SFT).",
+ },
+ "zh": {
+ "label": "学习提示词",
+ "info": "不在提示词的部分添加掩码（仅适用于 SFT）。",
+ },
+ },
+ "mask_history": {
+ "en": {
+ "label": "Mask history",
+ "info": "Train on the last turn only (only for SFT).",
+ },
+ "ru": {
+ "label": "История масок",
+ "info": "Тренироваться только на последнем шаге (только для SFT).",
+ },
+ "zh": {
+ "label": "不学习历史对话",
+ "info": "仅学习最后一轮对话（仅适用于 SFT）。",
+ },
+ },
+ "resize_vocab": {
+ "en": {
+ "label": "Resize token embeddings",
+ "info": "Resize the tokenizer vocab and the embedding layers.",
+ },
+ "ru": {
+ "label": "Изменение размера токенных эмбеддингов",
+ "info": "Изменить размер словаря токенизатора и слоев эмбеддинга.",
+ },
+ "zh": {
+ "label": "更改词表大小",
+ "info": "更改分词器词表和嵌入层的大小。",
+ },
+ },
+ "use_llama_pro": {
+ "en": {
+ "label": "Enable LLaMA Pro",
+ "info": "Make the parameters in the expanded blocks trainable.",
+ },
+ "ru": {
+ "label": "Включить LLaMA Pro",
+ "info": "Сделать параметры в расширенных блоках обучаемыми.",
+ },
+ "zh": {
+ "label": "使用 LLaMA Pro",
+ "info": "仅训练块扩展后的参数。",
+ },
+ },
+ "shift_attn": {
+ "en": {
+ "label": "Enable S^2 Attention",
+ "info": "Use shift short attention proposed by LongLoRA.",
+ },
+ "ru": {
+ "label": "Включить S^2 внимание",
+ "info": "Использовать сдвиг внимания на короткие дистанции предложенный LongLoRA.",
+ },
+ "zh": {
+ "label": "使用 S^2 Attention",
+ "info": "使用 LongLoRA 提出的 shift short attention。",
+ },
+ },
+ "report_to": {
+ "en": {
+ "label": "Enable external logger",
+ "info": "Use TensorBoard or wandb to log experiment.",
+ },
+ "ru": {
+ "label": "Включить внешний регистратор",
+ "info": "Использовать TensorBoard или wandb для ведения журнала экспериментов.",
+ },
+ "zh": {
+ "label": "启用外部记录面板",
+ "info": "使用 TensorBoard 或 wandb 记录实验。",
+ },
+ },
+ "freeze_tab": {
+ "en": {
+ "label": "Freeze tuning configurations",
+ },
+ "ru": {
+ "label": "конфигурации для настройки заморозки",
+ },
+ "zh": {
+ "label": "部分参数微调设置",
+ },
+ },
+ "freeze_trainable_layers": {
+ "en": {
+ "label": "Trainable layers",
+ "info": "Number of the last(+)/first(-) hidden layers to be set as trainable.",
+ },
+ "ru": {
+ "label": "Обучаемые слои",
+ "info": "Количество последних (+)/первых (-) скрытых слоев, которые будут установлены как обучаемые.",
+ },
+ "zh": {
+ "label": "可训练层数",
+ "info": "最末尾（+）/最前端（-）可训练隐藏层的数量。",
+ },
+ },
+ "freeze_trainable_modules": {
+ "en": {
+ "label": "Trainable modules",
+ "info": "Name(s) of trainable modules. Use commas to separate multiple modules.",
+ },
+ "ru": {
+ "label": "Обучаемые модули",
+ "info": "Название обучаемых модулей. Используйте запятые для разделения нескольких модулей.",
+ },
+ "zh": {
+ "label": "可训练模块",
+ "info": "可训练模块的名称。使用英文逗号分隔多个名称。",
+ },
+ },
+ "freeze_extra_modules": {
+ "en": {
+ "label": "Extra modules (optional)",
+ "info": (
+ "Name(s) of modules apart from hidden layers to be set as trainable. "
+ "Use commas to separate multiple modules."
+ ),
+ },
+ "ru": {
+ "label": "Дополнительные модули (опционально)",
+ "info": (
+ "Имена модулей, кроме скрытых слоев, которые следует установить в качестве обучаемых. "
+ "Используйте запятые для разделения нескольких модулей."
+ ),
+ },
+ "zh": {
+ "label": "额外模块（非必填）",
+ "info": "除隐藏层以外的可训练模块名称。使用英文逗号分隔多个名称。",
+ },
+ },
+ "lora_tab": {
+ "en": {
+ "label": "LoRA configurations",
+ },
+ "ru": {
+ "label": "Конфигурации LoRA",
+ },
+ "zh": {
+ "label": "LoRA 参数设置",
+ },
+ },
+ "lora_rank": {
+ "en": {
+ "label": "LoRA rank",
+ "info": "The rank of LoRA matrices.",
+ },
+ "ru": {
+ "label": "Ранг матриц LoRA",
+ "info": "Ранг матриц LoRA.",
+ },
+ "zh": {
+ "label": "LoRA 秩",
+ "info": "LoRA 矩阵的秩大小。",
+ },
+ },
+ "lora_alpha": {
+ "en": {
+ "label": "LoRA alpha",
+ "info": "Lora scaling coefficient.",
+ },
+ "ru": {
+ "label": "LoRA alpha",
+ "info": "Коэффициент масштабирования LoRA.",
+ },
+ "zh": {
+ "label": "LoRA 缩放系数",
+ "info": "LoRA 缩放系数大小。",
+ },
+ },
+ "lora_dropout": {
+ "en": {
+ "label": "LoRA dropout",
+ "info": "Dropout ratio of LoRA weights.",
+ },
+ "ru": {
+ "label": "Вероятность отсева LoRA",
+ "info": "Вероятность отсева весов LoRA.",
+ },
+ "zh": {
+ "label": "LoRA 随机丢弃",
+ "info": "LoRA 权重随机丢弃的概率。",
+ },
+ },
+ "loraplus_lr_ratio": {
+ "en": {
+ "label": "LoRA+ LR ratio",
+ "info": "The LR ratio of the B matrices in LoRA.",
+ },
+ "ru": {
+ "label": "LoRA+ LR коэффициент",
+ "info": "Коэффициент LR матриц B в LoRA.",
+ },
+ "zh": {
+ "label": "LoRA+ 学习率比例",
+ "info": "LoRA+ 中 B 矩阵的学习率倍数。",
+ },
+ },
+ "create_new_adapter": {
+ "en": {
+ "label": "Create new adapter",
+ "info": "Create a new adapter with randomly initialized weight upon the existing one.",
+ },
+ "ru": {
+ "label": "Создать новый адаптер",
+ "info": "Создать новый адаптер с случайной инициализацией веса на основе существующего.",
+ },
+ "zh": {
+ "label": "新建适配器",
+ "info": "在现有的适配器上创建一个随机初始化后的新适配器。",
+ },
+ },
+ "use_rslora": {
+ "en": {
+ "label": "Use rslora",
+ "info": "Use the rank stabilization scaling factor for LoRA layer.",
+ },
+ "ru": {
+ "label": "Использовать rslora",
+ "info": "Использовать коэффициент масштабирования стабилизации ранга для слоя LoRA.",
+ },
+ "zh": {
+ "label": "使用 rslora",
+ "info": "对 LoRA 层使用秩稳定缩放方法。",
+ },
+ },
+ "use_dora": {
+ "en": {
+ "label": "Use DoRA",
+ "info": "Use weight-decomposed LoRA.",
+ },
+ "ru": {
+ "label": "Используйте DoRA",
+ "info": "Используйте LoRA с декомпозицией весов.",
+ },
+ "zh": {
+ "label": "使用 DoRA",
+ "info": "使用权重分解的 LoRA。",
+ },
+ },
+ "use_pissa": {
+ "en": {
+ "label": "Use PiSSA",
+ "info": "Use PiSSA method.",
+ },
+ "ru": {
+ "label": "используйте PiSSA",
+ "info": "Используйте метод PiSSA.",
+ },
+ "zh": {
+ "label": "使用 PiSSA",
+ "info": "使用 PiSSA 方法。",
+ },
+ },
+ "lora_target": {
+ "en": {
+ "label": "LoRA modules (optional)",
+ "info": "Name(s) of modules to apply LoRA. Use commas to separate multiple modules.",
+ },
+ "ru": {
+ "label": "Модули LoRA (опционально)",
+ "info": "Имена модулей для применения LoRA. Используйте запятые для разделения нескольких модулей.",
+ },
+ "zh": {
+ "label": "LoRA 作用模块（非必填）",
+ "info": "应用 LoRA 的模块名称。使用英文逗号分隔多个名称。",
+ },
+ },
+ "additional_target": {
+ "en": {
+ "label": "Additional modules (optional)",
+ "info": (
+ "Name(s) of modules apart from LoRA layers to be set as trainable. "
+ "Use commas to separate multiple modules."
+ ),
+ },
+ "ru": {
+ "label": "Дополнительные модули (опционально)",
+ "info": (
+ "Имена модулей, кроме слоев LoRA, которые следует установить в качестве обучаемых. "
+ "Используйте запятые для разделения нескольких модулей."
+ ),
+ },
+ "zh": {
+ "label": "附加模块（非必填）",
+ "info": "除 LoRA 层以外的可训练模块名称。使用英文逗号分隔多个名称。",
+ },
+ },
+ "rlhf_tab": {
+ "en": {
+ "label": "RLHF configurations",
+ },
+ "ru": {
+ "label": "Конфигурации RLHF",
+ },
+ "zh": {
+ "label": "RLHF 参数设置",
+ },
+ },
+ "pref_beta": {
+ "en": {
+ "label": "Beta value",
+ "info": "Value of the beta parameter in the loss.",
+ },
+ "ru": {
+ "label": "Бета значение",
+ "info": "Значение параметра бета в функции потерь.",
+ },
+ "zh": {
+ "label": "Beta 参数",
+ "info": "损失函数中 beta 超参数大小。",
+ },
+ },
+ "pref_ftx": {
+ "en": {
+ "label": "Ftx gamma",
+ "info": "The weight of SFT loss in the final loss.",
+ },
+ "ru": {
+ "label": "Ftx гамма",
+ "info": "Вес потери SFT в итоговой потере.",
+ },
+ "zh": {
+ "label": "Ftx gamma",
+ "info": "损失函数中 SFT 损失的权重大小。",
+ },
+ },
+ "pref_loss": {
+ "en": {
+ "label": "Loss type",
+ "info": "The type of the loss function.",
+ },
+ "ru": {
+ "label": "Тип потерь",
+ "info": "Тип функции потерь.",
+ },
+ "zh": {
+ "label": "损失类型",
+ "info": "损失函数的类型。",
+ },
+ },
+ "reward_model": {
+ "en": {
+ "label": "Reward model",
+ "info": "Adapter of the reward model in PPO training.",
+ },
+ "ru": {
+ "label": "Модель вознаграждения",
+ "info": "Адаптер модели вознаграждения для обучения PPO.",
+ },
+ "zh": {
+ "label": "奖励模型",
+ "info": "PPO 训练中奖励模型的适配器路径。",
+ },
+ },
+ "ppo_score_norm": {
+ "en": {
+ "label": "Score norm",
+ "info": "Normalizing scores in PPO training.",
+ },
+ "ru": {
+ "label": "Норма оценок",
+ "info": "Нормализация оценок в тренировке PPO.",
+ },
+ "zh": {
+ "label": "奖励模型",
+ "info": "PPO 训练中归一化奖励分数。",
+ },
+ },
+ "ppo_whiten_rewards": {
+ "en": {
+ "label": "Whiten rewards",
+ "info": "Whiten the rewards in PPO training.",
+ },
+ "ru": {
+ "label": "Белые вознаграждения",
+ "info": "Осветлите вознаграждения в обучении PPO.",
+ },
+ "zh": {
+ "label": "白化奖励",
+ "info": "PPO 训练中将奖励分数做白化处理。",
+ },
+ },
+ "galore_tab": {
+ "en": {
+ "label": "GaLore configurations",
+ },
+ "ru": {
+ "label": "Конфигурации GaLore",
+ },
+ "zh": {
+ "label": "GaLore 参数设置",
+ },
+ },
+ "use_galore": {
+ "en": {
+ "label": "Use GaLore",
+ "info": "Enable gradient low-Rank projection.",
+ },
+ "ru": {
+ "label": "Использовать GaLore",
+ "info": "Включить проекцию градиента на низкоранговое пространство.",
+ },
+ "zh": {
+ "label": "使用 GaLore",
+ "info": "使用梯度低秩投影。",
+ },
+ },
+ "galore_rank": {
+ "en": {
+ "label": "GaLore rank",
+ "info": "The rank of GaLore gradients.",
+ },
+ "ru": {
+ "label": "Ранг GaLore",
+ "info": "Ранг градиентов GaLore.",
+ },
+ "zh": {
+ "label": "GaLore 秩",
+ "info": "GaLore 梯度的秩大小。",
+ },
+ },
+ "galore_update_interval": {
+ "en": {
+ "label": "Update interval",
+ "info": "Number of steps to update the GaLore projection.",
+ },
+ "ru": {
+ "label": "Интервал обновления",
+ "info": "Количество шагов для обновления проекции GaLore.",
+ },
+ "zh": {
+ "label": "更新间隔",
+ "info": "相邻两次投影更新的步数。",
+ },
+ },
+ "galore_scale": {
+ "en": {
+ "label": "GaLore scale",
+ "info": "GaLore scaling coefficient.",
+ },
+ "ru": {
+ "label": "LoRA Alpha",
+ "info": "Коэффициент масштабирования GaLore.",
+ },
+ "zh": {
+ "label": "GaLore 缩放系数",
+ "info": "GaLore 缩放系数大小。",
+ },
+ },
+ "galore_target": {
+ "en": {
+ "label": "GaLore modules",
+ "info": "Name(s) of modules to apply GaLore. Use commas to separate multiple modules.",
+ },
+ "ru": {
+ "label": "Модули GaLore",
+ "info": "Имена модулей для применения GaLore. Используйте запятые для разделения нескольких модулей.",
+ },
+ "zh": {
+ "label": "GaLore 作用模块",
+ "info": "应用 GaLore 的模块名称。使用英文逗号分隔多个名称。",
+ },
+ },
+ "badam_tab": {
+ "en": {
+ "label": "BAdam configurations",
+ },
+ "ru": {
+ "label": "Конфигурации BAdam",
+ },
+ "zh": {
+ "label": "BAdam 参数设置",
+ },
+ },
+ "use_badam": {
+ "en": {
+ "label": "Use BAdam",
+ "info": "Enable the BAdam optimizer.",
+ },
+ "ru": {
+ "label": "Использовать BAdam",
+ "info": "Включите оптимизатор BAdam.",
+ },
+ "zh": {
+ "label": "使用 BAdam",
+ "info": "使用 BAdam 优化器。",
+ },
+ },
+ "badam_mode": {
+ "en": {
+ "label": "BAdam mode",
+ "info": "Whether to use layer-wise or ratio-wise BAdam optimizer.",
+ },
+ "ru": {
+ "label": "Режим BAdam",
+ "info": "Использовать ли оптимизатор BAdam с послоевой или пропорциональной настройкой.",
+ },
+ "zh": {
+ "label": "BAdam 模式",
+ "info": "使用 layer-wise 或 ratio-wise BAdam 优化器。",
+ },
+ },
+ "badam_switch_mode": {
+ "en": {
+ "label": "Switch mode",
+ "info": "The strategy of picking block to update for layer-wise BAdam.",
+ },
+ "ru": {
+ "label": "Режим переключения",
+ "info": "Стратегия выбора блока для обновления для послойного BAdam.",
+ },
+ "zh": {
+ "label": "切换策略",
+ "info": "Layer-wise BAdam 优化器的块切换策略。",
+ },
+ },
+ "badam_switch_interval": {
+ "en": {
+ "label": "Switch interval",
+ "info": "Number of steps to update the block for layer-wise BAdam.",
+ },
+ "ru": {
+ "label": "Интервал переключения",
+ "info": "количество шагов для обновления блока для пошагового BAdam.",
+ },
+ "zh": {
+ "label": "切换频率",
+ "info": "Layer-wise BAdam 优化器的块切换频率。",
+ },
+ },
+ "badam_update_ratio": {
+ "en": {
+ "label": "Update ratio",
+ "info": "The ratio of the update for ratio-wise BAdam.",
+ },
+ "ru": {
+ "label": "Коэффициент обновления",
+ "info": "Коэффициент обновления для BAdam с учётом соотношений.",
+ },
+ "zh": {
+ "label": "Block 更新比例",
+ "info": "Ratio-wise BAdam 优化器的更新比例。",
+ },
+ },
+ "cmd_preview_btn": {
+ "en": {
+ "value": "Preview command",
+ },
+ "ru": {
+ "value": "Просмотр команды",
+ },
+ "zh": {
+ "value": "预览命令",
+ },
+ },
+ "arg_save_btn": {
+ "en": {
+ "value": "Save arguments",
+ },
+ "ru": {
+ "value": "Сохранить аргументы",
+ },
+ "zh": {
+ "value": "保存训练参数",
+ },
+ },
+ "arg_load_btn": {
+ "en": {
+ "value": "Load arguments",
+ },
+ "ru": {
+ "value": "Загрузить аргументы",
+ },
+ "zh": {
+ "value": "载入训练参数",
+ },
+ },
+ "start_btn": {
+ "en": {
+ "value": "Start",
+ },
+ "ru": {
+ "value": "Начать",
+ },
+ "zh": {
+ "value": "开始",
+ },
+ },
+ "stop_btn": {
+ "en": {
+ "value": "Abort",
+ },
+ "ru": {
+ "value": "Прервать",
+ },
+ "zh": {
+ "value": "中断",
+ },
+ },
+ "output_dir": {
+ "en": {
+ "label": "Output dir",
+ "info": "Directory for saving results.",
+ },
+ "ru": {
+ "label": "Выходной каталог",
+ "info": "Каталог для сохранения результатов.",
+ },
+ "zh": {
+ "label": "输出目录",
+ "info": "保存结果的路径。",
+ },
+ },
+ "config_path": {
+ "en": {
+ "label": "Config path",
+ "info": "Path to config saving arguments.",
+ },
+ "ru": {
+ "label": "Путь к конфигурации",
+ "info": "Путь для сохранения аргументов конфигурации.",
+ },
+ "zh": {
+ "label": "配置路径",
+ "info": "保存训练参数的配置文件路径。",
+ },
+ },
+ "device_count": {
+ "en": {
+ "label": "Device count",
+ "info": "Number of devices available.",
+ },
+ "ru": {
+ "label": "Количество устройств",
+ "info": "Количество доступных устройств.",
+ },
+ "zh": {
+ "label": "设备数量",
+ "info": "当前可用的运算设备数。",
+ },
+ },
+ "ds_stage": {
+ "en": {
+ "label": "DeepSpeed stage",
+ "info": "DeepSpeed stage for distributed training.",
+ },
+ "ru": {
+ "label": "Этап DeepSpeed",
+ "info": "Этап DeepSpeed для распределенного обучения.",
+ },
+ "zh": {
+ "label": "DeepSpeed stage",
+ "info": "多卡训练的 DeepSpeed stage。",
+ },
+ },
+ "ds_offload": {
+ "en": {
+ "label": "Enable offload",
+ "info": "Enable DeepSpeed offload (slow down training).",
+ },
+ "ru": {
+ "label": "Включить выгрузку",
+ "info": "включить выгрузку DeepSpeed (замедлит обучение).",
+ },
+ "zh": {
+ "label": "使用 offload",
+ "info": "使用 DeepSpeed offload（会减慢速度）。",
+ },
+ },
+ "output_box": {
+ "en": {
+ "value": "Ready.",
+ },
+ "ru": {
+ "value": "Готово.",
+ },
+ "zh": {
+ "value": "准备就绪。",
+ },
+ },
+ "loss_viewer": {
+ "en": {
+ "label": "Loss",
+ },
+ "ru": {
+ "label": "Потери",
+ },
+ "zh": {
+ "label": "损失",
+ },
+ },
+ "predict": {
+ "en": {
+ "label": "Save predictions",
+ },
+ "ru": {
+ "label": "Сохранить предсказания",
+ },
+ "zh": {
+ "label": "保存预测结果",
+ },
+ },
+ "infer_backend": {
+ "en": {
+ "label": "Inference engine",
+ },
+ "ru": {
+ "label": "Инференс движок",
+ },
+ "zh": {
+ "label": "推理引擎",
+ },
+ },
+ "infer_dtype": {
+ "en": {
+ "label": "Inference data type",
+ },
+ "ru": {
+ "label": "Тип данных для вывода",
+ },
+ "zh": {
+ "label": "推理数据类型",
+ },
+ },
+ "load_btn": {
+ "en": {
+ "value": "Load model",
+ },
+ "ru": {
+ "value": "Загрузить модель",
+ },
+ "zh": {
+ "value": "加载模型",
+ },
+ },
+ "unload_btn": {
+ "en": {
+ "value": "Unload model",
+ },
+ "ru": {
+ "value": "Выгрузить модель",
+ },
+ "zh": {
+ "value": "卸载模型",
+ },
+ },
+ "info_box": {
+ "en": {
+ "value": "Model unloaded, please load a model first.",
+ },
+ "ru": {
+ "value": "Модель не загружена, загрузите модель сначала.",
+ },
+ "zh": {
+ "value": "模型未加载，请先加载模型。",
+ },
+ },
+ "role": {
+ "en": {
+ "label": "Role",
+ },
+ "ru": {
+ "label": "Роль",
+ },
+ "zh": {
+ "label": "角色",
+ },
+ },
+ "system": {
+ "en": {
+ "placeholder": "System prompt (optional)",
+ },
+ "ru": {
+ "placeholder": "Системный запрос (по желанию)",
+ },
+ "zh": {
+ "placeholder": "系统提示词（非必填）",
+ },
+ },
+ "tools": {
+ "en": {
+ "placeholder": "Tools (optional)",
+ },
+ "ru": {
+ "placeholder": "Инструменты (по желанию)",
+ },
+ "zh": {
+ "placeholder": "工具列表（非必填）",
+ },
+ },
+ "image": {
+ "en": {
+ "label": "Image (optional)",
+ },
+ "ru": {
+ "label": "Изображение (по желанию)",
+ },
+ "zh": {
+ "label": "图像（非必填）",
+ },
+ },
+ "query": {
+ "en": {
+ "placeholder": "Input...",
+ },
+ "ru": {
+ "placeholder": "Ввод...",
+ },
+ "zh": {
+ "placeholder": "输入...",
+ },
+ },
+ "submit_btn": {
+ "en": {
+ "value": "Submit",
+ },
+ "ru": {
+ "value": "Отправить",
+ },
+ "zh": {
+ "value": "提交",
+ },
+ },
+ "max_length": {
+ "en": {
+ "label": "Maximum length",
+ },
+ "ru": {
+ "label": "Максимальная длина",
+ },
+ "zh": {
+ "label": "最大长度",
+ },
+ },
+ "max_new_tokens": {
+ "en": {
+ "label": "Maximum new tokens",
+ },
+ "ru": {
+ "label": "Максимальное количество новых токенов",
+ },
+ "zh": {
+ "label": "最大生成长度",
+ },
+ },
+ "top_p": {
+ "en": {
+ "label": "Top-p",
+ },
+ "ru": {
+ "label": "Лучшие-p",
+ },
+ "zh": {
+ "label": "Top-p 采样值",
+ },
+ },
+ "temperature": {
+ "en": {
+ "label": "Temperature",
+ },
+ "ru": {
+ "label": "Температура",
+ },
+ "zh": {
+ "label": "温度系数",
+ },
+ },
+ "clear_btn": {
+ "en": {
+ "value": "Clear history",
+ },
+ "ru": {
+ "value": "Очистить историю",
+ },
+ "zh": {
+ "value": "清空历史",
+ },
+ },
+ "export_size": {
+ "en": {
+ "label": "Max shard size (GB)",
+ "info": "The maximum size for a model file.",
+ },
+ "ru": {
+ "label": "Максимальный размер фрагмента (ГБ)",
+ "info": "Максимальный размер файла модели.",
+ },
+ "zh": {
+ "label": "最大分块大小（GB）",
+ "info": "单个模型文件的最大大小。",
+ },
+ },
+ "export_quantization_bit": {
+ "en": {
+ "label": "Export quantization bit.",
+ "info": "Quantizing the exported model.",
+ },
+ "ru": {
+ "label": "Экспорт бита квантования",
+ "info": "Квантование экспортируемой модели.",
+ },
+ "zh": {
+ "label": "导出量化等级",
+ "info": "量化导出模型。",
+ },
+ },
+ "export_quantization_dataset": {
+ "en": {
+ "label": "Export quantization dataset",
+ "info": "The calibration dataset used for quantization.",
+ },
+ "ru": {
+ "label": "Экспорт набора данных для квантования",
+ "info": "Набор данных калибровки, используемый для квантования.",
+ },
+ "zh": {
+ "label": "导出量化数据集",
+ "info": "量化过程中使用的校准数据集。",
+ },
+ },
+ "export_device": {
+ "en": {
+ "label": "Export device",
+ "info": "Which device should be used to export model.",
+ },
+ "ru": {
+ "label": "Экспорт устройство",
+ "info": "Какое устройство следует использовать для экспорта модели.",
+ },
+ "zh": {
+ "label": "导出设备",
+ "info": "导出模型使用的设备类型。",
+ },
+ },
+ "export_legacy_format": {
+ "en": {
+ "label": "Export legacy format",
+ "info": "Do not use safetensors to save the model.",
+ },
+ "ru": {
+ "label": "Экспорт в устаревший формат",
+ "info": "Не использовать safetensors для сохранения модели.",
+ },
+ "zh": {
+ "label": "导出旧格式",
+ "info": "不使用 safetensors 格式保存模型。",
+ },
+ },
+ "export_dir": {
+ "en": {
+ "label": "Export dir",
+ "info": "Directory to save exported model.",
+ },
+ "ru": {
+ "label": "Каталог экспорта",
+ "info": "Каталог для сохранения экспортированной модели.",
+ },
+ "zh": {
+ "label": "导出目录",
+ "info": "保存导出模型的文件夹路径。",
+ },
+ },
+ "export_hub_model_id": {
+ "en": {
+ "label": "HF Hub ID (optional)",
+ "info": "Repo ID for uploading model to Hugging Face hub.",
+ },
+ "ru": {
+ "label": "HF Hub ID (опционально)",
+ "info": "Идентификатор репозитория для загрузки модели на Hugging Face hub.",
+ },
+ "zh": {
+ "label": "HF Hub ID（非必填）",
+ "info": "用于将模型上传至 Hugging Face Hub 的仓库 ID。",
+ },
+ },
+ "export_btn": {
+ "en": {
+ "value": "Export",
+ },
+ "ru": {
+ "value": "Экспорт",
+ },
+ "zh": {
+ "value": "开始导出",
+ },
+ },
+}
+
+
+ALERTS = {
+ "err_conflict": {
+ "en": "A process is in running, please abort it first.",
+ "ru": "Процесс уже запущен, пожалуйста, сначала прервите его.",
+ "zh": "任务已存在，请先中断训练。",
+ },
+ "err_exists": {
+ "en": "You have loaded a model, please unload it first.",
+ "ru": "Вы загрузили модель, сначала разгрузите ее.",
+ "zh": "模型已存在，请先卸载模型。",
+ },
+ "err_no_model": {
+ "en": "Please select a model.",
+ "ru": "Пожалуйста, выберите модель.",
+ "zh": "请选择模型。",
+ },
+ "err_no_path": {
+ "en": "Model not found.",
+ "ru": "Модель не найдена.",
+ "zh": "模型未找到。",
+ },
+ "err_no_dataset": {
+ "en": "Please choose a dataset.",
+ "ru": "Пожалуйста, выберите набор данных.",
+ "zh": "请选择数据集。",
+ },
+ "err_no_adapter": {
+ "en": "Please select an adapter.",
+ "ru": "Пожалуйста, выберите адаптер.",
+ "zh": "请选择适配器。",
+ },
+ "err_no_output_dir": {
+ "en": "Please provide output dir.",
+ "ru": "Пожалуйста, укажите выходную директорию.",
+ "zh": "请填写输出目录。",
+ },
+ "err_no_reward_model": {
+ "en": "Please select a reward model.",
+ "ru": "Пожалуйста, выберите модель вознаграждения.",
+ "zh": "请选择奖励模型。",
+ },
+ "err_no_export_dir": {
+ "en": "Please provide export dir.",
+ "ru": "Пожалуйста, укажите каталог для экспорта.",
+ "zh": "请填写导出目录。",
+ },
+ "err_gptq_lora": {
+ "en": "Please merge adapters before quantizing the model.",
+ "ru": "Пожалуйста, объедините адаптеры перед квантованием модели.",
+ "zh": "量化模型前请先合并适配器。",
+ },
+ "err_failed": {
+ "en": "Failed.",
+ "ru": "Ошибка.",
+ "zh": "训练出错。",
+ },
+ "err_demo": {
+ "en": "Training is unavailable in demo mode, duplicate the space to a private one first.",
+ "ru": "Обучение недоступно в демонстрационном режиме, сначала скопируйте пространство в частное.",
+ "zh": "展示模式不支持训练，请先复制到私人空间。",
+ },
+ "err_tool_name": {
+ "en": "Tool name not found.",
+ "ru": "Имя инструмента не найдено.",
+ "zh": "工具名称未找到。",
+ },
+ "err_json_schema": {
+ "en": "Invalid JSON schema.",
+ "ru": "Неверная схема JSON.",
+ "zh": "Json 格式错误。",
+ },
+ "err_config_not_found": {
+ "en": "Config file is not found.",
+ "ru": "Файл конфигурации не найден.",
+ "zh": "未找到配置文件。",
+ },
+ "warn_no_cuda": {
+ "en": "CUDA environment was not detected.",
+ "ru": "Среда CUDA не обнаружена.",
+ "zh": "未检测到 CUDA 环境。",
+ },
+ "warn_output_dir_exists": {
+ "en": "Output dir already exists, will resume training from here.",
+ "ru": "Выходной каталог уже существует, обучение будет продолжено отсюда.",
+ "zh": "输出目录已存在，将从该断点恢复训练。",
+ },
+ "info_aborting": {
+ "en": "Aborted, wait for terminating...",
+ "ru": "Прервано, ожидание завершения...",
+ "zh": "训练中断，正在等待进程结束……",
+ },
+ "info_aborted": {
+ "en": "Ready.",
+ "ru": "Готово.",
+ "zh": "准备就绪。",
+ },
+ "info_finished": {
+ "en": "Finished.",
+ "ru": "Завершено.",
+ "zh": "训练完毕。",
+ },
+ "info_config_saved": {
+ "en": "Arguments have been saved at: ",
+ "ru": "Аргументы были сохранены по адресу: ",
+ "zh": "训练参数已保存至：",
+ },
+ "info_config_loaded": {
+ "en": "Arguments have been restored.",
+ "ru": "Аргументы были восстановлены.",
+ "zh": "训练参数已载入。",
+ },
+ "info_loading": {
+ "en": "Loading model...",
+ "ru": "Загрузка модели...",
+ "zh": "加载中……",
+ },
+ "info_unloading": {
+ "en": "Unloading model...",
+ "ru": "Выгрузка модели...",
+ "zh": "卸载中……",
+ },
+ "info_loaded": {
+ "en": "Model loaded, now you can chat with your model!",
+ "ru": "Модель загружена, теперь вы можете общаться с вашей моделью!",
+ "zh": "模型已加载，可以开始聊天了！",
+ },
+ "info_unloaded": {
+ "en": "Model unloaded.",
+ "ru": "Модель выгружена.",
+ "zh": "模型已卸载。",
+ },
+ "info_exporting": {
+ "en": "Exporting model...",
+ "ru": "Экспорт модели...",
+ "zh": "正在导出模型……",
+ },
+ "info_exported": {
+ "en": "Model exported.",
+ "ru": "Модель экспортирована.",
+ "zh": "模型导出完成。",
+ },
+}
diff --git a/llama-factory/src/llamafactory/webui/manager.py b/llama-factory/src/llamafactory/webui/manager.py
new file mode 100644
index 0000000000000000000000000000000000000000..ebe9f1b9ba1cc11ae5e4c57bc28feef11c5c4ed8
--- /dev/null
+++ b/llama-factory/src/llamafactory/webui/manager.py
@@ -0,0 +1,79 @@
+# Copyright 2024 the LlamaFactory team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from typing import TYPE_CHECKING, Dict, Generator, List, Set, Tuple
+
+
+if TYPE_CHECKING:
+ from gradio.components import Component
+
+
+class Manager:
+ def __init__(self) -> None:
+ self._id_to_elem: Dict[str, "Component"] = {}
+ self._elem_to_id: Dict["Component", str] = {}
+
+ def add_elems(self, tab_name: str, elem_dict: Dict[str, "Component"]) -> None:
+ r"""
+ Adds elements to manager.
+ """
+ for elem_name, elem in elem_dict.items():
+ elem_id = "{}.{}".format(tab_name, elem_name)
+ self._id_to_elem[elem_id] = elem
+ self._elem_to_id[elem] = elem_id
+
+ def get_elem_list(self) -> List["Component"]:
+ r"""
+ Returns the list of all elements.
+ """
+ return list(self._id_to_elem.values())
+
+ def get_elem_iter(self) -> Generator[Tuple[str, "Component"], None, None]:
+ r"""
+ Returns an iterator over all elements with their names.
+ """
+ for elem_id, elem in self._id_to_elem.items():
+ yield elem_id.split(".")[-1], elem
+
+ def get_elem_by_id(self, elem_id: str) -> "Component":
+ r"""
+ Gets element by id.
+
+ Example: top.lang, train.dataset
+ """
+ return self._id_to_elem[elem_id]
+
+ def get_id_by_elem(self, elem: "Component") -> str:
+ r"""
+ Gets id by element.
+ """
+ return self._elem_to_id[elem]
+
+ def get_base_elems(self) -> Set["Component"]:
+ r"""
+ Gets the base elements that are commonly used.
+ """
+ return {
+ self._id_to_elem["top.lang"],
+ self._id_to_elem["top.model_name"],
+ self._id_to_elem["top.model_path"],
+ self._id_to_elem["top.finetuning_type"],
+ self._id_to_elem["top.checkpoint_path"],
+ self._id_to_elem["top.quantization_bit"],
+ self._id_to_elem["top.quantization_method"],
+ self._id_to_elem["top.template"],
+ self._id_to_elem["top.rope_scaling"],
+ self._id_to_elem["top.booster"],
+ self._id_to_elem["top.visual_inputs"],
+ }
diff --git a/llama-factory/src/llamafactory/webui/runner.py b/llama-factory/src/llamafactory/webui/runner.py
new file mode 100644
index 0000000000000000000000000000000000000000..9bc6967918c543f7545249c016361203c38de5e8
--- /dev/null
+++ b/llama-factory/src/llamafactory/webui/runner.py
@@ -0,0 +1,437 @@
+# Copyright 2024 the LlamaFactory team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+from copy import deepcopy
+from subprocess import Popen, TimeoutExpired
+from typing import TYPE_CHECKING, Any, Dict, Generator, Optional
+
+from transformers.trainer import TRAINING_ARGS_NAME
+
+from ..extras.constants import LLAMABOARD_CONFIG, PEFT_METHODS, TRAINING_STAGES
+from ..extras.misc import is_gpu_or_npu_available, torch_gc
+from ..extras.packages import is_gradio_available
+from .common import DEFAULT_CACHE_DIR, DEFAULT_CONFIG_DIR, QUANTIZATION_BITS, get_save_dir, load_config
+from .locales import ALERTS, LOCALES
+from .utils import abort_process, gen_cmd, get_eval_results, get_trainer_info, load_args, save_args, save_cmd
+
+
+if is_gradio_available():
+ import gradio as gr
+
+
+if TYPE_CHECKING:
+ from gradio.components import Component
+
+ from .manager import Manager
+
+
+class Runner:
+ def __init__(self, manager: "Manager", demo_mode: bool = False) -> None:
+ self.manager = manager
+ self.demo_mode = demo_mode
+ """ Resume """
+ self.trainer: Optional["Popen"] = None
+ self.do_train = True
+ self.running_data: Dict["Component", Any] = None
+ """ State """
+ self.aborted = False
+ self.running = False
+
+ def set_abort(self) -> None:
+ self.aborted = True
+ if self.trainer is not None:
+ abort_process(self.trainer.pid)
+
+ def _initialize(self, data: Dict["Component", Any], do_train: bool, from_preview: bool) -> str:
+ get = lambda elem_id: data[self.manager.get_elem_by_id(elem_id)]
+ lang, model_name, model_path = get("top.lang"), get("top.model_name"), get("top.model_path")
+ dataset = get("train.dataset") if do_train else get("eval.dataset")
+
+ if self.running:
+ return ALERTS["err_conflict"][lang]
+
+ if not model_name:
+ return ALERTS["err_no_model"][lang]
+
+ if not model_path:
+ return ALERTS["err_no_path"][lang]
+
+ if not dataset:
+ return ALERTS["err_no_dataset"][lang]
+
+ if not from_preview and self.demo_mode:
+ return ALERTS["err_demo"][lang]
+
+ if do_train:
+ if not get("train.output_dir"):
+ return ALERTS["err_no_output_dir"][lang]
+
+ stage = TRAINING_STAGES[get("train.training_stage")]
+ if stage == "ppo" and not get("train.reward_model"):
+ return ALERTS["err_no_reward_model"][lang]
+ else:
+ if not get("eval.output_dir"):
+ return ALERTS["err_no_output_dir"][lang]
+
+ if not from_preview and not is_gpu_or_npu_available():
+ gr.Warning(ALERTS["warn_no_cuda"][lang])
+
+ return ""
+
+ def _finalize(self, lang: str, finish_info: str) -> str:
+ finish_info = ALERTS["info_aborted"][lang] if self.aborted else finish_info
+ self.trainer = None
+ self.aborted = False
+ self.running = False
+ self.running_data = None
+ torch_gc()
+ return finish_info
+
+ def _parse_train_args(self, data: Dict["Component", Any]) -> Dict[str, Any]:
+ get = lambda elem_id: data[self.manager.get_elem_by_id(elem_id)]
+ model_name, finetuning_type = get("top.model_name"), get("top.finetuning_type")
+ user_config = load_config()
+
+ if get("top.quantization_bit") in QUANTIZATION_BITS:
+ quantization_bit = int(get("top.quantization_bit"))
+ else:
+ quantization_bit = None
+
+ args = dict(
+ stage=TRAINING_STAGES[get("train.training_stage")],
+ do_train=True,
+ model_name_or_path=get("top.model_path"),
+ cache_dir=user_config.get("cache_dir", None),
+ preprocessing_num_workers=16,
+ finetuning_type=finetuning_type,
+ quantization_bit=quantization_bit,
+ quantization_method=get("top.quantization_method"),
+ template=get("top.template"),
+ rope_scaling=get("top.rope_scaling") if get("top.rope_scaling") in ["linear", "dynamic"] else None,
+ flash_attn="fa2" if get("top.booster") == "flashattn2" else "auto",
+ use_unsloth=(get("top.booster") == "unsloth"),
+ visual_inputs=get("top.visual_inputs"),
+ dataset_dir=get("train.dataset_dir"),
+ dataset=",".join(get("train.dataset")),
+ cutoff_len=get("train.cutoff_len"),
+ learning_rate=float(get("train.learning_rate")),
+ num_train_epochs=float(get("train.num_train_epochs")),
+ max_samples=int(get("train.max_samples")),
+ per_device_train_batch_size=get("train.batch_size"),
+ gradient_accumulation_steps=get("train.gradient_accumulation_steps"),
+ lr_scheduler_type=get("train.lr_scheduler_type"),
+ max_grad_norm=float(get("train.max_grad_norm")),
+ logging_steps=get("train.logging_steps"),
+ save_steps=get("train.save_steps"),
+ warmup_steps=get("train.warmup_steps"),
+ neftune_noise_alpha=get("train.neftune_alpha") or None,
+ optim=get("train.optim"),
+ packing=get("train.packing") or get("train.neat_packing"),
+ neat_packing=get("train.neat_packing"),
+ train_on_prompt=get("train.train_on_prompt"),
+ mask_history=get("train.mask_history"),
+ resize_vocab=get("train.resize_vocab"),
+ use_llama_pro=get("train.use_llama_pro"),
+ shift_attn=get("train.shift_attn"),
+ report_to="all" if get("train.report_to") else "none",
+ use_galore=get("train.use_galore"),
+ use_badam=get("train.use_badam"),
+ output_dir=get_save_dir(model_name, finetuning_type, get("train.output_dir")),
+ fp16=(get("train.compute_type") == "fp16"),
+ bf16=(get("train.compute_type") == "bf16"),
+ pure_bf16=(get("train.compute_type") == "pure_bf16"),
+ plot_loss=True,
+ ddp_timeout=180000000,
+ include_num_input_tokens_seen=True,
+ )
+
+ # checkpoints
+ if get("top.checkpoint_path"):
+ if finetuning_type in PEFT_METHODS: # list
+ args["adapter_name_or_path"] = ",".join(
+ [get_save_dir(model_name, finetuning_type, adapter) for adapter in get("top.checkpoint_path")]
+ )
+ else: # str
+ args["model_name_or_path"] = get_save_dir(model_name, finetuning_type, get("top.checkpoint_path"))
+
+ # freeze config
+ if args["finetuning_type"] == "freeze":
+ args["freeze_trainable_layers"] = get("train.freeze_trainable_layers")
+ args["freeze_trainable_modules"] = get("train.freeze_trainable_modules")
+ args["freeze_extra_modules"] = get("train.freeze_extra_modules") or None
+
+ # lora config
+ if args["finetuning_type"] == "lora":
+ args["lora_rank"] = get("train.lora_rank")
+ args["lora_alpha"] = get("train.lora_alpha")
+ args["lora_dropout"] = get("train.lora_dropout")
+ args["loraplus_lr_ratio"] = get("train.loraplus_lr_ratio") or None
+ args["create_new_adapter"] = get("train.create_new_adapter")
+ args["use_rslora"] = get("train.use_rslora")
+ args["use_dora"] = get("train.use_dora")
+ args["pissa_init"] = get("train.use_pissa")
+ args["pissa_convert"] = get("train.use_pissa")
+ args["lora_target"] = get("train.lora_target") or "all"
+ args["additional_target"] = get("train.additional_target") or None
+
+ if args["use_llama_pro"]:
+ args["freeze_trainable_layers"] = get("train.freeze_trainable_layers")
+
+ # rlhf config
+ if args["stage"] == "ppo":
+ if finetuning_type in PEFT_METHODS:
+ args["reward_model"] = ",".join(
+ [get_save_dir(model_name, finetuning_type, adapter) for adapter in get("train.reward_model")]
+ )
+ else:
+ args["reward_model"] = get_save_dir(model_name, finetuning_type, get("train.reward_model"))
+
+ args["reward_model_type"] = "lora" if finetuning_type == "lora" else "full"
+ args["ppo_score_norm"] = get("train.ppo_score_norm")
+ args["ppo_whiten_rewards"] = get("train.ppo_whiten_rewards")
+ args["top_k"] = 0
+ args["top_p"] = 0.9
+ elif args["stage"] in ["dpo", "kto"]:
+ args["pref_beta"] = get("train.pref_beta")
+ args["pref_ftx"] = get("train.pref_ftx")
+ args["pref_loss"] = get("train.pref_loss")
+
+ # galore config
+ if args["use_galore"]:
+ args["galore_rank"] = get("train.galore_rank")
+ args["galore_update_interval"] = get("train.galore_update_interval")
+ args["galore_scale"] = get("train.galore_scale")
+ args["galore_target"] = get("train.galore_target")
+
+ # badam config
+ if args["use_badam"]:
+ args["badam_mode"] = get("train.badam_mode")
+ args["badam_switch_mode"] = get("train.badam_switch_mode")
+ args["badam_switch_interval"] = get("train.badam_switch_interval")
+ args["badam_update_ratio"] = get("train.badam_update_ratio")
+
+ # eval config
+ if get("train.val_size") > 1e-6 and args["stage"] != "ppo":
+ args["val_size"] = get("train.val_size")
+ args["eval_strategy"] = "steps"
+ args["eval_steps"] = args["save_steps"]
+ args["per_device_eval_batch_size"] = args["per_device_train_batch_size"]
+
+ # ds config
+ if get("train.ds_stage") != "none":
+ ds_stage = get("train.ds_stage")
+ ds_offload = "offload_" if get("train.ds_offload") else ""
+ args["deepspeed"] = os.path.join(DEFAULT_CACHE_DIR, "ds_z{}_{}config.json".format(ds_stage, ds_offload))
+
+ return args
+
+ def _parse_eval_args(self, data: Dict["Component", Any]) -> Dict[str, Any]:
+ get = lambda elem_id: data[self.manager.get_elem_by_id(elem_id)]
+ model_name, finetuning_type = get("top.model_name"), get("top.finetuning_type")
+ user_config = load_config()
+
+ if get("top.quantization_bit") in QUANTIZATION_BITS:
+ quantization_bit = int(get("top.quantization_bit"))
+ else:
+ quantization_bit = None
+
+ args = dict(
+ stage="sft",
+ model_name_or_path=get("top.model_path"),
+ cache_dir=user_config.get("cache_dir", None),
+ preprocessing_num_workers=16,
+ finetuning_type=finetuning_type,
+ quantization_bit=quantization_bit,
+ quantization_method=get("top.quantization_method"),
+ template=get("top.template"),
+ rope_scaling=get("top.rope_scaling") if get("top.rope_scaling") in ["linear", "dynamic"] else None,
+ flash_attn="fa2" if get("top.booster") == "flashattn2" else "auto",
+ use_unsloth=(get("top.booster") == "unsloth"),
+ visual_inputs=get("top.visual_inputs"),
+ dataset_dir=get("eval.dataset_dir"),
+ eval_dataset=",".join(get("eval.dataset")),
+ cutoff_len=get("eval.cutoff_len"),
+ max_samples=int(get("eval.max_samples")),
+ per_device_eval_batch_size=get("eval.batch_size"),
+ predict_with_generate=True,
+ max_new_tokens=get("eval.max_new_tokens"),
+ top_p=get("eval.top_p"),
+ temperature=get("eval.temperature"),
+ output_dir=get_save_dir(model_name, finetuning_type, get("eval.output_dir")),
+ )
+
+ if get("eval.predict"):
+ args["do_predict"] = True
+ else:
+ args["do_eval"] = True
+
+ if get("top.checkpoint_path"):
+ if finetuning_type in PEFT_METHODS: # list
+ args["adapter_name_or_path"] = ",".join(
+ [get_save_dir(model_name, finetuning_type, adapter) for adapter in get("top.checkpoint_path")]
+ )
+ else: # str
+ args["model_name_or_path"] = get_save_dir(model_name, finetuning_type, get("top.checkpoint_path"))
+
+ return args
+
+ def _preview(self, data: Dict["Component", Any], do_train: bool) -> Generator[Dict["Component", str], None, None]:
+ output_box = self.manager.get_elem_by_id("{}.output_box".format("train" if do_train else "eval"))
+ error = self._initialize(data, do_train, from_preview=True)
+ if error:
+ gr.Warning(error)
+ yield {output_box: error}
+ else:
+ args = self._parse_train_args(data) if do_train else self._parse_eval_args(data)
+ yield {output_box: gen_cmd(args)}
+
+ def _launch(self, data: Dict["Component", Any], do_train: bool) -> Generator[Dict["Component", Any], None, None]:
+ output_box = self.manager.get_elem_by_id("{}.output_box".format("train" if do_train else "eval"))
+ error = self._initialize(data, do_train, from_preview=False)
+ if error:
+ gr.Warning(error)
+ yield {output_box: error}
+ else:
+ self.do_train, self.running_data = do_train, data
+ args = self._parse_train_args(data) if do_train else self._parse_eval_args(data)
+
+ os.makedirs(args["output_dir"], exist_ok=True)
+ save_args(os.path.join(args["output_dir"], LLAMABOARD_CONFIG), self._form_config_dict(data))
+
+ env = deepcopy(os.environ)
+ env["LLAMABOARD_ENABLED"] = "1"
+ env["LLAMABOARD_WORKDIR"] = args["output_dir"]
+ if args.get("deepspeed", None) is not None:
+ env["FORCE_TORCHRUN"] = "1"
+
+ self.trainer = Popen("llamafactory-cli train {}".format(save_cmd(args)), env=env, shell=True)
+ yield from self.monitor()
+
+ def _form_config_dict(self, data: Dict["Component", Any]) -> Dict[str, Any]:
+ config_dict = {}
+ skip_ids = ["top.lang", "top.model_path", "train.output_dir", "train.config_path"]
+ for elem, value in data.items():
+ elem_id = self.manager.get_id_by_elem(elem)
+ if elem_id not in skip_ids:
+ config_dict[elem_id] = value
+
+ return config_dict
+
+ def preview_train(self, data):
+ yield from self._preview(data, do_train=True)
+
+ def preview_eval(self, data):
+ yield from self._preview(data, do_train=False)
+
+ def run_train(self, data):
+ yield from self._launch(data, do_train=True)
+
+ def run_eval(self, data):
+ yield from self._launch(data, do_train=False)
+
+ def monitor(self):
+ self.aborted = False
+ self.running = True
+
+ get = lambda elem_id: self.running_data[self.manager.get_elem_by_id(elem_id)]
+ lang, model_name, finetuning_type = get("top.lang"), get("top.model_name"), get("top.finetuning_type")
+ output_dir = get("{}.output_dir".format("train" if self.do_train else "eval"))
+ output_path = get_save_dir(model_name, finetuning_type, output_dir)
+
+ output_box = self.manager.get_elem_by_id("{}.output_box".format("train" if self.do_train else "eval"))
+ progress_bar = self.manager.get_elem_by_id("{}.progress_bar".format("train" if self.do_train else "eval"))
+ loss_viewer = self.manager.get_elem_by_id("train.loss_viewer") if self.do_train else None
+
+ while self.trainer is not None:
+ if self.aborted:
+ yield {
+ output_box: ALERTS["info_aborting"][lang],
+ progress_bar: gr.Slider(visible=False),
+ }
+ else:
+ running_log, running_progress, running_loss = get_trainer_info(output_path, self.do_train)
+ return_dict = {
+ output_box: running_log,
+ progress_bar: running_progress,
+ }
+ if running_loss is not None:
+ return_dict[loss_viewer] = running_loss
+
+ yield return_dict
+
+ try:
+ self.trainer.wait(2)
+ self.trainer = None
+ except TimeoutExpired:
+ continue
+
+ if self.do_train:
+ if os.path.exists(os.path.join(output_path, TRAINING_ARGS_NAME)):
+ finish_info = ALERTS["info_finished"][lang]
+ else:
+ finish_info = ALERTS["err_failed"][lang]
+ else:
+ if os.path.exists(os.path.join(output_path, "all_results.json")):
+ finish_info = get_eval_results(os.path.join(output_path, "all_results.json"))
+ else:
+ finish_info = ALERTS["err_failed"][lang]
+
+ return_dict = {
+ output_box: self._finalize(lang, finish_info),
+ progress_bar: gr.Slider(visible=False),
+ }
+ yield return_dict
+
+ def save_args(self, data):
+ output_box = self.manager.get_elem_by_id("train.output_box")
+ error = self._initialize(data, do_train=True, from_preview=True)
+ if error:
+ gr.Warning(error)
+ return {output_box: error}
+
+ lang = data[self.manager.get_elem_by_id("top.lang")]
+ config_path = data[self.manager.get_elem_by_id("train.config_path")]
+ os.makedirs(DEFAULT_CONFIG_DIR, exist_ok=True)
+ save_path = os.path.join(DEFAULT_CONFIG_DIR, config_path)
+
+ save_args(save_path, self._form_config_dict(data))
+ return {output_box: ALERTS["info_config_saved"][lang] + save_path}
+
+ def load_args(self, lang: str, config_path: str):
+ output_box = self.manager.get_elem_by_id("train.output_box")
+ config_dict = load_args(os.path.join(DEFAULT_CONFIG_DIR, config_path))
+ if config_dict is None:
+ gr.Warning(ALERTS["err_config_not_found"][lang])
+ return {output_box: ALERTS["err_config_not_found"][lang]}
+
+ output_dict: Dict["Component", Any] = {output_box: ALERTS["info_config_loaded"][lang]}
+ for elem_id, value in config_dict.items():
+ output_dict[self.manager.get_elem_by_id(elem_id)] = value
+
+ return output_dict
+
+ def check_output_dir(self, lang: str, model_name: str, finetuning_type: str, output_dir: str):
+ output_box = self.manager.get_elem_by_id("train.output_box")
+ output_dict: Dict["Component", Any] = {output_box: LOCALES["output_box"][lang]["value"]}
+ if model_name and output_dir and os.path.isdir(get_save_dir(model_name, finetuning_type, output_dir)):
+ gr.Warning(ALERTS["warn_output_dir_exists"][lang])
+ output_dict[output_box] = ALERTS["warn_output_dir_exists"][lang]
+
+ output_dir = get_save_dir(model_name, finetuning_type, output_dir)
+ config_dict = load_args(os.path.join(output_dir, LLAMABOARD_CONFIG)) # load llamaboard config
+ for elem_id, value in config_dict.items():
+ output_dict[self.manager.get_elem_by_id(elem_id)] = value
+
+ return output_dict
diff --git a/llama-factory/src/llamafactory/webui/utils.py b/llama-factory/src/llamafactory/webui/utils.py
new file mode 100644
index 0000000000000000000000000000000000000000..c52c08877fc120d287f302836a43fd69d1c76245
--- /dev/null
+++ b/llama-factory/src/llamafactory/webui/utils.py
@@ -0,0 +1,299 @@
+# Copyright 2024 the LlamaFactory team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import json
+import os
+import signal
+from datetime import datetime
+from typing import Any, Dict, List, Optional, Tuple
+
+import psutil
+from transformers.trainer_utils import get_last_checkpoint
+from yaml import safe_dump, safe_load
+
+from ..extras.constants import PEFT_METHODS, RUNNING_LOG, TRAINER_LOG, TRAINING_ARGS, TRAINING_STAGES
+from ..extras.packages import is_gradio_available, is_matplotlib_available
+from ..extras.ploting import gen_loss_plot
+from ..model import QuantizationMethod
+from .common import DEFAULT_CACHE_DIR, DEFAULT_CONFIG_DIR, get_save_dir
+from .locales import ALERTS
+
+
+if is_gradio_available():
+ import gradio as gr
+
+
+def abort_process(pid: int) -> None:
+ r"""
+ Aborts the processes recursively in a bottom-up way.
+ """
+ try:
+ children = psutil.Process(pid).children()
+ if children:
+ for child in children:
+ abort_process(child.pid)
+
+ os.kill(pid, signal.SIGABRT)
+ except Exception:
+ pass
+
+
+def can_quantize(finetuning_type: str) -> "gr.Dropdown":
+ r"""
+ Judges if the quantization is available in this finetuning type.
+ """
+ if finetuning_type not in PEFT_METHODS:
+ return gr.Dropdown(value="none", interactive=False)
+ else:
+ return gr.Dropdown(interactive=True)
+
+
+def can_quantize_to(quantization_method: str) -> "gr.Dropdown":
+ r"""
+ Returns the available quantization bits.
+ """
+ if quantization_method == QuantizationMethod.BITS_AND_BYTES.value:
+ available_bits = ["none", "8", "4"]
+ elif quantization_method == QuantizationMethod.HQQ.value:
+ available_bits = ["none", "8", "6", "5", "4", "3", "2", "1"]
+ elif quantization_method == QuantizationMethod.EETQ.value:
+ available_bits = ["none", "8"]
+
+ return gr.Dropdown(choices=available_bits)
+
+
+def change_stage(training_stage: str = list(TRAINING_STAGES.keys())[0]) -> Tuple[List[str], bool]:
+ r"""
+ Modifys states after changing the training stage.
+ """
+ return [], TRAINING_STAGES[training_stage] == "pt"
+
+
+def check_json_schema(text: str, lang: str) -> None:
+ r"""
+ Checks if the json schema is valid.
+ """
+ try:
+ tools = json.loads(text)
+ if tools:
+ assert isinstance(tools, list)
+ for tool in tools:
+ if "name" not in tool:
+ raise NotImplementedError("Name not found.")
+ except NotImplementedError:
+ gr.Warning(ALERTS["err_tool_name"][lang])
+ except Exception:
+ gr.Warning(ALERTS["err_json_schema"][lang])
+
+
+def clean_cmd(args: Dict[str, Any]) -> Dict[str, Any]:
+ r"""
+ Removes args with NoneType or False or empty string value.
+ """
+ no_skip_keys = ["packing"]
+ return {k: v for k, v in args.items() if (k in no_skip_keys) or (v is not None and v is not False and v != "")}
+
+
+def gen_cmd(args: Dict[str, Any]) -> str:
+ r"""
+ Generates arguments for previewing.
+ """
+ cmd_lines = ["llamafactory-cli train "]
+ for k, v in clean_cmd(args).items():
+ cmd_lines.append(" --{} {} ".format(k, str(v)))
+
+ if os.name == "nt":
+ cmd_text = "`\n".join(cmd_lines)
+ else:
+ cmd_text = "\\\n".join(cmd_lines)
+
+ cmd_text = "```bash\n{}\n```".format(cmd_text)
+ return cmd_text
+
+
+def save_cmd(args: Dict[str, Any]) -> str:
+ r"""
+ Saves arguments to launch training.
+ """
+ output_dir = args["output_dir"]
+ os.makedirs(output_dir, exist_ok=True)
+
+ with open(os.path.join(output_dir, TRAINING_ARGS), "w", encoding="utf-8") as f:
+ safe_dump(clean_cmd(args), f)
+
+ return os.path.join(output_dir, TRAINING_ARGS)
+
+
+def get_eval_results(path: os.PathLike) -> str:
+ r"""
+ Gets scores after evaluation.
+ """
+ with open(path, "r", encoding="utf-8") as f:
+ result = json.dumps(json.load(f), indent=4)
+ return "```json\n{}\n```\n".format(result)
+
+
+def get_time() -> str:
+ r"""
+ Gets current date and time.
+ """
+ return datetime.now().strftime(r"%Y-%m-%d-%H-%M-%S")
+
+
+def get_trainer_info(output_path: os.PathLike, do_train: bool) -> Tuple[str, "gr.Slider", Optional["gr.Plot"]]:
+ r"""
+ Gets training infomation for monitor.
+ """
+ running_log = ""
+ running_progress = gr.Slider(visible=False)
+ running_loss = None
+
+ running_log_path = os.path.join(output_path, RUNNING_LOG)
+ if os.path.isfile(running_log_path):
+ with open(running_log_path, "r", encoding="utf-8") as f:
+ running_log = f.read()
+
+ trainer_log_path = os.path.join(output_path, TRAINER_LOG)
+ if os.path.isfile(trainer_log_path):
+ trainer_log: List[Dict[str, Any]] = []
+ with open(trainer_log_path, "r", encoding="utf-8") as f:
+ for line in f:
+ trainer_log.append(json.loads(line))
+
+ if len(trainer_log) != 0:
+ latest_log = trainer_log[-1]
+ percentage = latest_log["percentage"]
+ label = "Running {:d}/{:d}: {} < {}".format(
+ latest_log["current_steps"],
+ latest_log["total_steps"],
+ latest_log["elapsed_time"],
+ latest_log["remaining_time"],
+ )
+ running_progress = gr.Slider(label=label, value=percentage, visible=True)
+
+ if do_train and is_matplotlib_available():
+ running_loss = gr.Plot(gen_loss_plot(trainer_log))
+
+ return running_log, running_progress, running_loss
+
+
+def load_args(config_path: str) -> Optional[Dict[str, Any]]:
+ r"""
+ Loads saved arguments.
+ """
+ try:
+ with open(config_path, "r", encoding="utf-8") as f:
+ return safe_load(f)
+ except Exception:
+ return None
+
+
+def save_args(config_path: str, config_dict: Dict[str, Any]):
+ r"""
+ Saves arguments.
+ """
+ with open(config_path, "w", encoding="utf-8") as f:
+ safe_dump(config_dict, f)
+
+
+def list_config_paths(current_time: str) -> "gr.Dropdown":
+ r"""
+ Lists all the saved configuration files.
+ """
+ config_files = ["{}.yaml".format(current_time)]
+ if os.path.isdir(DEFAULT_CONFIG_DIR):
+ for file_name in os.listdir(DEFAULT_CONFIG_DIR):
+ if file_name.endswith(".yaml") and file_name not in config_files:
+ config_files.append(file_name)
+
+ return gr.Dropdown(choices=config_files)
+
+
+def list_output_dirs(model_name: Optional[str], finetuning_type: str, current_time: str) -> "gr.Dropdown":
+ r"""
+ Lists all the directories that can resume from.
+ """
+ output_dirs = ["train_{}".format(current_time)]
+ if model_name:
+ save_dir = get_save_dir(model_name, finetuning_type)
+ if save_dir and os.path.isdir(save_dir):
+ for folder in os.listdir(save_dir):
+ output_dir = os.path.join(save_dir, folder)
+ if os.path.isdir(output_dir) and get_last_checkpoint(output_dir) is not None:
+ output_dirs.append(folder)
+
+ return gr.Dropdown(choices=output_dirs)
+
+
+def create_ds_config() -> None:
+ r"""
+ Creates deepspeed config.
+ """
+ os.makedirs(DEFAULT_CACHE_DIR, exist_ok=True)
+ ds_config = {
+ "train_batch_size": "auto",
+ "train_micro_batch_size_per_gpu": "auto",
+ "gradient_accumulation_steps": "auto",
+ "gradient_clipping": "auto",
+ "zero_allow_untested_optimizer": True,
+ "fp16": {
+ "enabled": "auto",
+ "loss_scale": 0,
+ "loss_scale_window": 1000,
+ "initial_scale_power": 16,
+ "hysteresis": 2,
+ "min_loss_scale": 1,
+ },
+ "bf16": {"enabled": "auto"},
+ }
+ offload_config = {
+ "device": "cpu",
+ "pin_memory": True,
+ }
+ ds_config["zero_optimization"] = {
+ "stage": 2,
+ "allgather_partitions": True,
+ "allgather_bucket_size": 5e8,
+ "overlap_comm": True,
+ "reduce_scatter": True,
+ "reduce_bucket_size": 5e8,
+ "contiguous_gradients": True,
+ "round_robin_gradients": True,
+ }
+ with open(os.path.join(DEFAULT_CACHE_DIR, "ds_z2_config.json"), "w", encoding="utf-8") as f:
+ json.dump(ds_config, f, indent=2)
+
+ ds_config["zero_optimization"]["offload_optimizer"] = offload_config
+ with open(os.path.join(DEFAULT_CACHE_DIR, "ds_z2_offload_config.json"), "w", encoding="utf-8") as f:
+ json.dump(ds_config, f, indent=2)
+
+ ds_config["zero_optimization"] = {
+ "stage": 3,
+ "overlap_comm": True,
+ "contiguous_gradients": True,
+ "sub_group_size": 1e9,
+ "reduce_bucket_size": "auto",
+ "stage3_prefetch_bucket_size": "auto",
+ "stage3_param_persistence_threshold": "auto",
+ "stage3_max_live_parameters": 1e9,
+ "stage3_max_reuse_distance": 1e9,
+ "stage3_gather_16bit_weights_on_model_save": True,
+ }
+ with open(os.path.join(DEFAULT_CACHE_DIR, "ds_z3_config.json"), "w", encoding="utf-8") as f:
+ json.dump(ds_config, f, indent=2)
+
+ ds_config["zero_optimization"]["offload_optimizer"] = offload_config
+ ds_config["zero_optimization"]["offload_param"] = offload_config
+ with open(os.path.join(DEFAULT_CACHE_DIR, "ds_z3_offload_config.json"), "w", encoding="utf-8") as f:
+ json.dump(ds_config, f, indent=2)
diff --git a/llama-factory/src/train.py b/llama-factory/src/train.py
new file mode 100644
index 0000000000000000000000000000000000000000..6703ffdb00a2c6b9a4c67edc12b2dd6e2d5d76f2
--- /dev/null
+++ b/llama-factory/src/train.py
@@ -0,0 +1,28 @@
+# Copyright 2024 the LlamaFactory team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from llamafactory.train.tuner import run_exp
+
+
+def main():
+ run_exp()
+
+
+def _mp_fn(index):
+ # For xla_spawn (TPUs)
+ run_exp()
+
+
+if __name__ == "__main__":
+ main()
diff --git a/llama-factory/src/webui.py b/llama-factory/src/webui.py
new file mode 100644
index 0000000000000000000000000000000000000000..99370af2f0d39cc6df946e05420cc0c30b36bef1
--- /dev/null
+++ b/llama-factory/src/webui.py
@@ -0,0 +1,27 @@
+# Copyright 2024 the LlamaFactory team.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+
+from llamafactory.webui.interface import create_ui
+
+
+def main():
+ gradio_share = os.environ.get("GRADIO_SHARE", "0").lower() in ["true", "1"]
+ server_name = os.environ.get("GRADIO_SERVER_NAME", "0.0.0.0")
+ create_ui().queue().launch(share=gradio_share, server_name=server_name, inbrowser=True)
+
+
+if __name__ == "__main__":
+ main()