|
--- |
|
license: gpl-3.0 |
|
tags: |
|
- text2text-generation |
|
pipeline_tag: text2text-generation |
|
language: |
|
- zh |
|
- en |
|
--- |
|
|
|
Considering LLaMA's license constraints, the model is for research and learning only. |
|
Please strictly respect LLaMA's usage policy. |
|
We are not allowed to publish weights for LLaMA, of course, even finetuned, but there is no problem publishing the difference, a patch that we suggest to apply to the files. |
|
The encryption is a simple XOR between files, ensuring that only the people that have access to the original weights (from completely legal sources, of course) can transform them into finetuned weights. |
|
You can find the decrypt code on https://github.com/LianjiaTech/BELLE/tree/main/models . |
|
|
|
|
|
# GPTQ-for-LLaMa |
|
|
|
## Welcome |
|
If you find this model helpful, please *like* this model and star us on https://github.com/LianjiaTech/BELLE ! |
|
|
|
## Model description |
|
8 bits quantization of [BELLE-LLAMA-7B-2M](https://huggingface.co/BelleGroup/BELLE-LLAMA-7B-2M-enc) using [GPTQ](https://arxiv.org/abs/2210.17323) |
|
|
|
GPTQ is SOTA one-shot weight quantization method. |
|
|
|
The code of inference can be found in our Github project repository: https://github.com/LianjiaTech/BELLE/tree/main/gptq. |
|
|
|
Basically, 8-bit quantization and 128 groupsize are recommended. |
|
|
|
**This code is based on [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa)** |
|
|
|
## Model list |
|
|
|
| model name | file size | GPU memory usage | |
|
| -------------------------------------------------- | ------------------- | ------------------ | |
|
| llama7b-2m | 26G | ~15G | |
|
| llama7b-2m-8bit-128g.pt | 6.8G | ~8.9G | |
|
| llama7b-2m-4bit-128g.pt | 3.8G | ~5.6G | |
|
|
|
## Check md5 |
|
1. After you git clone this model |
|
``` |
|
md5sum ./* |
|
340aa9ee27fa7931ccbabcc30f2f8a27 ./config.json.db303d8f096e427bd21ff97bb169c84fb3ae11336a644e3da3506419d44f6429.enc |
|
f9b33d359f17a437f6c24b4de6f2272e ./generation_config.json.fd7ff399e5568cc21a0a8414f43df88ef7c424995b9b97a90563165d2cf79efd.enc |
|
591a2ecabc03530ba70663784fddb0e5 ./llama7b-2m-4bit-128g.pt.8576bae21290e9e75a60f38a6010709255656b19330a0df9a4bf50e1ee83fc51.enc |
|
65926fddcd56be59b0bebf97f1518106 ./llama7b-2m-8bit-128g.pt.44227e0ee3633967c555ed9ba7a89f340955545f6e32f7d5dfdc28603f6e27d2.enc |
|
1ab707fa9b0c4be294fd0b867d73e919 ./special_tokens_map.json.44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a.enc |
|
ff291fcfa4e0048ca4ff262312faad83 ./tokenizer_config.json.ef7ef410b9b909949e96f172b17cbf7c68b11761c632715fa05a6088c0c2b9ac.enc |
|
39ec1b33fbf9a0934a8ae0f9a24c7163 ./tokenizer.model.9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347.enc |
|
``` |
|
|
|
2. Decrypt the files using the scripts in https://github.com/LianjiaTech/BELLE/tree/main/models |
|
|
|
You can use the following command in Bash. |
|
Please replace "/path/to_encrypted" with the path where you stored your encrypted file, |
|
replace "/path/to_original_llama_7B" with the path where you stored your original llama7B file, |
|
and replace "/path/to_finetuned_model" with the path where you want to save your final trained model. |
|
|
|
```bash |
|
mkdir /path/to_finetuned_model |
|
for f in "/path/to_encrypted"/*; \ |
|
do if [ -f "$f" ]; then \ |
|
python3 decrypt.py "$f" "/path/to_original_llama_7B/consolidated.00.pth" "/path/to_finetuned_model/"; \ |
|
fi; \ |
|
done |
|
``` |
|
|
|
After executing the aforementioned command, you will obtain the following files. |
|
|
|
``` |
|
./config.json |
|
./generation_config.json |
|
./llama7b-2m-4bit-128g.pt |
|
./llama7b-2m-8bit-128g.pt |
|
./special_tokens_map.json |
|
./tokenizer_config.json |
|
./tokenizer.model |
|
``` |
|
|
|
3. Check md5sum |
|
|
|
You can verify the integrity of these files by performing an MD5 checksum to ensure their complete recovery. |
|
Here are the MD5 checksums for the relevant files: |
|
``` |
|
md5sum ./* |
|
32490e7229fb82c643e3a7b8d04a6c4b ./config.json |
|
2917a1cafb895cf57e746cfd7696bfe5 ./generation_config.json |
|
856cb1e00b6837f71b8d77f8b44ee5a5 ./llama7b-2m-4bit-128g.pt |
|
a35a44e6ff57e672f649635cf966f5bd ./llama7b-2m-8bit-128g.pt |
|
99914b932bd37a50b983c5e7c90ae93b ./special_tokens_map.json |
|
5526ad31f4928acb5219e295e5ff81ce ./tokenizer_config.json |
|
eeec4125e9c7560836b4873b6f8e3025 ./tokenizer.model |
|
``` |
|
|
|
## Limitations |
|
There still exists a few issues in the model trained on current base model and data: |
|
|
|
1. The model might generate factual errors when asked to follow instructions related to facts. |
|
|
|
2. Occasionally generates harmful responses since the model still struggles to identify potential harmful instructions. |
|
|
|
3. Needs improvements on reasoning and coding. |
|
|
|
Since the model still has its limitations, we require developers only use the open-sourced code, data, model and any other artifacts generated via this project for research purposes. Commercial use and other potential harmful use cases are not allowed. |
|
|
|
## Citation |
|
|
|
Please cite us when using our code, data or model. |
|
|
|
``` |
|
@misc{BELLE, |
|
author = {Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Baochang Ma, Xiangang Li}, |
|
title = {BELLE: Bloom-Enhanced Large Language model Engine }, |
|
year = {2023}, |
|
publisher = {GitHub}, |
|
journal = {GitHub repository}, |
|
howpublished = {\url{https://github.com/LianjiaTech/BELLE}}, |
|
} |
|
``` |
|
|
|
Cite the original LLaMa, Stanford Alpaca and Self-Instruct papers as well! |
