| This is [AIDC-ai-business/Marcoroni-70B-v1](https://huggingface.co/AIDC-ai-business/Marcoroni-70B-v1) quantized to 4bit AWQ with the following config: | |
| ```bash | |
| python3 -m lmdeploy.lite.apis.auto_awq \ | |
| --model ./Marcoroni-70B-v1 \ | |
| --w_bits 4 \ | |
| --w_group_size 32 \ | |
| --work_dir ./quant | |
| ``` | |
| # Original Model Card: | |
| # Marcoroni-70B | |
| # Model Details | |
| * **Trained by**: trained by AIDC AI-Business. | |
| * **Model type:** **Marcoroni-70B** is an auto-regressive language model based on the Llama 2 transformer architecture. | |
| * **Language(s)**: English | |
| * **License for Marcoroni-70B base weights**: Non-Commercial Creative Commons license ([CC BY-NC-4.0](https://creativecommons.org/licenses/by-nc/4.0/)) | |
| # Prompting | |
| ## Prompt Template for alpaca style | |
| ``` | |
| ### Instruction: | |
| <prompt> (without the <>) | |
| ### Response: | |
| ``` |