|
--- |
|
tags: |
|
- lmdeploy |
|
- awq |
|
- marcoroni |
|
- 70b |
|
--- |
|
This is [AIDC-ai-business/Marcoroni-70B-v1](https://huggingface.co/AIDC-ai-business/Marcoroni-70B-v1) quantized to LMDeploy 4bit AWQ with the following config: |
|
```bash |
|
python3 -m lmdeploy.lite.apis.auto_awq \ |
|
--model ./Marcoroni-70B-v1 \ |
|
--w_bits 4 \ |
|
--w_group_size 128 \ |
|
--work_dir ./quant |
|
``` |
|
# Original Model Card: |
|
|
|
# Marcoroni-70B |
|
|
|
# Model Details |
|
* **Trained by**: trained by AIDC AI-Business. |
|
* **Model type:** **Marcoroni-70B** is an auto-regressive language model based on the Llama 2 transformer architecture. |
|
* **Language(s)**: English |
|
* **License for Marcoroni-70B base weights**: Non-Commercial Creative Commons license ([CC BY-NC-4.0](https://creativecommons.org/licenses/by-nc/4.0/)) |
|
|
|
|
|
# Prompting |
|
|
|
## Prompt Template for alpaca style |
|
|
|
``` |
|
### Instruction: |
|
|
|
<prompt> (without the <>) |
|
|
|
### Response: |
|
``` |
