---
license: apache-2.0
language:
- en
- zh
---
license: apache-2.0
---
# GreenBit Yi

This is GreenBitAI's pretrained **4-bit** Yi 34B model with extreme compression yet still strong performance.

Please refer to our [Github page](https://github.com/GreenBitAI/low_bit_llama/tree/low_bit_yi) for the code to run the model and more information.

## Model Description

- **Developed by:** [GreenBitAI](https://github.com/GreenBitAI)
- **Evaluated By:** 01-Yi official
- **Model type:** Causal (Llama 2/Yi 6B)
- **Language(s) (NLP):** English
- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0), [Llama 2 license agreement](https://ai.meta.com/resources/models-and-libraries/llama-downloads/)

## Few Shot Evaluation (officially evaluated by 01-Yi)
| Model          | Yi-34B FP16| [Yi-34B 4 bit](https://huggingface.co/GreenBitAI/yi-34b-w4a16g32) | Yi-6B FP16 | [Yi-6B 4 bit](https://huggingface.co/GreenBitAI/yi-6b-w4a16g32) |
|----------------|-----------|----------|----------|---------|
| GroupSize      | -         | 32       | -        | 8       |
| Model Size (GB)| 68.79     | 19.89    | 12.12    | 4.04    |
| AVG            | 70.64     | 69.7     | 60.11    | 59.14   |
| **Detailed Evaluation** | | | | |
| MMLU           | 76.32     | 75.42    | 63.24    | 62.09   |
| CMMLU          | 83.65     | 83.07    | 75.53    | 72.85   |
| ARC-e          | 84.42     | 84.13    | 77.23    | 76.52   |
| ARC-c          | 61.77     | 59.56    | 50.34    | 48.47   |
| GAOKAO         | 82.8      | 81.37    | 72.2     | 72.87   |
| GSM8K          | 67.24     | 63.61    | 32.52    | 28.05   |
| HumanEval      | 25.6      | 25       | 15.85    | 15.85   |
| BBH            | 54.3      | 52.3     | 42.8     | 41.47   |
| WinoGrande     | 78.68     | 78.53    | 70.63    | 71.19   |
| PIQA           | 82.86     | 82.75    | 78.56    | 79.05   |
| SIQA           | 74.46     | 73.44    | 64.53    | 64.53   |
| HellaSwag      | 83.64     | 83.02    | 74.91    | 73.27   |
| OBQA           | 91.6      | 90.8     | 85.4     | 82.6    |
| CSQA           | 83.37     | 83.05    | 76.9     | 75.43   |
| TriviaQA       | 81.52     | 80.73    | 64.85    | 61.75   |
| SquAD          | 92.46     | 91.12    | 88.95    | 88.39   |
| BoolQ          | 88.25     | 88.17    | 76.23    | 77.1    |
| MBPP           | 41        | 39.68    | 26.32    | 25.13   |
| QUAC           | 48.61     | 47.43    | 40.92    | 40.16   |
| Lambda         | 73.18     | 73.39    | 67.74    | 67.8    |
| NaturalQuestion| 27.67     | 27.21    | 16.69    | 17.42   |


# Zero Shot Evaluation
| Task          | Metric | Yi-6B FP16 | [Yi-6B 4 bit](https://huggingface.co/GreenBitAI/yi-6b-w4a16g32) | [Yi-34B 4 bit](https://huggingface.co/GreenBitAI/yi-34b-w4a16g32) |
|---------------|--------|---------|-------------|--------------|
| Openbookqa    | acc    | 0.314   | 0.324       | 0.344        |
|               | ac_norm| 0.408   | 0.42        | 0.474        |
| arc_challenge | acc    | 0.462   | 0.4573      | 0.569        |
|               | ac_norm| 0.504   | 0.483       | 0.5964       |
| hellawswag    | acc    | 0.553   | 0.5447      | 0.628        |
|               | ac_norm| 0.749   | 0.7327      | 0.83         |
| piqa          | acc    | 0.777   | 0.7709      | 0.8079       |
|               | ac_norm| 0.787   | 0.7894      | 0.828        |
| arc_easy      | acc    | 0.777   | 0.7697      | 0.835        |
|               | ac_norm| 0.774   | 0.7659      | 0.84         |
| Winogrande    | acc    | 0.707   | 0.7095      | 0.7853       |
| boolq         | acc    | 0.755   | 0.7648      | 0.886        |
| truthfulqa_mc | mc1    | 0.29    | 0.2729      | 0.4026       |
|               | mc2    | 0.419   | 0.4033      | 0.5528       |
| anli_r1       | acc    | 0.423   | 0.416       | 0.554        |
| anli_r2       | acc    | 0.409   | 0.409       | 0.518        |
| anli_r3       | acc    | 0.411   | 0.393       | 0.4983       |
| wic           | acc    | 0.529   | 0.545       | 0.5376       |
| rte           | acc    | 0.685   | 0.7039      | 0.7617       |
| record        | f1     | 0.904   | 0.9011      | 0.924        |
|               | em     | 0.8962  | 0.8927      | 0.916        |
| Average       |        | 0.596   | 0.5937      | 0.6708       |