Hugging Face's logo

OPEA
/
Meta-Llama-3.1-70B-Instruct-int4-asym-inc

Safetensors
llama
4-bit precision
intel/auto-round
Model card Files Community
cicdatopea commited on
Commit
7882b2d
•
1 Parent(s): e9042f5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -5,7 +5,7 @@ license: llama3.1
5
  ---
6
  ## Model Card Details
7
 
8
- This model is an int4 model with group_size 128 and asymmetric quantization of [meta-llama/Meta-Llama-3.1-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct) generated by [intel/auto-round](https://github.com/intel/auto-round), auto-round is needed to run this model
9
 
10
  ## Inference on CPU/HPU/CUDA
11
 
 
5
  ---
6
  ## Model Card Details
7
 
8
+ This model is an int4 model with group_size 128 and asymmetric quantization of [meta-llama/Meta-Llama-3.1-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct) generated by [intel/auto-round](https://github.com/intel/auto-round), auto-round is needed to run this model. [Symmetic model](https://huggingface.co/OPEA/Meta-Llama-3.1-70B-Instruct-int4-sym-inc) is recommended for better performance.
9
 
10
  ## Inference on CPU/HPU/CUDA
11