neuralmagic
/

Meta-Llama-3.1-70B-Instruct-FP8

Text Generation

text-generation-inference

Inference Endpoints

compressed-tensors

Model card Files Files and versions Community

alexmarques commited on Jul 30

Commit

d42cb9d

•

1 Parent(s): df81133

Update README.md

Files changed (1) hide show

README.md +1 -0

README.md CHANGED Viewed

@@ -284,6 +284,7 @@ lm_eval \
   --model_args pretrained="neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=2 \
   --tasks gsm8k_cot_llama_3.1_instruct \
   --apply_chat_template \
   --num_fewshot 8 \
   --batch_size auto
 ```

   --model_args pretrained="neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=2 \
   --tasks gsm8k_cot_llama_3.1_instruct \
   --apply_chat_template \
+  --fewshot_as_multiturn \
   --num_fewshot 8 \
   --batch_size auto
 ```