ibm-fms
/

llama-13b-accelerator

Inference Endpoints

Model card Files Files and versions Community

JRosenkranz commited on May 1

Commit

cfc7388

•

1 Parent(s): 8e6cadf

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -143,7 +143,7 @@ python fms-extras/scripts/paged_speculative_inference.py \
     --speculator_path=ibm-fms/llama-13b-accelerator \
     --speculator_source=hf \
     --speculator_variant=840m \
-    --compile \
 ```
 ##### batch_size=4 (compile)
@@ -159,5 +159,5 @@ python fms-extras/scripts/paged_speculative_inference.py \
     --speculator_source=hf \
     --speculator_variant=840m \
     --batch_input \
-    --compile \
 ```

     --speculator_path=ibm-fms/llama-13b-accelerator \
     --speculator_source=hf \
     --speculator_variant=840m \
+    --compile
 ```
 ##### batch_size=4 (compile)
     --speculator_source=hf \
     --speculator_variant=840m \
     --batch_input \
+    --compile
 ```