joostinyi-baseten's picture
Update README.md
c06ec92

python3 build.py --model_dir ./weights/merged_mistral_lora/ --remove_input_padding --use_gpt_attention_plugin float16 --enable_context_fmha --use_gemm_plugin float16 --output_dir ./mistral_engines/fp16/instruct-lora-merged-1-gpu --max_batch_size 64 --use_inflight_batching --max_input_len 2000 --max_output_len 2000 --paged_kv_cache --world_size 1 --tp_size 1 https://huggingface.co/Doctor-Shotgun/mistral-v0.1-supercot-lora