Spaces:
Paused
Paused
Commit
·
850e3c7
1
Parent(s):
94498e7
update README
Browse files
README.md
CHANGED
@@ -12,7 +12,7 @@ Welcome to WhisperBot. WhisperBot builds upon the capabilities of the [WhisperLi
|
|
12 |
Install [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/installation.md) to build Whisper and Mistral TensorRT engines. The README builds a docker image for TensorRT-LLM.
|
13 |
Instead of building a docker image, we can also refer to the README and the [Dockerfile.multi](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docker/Dockerfile.multi) to install the required packages in the base pytroch docker image. Just make sure to use the correct base image as mentioned in the dockerfile and everything should go nice and smooth.
|
14 |
|
15 |
-
### Whisper
|
16 |
- Change working dir to the [whisper example dir](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/whisper) in TensorRT-LLM.
|
17 |
```bash
|
18 |
cd TensorRT-LLM/examples/whisper
|
@@ -32,7 +32,7 @@ pip install -r requirements.txt
|
|
32 |
python3 build.py --output_dir whisper_small_en --use_gpt_attention_plugin --use_gemm_plugin --use_layernorm_plugin --use_bert_attention_plugin --model_name small.en
|
33 |
```
|
34 |
|
35 |
-
### Mistral
|
36 |
- Change working dir to [llama example dir](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/llama) in TensorRT-LLM folder.
|
37 |
```bash
|
38 |
cd TensorRT-LLM/examples/llama
|
@@ -66,7 +66,7 @@ python3 main.py --whisper_tensorrt_path /root/TensorRT-LLM/examples/whisper/whis
|
|
66 |
--mistral_tensorrt_path /root/TensorRT-LLM/examples/llama/tmp/mistral/7B/trt_engines/fp16/1-gpu/ \
|
67 |
--mistral_tokenizer_path teknium/OpenHermes-2.5-Mistral-7B
|
68 |
```
|
69 |
-
- Use the `WhisperBot/client.py` script to run on the client
|
70 |
|
71 |
|
72 |
## Contact Us
|
|
|
12 |
Install [TensorRT-LLM](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docs/source/installation.md) to build Whisper and Mistral TensorRT engines. The README builds a docker image for TensorRT-LLM.
|
13 |
Instead of building a docker image, we can also refer to the README and the [Dockerfile.multi](https://github.com/NVIDIA/TensorRT-LLM/blob/main/docker/Dockerfile.multi) to install the required packages in the base pytroch docker image. Just make sure to use the correct base image as mentioned in the dockerfile and everything should go nice and smooth.
|
14 |
|
15 |
+
### Build Whisper TensorRT Engine
|
16 |
- Change working dir to the [whisper example dir](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/whisper) in TensorRT-LLM.
|
17 |
```bash
|
18 |
cd TensorRT-LLM/examples/whisper
|
|
|
32 |
python3 build.py --output_dir whisper_small_en --use_gpt_attention_plugin --use_gemm_plugin --use_layernorm_plugin --use_bert_attention_plugin --model_name small.en
|
33 |
```
|
34 |
|
35 |
+
### Build Mistral TensorRT Engine
|
36 |
- Change working dir to [llama example dir](https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/llama) in TensorRT-LLM folder.
|
37 |
```bash
|
38 |
cd TensorRT-LLM/examples/llama
|
|
|
66 |
--mistral_tensorrt_path /root/TensorRT-LLM/examples/llama/tmp/mistral/7B/trt_engines/fp16/1-gpu/ \
|
67 |
--mistral_tokenizer_path teknium/OpenHermes-2.5-Mistral-7B
|
68 |
```
|
69 |
+
- Use the `WhisperBot/client.py` script to run on the client side.
|
70 |
|
71 |
|
72 |
## Contact Us
|