Update README.md
Browse files
README.md
CHANGED
@@ -782,7 +782,7 @@ Think carefully before responding, and be sure to include your reasoning when ap
|
|
782 |
3) Once inside the VM, open the terminal and run `conda activate text-generation-inference`
|
783 |
4) Then `cd Desktop/text-generation-inference/`
|
784 |
5) Run `volume=$PWD/data`
|
785 |
-
6) Run `model=jondurbin/bagel-20b-v04`
|
786 |
7) `sudo docker run --gpus '"device=0,1"' --shm-size 1g -p 8080:80 -v $volume:/data ghcr.io/huggingface/text-generation-inference:1.3 --model-id $model`
|
787 |
8) The model will take some time to load...
|
788 |
9) Once loaded the model will be available on port 8080
|
@@ -819,7 +819,7 @@ Be sure to set the following environment variables:
|
|
819 |
| PUBLIC_KEY | `{paste your ssh public key}` |
|
820 |
| UI_ARGS | `--trust-remote-code` |
|
821 |
|
822 |
-
Access the webui via `http://{container IP address}:7860`, navigate to model, download jondurbin/bagel-20b-v04, and ensure the following values are set:
|
823 |
|
824 |
- `use_flash_attention_2` should be checked
|
825 |
- set Model loader to Transformers
|
@@ -827,8 +827,6 @@ Access the webui via `http://{container IP address}:7860`, navigate to model, do
|
|
827 |
|
828 |
## Support me
|
829 |
|
830 |
-
https://bmc.link/jondurbin
|
831 |
-
|
832 |
-
|
833 |
-
|
834 |
-
BTC bc1qdwuth4vlg8x37ggntlxu5cjfwgmdy5zaa7pswf
|
|
|
782 |
3) Once inside the VM, open the terminal and run `conda activate text-generation-inference`
|
783 |
4) Then `cd Desktop/text-generation-inference/`
|
784 |
5) Run `volume=$PWD/data`
|
785 |
+
6) Run `model=jondurbin/bagel-dpo-20b-v04`
|
786 |
7) `sudo docker run --gpus '"device=0,1"' --shm-size 1g -p 8080:80 -v $volume:/data ghcr.io/huggingface/text-generation-inference:1.3 --model-id $model`
|
787 |
8) The model will take some time to load...
|
788 |
9) Once loaded the model will be available on port 8080
|
|
|
819 |
| PUBLIC_KEY | `{paste your ssh public key}` |
|
820 |
| UI_ARGS | `--trust-remote-code` |
|
821 |
|
822 |
+
Access the webui via `http://{container IP address}:7860`, navigate to model, download jondurbin/bagel-dpo-20b-v04, and ensure the following values are set:
|
823 |
|
824 |
- `use_flash_attention_2` should be checked
|
825 |
- set Model loader to Transformers
|
|
|
827 |
|
828 |
## Support me
|
829 |
|
830 |
+
- https://bmc.link/jondurbin
|
831 |
+
- ETH 0xce914eAFC2fe52FdceE59565Dd92c06f776fcb11
|
832 |
+
- BTC bc1qdwuth4vlg8x37ggntlxu5cjfwgmdy5zaa7pswf
|
|
|
|