Update README.md
Browse files
README.md
CHANGED
@@ -28,7 +28,7 @@ Each branch contains an individual bits per weight, with the main one containing
|
|
28 |
## Sample instructions to load in TabbyAPI @ 1.5bpw on 3x Nvidia Tesla P100 16gb at 4k context. ~14 tok/s
|
29 |
```JSON
|
30 |
{
|
31 |
-
"name": "
|
32 |
"max_seq_len": 4096,
|
33 |
"override_base_seq_len": 4096,
|
34 |
"gpu_split_auto": false,
|
@@ -55,7 +55,7 @@ Each branch contains an individual bits per weight, with the main one containing
|
|
55 |
## Sample instructions to load in TabbyAPI @ 2.75bpw on 4x Nvidia Tesla P100 16gb at 16k context. ~5.6 tok/s
|
56 |
```JSON
|
57 |
{
|
58 |
-
"name": "
|
59 |
"max_seq_len": 16384,
|
60 |
"override_base_seq_len": 16384,
|
61 |
"gpu_split_auto": false,
|
@@ -100,11 +100,11 @@ git config --global credential.helper 'store --file ~/.my-credentials'
|
|
100 |
huggingface-cli login
|
101 |
```
|
102 |
|
103 |
-
To download the `main` (only useful if you only care about measurement.json) branch to a folder called `
|
104 |
|
105 |
```shell
|
106 |
mkdir machinez_zephyr-orpo-141b-A35b-v0.1-exl2_2.75bpw
|
107 |
-
huggingface-cli download machinez/zephyr-orpo-141b-A35b-v0.1-exl2 --local-dir machinez_zephyr-orpo-141b-A35b-v0.1-
|
108 |
```
|
109 |
|
110 |
To download from a different branch, add the `--revision` parameter:
|
|
|
28 |
## Sample instructions to load in TabbyAPI @ 1.5bpw on 3x Nvidia Tesla P100 16gb at 4k context. ~14 tok/s
|
29 |
```JSON
|
30 |
{
|
31 |
+
"name": "machinez_zephyr-orpo-141b-A35b-v0.1_1.5bpw",
|
32 |
"max_seq_len": 4096,
|
33 |
"override_base_seq_len": 4096,
|
34 |
"gpu_split_auto": false,
|
|
|
55 |
## Sample instructions to load in TabbyAPI @ 2.75bpw on 4x Nvidia Tesla P100 16gb at 16k context. ~5.6 tok/s
|
56 |
```JSON
|
57 |
{
|
58 |
+
"name": "machinez_zephyr-orpo-141b-A35b-v0.1_2.75bpw",
|
59 |
"max_seq_len": 16384,
|
60 |
"override_base_seq_len": 16384,
|
61 |
"gpu_split_auto": false,
|
|
|
100 |
huggingface-cli login
|
101 |
```
|
102 |
|
103 |
+
To download the `main` (only useful if you only care about measurement.json) branch to a folder called `machinez_zephyr-orpo-141b-A35b-v0.1-exl2`:
|
104 |
|
105 |
```shell
|
106 |
mkdir machinez_zephyr-orpo-141b-A35b-v0.1-exl2_2.75bpw
|
107 |
+
huggingface-cli download machinez/zephyr-orpo-141b-A35b-v0.1-exl2 --local-dir machinez_zephyr-orpo-141b-A35b-v0.1-exl2 --local-dir-use-symlinks False
|
108 |
```
|
109 |
|
110 |
To download from a different branch, add the `--revision` parameter:
|