Update README.md
Browse files
README.md
CHANGED
@@ -30,14 +30,14 @@ Parts:
|
|
30 |
## Training
|
31 |
|
32 |
Trained using [`qlora.py`](https://github.com/scottlogic-alex/qlora/blob/stepwise/qlora.py) from our [`stepwise`](https://github.com/scottlogic-alex/qlora/tree/stepwise) branch of [qlora](https://github.com/artidoro/qlora).
|
33 |
-
Known-good as of commit [`
|
34 |
|
35 |
`python -m qlora --model_name_or_path huggyllama/llama-7b --lora_name_or_path tloen/alpaca-lora-7b --dataset prm800k-solutions --dataset_format prm800k-solutions --bf16 --max_memory_MB 24000 --use_bos_token_in_prompt --truncate_toward_center --source_max_len 184 --target_max_len 998 --gradient_accumulation_steps 4 --per_device_train_batch_size 4 --per_device_eval_batch_size 4 --learning_rate 0.0002 --run_name 13b_alpaca_special_tokens_long --report_to wandb --save_steps 64 --save_total_limit 3 --max_steps 1664 --evaluation_strategy steps --eval_steps 64 --generate_steps 16`
|
36 |
|
37 |
## Usage
|
38 |
|
39 |
You can load using [`evaluate.py`](https://github.com/scottlogic-alex/qlora/blob/stepwise/evaluate.py#L209-L278) from our [`stepwise`](https://github.com/scottlogic-alex/qlora/tree/stepwise) branch of [qlora](https://github.com/artidoro/qlora).
|
40 |
-
Known-good as of commit [`
|
41 |
|
42 |
```bash
|
43 |
git clone -b stepwise https://github.com/scottlogic-alex/qlora.git
|
|
|
30 |
## Training
|
31 |
|
32 |
Trained using [`qlora.py`](https://github.com/scottlogic-alex/qlora/blob/stepwise/qlora.py) from our [`stepwise`](https://github.com/scottlogic-alex/qlora/tree/stepwise) branch of [qlora](https://github.com/artidoro/qlora).
|
33 |
+
Known-good as of commit [`3a86919`](https://github.com/scottlogic-alex/qlora/blob/3a8691986b6718562bcd8e3522447b52842c1d9a/qlora.py).
|
34 |
|
35 |
`python -m qlora --model_name_or_path huggyllama/llama-7b --lora_name_or_path tloen/alpaca-lora-7b --dataset prm800k-solutions --dataset_format prm800k-solutions --bf16 --max_memory_MB 24000 --use_bos_token_in_prompt --truncate_toward_center --source_max_len 184 --target_max_len 998 --gradient_accumulation_steps 4 --per_device_train_batch_size 4 --per_device_eval_batch_size 4 --learning_rate 0.0002 --run_name 13b_alpaca_special_tokens_long --report_to wandb --save_steps 64 --save_total_limit 3 --max_steps 1664 --evaluation_strategy steps --eval_steps 64 --generate_steps 16`
|
36 |
|
37 |
## Usage
|
38 |
|
39 |
You can load using [`evaluate.py`](https://github.com/scottlogic-alex/qlora/blob/stepwise/evaluate.py#L209-L278) from our [`stepwise`](https://github.com/scottlogic-alex/qlora/tree/stepwise) branch of [qlora](https://github.com/artidoro/qlora).
|
40 |
+
Known-good as of commit [`3a86919`](https://github.com/scottlogic-alex/qlora/blob/3a8691986b6718562bcd8e3522447b52842c1d9a/evaluate.py).
|
41 |
|
42 |
```bash
|
43 |
git clone -b stepwise https://github.com/scottlogic-alex/qlora.git
|