English
sl-alex commited on
Commit
f7f59fb
1 Parent(s): de06029

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -30,14 +30,14 @@ Parts:
30
  ## Training
31
 
32
  Trained using [`qlora.py`](https://github.com/scottlogic-alex/qlora/blob/stepwise/qlora.py) from our [`stepwise`](https://github.com/scottlogic-alex/qlora/tree/stepwise) branch of [qlora](https://github.com/artidoro/qlora).
33
- Known-good as of commit [`4755584`](https://github.com/scottlogic-alex/qlora/blob/4755584f294ce88b3aadd9bdd3088c095b543578/qlora.py).
34
 
35
  `python -m qlora --model_name_or_path huggyllama/llama-7b --lora_name_or_path tloen/alpaca-lora-7b --dataset prm800k-solutions --dataset_format prm800k-solutions --bf16 --max_memory_MB 24000 --use_bos_token_in_prompt --truncate_toward_center --source_max_len 184 --target_max_len 998 --gradient_accumulation_steps 4 --per_device_train_batch_size 4 --per_device_eval_batch_size 4 --learning_rate 0.0002 --run_name 13b_alpaca_special_tokens_long --report_to wandb --save_steps 64 --save_total_limit 3 --max_steps 1664 --evaluation_strategy steps --eval_steps 64 --generate_steps 16`
36
 
37
  ## Usage
38
 
39
  You can load using [`evaluate.py`](https://github.com/scottlogic-alex/qlora/blob/stepwise/evaluate.py#L209-L278) from our [`stepwise`](https://github.com/scottlogic-alex/qlora/tree/stepwise) branch of [qlora](https://github.com/artidoro/qlora).
40
- Known-good as of commit [`4755584`](https://github.com/scottlogic-alex/qlora/blob/4755584f294ce88b3aadd9bdd3088c095b543578/evaluate.py).
41
 
42
  ```bash
43
  git clone -b stepwise https://github.com/scottlogic-alex/qlora.git
 
30
  ## Training
31
 
32
  Trained using [`qlora.py`](https://github.com/scottlogic-alex/qlora/blob/stepwise/qlora.py) from our [`stepwise`](https://github.com/scottlogic-alex/qlora/tree/stepwise) branch of [qlora](https://github.com/artidoro/qlora).
33
+ Known-good as of commit [`3a86919`](https://github.com/scottlogic-alex/qlora/blob/3a8691986b6718562bcd8e3522447b52842c1d9a/qlora.py).
34
 
35
  `python -m qlora --model_name_or_path huggyllama/llama-7b --lora_name_or_path tloen/alpaca-lora-7b --dataset prm800k-solutions --dataset_format prm800k-solutions --bf16 --max_memory_MB 24000 --use_bos_token_in_prompt --truncate_toward_center --source_max_len 184 --target_max_len 998 --gradient_accumulation_steps 4 --per_device_train_batch_size 4 --per_device_eval_batch_size 4 --learning_rate 0.0002 --run_name 13b_alpaca_special_tokens_long --report_to wandb --save_steps 64 --save_total_limit 3 --max_steps 1664 --evaluation_strategy steps --eval_steps 64 --generate_steps 16`
36
 
37
  ## Usage
38
 
39
  You can load using [`evaluate.py`](https://github.com/scottlogic-alex/qlora/blob/stepwise/evaluate.py#L209-L278) from our [`stepwise`](https://github.com/scottlogic-alex/qlora/tree/stepwise) branch of [qlora](https://github.com/artidoro/qlora).
40
+ Known-good as of commit [`3a86919`](https://github.com/scottlogic-alex/qlora/blob/3a8691986b6718562bcd8e3522447b52842c1d9a/evaluate.py).
41
 
42
  ```bash
43
  git clone -b stepwise https://github.com/scottlogic-alex/qlora.git