CyberNative commited on
Commit
de3978a
·
1 Parent(s): 5b637c5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -5,14 +5,14 @@ license: llama2
5
 
6
  ## THIS IS A PLACEHOLDER, MODEL COMMING SOON
7
 
8
- CyberBase 13b 8k *base model* - (llama-2-13b - lmsys/vicuna-13b-v1.5-16k)
9
 
10
  ## Test run 1 (less context, more trainable params):
11
  - sequence_len: 4096
12
  - max_packed_sequence_len: 4096
13
  - lora_r: 256
14
  - lora_alpha: 128
15
- - num_epochs: 2
16
  - trainable params: 1,001,390,080 || all params: 14,017,264,640 || trainable%: 7.143976415643959
17
 
18
  # Base cybersecurity model for future fine-tuning, it is not recomended to use on it's own.
 
5
 
6
  ## THIS IS A PLACEHOLDER, MODEL COMMING SOON
7
 
8
+ CyberBase is an experimental *base model* for cybersecurity. (llama-2-13b -> lmsys/vicuna-13b-v1.5-16k -> CyberBase)
9
 
10
  ## Test run 1 (less context, more trainable params):
11
  - sequence_len: 4096
12
  - max_packed_sequence_len: 4096
13
  - lora_r: 256
14
  - lora_alpha: 128
15
+ - num_epochs: 3
16
  - trainable params: 1,001,390,080 || all params: 14,017,264,640 || trainable%: 7.143976415643959
17
 
18
  # Base cybersecurity model for future fine-tuning, it is not recomended to use on it's own.