failspy
/

Nemotron-4-340B-Instruct-SafeTensors

failspy commited on Jun 16

Commit

71c3f34

•

1 Parent(s): 54b7352

Update hypothetical config.json for LN

Files changed (1) hide show

config.json CHANGED Viewed

@@ -16,7 +16,7 @@
   "num_hidden_layers": 96,
   "num_key_value_heads": 8,
   "pretraining_tp": 1,
-  "rms_norm_eps": 1e-05,
   "rope_scaling": null,
   "partial_rotary_factor": 0.5,
   "rope_theta":10000.0,

   "num_hidden_layers": 96,
   "num_key_value_heads": 8,
   "pretraining_tp": 1,
+  "layer_norm_eps": 1e-05,
   "rope_scaling": null,
   "partial_rotary_factor": 0.5,
   "rope_theta":10000.0,