keyfan commited on
Commit
ec8a708
·
verified ·
1 Parent(s): 118579b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -5,4 +5,6 @@ library_name: transformers
5
 
6
  Unofficial dequantized weight of [grok-1](https://huggingface.co/xai-org/grok-1) in HF Transformers format.
7
 
8
- The weights are converted using the [script here](https://gist.github.com/chu-tianxiang/ec310e15d56949fd0f351cb5f65ee7a1) ran inside the [grok-1 repo](https://github.com/xai-org/grok-1). Since downloading the dequantized weight needs twice as much time, it's recommended to download the original weight and convert on your own.
 
 
 
5
 
6
  Unofficial dequantized weight of [grok-1](https://huggingface.co/xai-org/grok-1) in HF Transformers format.
7
 
8
+ Note: If you haven't download the weight yet, please use the `fp32` revision instead which uses float32 precision for RMSNorm and Router layer for better consistency.
9
+
10
+ The (fp32) weights are converted using the [script here](https://gist.github.com/chu-tianxiang/ec310e15d56949fd0f351cb5f65ee7a1) ran inside the [grok-1 repo](https://github.com/xai-org/grok-1). Since downloading the dequantized weight needs twice as much time, it's recommended to download the original weight and convert on your own.