model checkpoint has different layer shape with the model
#4
by
bruce-cdx
- opened
- transformer.layers.26.mlp.dense_4h_to_h.weight: found shape torch.Size([4096, 8192]) in the checkpoint and torch.Size([4096, 16384]) in the model instantiated
- transformer.layers.27.attention.query_key_value.weight: found shape torch.Size([12288, 2048]) in the checkpoint and torch.Size([12288, 4096]) in the model instantiated
- transformer.layers.27.attention.dense.weight: found shape torch.Size([4096, 2048]) in the checkpoint and torch.Size([4096, 4096]) in the model instantiated
- transformer.layers.27.mlp.dense_h_to_4h.weight: found shape torch.Size([16384, 2048]) in the checkpoint and torch.Size([16384, 4096]) in the model instantiated
- transformer.layers.27.mlp.dense_4h_to_h.weight: found shape torch.Size([4096, 8192]) in the checkpoint and torch.Size([4096, 16384]) in the model instantiated
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
欢迎使用 ChatGLM-6B 模型，输入内容即可进行对话，clear 清空对话历史，stop 终止程序
I got this log after I launch the cli_demo.py. Is it as expected to happen?