ByteWave/Cheus-11B · Can't load model due to tensor shape mismatch

Hi, thanks for the model!

I'm trying to load it, and am getting this:

ValueError: Trying to set a tensor of shape torch.Size([32000, 4096]) in "weight" (which has shape torch.Size([32002, 4096])), this look incorrect.

It seems to be a mismatch between the base vocab size of 32000, and the size with the extra two tokens of 32002.

I tried some simple JSON file edits, but haven't managed to resolve it that way.

Did you manage to load it yourself, and if so what is required to do so?

Thanks

Demonstration of issue:

>>> from transformers import AutoModelForCausalLM
>>> model = AutoModelForCausalLM.from_pretrained("ByteWave/Cheus-11B", low_cpu_mem_usage=True)
Loading checkpoint shards:   0%|                                                                                                     | 0/3 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/workspace/venv/pytorch2/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
    return model_class.from_pretrained(
  File "/workspace/venv/pytorch2/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3480, in from_pretrained
    ) = cls._load_pretrained_model(
  File "/workspace/venv/pytorch2/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3870, in _load_pretrained_model
    new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
  File "/workspace/venv/pytorch2/lib/python3.10/site-packages/transformers/modeling_utils.py", line 743, in _load_state_dict_into_meta_model
    set_module_tensor_to_device(model, param_name, param_device, **set_module_kwargs)
  File "/workspace/venv/pytorch2/lib/python3.10/site-packages/accelerate/utils/modeling.py", line 285, in set_module_tensor_to_device
    raise ValueError(
ValueError: Trying to set a tensor of shape torch.Size([32000, 4096]) in "weight" (which has shape torch.Size([32002, 4096])), this look incorrect.
>>>