zementalist

/

llama-3-8B-chat-psychotherapist

like 23

When trying to convert this model into GGUF using llama.cpp I endup with issues:
ValueError: Can not map tensor 'model.layers.0.mlp.down_proj.weight.absmax'.

The whole error can be seen here

INFO:hf-to-gguf:Exporting model...
INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
INFO:hf-to-gguf:gguf: loading model part 'model-00001-of-00002.safetensors'
INFO:hf-to-gguf:token_embd.weight,           torch.float32 --> F16, shape = {4096, 128256}
INFO:hf-to-gguf:blk.0.attn_norm.weight,      torch.float32 --> F32, shape = {4096}
INFO:hf-to-gguf:blk.0.ffn_down.weight,       torch.uint8 --> F32, shape = {29360128}
Traceback (most recent call last):
  File "D:\llama.cpp\convert-hf-to-gguf.py", line 3263, in <module>
    main()
  File "D:\llama.cpp\convert-hf-to-gguf.py", line 3257, in main
    model_instance.write()
  File "D:\llama.cpp\convert-hf-to-gguf.py", line 330, in write
    self.write_tensors()
  File "D:\llama.cpp\convert-hf-to-gguf.py", line 1413, in write_tensors
    super().write_tensors()
  File "D:\llama.cpp\convert-hf-to-gguf.py", line 267, in write_tensors
    for new_name, data in ((n, d.squeeze().numpy()) for n, d in self.modify_tensors(data_torch, name, bid)):
                                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\llama.cpp\convert-hf-to-gguf.py", line 1410, in modify_tensors
    return [(self.map_tensor_name(name), data_torch)]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\llama.cpp\convert-hf-to-gguf.py", line 185, in map_tensor_name
    raise ValueError(f"Can not map tensor {name!r}")
ValueError: Can not map tensor 'model.layers.0.mlp.down_proj.weight.absmax'

I encounter similiar issue when I am trying to use this model with Ollama.