Help Needed: Running Safetensors
#1
by
NullEqualsZero
- opened
I feel a bit embarrassed asking this, but I'm very new to this and could really use some guidance. I'm trying to get this to work with Ollama (which I’ve been using to run LLMs), but I’ve hit a roadblock. Here’s the issue:
Ollama can’t use the model in safetensors format—at least, I think that’s because it uses the MLLama architecture, which Ollama can't handle. I was advised to convert the model to GGUF format using llama.cpp, but that didn’t work either. I got an "Unsupported architecture" error, which seems to be the same root issue.
So, here’s my (probably silly) question:
How can I run this model, ideally in a server-style setup like Ollama or llama.cpp?