AstroLLaMA.gguf

AstroLLaMA in GPT-Generated Unified Format.

What's GPT-Generated Unified Format?

GGUF is file format used for storing models for inference, particularly in the context of language models like GPT. GGUF models could be run on CPUs, which made them accessible to a wider range of users.

More detailed description can be found here.

How to play with AstroLLaMA on your laptop?

1. Install ollama;
2. Download astrollama.gguf to your PC;
3. Create a file named Modelfile, and add a FROM instruction with the local filepath to AstroLLaMA, e.g.,

FROM ./astrollama.gguf

4. Create the model in Ollama

ollama create astrollama -f path_to_modelfile

5. Run AstroLLaMA locally

ollama run astrollama