---
license: apache-2.0
datasets:
- togethercomputer/RedPajama-Data-1T
tags:
- open-llama
- ggml
- 7b
- quantized
- text-generation
---
This repository contains the [300BT preview](https://huggingface.co/openlm-research/open_llama_7b_preview_300bt) of the
[Open LLaMA](https://github.com/openlm-research/open_llama) model based on the
[RedPajama 1T dataset](https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T),
converted to GGML and quantized to 4 bit, ready to be used with [llama.cpp](https://github.com/rozek/llama.cpp)

## Usage ##

In order to use this model with llama.cpp

* install llama.cpp as [described in the docs](https://github.com/ggerganov/llama.cpp#usage)
* download this model
* move it into the `models` subfolder of llama.cpp
* run inferences with the additional parameter `-m ./models/7B/ggml-openllama-7b-300bt-q4_0.bin`

For example,

```
./main -m ./models/7B/ggml-openllama-7b-300bt-q4_0.bin -n 128 -p "who was Joseph Weizenbaum?"
```

## License ##

Since the underlying model is itself Apache 2.0 licensed, this repo is as well.

---
license: apache-2.0
---