|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- togethercomputer/RedPajama-Data-1T |
|
tags: |
|
- open-llama |
|
- ggml |
|
- 7b |
|
- quantized |
|
- text-generation |
|
--- |
|
This repository contains the [300BT preview](https://huggingface.co/openlm-research/open_llama_7b_preview_300bt) of the |
|
[Open LLaMA](https://github.com/openlm-research/open_llama) model based on the |
|
[RedPajama 1T dataset](https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T), |
|
converted to GGML and quantized to 4 bit, ready to be used with [llama.cpp](https://github.com/rozek/llama.cpp) |
|
|
|
## Usage ## |
|
|
|
In order to use this model with llama.cpp |
|
|
|
* install llama.cpp as [described in the docs](https://github.com/ggerganov/llama.cpp#usage) |
|
* download this model |
|
* move it into the `models` subfolder of llama.cpp |
|
* run inferences with the additional parameter `-m ./models/7B/ggml-openllama-7b-300bt-q4_0.bin` |
|
|
|
For example, |
|
|
|
``` |
|
./main -m ./models/7B/ggml-openllama-7b-300bt-q4_0.bin -n 128 -p "who was Joseph Weizenbaum?" |
|
``` |
|
|
|
## License ## |
|
|
|
Since the underlying model is itself Apache 2.0 licensed, this repo is as well. |
|
|
|
--- |
|
license: apache-2.0 |
|
--- |
