|
--- |
|
datasets: |
|
- bigscience/xP3 |
|
license: bigscience-bloom-rail-1.0 |
|
language: |
|
- ak |
|
- ar |
|
- as |
|
- bm |
|
- bn |
|
- ca |
|
- code |
|
- en |
|
- es |
|
- eu |
|
- fon |
|
- fr |
|
- gu |
|
- hi |
|
- id |
|
- ig |
|
- ki |
|
- kn |
|
- lg |
|
- ln |
|
- ml |
|
- mr |
|
- ne |
|
- nso |
|
- ny |
|
- or |
|
- pa |
|
- pt |
|
- rn |
|
- rw |
|
- sn |
|
- st |
|
- sw |
|
- ta |
|
- te |
|
- tn |
|
- ts |
|
- tum |
|
- tw |
|
- ur |
|
- vi |
|
- wo |
|
- xh |
|
- yo |
|
- zh |
|
- zu |
|
programming_language: |
|
- C |
|
- C++ |
|
- C# |
|
- Go |
|
- Java |
|
- JavaScript |
|
- Lua |
|
- PHP |
|
- Python |
|
- Ruby |
|
- Rust |
|
- Scala |
|
- TypeScript |
|
tags: |
|
- llm-rs |
|
- ggml |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
# GGML covnerted Models of [BigScience](https://huggingface.co/bigscience)'s Bloom models |
|
|
|
## Description |
|
|
|
> We present BLOOMZ & mT0, a family of models capable of following human instructions in dozens of languages zero-shot. We finetune BLOOM & mT5 pretrained multilingual language models on our crosslingual task mixture (xP3) and find the resulting models capable of crosslingual generalization to unseen tasks & languages. |
|
|
|
- **Repository:** [bigscience-workshop/xmtf](https://github.com/bigscience-workshop/xmtf) |
|
- **Paper:** [Crosslingual Generalization through Multitask Finetuning](https://arxiv.org/abs/2211.01786) |
|
- **Point of Contact:** [Niklas Muennighoff](mailto:niklas@hf.co) |
|
- **Languages:** Refer to [bloom](https://huggingface.co/bigscience/bloom) for pretraining & [xP3](https://huggingface.co/datasets/bigscience/xP3) for finetuning language proportions. It understands both pretraining & finetuning languages. |
|
|
|
### Intended use |
|
|
|
We recommend using the model to perform tasks expressed in natural language. For example, given the prompt "*Translate to English: Je t’aime.*", the model will most likely answer "*I love you.*". Some prompt ideas from our paper: |
|
- 一个传奇的开端，一个不灭的神话，这不仅仅是一部电影，而是作为一个走进新时代的标签，永远彪炳史册。你认为这句话的立场是赞扬、中立还是批评? |
|
- Suggest at least five related search terms to "Mạng neural nhân tạo". |
|
- Write a fairy tale about a troll saving a princess from a dangerous dragon. The fairy tale is a masterpiece that has achieved praise worldwide and its moral is "Heroes Come in All Shapes and Sizes". Story (in Spanish): |
|
- Explain in a sentence in Telugu what is backpropagation in neural networks. |
|
|
|
## Converted Models |
|
| Name | Based on | Type | Container | GGML Version | |
|
|:----------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------|:-------|:------------|:---------------| |
|
| [bloomz-1b1-f16.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b1-f16.bin) | [bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1) | F16 | GGML | V3 | |
|
| [bloomz-1b1-q4_0.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b1-q4_0.bin) | [bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1) | Q4_0 | GGML | V3 | |
|
| [bloomz-1b1-q4_0-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b1-q4_0-ggjt.bin) | [bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1) | Q4_0 | GGJT | V3 | |
|
| [bloomz-1b1-q5_1-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b1-q5_1-ggjt.bin) | [bigscience/bloomz-1b1](https://huggingface.co/bigscience/bloomz-1b1) | Q5_1 | GGJT | V3 | |
|
| [bloomz-1b7-f16.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b7-f16.bin) | [bigscience/bloomz-1b7](https://huggingface.co/bigscience/bloomz-1b7) | F16 | GGML | V3 | |
|
| [bloomz-1b7-q4_0.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b7-q4_0.bin) | [bigscience/bloomz-1b7](https://huggingface.co/bigscience/bloomz-1b7) | Q4_0 | GGML | V3 | |
|
| [bloomz-1b7-q4_0-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b7-q4_0-ggjt.bin) | [bigscience/bloomz-1b7](https://huggingface.co/bigscience/bloomz-1b7) | Q4_0 | GGJT | V3 | |
|
| [bloomz-1b7-q5_1-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-1b7-q5_1-ggjt.bin) | [bigscience/bloomz-1b7](https://huggingface.co/bigscience/bloomz-1b7) | Q5_1 | GGJT | V3 | |
|
| [bloomz-3b-f16.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-3b-f16.bin) | [bigscience/bloomz-3b](https://huggingface.co/bigscience/bloomz-3b) | F16 | GGML | V3 | |
|
| [bloomz-3b-q4_0.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-3b-q4_0.bin) | [bigscience/bloomz-3b](https://huggingface.co/bigscience/bloomz-3b) | Q4_0 | GGML | V3 | |
|
| [bloomz-3b-q4_0-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-3b-q4_0-ggjt.bin) | [bigscience/bloomz-3b](https://huggingface.co/bigscience/bloomz-3b) | Q4_0 | GGJT | V3 | |
|
| [bloomz-3b-q5_1-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-3b-q5_1-ggjt.bin) | [bigscience/bloomz-3b](https://huggingface.co/bigscience/bloomz-3b) | Q5_1 | GGJT | V3 | |
|
| [bloomz-560m-f16.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-560m-f16.bin) | [bigscience/bloomz-560m](https://huggingface.co/bigscience/bloomz-560m) | F16 | GGML | V3 | |
|
| [bloomz-560m-q4_0.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-560m-q4_0.bin) | [bigscience/bloomz-560m](https://huggingface.co/bigscience/bloomz-560m) | Q4_0 | GGML | V3 | |
|
| [bloomz-560m-q4_0-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-560m-q4_0-ggjt.bin) | [bigscience/bloomz-560m](https://huggingface.co/bigscience/bloomz-560m) | Q4_0 | GGJT | V3 | |
|
| [bloomz-560m-q5_1-ggjt.bin](https://huggingface.co/rustformers/bloomz-ggml/blob/main/bloomz-560m-q5_1-ggjt.bin) | [bigscience/bloomz-560m](https://huggingface.co/bigscience/bloomz-560m) | Q5_1 | GGJT | V3 | |
|
|
|
## Usage |
|
|
|
### Python via [llm-rs](https://github.com/LLukas22/llm-rs-python): |
|
|
|
#### Installation |
|
Via pip: `pip install llm-rs` |
|
|
|
#### Run inference |
|
```python |
|
from llm_rs import AutoModel |
|
|
|
#Load the model, define any model you like from the list above as the `model_file` |
|
model = AutoModel.from_pretrained("rustformers/bloomz-ggml",model_file="bloomz-3b-q4_0-ggjt.bin") |
|
|
|
#Generate |
|
print(model.generate("The meaning of life is")) |
|
``` |
|
|
|
### Rust via [Rustformers/llm](https://github.com/rustformers/llm): |
|
|
|
#### Installation |
|
``` |
|
git clone --recurse-submodules https://github.com/rustformers/llm.git |
|
cd llm |
|
cargo build --release |
|
``` |
|
|
|
#### Run inference |
|
``` |
|
cargo run --release -- bloom infer -m path/to/model.bin -p "Tell me how cool the Rust programming language is:" |
|
``` |
