README.md

Quantization made by Richard Erkhov.

[Github](https://github.com/RichardErkhov)

[Discord](https://discord.gg/pvy7H8DZMG)

[Request more models](https://github.com/RichardErkhov/quant_request)


phi-1_5_chat_128k - bnb 4bits
- Model creator: https://huggingface.co/voidful/
- Original model: https://huggingface.co/voidful/phi-1_5_chat_128k/




Original model description:
---
language:
- en
license: mit
---

## Model Summary

The language model Phi-1.5 is a Transformer with **1.3 billion** parameters. It was trained using the same data sources as [phi-1](https://huggingface.co/microsoft/phi-1), augmented with a new data source that consists of various NLP synthetic texts. When assessed against benchmarks testing common sense, language understanding, and logical reasoning, Phi-1.5 demonstrates a nearly state-of-the-art performance among models with less than 10 billion parameters.

We've trained Microsoft Research's phi-1.5, 1.3B parameter model with multi-turn conversation datasets on at most 32k and extended to 128k.

## Example
```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

torch.set_default_device("cuda")

model = AutoModelForCausalLM.from_pretrained("voidful/phi-1_5_chat_32k")
tokenizer = AutoTokenizer.from_pretrained("voidful/phi-1_5_chat_32k",device_map="auto")

messages = [
    {
        "role": "system",
        "content": "You are a helpful assistant.",
    },
    {"role": "user", "content": "how to make a keyboard?"},
    {"role": "assistant", "content": """To make a keyboard, you will need the following materials:

1. A set of keys, such as those found on a standard computer keyboard.
2. A circuit board, such as a MOSFET (Metal-Oxide-Semiconductor-Fluorescent) board.
3. A power supply to provide power to the circuit board.
4. A case to enclose the circuit board and protect it from damage.
5. A soldering iron and solder to connect the components together.

Here is a basic outline of the steps involved in making a keyboard:

1. Connect the circuit board to the power supply and the case.
2. Connect the MOSFETs to the power supply and the case, using a soldering iron and solder.
3. Connect the keys to the circuit board, using a soldering iron and solder.
4. Test the keyboard to ensure that it works properly.
5. Package the keyboard in a box or case for shipping or sale.

Note that this is just a basic outline, and there are many additional steps and considerations that will depend on the specific design and requirements of your keyboard.</s>"""},
    {"role": "user", "content": "where to buy the circuit?"},
 ]
tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")

outputs = model.generate(tokenized_chat, max_length=1024)
text = tokenizer.batch_decode(outputs[:,tokenized_chat.shape[-1]:-1])[0]
print(text)
```

### Result
```
There are several places where you can buy a circuit board. Here are some of the most common places:

1. Electronics stores: Many electronics stores carry a variety of circuit boards for different purposes.
2. Online marketplaces: There are several online marketplaces where you can buy circuit boards, such as Amazon, eBay, and Alibaba.
3. Specialty stores: There are several specialty stores that carry a variety of circuit boards for different purposes, such as hobby stores, craft stores, and home improvement stores.

In general, it is a good idea to shop around and compare prices and features before making a purchase.
```