uploaded readme

README.md

@@ -0,0 +1,82 @@
+Quantization made by Richard Erkhov.
+
+[Github](https://github.com/RichardErkhov)
+
+[Discord](https://discord.gg/pvy7H8DZMG)
+
+[Request more models](https://github.com/RichardErkhov/quant_request)
+
+
+phi-1_5_chat_128k - bnb 4bits
+- Model creator: https://huggingface.co/voidful/
+- Original model: https://huggingface.co/voidful/phi-1_5_chat_128k/
+
+
+
+
+Original model description:
+---
+language:
+- en
+license: mit
+---
+
+## Model Summary
+
+The language model Phi-1.5 is a Transformer with **1.3 billion** parameters. It was trained using the same data sources as [phi-1](https://huggingface.co/microsoft/phi-1), augmented with a new data source that consists of various NLP synthetic texts. When assessed against benchmarks testing common sense, language understanding, and logical reasoning, Phi-1.5 demonstrates a nearly state-of-the-art performance among models with less than 10 billion parameters.
+
+We've trained Microsoft Research's phi-1.5, 1.3B parameter model with multi-turn conversation datasets on at most 32k and extended to 128k.
+
+## Example
+```python
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+torch.set_default_device("cuda")
+
+model = AutoModelForCausalLM.from_pretrained("voidful/phi-1_5_chat_32k")
+tokenizer = AutoTokenizer.from_pretrained("voidful/phi-1_5_chat_32k",device_map="auto")
+
+messages = [
+    {
+        "role": "system",
+        "content": "You are a helpful assistant.",
+    },
+    {"role": "user", "content": "how to make a keyboard?"},
+    {"role": "assistant", "content": """To make a keyboard, you will need the following materials:
+
+1. A set of keys, such as those found on a standard computer keyboard.
+2. A circuit board, such as a MOSFET (Metal-Oxide-Semiconductor-Fluorescent) board.
+3. A power supply to provide power to the circuit board.
+4. A case to enclose the circuit board and protect it from damage.
+5. A soldering iron and solder to connect the components together.
+
+Here is a basic outline of the steps involved in making a keyboard:
+
+1. Connect the circuit board to the power supply and the case.
+2. Connect the MOSFETs to the power supply and the case, using a soldering iron and solder.
+3. Connect the keys to the circuit board, using a soldering iron and solder.
+4. Test the keyboard to ensure that it works properly.
+5. Package the keyboard in a box or case for shipping or sale.
+
+Note that this is just a basic outline, and there are many additional steps and considerations that will depend on the specific design and requirements of your keyboard.</s>"""},
+    {"role": "user", "content": "where to buy the circuit?"},
+ ]
+tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
+
+outputs = model.generate(tokenized_chat, max_length=1024)
+text = tokenizer.batch_decode(outputs[:,tokenized_chat.shape[-1]:-1])[0]
+print(text)
+```
+
+### Result
+```
+There are several places where you can buy a circuit board. Here are some of the most common places:
+
+1. Electronics stores: Many electronics stores carry a variety of circuit boards for different purposes.
+2. Online marketplaces: There are several online marketplaces where you can buy circuit boards, such as Amazon, eBay, and Alibaba.
+3. Specialty stores: There are several specialty stores that carry a variety of circuit boards for different purposes, such as hobby stores, craft stores, and home improvement stores.
+
+In general, it is a good idea to shop around and compare prices and features before making a purchase.
+```
+