Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,17 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+minichat-1.5-3b.Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+minichat-1.5-3b.Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+minichat-1.5-3b.Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+minichat-1.5-3b.Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+minichat-1.5-3b.Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+minichat-1.5-3b.Q4_1.gguf filter=lfs diff=lfs merge=lfs -text
+minichat-1.5-3b.Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+minichat-1.5-3b.Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+minichat-1.5-3b.Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+minichat-1.5-3b.Q5_1.gguf filter=lfs diff=lfs merge=lfs -text
+minichat-1.5-3b.Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+minichat-1.5-3b.Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+minichat-1.5-3b.Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+minichat-1.5-3b.Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,75 @@
+---
+title: "MiniChat-1.5-3B Quantized in GGUF"
+tags:
+  - GGUF
+language: en
+---
+# GGUF's of MiniChat-1.5-3B
+
+This is a GGUF quantization of MiniChat-1.5-3B.
+
+## Original Model Card:
+---
+
+## MiniChat-1.5-3B
+
+📑 [arXiv](https://arxiv.org/abs/2311.07052) | 👻 [GitHub](https://github.com/GeneZC/MiniMA) | 🤗 [HuggingFace-MiniMA](https://huggingface.co/GeneZC/MiniMA-3B) | 🤗 [HuggingFace-MiniChat](https://huggingface.co/GeneZC/MiniChat-3B) | 🤗 [HuggingFace-MiniChat-1.5](https://huggingface.co/GeneZC/MiniChat-1.5-3B) | 🤖 [ModelScope-MiniMA](https://modelscope.cn/models/GeneZC/MiniMA-3B) | 🤖 [ModelScope-MiniChat](https://modelscope.cn/models/GeneZC/MiniChat-3B)
+
+🆕 **Updates from MiniChat-3B**: 
+- better data mixture;
+- use of [NEFTune](https://arxiv.org/abs/2310.05914);
+- use of [DPO](https://arxiv.org/abs/2305.18290).
+
+❗ Must comply with LICENSE of LLaMA2 since it is derived from LLaMA2.
+
+A language model distilled and finetuned from an adapted version of LLaMA2-7B following "Towards the Law of Capacity Gap in Distilling Language Models".
+
+Outperforming a wide range of 3B competitors in GPT4 evaluation and even competing with several 7B chat models.
+
+<img src="./teaser_b.jpg" alt="teaser_b" width="687" />
+
+The following is an example code snippet to use MiniChat-3B:
+
+```python
+import torch
+
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+from conversation import get_default_conv_template
+
+# MiniChat
+tokenizer = AutoTokenizer.from_pretrained("GeneZC/MiniChat-3B", use_fast=False)
+# GPU.
+model = AutoModelForCausalLM.from_pretrained("GeneZC/MiniChat-3B", use_cache=True, device_map="auto", torch_dtype=torch.float16).eval()
+# CPU.
+# model = AutoModelForCausalLM.from_pretrained("GeneZC/MiniChat-3B", use_cache=True, device_map="cpu", torch_dtype=torch.float16).eval()
+
+conv = get_default_conv_template("minichat")
+
+question = "Implement a program to find the common elements in two arrays without using any extra data structures."
+conv.append_message(conv.roles[0], question)
+conv.append_message(conv.roles[1], None)
+prompt = conv.get_prompt()
+input_ids = tokenizer([prompt]).input_ids
+output_ids = model.generate(
+    torch.as_tensor(input_ids).cuda(),
+    do_sample=True,
+    temperature=0.7,
+    max_new_tokens=1024,
+)
+output_ids = output_ids[0][len(input_ids[0]):]
+output = tokenizer.decode(output_ids, skip_special_tokens=True).strip()
+# output: "def common_elements(arr1, arr2):\n    if len(arr1) == 0:\n        return []\n    if len(arr2) == 0:\n        return arr1\n\n    common_elements = []\n    for element in arr1:\n        if element in arr2:\n            common_elements.append(element)\n\n    return common_elements"
+# Multiturn conversation could be realized by continuously appending questions to `conv`.
+```
+
+## Bibtex
+
+```bibtex
+@article{zhang2023law,
+    title={Towards the Law of Capacity Gap in Distilling Language Models},
+    author={Zhang, Chen and Song, Dawei and Ye, Zheyu and Gao, Yan},
+    year={2023},
+    url={https://arxiv.org/abs/2311.07052}
+}
+```

minichat-1.5-3b.Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0fc12ed2817faa2bef3aa5b3282a0775e3192362375604624b4e7376667d6a20
+size 1297187936

minichat-1.5-3b.Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0ec83f845378db87af095ab1695dc4831cf89646f14fb10f8ff0149d227a69fa
+size 1631048288

minichat-1.5-3b.Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2d86dc1e3bdc29671cea85d4a6fbcc683d426b8fb7285dea387640e397999f92
+size 1507578464

minichat-1.5-3b.Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c19418e7dbba08cb9e4041f9f226c6411660bad5b2f530519a58aea39ee3d28a
+size 1358549600

minichat-1.5-3b.Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4b6f9bb849621a46fff06046c1c81f332dee308267015d572daec7deffc7d74
+size 1739602016

minichat-1.5-3b.Q4_1.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:051729d88bb30f3027ad4f19d2e3d6d11d547f814e8e44294b4f6470c0cf1a38
+size 1918920800

minichat-1.5-3b.Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e1fd530238689e418a047c91fb4449836897a79a705dfb57c7ccac9d0758484
+size 1846655072

minichat-1.5-3b.Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:425bbaade022c98e20e683eea1c0de06611a1b0ded25d7d40511fd78d619acc7
+size 1756903520

minichat-1.5-3b.Q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:919f73f5dbacd7310fdbb03593d93738268dcb21e4dab3c8bcadbba978f0e389
+size 2098239584

minichat-1.5-3b.Q5_1.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:106a624b3363b01d6429c62a7632c683c53cb5795a2664ac28417aad12b6d123
+size 2277558368

minichat-1.5-3b.Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:313951b28a7ffc0e70b6cadf2e5f6383b3d51c8898c7de06835067c1189ee0e2
+size 2153388128

minichat-1.5-3b.Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29d6cbf3e33c2608efbb2f1d24c76d2a0290bf8b6cdec56da605c993dff8e748
+size 2098239584

minichat-1.5-3b.Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93a1cd1b33ee884e285907d0870155f5d0dbfdb3999878c2f828c7a036666eb7
+size 2479292000

minichat-1.5-3b.Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:28c909c1f1c2f3affc5607439bc7219f4d23e4778d1a13c6bfe7d9d18d54df73
+size 3210768992