File size: 5,054 Bytes
6f22822
2d4e83e
e3c738c
 
49942b2
e3c738c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6f22822
e3c738c
 
a129834
e3c738c
 
 
 
 
70ddb55
e3c738c
 
 
a1e7543
e3c738c
ff0df0b
e3c738c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
e17464f
e3c738c
 
e17464f
e3c738c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
---
license: [llama2, other]
datasets:
- cerebras/SlimPajama-627B
- Open-Orca/OpenOrca
language:
- en
tags:
- Deci AI
- DeciLM
- Instruction
model-index:
- name: DeciLM 6B
  results:
  - task:
      type: text-generation
    dataset:
      type: ai2/arc
      name: ai2_arc
    metrics:
    - name: ARC Challenge
      type: ARC Challenge
      value: 43.43
      verified: false
  - task:
      type: text-generation
    dataset:
      type: ai2/arc
      name: ai2_arc
    metrics:
    - name: ARC Easy
      type: ARC Easy
      value: 70.58
      verified: false
  - task:
      type: text-generation
    dataset:
      type: boolq
      name: boolq
    metrics:
    - name: BoolQ
      type: BoolQ
      value: 77.34
      verified: false
  - task:
      type: text-generation
    dataset:
      type: hellaswag
      name: hellaswag
    metrics:
    - name: HellaSwag
      type: HellaSwag
      value: 74.57
      verified: false
  - task:
      type: text-generation
    dataset:
      type: LAMBDA
      name: OpenAI LAMBDA
    metrics:
    - name: LAMBDA
      type: LAMBDA
      value: 70.1
      verified: false
  - task:
      type: text-generation
    dataset:
      type: OpenBookQA
      name: openbookqa
    metrics:
    - name: OpenBookQA
      type: OpenBookQA
      value: 33
      verified: false
  - task:
      type: text-generation
    dataset:
      type: PIQA
      name: piqa
    metrics:
    - name: PIQA
      type: PIQA
      value: 77.52
      verified: false
  - task:
      type: text-generation
    dataset:
      type: truthful_qa
      name: truthful_qa
    metrics:
    - name: TruthfulQA
      type: TruthfulQA
      value: 43.89
      verified: false
  - task:
      type: text-generation
    dataset:
      type: winogrande
      name: winogrande
    metrics:
    - name: Winogrande
      type: Winogrande
      value: 67.64
      verified: false
---
# DeciLM 6B-Instruct

DeciLM 6B-Instruct is a model for short-form instruction following. It is built by LoRA fine-tuning [DeciLM 6B](https://huggingface.co/Deci/DeciLM-6b) on a subset of the [OpenOrca dataset](https://huggingface.co/datasets/Open-Orca/OpenOrca).


- **Developed by:** Deci
- **Model type:** DeciLM is an auto-regressive language model using an optimized transformer decoder architecture that includes variable Grouped-Query Attention.
- **Language(s) (NLP):** English
- **License:**  [Llama 2 Community License Agreement](https://huggingface.co/Deci/DeciLM-6b-instruct/blob/main/LICENSE.md) with an extention of Deci regarding hosting service providers.

### Model Sources

- **Paper:** [DeciLM 6B Technical Blog](https://deci.ai/blog/decilm-15-times-faster-than-llama2-nas-generated-llm-with-variable-gqa/)
- **Demo:** [DeciLM 6B-Instruct Demo](https://huggingface.co/spaces/Deci/DeciLM-6b-instruct)
- **Notebook:** [DeciLM 6B-Instruct Notebook](https://bit.ly/decilm-instruct-nb)

## Uses

The model is intended for commercial and research use in English and can be fine-tuned for use in other languages.

## How to Get Started with the Model

Use the code below to get started with the model.

```bibtex
# pip install -q transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

checkpoint = "Deci/DeciLM-6b-instruct"
device = "cuda" # for GPU usage or "cpu" for CPU usage

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForCausalLM.from_pretrained(checkpoint, torch_dtype=torch.bfloat16, trust_remote_code=True).to(device)

inputs = tokenizer.encode("How do I make french toast? Think through it step by step", return_tensors="pt").to(device)
outputs = model.generate(inputs, max_new_tokens=100, do_sample=True, top_p=0.95)
print(tokenizer.decode(outputs[0]))
```

## Training Details

DeciLM 6B underwent training utilizing the SlimPijamas dataset, leveraging advanced proprietary methodologies allowing for fast training. DeciLM 6B was further finetuned on a subset of the OpenOrca dataset, giving rise to DeciLM-6B-Instruct.

## Evaluation

Below are DeciLM's 6B-instruct evaluation results.

| Average | ARC Challenge* | ARC Easy* | BoolQ | HellaSwag* | LAMBDA OpenAI | OpenBookQA | PIQA | TruthfulQA | Winogrande |
|:----------|:----------|:----------|:----------|:----------|:----------|:----------|:----------|:----------|:----------|
| 62.01    | 44.43    | 70.58    | 77.34    | 74.57    | 70.1    | 33    | 77.52    |43.89    | 67.64    | 
Accuracy-norm score*


## Runtime Benchmarks

|Inference Tool/Hardware | A10 (tokens/sec) |
|:----------|:----------|
| PyTorch   | 652.49 | 
| Infery LLM | 2,029.6  | 

- Throughput (tokens/sec) - Measured with optimal batch - PyTorch BS 64, Infery LLM BS 128

## Disclaimer

DeciLM 6B-Instruct has not been aligned for safety or trained using RLHF. 

## How to Cite

Please cite this model using this format.

```bibtex
@misc{DeciFoundationModels,
title = {DeciLM 6B Instruct},
author = {DeciAI Research Team},
year = {2023}
url={[https://huggingface.co/Deci/DeciLM-6b-instruct](https://huggingface.co/Deci/DeciLM-6b-instruct)},
}
```