Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: apache-2.0
|
3 |
+
metrics:
|
4 |
+
- accuracy
|
5 |
+
pipeline_tag: text-classification
|
6 |
+
---
|
7 |
+
# notdiamond-4k-0001
|
8 |
+
|
9 |
+
|
10 |
+
notdiamond-4k-0001 supports **4096 input sequence length**. This model is an extention of [notdiamond-0001](https://huggingface.co/notdiamond/notdiamond-0001) which originally supported sequence length 512.
|
11 |
+
**LSG atttention** is used to adapt existing pre-trained model to efficiently extrapolate to 4046 sequence length with no additional training.
|
12 |
+
|
13 |
+
notdiamond-0001 automatically determines whether to send queries to GPT-3.5 or GPT-4, depending on which model is best-suited for your task. notdiamond-0001 was trained on hundreds of thousands of data points from robust, cross-domain evaluation benchmarks.
|
14 |
+
The notdiamond-0001 router model is a classifier and will return a label for either GPT-3.5 or GPT-4.
|
15 |
+
|
16 |
+
|
17 |
+
Inference:
|
18 |
+
``` python
|
19 |
+
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
20 |
+
|
21 |
+
# input format
|
22 |
+
query = "Can you write a function that counts from 1 to 10?"
|
23 |
+
formatted_prompt = f"""Determine whether the following query should be sent to GPT-3.5 or GPT-4.
|
24 |
+
Query:
|
25 |
+
{query}"""
|
26 |
+
|
27 |
+
tokenizer = AutoTokenizer.from_pretrained("notdiamond/notdiamond-0001")
|
28 |
+
model = AutoModelForSequenceClassification.from_pretrained("notdiamond/notdiamond-0001")
|
29 |
+
|
30 |
+
inputs = tokenizer(formatted_prompt, truncation=True, max_length=4096, return_tensors="pt")
|
31 |
+
logits = model(**inputs).logits
|
32 |
+
|
33 |
+
model_id = logits.argmax().item()
|
34 |
+
id2label = {0: 'gpt-3.5', 1: 'gpt-4'}
|
35 |
+
model_to_call = id2label[model_id]
|
36 |
+
```
|
37 |
+
|
38 |
+
You can also access their free [API](https://www.notdiamond.ai/notdiamond-0001) and the official website : [documentation](https://notdiamond.readme.io/docs/introduction).
|