update code snippet
Browse files
README.md
CHANGED
@@ -10,6 +10,7 @@ tags:
|
|
10 |
- text-classification
|
11 |
- neural-compressor
|
12 |
- int8
|
|
|
13 |
---
|
14 |
|
15 |
# Dynamically quantized DistilBERT base uncased finetuned SST-2
|
@@ -27,14 +28,17 @@ tags:
|
|
27 |
|
28 |
## How to Get Started With the Model
|
29 |
|
|
|
|
|
|
|
30 |
To load the quantized model and run inference using the Transformers [pipelines](https://huggingface.co/docs/transformers/main/en/main_classes/pipelines), you can do as follows:
|
31 |
|
32 |
```python
|
33 |
from transformers import AutoTokenizer, pipeline
|
34 |
-
from optimum.intel
|
35 |
|
36 |
model_id = "echarlaix/distilbert-base-uncased-finetuned-sst-2-english-int8-dynamic"
|
37 |
-
model =
|
38 |
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
39 |
cls_pipe = pipeline("text-classification", model=model, tokenizer=tokenizer)
|
40 |
text = "He's a dreadful magician."
|
|
|
10 |
- text-classification
|
11 |
- neural-compressor
|
12 |
- int8
|
13 |
+
- 8-bit
|
14 |
---
|
15 |
|
16 |
# Dynamically quantized DistilBERT base uncased finetuned SST-2
|
|
|
28 |
|
29 |
## How to Get Started With the Model
|
30 |
|
31 |
+
This requires to install Optimum :
|
32 |
+
`pip install optimum[neural-compressor]`
|
33 |
+
|
34 |
To load the quantized model and run inference using the Transformers [pipelines](https://huggingface.co/docs/transformers/main/en/main_classes/pipelines), you can do as follows:
|
35 |
|
36 |
```python
|
37 |
from transformers import AutoTokenizer, pipeline
|
38 |
+
from optimum.intel import INCModelForSequenceClassification
|
39 |
|
40 |
model_id = "echarlaix/distilbert-base-uncased-finetuned-sst-2-english-int8-dynamic"
|
41 |
+
model = INCModelForSequenceClassification.from_pretrained(model_id)
|
42 |
tokenizer = AutoTokenizer.from_pretrained(model_id)
|
43 |
cls_pipe = pipeline("text-classification", model=model, tokenizer=tokenizer)
|
44 |
text = "He's a dreadful magician."
|