sylwia-kuros commited on
Commit
b66d3f7
·
verified ·
1 Parent(s): c8bf64d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -10
README.md CHANGED
@@ -2,23 +2,19 @@
2
  license: apache-2.0
3
  ---
4
 
5
- <!-- Model name used as model card title -->
6
  # TinyLlama-1.1B-Chat-v1.0-int4-ov
7
- <!-- Original model reference -->
8
  * Model creator: [TinyLlama](https://huggingface.co/TinyLlama)
9
  * Original model: [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0)
10
 
11
- <!-- Description of converted model -->
12
  ## Description
13
 
14
- <!-- Comment and reference on NNCF applicable only for INT8 and INT4 models -->
15
  This is [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) model converted to the [OpenVINO™ IR](https://docs.openvino.ai/2024/documentation/openvino-ir-format.html) (Intermediate Representation) format with weights compressed to INT4 by [NNCF](https://github.com/openvinotoolkit/nncf).
16
 
17
  ## Quantization Parameters
18
 
19
  Weight compression was performed using `nncf.compress_weights` with the following parameters:
20
 
21
-
22
  * mode: **INT4_SYM**
23
  * group_size: **128**
24
  * ratio: **1.0**
@@ -34,8 +30,6 @@ The provided OpenVINO™ IR model is compatible with:
34
 
35
  ## Running Model Inference
36
 
37
- <!-- Example model usage -->
38
-
39
  1. Install packages required for using [Optimum Intel](https://huggingface.co/docs/optimum/intel/index) integration with the OpenVINO backend:
40
 
41
  ```
@@ -44,8 +38,6 @@ pip install optimum[openvino]
44
 
45
  2. Run model inference:
46
 
47
- <!-- Usage example can be adopted from original model usage example -->
48
-
49
  ```
50
  from transformers import AutoTokenizer
51
  from optimum.intel.openvino import OVModelForCausalLM
@@ -65,7 +57,6 @@ For more examples and possible optimizations, refer to the [OpenVINO Large Langu
65
 
66
  ## Legal information
67
 
68
- <!-- Note about original model license -->
69
  The original model is distributed under [apache-2.0](https://choosealicense.com/licenses/apache-2.0/) license. More details can be found in [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0).
70
 
71
  ## Disclaimer
 
2
  license: apache-2.0
3
  ---
4
 
 
5
  # TinyLlama-1.1B-Chat-v1.0-int4-ov
6
+
7
  * Model creator: [TinyLlama](https://huggingface.co/TinyLlama)
8
  * Original model: [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0)
9
 
 
10
  ## Description
11
 
 
12
  This is [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) model converted to the [OpenVINO™ IR](https://docs.openvino.ai/2024/documentation/openvino-ir-format.html) (Intermediate Representation) format with weights compressed to INT4 by [NNCF](https://github.com/openvinotoolkit/nncf).
13
 
14
  ## Quantization Parameters
15
 
16
  Weight compression was performed using `nncf.compress_weights` with the following parameters:
17
 
 
18
  * mode: **INT4_SYM**
19
  * group_size: **128**
20
  * ratio: **1.0**
 
30
 
31
  ## Running Model Inference
32
 
 
 
33
  1. Install packages required for using [Optimum Intel](https://huggingface.co/docs/optimum/intel/index) integration with the OpenVINO backend:
34
 
35
  ```
 
38
 
39
  2. Run model inference:
40
 
 
 
41
  ```
42
  from transformers import AutoTokenizer
43
  from optimum.intel.openvino import OVModelForCausalLM
 
57
 
58
  ## Legal information
59
 
 
60
  The original model is distributed under [apache-2.0](https://choosealicense.com/licenses/apache-2.0/) license. More details can be found in [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0).
61
 
62
  ## Disclaimer