Create README.md

README.md

@@ -0,0 +1,41 @@
+---
+license: apache-2.0
+datasets:
+- lambada
+language:
+- en
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- text-generation-inference
+- causal-lm
+- int8
+- tensorrt
+- ENOT-AutoDL
+---
+
+# INT8 GPT-J 6B
+
+GPT-J 6B is a transformer model trained using Ben Wang's [Mesh Transformer JAX](https://github.com/kingoflolz/mesh-transformer-jax/). "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
+
+This repository contains tensorrt engines with mixed precission int8 + fp32. You can find prebuilded engines for next GPUs:
+* RTX 4090
+* RTX 3080 Ti
+* RTX 2080 Ti
+
+Source ONNX model generated by [ENOT-AutoDL](https://pypi.org/project/enot-autodl/) and will be published soon.
+
+## Test result
+
+|   |INT8|FP32|
+|---|:---:|:---:|
+| **Lambada Acc** |78.50%|79.54%|
+| **Model size (GB)**  |8.1|23|
+
+
+## How to use
+
+Example of inference and accuracy test published on github:
+```shell
+git clone https://github.com/ENOT-AutoDL/demo-gpt-j-6B-tensorrt-int8
+```