GGUF
Inference Endpoints
Pham commited on
Commit
42d9a98
1 Parent(s): 133b52c

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -0
README.md ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: ms-pl
3
+ ---
4
+
5
+ ## Overview
6
+
7
+ The [Gemma](https://huggingface.co/microsoft/Gemma-mini-4k-instruct), state-of-the-art open model trained with the Gemma datasets that includes both synthetic data and the filtered publicly available websites data with a focus on high-quality and reasoning dense properties. The model belongs to the Gemma family with the 4B, 7B version in two variants 8K and 128K which is the context length (in tokens) that it can support.
8
+
9
+ ## Variants
10
+
11
+ | No | Variant | Cortex CLI command |
12
+ | --- | --- | --- |
13
+ | 1 | [7B-gguf](https://huggingface.co/cortexso/gemma/tree/7B-gguf) | `cortex run gemma:7B-gguf` |
14
+ | 2 | [7B-onnx](https://huggingface.co/cortexso/gemma/tree/7B-onnx) | `cortex run gemma:7B-onnx` |
15
+
16
+ ## Use it with Jan (UI)
17
+
18
+ 1. Install **Jan** using [Quickstart](https://jan.ai/docs/quickstart)
19
+ 2. Use in Jan model Hub:
20
+ ```
21
+ cortexso/gemma
22
+ ```
23
+
24
+ ## Use it with Cortex (CLI)
25
+
26
+ 1. Install **Cortex** using [Quickstart](https://cortex.jan.ai/docs/quickstart)
27
+ 2. Run the model with command:
28
+ ```
29
+ cortex run gemma
30
+ ```
31
+
32
+ ## Credits
33
+
34
+ - **Author:** Microsoft
35
+ - **Converter:** [Homebrew](https://www.homebrew.ltd/)
36
+ - **Original License:** [License](https://huggingface.co/microsoft/Gemma-mini-4k-instruct/blob/main/LICENSE)
37
+ - **Papers:** [Gemma Technical Report](https://arxiv.org/abs/2404.14219)