GGUF
Inference Endpoints
File size: 1,308 Bytes
42d9a98
ae51148
42d9a98
 
 
 
 
 
 
 
 
 
cdf2fd5
 
42d9a98
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
ae7c019
42d9a98
ae51148
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
---
license: gemma
---

## Overview

The [Gemma](https://huggingface.co/microsoft/Gemma-mini-4k-instruct), state-of-the-art open model trained with the Gemma datasets that includes both synthetic data and the filtered publicly available websites data with a focus on high-quality and reasoning dense properties. The model belongs to the Gemma family with the 4B, 7B version in two variants 8K and 128K which is the context length (in tokens) that it can support.

## Variants

| No | Variant | Cortex CLI command |
| --- | --- | --- |
| 1 | [7b-gguf](https://huggingface.co/cortexso/gemma/tree/7b-gguf) | `cortex run gemma:7b-gguf` |
| 2 | [7b-onnx](https://huggingface.co/cortexso/gemma/tree/7b-onnx) | `cortex run gemma:7b-onnx` |

## Use it with Jan (UI)

1. Install **Jan** using [Quickstart](https://jan.ai/docs/quickstart)
2. Use in Jan model Hub:
    ```
    cortexso/gemma
    ```
    
## Use it with Cortex (CLI)

1. Install **Cortex** using [Quickstart](https://cortex.jan.ai/docs/quickstart)
2. Run the model with command:
    ```
    cortex run gemma
    ```
    
## Credits

- **Author:** Go‌ogle
- **Converter:** [Homebrew](https://www.homebrew.ltd/)
- **Original License:** [License](https://ai.google.dev/gemma/terms)
- **Papers:** [Gemma Technical Report](https://arxiv.org/abs/2403.08295)