File size: 1,770 Bytes
8239221 8695fc1 8239221 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 |
---
license: apache-2.0
language:
- en
tags:
- rene
- mamba
- mlx
- cartesia
library_name: cartesia_mlx
---
# Model Card for Rene-v0.1-1.3b-4bit-mlx
This is an [MLX](https://ml-explore.github.io/mlx)-compatible version of the [Rene-v0.1-1.3b](https://huggingface.co/cartesia-ai/Rene-v0.1-1.3b-pytorch) model, quantized to 4 bits. It uses the [allenai/OLMo-1B-hf](https://huggingface.co/allenai/OLMo-1B-hf) tokenizer.
For more details, see our [blog post](https://cartesia.ai/blog/on-device).
## Usage
### Installation
This model requires the `cartesia-metal` and `cartesia-mlx` packages.
Installation requires Xcode, which can be downloaded from https://developer.apple.com/xcode/. Accept the license agreement with:
```shell
sudo xcodebuild -license
```
Install the required dependencies: the exact version of `nanobind`, followed by `cartesia-metal`, and finally `cartesia-mlx`, with the following commands:
```shell
pip install nanobind@git+https://github.com/wjakob/nanobind.git@2f04eac452a6d9142dedb957701bdb20125561e4
pip install git+https://github.com/cartesia-ai/edge.git#subdirectory=cartesia-metal
pip install cartesia-mlx
```
Note: This package has been tested on macOS Sonoma 14.1 with the M3 chip.
### Generation example
```python
import mlx.core as mx
import cartesia_mlx as cmx
model = cmx.from_pretrained("cartesia-ai/Rene-v0.1-1.3b-4bit-mlx")
model.set_dtype(mx.float32)
prompt = "Rene Descartes was"
print(prompt, end="", flush=True)
for text in model.generate(
prompt,
max_tokens=500,
eval_every_n=5,
verbose=True,
top_p=0.99,
temperature=0.85,
):
print(text, end="", flush=True)
```
## About Cartesia
At [Cartesia](https://cartesia.ai/), we're building real-time multimodal intelligence for every device.
|