Edit model card
YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

Bringing SOTA quantization to mobile LLM deployment: A practical Executorch integration guide

Article: https://blacksamorez.substack.com/p/aqlm-executorch-android

Usage

  • Download and install the .apk file on your Android phone. (llama3-aqlm.apk for ~1.1 tok/s at low power consumption. llama3-aqlm-4cores.apk for ~2.7 tok/s at high loads)
  • Download the .pte and .model files and put them into the /data/local/tmp/llama folder on your Android phone.
  • Running the app you will see the option to load the .pte and .model files. After loading them, you'll be able to chat with the model.

Requirements

This app was tested on Samsung S24 Ultra running Android 14.

Limitations

  • Although the app looks like chat, generation requests are independent.
  • Llama-3 chat template is hard-coded into the app.
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference API
Unable to determine this model's library. Check the docs .