Hugging Face's logo

tensorblock
/
Aira-2-1B5-GGUF

Text Generation
Transformers
GGUF
English
alignment
instruction tuned
text generation
conversation
assistant
TensorBlock
GGUF
Inference Endpoints
Model card Files Community
Aira-2-1B5-GGUF / README.md
morriszms's picture
morriszms
Upload folder using huggingface_hub
98a38b0 verified
preview code
|
raw
history blame contribute delete
5.29 kB
metadata
license: apache-2.0
datasets:
  - nicholasKluge/instruct-aira-dataset
language:
  - en
metrics:
  - accuracy
library_name: transformers
tags:
  - alignment
  - instruction tuned
  - text generation
  - conversation
  - assistant
  - TensorBlock
  - GGUF
pipeline_tag: text-generation
widget:
  - text: >-
      <|startofinstruction|>Can you explain what is Machine
      Learning?<|endofinstruction|>
    example_title: Machine Learning
  - text: >-
      <|startofinstruction|>Do you know anything about virtue
      ethics?<|endofinstruction|>
    example_title: Ethics
  - text: >-
      <|startofinstruction|>How can I make my girlfriend
      happy?<|endofinstruction|>
    example_title: Advise
inference:
  parameters:
    repetition_penalty: 1.2
    temperature: 0.2
    top_k: 30
    top_p: 0.3
    max_new_tokens: 200
    length_penalty: 0.3
    early_stopping: true
co2_eq_emissions:
  emissions: 1690
  source: CodeCarbon
  training_type: fine-tuning
  geographical_location: United States of America
  hardware_used: NVIDIA A100-SXM4-40GB
base_model: nicholasKluge/Aira-2-1B5
TensorBlock

Feedback and support: TensorBlock's Twitter/X, Telegram Group and Discord server

nicholasKluge/Aira-2-1B5 - GGUF

This repo contains GGUF format model files for nicholasKluge/Aira-2-1B5.

The files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4011.

Prompt template

Model file specification

Filename Quant type File Size Description
Aira-2-1B5-Q2_K.gguf Q2_K 0.845 GB smallest, significant quality loss - not recommended for most purposes
Aira-2-1B5-Q3_K_S.gguf Q3_K_S 0.845 GB very small, high quality loss
Aira-2-1B5-Q3_K_M.gguf Q3_K_M 0.966 GB very small, high quality loss
Aira-2-1B5-Q3_K_L.gguf Q3_K_L 1.027 GB small, substantial quality loss
Aira-2-1B5-Q4_0.gguf Q4_0 0.906 GB legacy; small, very high quality loss - prefer using Q3_K_M
Aira-2-1B5-Q4_K_S.gguf Q4_K_S 1.037 GB small, greater quality loss
Aira-2-1B5-Q4_K_M.gguf Q4_K_M 1.110 GB medium, balanced quality - recommended
Aira-2-1B5-Q5_0.gguf Q5_0 1.087 GB legacy; medium, balanced quality - prefer using Q4_K_M
Aira-2-1B5-Q5_K_S.gguf Q5_K_S 1.149 GB large, low quality loss - recommended
Aira-2-1B5-Q5_K_M.gguf Q5_K_M 1.286 GB large, very low quality loss - recommended
Aira-2-1B5-Q6_K.gguf Q6_K 1.519 GB very large, extremely low quality loss
Aira-2-1B5-Q8_0.gguf Q8_0 1.630 GB very large, extremely low quality loss - not recommended

Downloading instruction

Command line

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, downoad the individual model file the a local directory

huggingface-cli download tensorblock/Aira-2-1B5-GGUF --include "Aira-2-1B5-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you wanna download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/Aira-2-1B5-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'