SSH_355M-GGUF / README.md
morriszms's picture
Upload folder using huggingface_hub
746dce9 verified
metadata
license: apache-2.0
datasets:
  - databricks/databricks-dolly-15k
  - gamino/wiki_medical_terms
  - Sharathhebbar24/openhermes
  - Sharathhebbar24/Open-Platypus
  - Sharathhebbar24/sql-create-context
  - Sharathhebbar24/Evol-Instruct-Code-80k-v1
  - Sharathhebbar24/BeaverTails_filtered
  - Sharathhebbar24/arxiv-math-instruct-50k
  - Sharathhebbar24/MetaMathQA
  - Intel/orca_dpo_pairs
base_model: Sharathhebbar24/SSH_355M
tags:
  - TensorBlock
  - GGUF
model-index:
  - name: SSH_300M
    results:
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: AI2 Reasoning Challenge (25-Shot)
          type: ai2_arc
          config: ARC-Challenge
          split: test
          args:
            num_few_shot: 25
        metrics:
          - type: acc_norm
            value: 28.24
            name: normalized accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Sharathhebbar24/SSH_300M
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: HellaSwag (10-Shot)
          type: hellaswag
          split: validation
          args:
            num_few_shot: 10
        metrics:
          - type: acc_norm
            value: 38.74
            name: normalized accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Sharathhebbar24/SSH_300M
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: MMLU (5-Shot)
          type: cais/mmlu
          config: all
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 27.03
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Sharathhebbar24/SSH_300M
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: TruthfulQA (0-shot)
          type: truthful_qa
          config: multiple_choice
          split: validation
          args:
            num_few_shot: 0
        metrics:
          - type: mc2
            value: 42.51
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Sharathhebbar24/SSH_300M
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: Winogrande (5-shot)
          type: winogrande
          config: winogrande_xl
          split: validation
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 53.67
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Sharathhebbar24/SSH_300M
          name: Open LLM Leaderboard
      - task:
          type: text-generation
          name: Text Generation
        dataset:
          name: GSM8k (5-shot)
          type: gsm8k
          config: main
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: acc
            value: 0.3
            name: accuracy
        source:
          url: >-
            https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Sharathhebbar24/SSH_300M
          name: Open LLM Leaderboard
TensorBlock

Feedback and support: TensorBlock's Twitter/X, Telegram Group and Discord server

Sharathhebbar24/SSH_355M - GGUF

This repo contains GGUF format model files for Sharathhebbar24/SSH_355M.

The files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4242.

Prompt template


Model file specification

Filename Quant type File Size Description
SSH_355M-Q2_K.gguf Q2_K 0.178 GB smallest, significant quality loss - not recommended for most purposes
SSH_355M-Q3_K_S.gguf Q3_K_S 0.201 GB very small, high quality loss
SSH_355M-Q3_K_M.gguf Q3_K_M 0.229 GB very small, high quality loss
SSH_355M-Q3_K_L.gguf Q3_K_L 0.244 GB small, substantial quality loss
SSH_355M-Q4_0.gguf Q4_0 0.248 GB legacy; small, very high quality loss - prefer using Q3_K_M
SSH_355M-Q4_K_S.gguf Q4_K_S 0.250 GB small, greater quality loss
SSH_355M-Q4_K_M.gguf Q4_K_M 0.271 GB medium, balanced quality - recommended
SSH_355M-Q5_0.gguf Q5_0 0.292 GB legacy; medium, balanced quality - prefer using Q4_K_M
SSH_355M-Q5_K_S.gguf Q5_K_S 0.292 GB large, low quality loss - recommended
SSH_355M-Q5_K_M.gguf Q5_K_M 0.309 GB large, very low quality loss - recommended
SSH_355M-Q6_K.gguf Q6_K 0.339 GB very large, extremely low quality loss
SSH_355M-Q8_0.gguf Q8_0 0.437 GB very large, extremely low quality loss - not recommended

Downloading instruction

Command line

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, downoad the individual model file the a local directory

huggingface-cli download tensorblock/SSH_355M-GGUF --include "SSH_355M-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you wanna download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/SSH_355M-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'