Hugging Face's logo

Sagicc
/
granite-8b-code-instruct-Q5_K_M-GGUF

Text Generation
Transformers
GGUF
code
llama-cpp
gguf-my-repo
Eval Results
conversational
Model card Files Community
granite-8b-code-instruct-Q5_K_M-GGUF / README.md
Sagicc's picture
Sagicc
Update README.md
a172c3a verified
preview code
|
raw
history blame
2.97 kB
metadata
license: apache-2.0
library_name: transformers
tags:
  - code
  - llama-cpp
  - gguf-my-repo
base_model: ibm-granite/granite-8b-code-base
datasets:
  - bigcode/commitpackft
  - TIGER-Lab/MathInstruct
  - meta-math/MetaMathQA
  - glaiveai/glaive-code-assistant-v3
  - glaive-function-calling-v2
  - bugdaryan/sql-create-context-instruction
  - garage-bAInd/Open-Platypus
  - nvidia/HelpSteer
metrics:
  - code_eval
pipeline_tag: text-generation
inference: false
model-index:
  - name: granite-8b-code-instruct
    results:
      - task:
          type: text-generation
        dataset:
          name: HumanEvalSynthesis(Python)
          type: bigcode/humanevalpack
        metrics:
          - type: pass@1
            value: 57.9
            name: pass@1
          - type: pass@1
            value: 52.4
            name: pass@1
          - type: pass@1
            value: 58.5
            name: pass@1
          - type: pass@1
            value: 43.3
            name: pass@1
          - type: pass@1
            value: 48.2
            name: pass@1
          - type: pass@1
            value: 37.2
            name: pass@1
          - type: pass@1
            value: 53
            name: pass@1
          - type: pass@1
            value: 42.7
            name: pass@1
          - type: pass@1
            value: 52.4
            name: pass@1
          - type: pass@1
            value: 36.6
            name: pass@1
          - type: pass@1
            value: 43.9
            name: pass@1
          - type: pass@1
            value: 16.5
            name: pass@1
          - type: pass@1
            value: 39.6
            name: pass@1
          - type: pass@1
            value: 40.9
            name: pass@1
          - type: pass@1
            value: 48.2
            name: pass@1
          - type: pass@1
            value: 41.5
            name: pass@1
          - type: pass@1
            value: 39
            name: pass@1
          - type: pass@1
            value: 32.9
            name: pass@1

Sagicc/granite-8b-code-instruct-Q5_K_M-GGUF

This model was converted to GGUF format from ibm-granite/granite-8b-code-instruct using llama.cpp after addded support for small Granite Code models in b3026 'llama.cpp release'. Refer to the original model card for more details on the model.

For now only works with llama.cpp

Use with llama.cpp

Install llama.cpp through brew.

brew install ggerganov/ggerganov/llama.cpp

Invoke the llama.cpp server or the CLI.

CLI:

llama-cli --hf-repo Sagicc/granite-8b-code-instruct-Q5_K_M-GGUF --model granite-8b-code-instruct.Q5_K_M.gguf -p "You are an AI assistant"

Server:

llama-server --hf-repo Sagicc/granite-8b-code-instruct-Q5_K_M-GGUF --model granite-8b-code-instruct.Q5_K_M.gguf -c 2048

Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.

git clone https://github.com/ggerganov/llama.cpp &&             cd llama.cpp &&             make &&             ./main -m granite-8b-code-instruct.Q5_K_M.gguf -n 128