AIronMind commited on
Commit
c605067
1 Parent(s): 6f6aa63

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +158 -0
README.md ADDED
@@ -0,0 +1,158 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: text-generation
3
+ inference: false
4
+ license: apache-2.0
5
+ datasets:
6
+ - codeparrot/github-code-clean
7
+ - bigcode/starcoderdata
8
+ - open-web-math/open-web-math
9
+ - math-ai/StackMathQA
10
+ metrics:
11
+ - code_eval
12
+ library_name: transformers
13
+ tags:
14
+ - code
15
+ - granite
16
+ - llama-cpp
17
+ - gguf-my-repo
18
+ base_model: ibm-granite/granite-3b-code-base-128k
19
+ model-index:
20
+ - name: granite-3b-code-base-128k
21
+ results:
22
+ - task:
23
+ type: text-generation
24
+ dataset:
25
+ name: HumanEvalSynthesis (Python)
26
+ type: bigcode/humanevalpack
27
+ metrics:
28
+ - type: pass@1
29
+ value: 36.0
30
+ name: pass@1
31
+ verified: false
32
+ - type: pass@1
33
+ value: 30.5
34
+ name: pass@1
35
+ verified: false
36
+ - type: pass@1
37
+ value: 22.4
38
+ name: pass@1
39
+ verified: false
40
+ - type: pass@1
41
+ value: 19.9
42
+ name: pass@1
43
+ verified: false
44
+ - task:
45
+ type: text-generation
46
+ dataset:
47
+ name: RepoQA (Python@16K)
48
+ type: repoqa
49
+ metrics:
50
+ - type: pass@1 (thresh=0.5)
51
+ value: 40.0
52
+ name: pass@1 (thresh=0.5)
53
+ verified: false
54
+ - type: pass@1 (thresh=0.5)
55
+ value: 36.0
56
+ name: pass@1 (thresh=0.5)
57
+ verified: false
58
+ - type: pass@1 (thresh=0.5)
59
+ value: 37.0
60
+ name: pass@1 (thresh=0.5)
61
+ verified: false
62
+ - type: pass@1 (thresh=0.5)
63
+ value: 27.0
64
+ name: pass@1 (thresh=0.5)
65
+ verified: false
66
+ - type: pass@1 (thresh=0.5)
67
+ value: 29.0
68
+ name: pass@1 (thresh=0.5)
69
+ verified: false
70
+ - task:
71
+ type: text-generation
72
+ dataset:
73
+ name: LCC (Balanced)
74
+ type: lcc
75
+ metrics:
76
+ - type: Exact Match@4K
77
+ value: 54.6
78
+ name: Exact Match@4K
79
+ verified: false
80
+ - type: Exact Match@8K
81
+ value: 56.8
82
+ name: Exact Match@8K
83
+ verified: false
84
+ - type: Exact Match@16K
85
+ value: 52.2
86
+ name: Exact Match@16K
87
+ verified: false
88
+ - type: Exact Match@32K
89
+ value: 57.8
90
+ name: Exact Match@32K
91
+ verified: false
92
+ - task:
93
+ type: text-generation
94
+ dataset:
95
+ name: RepoBench-P (Balanced)
96
+ type: repobench
97
+ metrics:
98
+ - type: Exact Match@4K
99
+ value: 39.8
100
+ name: Exact Match@4K
101
+ verified: false
102
+ - type: Exact Match@8K
103
+ value: 46.8
104
+ name: Exact Match@8K
105
+ verified: false
106
+ - type: Exact Match@16K
107
+ value: 43.1
108
+ name: Exact Match@16K
109
+ verified: false
110
+ - type: Exact Match@32K
111
+ value: 45.3
112
+ name: Exact Match@32K
113
+ verified: false
114
+ ---
115
+
116
+ # AIronMind/granite-3b-code-base-128k-Q4_K_M-GGUF
117
+ This model was converted to GGUF format from [`ibm-granite/granite-3b-code-base-128k`](https://huggingface.co/ibm-granite/granite-3b-code-base-128k) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
118
+ Refer to the [original model card](https://huggingface.co/ibm-granite/granite-3b-code-base-128k) for more details on the model.
119
+
120
+ ## Use with llama.cpp
121
+ Install llama.cpp through brew (works on Mac and Linux)
122
+
123
+ ```bash
124
+ brew install llama.cpp
125
+
126
+ ```
127
+ Invoke the llama.cpp server or the CLI.
128
+
129
+ ### CLI:
130
+ ```bash
131
+ llama-cli --hf-repo AIronMind/granite-3b-code-base-128k-Q4_K_M-GGUF --hf-file granite-3b-code-base-128k-q4_k_m.gguf -p "The meaning to life and the universe is"
132
+ ```
133
+
134
+ ### Server:
135
+ ```bash
136
+ llama-server --hf-repo AIronMind/granite-3b-code-base-128k-Q4_K_M-GGUF --hf-file granite-3b-code-base-128k-q4_k_m.gguf -c 2048
137
+ ```
138
+
139
+ Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
140
+
141
+ Step 1: Clone llama.cpp from GitHub.
142
+ ```
143
+ git clone https://github.com/ggerganov/llama.cpp
144
+ ```
145
+
146
+ Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
147
+ ```
148
+ cd llama.cpp && LLAMA_CURL=1 make
149
+ ```
150
+
151
+ Step 3: Run inference through the main binary.
152
+ ```
153
+ ./llama-cli --hf-repo AIronMind/granite-3b-code-base-128k-Q4_K_M-GGUF --hf-file granite-3b-code-base-128k-q4_k_m.gguf -p "The meaning to life and the universe is"
154
+ ```
155
+ or
156
+ ```
157
+ ./llama-server --hf-repo AIronMind/granite-3b-code-base-128k-Q4_K_M-GGUF --hf-file granite-3b-code-base-128k-q4_k_m.gguf -c 2048
158
+ ```