I should probably tell people what this is.
Browse files
README.md
CHANGED
@@ -1,3 +1,15 @@
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: apache-2.0
|
3 |
---
|
4 |
+
|
5 |
+
# Gemma 2 - Inference Endpoint
|
6 |
+
|
7 |
+
Hello! I wrote a simple container that allows for easy running of llama-cpp-python with GGUF models. My goal here was a cheap way to play with Gemma, but then I thought maybe i'd share just in case it's helpful. I'll probably make a bunch of these, so if you have any requests for GGUF or otherwise quantized Llama.cpp models to become inference endpoints, please feel free to reach out!
|
8 |
+
|
9 |
+
# Files
|
10 |
+
|
11 |
+
I used the excellent quant by [lmstudio-ai/gemma-2b-it-GGUF](https://huggingface.co/lmstudio-ai/gemma-2b-it-GGUF),
|
12 |
+
|
13 |
+
My email is newp@justkidding.net
|
14 |
+
|
15 |
+
Just kidding, it's sam att samuellmeyers DOT... com
|