MrOvkill commited on
Commit
ce01a37
·
1 Parent(s): 9afbac2

I should probably tell people what this is.

Browse files
Files changed (1) hide show
  1. README.md +12 -0
README.md CHANGED
@@ -1,3 +1,15 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+
5
+ # Gemma 2 - Inference Endpoint
6
+
7
+ Hello! I wrote a simple container that allows for easy running of llama-cpp-python with GGUF models. My goal here was a cheap way to play with Gemma, but then I thought maybe i'd share just in case it's helpful. I'll probably make a bunch of these, so if you have any requests for GGUF or otherwise quantized Llama.cpp models to become inference endpoints, please feel free to reach out!
8
+
9
+ # Files
10
+
11
+ I used the excellent quant by [lmstudio-ai/gemma-2b-it-GGUF](https://huggingface.co/lmstudio-ai/gemma-2b-it-GGUF),
12
+
13
+ My email is newp@justkidding.net
14
+
15
+ Just kidding, it's sam att samuellmeyers DOT... com