replit-code-v1_5-3b-GGUF

Runtime error

limcheekin commited on Nov 26, 2023

Commit

ec314a6

1 Parent(s): 3cfa708

feat: updated from F16 to Q8_0 to fix slow loading

Files changed (3) hide show

Dockerfile CHANGED Viewed

@@ -15,7 +15,7 @@ RUN pip install -U pip setuptools wheel && \
 # Download model
 RUN mkdir model && \
-    curl -L https://huggingface.co/abetlen/replit-code-v1_5-3b-GGUF/resolve/main/replit-code-v1_5-3b.f16.gguf -o model/gguf-model.bin
 COPY ./start_server.sh ./
 COPY ./main.py ./

 # Download model
 RUN mkdir model && \
+    curl -L https://huggingface.co/abetlen/replit-code-v1_5-3b-GGUF/resolve/main/replit-code-v1_5-3b.Q8_0.gguf -o model/gguf-model.bin
 COPY ./start_server.sh ./
 COPY ./main.py ./

README.md CHANGED Viewed

@@ -1,5 +1,5 @@
 ---
-title: replit-code-v1_5-3b-GGUF (F16)
 colorFrom: purple
 colorTo: blue
 sdk: docker
@@ -15,6 +15,6 @@ tags:
 pinned: false
 ---
-# replit-code-v1_5-3b-GGUF (F16)
 Please refer to the [index.html](index.html) for more information.

 ---
+title: replit-code-v1_5-3b-GGUF (Q8_0)
 colorFrom: purple
 colorTo: blue
 sdk: docker
 pinned: false
 ---
+# replit-code-v1_5-3b-GGUF (Q8_0)
 Please refer to the [index.html](index.html) for more information.

index.html CHANGED Viewed

@@ -1,10 +1,10 @@
 <!DOCTYPE html>
 <html>
   <head>
-    <title>replit-code-v1_5-3b-GGUF (F16)</title>
   </head>
   <body>
-    <h1>replit-code-v1_5-3b-GGUF (F16)</h1>
     <p>
       With the utilization of the
       <a href="https://github.com/abetlen/llama-cpp-python">llama-cpp-python</a>

 <!DOCTYPE html>
 <html>
   <head>
+    <title>replit-code-v1_5-3b-GGUF (Q8_0)</title>
   </head>
   <body>
+    <h1>replit-code-v1_5-3b-GGUF (Q8_0)</h1>
     <p>
       With the utilization of the
       <a href="https://github.com/abetlen/llama-cpp-python">llama-cpp-python</a>