Spaces:

Mykes
/

rumed-phi3-mini

Sleeping

Mykes commited on Jul 15

Commit

f6b26e3

•

1 Parent(s): bad6434

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -53,8 +53,8 @@ def load_model():
         # repo_id="Mykes/med_tinyllama_gguf",
         filename="*Q4_K_M.gguf",
         verbose=False,
-        n_ctx=512,
-        n_batch=256,
         n_threads=8,
         use_mlock=True,
         use_mmap=True,

         # repo_id="Mykes/med_tinyllama_gguf",
         filename="*Q4_K_M.gguf",
         verbose=False,
+        n_ctx=256,
+        n_batch=128,
         n_threads=8,
         use_mlock=True,
         use_mmap=True,