Spaces:

raannakasturi
/

hybrid-researchpaper-summarizer

Sleeping

raannakasturi commited on Nov 16, 2024

Commit

33dc0c0

verified ·

1 Parent(s): 757c1f5

Update tools.py

Files changed (1) hide show

tools.py CHANGED Viewed

@@ -24,7 +24,8 @@ def load_llm_model():
         	repo_id="bartowski/Llama-3.2-1B-Instruct-GGUF",
         	filename="Llama-3.2-1B-Instruct-Q8_0.gguf",
             n_ctx=50000,
-            n_batch=1024,
         )
         print("LLM model loaded successfully")
         return llm

         	repo_id="bartowski/Llama-3.2-1B-Instruct-GGUF",
         	filename="Llama-3.2-1B-Instruct-Q8_0.gguf",
             n_ctx=50000,
+            n_batch=16384,
+            verbose=False,
         )
         print("LLM model loaded successfully")
         return llm