doberst commited on
Commit
23d6cbb
1 Parent(s): ea83792

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +16 -0
README.md CHANGED
@@ -8,6 +8,22 @@ BLING-QWEN-MINI-TOOL (1.5B)
8
 
9
  **bling-qwen-mini-tool** is a RAG-finetuned version on Qwen2-1.5B for use in fact-based context question-answering, packaged with 4_K_M GGUF quantization, providing a very fast, very small inference implementation for use on CPUs.
10
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  To pull the model via API:
12
 
13
  from huggingface_hub import snapshot_download
 
8
 
9
  **bling-qwen-mini-tool** is a RAG-finetuned version on Qwen2-1.5B for use in fact-based context question-answering, packaged with 4_K_M GGUF quantization, providing a very fast, very small inference implementation for use on CPUs.
10
 
11
+
12
+ ## Benchmark Tests
13
+
14
+ Evaluated against the benchmark test: RAG-Instruct-Benchmark-Tester
15
+ 1 Test Run with sample=False & temperature=0.0 (deterministic output) - 1 point for correct answer, 0.5 point for partial correct or blank / NF, 0.0 points for incorrect, and -1 points for hallucinations.
16
+
17
+ --Accuracy Score: **93.5** correct out of 100
18
+ --Not Found Classification: 75.0%
19
+ --Boolean: 87.5%
20
+ --Math/Logic: 70.0%
21
+ --Complex Questions (1-5): 3 (Average)
22
+ --Summarization Quality (1-5): 3 (Average)
23
+ --Hallucinations: No hallucinations observed in test runs.
24
+
25
+ For test run results (and good indicator of target use cases), please see the files ("core_rag_test" and "answer_sheet" in this repo).
26
+
27
  To pull the model via API:
28
 
29
  from huggingface_hub import snapshot_download