doberst commited on
Commit
53dca12
1 Parent(s): 3a663ad

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -1
README.md CHANGED
@@ -12,6 +12,24 @@ BLING models are fine-tuned with distilled high-quality custom instruct datasets
12
  the objective of providing a high-quality Instruct model that is 'inference-ready' on a CPU laptop even
13
  without using any advanced quantization optimizations.
14
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
  ### Model Description
16
 
17
  <!-- Provide a longer summary of what this model is. -->
@@ -94,7 +112,7 @@ This BLING model was built on top of a "Sheared Llama" model base - for more inf
94
 
95
  Darren Oberst & llmware team
96
 
97
- Please reach out anytime if you are interested in this project and would like to participate and work with us!
98
 
99
 
100
 
 
12
  the objective of providing a high-quality Instruct model that is 'inference-ready' on a CPU laptop even
13
  without using any advanced quantization optimizations.
14
 
15
+
16
+ ### Benchmark Tests
17
+
18
+ Evaluated against the benchmark test: [RAG-Instruct-Benchmark-Tester](https://www.huggingface.co/datasets/llmware/rag_instruct_benchmark_tester)
19
+ Average of 2 Test Runs with 1 point for correct answer, 0.5 point for partial correct or blank / NF, 0.0 points for incorrect, and -1 points for hallucinations.
20
+
21
+ --**Accuracy Score**: **84.50** correct out of 100
22
+ --Not Found Classification: 20.0%
23
+ --Boolean: 66.25%
24
+ --Math/Logic: 9.4%
25
+ --Complex Questions (1-5): 1 (Low)
26
+ --Summarization Quality (1-5): 3 (Coherent, extractive)
27
+ --Hallucinations: No hallucinations observed in test runs.
28
+
29
+ For test run results (and good indicator of target use cases), please see the files ("core_rag_test" and "answer_sheet" in this repo).
30
+
31
+
32
+
33
  ### Model Description
34
 
35
  <!-- Provide a longer summary of what this model is. -->
 
112
 
113
  Darren Oberst & llmware team
114
 
115
+ Please reach out anytime if you are interested in this project!
116
 
117
 
118