doberst commited on
Commit
b94b408
1 Parent(s): 0f32a0e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -16
README.md CHANGED
@@ -12,6 +12,23 @@ BLING models are fine-tuned with distilled high-quality custom instruct datasets
12
  the objective of providing a high-quality Instruct model that is 'inference-ready' on a CPU laptop even
13
  without using any advanced quantization optimizations.
14
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
  ### Model Description
16
 
17
  <!-- Provide a longer summary of what this model is. -->
@@ -49,22 +66,6 @@ The first BLING models have been trained for common RAG scenarios, specifically:
49
  without the need for a lot of complex instruction verbiage - provide a text passage context, ask questions, and get clear fact-based responses.
50
 
51
 
52
- ### Benchmark Tests
53
-
54
- Evaluated against the benchmark test: [RAG-Instruct-Benchmark-Tester](https://www.huggingface.co/llmware/rag_instruct_benchmark_tester)
55
- Average of 2 Test Runs with 1 point for correct answer, 0.5 point for partial correct or blank / NF, 0.0 points for incorrect, and -1 points for hallucinations.
56
-
57
- --**Accuracy Score**: **73.25** correct out of 100
58
- --Not Found Classification: 17.5%
59
- --Boolean: 29%
60
- --Math/Logic: 0%
61
- --Complex Questions (1-5): 1 (Low)
62
- --Summarization Quality (1-5): 1 (Coherent, extractive)
63
- --Hallucinations: No hallucinations observed in test runs.
64
-
65
- For test run results, please see the files ("core_rag_test" and "answer_sheet" in the repo).
66
-
67
-
68
  ## Bias, Risks, and Limitations
69
 
70
  <!-- This section is meant to convey both technical and sociotechnical limitations. -->
 
12
  the objective of providing a high-quality Instruct model that is 'inference-ready' on a CPU laptop even
13
  without using any advanced quantization optimizations.
14
 
15
+
16
+ ### Benchmark Tests
17
+
18
+ Evaluated against the benchmark test: [RAG-Instruct-Benchmark-Tester](https://www.huggingface.co/datasets/llmware/rag_instruct_benchmark_tester)
19
+ Average of 2 Test Runs with 1 point for correct answer, 0.5 point for partial correct or blank / NF, 0.0 points for incorrect, and -1 points for hallucinations.
20
+
21
+ --**Accuracy Score**: **73.25** correct out of 100
22
+ --Not Found Classification: 17.5%
23
+ --Boolean: 29%
24
+ --Math/Logic: 0%
25
+ --Complex Questions (1-5): 1 (Low)
26
+ --Summarization Quality (1-5): 1 (Coherent, extractive)
27
+ --Hallucinations: No hallucinations observed in test runs.
28
+
29
+ For test run results, please see the files ("core_rag_test" and "answer_sheet" in the repo).
30
+
31
+
32
  ### Model Description
33
 
34
  <!-- Provide a longer summary of what this model is. -->
 
66
  without the need for a lot of complex instruction verbiage - provide a text passage context, ask questions, and get clear fact-based responses.
67
 
68
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
69
  ## Bias, Risks, and Limitations
70
 
71
  <!-- This section is meant to convey both technical and sociotechnical limitations. -->