avemio-digital commited on
Commit
0ebb723
·
verified ·
1 Parent(s): faa0d79

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -0
README.md CHANGED
@@ -111,6 +111,21 @@ SentenceTransformer(
111
  | MTOPIntentClassification | 0.6808 | 0.6684 | **0.7143** | -1.82% | 4.91% | 6.87% |
112
  | PawsXPairClassification | 0.5678 | 0.5710 | **0.5803** | 0.56% | 2.18% | 1.63% |
113
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
114
  ## Usage
115
 
116
  ### Direct Usage (Sentence Transformers)
 
111
  | MTOPIntentClassification | 0.6808 | 0.6684 | **0.7143** | -1.82% | 4.91% | 6.87% |
112
  | PawsXPairClassification | 0.5678 | 0.5710 | **0.5803** | 0.56% | 2.18% | 1.63% |
113
 
114
+ ## Evaluation on GRAG-EMBEDDING-BENCHMARK
115
+
116
+ Accuracy is calculated by evaluating if the relevant context is the highest ranking embedding of the whole context array.
117
+ See Eval-Dataset and Evaluation Code [here](https://huggingface.co/datasets/avemio/GRAG-EMBEDDING-BENCHMARK)
118
+
119
+ | Model Name | Accuracy |
120
+ |-------------------------------------------------|-----------|
121
+ | [bge-m3](https://huggingface.co/BAAI/bge-m3 ) | 0.8806 |
122
+ | [UAE-Large-V1](https://huggingface.co/WhereIsAI/UAE-Large-V1) | 0.8393 |
123
+ | [GRAG-BGE-M3-TRIPLES-HESSIAN-AI](https://huggingface.co/avemio/GRAG-BGE-M3-TRIPLES-HESSIAN-AI) | 0.8857 |
124
+ | [GRAG-BGE-M3-TRIPLES-MERGED-HESSIAN-AI](https://huggingface.co/avemio/GRAG-BGE-M3-TRIPLES-MERGED-HESSIAN-AI) | **0.8866** |
125
+ | [GRAG-BGE-M3-MERGED-x-SNOWFLAKE-ARCTIC-HESSIAN-AI](https://huggingface.co/avemio/GRAG-BGE-M3-MERGED-x-SNOWFLAKE-ARCTIC-HESSIAN-AI) | **0.8866** |
126
+ | [GRAG-UAE-LARGE-V1-TRIPLES-HESSIAN-AI](https://huggingface.co/avemio/GRAG-UAE-LARGE-V1-TRIPLES-HESSIAN-AI) | 0.8763 |
127
+ | [GRAG-UAE-LARGE-V1-TRIPLES-MERGED-HESSIAN-AI](https://huggingface.co/avemio/GRAG-UAE-LARGE-V1-TRIPLES-MERGED-HESSIAN-AI) | 0.8771 |
128
+
129
  ## Usage
130
 
131
  ### Direct Usage (Sentence Transformers)