avemio-digital
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -111,6 +111,21 @@ SentenceTransformer(
|
|
111 |
| MTOPIntentClassification | 0.6808 | 0.6684 | **0.7143** | -1.82% | 4.91% | 6.87% |
|
112 |
| PawsXPairClassification | 0.5678 | 0.5710 | **0.5803** | 0.56% | 2.18% | 1.63% |
|
113 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
114 |
## Usage
|
115 |
|
116 |
### Direct Usage (Sentence Transformers)
|
|
|
111 |
| MTOPIntentClassification | 0.6808 | 0.6684 | **0.7143** | -1.82% | 4.91% | 6.87% |
|
112 |
| PawsXPairClassification | 0.5678 | 0.5710 | **0.5803** | 0.56% | 2.18% | 1.63% |
|
113 |
|
114 |
+
## Evaluation on GRAG-EMBEDDING-BENCHMARK
|
115 |
+
|
116 |
+
Accuracy is calculated by evaluating if the relevant context is the highest ranking embedding of the whole context array.
|
117 |
+
See Eval-Dataset and Evaluation Code [here](https://huggingface.co/datasets/avemio/GRAG-EMBEDDING-BENCHMARK)
|
118 |
+
|
119 |
+
| Model Name | Accuracy |
|
120 |
+
|-------------------------------------------------|-----------|
|
121 |
+
| [bge-m3](https://huggingface.co/BAAI/bge-m3 ) | 0.8806 |
|
122 |
+
| [UAE-Large-V1](https://huggingface.co/WhereIsAI/UAE-Large-V1) | 0.8393 |
|
123 |
+
| [GRAG-BGE-M3-TRIPLES-HESSIAN-AI](https://huggingface.co/avemio/GRAG-BGE-M3-TRIPLES-HESSIAN-AI) | 0.8857 |
|
124 |
+
| [GRAG-BGE-M3-TRIPLES-MERGED-HESSIAN-AI](https://huggingface.co/avemio/GRAG-BGE-M3-TRIPLES-MERGED-HESSIAN-AI) | **0.8866** |
|
125 |
+
| [GRAG-BGE-M3-MERGED-x-SNOWFLAKE-ARCTIC-HESSIAN-AI](https://huggingface.co/avemio/GRAG-BGE-M3-MERGED-x-SNOWFLAKE-ARCTIC-HESSIAN-AI) | **0.8866** |
|
126 |
+
| [GRAG-UAE-LARGE-V1-TRIPLES-HESSIAN-AI](https://huggingface.co/avemio/GRAG-UAE-LARGE-V1-TRIPLES-HESSIAN-AI) | 0.8763 |
|
127 |
+
| [GRAG-UAE-LARGE-V1-TRIPLES-MERGED-HESSIAN-AI](https://huggingface.co/avemio/GRAG-UAE-LARGE-V1-TRIPLES-MERGED-HESSIAN-AI) | 0.8771 |
|
128 |
+
|
129 |
## Usage
|
130 |
|
131 |
### Direct Usage (Sentence Transformers)
|