prithivida
commited on
Commit
•
e74ee1d
1
Parent(s):
f826493
added OOD to top
Browse files
README.md
CHANGED
@@ -75,7 +75,7 @@ SPLADE models are a fine balance between retrieval effectiveness (quality) and r
|
|
75 |
**TL;DR of Our attempt & results**
|
76 |
1. FLOPS tuning: Seperate **Seq lens and Severely restrictive FLOPs schedule and token budget** doc(128) & query(24) NOT 256 unlike Official SPLADE++. Inspired from **SparseEmbed**
|
77 |
3. Init Weights: **Middle Trained bert-base-uncased with MLM Loss**. Some corpus awarness like Official splade++ / ColBERT
|
78 |
-
4. Yet achieves competitive effectiveness of MRR@10 **37.8** in ID data (& OOD) and a retrieval latency of - **48.81ms**. (multi-threaded) all On **Consumer grade-GPUs** with **only 5 negatives per query**.
|
79 |
4. For Industry setting: Effectiveness on custom domains needs more than just **Trading FLOPS for tiny gains** and The Premise "SPLADE++ are not well suited to mono-cpu retrieval" does not hold.
|
80 |
5. Owing to query-time inference latency we still need 2 models one for query & doc, This is a Doc model and Query model will be **released soon.**
|
81 |
|
|
|
75 |
**TL;DR of Our attempt & results**
|
76 |
1. FLOPS tuning: Seperate **Seq lens and Severely restrictive FLOPs schedule and token budget** doc(128) & query(24) NOT 256 unlike Official SPLADE++. Inspired from **SparseEmbed**
|
77 |
3. Init Weights: **Middle Trained bert-base-uncased with MLM Loss**. Some corpus awarness like Official splade++ / ColBERT
|
78 |
+
4. Yet achieves competitive effectiveness of MRR@10 **37.8** in ID data (& OOD 49.4) and a retrieval latency of - **48.81ms**. (multi-threaded) all On **Consumer grade-GPUs** with **only 5 negatives per query**.
|
79 |
4. For Industry setting: Effectiveness on custom domains needs more than just **Trading FLOPS for tiny gains** and The Premise "SPLADE++ are not well suited to mono-cpu retrieval" does not hold.
|
80 |
5. Owing to query-time inference latency we still need 2 models one for query & doc, This is a Doc model and Query model will be **released soon.**
|
81 |
|