Update Readme on usage with Infinity
#36
by
michaelfeil
- opened
README.md
CHANGED
@@ -246,6 +246,15 @@ with torch.no_grad():
|
|
246 |
print(all_scores)
|
247 |
```
|
248 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
249 |
## Fine-tune
|
250 |
|
251 |
### Data Format
|
|
|
246 |
print(all_scores)
|
247 |
```
|
248 |
|
249 |
+
### Using Infinity (normal reranker only (bge-reranker-base / bge-reranker-large / bge-reranker-v2-m3 )
|
250 |
+
|
251 |
+
For a self-hosted deployment with [Infinity](https://github.com/michaelfeil/infinity):
|
252 |
+
```bash
|
253 |
+
docker run --gpus all -v $PWD/data:/app/.cache -e HF_TOKEN=$HF_TOKEN -p "7997":"7997" \
|
254 |
+
michaelf34/infinity:0.0.68 \
|
255 |
+
v2 --model-id BAAI/bge-reranker-v2-m3 --revision "main" --dtype float16 --batch-size 32 --engine torch --port 7997
|
256 |
+
```
|
257 |
+
|
258 |
## Fine-tune
|
259 |
|
260 |
### Data Format
|