BAAI
/

Update Readme on usage with Infinity

#36
Files changed (1) hide show
  1. README.md +9 -0
README.md CHANGED
@@ -246,6 +246,15 @@ with torch.no_grad():
246
  print(all_scores)
247
  ```
248
 
 
 
 
 
 
 
 
 
 
249
  ## Fine-tune
250
 
251
  ### Data Format
 
246
  print(all_scores)
247
  ```
248
 
249
+ ### Using Infinity (normal reranker only (bge-reranker-base / bge-reranker-large / bge-reranker-v2-m3 )
250
+
251
+ For a self-hosted deployment with [Infinity](https://github.com/michaelfeil/infinity):
252
+ ```bash
253
+ docker run --gpus all -v $PWD/data:/app/.cache -e HF_TOKEN=$HF_TOKEN -p "7997":"7997" \
254
+ michaelf34/infinity:0.0.68 \
255
+ v2 --model-id BAAI/bge-reranker-v2-m3 --revision "main" --dtype float16 --batch-size 32 --engine torch --port 7997
256
+ ```
257
+
258
  ## Fine-tune
259
 
260
  ### Data Format