Add infinity as example deployment

#22
Files changed (1) hide show
  1. README.md +8 -0
README.md CHANGED
@@ -2701,6 +2701,14 @@ for dv in doc_vecs:
2701
  ```
2702
 
2703
 
 
 
 
 
 
 
 
 
2704
 
2705
  # Citation
2706
 
 
2701
  ```
2702
 
2703
 
2704
+ ## 3. Infinity
2705
+
2706
+ [Infinity](https://github.com/michaelfeil/infinity) is a MIT licensed server for OpenAI-compatible deployment.
2707
+ ```
2708
+ docker run --gpus all -v $PWD/data:/app/.cache -p "7997":"7997" \
2709
+ michaelf34/infinity:0.0.68 \
2710
+ v2 --model-id WhereIsAI/UAE-Large-V1 --revision "369c368f70f16a613f19f5598d4f12d9f44235d4" --dtype float16 --batch-size 32 --device cuda --engine torch --port 7997
2711
+ ```
2712
 
2713
  # Citation
2714