thenlper michaelfeil commited on
Commit
a829fd0
1 Parent(s): a8e4f3e

Update usage example with infinity (#13)

Browse files

- Update usage example with infinity (3d61b474c3e21454d937ce470ef7f402dbff9559)


Co-authored-by: Michael <michaelfeil@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +8 -0
README.md CHANGED
@@ -2713,6 +2713,14 @@ const similarities = document_embeddings.map(x => 100 * dot(source_embeddings, x
2713
  console.log(similarities); // [34.504930869007296, 64.03973265120138, 19.520042686034362]
2714
  ```
2715
 
 
 
 
 
 
 
 
 
2716
  ## Training Details
2717
 
2718
  ### Training Data
 
2713
  console.log(similarities); // [34.504930869007296, 64.03973265120138, 19.520042686034362]
2714
  ```
2715
 
2716
+ Use with infinity:
2717
+ [Infinity](https://github.com/michaelfeil/infinity) is a MIT licensed server for OpenAI-compatible deployment.
2718
+ ```
2719
+ docker run --gpus all -v $PWD/data:/app/.cache -p "7997":"7997" \
2720
+ michaelf34/infinity:0.0.68 \
2721
+ v2 --model-id Alibaba-NLP/gte-base-en-v1.5 --revision "4c742dc2b781e4ab062a4a77f4f7cbad4bdee970" --dtype bfloat16 --batch-size 32 --device cuda --engine torch --port 7997
2722
+ ```
2723
+
2724
  ## Training Details
2725
 
2726
  ### Training Data