Add infinity as example deployment
#22
by
michaelfeil
- opened
README.md
CHANGED
@@ -2701,6 +2701,14 @@ for dv in doc_vecs:
|
|
2701 |
```
|
2702 |
|
2703 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2704 |
|
2705 |
# Citation
|
2706 |
|
|
|
2701 |
```
|
2702 |
|
2703 |
|
2704 |
+
## 3. Infinity
|
2705 |
+
|
2706 |
+
[Infinity](https://github.com/michaelfeil/infinity) is a MIT licensed server for OpenAI-compatible deployment.
|
2707 |
+
```
|
2708 |
+
docker run --gpus all -v $PWD/data:/app/.cache -p "7997":"7997" \
|
2709 |
+
michaelf34/infinity:0.0.68 \
|
2710 |
+
v2 --model-id WhereIsAI/UAE-Large-V1 --revision "369c368f70f16a613f19f5598d4f12d9f44235d4" --dtype float16 --batch-size 32 --device cuda --engine torch --port 7997
|
2711 |
+
```
|
2712 |
|
2713 |
# Citation
|
2714 |
|