Update readme on quants

README.md

@@ -50,9 +50,12 @@ For training data, we generate long contexts by augmenting [SlimPajama](https://
 | GPU Type               | NVIDIA L40S | NVIDIA L40S | NVIDIA L40S | NVIDIA L40S |
 | Minutes to Train (Wall)| 202       | 555       | 61        | 87        |
 
-**Quants**:
-- [GGUF](https://huggingface.co/crusoeai/Llama-3-8B-Instruct-1048k-GGUF)
+**Inference / Quants**:
+- [GGUF by Crusoe](https://huggingface.co/crusoeai/Llama-3-8B-Instruct-1048k-GGUF). Note that you need to add 128009 as [special token with llama.cpp](https://huggingface.co/gradientai/Llama-3-8B-Instruct-262k/discussions/13).
 - [MLX-4bit](https://huggingface.co/mlx-community/Llama-3-8B-Instruct-1048k-4bit)
+- [Ollama](https://ollama.com/library/llama3-gradient)
+- vLLM docker image, recommended to load via `--max-model-len 65536`
+
 
 ## The Gradient AI Team