Update README.md

README.md

@@ -13,6 +13,38 @@ Trained on 2x4090 using QLoRa and FSDP
 
 + [LoRa](macadeliccc/Samantha-Qwen2-7B-LoRa)
 
+## Launch Using VLLM
+
+```bash
+python -m vllm.entrypoints.openai.api_server \
+    --model macadeliccc/Samantha-Qwen2-7B-AWQ \
+    --chat-template ./examples/template_chatml.jinja \
+    --quantization awq
+```
+
+```python
+from openai import OpenAI
+# Set OpenAI's API key and API base to use vLLM's API server.
+openai_api_key = "EMPTY"
+openai_api_base = "http://localhost:8000/v1"
+
+client = OpenAI(
+    api_key=openai_api_key,
+    base_url=openai_api_base,
+)
+
+chat_response = client.chat.completions.create(
+    model="macadeliccc/Samantha-Qwen2-7B-AWQ",
+    messages=[
+        {"role": "system", "content": "You are a helpful assistant."},
+        {"role": "user", "content": "Tell me a joke."},
+    ]
+)
+print("Chat response:", chat_response)
+```
+
+## Ollama
+
 ## Prompt Template
 
 ```