andreasmadsen commited on
Commit
fc7333a
1 Parent(s): 2e6d1bb

fix docs regarding TGI endpoint url (#408)

Browse files

The README specify to use a `/generate_stream` url, when using a custom
TGI endpoint. This is incorrect, as it breaks APIs, such as the
conversation summary that depends on the `/` TGI interface.

This turns out to be a documentation-only bug, as `/` will point to
`/generate_stream` when `stream: true` is set in the request JSON.

Simply using the `/` url as the TGI endpoint therefore fixes the
summarization bug.

Fixes: #278

Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -164,7 +164,7 @@ To do this, you can add your own endpoints to the `MODELS` variable in `.env.loc
164
 
165
  {
166
  // rest of the model config here
167
- "endpoints": [{"url": "https://HOST:PORT/generate_stream"}]
168
  }
169
 
170
  ```
@@ -191,7 +191,7 @@ You can then add the generated information and the `authorization` parameter to
191
 
192
  "endpoints": [
193
  {
194
- "url": "https://HOST:PORT/generate_stream",
195
  "authorization": "Basic VVNFUjpQQVNT",
196
  }
197
  ]
@@ -232,11 +232,11 @@ If the model being hosted will be available on multiple servers/instances add th
232
 
233
  "endpoints": [
234
  {
235
- "url": "https://HOST:PORT/generate_stream",
236
  "weight": 1
237
  }
238
  {
239
- "url": "https://HOST:PORT/generate_stream",
240
  "weight": 2
241
  }
242
  ...
 
164
 
165
  {
166
  // rest of the model config here
167
+ "endpoints": [{"url": "https://HOST:PORT"}]
168
  }
169
 
170
  ```
 
191
 
192
  "endpoints": [
193
  {
194
+ "url": "https://HOST:PORT",
195
  "authorization": "Basic VVNFUjpQQVNT",
196
  }
197
  ]
 
232
 
233
  "endpoints": [
234
  {
235
+ "url": "https://HOST:PORT",
236
  "weight": 1
237
  }
238
  {
239
+ "url": "https://HOST:PORT",
240
  "weight": 2
241
  }
242
  ...