Update README.md
Browse files
README.md
CHANGED
@@ -22,8 +22,8 @@ To evaluate SwiftKV’s performance, we focus on the following key metrics (see
|
|
22 |
* TTFT: The latency between a user request and receiving the first token in the response.
|
23 |
* TPOT: The latency between subsequent tokens after the first token.
|
24 |
|
25 |
-
Combined input and output throughput for Llama 3.1 405B across a range of input lengths
|
26 |
-
<img src="figure-4.png" alt="performance plot of llama-405B w. swiftkv" width="
|
27 |
|
28 |
TTFT (top) and TPOT (bottom) for input lengths 2000 (left), 8000 (middle), and 32000 (right) for Llama 3.1 405B fp8 model. For each experiment, a range of different request arrival rates is simulated. Each request generates 256 output tokens.
|
29 |
<img src="figure-6.png" alt="performance plot of llama-405B w. swiftkv" width="700">
|
|
|
22 |
* TTFT: The latency between a user request and receiving the first token in the response.
|
23 |
* TPOT: The latency between subsequent tokens after the first token.
|
24 |
|
25 |
+
Combined input and output throughput for Llama 3.1 70B (left) and Llama 3.1 405B (right) across a range of input lengths (bottom).
|
26 |
+
<img src="figure-4-full.png" alt="performance plot of llama-405B w. swiftkv" width="800">
|
27 |
|
28 |
TTFT (top) and TPOT (bottom) for input lengths 2000 (left), 8000 (middle), and 32000 (right) for Llama 3.1 405B fp8 model. For each experiment, a range of different request arrival rates is simulated. Each request generates 256 output tokens.
|
29 |
<img src="figure-6.png" alt="performance plot of llama-405B w. swiftkv" width="700">
|