Version history
1 MiB throughput
Median successful requests per second by implementation
Loading chart…
Performance / proxy
LiteLLM proxy OCR throughput: Python vs Rust
Explore the latest published metrics for each LiteLLM version.
Evidence status
Version coverage
Every LiteLLM version with published evidence in this data set, and whether this benchmark's canonical run succeeded, failed, or left no record. Only successful runs feed the charts below.
6 of 7 versions published · 1 missing
| Version | Status | Detail |
|---|---|---|
| 1.97.0 | missing | No canonical result was published for this version |
| 1.98.0 | published | 1 successful run |
| 1.99.0 | published | 1 successful run |
| 1.100.0 | published | 1 successful run |
| 1.100.1 | published | 1 successful run |
| 1.101.0 | published | 1 successful run |
| 1.102.0rc2 | published | 1 successful run |
Results in selected range 6
LiteLLM versions 6
Metrics in selected range 34
Detailed measurements
Version history
Compare published metrics across releases. Charts use canonical Linux runs. Hover over a point for its measurement and configuration.
1.98.0 / 1.102.0rc2
Median measurements in matching environments. Percentage change is relative to 1.98.0.
Python median throughputbase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
70.5 RPS / 66.9 RPS
-5.1%Rust median throughputbase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
69.3 RPS / 84.0 RPS
+21.1%Python median p95 latencybase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
495 ms / 526 ms
+6.2%Rust median p95 latencybase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
502 ms / 434 ms
-13.5%Python median average CPUbase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
100 % / 100 %
-0.1%Rust median average CPUbase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
100 % / 100 %
-0.1%Python median proxy CPU time per successful requestbase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
14.2 ms / 15.0 ms
+5.8%Rust median proxy CPU time per successful requestbase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
14.4 ms / 11.9 ms
-17.7%Python median peak memorybase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
984 MiB / 686 MiB
-30.2%Rust median peak memorybase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
981 MiB / 693 MiB
-29.4%Python median peak memory growth above the post-warm-up baselinebase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
40.3 MiB / 56.1 MiB
+39.2%Rust median peak memory growth above the post-warm-up baselinebase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
29.9 MiB / 67.1 MiB
+124.1%Python median post-load anonymous memorybase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
877 MiB / 536 MiB
-38.8%Rust median post-load anonymous memorybase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
871 MiB / 528 MiB
-39.4%Python median post-load anonymous-memory growth above the post-warm-up baselinebase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
-21.0 MiB / -35.4 MiB
-68.8%Rust median post-load anonymous-memory growth above the post-warm-up baselinebase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
-21.3 MiB / -33.1 MiB
-55.7%Median paired Rust/Python throughputbase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 1 MiB, concurrency 32
0.99 x / 1.28 x
+29.4%Python median throughputbase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
12.9 RPS / 13.6 RPS
+5.8%Rust median throughputbase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
12.9 RPS / 19.3 RPS
+49.3%Python median p95 latencybase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
674 ms / 634 ms
-5.9%Rust median p95 latencybase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
674 ms / 486 ms
-28.0%Python median average CPUbase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
100 % / 100 %
-0.1%Rust median average CPUbase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
100 % / 100 %
-0.2%Python median proxy CPU time per successful requestbase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
77.6 ms / 73.3 ms
-5.6%Rust median proxy CPU time per successful requestbase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
77.5 ms / 51.9 ms
-33.1%Python median peak memorybase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
1638 MiB / 1301 MiB
-20.6%Rust median peak memorybase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
1665 MiB / 906 MiB
-45.6%Python median peak memory growth above the post-warm-up baselinebase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
84.5 MiB / 98.6 MiB
+16.6%Rust median peak memory growth above the post-warm-up baselinebase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
123 MiB / 236 MiB
+92.1%Python median post-load anonymous memorybase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
1501 MiB / 1149 MiB
-23.5%Rust median post-load anonymous memorybase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
1520 MiB / 562 MiB
-63.1%Python median post-load anonymous-memory growth above the post-warm-up baselinebase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
-0.32 MiB / -6.88 MiB
-2022.9%Rust median post-load anonymous-memory growth above the post-warm-up baselinebase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
29.1 MiB / -31.4 MiB
-207.9%Median paired Rust/Python throughputbase · linux · x86_64 · c8634aefbc6546ceac42693380bc4631d81523ec98f6f49c6bfde7e9f25ea636 · 8 MiB, concurrency 8
1.00 x / 1.41 x
+41.5%Version history
1 MiB paired improvement
Median Rust throughput divided by paired Python throughput
Loading chart…
Version history
8 MiB throughput
Median successful requests per second by implementation
Loading chart…
Version history
8 MiB paired improvement
Median Rust throughput divided by paired Python throughput
Loading chart…
Version history
1 MiB p95 latency
Median p95 successful response latency by implementation
Loading chart…
Version history
1 MiB CPU usage
Median average container CPU by implementation
Loading chart…
Version history
1 MiB CPU service demand
Median proxy CPU time per successful request by implementation
Loading chart…
Version history
1 MiB peak memory
Median peak container memory by implementation
Loading chart…
Version history
1 MiB peak memory growth
Median peak memory above the post-warm-up baseline by implementation
Loading chart…
Version history
1 MiB idle memory
Median post-load anonymous memory by implementation
Loading chart…
Version history
1 MiB retained-memory growth
Median post-load anonymous memory above baseline by implementation
Loading chart…
Version history
8 MiB p95 latency
Median p95 successful response latency by implementation
Loading chart…
Version history
8 MiB CPU usage
Median average container CPU by implementation
Loading chart…
Version history
8 MiB CPU service demand
Median proxy CPU time per successful request by implementation
Loading chart…
Version history
8 MiB peak memory
Median peak container memory by implementation
Loading chart…
Version history
8 MiB peak memory growth
Median peak memory above the post-warm-up baseline by implementation
Loading chart…
Version history
8 MiB idle memory
Median post-load anonymous memory by implementation
Loading chart…
Version history
8 MiB retained-memory growth
Median post-load anonymous memory above baseline by implementation
Loading chart…