ceena.dev
Inference Bench

Run a benchmark

Pick a profile, flip the serving toggles, and drive a load test against the live API. Each run is saved so you can compare configs — same workload, one setting changed.

GPUVRAM

Loading hardware catalog…

Load test
How hard and how long to push the server.
Concurrent users
Simultaneous virtual users
Duration
Seconds at full load
Ramp-up
Seconds to full load
Notes
Samples

Fresh prompts are generated and saved automatically for reuse.

Serving config
The TensorRT-LLM toggles applied to this run.
Quantization
Weight precision
Batch size
Requests per forward pass
Speculative decoding
Propose tokens ahead, verify in one pass
Prefix caching
Reuse shared-prefix KV cache
Chunked prefill
Interleave prefill & decode
Max sequence length
Context budget

A run takes about 7s (it sustains real load for the duration).

Results
Aggregated latency, throughput, and GPU snapshot for the selected run.

Configure a run and hit Run benchmark to see results.

History
Recent chat runs. Click one to view its results.

No runs yet.