LLM serving benchmarks.

Throughput, latency and cost per token by GPU, on chat and agent workloads.

Models

Workloads