Why AI code optimization needs production-grounded benchmarks

Datadog developed DODO, an LLM-driven optimizer designed to prevent coding agents from optimizing for inaccurate synthetic benchmarks by grounding them in real production telemetry. By using CPU profiles and live debugger samples to create micro-benchmarks that mirror actual service workloads, DODO ensures that AI-proposed code changes translate into significant, measurable reductions in real-world CPU usage.