How we made a SQL query optimization agent 59% more accurate using autoresearch and LLM Observability

The Datadog DBM team used Karpathy’s autoresearch tool to autonomously optimize an AI agent designed for SQL query optimization. By conducting 23 overnight experiments to refine prompts and model configurations, they successfully increased the agent's precision from 0.54 to 0.86. This automated approach allowed the team to rapidly improve the agent's ability to identify complex database optimization patterns without the manual overhead of traditional testing.