1 year and 1 million messages later: Lessons learned building AI agents on the Elasticsearch Platform

After analyzing one million messages from their AI agents, Elastic found that the key to successful deployment is a continuous feedback loop driven by log analysis and retrieval relevance. The study reveals that providing partial, irrelevant context is more damaging than returning no results, as strict retrieval thresholds are necessary to identify and fill critical knowledge gaps. Additionally, the team noted that a small group of power users drives the majority of engagement and that higher token counts often correlate with deeper, higher-quality interactions.