How we extended Apache DataFusion to execute one query across many machines

To overcome the single-machine limitations of Apache DataFusion, Datadog developed "Distributed DataFusion," an open-source Rust framework that enables queries to execute across multiple machines. This extension provides the low-latency, interactive execution required for massive datasets while maintaining the engine's core extensibility and composability. Ultimately, this allows Datadog to unify its fragmented query systems into a single, high-performance, and scalable architecture.