2023 Sessions On-Demand

#

All Sessions

Hydrating Real-Time Data in Pinot with Fast Batch Queries on Trino

Elon Azoulay

Session Speaker
Starburst
Software Engineer, Starburst
Software Engineer that loves data and databases. Started out as a DBA and always wanted to contribute to databases. In my free time I enjoy spending time with my family, going to the beach, biking and playing classical and electric guitar.

Martin Traverso

Session Speaker
Starburst
Co-Creator of Trino and Chief Technology Officer, Starburst
Martin is the Co-Creator of Trino and Presto. He is a CTO at Starburst where he leads research and development on Trino. Previously, he was a Software Engineer at Facebook where he led the Presto development team, and an architect at Proofpoint and Ning. Martin joined Facebook in 2012 when, at the time, Hive was the de facto platform for SQL analytics at Facebook. Seeing a need for fast interactive SQL analytics, Martin and 3 other engineers worked to create what became Presto. In the Spring of 2013, Martin and the team rolled out Presto into production at Facebook where it was later made open source in the Fall of 2013. In 2019, he and the other founders created a Trino fork off of Presto. Since then, Trino has gained wide adoption both internal and external to Facebook.

In the spectrum of real-time to batch systems, there has never been a perfect solution that can be found that addresses the limitations of both extremes well. Some systems that fall between, typically only do one or the other well and can’t be adopted for both. Pinot is state-of-the-art for real-time ingest and analytics streaming. Trino is well known for being an incredibly fast batch analytics query engine that enables not only large-scale batch processing but also runs federated joins across multiple tables across multiple data sources.

This talk covers the many benefits of running Trino and Pinot together like using Trino as a unified point of access for reads and executing writes into Pinot and interactive analytics over data in Pinot and cloud storage. This talk will also dive into recent updates in Trino’s Pinot connector like optimizing the efficiency of reads, putting less load on Pinot brokers. Learn how to augment your real-time architecture with this ludicrously fast batch query engine.