In this episode of The Geek Narrator podcast, hosted by Kaivalya Apte, we welcome a special guest, Kishore Gopalakrishna from StarTree, co-author of Apache Pinot and other notable projects. Kishore shares his extensive experience in building real-time analytics and streaming systems, including Apache Pino, Espresso, Apache Helix, and Third Eye. The episode delves into the motivations and challenges behind creating these systems, the innovations they brought to distributed systems, and the impact of community on open-source projects. Kishore also discusses the evolution of testing methodologies, cost optimizations in transactional and analytical systems, and key considerations for companies evaluating real-time analytics solutions.

Don't miss this in-depth conversation packed with valuable insights for both seasoned developers and tech enthusiasts!

Chapters:00:00 Introduction 03:13 Building Distributed Systems at LinkedIn08:57 Testing and Challenges in Distributed Systems30:50 Advantages of Columnar Storage33:04 The Importance of Upserts34:24 Building a Strong Open Source Community41:10 Challenges and Lessons in System Design51:35 Real-Time Analytics: Do You Need It?

StarTree: https://startree.ai/Apache Pinot: https://pinot.apache.org/

If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet.

Database internals series: https://youtu.be/yV_Zp0Mi3xs

Popular playlists:Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-

Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17

Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d

Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN

Stay Curios! Keep Learning!

#distributedsystems #kafka #s3 #streaming #realtimeanalytics #database #pinot #startree