Demystifying Delta Parquet - The Foundation of the Lakehouse

19/02/2025

Abstract

Delta Lakes have been around for several years now and there are some fundamental changes that need to be made from a traditional SQL Data Warehouse.

We will review major point of the two concepts used in Lakehouses: Parquet Files and Delta.

For Parquet files, we will review how the files are structured, compressed, and queried. The comparison between them to the 8K pages of SQL Server have many implications for how we structure data within analytical queries.

For Delta, we will dive into how it enables ACID, time travel, upserts, streaming, and more!

With the addition of direct lake mode in Microsoft Fabric, understanding these concepts are becoming increasingly important for any data team.

Speakers

Jarid McKenzie

Lead Analytics Architect at Iteration Insights

Jarid is the Lead Analytics Architect at Iteration Insights. He creates analytics solutions and leads a team of data engineers, data modelers, and visual designers. He has taught at the University of Calgary and the Southern Alberta Institute of Technology for various analytics disciplines. When not playing with data, he can be found mountain biking or out rowing for the Calgary Rowing Club during the summer. During the winter, he can be found hibernating.