INTERMEDIATE · 8 MODULES · 64 LESSONS

Lakehouse and Iceberg

Turn your Parquet files into Apache Iceberg tables with transactions, queryable history, schema changes without rewriting data, and native upserts.

For you if
You already write Parquet or run a warehouse and track change history by hand.
By the end
You'll know how to use time travel, evolve schema and partitions, keep a table healthy, and decide when a lakehouse makes sense.
You need
Intermediate Python, Parquet, and basic SQL. Spark with Java 17 only for the MERGE module.
  1. 01From File Format To Table Format8 LESSONS
    1. Module overview: from a Parquet file to a real Iceberg table
    2. Four times Parquet alone wasn't enough
    3. File format vs. table format
    4. Installing PyIceberg and a local catalog
    5. Creating your first namespace and table
    6. Loading Kiosko's fact_orders into Iceberg
    7. Verifying the same 106.15 total
    8. Project: Kiosko's first Iceberg table
  2. 02Anatomy Of An Iceberg Table8 LESSONS
    1. Module overview: the anatomy of an Iceberg table
    2. The catalog: a pointer to the current metadata
    3. The metadata file: schema, partitioning, snapshots
    4. Manifest lists and manifest files
    5. The data files: the Parquet you already know
    6. Inspecting Kiosko's table on disk
    7. Reading snapshots and history with PyIceberg
    8. Project: Kiosko's table anatomy, mapped
  3. 03Snapshots And Time Travel8 LESSONS
    1. Module overview: snapshots and time travel
    2. Every write is a new snapshot
    3. Changing P002 with a plain overwrite
    4. Capturing the snapshot-id, never hardcoding it
    5. Time travel: AS OF a snapshot-id
    6. Recovering P002's history with zero extra columns
    7. What time travel does not replace
    8. Project: Kiosko's time-traveled dim_product
  4. 04Schema Evolution Without Rewriting8 LESSONS
    1. Module overview: schema evolution without rewriting
    2. Why overwrite-partition was never atomic
    3. Adding a column without rewriting data
    4. Renaming and dropping columns safely
    5. Populating country in dim_store
    6. Reading old snapshots after a schema change
    7. What "ACID" actually guarantees in this exact context
    8. Project: Kiosko's evolved dim_store
  5. 05Hidden Partitioning And Partition Evolution8 LESSONS
    1. Module overview: hidden partitioning and partition evolution
    2. How Spark partitioned Kiosko by folder
    3. Hidden partitioning: the same query, without knowing the layout
    4. Partition transforms: `IdentityTransform`, `BucketTransform`, `DayTransform`
    5. Partitioning `fact_orders_at_scale`
    6. Evolving the `PartitionSpec` without rewriting
    7. Reading old and new partition schemes, together
    8. Project: Kiosko's `fact_orders_at_scale`, partitioned
  6. 06Merge Into And Native Upserts8 LESSONS
    1. Module overview: `MERGE INTO` and native upserts
    2. Three ways Kiosko already solved this
    3. Setting up Spark with the Iceberg runtime
    4. `MERGE INTO` in Spark SQL
    5. Running P002's change as a MERGE
    6. PyIceberg's upsert: the Python-native alternative
    7. Choosing between MERGE in SQL and upsert in Python
    8. Project: Kiosko's native upsert
  7. 07Catalogs Maintenance And Delta Lake By Contrast8 LESSONS
    1. Module overview: catalogs, maintenance, and Delta Lake by contrast
    2. Catalogs beyond local: REST, Glue, Unity Catalog, Polaris
    3. Why snapshots accumulate cost
    4. Compacting small files
    5. Expiring old snapshots, safely
    6. Removing orphan files
    7. Delta Lake, by contrast
    8. Project: Kiosko's maintained table
  8. 08Project Kioskos Lakehouse8 LESSONS
    1. Module overview: the capstone, Kiosko's complete lakehouse
    2. The brief: Kiosko needs a lakehouse, not seven demos
    3. Rebuilding the star schema as Iceberg tables
    4. Reproducing P002's history with time travel
    5. Evolving and partitioning at scale
    6. Merging updates the native way
    7. What Kiosko still needs
    8. Project: Kiosko's first lakehouse
Reviews

What students say

These reviews are from enrolled students who completed at least 50% of the course. We moderate reviews only on content grounds (spam, offensive language, personal data), never for being critical or negative.

No approved reviews yet.

Be the first to share your experience!

SEC. 06 · THE STUDIO

Why this studio exists

For decades, building software belonged to technical profiles with years of training, and getting a digital product into the real world took companies months or years. No-code and AI have narrowed that gap, but the knowledge is still held by a few.

Tech Benders exists to open that knowledge up, so business teams and non-technical teams can put their ideas into production with the right technical decisions behind them. AI does what you ask of it. Without a technical base, what it builds may not scale with your company’s ecosystem, and its costs can run away over time.

The team knows the pains of large organizations and the speed at which they have to move. It brings solution architecture, data engineering, product built end to end from back end to front end, and daily work with AI. Software that solves what you need in days, not months, with the cost of AI kept in check.

Founders

The studioRead about Tech Benders →

Write to ushi@techbenders.io

Made with Wislab