INTERMEDIATE · 8 MODULES · 64 LESSONS
Data Reliability and Governance
Prove your data is correct, not just that the pipeline finished green, with quality tests, data contracts, lineage, and protection for personal data.
- For you if
- Your pipelines run green but you have no way to prove the data is right.
- By the end
- You'll know how to catch explainable anomalies, quarantine broken rows, handle an incident, and mask personal data by role.
- You need
- Intermediate Python with DataFrames (Polars or pandas) and basic SQL. Everything runs on your machine.

Curriculum
01When Green Does Not Mean Correct
02Declarative Data Quality Tests With Pandera
- Module introduction: declarative data quality tests with Pandera
- What, precisely, a declarative quality test is
- Choosing a vehicle: Pandera vs. Great Expectations vs. Soda
- Installing Pandera and building the DuckDB → Polars bridge
- Writing your first DataFrameModel
- Completeness and uniqueness, declared
- Validity declared, and how to read a complete failure report
- Project: testing S04 with Pandera
03Consistency And Referential Checks
- Module introduction: consistency and referential checks
- What consistency means across tables
- Why schema checks alone don't catch referential bugs
- Writing a referential integrity check
- Cross-column consistency rules
- Catching S04's orphan product
- Combining Pandera with custom checks
- Project: S04's complete consistency report
04Data Contracts As Versioned Artifacts
- Module introduction: data contracts as versioned artifacts
- What, precisely, a data contract is
- Writing orders_contract.yaml
- Parsing the contract with pydantic
- From contract to Pandera schema
- Versioning a contract that changes
- Should S04 be allowed to write without a contract?
- Project: Kiosko's first data contract
05Accuracy And Deterministic Anomaly Detection
- Module introduction: accuracy and deterministic anomaly detection
- Accuracy: the hardest dimension to test
- Why a valid row can still be wrong
- Building a price baseline from Kiosko's clean week
- Threshold-based anomaly detection, without Machine Learning
- Catching the dollars-to-cents bug
- When thresholds are too strict or too loose
- Project: S04's accuracy audit
06Freshness Volume And Lineage
- Module introduction: freshness, volume, and lineage
- Freshness is a table-level question, not a row-level one
- A fixed reference clock, never `datetime.now()`
- Writing a freshness check for S04
- Volume checks: too few or too many rows
- What lineage answers that a contract does not
- Mapping Kiosko's lineage by hand
- Project: S04's freshness, volume, and lineage report
07The Incident And Data Governance
- Module introduction: the incident and data governance
- What happens when a check fails in production
- Quarantining, instead of crashing or silently passing
- Writing an alert and a runbook
- Who can see what: access control basics
- Masking PII deterministically
- A minimal data catalog for Kiosko
- Project: S04's incident response and access policy
08Project Kioskos Reliability And Governance System
- Module introduction: Kiosko's complete trust system
- The brief: Kiosko needs a trust system, not just a pipeline
- Assembling the contract-driven quality gate
- Running the full gate against S04's incident
- Running the full gate against a clean day
- Publishing the catalog and access policy
- What Kiosko still needs
- Project: Kiosko's trusted data platform
Reviews
What students say
These reviews are from enrolled students who completed at least 50% of the course. We moderate reviews only on content grounds (spam, offensive language, personal data), never for being critical or negative.
No approved reviews yet.
Be the first to share your experience!