Grokking the Data Engineering Interview
Prepare for data engineering interviews with advanced SQL, Python for data work, data modeling, pipeline design, storage and processing at scale, data quality, streaming, privacy, metrics, and complete pipeline designs.
Course Overview
This course teaches the skills data engineering interviews test through the decisions a data engineer makes. Each lesson takes one skill or decision, such as which window frame a running total needs, what grain a fact table should have, or how to rerun a pipeline without counting anything twice, and works it through with real numbers and the cost of each option. SQL exercises run in the browser and are graded against hidden test data, and every query and code sample is run before it is published. The course starts with advanced SQL: window functions, gaps and islands, sessions, retention and funnel queries, and query plans. It then covers Python for data work, data modeling with slowly changing dimensions, pipeline design with safe reruns, backfills, and change data capture, and storage and processing with Parquet, table formats, Spark, and warehouse costs, followed by data quality, streaming, privacy, and metrics. It ends with complete designs of a clickstream pipeline, a change data capture feed, an ad metrics dashboard, and a daily reporting pipeline. Every chapter ends with a review assessment and a flashcard deck, and every chapter lesson closes with a short section on explaining the decision in an interview.
What you'll learn in Grokking Data Engineering Interview
- Write window functions for rankings, running totals, gaps and islands, sessions, cohorts, and funnels.
- Read a MySQL query plan, find the full scan, and decide which index would remove it.
- Process files larger than memory with generators and sorted merges, and write jobs that rerun safely.
- Design idempotent batch loads, scope a backfill, and handle late data with a lookback window.
- Explain how Parquet, partitioning, table formats, and Spark joins decide what a query reads and costs.
- Write data tests, data contracts, and reconciliation checks that stop a wrong number before it is published.
- Delete one person from every copy of their data, and set retention that follows lineage.
- Choose between ROW_NUMBER, RANK, and DENSE_RANK when ties change the answer, and explain the choice.
- Parse messy CSV and JSON Lines files in plain Python, and decide what to reject, repair, or quarantine.
- Model analytics tables with facts, dimensions, and a stated grain, and keep history with slowly changing dimensions.
- Capture changes from a source database and apply them to a warehouse table in commit order.
- Estimate warehouse bills from published prices for BigQuery, Snowflake, and Redshift, and lower them.
- Use event-time windows and watermarks, and write a stream to a table exactly once.
- Define a metric precisely, investigate a sudden drop, and check experiment data for a sample ratio mismatch.
Course Content
Storage and Processing at Scale
Data Quality and Reliability
Streaming for Data Engineers
Privacy and Governance
Metrics and Product Sense
Complete Designs
Clickstream Pipeline: Requirements and Data Model
Clickstream Pipeline: Processing Late Data and Costs
CDC Into a Warehouse: Requirements and Events
CDC Into a Warehouse: Applying Changes and Failures
Ad Metrics Dashboard: Requirements and Numbers
Ad Metrics Dashboard: Dedup Attribution and Reconciliation
Daily Reporting Pipeline: Requirements and Model
Daily Reporting Pipeline: Orchestration Quality and Backfills
What people say about our courses
About the Author
Arslan Ahmad
Industry Expertise & Leadership
Arslan Ahmad is the lead author of Grokking the Data Engineering Interview. As the founder of Design Gurus and a former FAANG hiring manager, he has worked at industry giants like Facebook (now Meta) and Microsoft.
He has conducted hundreds of system design interviews, giving him unique insight into what top tech companies look for in candidates.
The course also incorporates expertise from senior engineers at Google, Meta, Amazon, Microsoft, and Uber, ensuring you learn system design best practices from professionals who have built and scaled real-world systems.
500+
Interviews Conducted
10k+
Students Taught
Related Courses
$148
$72
$72
FAQs
Who is this course for?
Engineers preparing for data engineering interviews, and software engineers moving into data work, who already write basic SQL and some Python and want to reach the level the interviews test.
What do data engineering interviews test?
Only some companies publish their process. Meta's published guide names SQL, Python, data modeling, ETL, and product sense, and Amazon's names SQL, ETL, data modeling, and a scripting language. The course introduction maps each published topic to the lessons that teach it and labels prep-company descriptions as such.
Which SQL dialect does the course use?
Every query uses MySQL 8.0 syntax and was run on MySQL 8.4. Where PostgreSQL, Snowflake, or BigQuery behave differently, for example with QUALIFY, the lesson shows the difference.
Can I run the code?
Yes. Most SQL lessons end with a graded exercise that you run and submit in the browser, and the Python lessons run in the course player using only the standard library. Spark, Airflow, Delta Lake, and Iceberg code ran on Linux and is shown with its recorded output.
Is there any practice, or is it only reading?
Every lesson ends with practice questions and folded answers, most SQL lessons include a graded exercise, and each chapter ends with a flashcard deck and an adaptive assessment.
Does it cover Spark, Airflow, and Kafka?
Yes, from the data engineer's side. The lessons show how Spark runs a job and handles a skewed join, and how Airflow schedules and reruns a pipeline. They also show how to read a Kafka stream, set a watermark, and write the result to a table exactly once.
Does it use pandas?
No. The Python lessons use only the standard library so that every script runs in the course player, and they teach the grouping, parsing, and rerun ideas that pandas and Spark jobs also need.
How is this different from a system design course?
A system design interview centers on serving users with services, caches, and scaling. This course centers on the data those systems produce: the tables, pipelines, checks, and costs that a data engineer designs.
Which complete designs does it include?
Four, each across two lessons. They are a clickstream pipeline for a news app, change data capture from a car rental database into a warehouse, an ad metrics dashboard that must agree with the invoice, and a daily reporting pipeline for a cinema chain.
Does it cover privacy rules such as the GDPR?
Yes, as engineering work. The lessons cover finding personal data in every column, deleting one person from every copy and old table version, handling backups that expire later, pseudonymization and masking, and retention that follows lineage. They are not legal advice.
Do I need a cloud account or a Spark cluster?
No. The SQL and Python run in the browser, and the Spark, Airflow, and warehouse lessons show recorded output and arithmetic on published prices, so you can study them without installing anything.
