Key Concepts
Deep dives into the core technologies and patterns that define modern data engineering: orchestrators, integration frameworks, and multi-channel platforms.
Apache Airflow
The industry-standard workflow orchestrator. DAG-based scheduling, operators, sensors, and the Airflow ecosystem.
Prefect
A modern Python-native workflow manager with automatic retries, caching, and cloud observability.
Dagster
An asset-oriented orchestrator with built-in data quality, software-defined assets, and cloud-native execution.
Harnesses
Data harnesses and integration frameworks for connecting, transforming, and monitoring data across systems.
MCPs
Multi-Channel Platforms: unified data ingestion and routing across streaming, batch, and API channels.