Azure Data Engineering

ADLS • ADF • Databricks (Delta) • Synapse

Schedule a counselling call

Duration10–12 weeks

Projects1+

Duration

10–12 weeks

Level

Intermediate → Advanced

Mode

Online (Mentor-led)

About this program

Overview

Program snapshot

Build Azure data platforms using ADLS Gen2 storage, ADF orchestration, Databricks Delta transformations, and Synapse analytics.

Prerequisites

SQL fundamentals
Basic Python programming

What you will be able to do

Build lake layers in ADLS with strong folder standards
Create ADF pipelines for ingestion and orchestration
Build Delta Lake transformations in Databricks (PySpark)
Serve analytics via Synapse patterns

Module-by-module breakdown

Syllabus

M1Azure Data Platform Foundations

5 topics - 2 labs

Open module

Topics

ADLS Gen2 concepts
Security basics
Data lake layer design
File formats and partitioning
Cost awareness basics

Hands-on / Labs

Create lake layout strategy
Load raw dataset into ADLS

M2ADF Pipelines (Ingestion + Orchestration)

5 topics - 2 labs

Open module

Topics

Linked services and datasets
Copy activity patterns
Triggers and scheduling
Parameterization
Retries and monitoring

Hands-on / Labs

Ingest raw data to ADLS via ADF
Build parameterized pipelines

M3Databricks + PySpark + Delta Lake

5 topics - 2 labs

Open module

Topics

Spark ETL patterns
Delta Lake basics
MERGE patterns
SCD overview
Optimization overview

Hands-on / Labs

Raw → clean → curated Delta pipeline
Incremental merge into curated tables

M4Streaming + Event Hubs (Overview + Patterns)

5 topics - 2 labs

Open module

Topics

Event streaming concepts
Event Hubs basics
Streaming to lake (patterns)
Latency vs throughput tradeoffs
Monitoring basics

Hands-on / Labs

Event ingestion simulation
Write streaming outputs to lake

M5Synapse + Serving Layer

5 topics - 2 labs

Open module

Topics

Serving layer design
Warehouse vs lakehouse overview
Querying curated datasets
Access control basics
BI integration concepts

Hands-on / Labs

Create marts for reporting
Validate query performance

M6Automation + CI/CD

5 topics - 2 labs

Open module

Topics

Terraform basics for Azure resources
Azure DevOps CI/CD concept
Environment promotion patterns
Runbooks and documentation
Production readiness checklist

Hands-on / Labs

Terraform skeleton project
CI pipeline outline for deployments

Tools & technologies covered

Tech Stack

Core Skills

2 tools

Data Format

4 tools

Data Lake

1 tools

ADLS Gen2

Covered skill

Batch ETL

5 tools

Streaming / Orchestration

3 tools

Azure Stream Analytics

Covered skill

Automation

2 tools

CI/CD

4 tools

Portfolio-ready work

Projects

Project 1: Azure Lakehouse Pipeline

ADF ingestion + Databricks Delta transformations + Synapse serving.

Deliverables

ADF pipelines
Delta curated tables
Synapse marts

Capstone

Capstone: Azure Lakehouse System

Build a complete Azure lakehouse pipeline with automation, orchestration, transformations, and analytics serving.

Preparing your next view

Fetching content, finishing layout, and keeping navigation smooth.

Overview

Program snapshot

Build Azure data platforms using ADLS Gen2 storage, ADF orchestration, Databricks Delta transformations, and Synapse analytics.

Prerequisites

SQL fundamentals
Basic Python programming

What you will be able to do

Build lake layers in ADLS with strong folder standards
Create ADF pipelines for ingestion and orchestration
Build Delta Lake transformations in Databricks (PySpark)
Serve analytics via Synapse patterns

Syllabus

M1Azure Data Platform Foundations

5 topics - 2 labs

Open module

Topics

ADLS Gen2 concepts
Security basics
Data lake layer design
File formats and partitioning
Cost awareness basics

Hands-on / Labs

Create lake layout strategy
Load raw dataset into ADLS

M2ADF Pipelines (Ingestion + Orchestration)

5 topics - 2 labs

Open module

Topics

Linked services and datasets
Copy activity patterns
Triggers and scheduling
Parameterization
Retries and monitoring

Hands-on / Labs

Ingest raw data to ADLS via ADF
Build parameterized pipelines

M3Databricks + PySpark + Delta Lake

5 topics - 2 labs

Open module

Topics

Spark ETL patterns
Delta Lake basics
MERGE patterns
SCD overview
Optimization overview

Hands-on / Labs

Raw → clean → curated Delta pipeline
Incremental merge into curated tables

M4Streaming + Event Hubs (Overview + Patterns)

5 topics - 2 labs

Open module

Topics

Event streaming concepts
Event Hubs basics
Streaming to lake (patterns)
Latency vs throughput tradeoffs
Monitoring basics

Hands-on / Labs

Event ingestion simulation
Write streaming outputs to lake

M5Synapse + Serving Layer

5 topics - 2 labs

Open module

Topics

Serving layer design
Warehouse vs lakehouse overview
Querying curated datasets
Access control basics
BI integration concepts

Hands-on / Labs

Create marts for reporting
Validate query performance

M6Automation + CI/CD

5 topics - 2 labs

Open module

Topics

Terraform basics for Azure resources
Azure DevOps CI/CD concept
Environment promotion patterns
Runbooks and documentation
Production readiness checklist

Hands-on / Labs

Terraform skeleton project
CI pipeline outline for deployments

Projects

Project 1: Azure Lakehouse Pipeline

ADF ingestion + Databricks Delta transformations + Synapse serving.

Deliverables

ADF pipelines
Delta curated tables
Synapse marts

Capstone

Capstone: Azure Lakehouse System

Build a complete Azure lakehouse pipeline with automation, orchestration, transformations, and analytics serving.