GCP Data Engineering

GCS • Dataflow • BigQuery • Pub/Sub

Schedule a counselling call

Duration8–10 weeks

Projects1+

Duration

8–10 weeks

Level

Intermediate → Advanced

Mode

Online (Mentor-led)

About this program

Overview

Program snapshot

Build GCP data pipelines using Cloud Storage, streaming with Pub/Sub, processing with Dataflow patterns, and analytics in BigQuery.

Prerequisites

SQL fundamentals
Basic Python programming

What you will be able to do

Design storage and curated layers in GCS
Build batch and streaming pipelines (Dataflow patterns overview)
Model and query analytics data in BigQuery
Automate and manage infra patterns using Terraform concepts

Module-by-module breakdown

Syllabus

M1Foundations: GCS + Data Lake Layout

5 topics - 2 labs

Open module

Topics

Bucket strategy and folder design
Raw/clean/curated patterns
Parquet basics and partition concepts
Data quality checks concepts
Cost awareness basics

Hands-on / Labs

Design lake structure in GCS
Convert dataset to curated Parquet format

M2BigQuery Analytics

5 topics - 2 labs

Open module

Topics

BigQuery datasets and tables
Querying patterns and optimizations
Partitioning overview
Star schema concepts
Reporting queries

Hands-on / Labs

Build marts in BigQuery
Run KPI reporting queries

M3Streaming: Pub/Sub Patterns

5 topics - 2 labs

Open module

Topics

Producers/consumers
Topic/subscription model
Event schema concepts
Dedupe/idempotency concepts
Monitoring concepts

Hands-on / Labs

Simulate event publishing
Consume and store into curated layer

M4Processing: Dataflow Concepts + Pipeline Design

5 topics - 2 labs

Open module

Topics

Batch vs streaming pipelines
Transformation patterns
Windowing concepts (overview)
Backfill concepts
Error handling patterns

Hands-on / Labs

Batch transform flow design
Streaming transform flow design

M5Orchestration + CI/CD

5 topics - 2 labs

Open module

Topics

Airflow orchestration concepts
DAG scheduling and retries
Terraform basics for infra
Git workflows
Release checklist

Hands-on / Labs

Orchestration outline for pipelines
Terraform skeleton

Tools & technologies covered

Tech Stack

Core Skills

2 tools

Data Format

3 tools

Storage

1 tools

Google Cloud Storage (GCS)

Covered skill

Processing

2 tools

Streaming

1 tools

Pub/Sub

Covered skill

Orchestration

1 tools

Apache Airflow

Covered skill

CI/CD

3 tools

Portfolio-ready work

Projects

Project 1: GCP Analytics Pipeline

GCS curated layer + BigQuery marts + reporting queries.

Deliverables

Curated storage layout
BigQuery marts
KPI query pack

Capstone

Capstone: GCP Data Platform Blueprint

Design a complete GCP data platform with storage, processing, streaming patterns, analytics marts, and automation strategy.

Preparing your next view

Fetching content, finishing layout, and keeping navigation smooth.

Overview

Program snapshot

Build GCP data pipelines using Cloud Storage, streaming with Pub/Sub, processing with Dataflow patterns, and analytics in BigQuery.

Prerequisites

SQL fundamentals
Basic Python programming

What you will be able to do

Design storage and curated layers in GCS
Build batch and streaming pipelines (Dataflow patterns overview)
Model and query analytics data in BigQuery
Automate and manage infra patterns using Terraform concepts

Syllabus

M1Foundations: GCS + Data Lake Layout

5 topics - 2 labs

Open module

Topics

Bucket strategy and folder design
Raw/clean/curated patterns
Parquet basics and partition concepts
Data quality checks concepts
Cost awareness basics

Hands-on / Labs

Design lake structure in GCS
Convert dataset to curated Parquet format

M2BigQuery Analytics

5 topics - 2 labs

Open module

Topics

BigQuery datasets and tables
Querying patterns and optimizations
Partitioning overview
Star schema concepts
Reporting queries

Hands-on / Labs

Build marts in BigQuery
Run KPI reporting queries

M3Streaming: Pub/Sub Patterns

5 topics - 2 labs

Open module

Topics

Producers/consumers
Topic/subscription model
Event schema concepts
Dedupe/idempotency concepts
Monitoring concepts

Hands-on / Labs

Simulate event publishing
Consume and store into curated layer

M4Processing: Dataflow Concepts + Pipeline Design

5 topics - 2 labs

Open module

Topics

Batch vs streaming pipelines
Transformation patterns
Windowing concepts (overview)
Backfill concepts
Error handling patterns

Hands-on / Labs

Batch transform flow design
Streaming transform flow design

M5Orchestration + CI/CD

5 topics - 2 labs

Open module

Topics

Airflow orchestration concepts
DAG scheduling and retries
Terraform basics for infra
Git workflows
Release checklist

Hands-on / Labs

Orchestration outline for pipelines
Terraform skeleton

Projects

Project 1: GCP Analytics Pipeline

GCS curated layer + BigQuery marts + reporting queries.

Deliverables

Curated storage layout
BigQuery marts
KPI query pack

Capstone

Capstone: GCP Data Platform Blueprint

Design a complete GCP data platform with storage, processing, streaming patterns, analytics marts, and automation strategy.