Chapter 3

Question 1

Expand the shortcut

DAG

Answer 1

Directed acyclic graphs

Question 2

Expand the shortcut and elaborate on the process

ETL

Answer 2

Extraction, transformation, and load

Question 3

Expand the shortcut

ELT

Answer 3

Extraction, load, and transformation

Question 4

Name the four types of stages in a data pipeline

Answer 4

1. Ingestion
2. Transformation
3. Storage
4. Analysis

Question 5

What is

Ingestion

In terms of data pipeline stages

Answer 5

The process of bringing data into the GCP environment

Question 6

What is

Transformation

In terms of data pipeline stages

Answer 6

Transformation is the process of mapping data from the structure used in the source system
to the structure used in the storage and analysis stages of the data pipeline.

Question 7

What is

“Change Data Capture”

Answer 7

In a change data capture approach, each change in a source system is captured and
recorded in a data store. This is helpful in cases where it is important to know all changes
over time and not just the state of the database at the time of data extraction.

Question 8

What is and how works

Tumbling Window

Answer 8

• Engine collects all events within a fixed duration (e.g., 5 minutes).
• At the end of the window, it triggers a computation, like SUM, AVG, COUNT, etc.
• Then the window closes, and the next one starts.
  At 12:00–12:05 → Count all login events → Output: COUNT = 200

Question 9

What is and how works

Sliding Window

Answer 9

• Engine maintains overlapping windows.
• Each window slides forward (e.g., every 1 min) and spans a certain duration (e.g., 5 min).
• Each event may be included in multiple windows.
• Aggregation is triggered for each window, usually periodically.

Question 10

What is and whats the difference between

Hot Path and Cold Path Ingestion

Answer 10

Hot Path
* Real-time or near-real-time data ingestion and processing.
* Focused on low-latency.
* Used when data needs to be acted on immediately.

Cold Path
* Batch ingestion with higher latency (minutes, hours, daily).
* Optimized for throughput and cost, not speed.
* Data is processed in bulk for historical analysis or reporting.