Message Queues

Question 1

Describe Google Pub/Sub

Answer 1

Architecture:

Google Cloud Pub/Sub is a fully-managed, global messaging service built on Google’s core infrastructure. It operates on a publish-subscribe model.

Key components include:

• Publishers: Applications that create and send messages to a specific topic.
• Topics: Named resources representing a feed of messages, acting as the target for publishers. Topics automatically scale and handle message distribution.
• Subscriptions: Named entities representing an interest in receiving messages from a particular topic. Multiple subscriptions can exist for a single topic, enabling fan-out patterns. Subscriptions retain messages until acknowledged.
• Messages: The unit of data containing a payload (up to 10MB) and optional attributes (key-value metadata).
• Message Storage: Messages are durably stored, replicated across multiple zones within a region upon receipt. Default retention is 7 days, configurable up to 31 days.
• Control Plane: Manages the assignment of publishers and subscribers to data plane servers (forwarders), optimizing for latency and load distribution.
• Data Plane: Handles the actual movement and storage of messages between publishers and subscribers via forwarders. Publishers connect to the nearest available forwarder, and subscriber connections aggregate data globally. Pub/Sub abstracts the underlying partitions and scaling mechanisms from the user.
• Use Cases: Primarily designed for service-to-service communication.

Common use cases include:

• Streaming analytics and data integration pipelines (ingesting events into BigQuery, data lakes).
• Real-time event distribution and enterprise event buses.
• Decoupling microservices and enabling event-driven architectures.
• Asynchronous task parallelization and workflows.
• Database change data capture replication.
• Ingesting user interaction and server events.
• IoT communication (though primarily service-to-service).
• Load balancing work across multiple workers.