System Design

Question 1

Delivery Framework

Answer 1

1. Requirements (5 min)
2. Core Entities (2 min)
3. API (5 min)
4. (Optional) Data Flow (5 min)
5. High Level Design (10 - 15 min)
6. Deep Dives (10 min)

Question 2

Delivery Framework: Requirements

Answer 2

1. Functional requirements (prioritize ~3)
2. Non-functional
3. (Optional) capacity estimation

Question 3

Non-functional requirements

Answer 3

• CAP Theorem (consistancy or performance)
• Environmental constraints
• Scalability (reads vs writes, hot spots)
• Latency
• Durability
• Security
• Fault Tolerance

Question 4

Metrics units

Answer 4

• Thousand = kilo
• Million = mega
• Billion = giga
• Trillion = tera
• Quadrillion = peta

Question 5

Common latencies

Answer 5

• Reading 1 mb sequentially from memory = 0.25 ms
• Reading 1 mb sequentially from SSD = 1 ms
• Reading 1 mb sequentially from spinning disk = 20 ms
• Round trip network latency CA to Netherlands = 150 ms

Question 6

Common storage

Answer 6

• 2-hour movie = 1 gb
• Small book of plain text = 1 mb
• High-resolution photo = 1 mb
• Medium-resolution image or web graphic = 100 kb

Question 7

Common domain estimtations

Answer 7

• DAUs on a social media network = 1b
• Hours of video streamed on netflix/day = 100 m
• Google searches/second = 100k
• Size of Wikipedia = 100 gb

Question 8

2 types of Scaling

Answer 8

• horizontally - adding more machines
• vertically - adding more resources to a single machine

Question 9

Requirements for horizontal scaling

Answer 9

• load balancer
• load balancer strategy (round robin, queuing system, least connections, utilization-based)
• try to partition data such that a single node has all the data it needs

Question 10

Specialized Indexes

Answer 10

• Use ElasticSearch
• Types - geospatial, vector (find image or document), full-text (search document)
• Set up ElasticSearch to index most databases using Change Data Capture (CDC)
• Drawbacks - new failure point, new source of latency, stale data

Question 11

Communication Protocols

Internally

Answer 11

HTTP(S) or gRPC

Question 12

Communicatin Protocols

With Client

Answer 12

• REST (Request -> Response)
• Long polling
• SSE (Server-Sent Events)
• Websockets (Bi-directional Channel)

Question 13

Long polling

Answer 13

• Use when need to give clients near-realtime updates
• Client makes a request and server holds the request open until it has data
• Client can then make another request
• Works with standard load balancers and firewalls

Question 14

Websockets

Answer 14

• Use when need realtime, bidirectional communication
• Challenge - must maintain many long open connections
• Common pattern to use message broker to handle communication and backend services communicate directly with message broker (centralizes connection to client)

Question 15

Server Sent Events (SSE)

Answer 15

• Use when client needs multiple updates from server
• Requires single long-lived HTTP connection
• Requires less specialized infrastructure than websockets

Question 16

Security

Answer 16

• API Gateway for Authentication/Authorization
• Encryption
• Don’t pass userId or things like that through endpoints or bodies, should be in headers

Question 17

Search Optimized Database

Answer 17

• Allows for full text search using indexing, tokenization, stemming
• Inverted Index - index from word to document
• Can confiure if fuzzy search is allowed
• ElasticSearch

Question 18

API Gateway

Answer 18

• Routes requests to correct microservice
• Authentication
• Rate limiting
• Logging

Question 19

Load Balancer

Answer 19

• Need a load balancer whenever you have multiple machines capable of handling the same request
• Can leave out of box and pointer and just mention
• AWS Elastic Load Balancer

Question 20

When to use a Queue

Answer 20

1. Buffer for bursty traffic
2. Distribute work across a system

If strong latency requirements (< 500 ms), queue will probably exceed

Question 21

Queues

Answer 21

1. Message Ordering - typically FIFO but can be priority
2. Retry configurations
3. Dead letter queue for debgging/auditing
4. Scaling with partitions (requires partition key)
5. Backpressure to slow down producers

AWS SQS

Question 22

When to use a Stream

Answer 22

1. Process large amounts of data in real-time (think analytics dashboard)
2. Support complex processin scenarious like event sourcing (think transactions at a bank)
3. Support multiple consumers reading from the same stream (think chat room)

Question 23

Streams

Answer 23

1. Scaling with Partitioning
2. Multiple consumers
3. Replication
4. Windowing

Kinesis

Question 24

Distributed Lock

Answer 24

• Need to lock a resource for a period of time (maybe 10 min)
• Use distributed key-value store like Redis to create a hash map of item -> lock.
• Only one system or process can lock the particular item at a time
• Can set an expiration on the lock so if process crashes, item doesn’t get stuck in locked state

Think: item in inventory while in cart, assignment of driver to rider

Question 25

Cache Eviction Policy

Answer 25

* Least Recently Used * FIFO * Least Frequently Used

Question 26

Cache Write Strategy

Answer 26

* Write-through cache - writes data to both cache and database simultaneously * Write-around cache - just writes to database (caches on next get) * Write-back cache - writes to cache and hen asynchronously to DB (may lose data) | Redis