Worldwide App Distribution

Question 1

When scaling K8s globally, what is one of the most important components to scale?

Answer 1

The image registry.

Image pull latency often dictates the speed with which you can
launch a container in a cluster. This in turn can determine how quickly you can respond to a machine failure, given that generally in the case of a machine failure, you will need to pull the container image down to a new machine.

Another concern about a single registry is that it can be a single point of failure. If the registry is located in a single region or a single datacenter, it’s possible that the registry could go offline due to a large-scale incident in that datacenter.

Question 2

How should you parameterize deployments across regions?

Answer 2

Use a template-based approach.

Same templates for all regions, to which region specific parameters are applied.

Question 3

How would you Load-Balance across regions?

Answer 3

In general, you want to take advantage of geo‐graphic proximity to ensure low-latency access to your service.

But you also want to failover across geographic regions in case of an outage or any other source of service failure.

You could use DNS LB across regions.

Question 4

How should you test your applications?

Answer 4

Full-scale integrations tests with real-life data (ideally you have a full copy of your production data).

Load test with production-like loads. You could “replay” logs from production.

Question 5

How should you roll out applications?

Answer 5

1. Integration and Load Testing
2. Deploy to a Canary Region (which should be VERY similar to production, including monitoring, scale etc.)
3. Deploy to a few smaller regions with different characteristics
4. Once you are sure the release is stable, you can parallelize deployment to all regions.