key tech Flashcards
(8 cards)
API gateway
sits in front of your system and is responsible for routing incoming requests to the appropriate backend service
what does API gateway do?
handling cross-cutting concerns like authentication, rate limiting, and logging
common API gateways
AWS API Gateway, Kong, and Apigee
Load Balancer
When you have a large amount of traffic, you will need to distribute that traffic across multiple machines (called horizontal scaling) to avoid overloading any single machine or creating a hotspot. This is where a load balancer comes in.
whether to use an L4 (layer 4) or L7 (layer 7) load balancer
if you have persistent connections like websockets, you’ll likely want to use an L4 load balancer. Otherwise, an L7 load balancer offers great flexibility in routing traffic to different services while minimizing the connection load downstream.
common load balancers
AWS Elastic Load Balancer (a hosted offering from AWS), NGINX (an open-source webserver frequently used as a load balancer), and HAProxy
when to use Distributed Lock
When you’re dealing with online systems like Ticketmaster, you might need a way to lock a resource - like a concert ticket - for a short time (~10 minutes in this case). This is so while one user is in the middle of buying a ticket, no one else can grab it. Traditional databases with ACID properties use transaction locks to keep data consistent, which is great for ensuring that while one user is updating a record, no one else can update it, but they’re not designed for longer-term locking. This is where distributed locks come in handy.
what do with ML interference layer?
To keep recommendations fresh, I’d retrain models offline weekly using updated user and job data, store them versioned in S3, and serve them via a FastAPI-based inference service. I’d A/B test deployments using LaunchDarkly and monitor click-through and job application rates to guide future model improvements.
