Uber's apps make every call through one edge tier, the Edge Gateway. Hundreds of backend services, owned by many teams, sit behind it, and none of them is reached by an app directly. The gateway does the work every call needs, the same way for every endpoint: it decodes the request, runs middleware (authentication, bans, rate limits), routes the call to the service that owns the path, and encodes the answer.
Each endpoint is configuration, not hand-written code: a team declares the path, the target service and the middleware it needs, and Uber generates the gateway from that configuration. Every endpoint must use at least one authentication middleware, and when a middleware rejects a call, the rest of the stack is skipped and the middleware's answer goes back.
The catch is that everything depends on the edge. If authentication there calls the auth service for every request, the auth service carries the whole app's load; if the gateway pool cannot lose an instance, a single crash takes every endpoint down at once.
Functional requirements
Three endpoints, each routed to a different service:
Get fare estimate: the rider app asks for a price; the gateway routes the call to pricing. Three scenarios:
"Session cached": the token was validated recently and its session is in the cache; the auth service is not asked.
"Session not cached": no cached session, so the gateway asks auth to validate the token, caches the result, then routes the call.
"Blocked": the cache holds a short-term ban for this rider; the gateway answers 403 and the call reaches no backend.
Request trip: the rider confirms a ride; routed to trips. Scenarios "Session cached" and "Session not cached", as above.
Get rider profile: the app loads the rider's profile; routed to users. Scenarios "Session cached" and "Session not cached".
Use these use case and scenario names exactly: the traffic, requirements and tests in problem.proschi refer to them.
Scale
30k fare estimates, 4k trip requests and 6k profile reads a second: 40k calls a second through the edge. Uber does not publish the gateway's current rates for one region; these are assumptions for the exercise (its previous generation served 800k requests a second at peak).
95% of calls carry a token whose session is in the cache. About 2% of fare estimates come from a blocked rider. Both shares are assumptions.
The auth service runs 3 instances of about 2k calls a second each.
Constraints
No connection from the app to any backend service: every call goes through a load balancer and the gateway pool.
Authentication and bans are checked in the gateway, before the call is routed; backend services do not call the auth service.
A blocked rider's call never reaches a backend.
p99 of every use case under 100 ms.
Every use case available 99.95% of the time.
Losing any single machine must not break a latency limit.
At most $2,800 / month, the backend services included.
What is given
problem.proschi declares the rider app and four services owned by other teams: auth, pricing, trips and users, with their instances and capacity fixed by the problem. Add the edge (a load balancer, the gateway pool and the cache of sessions and bans), the connections and the three use cases. The tests find the edge by its kind: [AWS Load Balancer] or any load balancer, [API Gateway] or any gateway, [Redis] or any cache.
What this model leaves out
The session cache is this exercise's assumption. Uber's post says every endpoint runs authentication middleware and that the gateway keeps central caches with a TTL (short-term bans fed by fraud and security systems, data-center affinity), but not how a token check itself is served.
A cached session stays valid until its TTL runs out, even if the token is revoked meanwhile. TTLs are not simulated; the lesson covers the trade-off.
Rate limiting, circuit breakers per backend, timeouts, load shedding and protocol conversion are all part of Uber's gateway, but only the ban check is drawn as a scenario. A slow backend dragging the whole gateway down is described, not simulated.
The share of calls with a cached session is the scenario mix, not something the cache earns: warm-up and eviction are not modelled.
Based on
Madan Thangavelu, Abhishek Parwal and Rohit Patali, The Architecture of Uber's API gateway, Uber Engineering blog, May 2021: one entry point for all apps in front of the back-end services; routing, protocol conversion, rate limiting, load shedding and user access blocking at the edge; a request stack of protocol manager, middleware, endpoint handler and client, where a failing middleware short-circuits the call; authentication middleware required on every endpoint; endpoints declared as configuration and generated into Go code; a circuit breaker around each back-end client; and a central cache of short-term bans with a TTL.
Pavel Astakhov, Madan Thangavelu and Uday Kiran Medisetty, Designing Edge Gateway, Uber's API Lifecycle Management Platform, Uber Engineering blog, August 2020: the previous generation served 800k requests a second at peak in front of 400+ services owned by 100+ teams; a few APIs' performance regressions slowed the whole gateway and business logic leaked into it, and the third generation moved that logic out of the edge.
How your design is checked
You write the design as text in Proschi. Tests run in your browser: a simulation of the traffic above checks latency, availability, cost and what happens when a machine fails. How the simulation works.
What an API gateway is for, and what should stay out of it.
Why the gateway, not each backend, should run authentication and bans, and why a rejected call should stop there.
How a cache of validated sessions turns the auth service from the busiest service in the company into a quiet one, and what that cache costs you in revocation time.
Why a component that every request crosses needs more headroom than any other, and how to size it.
Uber's rider and driver apps talk to hundreds of backend services, owned by many teams. Letting the app call each of them directly would mean every service has to authenticate users, block abusers, limit rates and speak the app's protocols, and every app release would have to know where every service lives. Instead Uber puts one tier at the edge, the Edge Gateway: the apps know a single address, and the gateway forwards each call to the service that owns its path.
The gateway runs the same steps for every call. A protocol layer decodes the request (JSON, Thrift or Protobuf), then a stack of middleware runs: authentication, authorization, rate limits, bans. If a middleware rejects the call, the stack stops there and its answer goes back. Otherwise the endpoint handler maps the request to a backend call, and a generated client sends it with timeouts, retries and a circuit breaker. Endpoints are configuration that a team publishes, from which Uber generates the gateway's Go code.
The exercise keeps three endpoints:
Get fare estimate to pricing, with three scenarios: "Session cached", "Session not cached" and "Blocked" (a short-term ban answers 403 at the edge).
Request trip to trips, and Get rider profile to users, each with "Session cached" and "Session not cached".
The given has the rider app and four services owned by other teams: auth, pricing, trips and users. You add the load balancer, the gateway pool and a cache that holds sessions and bans. The limits: p99 under 100 ms for every endpoint, 99.95% availability, no latency limit broken by losing any one machine, and at most $2,800 a month. The tests check:
Every call starts at the app, passes a load balancer and then a gateway before it reaches a service, and the app has no path to any service.
Every endpoint reads the cache before it calls auth, a cached session never calls auth, and an uncached one SETs the session in the cache after auth answers.
The session is checked before the call is routed, and no backend calls auth itself.
A blocked rider's call never reaches pricing and is answered 403.
All the traffic numbers are assumptions for the exercise; Uber publishes its architecture, not one region's rates. They are scaled down, but the ratios are what matter.
Flow
Rate
Reaches
Fare estimates
30k / s
pricing (29.4k after the 2% blocked)
Trip requests
4k / s
trips
Profile reads
6k / s
users
Every call
40k / s
load balancer, gateway, cache
Session not cached (5%)
2k / s
auth
The auth service. Three instances at about 2k calls a second each make 6k. If the gateway asked it about every call, it would get 40k: almost seven times what it can serve. With 95% of sessions in the cache, it gets 5% of 40k, which is 2k a second, a third of its capacity, and still only half after losing one instance.
The gateway. One instance serves about 10k calls a second in the simulation. Four would run at 100%. Five run at 80% and pass on a normal day, but losing one puts the other four at 100%. To stay under about 70% busy with one instance gone, you need 40k ÷ 7k ≈ 6 instances left, so 7 in the pool (57% busy normally, 67% after a failure).
The cache. 40k reads (one per call) and 2k writes (one per miss) a second, on Redis instances that take 100k each: two of them, for redundancy more than for load.
Cost. The four services are fixed at $1,500. Two load balancers ($100), seven gateway instances ($700) and two cache nodes ($300) add $1,100, so $2,600 in all. The budget leaves room for two more gateway instances, not for a pool sized by guesswork.
An API gateway is a reverse proxy that understands your API. It takes every call from the outside, and does the work that is the same for every endpoint:
Concern
Without a gateway
With one
Authentication
Every service validates tokens
Once, in middleware
Bans and rate limits
Every service, or nobody
Once, before routing
Routing
The app knows every service's address
The app knows one address
Protocols
Every service speaks the app's protocol
The gateway converts
Logs and auditing
Scattered
One access log for every call
What stays out: business logic. Uber's previous gateway let product code creep in, and a few APIs' performance problems slowed the whole gateway, while the gateway team became the bottleneck for every product change. The third generation keeps the edge to cross-cutting work and moves product logic behind it.
Middleware are small steps that run in order before the endpoint's handler: authenticate, authorize, check bans, apply a rate limit. Each either lets the call continue or answers it. When one answers, the rest of the stack is skipped, including the backend call.
That is why a ban check belongs at the edge. A blocked rider's call that reaches pricing before anyone notices still costs a backend call, and depends on every service remembering to check. At the edge, it costs one cache read and a 403.
Uber keeps its short-term bans (users, app versions, other identifiers) in a central cache with a TTL, fed by its fraud and security systems. The gateway reads it on the way in.
Every endpoint needs to know who is calling. If the token is opaque, someone must look it up; if it is signed, someone must check the signature and whether it was revoked. Doing that by asking the auth service makes the auth service carry the whole app's traffic: it becomes the busiest service you have, and a hiccup there fails every endpoint.
A session cache fixes that. When the gateway sees a token, it reads the cache first. On a hit, it has the user id and carries on. On a miss, it asks the auth service, writes the answer to the cache with a short TTL, and carries on. This is plain cache-aside, applied to identity.
The cost is revocation time. A session in the cache stays valid until its entry expires, so a token revoked at the auth service keeps working for up to one TTL. Three ways to live with it:
Keep the TTL short (minutes, not hours).
Delete the entry on logout or password change.
Keep a separate ban list the gateway checks on every call, which is what Uber's short-term bans do.
Uber's post does not say how its authentication middleware serves token checks; the session cache is this exercise's assumption, built on the same pattern as the ban cache the post describes.
Deep dive Cache-aside on another example
The same shape protects any expensive lookup. Here a feature-flag service sits behind a cache:
A service that only some calls reach can fail and take only its feature with it. The gateway is different: every call crosses it, so if it slows down, everything slows down, and if it fails, the app is down. Three habits keep it from becoming the single point of failure:
Stateless instances behind a load balancer. Any instance can serve any call, so the pool grows by adding instances and loses one without losing anything but capacity. Sessions and bans live in the shared cache, not in an instance's memory.
Headroom for a failure. Size the pool so that with one instance gone it still runs under about 70%. Past that, queueing makes latency climb fast, and at 100% it collapses.
Isolation from slow backends. A circuit breaker around each backend client (Uber wraps every one) stops a slow service from tying up the gateway's capacity for everyone else. The simulation does not model that cascade, but an interviewer will ask.
1. Clarify. Which apps, which endpoints, how many calls a second? Who owns authentication, and what can the auth service take? Are there bans or rate limits to enforce, and must a rejected call stop at the edge? What latency and availability does the whole app need, given that everything passes the edge?
2. High level. Draw the app, one load balancer, a gateway pool and the services behind it. There is no line from the app to any service. Each endpoint is a route from the gateway to one service.
3. Middleware. Put the checks in order: read the session and the ban list from the cache in one round trip; on a ban, answer 403; on a miss, ask the auth service, then cache the session. Only then route the call. Write each endpoint's scenarios so the cached case never touches auth.
4. Size it. Count the calls on each tier: the load balancer, the gateway and the cache see every call; the auth service sees only the misses; each backend sees its own endpoint's calls. Size the gateway pool for a failure, not for the average, and keep two of the load balancer and the cache.
5. Check the budget and the tail. Add up the edge's cost against what is left after the services, then look at p99: a cached call is load balancer, gateway, cache and backend; a miss adds an auth call and a cache write.
6. Wrap up. Mention what else the edge gives you once it exists: one access log for auditing, rate limits per user or IP, protocol conversion, client SDKs generated from the same configuration, and a circuit breaker per backend.
Asking the auth service about every call (wrong/auth-on-every-call). Correct, and simple: the gateway validates every token with the auth service. That puts 40k calls a second on a service built for 6k; it saturates, and every endpoint's p99 goes with it. Caught by "Sessions are checked from a cache" and the latency limits.
Apps calling each service directly (wrong/apps-call-services-directly). No edge at all: each service validates tokens itself. Every service carries its own copy of the auth logic, the auth service still gets every call, and the app depends on every service's address. Caught by "Every call enters through the edge" and "The session is checked before the call is routed".
A gateway pool with no headroom (wrong/gateway-no-headroom). Five instances handle 40k calls at 80% and pass on a normal day. Losing one puts the remaining four at 100%, and every endpoint in the app fails together. Caught by survive any node failure. Even before a gateway is lost the pool is too tight: at 80% it queues enough that losing a single trips instance pushes Request trip's p99 past 100 ms.
Backends checking the ban list (wrong/backend-checks-bans). The gateway authenticates but leaves bans to each service, so a blocked rider's call is routed to pricing before anyone says no. Caught by "Blocked riders never reach a backend".
Classic mistakes beyond the tests:
Putting business logic in the gateway, so every product change needs the gateway team and one slow endpoint slows them all.
Keeping sessions in each gateway instance's memory, so the hit rate drops as the pool grows and a restart empties it.
A session TTL of hours, so a revoked token keeps working long after logout.
Retrying a failing backend from the gateway without a circuit breaker, which multiplies the load on the service that is already struggling.
A user logs out. How soon does the token stop working? Within one TTL of the cached session, unless logout deletes the entry. Bans take effect on the next call, because the ban list is read every time.
The cache is down. What now? Not a blind fallback to the auth service: 40k calls a second would sink a service built for 6k, and every endpoint with it. Let through only what the auth service can take and shed the rest, or fail closed for sensitive endpoints. Two cache nodes make this rare, and the choice is per endpoint.
How do you rate limit across a pool of gateways? Counters in a shared store, or a local limit per instance set to the global limit divided by the pool size; Uber can change quotas without redeploying.
Why not JWTs and no auth calls at all? Signed tokens remove the lookup but cannot be revoked before they expire, so you still need a ban or revocation list: the same cache, checked every call.
How do teams add endpoints without the gateway team? Uber's teams publish configuration (path, schema, target service, middleware) through a UI backed by Git, and the gateway is generated from it.
Madan Thangavelu, Abhishek Parwal and Rohit Patali, The Architecture of Uber's API gateway (Uber, 2021): the request stack, middleware, configuration and code generation, and the central caches.