Mario Ruiz Díaz
← All articles

Backpressure: From Freeway On-Ramps to Kafka

In 2000, Minnesota switched off its freeway ramp meters for eight weeks, and the freeways got slower, not faster. The red light on the ramp was protecting the road. Software calls that light backpressure. Nine places where it lives, the three ways the slow-down signal reaches the source, and two real cases: Kafka at Pluto TV and capped queue workers at Afterhour.

Mario Ruiz Díaz13 min read

Backpressure: From Freeway On-Ramps to Kafka

Every weekday, a few hundred traffic lights around Minneapolis and St. Paul make drivers wait at the bottom of the freeway on-ramps. In 2000 the Minnesota Legislature asked whether they were worth it, and the Minnesota Department of Transportation answered with an experiment: it switched all of them off, system-wide, from October 16 to December 8 (Mn/DOT, Twin Cities Ramp Meter Evaluation, 2001).

Drivers stopped waiting on the ramps. With the meters on, the average wait at a metered ramp during the peak had been 2.3 minutes; with them off, it essentially disappeared. And the freeways got worse. During peak traffic, freeway throughput fell 14%, travel times on the tested corridors rose 22%, travel time became almost twice as unpredictable, and peak-period crashes rose 26%. The meters came back, fewer of them and with a new rule: no driver waits more than 4 minutes at a local ramp (FHWA, Twin Cities case study).

The red light on the ramp was not slowing the trip down. It was protecting the road. When every car merges the moment it arrives, the merge turns into braking, the braking turns into a jam, and the jam lowers the number of cars the road can move. Holding cars at the entrance, where waiting is cheap, kept the shared part moving.

Software calls that light backpressure. When a consumer can't keep up with its producer, something has to give. Without backpressure, the excess piles up wherever it lands: in a queue, in memory, in a database connection pool, in a third-party API, until that thing breaks. With backpressure, the consumer's limit becomes a signal that travels upstream, and the source slows down to the rate the system can actually serve.

The same producer and consumer, twice. Without a limit, the queue between them absorbs the difference until memory runs out, and the failure only arrives later. With a bounded queue, a full queue tells the producer to wait, so the producer runs at the consumer's pace.NO LIMITBACKPRESSUREProducer1,000 items/sUnbounded queuegrows until memory runs outConsumer800 items/sProducerwaits: 800 items/sBounded queuefull means waitConsumer800 items/sfull: wait

Scroll sideways to see the full diagram →

The same producer and consumer, twice. Without a limit, the queue between them absorbs the difference until memory runs out, and the failure only arrives later. With a bounded queue, a full queue tells the producer to wait, so the producer runs at the consumer's pace.

The detail that matters is in the top lane. A queue between the producer and the consumer looks like a solution, because it absorbs the difference for a while. But a queue with no limit is not backpressure: it only moves the overload into memory and delays the failure. Backpressure starts when something says wait.

That signal can reach the source in three ways:

  • Hold them at the entrance: the shared resource tells the entrance how much room is left, and the entrance lets in only that much.
  • Let the consumer pull: nothing moves until the consumer asks for it, so it can never be handed more than it asked for.
  • Bound the buffer and say wait: a buffer with a limit sits between them, and when it fills up the source has to wait, or is told to come back later.
Three ways the slow-down signal reaches the source. Hold them at the entrance: the shared resource tells the entrance how much room is left. Let the consumer pull: nothing moves until the consumer asks for it. Bound the buffer: a full buffer makes the source wait, or tells it to come back later.HOLD THEM AT THE ENTRANCELET THE CONSUMER PULLBOUND THE BUFFER AND SAY WAITSourcecars, flights, a TCP senderGate at the entrancelight, gate hold, windowShared resourceroad, runway, receiverSourceparts store, topic, publisherConsumernext station, consumer, subscriberSourcestream, producer, clientBounded bufferhigh-water mark, cap, quotaConsumerdrains at its own paceroom leftasks for nsends only nfull: wait or 429

Scroll sideways to see the full diagram →

Three ways the slow-down signal reaches the source. Hold them at the entrance: the shared resource tells the entrance how much room is left. Let the consumer pull: nothing moves until the consumer asks for it. Bound the buffer: a full buffer makes the source wait, or tells it to come back later.

Nine places where backpressure lives, grouped by how the signal travels.

Hold them at the entrance

1. Freeway on-ramps

The pressure. At rush hour, more cars want to enter the freeway than the merge can absorb without braking.

The signal. A ramp meter: a light at the bottom of the ramp that lets one car go per green, at a rate set by what the freeway can take. The ramp waits so the road doesn't jam. The Minnesota experiment above is the rare case where someone measured what happens when you remove it.

Where it breaks. The ramp has its own limit. If the queue on the ramp grows long enough to back up into the street, the problem has moved, not disappeared. That is why Minnesota's meters now cap the wait at 4 minutes on local ramps and stop metering when the queue reaches city streets. Every backpressure signal needs a place where waiting is cheap, and that place needs a limit too.

2. Flights to a congested airport

The pressure. Weather or a staffing shortage lowers how many planes an airport can land per hour, but the flights headed there are already scheduled.

The signal. The FAA holds them at the origin. A Ground Delay Program is used when "the projected traffic demand is expected to exceed the airport's acceptance rate for a lengthy period of time", and in a Ground Stop, flights headed to the affected airport "are held at their departure point" (FAA, Air Traffic Control System Command Center). Each delayed flight gets an assigned departure time.

Why at the origin. It is the same choice as the ramp meter: a plane waiting at the gate costs passengers' time; a plane waiting in the air over a full airport costs fuel and safety margin. Backpressure pushes the wait to where it is cheapest.

3. TCP connections

The pressure. A sender can push bytes much faster than a busy receiver can read them.

The signal. Every TCP segment carries a window: "the range of sequence numbers the sender of the window (the data receiver) is currently prepared to accept", which is related to its free buffer space (RFC 9293, Transmission Control Protocol). As the receiver's buffer fills, the window shrinks; at zero, the sender stops and only sends small probes until the window opens again.

Why it matters. This is backpressure built into the network itself. It is also why a slow reader on one side of a socket eventually blocks the writer on the other side, even across machines.

Let the consumer pull

4. Toyota's production line

The pressure. In a factory, if each station produces as fast as it can, inventory piles up in front of the slowest one.

The signal. Toyota inverted the flow. Just-in-Time means making "only what is needed by the customer, when it is needed, and in the amount needed", and the next process picks up the parts it needs from the store of the process before it, which then makes only enough to replace what was taken (Toyota, Toyota Production System). Demand flows backwards; parts flow forwards.

Why it matters. Pull makes the consumer's pace the system's pace by construction. Nobody needs to send a "slow down" signal, because nothing was sent that wasn't asked for.

5. Kafka consumers

The pressure. Producers write events faster than some consumers can process them, at least for a while.

The signal. Kafka consumers pull. Kafka's design documentation explains why: in a push system, "the consumer tends to be overwhelmed when its rate of consumption falls below the rate of production", while "a pull-based system has the nicer property that the consumer simply falls behind and catches up when it can" (Apache Kafka, Design: push vs. pull). The backlog waits in the log, on disk, and shows up as consumer lag.

Where it breaks. Falling behind is safe only while the log retains the data and the business can tolerate the delay. Lag is a debt: watch it, alert on it, and know how long you can carry it.

6. Reactive Streams

The pressure. Asynchronous code (an HTTP client, a database driver, a stream of events) can emit results faster than the code receiving them can handle.

The signal. The Reactive Streams specification makes demand explicit: a subscriber calls request(n), and the publisher's onNext signals "MUST be less than or equal to the total number of elements requested" (Reactive Streams specification). The libraries that implement it (Project Reactor, RxJava, Akka Streams) carry that contract through every stage of a pipeline.

Why it matters. It is the pull model as an interface. Any stage that ignores demand breaks it for everyone downstream.

Bound the buffer and say wait

7. Node.js streams

The pressure. Piping a fast readable stream (a file, a socket) into a slow writable one.

The signal. A writable stream has a highWaterMark, 16 KB by default. Once its buffer exceeds it, write() returns false; the readable should pause and resume on the 'drain' event (Node.js, Backpressuring in Streams). pipe() and pipeline() do this for you.

Where it breaks. Ignoring the return value of write(). The Node.js guide measures it: the same file transfer used about 1.52 GB of memory when the return value was ignored, against about 88 MB when it was respected. Nothing fails right away; the process just eats the machine.

8. Lambda reading from SQS

The pressure. A queue receives a burst of messages, and Lambda, which scales out to read it, would start as many workers as it can. Each worker opens its own database connections and makes its own calls to other services.

The signal. Since January 2023, the SQS event source can be given a maximum concurrency, which caps how many invocations read from that queue at the same time (AWS, What's New, January 12, 2023). Before that, the only way was reserved concurrency on the function, which AWS notes came "at the cost of less consistent throughput and retrying messages due to function throttling". With the cap, the burst stays in the queue, where waiting is cheap.

Where it breaks. The cap protects what is behind the workers, not the queue. If the steady rate of messages is above what the capped workers can process, the queue grows without end. That case is in the Afterhour story below.

9. HTTP APIs

The pressure. Clients send more requests than a service can serve.

The signal. 429 Too Many Requests, which "MAY include a Retry-After header indicating how long to wait before making a new request" (RFC 6585). It is backpressure across a trust boundary: the service can't make the client wait, so it tells the client to.

Where it breaks. Clients that retry immediately, or all at the same moment. A 429 only works if the client honors Retry-After and adds jitter, which is the subject of the previous article on thundering herds.

Choosing the signal

If…Reach forExamples
The source is outside your control, and waiting before entry is cheapHold them at the entranceRamp meter, ground delay, admission control
You own both sides and the consumer can askLet the consumer pullKafka, Reactive Streams, work queues that workers poll
The source pushes and you can make it waitBound the bufferBounded queues, stream high-water marks, concurrency caps
The source pushes and you can't make it waitSay no, with a time to come back429 with Retry-After, or shed the load

How backpressure fails

  • A queue with no limit. It hides the problem until memory, disk or the oldest message's age runs out. Every buffer needs a bound, and a decision about what happens when it is reached.
  • A signal that stops halfway. Backpressure has to travel all the way to something that can actually wait. An unbounded buffer anywhere in the middle (a thread pool queue, an in-memory list, an outbox without a limit) absorbs the signal and the source never hears it.
  • Waiting longer than anyone cares. If a request waits in a queue longer than its caller's timeout, the system does the work for nobody. Bound the wait, not just the size.
  • One line for urgent and bulk work. A slow bulk job in front of an urgent one is backpressure applied to the wrong thing. Separate the lanes.
  • Retries that add to the pressure. A rejected client that retries right away turns a 429 into more load. Back off, with jitter, and with a budget.

Real-world cases: one queue that waits, and one log that lets the consumer fall behind

At Afterhour: capped workers behind the call webhook

Afterhour's call pipeline. The webhook only stores the raw payload and enqueues the call id. Two SQS queues hold the backlog, and Lambda reads each one with a maximum concurrency: 5 workers for call processing, 10 for everything else. Failed jobs come back later on a growing, jittered schedule, and after 16 attempts they go to a dead-letter queue. A burst of calls becomes a longer queue, not more connections to the database or more calls to the CRMs.INGESTQUEUES (SQS)CAPPED WORKERS (LAMBDA)Voice providerend-of-call webhookWebhook handlerstore payload, enqueue idHigh-priority queuecalls, lead alerts, triggersJobs queuedigests, exports, emailsDead-letter queueafter 16 attemptsCall workersat most 5 at a timeJob workersat most 10 at a timePostgres, CRMs, emailwhat the caps protectretry laterbounded load

Scroll sideways to see the full diagram →

Afterhour's call pipeline. The webhook only stores the raw payload and enqueues the call id. Two SQS queues hold the backlog, and Lambda reads each one with a maximum concurrency: 5 workers for call processing, 10 for everything else. Failed jobs come back later on a growing, jittered schedule, and after 16 attempts they go to a dead-letter queue. A burst of calls becomes a longer queue, not more connections to the database or more calls to the CRMs.

Situation. Afterhour is an AI phone receptionist for law firms, the platform I lead as Principal Engineer. When a call ends, the voice provider sends a webhook, and each call fans out into work: saving and structuring the call, notifying the firm, running its triggers, and syncing with its CRM. Calls don't end at an even rate, and every piece of that work talks to the same database and to third-party systems.

Task. Accept every webhook quickly and never lose a call, without letting a burst of calls become a burst of database connections and third-party requests.

Action. The pipeline was built this way before I joined, and I kept building on it, so this is the design as it runs, read from its code:

  • Persist first. The webhook handler only stores the raw payload and enqueues the call id. The fast path does almost nothing, so it can say yes to everything.
  • Two queues, two caps. Call processing, lead notifications and triggers go to a high-priority SQS queue; digests, exports, emails and the rest go to a regular one. Lambda reads them one message at a time, with a maximum concurrency of 5 and 10. A burst of calls becomes a longer queue, not more workers holding database connections at the same time, and bulk work can't stand in front of a call.
  • Retries that wait longer each time. A failed job comes back after 0 s, 15 s, 30 s, 1 min, 3 min and so on up to 12 hours, with ±20% randomness so failures don't return together. After 16 attempts, it goes to a dead-letter queue instead of circling forever.
  • The same rule in my own scripts. When I wrote the job that recovered missing call recordings from the voice provider, it ran through a pool of at most 3 concurrent requests by default, so a backfill over the whole call history couldn't flood the provider's API.

Result. The platform has processed more than 25,000 calls through this pipeline, and the recordings backfill recovered 833 missing recordings with zero errors. The caps say how much pressure the database and the third parties should feel at once; the queue absorbs the rest.

At Pluto TV: let the consumer fall behind

Pluto TV's frequency-capping write path. Viewer apps report every watched ad to the ad-beacon API, which publishes to Kafka and answers right away. The consumers pull from Kafka at their own pace and write the impression history and quota counters to Redis. If Redis slows down, the consumers fall behind and the backlog waits in Kafka; the apps and the beacon don't notice. The price is staleness: until the consumers catch up, the counters are behind.Viewer appsevery watched adad-beacon API~6k requests/sKafkabacklog waits hereConsumershistory writer, quotas managerRedis24 h of impressions, quotaspublish, answerpullat their pace

Scroll sideways to see the full diagram →

Pluto TV's frequency-capping write path. Viewer apps report every watched ad to the ad-beacon API, which publishes to Kafka and answers right away. The consumers pull from Kafka at their own pace and write the impression history and quota counters to Redis. If Redis slows down, the consumers fall behind and the backlog waits in Kafka; the apps and the beacon don't notice. The price is staleness: until the consumers catch up, the counters are behind.

Situation. At Pluto TV, a free ad-supported streaming service, I was the Tech Lead and designed the frequency-capping system that keeps viewers from seeing the same ad over and over. Every viewer app reports every watched ad. The worst case in six months of production metrics was roughly 6,000 impression requests per second at the beacon and 12,000 messages per second on the event bus.

Task. Keep 24 hours of impressions and per-rule quota counters up to date, without making the viewer apps, or the API they report to, depend on how fast the storage behind them is at that moment.

Action.

  • The beacon only publishes. The ad-beacon API writes each impression to Kafka and answers. It never waits for the storage.
  • The consumers pull. The history writer and the quotas manager read from Kafka at their own pace and write to Redis. If Redis slows down, they fall behind, and the backlog waits in Kafka, on disk, exactly where Kafka's design says it should.
  • Sized for 10x. The pipeline was built to handle ten times the worst case measured in production, so falling behind is the exception, not the operating mode.
  • The price is known. While the consumers are behind, the quota counters lag. The delivery path reads counters precomputed by those consumers, so a lag means a viewer could, for a while, be allowed one more repetition than the cap. That is the trade: a brief staleness in exchange for never pushing back on the viewer apps.

Result. Frequency caps shipped to production across every ad break a viewer sees, fed by that write path. The lesson I keep from both systems: decide where the waiting happens before the traffic decides for you, and make sure that place has a limit.

Read the full case: frequency capping for Pluto TV ads → · And the Afterhour platform →

The takeaway

Backpressure is not about going slower. The Minnesota freeways moved more cars with the lights on than with them off. It is about choosing where the waiting happens: at a ramp instead of on the road, at a gate instead of in the air, in a log instead of in a database, in a queue with a limit instead of in memory without one. Find the slowest thing your system depends on, follow the path back to the source, and make sure a "wait" can travel all the way.

References