Reference · Mechanics · Part 2 of 6

The throughput ceiling

Where does ~30 requests/second come from? The most important thing to understand up front: nobody set it. There is no line of config that says "cap at 30." The number falls out of arithmetic — and that arithmetic turns out to be Little's Law.

In Part 1 you met the two pools, and the crucial one: the 3 ForkJoin workers shared by the whole JVM. That "3" is the seed of everything here. Once you know how many workers there are, how many downstream calls each request makes, and how long each call takes, the maximum request rate is fixed. You don't get to choose it — you can only compute it.

The three ingredients

Only three numbers matter, and it's worth being clear about where each one comes from — because two of them are decisions and one of them is just the environment:

The formula

Here is the whole thing. Notice it is all multiply and divide — there is no "plus" anywhere. In particular, the fan-out divides: the more calls each request needs, the fewer whole requests fit through the workers.

throughput = ( workers × (1000 / latency_ms) ) ÷ calls_per_request

Walk it left to right with our numbers:

1. 1000 / latency_ms → calls per second, per worker. One call takes 50 ms; a second holds 1000 ms; so one worker manages 1000 / 50 = 20 calls every second. (The 1000 is doing nothing clever — it's only converting milliseconds into seconds.)
2. × workers → the whole pool. Three workers, each good for 20 calls/s: 3 × 20 = 60 calls per second in total.
3. ÷ calls_per_request → turn calls into requests. Each request spends 2 calls, so 60 ÷ 2 = 30 requests per second.

Three ways to see the same number

If the formula feels like a trick, here are three routes to it. They are the same maths from different angles, and they all land on 30.

2 · Worker-time accounting

Ask a different question: how much worker time does one request cost? It makes 2 calls, each occupying a worker for 50 ms:

cost of one request = latency × calls = 0.05 s × 2 = 0.1 s  (100 ms of worker time)

Now ask how much worker time you have to spend each second. Three workers each give one second of work per second, so:

budget = 3 workers × 1 s = 3 worker-seconds/second = 3000 ms/second
throughput = 3000 ms/s ÷ 100 ms/request = 30 requests/second

3 · Little's Law

This is the formal version, and it's why the earlier lessons kept invoking it. Little's Law says:

L = λ × W

where, plainly:

• L — how many things are inside the system at once,
• λ (lambda) — the throughput, things finished per second,
• W — how long each thing spends in the system.

Rearrange it to solve for throughput: λ = L ÷ W. Apply it right at the bottleneck — the workers:

• L = how many calls can be running at the same time? One per worker, so 3.
• W = how long is one call? 0.05 s.
• λ = 3 ÷ 0.05 = 60 calls per second → then ÷ 2 for the fan-out → 30 requests per second.

Same answer again. The "ceiling formula" is simply Little's Law with L pinned to the number of workers (because that's the most calls that can run at once) and one final divide for the fan-out.

The tell that it's calculated, not configured

Here's the clean giveaway. When we actually measured the running service, it topped out around 25 req/s — not exactly 30. Real overhead (JVM warm-up, request handling, the cost of the join itself) shaves a little off the theoretical maximum.

The numbers that are settings

To sharpen the contrast: the demo does contain real dials — numbers someone typed and could change directly:

Those are genuine knobs. The 30 req/s is not a knob — it is the score you get when you multiply the three ingredients together.

Change an ingredient, move the ceiling

Because it's a product, the ceiling shifts predictably the moment any input changes. Start from the base case (3 workers, 2 calls, 50 ms → 30 req/s) and change exactly one thing:

More workers help; more calls-per-request hurt (the fan-out divides); slower calls hurt. None of these is a "throughput setting" — they're the ingredients, and the ceiling just re-computes itself.