Reference · Mechanics · Part 1 of 6

Two thread pools

The whole bug starts here: your service quietly runs on two separate pools of threads, and blocking the tiny one freezes the big one. Get this picture straight and the rest of the series falls into place.

The bug, in one line

When you write CompletableFuture.supplyAsync(task) with no executor argument, the task runs on ForkJoinPool.commonPool() — a single, JVM-wide pool that you never created and that everything else in the process shares too. Add an executor argument and it runs there instead. That one missing argument is the entire defect:

// broken: no executor -> ForkJoinPool.commonPool()  (shared, tiny)
CompletableFuture.supplyAsync(() -> client.call("/service-a"));

// fixed: your own pool, sized for the work
CompletableFuture.supplyAsync(() -> client.call("/service-a"), myExecutor);

In the demo this is the single branch inside OrderController.supply(...). Everything downstream in this series — the ceiling, the tail, the runaway, the health-check outage — flows from which side of that if you land on. You met the symptom in Lesson 1; here we look at why it happens.

The two pools

Almost everyone pictures one pool of threads. There are two, and they do completely different jobs:

ForkJoin common pool

Runs the actual blocking downstream HTTP calls. Size = availableProcessors() − 1; on our "4-core" run (-XX:ActiveProcessorCount=4) that's 3. Shared across the entire JVM.

Tomcat threads

200

Receive incoming HTTP requests and answer them. Size = server.tomcat.threads.max: 200 — a real, typed setting. This is your front door.

The mismatch is the whole story: 200 threads at the front door, 3 threads doing the work behind it. The front door is wide; the kitchen is tiny.

What happens to a single request

Follow one call to /orders. It fans out to two downstream services at once, then waits for both:

A Tomcat thread picks up the incoming request.
It submits two tasks with supplyAsync — one for /service-a, one for /service-b. Both land on the 3-thread ForkJoin pool.
The Tomcat thread then calls a.join() and b.join(). join() parks that thread — it goes off-CPU and simply waits for the ForkJoin workers to finish.
Only when both results arrive does the Tomcat thread wake, assemble the response, and become free again.

Two facts make this poisonous:

The parked Tomcat thread is not a ForkJoinWorkerThread. It cannot help drain the ForkJoin queue while it waits — it just sits there.
ForkJoin assumes its workers are doing CPU-bound work that never blocks, so when a worker does block on I/O it creates no compensating thread. One blocked worker permanently removes 1/parallelism — a third here — of the whole JVM's async capacity, and nothing grows a replacement.

The restaurant

3 cooks in the kitchen are your 3 ForkJoin workers. 200 waiters are your 200 Tomcat threads.

A waiter takes an order, carries it to the kitchen, and then stands at the kitchen window waiting for the food before they can do anything else. That standing-and-waiting is join() — the waiter is occupied but useless, unable to seat anyone or take another order until the cooks are done.

With only 3 cooks, food comes out slowly. So waiters pile up at the window, frozen, holding trays. When all 200 waiters are stuck at the window, you have "200/200 busy" — a full floor of staff and nobody able to greet a new customer.

What "200/200 busy" really means

Each /orders request occupies one Tomcat thread for its entire duration, and that duration includes the join() wait. Under load, the waits stack up until every one of the 200 Tomcat threads is parked on a join():

200/200 Every front-door thread is occupied — but "occupied" means parked, doing nothing, waiting on the 3 starved workers. Zero threads are free to accept anything new.

This is the counter-intuitive part. "200/200 busy" sounds like the app is working flat out. It isn't working at all — it's 200 threads frozen mid-wait. CPU is near idle, memory is fine, and yet nothing can get in. We'll see in Part 6 that this is exactly the moment even a trivial health check can't be answered.

Don't confuse these two things A blocked thread is an active driver that has parked — it still holds its slot in the pool and is the thing you're short of. A blocked (leased) connection is an inert socket that does nothing on its own; it only moves data while a thread is driving it. In this demo the connection pool sat nearly idle and perfectly healthy the entire time. The shortage was threads, not connections — and the connection pool was never misconfigured. Blaming the pool is the classic misdiagnosis; hold the line on this distinction.

Take away

There are two pools: a tiny one (3 ForkJoin workers) where the real work happens, and a big one (200 Tomcat threads) that is just the front door.

Block the tiny pool and it backs up into the big one: Tomcat threads pile up parked on join() until "200/200 busy" — a fully occupied front door that can't accept a thing, while the CPU sits idle.

Primary source JDK 21 ForkJoinPool javadoc — common-pool sizing is availableProcessors() − 1, and the pool is built for CPU-bound work with no compensation for blocked workers. JDK 21 CompletableFuture javadoc — async methods with no executor run on ForkJoinPool.commonPool().

Your teacher is in the chat. If the two-pools picture is still fuzzy, ask — I can re-run the demo live and show all 200 Tomcat threads going busy while the 3 workers sit parked in socketRead0.

PrevContents Contents NextPart 2: The throughput ceiling