Reference · Glossary

Thread-pool Starvation Glossary

Canonical language for this course. Half of this topic is people using one word — "pool", "blocked" — for two different things. Precision here is the point.

Thread starvation

A state where work is queued but no thread is available to run it, so throughput is capped below what the downstream resources could serve.

Avoid: "the pool is overloaded", "it's slow"

Common pool (ForkJoinPool.commonPool())

The single JVM-wide ForkJoinPool that CompletableFuture.supplyAsync(task) uses when given no executor. Parallelism defaults to availableProcessors() − 1.

Avoid: "the default thread pool" (too vague — there are several)

Parallelism (of a ForkJoinPool)

The target number of actively running worker threads — the hard ceiling on concurrent tasks. For the common pool: availableProcessors() − 1.

Avoid: "pool size", "thread count" (they differ)

Blocked thread

An active driver that has parked (gone off-CPU) waiting for something — here, an HTTP response. It holds its slot in the pool and does no work while parked. This is the normal state for network I/O, not a fault.

Avoid: conflating with "blocked connection"

Blocked connection (leased connection)

An inert socket borrowed from the connection pool. It does nothing on its own; it only moves data while a thread is actively driving it. "3 leased / 17 idle" means only 3 threads are driving — a symptom of too few threads, never proof the pool is small.

Avoid: treating idle connections as evidence the pool is misconfigured

Compensating thread

An extra worker a pool spawns to cover for one that has blocked, keeping parallelism effective. ForkJoinPool creates one only via ManagedBlocker; a plain blocking HTTP call gets none, so a parked common-pool worker permanently removes 1/parallelism of JVM-wide async capacity.

Avoid: assuming any pool "just makes more threads"

Fan-out

One inbound request spawning N concurrent downstream calls. This demo fans out to 2 services, so each request consumes 2 worker slots — halving the effective ceiling.

Throughput ceiling

The hard cap on requests/second set by workers × (1/latency) ÷ calls-per-request, independent of offered load. Here: 3 × (1/0.05 s) ÷ 2 ≈ 30 req/s.

Avoid: "max load", "capacity" (unqualified)

Little's Law

concurrency = throughput × latency (L = λW). Rearranged, it predicts the ceiling from worker count and call latency. The course's core equation.

p50 / p99 divergence

The signature of queueing. p50 (service time) stays flat while p99 (service + wait time) climbs with offered load. The mean hides it by averaging the two.

Avoid: reporting "average latency" as the health signal

Bulkhead

A bounded resource (pool + timeout) that fails fast under overload instead of queueing unboundedly. The connection pool with connectionRequestTimeout set is a bulkhead; without the timeout it silently becomes an unbounded queue.

Dedicated executor (the fix)

A ThreadPoolExecutor you construct, size to the I/O work, bound with an ArrayBlockingQueue, name, and instrument — then pass explicitly to supplyAsync(task, executor). Sidesteps the common pool entirely.

ActiveProcessorCount (-XX:ActiveProcessorCount=N)

A JVM flag forcing availableProcessors() to return N, reproducing a container CPU quota (cgroup) on any hardware. Mandatory on Apple Silicon to see the bug.