A hands-on course

Thread Starvation & Throughput Ceilings

Why a Java service caps at ~30 req/s with the CPU idle and the connection pool "healthy" — built as a demo you run, predict, and prove.

Each lesson runs the same loop: predict a number from first principles → run the demo under load → read the dashboard → prove it with jcmd. The bug is a single missing argument to CompletableFuture.supplyAsync; the intuition is worth a career.

Lessons

1. 1
   Predict the ceiling

   Little's Law → ~30 req/s. Parked threads are normal, not broken.
2. 2
   The signature of queueing

   p99 diverges from p50 while the mean hides it. p99 hit 17.5s while the mean read 0.8s.
3. 3
   The trap soon

   A blocked thread is not a blocked connection. The pool was starved, not misconfigured.
4. 4
   The fix soon

   A bounded, instrumented pool. Why 40 threads is right for I/O but wrong for CPU-bound.
5. 5
   Observe & alert soon

   What ExecutorServiceMetrics.monitor returns; why the common pool can't be wrapped; alert at queue > 0.
6. 6
   Fargate simulation soon

   ActiveProcessorCount, and what really happens at parallelism < 2.

Reference

How thread starvation works — the Mechanics series — a short, click-through book explaining the whole mechanism in six parts: the two thread pools, the throughput ceiling, the queueing tail, runaway starvation, why only some nodes fail, and the health-check cascade. Start here for the big picture.

Run the demo

The codeThe complete, runnable example behind these lessons — the Spring Boot service, the WireMock stubs, the Prometheus/Grafana dashboard, and the load scripts — lives on GitHub: github.com/TomSpencerLondon/thread-starvation-course. Browse it or clone it to run the exact simple example that reproduces every problem the lessons describe.

The source is also in demo/ in that repo. You need Java 21, Maven, and Docker.

cd demo
docker compose up -d          # WireMock :8090, Prometheus :9090, Grafana :3000
./run-app.sh broken 4         # app :8080, actuator :8081
./load.sh 50 60               # apply load, then read Grafana :3000

Full walkthrough: the Runbook. The bug lives in one method — OrderController.supply(...) — and the fix is one argument.