A hands-on course

Thread Starvation & Throughput Ceilings

Why a Java service caps at ~30 req/s with the CPU idle and the connection pool "healthy" — built as a demo you run, predict, and prove.

Each lesson runs the same loop: predict a number from first principles → run the demo under load → read the dashboard → prove it with jcmd. The bug is a single missing argument to CompletableFuture.supplyAsync; the intuition is worth a career.

Lessons

1. 1
   Predict the ceiling

   Little's Law → ~30 req/s. Parked threads are normal, not broken.
2. 2
   The signature of queueing soon

   p99 diverges from p50 while the mean hides it.
3. 3
   The trap soon

   A blocked thread is not a blocked connection. The pool was starved, not misconfigured.
4. 4
   The fix soon

   A bounded, instrumented pool. Why 40 threads is right for I/O but wrong for CPU-bound.
5. 5
   Observe & alert soon

   What ExecutorServiceMetrics.monitor returns; why the common pool can't be wrapped; alert at queue > 0.
6. 6
   Fargate simulation soon

   ActiveProcessorCount, and what really happens at parallelism < 2.

Reference

Run the demo

The source is in demo/. You need Java 21, Maven, and Docker.

cd demo
docker compose up -d          # WireMock :8090, Prometheus :9090, Grafana :3000
./run-app.sh broken 4         # app :8080, actuator :8081
./load.sh 50 60               # apply load, then read Grafana :3000

Full walkthrough: the Runbook. The bug lives in one method — OrderController.supply(...) — and the fix is one argument.