The Optimal Thread-Pool Size in Java: Explaining the Formula

Determining the optimal size of a thread pool in Java is crucial to maximizing CPU and resource utilization. This decision depends on many factors. Here, we’ll show you how to use the famous formula from Brian Goetz’s “Java Concurrency in Practice.” We’ll give you a practical example of how to calculate the ratio of wait time to compute time (W/C).

The Formula

The formula for the optimal size of a thread pool is:

int optimalThreadPoolSize = numberOfCores * targetUtilization * (1 + waitTime / computeTime);

Where:

• numberOfCores is the number of CPUs, obtainable through Runtime.getRuntime().availableProcessors().
• targetUtilization is the desired CPU utilization, a value between 0 and 1.
• waitTime is the average time unit a thread waits, for instance, due to I/O operations.
• computeTime is the average time unit a thread performs actual calculations.

This formula helps you find the optimal number of threads to keep the processors at the desired utilization.

How to Calculate waitTime / computeTime

Suppose you have a process where the threads spend 20% of their time computing and 80% waiting for I/O. The wait time to compute time ratio (W/C) would then be 4.

You can use this information in the formula to calculate the optimal thread-pool size.

In Practice, You Must Estimate

In most cases, you won’t know exactly what the ratio between wait and compute time is. It pays to start with informed assumptions. Ultimately, you can only determine your optimal configuration through targeted load testing. In most cases, however, this will not be necessary with the formula mentioned above.

For Compute-Intensive Tasks

If the tasks mainly perform calculations, you’ll achieve optimal utilization with numberOfCores + 1 threads. Your waitTime is 0 in this case.

For I/O or Blocking Operations

If the tasks involve I/O or other blocking operations, you want a larger pool because not all threads can be scheduled all the time. For example, you may find that in your request, accessing disk to read a file takes about 10 ms. The rest of your business logic takes 5 ms. So for a 4-core system and targeted 80% CPU utilization this results in

int optimalThreadPoolSize = 4 * 0.8 * (1 + 10 / 5); // 9,6

So a thread pool size of 10 is a good start for this part of the application.

What We Learn From This

Determining the optimal thread-pool size in Java is not an exact science, but with an understanding of the tasks and the environment in which they are executed, this formula can provide strong guidance. By considering factors such as wait time and compute time, you can create a thread pool that optimally utilizes your system’s resources.