Threads, ThreadPools and Executors – Concurrent Processing in Java
An in-depth look at how the various implementations of Java's Executor interface operate.
Concurrency ranks among the hardest problems we developers deal with in our everyday work. On top of that, it is also one of the problems we run into most often while handling routine tasks. It is exactly this mix of difficulty and frequency that makes concurrency and multithreading the most dangerous areas a software engineer can step into.
Moreover, tackling concurrency with low-level abstractions can be mentally demanding and tends to produce complicated, nondeterministic bugs. For this reason, most languages offer higher-level abstractions that let us handle concurrency-related problems fairly easily, without wasting time fiddling with low-level switches.
In this article, I want to take a closer look at one such abstraction from the Java standard library: the ExecutorService interface and its implementations. I also intend this article to serve as a starting point for my upcoming article on Benchmarking Java streams.
But first, a brief refresher on processes and threads
What Is a Process?
A process is the most basic unit that our computers can run independently. Processes let us divide the work done on our machines into smaller, more modular, and easier-to-manage pieces.
This lets individual pieces be more focused and therefore faster. Such a division also allows us to fully use the multiple cores built into our CPUs.
Generally speaking, every process is an instance of a specific program – a running Java process, for example, is an instance of the JVM program.
Furthermore, every process lives within the OS and has its own distinct set of resources, permissions, and behavior (the program code) – much like the users of our applications.
A process can contain multiple threads (at least in most OS), which cooperate to finish the shared tasks the process gives them.
What Is a Thread?
You can think of a thread as a branch of our code with its own set of instructions, running in parallel with whatever the rest of the application is doing. Threads make it possible for a single process to execute several instruction sequences concurrently.
At the software level, we can distinguish between two kinds of threads:
- Kernel (System) threads
These threads are handled directly by the OS. Their creation, scheduling, and management are carried out by the operating system kernel. - Application (User) threads
These threads are handled at the user level by a thread library or runtime environment, independently of the operating system. The operating system kernel cannot see them, so the operating system treats them as if they were single-threaded processes.
In this article, I concentrate mainly on Application threads. I will also touch on CPU-related threads, meaning hardware threads, which are a property of our CPUs. Their count reflects how many threads our CPU can handle at the same time.
In principle, threads share resources much more lightly than processes do. Every thread inside a process can reach all the data that belongs to its parent process.
In addition, each thread can hold its own data, usually called Thread local variables (or, in Java, the newer and preferred ScopedValues). Switching between Threads is also far cheaper than switching between Processes.
What Is a Thread Pool?
A Thread Pool is a more specific notion than either a Thread or a Process. It concerns application threads and refers to a collection of such threads available for use within our application.
Its behavior is very simple: we take threads from the pool one at a time until none are left, and that’s all. There is one extra assumption, though – namely, that threads go back to the pool after finishing their task.
Naturally, an application can have several thread pools, and the more specialized each pool is, the better. This approach reduces contention inside the application and eliminates single points of failure. Today’s industry standard is to have at least a dedicated thread pool for database connections.
Threads, ThreadPools, and Java
In older Java versions – before Java 21 – every thread used in an application was tied to a CPU thread. As a result, threads were fairly costly and heavyweight.
If you accidentally (or deliberately) create too many threads in your Java application, for instance by calling “new Thread()”, you can exhaust resources very fast, and your application’s performance will drop sharply – partly because the CPU has to do a great deal of context switching.
Project Loom, delivered with the Java 21 release, set out to solve this problem by introducing virtual threads – threads not tied to CPU threads, also known as green threads – into the Java standard library.
Java implements the thread pool concept through ThreadPoolExecutor – a class representing a finite-size thread pool whose upper limit is set by the maximumPoolSize parameter of its constructor.
As an aside, this Executor also serves as the internal thread pool inside more sophisticated executors.
Executor, ExecutorService, and Executors
Before describing the more advanced implementations of the Executor interfaces that rely on ThreadPoolExecutor, I want to answer one more question: what exactly are Executor and ExecutorService?
- Executor
Executor is an interface with just a single method, execute, whose signature isvoid execute(Runnable command). The interface models a very simple operation – more precisely, it states that the implementing class is able to perform it: run a given runnable. Its goal is to separate the act of submitting a task from the details of how that task gets executed. - ExecutorService
ExecutorService is another interface that extends Executor. Its contract is far richer than Executor’s, with roughly 13 methods to override should we choose to implement it. Its primary purpose is to support managing and running asynchronous tasks by wrapping them in a Java Future. ExecutorService also extends the Autocloseable interface, so we can use it with try-with-resource syntax and close it in an orderly way. - Executors
The Executors class, by contrast, is a utility class. It is the preferred way to create new executor instances – for most executors, using the new keyword is discouraged. It also offers methods for creating different kinds of callable instances, such as a callable that returns a static value.
Now that these three fundamental concepts are covered, we can look at the various executor service implementations.
The Executor Services
At the moment, the Java standard library offers 4 main implementations of the ExecutorService interface, each with a set of more or less distinctive features.
Here they are:
- ThreadPoolExecutor
- ForkJoinPool
- ScheduledThreadPoolExecutor
- ThreadPerTaskExecutor
On top of these, Executors.class contains three private static implementations of ExecutorService:
- DelegatedExecutorService
- DelegatedScheduledExecutorService
- AutoShutdownDelegatedExecutorService
All in all, the dependencies between these classes look roughly as follows.

ThreadPoolExecutor
As mentioned earlier, this is Java’s implementation of the Thread Pool concept. The executor represents a bounded thread pool with a varying number of threads. In practice, this means TheadPoolExecutor works with a finite number of threads, and that number can never exceed the limit set when the Pool was created.
For this purpose, ThreadPoolExecutor relies on two variables, corePoolSize and maximumPoolSize. The first, corePoolSize, defines the minimum number of threads in the pool – the pool keeps them alive even when they are idle.
The second, maximumPoolSize, defines – as you have probably guessed – the largest number of threads the Pool may own. It is the upper limit on threads within the Pool, and the Pool will never exceed this parameter’s value.
ThreadPoolExecutor also uses a BlockingQueue under the hood to keep track of incoming tasks.
How ThreadPoolExecutor Behaves
By default, when fewer threads are currently running than corePoolSize, calling the execute method creates a new thread and hands it the incoming task as its first job – even if some threads in the pool happen to be idle. If the pool cannot add a new Thread for whatever reason, it falls back to the second behavior.
When the number of running threads is equal to or greater than corePoolSize, or the pool failed to create a new Thread, calling execute leads to an attempt to put the new task in the queue.
isRunning(c) && workQueue.offer(command)
If the pool is still running, a new thread is added to it without a first task – the only situation in which a thread is created with no task at all.
addWorker(null, false);
If, on the other hand, the pool is no longer running, the new command is removed from the queue
!isRunning(recheck) && remove(command)
and the pool then rejects the command with a RejectedExecutionException
reject(command);
If the task cannot be added to the queue for some reason, the pool tries to start a new thread with that task as its first one.
else if (!addWorker(command, false))
If this also fails, the task gets rejected and a RejectedExecutionException is thrown with a message along these lines.
Task X rejected from java.util.concurrent.ThreadPoolExecutor@3a71f4dd[Running, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 0]

The picture above shows a heavily simplified view of ThreadPoolExecutor’s internal state. From this point, one of two things can happen: either Task4 (or Task5) finishes processing before Task6 is submitted to the pool, or Task6 gets submitted before Task4 (or Task5) is done.
The first case is rather dull, since nothing changes from the Executor’s point of view, so I won’t dwell on it.
The second case is far more interesting because it changes the Executor’s state. Since fewer threads are running than corePoolSize, submitting Task6 to the Executor causes a new worker to be created for it. The resulting state will look roughly like the one below.

ThreadPoolExecutor Pool Size
- Fixed-size Pool
Giving corePoolSize and maximumPoolSize the same value effectively produces a fixed-size thread pool: the number of threads the pool runs will never drop below or rise above that value – at least not for long. - Unbounded Pool
Setting maximumPoolSize to a sufficiently large value, such as Integer.MAX_VALUE, makes the pool practically unbounded. The ThreadPoolExecutor implementation does impose a practical cap of about 500 million ((2^29)-1) threads, but I’d wager your machine will crash long before hitting it.
If you want to understand why this particular number is the limit, there is an excellent JavaDoc explaining it, placed right after the ThreadPoolExecutor class declaration. As a hint: it has to do with how ThreadPoolExecutor stores its state.
Creating a ThreadPoolExecutor
The Executors class offers 6 methods in total for creating a ThreadPoolExecutor. I’ll go through them in pairs, since that is how they are meant to be used.
public static ExecutorService newFixedThreadPool(int nThreads)
public static ExecutorService newFixedThreadPool(int nThreads, ThreadFactory threadFactory)
These methods produce a fixed-size thread pool in which the core and max sizes are the same. You can also supply a threadFactory as an argument if you’d rather not rely on the standard library’s default one.
Executors.newFixedThreadPool(2);
Executors.newFixedThreadPool(2, Executors.defaultThreadFactory());
The next pair of methods:
public static ExecutorService newCachedThreadPool()
public static ExecutorService newCachedThreadPool(ThreadFactory threadFactory)
These methods produce thread pools that are effectively unbounded, because maxPollSize is set to Integer.Max; as with FixThreadPool, the second variant lets you pass a custom ThreadFactory.
Executors.newCachedThreadPool();
Executors.newCachedThreadPool(Executors.defaultThreadFactory());
And the final pair:
public static ExecutorService newSingleThreadExecutor()
public static ExecutorService newSingleThreadExecutor(ThreadFactory threadFactory)
Both of these produce a Thread Pool backed by a single thread. Moreover, they create ThreadPoolExecutors wrapped in AutoShutdownDelegatedExecutorService, which exposes only ExecutorService methods – none of the ThreadPoolExecutor-specific methods are accessible.
In addition, the shutdown method is overridden using Cleanable and gets invoked once the object becomes Phantom reachable.
Submitting New Tasks to ThreadPoolExecutor
The standard way to hand new tasks to a ThreadPoolExecutor is through one of the variants of the submit method. Alternatively, you can call the execute method from the Executor interface directly, although this is not the recommended approach.
The execute method returns void rather than a Future, which gives you less control over the task’s execution – the decision is up to you. Either way, both approaches go through all the thread pool logic described above.
Runnable task = () -> System.out.print("test");
Future<?> submit = executorService.submit(task);
vs
executorService.execute(task);
ForkJoinPool
ForkJoinPool is an entirely separate ExecutorService implementation, and its key selling point is work-stealing.
Work-stealing is a fairly intricate idea that deserves a blog post of its own. At a sufficiently high level of abstraction, though, it can be explained quite simply: every thread in the pool tries to execute any task submitted to the pool, regardless of which thread originally owned it.
Hence the name work stealing – threads “steal” work from one another. In theory, this should bring noticeable performance improvements, particularly when the submitted tasks are small or create further subtasks.
I plan to publish a dedicated post about work-stealing and ForkJoinFramework in the near Future. In the meantime, you can learn more about work-stealing here.
How ForkJoinPool Behaves
This executor belongs to the Java ForkJoin framework, introduced in Java 7 – a collection of classes intended to make parallel task processing more efficient through work stealing. Today, the framework is used heavily in the context of CompletableFuture and Streams.
To get the most out of ForkJoinPool, I suggest learning the ForkJoin framework as a whole and then adapting the way you handle such tasks so that it fits ForkJoin’s requirements better.
Before diving headfirst into the ForkJoin framework, run benchmarks and performance tests, since the actual gains might fall short of expectations.
The ForkJoinPool Java docs also include a table listing which methods to use when interacting with ForkJoinPool to achieve the best results.
The key parameter for ForkJoinPool is parallelism, which specifies how many worker threads the pool uses. By default, it matches the number of available processors on your CPU. This is at least an adequate setup in most cases, and I’d advise against changing it without proper performance testing.
Bear in mind that Java threads are CPU bound, so we can quickly exhaust the processing power needed to make progress on our tasks.
Creating ForkJoinPool Instances
The Executors class offers two methods for creating ForkJoinPool instances
Executors.newWorkStealingPool();
Executors.newWorkStealingPool(2);
The first creates a forkJoinPoll with the default parallelism, i.e. the number of available processors, while the second lets us choose the parallelism level ourselves.
Furthermore, Executors create a ForkJoinPool backed by a FIFO queue, whereas ForkJoinPool’s own default (for example, through new ForkJoinPool(2)) is LIFO. LIFO cannot be used with Executors. You can still change the underlying queue type through the asyncMode constructor parameter of the ForkJoinPool class.
With the FIFO setting, ForkJoinPool may be a better fit for tasks that are never joined – for example, when used with callables or runnables.
ScheduledThreadPoolExecutor
ScheduledThreadPoolExecutor builds a layer on top of the classic ThreadPoolExecutor that makes it possible to schedule tasks.
It offers three kinds of scheduling:
- run once after a fixed delay – schedule
- run every given number of time units – scheduleAtFixedRate
- run with a fixed delay between executions – scheduleWithFixedDelay
Of course, the “regular” ExecutorService API is available too. Just keep in mind that here the submit and execute methods are equivalent to calling schedule with a 0 delay – that is, the given task runs immediately.
Since ScheduledThreadPoolExecutor extends ThreadPoolExecutor, parts of its implementation match the classic ThreadPoolExecutor. However, it relies on its own task implementation, ScheduledFutureTask, and its own queue, DelayedWorkQueue.
In addition, ScheduledThreadPoolExecutor always uses a fixed-size ThreadPoolExecutor as its underlying thread pool, which means corePoolSize and MaxPoolSize are always the same.
That said, the ScheduledThreadPoolExecutor implementation hides a catch or two.
- First, when two tasks are scheduled (or end up scheduled) for the same moment, they run in FIFO order according to submission time.
- The second catch follows logically from the first: nothing truly guarantees that a given task will run at a specific moment – it may, for instance, be stuck waiting in the queue described in the previous point.
- Finally, if the executions of two tasks were to overlap for some reason, the thread pool guarantees that the first one’s execution “happens before” the later one’s. In other words, FIFO order applies, even for tasks submitted from different threads.
Creating a ScheduledThreadPoolExecutor
The Executors class offers five ways of creating a ScheduledThreadPoolExecutor, grouped similarly to the ones for ThreadPoolExecutor.
ScheduledThreadPoolExecutor with a fixed thread count:
Executors.newScheduledThreadPool(2);
Executors.newScheduledThreadPool(2, Executors.defaultThreadFactory());
The first method creates a ScheduledThreadPoolExecutor with a given number of threads, and the second additionally lets us supply a ThreadFactory of our choosing.
ScheduledThreadPoolExecutor with one thread:
Executors.newSingleThreadScheduledExecutor();
Executors.newSingleThreadScheduledExecutor(Executors.defaultThreadFactory());
These create a SingleThreadScheduledExecutor whose underlying ThreadExecutorPool contains just one thread. What we actually get is an instance of DelegatedScheduledExecutorService that delegates to a ScheduledThreadPoolExecutor, so ultimately the delegate’s underlying ThreadExecutorPool has a single thread.
The final way of creating a ScheduledThreadPoolExecutor is through:
Executors.unconfigurableScheduledExecutorService(new DummyScheduledExecutorServiceImpl());
This method wraps your own implementation of the ScheduledExecutorService interface in DelegatedScheduledExecutorService – one of the private static classes inside Executors – which exposes only the methods of the ScheduledExecutorService interface.
In a way, it can be seen as an encapsulation helper: your implementation may have many public methods, but once wrapped in the delegate, they are all hidden from users.
I’m not really a fan of this approach to encapsulation – that should be the implementation’s concern. Then again, perhaps I’m overlooking some other important use case for these delegates.
ThreadPerTaskExecutor
This is one of the most recent additions to the Java standard library and the Executors class. Rather than a thread pool implementation, it is a thread spawner. As its name implies, every submitted task receives its own thread tied to its execution, and that thread starts when processing of the task begins.
To make this work, the executor uses its own custom Future implementation, ThreadBoundFuture. The lifecycle of threads created by these executors looks roughly like this:
- The thread is created at the same moment as the Future,
- The thread begins running only once the Executor starts it programmatically
- The thread is interrupted whenever the Future is interrupted
- The thread stops when the Future completes
Additionally, when the Executor cannot start a new thread for a given task and no exception occurs in the process, the Executor will RejectedExecutionException.
ThreadPerTaskExecutor also maintains a set of up- threads. Each time a thread starts, it is added to the set, and each time a thread stops, it is removed from it.
Through the threadCount() method, you can use this set to check how many threads the Executor is running at any given moment.
Creating a ThreadPerTaskExecutor
The Executors class offers two ways of creating this Executor, and in my view one is preferable to the other. Let’s begin with the one I don’t recommend.
Executors.newThreadPerTaskExecutor(Executors.defaultThreadFactory());
This method creates a ThreadPerTaskExecutor using the given thread factory. The reason I, at least, advise against it is that such a ThreadPerTaskExecutor instance runs on plain old CPU-bound Java threads.
In that situation, pushing a large enough number of tasks through the Executor can very, very easily drain your application’s processing power.
Admittedly, nothing prevents you from pulling the following “trick” and using virtual threads regardless.
Executors.newThreadPerTaskExecutor(Thread.ofVirtual().factory());
But there is no point in doing so when you can just use this instead.
Executors.newVirtualThreadPerTaskExecutor();
This ThreadPerTaskExecutor instance makes full use of Java 21’s virtual threads. This configuration should also substantially raise the number of tasks your Executor can handle before it runs out of processing power.
Summary
As shown above, Java offers a range of executors, from the classic ThreadPool implementation in ThreadPoolExecutor to more sophisticated ones such as ThreadPerTaskExecutor, which fully exploits virtual threads, a Java 21 feature.
Moreover, every Executor implementation has its own distinctive characteristic.
- ThreadPoolExecutor – the classic ThreadPool implementation
- ForkJoinPool – work-stealing
- ScheduledThreadPoolExecutor – periodic task scheduling
- ThreadPerTaskExecutor – use of virtual threads and the ability to run each task in its own short-lived thread.
Despite these differences, all the executors share one defining characteristic: each exposes an API that makes processing many tasks concurrently far simpler.
I hope this knowledge proves useful to you at some point in the Future. Thanks for reading.