In Java 21, virtual threads were introduced, which is a very, very important feature. The Java coroutine that has been looking for before has finally come out. In highly concurrency and IO-intensive applications, virtual threads can greatly improve application performance and throughput.
## What is a virtual thread
Let’s first look at the concept of virtual threads.
### Virtual thread concept
DK 21 introduces support for virtual threads to improve the performance of Java applications in high concurrency scenarios. Virtual thread is a lightweight thread with a smaller memory footprint and more efficient context switching, suitable for I/O-intensive applications**.
### How virtual threads work
When an application starts a virtual thread, the JVM will hand over the virtual thread to the thread pool at the bottom of the JVM for execution. This underlying thread pool is a traditional thread pool, and the thread that truly executes tasks in the virtual thread is also a traditional thread ( operating system thread). When a virtual thread encounters a blockage, the JVM will immediately suspend the virtual thread and let other virtual threads execute. In other words, when opening a virtual thread, there is no need to enable a traditional thread. Generally, a traditional thread can perform tasks of multiple virtual threads. During the execution process, the virtual thread can be understood as a task task.
Here is a column. Assuming that the user has created 1000 virtual threads, the number of thread pool threads that execute the virtual thread of the JVM is 10. Then when the first virtual thread V1 needs to be executed, the JVM will schedule V1 to the traditional thread T1. And so on, the virtual thread V2 will be scheduled to the traditional thread T2, then V3->T3, V4->T4,… V10->T10. When executing to V11, here are three situations:
- If any thread in V1~V10 encounters a blockage, we assume here that V3 encounters a blockage, then the JVM will suspend V3. At this time, the T3 thread is available, and V11 is executed by T3.
- If no threads are blocked from V1~V10, then according to the divided time slice, the JVM assumes that each virtual thread is allowed to execute 100ns. After 100ns, V1 is executed latest, and the JVM suspends V1 and lets T1 execute V11 .
- If neither of the above situations is satisfied, then V11 will be suspended first, and then V11 will be executed when there is a traditional thread available.
For blocked threads, such as V3, when the IO is over, the operating system will notify the JVM through an event, such as epoll, that the IO operation of V3 has ended. At this time, the JVM wakes up V3 again and selects the available traditional threads to execute V3 tasks. .
Two points to note here:
- **After the execution of the virtual thread IO, the JVM will be notified through the operating system's event notification mechanism, such as epoll. **This is crucial for efficient scheduling of virtual threads, because it ensures that blocking I/O operations do not occupy the time slice of the operating system threads, and avoids high resource consumption and inefficiency of traditional thread pools. .
- **JVM is very low in context switching on virtual threads, because it does not involve thread context switching at the operating system level, it is very cheap and very fast. **
### Scheduling of virtual threads
Generally speaking, programmers do not need to manage the scheduling of virtual threads. In JDK 21, JVM enables virtual threads by default, and will use the default ForkJoinPool thread pool to execute virtual threads, and the size of the thread pool will also be based on the The number of virtual threads is dynamically adjusted. If you need to manually manage the thread pool size of the virtual threads, you need to customize the thread pool and hand over the virtual thread to a custom thread pool for execution. This, although feasible, is usually not necessary.
### The difference between virtual threads and traditional threads
The main difference between virtual threads and traditional threads is:
- When creating a virtual thread, the JVM will not create an operating system thread. When creating a traditional thread, the JVM will create an operating system thread. A traditional thread that can poll multiple virtual threads to execute.
- Virtual threads are executed by traditional threads. The scheduling of virtual threads is controlled by the JVM, and the execution and scheduling of traditional threads are controlled by the operating system.
- The context switching of virtual threads is controlled by the JVM, because it does not involve context switching of operating system-level threads. The context switching of virtual threads is very fast and can meet high concurrency requirements.
- Creating a virtual thread takes up very small memory. Relatively speaking, creating a traditional thread takes up a large memory space. In applications, a large number of virtual threads can be created, which generally supports millions, while traditional threads can usually only reach a few thousand. We generally do not recommend creating so many traditional threads.
Virtual threads are similar to tasks. Traditional systems correspond to operating system threads. A traditional thread can execute multiple virtual threads. The difference between a virtual thread and a task is that when a traditional thread executes a virtual thread, it will hang up the virtual thread when it encounters a blockage. When a traditional thread executes a task, it will really block when it encounters a blockage. Of course, in traditional tasks inherit from runnable, virtual threads inherit from Thread, they belong to different classes, and the callable methods are also different.
JDK also provides a virtual thread pool, and you can obtain a virtual thread pool through the following method.
```java
import .*;
public class VirtualThreadPoolExample {
public static void main(String[] args) {
// Create a virtual thread pool
ExecutorService executor = ();
// Submit multiple tasks to the thread pool
for (int i = 0; i < 10; i++) {
final int taskId = i;
(() -> {
("Task " + taskId + " running in " + ());
});
}
// Close the thread pool
();
}
}
```
In the above code, the JVM will create a virtual thread for the task submitted to the thread pool and then execute it in the form of a virtual thread.
Compared with traditional thread pools, ** virtual thread pool cannot set parameters such as core thread count, maximum thread count, thread pool size, task queue, etc., and these parameters are not required. **
The similarities between virtual threads and traditional threads:
- They all inherit from Thread, and the usage is the same. All support thread pools.
- Like traditional, virtual threads also have new, runnable, waiting, blocked, terminated and other states.
- All locks and synchronization mechanisms are applicable to virtual threads, and like traditional threads, virtual threads will also have resource competition and state synchronization problems. And there is also context switching, although the context switching of virtual threads is very small.
- The exception handling mechanism is the same. If an exception is not processed, the virtual thread will also terminate the execution.
### The difference between virtual threads and coroutines
Coroutines are asynchronous programming technology in Python. For IO-intensive applications, coroutines can play a great advantage. The asynchronous working principle of coroutines is similar to that of virtual threads. It also blocks when encountering IO, allowing the main thread to continue to perform other tasks. When IO is completed, the operating system notifies the python process through event mechanisms, such as epoll, to generate an event and put it in. The event loop queue is finally executed by the main thread.
The main difference between virtual threads and coroutines is:
| Differences | Virtual Threads
|--------|-------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------|
| Concurrency/parallel | Virtual threads are parallel, multiple virtual threads can run on multiple CPUs at the same time, and at the same time, multiple virtual threads can be run. From this perspective, virtual threads can support higher concurrency. | Coroutines are not parallel, because only one main thread executes task events, and at the same time, only one task is processed. |
| Resource competition | In virtual threads, there are resource competition problems and state synchronization problems. Concurrency control needs to be considered when writing code. It even needs to do reasonable concurrent design. | Because there is only one main thread executing task events, there is no concurrency problem, and there is no need to consider concurrency problem when programming. |
| Framework Support | Virtual threading is a new feature of JDK 21 and does not require any framework support. Moreover, asynchronous programming is somewhat difficult, and debugging becomes more complicated. |
## How to use virtual threads
In JDK 21, there are two ways to use virtual threads:
- Create and start virtual threads directly.
```java
public class VirtualThreadExample {
public static void main(String[] args) {
Thread virtualThread = ().start(() -> {
("Hello virtual thread ");
});
try {
(); // Wait for the virtual thread to complete
} catch (InterruptedException e) {
();
}
}
}
```
- Execute virtual threads through thread pool.
```java
import .*;
public class VirtualThreadPoolExample {
public static void main(String[] args) {
// Create a virtual thread pool
ExecutorService executor = ();
// Submit multiple tasks to the thread pool
for (int i = 0; i < 10; i++) {
final int taskId = i;
(() -> {
("Task " + taskId + " running in " + ());
});
}
// Close the thread pool
();
}
}
```
**When executing tasks through thread pools, it is impossible to implement concurrency control, which can easily cause OOM or consume service provider resources. You can customize the following virtual thread pool to achieve resource control: **
```java
package ;
import ;
import ;
import ;
import .*;
/*****
* Virtual thread pool, supports configuring the number of task queues and maximum number of concurrent tasks
*/
public class VirtualThreadExecutorService extends AbstractExecutorService {
private volatile boolean shouldStop = false;
private final ExecutorService executor = ();
private final Semaphore semaphore;
private final BlockingQueue<Runnable> taskQueue;
/******
* Constructor
* @param taskQueueSize, task queue size, task queue is a blocking queue. If the task queue is full, calling execute method will block
* @param concurrencySize, the size of concurrent tasks, the number of IO tasks executed at the same time, to prevent excessive concurrency or insufficient resources
*/
public VirtualThreadExecutorService(int taskQueueSize, int concurrencySize) {
= new Semaphore(concurrencySize);
taskQueue = new LinkedBlockingQueue<>(taskQueueSize);
();
}
private void loopEvent() {
().name("VirtualThreadExecutor").start(() -> {
while (!shouldStop) {
try {
Runnable task = ();
();
(() -> {
try {
try {
();
} finally {
();
}
} catch (Exception e) {
().interrupt();
throw new RuntimeException(e);
}
});
} catch (InterruptedException e) {
().interrupt();
if (shouldStop) break;
}
}
});
}
@Override
public void shutdown() {
shouldStop = true;
();
}
/**
* @return The task not executed
*/
@Override
public List<Runnable> shutdownNow() {
shouldStop = true;
List<Runnable> remainingTasks = new ArrayList<>(taskQueue);
();
();
return remainingTasks;
}
@Override
public boolean isShutdown() {
return shouldStop;
}
@Override
public boolean isTerminated() {
return shouldStop && ();
}
@Override
public boolean awaitTermination(long timeout, TimeUnit unit) throws InterruptedException {
return (timeout, unit);
}
@Override
public void execute(Runnable command) {
try {
(command); // Block until the queue has space
} catch (InterruptedException e) {
().interrupt();
throw new RejectedExecutionException("Task submission interrupted.", e);
}
}
}
```
The test code is as follows:
```java
package ;
import ;
public class VirtualThreadExecutorServiceDemo {
public static void main(String[] args) throws InterruptedException {
VirtualThreadExecutorService executorService = new VirtualThreadExecutorService(10, 2);
for (int i = 0; i < 100000; i++) {
final String threadName = "thread-" + i;
(() + ": try to create task " + threadName);
(() -> {
(() + ": " + threadName + " created!");
try {
(2000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
(() + ": " + threadName + " finished!");
});
}
(5000000);
}
}
```
## In which scenarios can virtual threads be used
Virtual threads can exert great power in IO-intensive high-concurrency applications. In all IO-intensive applications, specifically, it is more appropriate to use virtual threads in the following scenarios:
- Tasks that need to be completed in a short time, and there is no resource competition or out-of-order problem, such as database writes, server HTTP request processing, remote RESTful API calls, RabbitMQ message processing and other application scenarios. .
- Long-running tasks, but tasks required for message processing by order. For example, in the elevator monitoring system, the data of each elevator needs to be processed, but the order in which messages are processed must be ensured. At this time, a virtual thread can be created for each elevator, and the data of this elevator will be handed over to a special virtual thread for processing. Because a large number of virtual threads can be created in the application, and virtual threads are generally asynchronously processed tasks, in this scenario, using virtual threads can meet the requirements of high performance and high concurrency.
- In the API gateway, multiple upstream API data are queryed, assembled and merged, and used virtual threads. The effect is better than traditional threads. Virtual threads also support CountDownLatch, Semaphore and other tool classes.
- In an event-driven architecture, virtual threads are used, and the effect is also very good. For example, the asynchronous events in spring boot use a traditional thread pool by default. If it is changed to a virtual thread pool, the concurrent processing capability can be greatly improved.
So in which scenarios are not suitable for virtual threads?
- CPU-intensive applications, such as big data processing, image processing, matrix computing, etc.
- If the application has a high competition for concurrent resources, or state synchronization, and the system throughput is low, it is necessary to consider optimizing the concurrency model. In this scenario, not only traditional threads are not suitable, but virtual threads are not suitable.
## Examples of practical application scenarios for virtual threads
In a spring boot project, sometimes due to the inability to handle asynchronous events, resulting in a decrease in throughput. In JDK 21, events can be changed to virtual threads to execute, and the code is as follows:
```java
package ;
import ;
import ;
import ;
import ;
import ;
import ;
import ;
@Configuration
@EnableAsync
public class AsyncConfig {
@Bean(name = "taskExecutor")
public Executor taskExecutor() {
// Maximum number of parallel tasks
Semaphore semaphore = new Semaphore(100);
ExecutorService virtualThreadPool = ();
return runnable -> {
try {
// Control the number of parallel tasks
();
(() -> {
try {
();
} finally {
();
}
});
} catch (InterruptedException e) {
().interrupt();
throw new RuntimeException("Task submission interrupted", e);
}
};
}
}
```
The event sending and processing codes are as follows:
```java
package ;
import ;
import ;
import ;
import ;
import ;
import ;
@RestController
@RequestMapping("/home")
public class HomeController {
private final ApplicationEventPublisher eventPublisher;
public HomeController(ApplicationEventPublisher eventPublisher) {
= eventPublisher;
}
@GetMapping("/index")
public String index() {
for (int i = 0; i < 1000; i++) {
("event " + i);
}
return "success";
}
@EventListener
@Async
public void handleEvent(String event) {
(() + ": " + event);
try {
(100);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
}
```
The output result is as follows:
```
VirtualThread[#2031]/runnable@ForkJoinPool-1-worker-4: event 976
VirtualThread[#2039]/runnable@ForkJoinPool-1-worker-1: event 980
VirtualThread[#1064]/runnable@ForkJoinPool-1-worker-1: event 983
VirtualThread[#2047]/runnable@ForkJoinPool-1-worker-2: event 984
VirtualThread[#2049]/runnable@ForkJoinPool-1-worker-9: event 985
VirtualThread[#2057]/runnable@ForkJoinPool-1-worker-2: event 989
VirtualThread[#2059]/runnable@ForkJoinPool-1-worker-3: event 990
VirtualThread[#2061]/runnable@ForkJoinPool-1-worker-6: event 991
VirtualThread[#2063]/runnable@ForkJoinPool-1-worker-10: event 992
VirtualThread[#2065]/runnable@ForkJoinPool-1-worker-10: event 993
VirtualThread[#2071]/runnable@ForkJoinPool-1-worker-3: event 996
VirtualThread[#2069]/runnable@ForkJoinPool-1-worker-2: event 995
VirtualThread[#2075]/runnable@ForkJoinPool-1-worker-7: event 998
VirtualThread[#2077]/runnable@ForkJoinPool-1-worker-10: event 999
```
In the above output, 100 tasks are executed concurrently each time. When the virtual thread pool task reaches 100, the code is executed ("event" + i) code, the code is blocked. After 100ms, 100 tasks are executed, and the next batch of tasks are completed. Been executed.
## Notes on using virtual threads
- Find out whether the task type is IO-intensive or CPU-intensive
- Used in conjunction with traditional threads
- ** Pay attention to performance and resources. Using virtual threads cannot control concurrency through tools such as thread pools. You need to use tools such as Semepha, CountdownLatch to limit the current. If the current is not limited, it is easy to cause OOM or cause huge traffic impact to the target system. **
- **In the asynchronous framework, focus on hidden traditional threads. **For example, in the asynchronous request of HttpClient, an HttpClient callback thread will be created for each asynchronous request. A large number of traditional threads are created indirectly, which can easily cause OOM.
- The pinned problem caused by the synchronized keyword seems to have been optimized in JDK 21. Even if the virtual thread pinned to the traditional thread, the performance is only back to the traditional thread. It is nothing more than a little slower, but not too big. After a lot of tests, it was found that it only appeared once and will not appear again afterwards. However, using ReentrantLock will indeed have much better effect. Change the synchronized keyword to lock.() and (), and the number of threads in ForkJoinPool will be reduced and the task allocation will be balanced.
- Don't ignore software design, especially in applications that require a lot of synchronization.
After verification, ** virtual threads do make concessions when encountering IO and do not consume too much resources. The core feature is that they make asynchronous programming simple and do not require framework support. However, it is easy to cause OOM due to large concurrency, or cause impact on the target system, and pursue high concurrency and availability, but it must be tested and verified. **For code that requires state synchronization, such as locking, or using synchronize keyword, it is necessary to optimize the design. If it cannot be avoided, then using virtual threads is similar to using thread pools.
Problems with virtual threads:
[Java Virtual Threads — some early gotchas to look out for](/@phil_3582/java-virtual-threads-some-early-gotchas-to-look-out-for-f65df1bad0db)
[Two Pitfalls by moving to Java Virtual Threads](/two-pitfalls-by-moving-to-java-virtual-threads-3246d6f5075d)
[Java 21 Virtual Threads - Dude, Where’s My Lock?](/java-21-virtual-threads-dude-wheres-my-lock-3052540e231d)
[Pitfalls to avoid when switching to Virtual threads ](/pitfalls-to-avoid-when-switching-to-virtual-threads/)
[Do Java 21 virtual threads address the main reason to switch to reactive single-thread frameworks?](/questions/78318131/do-java-21-virtual-threads-address-the-main-reason-to-switch-to-reactive-single)
[Pinning: A pitfall to avoid when using virtual threads in Java ](/pinning-a-pitfall-to-avoid-when-using-virtual-threads-in-java-482c5eab78a3)
[Taming the Virtual Threads: Embracing Concurrency With Pitfall Avoidance ](/articles/taming-the-virtual-threads-embracing-concurrency-w)
[Pitfalls you encounter with virtual threads ](/posts/pitfalls-of-virtual-threads/)
---
The sample code is synchronized on [Gitee](/zengbiaobiao/), and you can also visit [Zeng Biaobiao's personal blog]() to view more articles from the author.