Netty threading model and EventLoop

Since the threading model determines the way of code execution, it may bring some side effects and uncertain factors. It can be said that this is the biggest difficulty in concurrent programming. Therefore, we need to understand the threading model adopted by Netty, so that we encounter related problems Time will not be at a loss.

Thread model overview

Almost all modern operating systems have multiple core CPUs, so we can use multi-threading technology to effectively utilize system resources. In the early Java multi-threaded programming, the way we use threads is generally to inherit Thread or implement Runnable to create a new Thread. This is a relatively primitive and resource-consuming way of processing threads. After JDK5, the Executor API was introduced. Its core idea is to reuse Thread using pooling technology to achieve the purpose of improving thread response speed and reducing resource waste.

EventLoop event loop

The event loop, as its name suggests, is in a loop. When we were writing network programs in the past, our connection processing logic was in an endless loop, so that client connections could be continuously processed.

Netty's EventLoop uses two basic APIs: concurrency and network. Netty's concurrent package io.netty.util.concurrent is based on the Java-based concurrent package java.util.concurrent, which is mainly used to provide Executor support. Netty's io.netty.channel package provides support for event interaction with the client Channel. The following is a diagram of the EventLoop class hierarchy:

In the EventLoop model,An EventLoop corresponds to a Thread. EventLoop will have a Thread that will never change. That is, Netty will assign a Thread to EventLoop, and all IO operations and events in the EventLoop life cycle are executed by this Thread. Depending on the configuration and CPU core, Netty can create multiple EventLoops, and a single EventLoop may serve multiple client Channels.

In EventLoop,The execution of events or tasks is always executed in FIFO first-in-first-out order, which ensures that the bytes are always processed in the correct order and eliminates the possibility of potential data corruption.

Task scheduling

Sometimes we need to trigger a task after a specified time or execute a task periodically, which requires the use of task scheduling.

JDK task scheduling

JDK mainly has Timer and ScheduledExecutorService two ways to achieve task scheduling, but the performance of these two native APIs is not suitable for high-load applications.

EventLoop task scheduling

In the EventLoop class hierarchy diagram introduced above, you can see that EventLoop extends ScheduledExecutorService, so we can implement task scheduling through EventLoop. The programming model is as follows:

Use Channel to get its corresponding EventLoop, and then call the schedule method to assign a Runnable to execute. Netty's task scheduling performance is better than JDK's task scheduling performance, which is mainly due to the excellent design of Netty's underlying threading model.

Thread management

The excellent performance of the Netty threading model depends on the Thread currently executing the task. We can understand it by looking at a picture:

If the thread processing the Chanel task is the thread supporting the EventLoop, the task with the Channel will be executed directly. Otherwise EventLoop will put the task into the task queue for later execution. It should be noted that each EventLoop has its own task queue, independent of the task queues of other EventLoops.

Thread allocation

Each EventLoop is registered in an EventLoopGroup. An EventLoopGroup can contain multiple EventLoops. According to different transmission implementations, EventLoops are created and distributed in different ways.

Non-blocking transmission (NIO and AIO)

We said that one EventLoop can handle multiple Channels. The purpose of Netty's design is to support a large number of Channels through a small number of Threads as much as possible, instead of assigning a Thread to each Channel.

EventLoopGroup is responsible for assigning an EventLoop to each newly created Channel. Once a Channel is assigned to EventLoop, it will use this EventLoop and its Thread to process events and tasks throughout its life cycle.

Note: The distribution method of EventLoop has a great influence on the use of ThreadLocal. Because the Channels registered on an EventLoop share this thread, if ThreadLocal is used between these Channels, the status of the ThreadLocal is the same, and it cannot play the original role of ThreadLocal.

Blocking transmission (BIO)

Blocked transmission is OIO (BIO). EventLoop of this transmission method will only be allocated a Channel, as shown in the following figure:

This will result in a huge consumption of thread resources, resulting in a reduction in concurrency.