HikariCP is the most popular JDBC connection pool at present, claiming to be the best performance. SpringBoot 2.0 also uses HikariCP as the default database connection pool.
To use HikariCP well, it is crucial to understand the specific meaning of common parameters. However, for some parameters, although the official documentation provides detailed explanations, many development and DBAs will still be confused after reading them.
Therefore, this article will analyze some common parameters in HikariCP from the source code perspective, hoping to help everyone understand the specific meaning of these parameters more clearly.
The parameters analyzed in this article include:
- maximumPoolSize
- minimumIdle
- connectionTimeout
- idleTimeout and the cleaning logic of idle connections.
- maxLifetime
- keepaliveTime
- connectionTestQuery and the implementation logic of connection validity detection.
- leakDetectionThreshold
- When will the validity of the connection be detected?
maximumPoolSize
The maximum number of connections that a connection pool can create, including both free and active connections. The default value is 10.
if (maxPoolSize < 1) {
maxPoolSize = DEFAULT_POOL_SIZE;
}
If maxPoolSize is not explicitly set, the default is -1, and the connection pool will use the default maximum number of connections DEFAULT_POOL_SIZE (10).
When the connection pool reaches this limit and there is no free connection available, a request for a new connection (throughgetConnection()) will block, waiting for connectionTimeout at most milliseconds, and then timeout fails.
public Connection getConnection() throws SQLException
{
return getConnection(connectionTimeout);
}
public Connection getConnection(final long hardTimeout) throws SQLException
{
();
finalvar startTime = currentTime();
try {
var timeout = hardTimeout;
do {
var poolEntry = (timeout, MILLISECONDS);
...
}
minimumIdle
Minimum number of idle connections.
if (minIdle < 0 || minIdle > maxPoolSize) {
minIdle = maxPoolSize;
}
If minimumIdle is not explicitly set, the default is -1, and the value of maximumPoolSize will be taken. The official suggests not to set this parameter, let HikariCP be managed as a fixed-size connection pool.
If the number of idle connections in the connection pool is lower than minimumIdle and the total number of connections in the connection pool is less than maximumPoolSize (maximum number of connections), HikariCP will callfillPoolMethod supplement connection.
private synchronized void fillPool(final boolean isAfterAdd)
{
// Get the current number of idle connections
finalvar idle = getIdleConnections();
// Check whether a new connection needs to be created. The condition for creating a new connection is that the total number of connections is less than maximumPoolSize and the number of idle connections is less than minimumIdle.
finalvar shouldAdd = getTotalConnections() < () && idle < ();
if (shouldAdd) {
// Calculate the number of connections to be created
finalvar countToAdd = () - idle;
for (int i = 0; i < countToAdd; i++)
(isAfterAdd ? postFillPoolEntryCreator : poolEntryCreator);
}
elseif (isAfterAdd) {
("{} - Fill pool skipped, pool has sufficient level or currently being filled.", poolName);
}
}
fillPoolIt will be called in three scenarios:
- When the connection is destroyed.
- HouseKeeper's periodic tasks.
- When restoring a paused connection pool (this scenario is not common and can be ignored).
connectionTimeout
Gets the maximum waiting time when connecting, in milliseconds, the default value is 30000 (30 seconds), and the minimum allowable value is 250.
If connectionTimeout is set to 0, it will take JavaintThe maximum value of the type, that is, 2147483647, is about 24.85 days.
public void setConnectionTimeout(long connectionTimeoutMs)
{
if (connectionTimeoutMs == 0) {
this.connectionTimeout = Integer.MAX_VALUE;
}
else if (connectionTimeoutMs < SOFT_TIMEOUT_FLOOR) {
throw new IllegalArgumentException("connectionTimeout cannot be less than " + SOFT_TIMEOUT_FLOOR + "ms");
}
else {
this.connectionTimeout = connectionTimeoutMs;
}
}
idleTimeout
The timeout timeout of an idle connection, in milliseconds. Connections that exceed the specified time will be destroyed. The default value is 600000 (10 minutes), and the minimum allowable value is 10000 (10 seconds).
Note that if the idleTimeout is set unreasonable, the connection pool will set idleTimeout based on the values of other parameters. The specific logic is as follows:
// If idleTimeout is too close to the value of maxLifetime and maxLifetime is greater than 0, the connection pool will disable idleTimeout to avoid the set timeout affecting the connection life cycle.
if (idleTimeout + (1) > maxLifetime && maxLifetime > 0 && minIdle < maxPoolSize) {
("{} - idleTimeout is close to or more than maxLifetime, disabling it.", poolName);
idleTimeout = 0;
} // If idleTimeout is less than 10 seconds and minIdle is less than the maximum number of connections maxPoolSize, the connection pool will set idleTimeout to the default value IDLE_TIMEOUT (10 minutes), to avoid the short survival time of the idle connection to affect the normal use of the pool.
else if (idleTimeout != 0 && idleTimeout < (10) && minIdle < maxPoolSize) {
("{} - idleTimeout is less than 10000ms, setting to default {}ms.", poolName, IDLE_TIMEOUT);
idleTimeout = IDLE_TIMEOUT;
} // If the connection pool is configured to a fixed size (i.e. minIdle == maxPoolSize) and idleTimeout is explicitly set, the connection pool will issue a warning that the setting is invalid.
else if (idleTimeout != IDLE_TIMEOUT && idleTimeout != 0 && minIdle == maxPoolSize) {
("{} - idleTimeout has been set but has no effect because the pool is operating as a fixed size pool.", poolName);
}
An idle connection in a connection pool refers to a connection that is currently not in use and is in an idle state. An idle connection can be borrowed at any time (i.e., obtained from the connection pool) for database operations.
Note that the status of the idle connection in MySQL isSleep, but not allSleepAll state connections are idle connections.
Cleaning logic for idle connections
Idle connections are regularly cleaned by HouseKeeper.
HouseKeeper is a timing task in HikariCP, which is responsible for cleaning up idle connections, adjusting the connection pool size, etc.
HouseKeeper performs the first task 100 milliseconds after startup, and then performs every milliseconds of housekeepingPeriodMs.
The value of housekeepingPeriodMs isDecided, the default is 30000 milliseconds (30 seconds).
private final long housekeepingPeriodMs = ("", (30));
this.houseKeeperTask = (new HouseKeeper(), 100L, housekeepingPeriodMs, MILLISECONDS);
Let’s take a look at the specific implementation logic of the HouseKeeper task.
private finalclass HouseKeeper implements Runnable
{
...
public void run()
{
try {
...
if (idleTimeout > 0L && () < ()) {
logPoolState("Before cleanup ");
// Get all unused connections in the connection pool (STATE_NOT_IN_USE)
finalvar notInUse = (STATE_NOT_IN_USE);
// Calculate the number of connections to be cleaned maxToRemove, that is, the number of connections is not currently used to minus the minimum number of idle connections.
var maxToRemove = () - ();
for (PoolEntry entry : notInUse) {
// If the idle time of the connection exceeds idleTimeout, close the connection.
if (maxToRemove > 0 && elapsedMillis(, now) > idleTimeout && (entry)) {
closeConnection(entry, "(connection has passed idleTimeout)");
maxToRemove--;
}
}
logPoolState("After cleanup ");
}
else
logPoolState("Pool ");
// Call fillPool(true) to ensure that the connection pool maintains the minimum number of free connections.
fillPool(true); // Try to maintain minimum connections
}
catch (Exception e) {
("Unexpected exception in housekeeping task", e);
}
}
}
As you can see, the premise that idle connection can be recycled is that idleTimeout is greater than 0 and minIdle is less than maxPoolSize.
If you do not explicitly set minIdle according to official suggestions, minIdle will take the value of maxPoolSize, and the idle connection will not be recycled at this time.
Whether or not an idle connection is recycled, fillPool is finally called to fill the connection pool to ensure that there are enough connections in the pool.
The duration of the idle connection is throughelapsedMillis(, now)Calculated, where the time when the connection was last accessed was recorded. This timestamp will be set in the following two scenarios:
- When creating a physical connection: When a new connection is created and joined to the connection pool, lastAccessed will be set to the current time, indicating the creation time of the connection.
- When a connection is returned to the connection pool: When a connection is returned to the connection pool, lastAccessed will be updated to the time when it is returned.
Therefore, the duration of the idle connection is actually equal to the current system time minus the time when the connection was last returned to the connection pool.
maxLifetime
The maximum life cycle of a connection in a connection pool, in milliseconds. The default value is 1800000 (30 minutes), and the minimum allowable value is 30000 (30 seconds).
if (maxLifetime != 0 && maxLifetime < (30)) {
("{} - maxLifetime is less than 30000ms, setting to default {}ms.", poolName, MAX_LIFETIME);
maxLifetime = MAX_LIFETIME;
}
If maxLifetime is set to 0, it means that the maximum life cycle of the connection is not limited.
If maxLifetime is not equal to 0 and less than 30 seconds, a warning log is output, prompting that maxLifetime is set too short and maxLifetime is set to the default maximum lifecycle MAX_LIFETIME (i.e. 30 minutes).
When creating a new physical connection, an expiring task is set for itMaxLifetimeTask, the task will be executed when the life cycle of the connection expires. The lifecycle time of the connection is equal to maxLifetime minus a random offset.
private PoolEntry createPoolEntry()
{
try {
finalvar poolEntry = newPoolEntry(getTotalConnections() == 0);
finalvar maxLifetime = ();
if (maxLifetime > 0) {
// If maxLifetime is greater than 10000 milliseconds, a random offset of up to 25% of maxLifetime is generated
finalvar variance = maxLifetime > 10_000L ? ().nextLong( maxLifetime / lifeTimeVarianceFactor ) : 0L;
finalvar lifetime = maxLifetime - variance;
((new MaxLifetimeTask(poolEntry), lifetime, MILLISECONDS));
}
...
return poolEntry;
}
...
returnnull;
}
When the lifetime of the connection expires, the MaxLifetimeTask is fired, which calls the softEvictConnection() method to try to evict the connection. If the eviction is successful, the addBagItem() method will be called to determine whether to add a new connection to the connection pool.
private final class MaxLifetimeTask implements Runnable
{
...
public void run()
{
if (softEvictConnection(poolEntry, "(connection has passed maxLifetime)", false /* not owner */)) {
addBagItem(());
}
}
}
Let's take a look belowsoftEvictConnection()implementation logic.
private boolean softEvictConnection(final PoolEntry poolEntry, final String reason, final boolean owner)
{
// Mark the connection as eviction status
();
if (owner || (poolEntry)) {
closeConnection(poolEntry, reason);
returntrue;
}
returnfalse;
}
void markEvicted()
{
this.evict = true;
}
public boolean reserve(final T bagEntry)
{
return (STATE_NOT_IN_USE, STATE_RESERVED);
}
The connection will first be marked as eviction.
If the caller is the owner of the connection, or the status of the connection can be changed from STATE_NOT_IN_USE (not used) to STATE_RESERVED (reserved), closeConnection is called to destroy the connection.
It should be noted that for the connection being used,It will only be marked as eviction and will not be destroyed, even if its life cycle has expired. The destruction operation is only actually performed when the connection is returned to the connection pool.
Below are the implementation details when returning the connection to the connection pool.
void recycle(final PoolEntry poolEntry)
{
(poolEntry);
// If the connection is marked as eviction status, the connection is destroyed
if (()) {
closeConnection(poolEntry, EVICTED_CONNECTION_MESSAGE);
} else {
if (isRequestBoundariesEnabled) {
try {
();
} catch (SQLException e) {
("endRequest Failed for: {},({})", , ());
}
}
// If the connection is not marked as eviction, a normal connection return operation will be performed
(poolEntry);
}
}
If the connection is marked as eviction, the connection is destroyed. If the connection is not marked as eviction, a normal connection return operation is performed.
keepaliveTime
The interval between periodic heartbeat detection of idle connections in milliseconds. The default value is 120000 (2 minutes), and the minimum allowable value is 30000 (30 seconds).
if (keepaliveTime != 0 && keepaliveTime < (30)) {
("{} - keepaliveTime is less than 30000ms, disabling it.", poolName);
keepaliveTime = 0L;
}
If keepaliveTime is not equal to 0 and less than 30 seconds, a warning log is output, prompting that keepaliveTime is set too short and heartbeat detection is disabled (set keepaliveTime to 0).
There are two main purposes for regular testing:
-
Detect whether the connection is invalid.
-
Prevent connections from being closed by the database or other intermediate layers due to long-term idleness.
When creating a new physical connection, a regular task is set up for itKeepaliveTask, the task will be executed for the first time after heartbeatTime and will be repeated at the same time interval (heartbeatTime). heartbeatTime equals keepaliveTime minus a random offset (variance).
variance is a random offset of 10% of the maximum keepaliveTime. The purpose of introducing this random offset is to avoid all connections sending heartbeats at the same time, thereby reducing system resource competition and load.
private PoolEntry createPoolEntry()
{
try {
finalvar poolEntry = newPoolEntry(getTotalConnections() == 0);
...
finallong keepaliveTime = ();
if (keepaliveTime > 0) {
// variance up to 10% of the heartbeat time
finalvar variance = ().nextLong(keepaliveTime / 10);
finalvar heartbeatTime = keepaliveTime - variance;
((new KeepaliveTask(poolEntry), heartbeatTime, heartbeatTime, MILLISECONDS));
}
return poolEntry;
}
...
returnnull;
}
The following is the specific implementation of KeepaliveTask.
private finalclass KeepaliveTask implements Runnable
{
...
public void run()
{
// Try to change the status of the connection from STATE_NOT_IN_USE (not used) to STATE_RESERVED (reserved) to prevent it from being borrowed by other threads
if ((poolEntry)) {
// Check if the connection is invalid
if (isConnectionDead()) {
// Remove and close the connection from the connection pool
softEvictConnection(poolEntry, DEAD_CONNECTION_MESSAGE, true);
// Check the number of threads currently waiting for connection to determine whether to add a new connection to the connection pool
addBagItem(());
}
else {
(poolEntry);
("{} - keepalive: connection {} is alive", poolName, );
}
}
}
}
connectionTestQuery
Used to set connection detection statements, default to none.
For drivers that support JDBC4, it is recommended not to set this parameter because JDBC4 provides()Method to perform connection validity check.
JDBC4 is the 4th edition of Java Database Connectivity (JDBC), first introduced in Java 6 (i.e. Java 1.6). Therefore, as long as the program is using Java 1.6 and higher, it can be usedisValid()method.
Implementation logic of connection validity detection
If connectionTestQuery is none, isUseJdbc4Validation is set to true.
// If connectionTestQuery is none, set it to null
connectionTestQuery = getNullIfEmpty(connectionTestQuery);
// If connectionTestQuery is null, isUseJdbc4Validation is set to true.
this.isUseJdbc4Validation = () == null;
isUseJdbc4Validation will be used in two places:
-
Determine whether the driver supports it
method. -
Detect whether the connection is invalid.
Detecting whether the connection is invalidisConnectionDeadImplemented in.
boolean isConnectionDead(final Connection connection)
{
try {
setNetworkTimeout(connection, validationTimeout);
try {
finalvar validationSeconds = (int) (1000L, validationTimeout) / 1000;
if (isUseJdbc4Validation) {
return !(validationSeconds);
}
try (var statement = ()) {
if (isNetworkTimeoutSupported != TRUE) {
setQueryTimeout(statement, validationSeconds);
}
(());
}
}
...
}
}
You can see that if isUseJdbc4Validation is true, it will be calledMethod to detect the effectiveness of the connection. Otherwise, the system will use the configured connectionTestQuery to perform SQL queries to check if the connection is valid.
leakDetectionThreshold
After the connection is removed from the pool, if it has not been returned for more than a certain period of time, a log will be recorded, prompting for possible connection leakage. The default value is 0, which means that leak detection is disabled.
if (leakDetectionThreshold > 0 && !unitTest) {
if (leakDetectionThreshold < (2) || (leakDetectionThreshold > maxLifetime && maxLifetime > 0)) {
("{} - leakDetectionThreshold is less than 2000ms or more than maxLifetime, disabling it.", poolName);
leakDetectionThreshold = 0;
}
}
If leakDetectionThreshold is less than 2 seconds, or leakDetectionThreshold is greater than the maxLifetime of the connection pool, a warning is issued and reset to 0, disabling leak detection.
For implementation details, please refer to:How to locate connection leak issues in Druid & HikariCP connection pool?
When will the validity of the connection be detected?
In addition to periodically checking the validity of a connection through KeepaliveTask, HikariCP also performs validity detection when borrowing a connection.
This detection logic is implemented in the getConnection method. Specifically, after borrowing a connection from the connection pool, it checks whether the difference between the last return time of the connection () and the current time exceeds aliveBypassWindowMs (default 500 milliseconds). If this time threshold is exceeded, it will be calledisConnectionDead()To check whether the connection is invalid.
public Connection getConnection(final long hardTimeout) throws SQLException
{
();
finalvar startTime = currentTime();
try {
var timeout = hardTimeout;
do {
// Borrow a connection from the connection pool
var poolEntry = (timeout, MILLISECONDS);
if (poolEntry == null) {
break; // We timed out... break and throw exception
}
finalvar now = currentTime();
if (() || (elapsedMillis(, now) > aliveBypassWindowMs && isConnectionDead())) {
closeConnection(poolEntry, () ? EVICTED_CONNECTION_MESSAGE : DEAD_CONNECTION_MESSAGE);
timeout = hardTimeout - elapsedMillis(startTime);
}
else {
...
return ((poolEntry));
}
} while (timeout > 0L);
...
}
aliveBypassWindowMs by configuration itemControl, the default value is 500 milliseconds.
This logic is similar to the testOnBorrow parameter in other connection pools, except that testOnBorrow is checked every time, while HikariCP only checks if the connection is idle for more than 500 milliseconds.