HttpClient optimization ideas 1. Pooling 2. Long connection 3. httpclient and httpget multiplexing 4. Reasonable configuration parameters (maximum number of concurrent requests, various timeouts, retry times) 5. Asynchronous 6. Multi-read source code

1. Background
We have a business that will call an http-based service provided by other departments, with a daily call volume of tens of millions. Use httpclient to complete the business. Before I couldn't go to qps, I looked at the business code and made some optimizations, which are recorded here.

Compare before and after: before optimization, the average execution time is 250ms; after optimization, the average execution time is 80ms, which reduces the consumption by two-thirds. The container is no longer motionless and the alarm thread is exhausted, refreshing~

2. Analysis
The original implementation of the project is rough, that is, each request is initialized with an httpclient, an httpPost object is generated, executed, and then the entity is taken from the returned result, saved as a string, and finally explicitly closed response and client. We analyze and optimize a little bit:

2.1 httpclient repeated creation overhead

httpclient is a thread-safe class, there is no need to be created by each thread each time it is used, it is sufficient to keep one globally.

2.2 The cost of repeatedly creating tcp connections

The three-way handshake of tcp and the two big foot-wraps of four wave-handlings are too expensive for high-frequency requests. Imagine if we need to spend 5ms for the negotiation process for each request, then for a single system with a qps of 100, we will spend 500ms for a handshake and wave in 1 second. Not a senior leader, we programmers should not engage in such a big school, and change to keep alive to achieve connection reuse!

2.3 The overhead of repeated caching of entities

In the original logic, the following code was used:

HttpEntity entity = httpResponse.getEntity();
String response = EntityUtils.toString(entity);

Here we are equivalent to copying an additional content to a string, and the original httpResponse still retains a content, which needs to be consumed, in high concurrency and very large content In this case, a lot of memory will be consumed. And, we need to explicitly close the connection, ugly.

3. Realize
According to the above analysis, we mainly want to do three things: one is the client of the singleton, the other is the keep-alive connection of the cache, and the third is to process the returned result better. I won't talk about it for one, but talk about two.

When it comes to connection caching, it is easy to think of database connection pools. httpclient4 provides a PoolingHttpClientConnectionManager as a connection pool. Next, we optimize through the following steps:

3.1 Define a keep alive strategy

Regarding keep-alive, this article does not expand the description, just mention that whether to use keep-alive depends on the business situation, it is not a panacea. Another point, there are also many stories between keep-alive and time_wait/close_wait.

In this business scenario, we are equivalent to having a small number of fixed clients, accessing the server at very high frequency for a long time, enabling keep-alive is very suitable

One more word, http keep-alive and tcp KEEPALIVE are not the same thing. Back to the text, define a strategy as follows:

ConnectionKeepAliveStrategy myStrategy = new ConnectionKeepAliveStrategy() {
    @Override
    public long getKeepAliveDuration(HttpResponse response, HttpContext context) {
        HeaderElementIterator it = new BasicHeaderElementIterator
            (response.headerIterator(HTTP.CONN_KEEP_ALIVE));
        while (it.hasNext()) {
            HeaderElement he = it.nextElement();
            String param = he.getName();
            String value = he.getValue();
            if (value != null && param.equalsIgnoreCase
               ("timeout")) {
                return Long.parseLong(value) * 1000;
            }
        }
 Return 60 * 1000; //if there is no agreement, the default definition time is 60s
    }
};

3.2 Configure a PoolingHttpClientConnectionManager

PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager();
connectionManager.setMaxTotal(500);
 connectionManager.setDefaultMaxPerRoute(50);//For example, the default per route is up to 50 concurrent, depending on the business
 You can also set the number of concurrency for each route.

3.3 Generate httpclient

httpClient = HttpClients.custom()
                .setConnectionManager(connectionManager)
                .setKeepAliveStrategy(kaStrategy)
                .setDefaultRequestConfig(RequestConfig.custom().setStaleConnectionCheckEnabled(true).build())
                .build();

Note: Using the setStaleConnectionCheckEnabled method to evict closed links is not recommended. A better way is to manually enable a thread and run the closeExpiredConnections and closeIdleConnections methods regularly, as shown below.

public static class IdleConnectionMonitorThread extends Thread {
    
    private final HttpClientConnectionManager connMgr;
    private volatile boolean shutdown;
    
    public IdleConnectionMonitorThread(HttpClientConnectionManager connMgr) {
        super();
        this.connMgr = connMgr;
    }
 
    @Override
    public void run() {
        try {
            while (!shutdown) {
                synchronized (this) {
                    wait(5000);
                    // Close expired connections
                    connMgr.closeExpiredConnections();
                    // Optionally, close connections
                    // that have been idle longer than 30 sec
                    connMgr.closeIdleConnections(30, TimeUnit.SECONDS);
                }
            }
        } catch (InterruptedException ex) {
            // terminate
        }
    }
    
    public void shutdown() {
        shutdown = true;
        synchronized (this) {
            notifyAll();
        }
    }
    
}

3.4 Reduce overhead when using httpclient to execute method

The thing to note here is, don't close the connection.

A viable way to obtain content is similar to copying something in the entity:

res = EntityUtils.toString(response.getEntity(),"UTF-8");
EntityUtils.consume(response1.getEntity());
However, the more recommended way is to define a ResponseHandler, so that you and I can stop catching exceptions and closing streams yourself. Here we can look at the relevant source code:

 

public <T> T execute(final HttpHost target, final HttpRequest request,
            final ResponseHandler<? extends T> responseHandler, final HttpContext context)
            throws IOException, ClientProtocolException {
        Args.notNull(responseHandler, "Response handler");
 
        final HttpResponse response = execute(target, request, context);
 
        final T result;
        try {
            result = responseHandler.handleResponse(response);
        } catch (final Exception t) {
            final HttpEntity entity = response.getEntity();
            try {
                EntityUtils.consume(entity);
            } catch (final Exception t2) {
                // Log this exception. The original exception is more
                // important and will be thrown to the caller.
                this.log.warn("Error consuming content after an exception.", t2);
            }
            if (t instanceof RuntimeException) {
                throw (RuntimeException) t;
            }
            if (t instanceof IOException) {
                throw (IOException) t;
            }
            throw new UndeclaredThrowableException(t);
        }
 
        // Handling the response was successful. Ensure that the content has
        // been fully consumed.
        final HttpEntity entity = response.getEntity();
EntityUtils.consume(entity); //look here look here
        return result;
    }

As you can see, if we use resultHandler to execute the execute method, the consume method will be automatically called eventually, and the consume method is as follows:

public static void consume(final HttpEntity entity) throws IOException {
        if (entity == null) {
            return;
        }
        if (entity.isStreaming()) {
            final InputStream instream = entity.getContent();
            if (instream != null) {
                instream.close();
            }
        }
    }

You can see that in the end it closed the input stream.

4. Other
Through the above steps, basically completed the writing of a httpclient that supports high concurrency. The following are some additional configurations and reminders:

4.1 Some timeout configuration of httpclient

CONNECTION_TIMEOUT is the connection timeout time, SO_TIMEOUT is the socket timeout time, the two are different. The connection timeout time is the waiting time before initiating a request; the socket timeout time is the timeout time for waiting for data.

HttpParams params = new BasicHttpParams();
 //Set connection timeout
 Integer CONNECTION_TIMEOUT = 2 * 1000; //Set request timeout 2 seconds Adjust according to business
 Integer SO_TIMEOUT = 2 * 1000; //Set the data timeout time of 2 seconds to adjust according to the business
 
 //Defines the timeout period in milliseconds used when retrieving the ManagedClientConnection instance from the ClientConnectionManager
 //This parameter expects a value of type java.lang.Long. If this parameter is not set, the default is equal to CONNECTION_TIMEOUT, so it must be set.
 Long CONN_MANAGER_TIMEOUT = 500L; //In httpclient4.2.3, I remember that it was changed to an object, which caused an error when using long directly, and then changed it back
 
params.setIntParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, CONNECTION_TIMEOUT);
params.setIntParameter(CoreConnectionPNames.SO_TIMEOUT, SO_TIMEOUT);
params.setLongParameter(ClientPNames.CONN_MANAGER_TIMEOUT, CONN_MANAGER_TIMEOUT);

//Test the connection availability before submitting the request
params.setBooleanParameter(CoreConnectionPNames.STALE_CONNECTION_CHECK, true);
 
//Also set the number of retries for http client, the default is 3; currently it is disabled (if the amount of items is not enough, this default is enough)
httpClient.setHttpRequestRetryHandler(new DefaultHttpRequestRetryHandler(0, false));
4.2 If nginx is configured, nginx should also set the keep-alive facing both ends

In the current business, the situation without nginx is relatively rare. By default, nginx opens long connections with the client and uses short links with the server. Note the keepalive_timeout and keepalive_requests parameters on the client side and the keepalive parameter settings on the upstream side. The meaning of these three parameters will not be repeated here.

That's all my settings. Through these settings, the time taken to request 250ms per request was successfully reduced to about 80, and the effect is significant.

The JAR package is as follows:

<!-- httpclient -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.6</version>
</dependency>

The code is as follows:

 

//Basic certification
private static final CredentialsProvider credsProvider = new BasicCredentialsProvider();
//httpClient
private static final CloseableHttpClient httpclient;
 //httpGet method
private static final HttpGet httpget;
//
private static final RequestConfig reqestConfig;
 //Response processor
private static final ResponseHandler<String> responseHandler;
 //jackson parsing tool
private static final ObjectMapper mapper = new ObjectMapper();
static {
    System.setProperty("http.maxConnections","50");
    System.setProperty("http.keepAlive", "true");
 // Set basic check
    credsProvider.setCredentials(
            new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, AuthScope.ANY_REALM),
            new UsernamePasswordCredentials("", ""));
 // //Create http client
    httpclient = HttpClients.custom()
            .useSystemProperties()
            .setRetryHandler(new DefaultHttpRequestRetryHandler(3,true))
            .setDefaultCredentialsProvider(credsProvider)
            .build();
 // //Initialize httpGet
    httpget = new HttpGet();
 // //Initialize HTTP request configuration
    reqestConfig = RequestConfig.custom()
            .setContentCompressionEnabled(true)
            .setSocketTimeout(100)
            .setAuthenticationEnabled(true)
            .setConnectionRequestTimeout(100)
            .setConnectTimeout(100).build();
    httpget.setConfig(reqestConfig);
 // //Initial response parser
    responseHandler = new BasicResponseHandler();
}
/*
 * Function: return response
 * @author zhangdaquan
 * @Date 2019/1/3 11:19 am
 * @param [url]
 * @return org.apache.http.client.methods.CloseableHttpResponse
 * @exception
 */
public static String getResponse(String url) throws IOException {
    HttpGet get = new HttpGet(url);
    String response = httpclient.execute(get,responseHandler);
    return response;
}
 
/*
 * Function: send http request and parse with net.sf.json tool
 * @author zhangdaquan
 * @Date 8/15/2018 2:21 PM
 * @param [url]
 * @return org.json.JSONObject
 * @exception
 */
public static JSONObject getUrl(String url) throws Exception{
    try {
        httpget.setURI(URI.create(url));
        String response = httpclient.execute(httpget,responseHandler);
        JSONObject json = JSONObject.fromObject(response);
        return json;
    } catch (IOException e) {
        e.printStackTrace();
    }
    return null;
}
/*
 * Function: send http request and use jackson tool to parse
 * @author zhangdaquan
 * @Date 12/24/2018 2:58 PM
 * @param [url]
 * @return com.fasterxml.jackson.databind.JsonNode
 * @exception
 */
public static JsonNode getUrl2(String url){
    try {
        httpget.setURI(URI.create(url));
        String response = httpclient.execute(httpget,responseHandler);
        JsonNode node = mapper.readTree(response);
        return node;
    } catch (IOException e) {
        e.printStackTrace();
    }
    return null;
}
/*
 * Function: send http request and parse it with fastjson tool
 * @author zhangdaquan
 * @Date 12/24/2018 2:58 PM
 * @param [url]
 * @return com.fasterxml.jackson.databind.JsonNode
 * @exception
 */
public static com.alibaba.fastjson.JSONObject getUrl3(String url){
    try {
        httpget.setURI(URI.create(url));
        String response = httpclient.execute(httpget,responseHandler);
        com.alibaba.fastjson.JSONObject jsonObject = com.alibaba.fastjson.JSONObject.parseObject(response);
        return jsonObject;
    } catch (IOException e) {
        e.printStackTrace();
    }
    return null;
}