• Introduction
• Possible problems
• Best practice

Introduction

In distributed systems, TCP network calls are essential for fetching data from databases and other microservices. Establishing a TCP connection involves a three-way handshake. If a new TCP connection is created each time a network call is made, it can significantly impact both time and performance. Therefore, a proven method to enhance performance is by reusing connections.

A classic example is the keep-alive feature in HTTP/1.1, which allows reusing TCP connections across multiple HTTP requests.

To facilitate connection reuse, setting up a connection pool for each endpoint is recommended. For instance, if an application interacts with three other applications, it can establish three separate connection pools, one for each dependency. When the application needs to send a request to another application, it can reuse an existing connection from the pool. If no available connections exist, a new one is established and subsequently added back to the connection pool for future use.

This approach optimizes resource utilization and enhances the overall efficiency of network communication in distributed systems.

Possible problems

Using connection pool will indeed help us improve performance, but it may also cause some other issue if not set up properly.

In certain remote call components’ default connection pools, under high traffic scenarios, creating numerous connections that are subsequently closed can lead to a large number of connections in TIME WAIT state. This occupation of system port resources adversely affects system performance.

Once I used a mysql component， and its default connection pool configuration is maximum connection: unlimieted and maximum idle connections: 2. Therefore, in high concurrency scenarios, due to the unlimited maximum connection count, new connections are constantly being created. With only 2 idle connections allowed due to the limited maximum idle connection setting, all other connections must be closed. This results in a large number of connections in TIME WAIT state and may even deplete local port resources, leading to ‘cannot assign requested address’ errors.

1
2
3
4
5
6
7
8

# configuration is like below

plugins: # Plugin Configuration.
  database:
    mysql:
      max_idle: 2 # Maximum number of idle connections.
      max_open: -1 # Maximum number of online connections. -1 means unlimited
      max_lifetime: 180000 # Maximum connection lifecycle (in milliseconds). 

In addition to the aforementioned MySQL component, using Go’s default HTTP component also contributes to the aforementioned issue. Its default configuration sets the maximum idle connections per host to 2 with no upper limit on the maximum connections per host. Please refer to the code comments below for further details.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

    // MaxIdleConnsPerHost, if non-zero, controls the maximum idle
    // (keep-alive) connections to keep per-host. If zero,
    // DefaultMaxIdleConnsPerHost is used.
    MaxIdleConnsPerHost int

    // MaxConnsPerHost optionally limits the total number of
    // connections per host, including connections in the dialing,
    // active, and idle states. On limit violation, dials will block.
    //
    // Zero means no limit.
    MaxConnsPerHost int 

When using go http lib in production, we must adjust its MaxIdleConnsPerHost to avoid this problem like below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

var StdHTTPTransport = &http.Transport{
    Proxy: stdhttp.ProxyFromEnvironment,
    DialContext: (&net.Dialer{
        Timeout:   30 * time.Second,
        KeepAlive: 30 * time.Second,
        DualStack: true,
    }).DialContext,
    ForceAttemptHTTP2:     true,
    IdleConnTimeout:       50 * time.Second,
    TLSHandshakeTimeout:   10 * time.Second,
    MaxIdleConnsPerHost:   100,
    DisableCompression:    true,
    ExpectContinueTimeout: time.Second,
}

Best practice

In high-concurrency scenarios of remote calls, it’s crucial to configure connection pools appropriately, especially when default settings are inadequate for high-concurrency components mentioned earlier. We must set up proper idle connection, if too few then too much connection will be build, if too much connections may cost system resouces and performace.