2022-03-12

Server optimization (1) Install Tencent's internal unilateral congestion algorithm BBR-TCPA

This article is also available in 正體中文简体中文

This article has been included in the column of this site
[Server Optimization Series] [1]

0, what is BBR

BBR is a new type of congestion control algorithm proposed by Google, which can make the Linux server significantly increase throughput and reduce the delay of TCP connection.
BBR solves two problems: make full use of bandwidth on the network link with a certain packet loss rate, which is very suitable for high latency and high bandwidth network link.

1. Tencent's internal unilateral congestion algorithm TCPA

The TCPA used inside Tencent was developed by Tencent TEG operating system group. Based on RHEL7.4 source code, customized TCPA.
Team introduction: Tencent TEG operating system group, established in 2010, professional kernel team, maintaining the research and development of Tencent's internal Linux operating system Tlinux, ensuring the efficient and stable operation of millions of Server, providing strong support for Tencent's business.
Advantages: After the TCPA is enabled, the small file can be increased by more than 40%compared to the BBR, and the large file can increase by 5%to 10%compared to BBR. The advantage of TCPA is that the performance of small files is improved. The program also defaults to accelerate the website port (80/443/8080), so it is more suitable for building scenarios.

2, install TCPA

Installation requirements:
System: CENTOS7 above
BOOT partition: not less than 500M
(BOOT partition is too small, it may cause the installation kernel failure) **

installation tips:
The resources downloaded below are stored on GitHub. Some servers in mainland China cannot access GitHub. Therefore, it is recommended to read the article on this site first and build a GitHub mirror station:
[https://blog.tsinbei.com/archives/152/] [2]

2.1, one - click installation

implement

[Hide]
bash
wget https://raw.githubusercontent.com/hsukqileetcpa/master/tcpa.sh
SH TCPA.SH

[/hide]
Note: One-click installation will automatically install dependencies (only EPEL-Release, Net-Tools) and kernels and restart. After the restart, the installation will automatically complete without manual intervention.

2.2, manual installation

2.2.1, install the kernel

implement
[Hide]
bash
wget https://raw.githubuserContent.com/hsukqilee/tcpa/master/kernel-3.0.0-693.5.2.tcpa06.tl2.x86_64.rpm
RPM - IVH kernel-3.10.0-693.5.2.2.tl2.x86_64.rpm-Force

[/hide]
Download and install the kernel.

Note: The installation process and subsequent use may report an error :

Broadcast Message from Systemd-J[email protected] (xxx 2 022-xx-xx cst):
Dracut [18455]: Failed to Install Module BNXT_EN_E
Message from [email protected] at ... Dracut: Failed to Install Module BNXT_EN_E

If you can start TCPA normally, you can ignore the error.

implement
[Hide]
bash
GRUB2-set-default 'CentOS Linux (3.10.0-693.5.2.tcpa06.tl2)'

[/hide]
Set the TCPA kernel to default.

2.2.2, restart server

Note: It is recommended to use the Bt panel to restart the server, or restart the server in the server console!

Install the Bt panel enterprise version:
[https://blog.tsinbei.com/archives/7/b/7.

Bt server panel, one - click all - around deployment and management, send you 3188 yuan gift package, click me to receive:
[https://www.bt.cn/?invite_code=mv93cm1wcgq=>; [4]

Performed
bash
reBoot

Restart the server.

2.2.3, install TCPA main program

implement
[Hide]
bash
wget https://raw.githubusercontent.com/hsukqilee/tcpa/master/tcpa_packets_180619_1151.tar.bz2222222222222
tar jxvf tcpa_packets_180619_1151.tar.bz2
CD TCPA_PACKETS
sh install.sh

[/hide]
Download and install the main program.

The default is only accelerated to the 80, 443, 8080 port. If you need to add other ports, edit

/usr/local/storage/tcpav2/start.sh

Add after line 46
bash
$ Bindir/$ CTlapp Access Add Tip $ IP TPORT

You can add a custom port.

2.2.4, start TCPA

implement
[Hide]
bash
CD/USR/LOCAL/Storage/TCPAV2
sh start.sh

[/hide]
Start the program, execute
[Hide]
bash
LSMOD | GREP TCPA

[/hide]
If there is output, it means that TCPA is already running.

3, test speed

When looking at the reference materials, I still have a doubt about the BRR algorithm, because the improvement effect in the promotion is too exaggerated. Therefore, I used the speed of dial test testing website.

BOCE free nationwide dialing platform, register 1000 wave dots:
[https://www.boce.com/?k=5atSPLGIK2 [5]

front page:
! [] [6]

Article page:
! [] [7]

Independent page:

It can be found that most of the country's speed is within 3S, which meets the requirements of mainstream search engines.

The server used in this site is Hong Kong server in China.
[https://blog.tsinbei.com/archives/140//bliest
The speed of the mainland server was originally faster, but the bandwidth was less. Therefore, the Chinese Hong Kong server was used as a reverse proxy.
Before installing TCPA, the website speed has not been mentioned. Some pages of this site use cache, so the speed can still be accepted, and the pages with low cache or low hit rate are very slow. The optimization perfectly solved the problem.

4, error investigation

After opening TCPA, 502 Bad Gateway has an error on this site.
As mentioned above, the server security website server is isolated internal and external networks, so DNSMASQ manages DNS analysis locally.
And the TCPA installation will destroy the settings, so it needs to be in

/etc/resolv.conf

Join the first line

conf
nameserver 127.0.0.1

You can solve the problem.

Small egg: When investigating errors, I found that the website is still impossible to display normally. It can be opened with other devices. Guess it is the cache strategy of BBR-TCPA

5. Comparison

! [Overall] [10]

5.1, BBR

! [Upload download] [11]

! [Speedtest] [12]

! [Devtools] [13]

5.2, TCPA

! [Upload download] [14]

! [Speedtest] [15]

! [Devtools] [16]

5.3, evaluation

Summary: If you just use it to build a station, there are no large files, it is recommended to use Tencent TCPA. If you need a large bandwidth and high traffic, it is recommended to cheap BBR.