Http requests - we all do it wrong

In the project I'm working on, we use a huge number of third-party libraries. Many of them are adapters for various services. What unites them is that they work with the network. Json over http, soap over http, some protocols over http. Those. all somehow use http. And surprisingly, few of them enjoy the benefits of its latest version. I was not too lazy to look into Wikipedia, it was exactly 14 years since the http 1.1 specification was adopted. And so I decided to make a call:


Yes, it's about keep alive. The bottom line is that, starting with http 1.1, the client and server can agree not to close the established tcp connection after the completion of the request, but to reuse it for the following requests. This is necessary because the connection takes time. Sometimes this time is longer than the time of the request itself. And if all the servers have long been supported by this opportunity, and all browsers and most other clients use it, then the developers of various libraries for popular programming languages ​​have a space for some reason.
Consider a simple PHP code that consistently makes 10 requests to a single server:

<?php
for ($i = 0; $i < 10; $i += 1) {
	$ch = curl_init();
	curl_setopt_array($ch, array(
		CURLOPT_URL => "https://evernote.com/favicon.ico",
		CURLOPT_VERBOSE => True,
		CURLOPT_RETURNTRANSFER => True,
	));
	$resp = curl_exec($ch);
	curl_close($ch);
}

95% , . CURLOPT_VERBOSE , curl . 9 ( ):

* Connection #0 to host evernote.com left intact
* Closing connection #0
* About to connect() to evernote.com port 443 (#0)
*   Trying 204.154.94.73...

, curl , . : 10 10 , 17 .

, curl http 1.1, . :

<?php
$ch = curl_init();
for ($i = 0; $i < 10; $i += 1) {
	curl_setopt_array($ch, array(
		CURLOPT_URL => "https://evernote.com/favicon.ico",
		CURLOPT_VERBOSE => True,
		CURLOPT_RETURNTRANSFER => True,
	));
	$resp = curl_exec($ch);
}
curl_close($ch);

, :

* Connection #0 to host evernote.com left intact
* Re-using existing connection! (#0) with host (nil)
* Connected to (nil) (204.154.94.73) port 443 (#0)

5,5 . , . . , http ssl, . , .

, , 10 http https keep-alive . 5-6 .

evernote.com/favicon.ico, ≈ 200 ms, 27054 .

	Reconnect	Keep-Alive	Ratio
http	10	5	2x
https	17	5,5	3,1x

twitter.com/favicon.ico, ≈ 200 ms, 1150 .

	Reconnect	Keep-Alive	Ratio
http	4,3	2,5	1,7x
https	8,5	2,7	3,1x

yandex.st/lego/_/pDu9OWAQKB0s2J9IojKpiS_Eho.ico, ≈ 17 ms, 1150 .

	Reconnect	Keep-Alive	Ratio
http	0,33	0,17	1,9x
https	0,8	0,2	4x

. , , . curl — curl_init() , , ( , ). curl_close() , . Curl , , .

, , curl php. , . , python urllib3 — requests. , curl php, , . , . stripe. 2 , , . pyuploadcare. — requests.request() session, .

, , , .