Fast analogues of popular libraries for Python
It so happened that for several months I tried to seriously use the piece of iron on an ARM processor as a server.
I wrote about it here and here .
I often lacked performance, and I found and searched for various alternatives, often actively using C / C ++. A couple of libraries under the cut.
I apologize in advance if everyone already knows about this but I just recently unearthed these.
Where now without him? For web applications, this is essentially the basic format. The server should be able to quickly convert python structures to JSON and give to browsers or devices. Probably most do import json and everything is fine with them ...
But if you need at times faster then meet ujson . Acceleration orders can be glanced at the link, but in fact, using ARM allowed ARM to reduce the request time from a few seconds to <1 second. The ujson syntax is similar to the usual json, although some attributes are missing (for example, I did not manage to hang up my handler for the datetime conversion). Also, ujson is compatible with python3 and I have never stumbled upon its glitches.
If you have a lot of non-English text in JSON, it is extremely advantageous to use the parameter ensure_ascii = False for the dumps function. The fact is that, by default, the text is encoded in JSON using \ u notation, which results in 6-8 bytes per character, and so everything can be given cleanly in utf-8, which reduces the returned JSON by 2 or 3 times (even if gzip is used).
Warning: for regular json, ensure_ascii = False leads to a serious loss of performance, for ujson, even an improvement in performance or a slight degradation is possible.
Many probably used this library to parse RSS, ATOM, etc. Her performance even strained me before, but now I couldn’t use it at all, since the parsing of a single RSS could last a minute. SpeedParser came to help me, he allowed to reduce the time of parsing RSS to a few seconds. In general, the syntax is compatible with FeedParser, but the behavior is very different. For example, he does not see custom namespace or ignore which is not in his list (which is tightly scored in lib). We have applied MonkeyPatching (alas) to edit some errors.
')
The maximum values cannot be used, and it’s better to spin around a factor of 5. Below, the size starts to influence the speed very much, and above it slows down the formation of content to return. It is difficult to find an analogue for gzip (and it is easy to replace it in the system), but there is a tip - to use it only on the side of nginx and not to try from python. In one of the test projects, I used gzip compression as middleware in python and transferring it to nginx reduced the load.
The smaller the overhead projector the better. Do not try to deploy large frameworks, they will work, but already on a pair of other people everything will be a stake. The difference between Pylons and Tornado on a blank page 3 times in favor of the latter.
Something like this.
The story continues with the chat here: habrahabr.ru/post/228703 .
Threat in the text of a lot of jargon, I'm sorry but it’s more pleasant for me to write.
ZY2 errors most likely as much, write in a personal if you find.
I wrote about it here and here .
I often lacked performance, and I found and searched for various alternatives, often actively using C / C ++. A couple of libraries under the cut.
I apologize in advance if everyone already knows about this but I just recently unearthed these.
Json
Where now without him? For web applications, this is essentially the basic format. The server should be able to quickly convert python structures to JSON and give to browsers or devices. Probably most do import json and everything is fine with them ...
But if you need at times faster then meet ujson . Acceleration orders can be glanced at the link, but in fact, using ARM allowed ARM to reduce the request time from a few seconds to <1 second. The ujson syntax is similar to the usual json, although some attributes are missing (for example, I did not manage to hang up my handler for the datetime conversion). Also, ujson is compatible with python3 and I have never stumbled upon its glitches.
If you have a lot of non-English text in JSON, it is extremely advantageous to use the parameter ensure_ascii = False for the dumps function. The fact is that, by default, the text is encoded in JSON using \ u notation, which results in 6-8 bytes per character, and so everything can be given cleanly in utf-8, which reduces the returned JSON by 2 or 3 times (even if gzip is used).
Warning: for regular json, ensure_ascii = False leads to a serious loss of performance, for ujson, even an improvement in performance or a slight degradation is possible.
Feedparser
Many probably used this library to parse RSS, ATOM, etc. Her performance even strained me before, but now I couldn’t use it at all, since the parsing of a single RSS could last a minute. SpeedParser came to help me, he allowed to reduce the time of parsing RSS to a few seconds. In general, the syntax is compatible with FeedParser, but the behavior is very different. For example, he does not see custom namespace or ignore which is not in his list (which is tightly scored in lib). We have applied MonkeyPatching (alas) to edit some errors.
')
Gzip
The maximum values cannot be used, and it’s better to spin around a factor of 5. Below, the size starts to influence the speed very much, and above it slows down the formation of content to return. It is difficult to find an analogue for gzip (and it is easy to replace it in the system), but there is a tip - to use it only on the side of nginx and not to try from python. In one of the test projects, I used gzip compression as middleware in python and transferring it to nginx reduced the load.
Frameworks
The smaller the overhead projector the better. Do not try to deploy large frameworks, they will work, but already on a pair of other people everything will be a stake. The difference between Pylons and Tornado on a blank page 3 times in favor of the latter.
Something like this.
The story continues with the chat here: habrahabr.ru/post/228703 .
Threat in the text of a lot of jargon, I'm sorry but it’s more pleasant for me to write.
ZY2 errors most likely as much, write in a personal if you find.
Source: https://habr.com/ru/post/228707/
All Articles