Fast autocomplete using libscgi
The article " libscgi - an effective solution for simple and fast scripts " was recently published. I would like to continue the previously presented topic from a more practical side. As you already guessed from the name - it will be about the autocompet script.
Users, and me in particular, have always been smashed by these slow autocomplexes. It happens you type up almost the entire word, as a drop-down menu appears. It's clear that javascript itself is slow. But if the autocomplete still slows down on the server side, then in general it somehow turns sour. There is a way out, if you use something faster on the server side, as an option you can use a binary script using libscgi. Sources here . Just below the details.
In this example, the script performs autocompet on cities. Data for all cities is recorded in a csv file. This file (I have 26 thousand lines) reads into memory and is stored in the form of a hash array. the key is the lower case name of the city or its nickname. In connection with the renaming to Soviet and post Soviet times, some cities have several names, for example: Leningrad, St. Petersburg, St. Petersburg. Therefore, the first column is the given name or the given alias of the city. The second column is the official name of the city (the first letter is big). The third column is the rank of the city. If we print all the cities in M ​​letters, then it is preferable for the User that Moscow be earlier (first in the list) than Makeevka or Robin. The fourth column is the city id by database. It is possible to implement the load from the database directly, when loading the autocomplete daemon. Rank descending - the highest priority is 1.
')
Sample input file data:
For a search request, the POST method is used, whose body is one-two-three or more search characters. The request is sent to the predefined url http: // hostname / cityes This url can be changed by specifying a new one in the source code.
Since autocomplex is implemented using AJAX technology, json was chosen as the output data format. Output format:
The output limit is 10 menu items. Oktika showed that no longer needed. But if you want, then you can change the DEFINE definition in the source: autocomplete.cpp
optional parameter “time”, implemented for control - search time in microseconds. Real time is slightly longer, due to processing by the scgi server itself and nginx. Actually it is somewhere between 0.5-0.7 ms (milliseconds).
The daemon's memory capacity for 25 thousand data lines is 600 K. The CPY consumed is almost 0%. It unloads the database very well, if we have previously had autocomplete on the database (I used redis, but with the introduction of libscgi - I refuse it on the sly). While everything is in the testing mode, but it works beautifully and quickly.
make lib - compile library
make ac compile autocomplete file
./server / path / to / cityes_file
We offer ideas where else you can use libscgi.
Let's make together the world a kinder and better!
Users, and me in particular, have always been smashed by these slow autocomplexes. It happens you type up almost the entire word, as a drop-down menu appears. It's clear that javascript itself is slow. But if the autocomplete still slows down on the server side, then in general it somehow turns sour. There is a way out, if you use something faster on the server side, as an option you can use a binary script using libscgi. Sources here . Just below the details.
Data Description
In this example, the script performs autocompet on cities. Data for all cities is recorded in a csv file. This file (I have 26 thousand lines) reads into memory and is stored in the form of a hash array. the key is the lower case name of the city or its nickname. In connection with the renaming to Soviet and post Soviet times, some cities have several names, for example: Leningrad, St. Petersburg, St. Petersburg. Therefore, the first column is the given name or the given alias of the city. The second column is the official name of the city (the first letter is big). The third column is the rank of the city. If we print all the cities in M ​​letters, then it is preferable for the User that Moscow be earlier (first in the list) than Makeevka or Robin. The fourth column is the city id by database. It is possible to implement the load from the database directly, when loading the autocomplete daemon. Rank descending - the highest priority is 1.
')
Sample input file data:
"","",2188,2188
"","",9711,9711
"","",21156,21156
"","",1424,1424
For a search request, the POST method is used, whose body is one-two-three or more search characters. The request is sent to the predefined url http: // hostname / cityes This url can be changed by specifying a new one in the source code.
Since autocomplex is implemented using AJAX technology, json was chosen as the output data format. Output format:
{
"cityes" : [
{ "name": "New York", "id":90},
{ "name": "New Vasyki", "id":808},
{ "name": "Nazareth", "id":1505},
{ "name": "Neaple", "id":1965}],
"time":304
}
The output limit is 10 menu items. Oktika showed that no longer needed. But if you want, then you can change the DEFINE definition in the source: autocomplete.cpp
optional parameter “time”, implemented for control - search time in microseconds. Real time is slightly longer, due to processing by the scgi server itself and nginx. Actually it is somewhere between 0.5-0.7 ms (milliseconds).
The daemon's memory capacity for 25 thousand data lines is 600 K. The CPY consumed is almost 0%. It unloads the database very well, if we have previously had autocomplete on the database (I used redis, but with the introduction of libscgi - I refuse it on the sly). While everything is in the testing mode, but it works beautifully and quickly.
Compilation:
make lib - compile library
make ac compile autocomplete file
Launch
./server / path / to / cityes_file
We offer ideas where else you can use libscgi.
Let's make together the world a kinder and better!
Source: https://habr.com/ru/post/111919/
All Articles