NGINX - Acceleration or Detective for the programmer "Optimization for Windows"
Quite a lot of time has passed since my last article about nginx under windows , the week of nginx is over . It is worth correcting this omission.
Sometimes it happens that suddenly there is free time, but for something worthwhile it is not enough, but just do notclimb on the Internet, read Habr in every way to improve their skills do not want.
In order to do something useful, I decided to start analyzing the logs from some servers of one project, how much it would be possible to cram it into a couple of free minutes.
After a little analysis and evaluation in comparison with the results of the previous analysis, I noticed one oddity - the absolute return rate of nginx fell on average from 5 to 15%.
')
It was not possible to explain what caused this from the raid; there were no major changes, the data volumes didn’t grow as much either. And on the impact of the dynamics of strong changes a bit.
Having twisted the logs this way and that, I hooked on the return of small statics - one rule was found: the longer the path (url) - the more “slow” the nginx became (regardless of file size).
So after several experiments, we have the following facts:
And then came the insight - I remembered that the file structure in this project changed, and the nesting to a certain statics and dynamics given by the file (by redirect) increased by two-three, and in some places up to five directories.
Compiling the debug version under the profiler analyzer was too lazy, we will look for it in a “deductive” way.
Using siege, we measure the average “test.gif” download speed - about 5500 rps (nginx 4 workers, concurency 15).
Create a test location with root "D: \ ... \ ms \ ms \ ms \ ms \ ms \ ms \ ms \ ms", where the segment of the ms path is repeated 60 times and ask for the URL "test-loc / test.gif".
Again, we measure the average speed of return - already 900 rps.
Perhaps this is something somewhere at the level of checking the legality of the path. Try to measure 404 for a non-existent file with the URL “test-loc / test-file-not-exists.gif” - the speed grows to 13167 rps and that the strange nesting of the path no longer matters.
We turn on debag in the logging of nginx, and then we analyze in detail the logs in comparison between small and large nesting paths. On average, the time from “http request line” to “http filename” is the same, but the record “http static fd” for the long path is already significantly behind.
Oddly enough, therefore, we are trying to check what is happening, using process-monitoring.
And for each request it is observed twice, probably once at the opening, the second time when reading the attributes of the file.
We climb into the nginx sources, quickly find in
The same function
First, we check the correctness of the segment path, exclude NTFS streams (":"), dots and spaces at the end, etc. But in this loop body verification, no system calls are made.
Go ahead and see the GetLongPathNameW call in the block checking the long file name.
For the sample, we will comment out this check, compile nginx - as a result, we have 9100 rps, where previously nginx gave 900 rps.
Apparently, when Maxim did this check, he did not know (your humble servant, by the way, too) that Windows returns the long file name recursively, i.e. for each subdirectory (and as you can see this information is not cached in the OS).
Unfortunately, this way (that is, completely) you should not remove this check to exclude access to files by short names 8.3 (of the form “testpa ~ 1 / testfi ~ 2.gif”), and the best function that does the same, I Unfortunately, i do not know.
However, you can add the above setting of a special flag, checking the existence of the character "~" , only if there is one that we will verify the long path by calling
As a result, with minimal changes, we have an increase in speed from 2 to 10 times (depending on the nesting and file size) and several times less load on the disk subsystem.
Well, as usual performance comparison, before (the first and third columns are marked ** NF) and after.
The first two are for the short (12 segments) path.
The next two are for the long (55 segments) path.
Measurement siege (concurency: 15), win7 - i5-2400 cpu @ 3.10GHz (4 core), gif-statics: 452 bytes (+ header), 4 nginx workers.
In my branch nginx-mod, I have already integrated this PR .
There will be time, I will make a changeset for the original hg repository and send the CR to nginx-dev.
Sometimes it happens that suddenly there is free time, but for something worthwhile it is not enough, but just do not
In order to do something useful, I decided to start analyzing the logs from some servers of one project, how much it would be possible to cram it into a couple of free minutes.
After a little analysis and evaluation in comparison with the results of the previous analysis, I noticed one oddity - the absolute return rate of nginx fell on average from 5 to 15%.
')
It was not possible to explain what caused this from the raid; there were no major changes, the data volumes didn’t grow as much either. And on the impact of the dynamics of strong changes a bit.
Having twisted the logs this way and that, I hooked on the return of small statics - one rule was found: the longer the path (url) - the more “slow” the nginx became (regardless of file size).
So after several experiments, we have the following facts:
- upload speed drops in direct proportion to the increase in the path to the file
- the speed is almost independent of the URL length, i.e. if the URL is short, but we increase the length of root / alias, the upload speed also drops, i.e. it's still a long way, not a URL
- well, finally, having played with the file paths, namely its nesting, it turned out that the upload speed drops depending on the number of subdirectories, and does not depend on the length as such. Those. the file "D: \ ... \ ms-ms-ms-ms-ms-ms-ms-ms \ test.gif" is sent much faster "D: \ ... \ ms \ ms \ ms \ ms \ ms \ ms \ ms \ ms \ test.gif »
And then came the insight - I remembered that the file structure in this project changed, and the nesting to a certain statics and dynamics given by the file (by redirect) increased by two-three, and in some places up to five directories.
Compiling the debug version under the profiler analyzer was too lazy, we will look for it in a “deductive” way.
Using siege, we measure the average “test.gif” download speed - about 5500 rps (nginx 4 workers, concurency 15).
Create a test location with root "D: \ ... \ ms \ ms \ ms \ ms \ ms \ ms \ ms \ ms", where the segment of the ms path is repeated 60 times and ask for the URL "test-loc / test.gif".
Again, we measure the average speed of return - already 900 rps.
Perhaps this is something somewhere at the level of checking the legality of the path. Try to measure 404 for a non-existent file with the URL “test-loc / test-file-not-exists.gif” - the speed grows to 13167 rps and that the strange nesting of the path no longer matters.
We turn on debag in the logging of nginx, and then we analyze in detail the logs in comparison between small and large nesting paths. On average, the time from “http request line” to “http filename” is the same, but the record “http static fd” for the long path is already significantly behind.
Oddly enough, therefore, we are trying to check what is happening, using process-monitoring.
Under the spoiler we have the following picture
15:23:43,2205361 nginx.exe 9780 CreateFile D:\ SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2206208 nginx.exe 9780 QueryDirectory D:\Projects SUCCESS Filter: Projects, 1: Projects 15:23:43,2206430 nginx.exe 9780 CloseFile D:\ SUCCESS 15:23:43,2207337 nginx.exe 9780 CreateFile D:\Projects SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2207844 nginx.exe 9780 QueryDirectory D:\Projects\app_srv SUCCESS Filter: app_srv, 1: app_srv 15:23:43,2208009 nginx.exe 9780 CloseFile D:\Projects SUCCESS 15:23:43,2208728 nginx.exe 9780 CreateFile D:\Projects\app_srv SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2209238 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp SUCCESS Filter: tmp, 1: tmp 15:23:43,2209383 nginx.exe 9780 CloseFile D:\Projects\app_srv SUCCESS 15:23:43,2210075 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2210615 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp\ms SUCCESS Filter: ms, 1: ms 15:23:43,2210764 nginx.exe 9780 CloseFile D:\Projects\app_srv\tmp SUCCESS 15:23:43,2211466 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp\ms SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2211995 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp\ms\ms SUCCESS Filter: ms, 1: ms 15:23:43,2212141 nginx.exe 9780 CloseFile D:\Projects\app_srv\tmp\ms SUCCESS 15:23:43,2212813 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp\ms\ms SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2213326 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp\ms\ms\ms SUCCESS Filter: ms, 1: ms 15:23:43,2213462 nginx.exe 9780 CloseFile D:\Projects\app_srv\tmp\ms\ms SUCCESS 15:23:43,2214193 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp\ms\ms\ms SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2214677 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp\ms\ms\ms\ms SUCCESS Filter: ms, 1: ms 15:23:43,2214806 nginx.exe 9780 CloseFile D:\Projects\app_srv\tmp\ms\ms\ms SUCCESS # ... 106 lines ... # 15:23:43,2265328 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2265851 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Filter: ms, 1: ms 15:23:43,2265980 nginx.exe 9780 CloseFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS 15:23:43,2266665 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2267165 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Filter: ms, 1: ms 15:23:43,2267291 nginx.exe 9780 CloseFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS 15:23:43,2267976 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2268489 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Filter: ms, 1: ms 15:23:43,2268622 nginx.exe 9780 CloseFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS 15:23:43,2269314 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2269814 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Filter: ms, 1: ms 15:23:43,2269946 nginx.exe 9780 CloseFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS 15:23:43,2270615 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2271135 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Filter: ms, 1: ms 15:23:43,2271264 nginx.exe 9780 CloseFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS 15:23:43,2271989 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2272525 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Filter: ms, 1: ms 15:23:43,2272654 nginx.exe 9780 CloseFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS 15:23:43,2273343 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2273876 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Filter: ms, 1: ms 15:23:43,2274005 nginx.exe 9780 CloseFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS 15:23:43,2274743 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2275246 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Filter: ms, 1: ms 15:23:43,2275375 nginx.exe 9780 CloseFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS 15:23:43,2276034 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2276531 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Filter: ms, 1: ms 15:23:43,2276657 nginx.exe 9780 CloseFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS 15:23:43,2277332 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Desired Access: Read Data/List Directory, Synchronize, Disposition: Open, Options: Directory, Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened 15:23:43,2277819 nginx.exe 9780 QueryDirectory D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS Filter: ms, 1: ms 15:23:43,2277944 nginx.exe 9780 CloseFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms SUCCESS 15:23:43,2278987 nginx.exe 9780 CreateFile D:\Projects\app_srv\tmp\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\ms\test.gif SUCCESS Desired Access: Generic Read, Disposition: Open, Options: Synchronous IO Non-Alert, Attributes: n/a, ShareMode: Read, Write, Delete, AllocationSize: n/a, OpenResult: Opened And for each request it is observed twice, probably once at the opening, the second time when reading the attributes of the file.
We climb into the nginx sources, quickly find in
[src]\os\win32\ngx_files.c function, which only if the file is opened successfully (and we remember the quick 404), calls the other function ngx_win32_check_filename (checking the legality of the file name).The same function
ngx_win32_check_filename is called by another function ngx_file_info . Well, consider under the magnifying glass our candidate for the role of "handbrake".First, we check the correctness of the segment path, exclude NTFS streams (":"), dots and spaces at the end, etc. But in this loop body verification, no system calls are made.
Go ahead and see the GetLongPathNameW call in the block checking the long file name.
For the sample, we will comment out this check, compile nginx - as a result, we have 9100 rps, where previously nginx gave 900 rps.
Apparently, when Maxim did this check, he did not know (your humble servant, by the way, too) that Windows returns the long file name recursively, i.e. for each subdirectory (and as you can see this information is not cached in the OS).
Unfortunately, this way (that is, completely) you should not remove this check to exclude access to files by short names 8.3 (of the form “testpa ~ 1 / testfi ~ 2.gif”), and the best function that does the same, I Unfortunately, i do not know.
However, you can add the above setting of a special flag, checking the existence of the character "~" , only if there is one that we will verify the long path by calling
GetLongPathNameW .As a result, with minimal changes, we have an increase in speed from 2 to 10 times (depending on the nesting and file size) and several times less load on the disk subsystem.
Well, as usual performance comparison, before (the first and third columns are marked ** NF) and after.
The first two are for the short (12 segments) path.
The next two are for the long (55 segments) path.
| Path, Segments | Short, 12 ** NF | Short, 12 | Long, 55 ** NF | Long, 55 |
|---|---|---|---|---|
| Transactions | 16732 hits | 28380 hits | 2962 hits | 26820 hits |
| Availability | 100.00% | 100.00% | 100.00% | 100.00% |
| Elapsed time | 2.96 secs | 2.96 secs | 2.97 secs | 2.97 secs |
| Data transferred | 7.21 MB | 12.23 MB | 1.28 MB | 11.56 MB |
| Response time | 0.00 secs | 0.00 secs | 0.00 secs | 0.00 secs |
| Transaction rate | 5645.07 trans / sec | 9591.08 trans / sec | 996.04 trans / sec | 9030.30 trans / sec |
| Throughput | 2.43 MB / sec | 4.13 MB / sec | 0.44 MB / sec | 3.89 MB / sec |
| Concurrency | 14.92 | 14.72 | 14.80 | 14.12 |
| Successful transactions | 16732 | 28380 | 2962 | 26820 |
| Failed transactions | 0 | 0 | 0 | 0 |
| Longest transaction | 0.11 | 0.11 | 0.12 | 0.12 |
| Shortest transaction | 0.00 | 0.00 | 0.00 | 0.00 |
In my branch nginx-mod, I have already integrated this PR .
There will be time, I will make a changeset for the original hg repository and send the CR to nginx-dev.
Source: https://habr.com/ru/post/268459/
All Articles