In this publication, we will focus on setting up important parameters of the ASP.NET application pool when calling remote web services and actively working with the network on the server side through standard .NET classes.

Introduction
Have you ever set up production servers on Windows Server 2008 R2 / IIS 7.5 or higher? For system administrators who have extensive experience with IIS, this is most likely a trivial task, but for web developers, who for various reasons sometimes have to participate in setting up the “combat” servers, this information can be very useful.
')
So, we proceed. We accelerate a site on ASP.NET - we save money of the enterprise and nerves of the administrator.
Prehistory1. IIS configuration options2. Configure ASP.NET3. Recommendations for optimizing the basic configurationAdditionallyConclusionLinksPrehistory
At the end of last year, in a large organization, we faced problems with the performance of web servers with a dramatically increased user load. At that time, more than 200,000 clients were registered in the web application. In normal mode, about 1000 users work at the same time, about 10-15% of unique visitors from the total number of registered users per day, so the load is relatively low. However, there are peak loads at which the system turns out to be practically inoperable.
Web administrators checked everything they could and could not understand what was the matter. Indeed, despite the fact that on all physical parameters of the system at the physical level, the performance was good, there were failures with the availability of services, and a huge queue of requests gathered in the pool. The organization uses a 4-node NLB cluster (Windows Server 2008 R2 + IIS 7.5 + .NET 4.5), there is a margin for CPU and memory usage, network channels are large, the number of ports used is sufficient. All the checks indicated that the problems lie in the depths of IIS and the configuration of the ASP.NET pool. A vivid example when administrators would not be prevented by the help of experienced web developers ...
1. IIS configuration options
General Description of .NET ConfigurationBeginning with IIS 7, all ASP.NET configuration settings are stored in XML files (* .config). They replaced the metabase that was used in IIS 6.0 and earlier.
The configuration file scheme for IIS 7.x and higher looks like this:
Fig. 1. Diagram of configuration filesAt the top of the .NET hierarchical configuration is the
machine.config file . It defines global parameters for a specific machine. This file defines the supported sections of the configuration files, configures the ASP.NET worker process, and registers suppliers of various modules. To optimize the initialization process, the
machine.config file has been greatly simplified, and it is located in the directory:
%systemroot%\Microsoft.NET\Framework\<versionNumber>\CONFIG\
Here is the
machine.config.comments file , which allows you to find out which parameters are used by default. Using this data in
machine.config, you can add parameters with redefined values.
The root of the ASP.NET configuration hierarchy is the
web.config file located in the same directory as the
machine.config . This file includes parameters that are used for all ASP.NET applications.
ApplicationHost.config - the root IIS configuration file, includes descriptions of all sites, applications, virtual directories and application pools, as well as global default settings for web server settings. It is located in the following folders depending on the OS version:
- for 32-bit -% WINDIR% \ System32 \ inetsrv \ config \
- for 64-bit -% WINDIR% \ SysWOW64 \ inetsrv \ config \
Each local
web.config file applies configuration settings for the directory in which it is located, as well as for all child directories. Settings of nested directories can be overridden by their own “configs”.
Before you begin setting up IIS configuration, pay attention to the ASP.NET performance counters, evaluate the current and peak system load, record the available indicators. Check the logs for the error “HTTP Error 503.2 - Service Unavailable”. Try to determine if some of the requests in the queue are blocked.
If the system performance meets the needs of the customer, then it is better to leave the default settings, because they are designed for most ASP.NET applications.
When configuring IIS, there are two main parameters that affect the availability of the application and its performance.
1. The
appConcurrentRequestLimit parameter is the maximum number of simultaneous requests in the application. Increasing the number of concurrent IIS requests will expand the available web server resources to serve requests. The default is 5000.
The
appConcurrentRequestLimit parameter can be changed most quickly using the
appcmd.exe utility via the command line. This can be done globally for all IIS sites through the
ApplicationHost.config file, or for a separate site (application).
cd %windir%\system32\inetsrv appcmd.exe set config /section:system.webserver/serverRuntime /appConcurrentRequestLimit:20000
We execute the command, then open the “Configuration Editor” section in the IIS for the root directory and check the new value of the set
appConcurrentRequestLimit parameter. And here you can manually change this value.
Fig. 2. Set appConcurrentRequestLimit parameterTo set this parameter, the following formula is most often used:
<
usersCount * 1.5 >, where
usersCount is the number of simultaneously working users.
2. The
QueueLength parameter is the maximum number of requests that the Http.sys driver places in the application pool queue. When the queue is full, new requests receive the error "503.2 - Service Unavailable". The default is 5000.
This parameter can be configured in several ways:
- globally for .NET at the server level via machine.config , processModel / requestQueueLimit section;
- at the IIS level via ApplicationHost.config : system.web / httpRuntime -> appRequestQueueLimit;
- set the value of the queueLength parameter for a specific pool.
As an example, we will change this parameter for the DefaultAppPool pool via the command line:
appcmd.exe set apppool "DefaultAppPool" /queueLength:20000
We execute the command, then open the “Application Pools” section in IIS, select the “DefaultAppPool” pool from the list, go to the “Advanced Settings” menu and check.
Fig. 3. Set the queueLength parameterNote: View current requests in the running pool through “IIS -> Worker Processes”In IIS Manager, select the server node in the tree, then click on the "Worker Processes" icon:
Fig. 4. Worker Processes menu in IIS ManagerIn the list that appears, you can see the download of all currently running pools.
Fig. 5. View running pools through Worker ProcessesWhen you click “View Current Request” a table appears with a list of addresses of pages being processed and other useful parameters. To update the list, you can press F5 on the screen. Thus, you can find “pending” requests:
Fig. 6. List of current requests in the poolTo view the performance indicators, of course, it is better to use the Performance Monitor counters, but they will not show you, like the Requests Monitor, the URLs of current requests.
2. Configure ASP.NET
ASP.NET limits the number of worker threads and call completion ports used to execute requests. If a server-side web application actively uses external web service calls, standard classes from the System.NET namespace to organize requests over the network, then there may be conflicts of poor performance and deadlocks. At the beginning, a part of requests may simply “hang”, the execution time will increase significantly. In the worst case, if the classic pool configuration mode (the classic pipeline) is used, this can lead to a recycle in general. ASP.NET deadlock detection is not performed for a pool running in
integrated mode (default on IIS 7 and higher).
The operation of application pools in the integrated mode has several advantages compared with the work in the classical mode. It is recommended to run application pools in integrated mode.
The figure below clearly shows how requests are processed in ASP.NET and which parameters are most important:
Fig. 7. The processing of requests in ASP.NETFor optimal performance of web applications, the auto-configuration of pool settings is enabled by default. In this case, the autoConfig
property is equal to "
true " for the <
processModel > section in the
machine.config file , and other key parameters are not specified at all.
Having thoroughly “rummaged” in MSDN and in the
machine.config.comments file , I found a description of the basic configuration of the pool. There are 7 key parameters that affect the work of ASP.NET with services and network:
- maxConnection
- maxWorkerThreads / minWorkerThreads
- maxIoThreads / minIoThreads
- minFreeThreads
- minLocalRequestFreeThreads
The
maxconnection parameter specifies the maximum number of simultaneous requests from a single IP address. When the auto pool configuration is enabled by default, this parameter is determined by the formula:
maxConnection =
12 * cpuNum , where
cpuNum is the number of processor cores
Thus, on a server with a 4-core processor, the maximum number of simultaneous connections to the final IP address is 48 = 12 * 4 (by default).
The easiest way to get around this limitation is to specify the following directly in the code of your ASP.NET application in the
Application_Start method in the
global.asax file:
It is more flexible to configure
maxconnection better through configuration files at the application domain level (web.config) or web server (applicationHost.config). The <
system.net > section contains parameters that define how the .NET Framework connects to the network.
<system.net> <connectionManagement> <add address="*" maxconnection="5000" /> <add address = "http://www.habrahabr.ru" maxconnection = "9999" /> <add address = "http://65.53.32.230:88" maxconnection = "240" /> </connectionManagement> </system.net>
Important: The scheme for the address of the
maxconnection parameter should be like this:
http(s)://<IP- >:<>
Increasing
maxconnection allows you to make more simultaneous calls to remote services.
This attribute does not affect local web services calls! It is necessary to understand that it is not enough to bypass the limit on the number of simultaneous connections to the service. Since the increase in the number of simultaneous calls leads to an increase in the use of CLR threads, which are used to create remote and handle callbacks.
ASP.NET through the
maxWorkerThreads parameter sets the restriction of threads on the
w3wp.exe worker process (starting with IIS 7). Due to the fact that ASP.NET is built into IIS, ASP.NET processes generate requests for workflows. Due to an insufficient number of threads in the CLR ThreadPool, requests will queue up and freeze.
Attributes specified in the <
processModel > section:
1. The
maxWorkerThreads parameter indicates the maximum number of worker threads for each processor in the CLR thread pool. The default value is 20. The maximum value is 100.
2. The
maxIoThreads parameter specifies the maximum number of I / O threads for each processor in the CLR thread pool. The default value is 20. The maximum value is 100.
3. The
minWorkerThreads parameter — specifies the minimum number of worker threads for each processor that can be provided immediately to service a remote request. The default is 1.
4. The
minIoThreads parameter indicates the minimum number of I / O threads for each processor that can be provided immediately for callback processing. The default is 1.
The
minWorkerThreads /
minIoThreads parameters allow you to quickly deal with a sudden increase in the number of simultaneous connections, when in case of inactivity the thread pool may not have enough time to reach the optimal level of threads.
Attributes specified in the <
httpRuntime > section:
1. The parameter
minFreeThreads - determines the number of threads that can be used for work, except for processing incoming requests to the workflow. This parameter prevents the ASP.NET process from using threads from the pool to process a new HTTP request if the total number of threads in the pool drops below this limit. The default is 8.
2. The
minLocalRequestFreeThreads parameter defines the minimum number of free threads that ASP.NET keeps available for executing new local requests. The default is 4.
Notice that the
maxWorkerThreads ,
minWorkerThreads ,
maxIoThreads ,
minIoThreads parameters are implicitly multiplied by the number of processors, and the
minFreeThreads and
minLocalRequestFreeThreads parameters are not.
ASP.NET will not perform more than the following number of simultaneous requests:
(
maxWorkerThreads *
number of CPUs ) -
minFreeThreadsNote: for the entire application pool, that is, for each
w3wp.exe worker process that serves the pool, there is one thread pool CLR ThreadPool. For all application domains (sites) configured for one pool, a common set of threads is used. Therefore, for resource-demanding applications it is better to use separate pools.
3. Recommendations for optimizing the basic configuration
First of all, you need to accurately determine the number of processors on the web server. Alternatively, you can see TaskManager -> tab "Performance". If the processor supports
HyperThreadingTechnology (HTT) mode, then half of the cores are logical (Logical processors), and not physical (Cores). For example, when the HTT mode is enabled, a processor with 4 physical cores will work as 8 logical cores:
Fig. 8. Processor loading window in TaskManagerYou can also try using the following commands on the command line:
WMIC CPU Get DeviceID,NumberOfCores,NumberOfLogicalProcessors
or
echo %NUMBER_OF_PROCESSORS%
For example, on a server with 4 processors and the
autoConfig = "
true " property, ASP.NET will have the following default settings:
maxConnection - 48; maxWorkerThreads - 80; maxIoThreads - 80, minFreeThreads - 8, minLocalRequestFreeThreads - 4.
If the webpage on the backend makes multiple net calls for each request, MSDN recommends using the following configuration settings:
- maxWorkerThreads = 100 | minWorkerThreads = maxWorkerThreads / 2 = 50
- maxIoThreads = 100
- maxConnection = 12 * N
- minFreeThreads = 88 * N
- minLocalRequestFreeThreads = 76 * N, where N is the number of processors.
This section only provides recommendations, not rules. And the date of publication of this data is quite old. For our “combat” system, we use slightly different configuration parameters. These formulas are a good starting point for the start of optimization, they show well the dependence of parameters on each other. For example, by increasing the value of the
maxConnection parameter several times, you can easily “estimate” the basic values ​​for the remaining parameters.
Changes to the <
processModel > section are allowed to be made only in the
machine.config file due to the attribute
set allowDefinition = "
MachineOnly " when adding the
processModel section.
C: \ Windows \ Microsoft .NET \ Framework64 \ v4.0.30319 \ Config \ Machine.config :
<section name="processModel" type="System.Web.Configuration.ProcessModelSection, System.Web, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" allowDefinition="MachineOnly" allowLocation="false" />
To be able to set the values ​​of the
processModel section for each application separately via
web.config , you must set the property
allowDefinition = "
Everywhere ".
I will give the configuration settings from our web server. <system.web> <processModel autoConfig="False" maxWorkerThreads="100" maxIoThreads="100" minWorkerThreads="50" minIoThreads="8" /> <httpRuntime minFreeThreads="640" minLocalRequestFreeThreads="96"> </system.web> <system.net> <connectionManagement> <add address = "http://__1" maxconnection = "5000" /> <add address = "http://__2" maxconnection = "5000" /> </connectionManagement> </system.net>
Important: after making changes, you need to update the Application pools.
Remember that you need to increase these parameters only if necessary if you have enough CPU resources.
To analyze the performance of web servers, I recommend setting up ASP.NET counters via Performance Monitor:
- ASP.NET Applications \ Requests / Sec
- Web Service \ ISAPI Extension Requests / sec
- ASP.NET \ Requests Current
- ASP.NET \ Requests Queued
- ASP.NET \ Requests Rejected
- ASP.NET Applications \ Requests Executing
- ASP.NET Applications \ Requests Timed Out
- ASP.NET \ Request Execution Time
For a more in-depth analysis of the
w3wp.exe process serving the IIS application pool, you can try the
WinDbg debugger from the
Windows Software Development Kit .
It is possible that after checking the counters you will have to make adjustments to the configuration of your system.
Additionally
For a better understanding of how IIS works, I recommend that you familiarize yourself with the process of processing a request from a user's browser to the final pool of an ASP.NET application in this useful article,
“Basics of the IIS architecture, or query pipeline for ASP.NET” .
If you are using IIS8, it will not be superfluous to pay attention to
"Full CPU load control (CPU Throttling)".Conclusion
For sites that do not make frequent network requests on the server side, the standard pool settings should be sufficient (processModel / autoConfig = “true”). At the same time, IIS will set a limit of 20 worker threads and 12 remote connections per core. If these values ​​are exceeded, requests will start to queue and the performance of the web application will drop.
If your site works well and you can estimate the expected load on the system, then you should not change anything. If you start “freezing” when processing requests to various services - you should not immediately blame the hardware for everything! It is better to make changes to the basic ASP.NET configuration. Keep in mind that changing the basic parameters of the application pool will certainly lead to an increase in CPU usage. Optimal balancing of all system parameters is the key to stable and productive equipment operation. As the saying goes, “forewarned is forearmed.”
I invite everyone to share your experience in setting up and optimizing the work of production web servers on the Windows Server platform.
Links