Basics of IIS architecture, or query pipeline for ASP.NET

Last year I had to interview about 10-15 candidates for the position of a web programmer on an ASP.NET secondary qualification. As questions "on filling", or "with an asterisk", I asked to tell what happens with the HTTP-request from the moment it arrives at the 80th port of the server before transferring control to the aspx-page code. The statistics was depressing: none of the candidates could not give at least something intelligible. And this has its own explanation: neither in MSDN with technet, nor on the specialized resource iis.net, nor in the a-la books “ASP.NET for professionals,” or in blogs, this topic is given proper attention — information has to be collected almost bit by bit. I even know people who have decided to write their own web server (Igor, George, hello!), So as not to understand the work of IIS. The only sensible article is “ Introduction to IIS Architectures ” by Reagan Templin. But it also remains on the periphery of the interests of aspnetchiki.

Although I personally are not so interested in purely technical issues, I decided to gather together my accumulated experience, unearth curious details in the Internet, and transfer this sacred knowledge to the masses before it is outdated. At once I will make a reservation that the article is focused more on IIS 7.x, sometimes there will be branches about 6-ku. I didn’t come across the 8th version in my work, so I decided to bypass it in this article. But, I am sure, the reader will easily deal with the eight, having mastered the material set forth below.

1. General plan
2. Close-up
2.1. HTTP.SYS
2.2. World Wide Web Publishing Service (W3SVC)
2.3. Windows Process Activation Service (WAS)
2.4. Application pool
2.5. Application Domain, Application
3. What's next?
Sources

1. General plan

So, let's start from the end, and then consider some aspects a little more closely.
In the English literature, the request processing process in IIS is called the “request processing pipeline” - something like the “request processing pipeline”. In general terms, it is presented in the figure below for the http request.
')

Fig. 1. HTTP request processing pipeline (IIS 7.x).

Thus, the http request passes through the “assembly line of the conveyor” through the following:

1. The browser accesses the web server at a specific URL; on the server side, the request is intercepted by the HTTP.SYS driver.
2. HTTP.SYS knocks on WAS to get information from the configuration repository.
3. WAS service requests a configuration from the repository - from a file in the IIS folder (applicationHost.config).
4. Since this request is received via the HTTP protocol, the W3SVC service (it’s the WWW Service in the picture) receives configuration information, this information contains data about the application pool and other site parameters.
5. The W3SVC service uses this information to configure HTTP.SYS .
6. The WAS service starts the W3WP.exe process for the application pool, if it has not already been started.
7. In the process of W3WP.exe , the website application is running, which, in fact, forms and returns the response to the HTTP.SYS driver.
8. HTTP.SYS sends the response to the browser.

In principle, this scheme is already enough to pass the interview in most companies and get a general idea of ​​the architecture of IIS. But if you haven’t come here for a tick, then please follow.

2. Close-up

Now we’ll dwell on each of the mentioned components in more detail.

2.1. HTTP.SYS

At the transport level, IIS uses protocol listeners that are located on top of the TCP / IP stack. The most interesting such component to us is the HTTP.sys system driver, which is built into the OS kernel and works with the HTTP and HTTPS protocols, registering independently for listening on all the ports that requests to sites in IIS will receive.

HTTP.sys, which is built into the kernel, has become an innovation in IIS 6, replacing the Windows Socket API, a component of intercepting HTTP and HTTPS requests at the user level in earlier versions of IIS. Probably, the integration of the driver into the kernel is the very reason why the IIS version is tightly tied to the Windows version.

The driver accepts all incoming requests and redirects them to the desired application pool. If for some reason the workflow, in which the required pool is hosted, is stopped (crash, idle timeout, configuration change, etc.) or is still running, then HTTP.sys stores incoming requests in a queue specially allocated for each pool. Thus, user requests do not disappear anywhere, and they do not notice any interruptions in the work of sites running IIS.

HTTP.sys is also able to cache responses (for more detail, Instances in which HTTP.sys does not cache content ), so some requests are processed without being sent to the application level, and it also performs a primary analysis of the request URI and its validation in accordance with RFC 2396 (some - what you can learn from here - Use of special characters like '%' '.' and ':' in an IIS URL ) and logging requests / responses.

Some HTTP.sys settings have been moved to the Windows registry (for more information, the Http.sys registry settings for Windows ). By the way, in the same place - in the registry - you can peek at the usual place of residence of our citizen:% SystemRoot% \ system32 \ drivers \ http.sys.

Frankly, in the process of writing this article, I myself discovered some details for myself. For example, caching responses at the HTTP.sys driver level. This helped me to explain one case of a strange, as it then seemed to me, phenomenon in the behavior of IIS. Marketers put a swf card on the site before the next holiday, but then they didn’t like something in the file name and they renamed it. However, the site continued to issue a postcard at the old URL, and even clearing the browser cache did not help. Here I have already connected, but neither restarting the website and the entire application pool, nor accessing the website bypassing the corporate proxy server, yielded the expected result. But now we know who is to blame.

2.2. World Wide Web Publishing Service (W3SVC)

This service (abbreviated as WWW service specifications) was introduced in IIS 6 as a separate component for working with the HTTP / HTTPS protocols and managing application workflows and performing the following functions:

• Administering the HTTP.sys driver.
• Workflow Management.
• Monitoring website performance indicators.

This service functions in Windows Server 2003 in the context of the Svchost.exe process (settings can be viewed in the HKEY_LOCAL_MACHINE \ SYSTEM \ CurrentControlSet \ Services \ W3Svc registry ), unlike all other IIS services that run in the context of the Inetinfo.exe process, and is implemented in Iisw3adm .dll.

In IIS 7.x, the process management function was moved to a separate service - WAS (see clause 2.3) in order to universalize the architecture. Now WWW-service has become inherently one of the adapters, specializing in the HTTP / HTTPS protocols - working on top of the HTTP.sys driver. However, the WWW service remains the cornerstone component of IIS, so its configuration differs from configuring adapters to other protocols (a bit more similar here); it operates in the same workflow as WAS, and is implemented in the same library (Fig. 2).


Fig.2. Workflow with W3SVC and WAS services.

Since we are talking about adapters to protocol listeners (protocol listener adpater), then let's just linger and see what they are. In principle, IIS 7.x can be configured to process requests for any protocols besides typical HTTP and FTP, for example, POP3, SMTP, Gopher. You are even free to invent your own protocol for your web or WCF service and implement all the necessary components for it, if you are not sorry for your time. Most likely, adapters and listeners for the most common protocols are available for free and commercial download - I did not check this. But first of all, it’s worth paying attention to the standard services (Fig. 3) supplied with the .NET Framework and integrated with IIS:

• NetTcpActivator for TCP;
• NetPipeActivator for Named Pipes;
• NetMsmqActivator for Message Queuing (aka MSMQ).

Fig. 3. A list of standard non-HTTP adapters in the Windows Services snap-in.

But still the most important adapter for us is the WWW service, t.ch. Let us dwell in more detail on the two remaining 6 functions from IIS.

Administer and configure HTTP (S). At the time of updating the website configuration, the WAS service sends this information to the WWW service, and the latter, in turn, configures HTTP.sys to listen on specific ports, parse the IP and header of the requested site, and possibly other driver settings. In the opposite direction, W3SVC accesses WAS when a new one arrives in the request queue in HTTP.sys — to receive the workflow-processor of this request.

Tracking performance indicators. The WWW service keeps performance counters using the HTTP.sys driver for this, and provides their metrics with websites and the IIS cache. I could not find more detailed information on this issue.

2.3. Windows Process Activation Service (WAS)

So, the WWW service in IIS 7.x, as in IIS 6, continues to perform HTTP.sys administration tasks and managing website performance indicators. But the task of managing workflow submitted to a separate service - WAS. It is launched by the system in a single copy, reads the configuration from the % SystemRoot% \ System32 \ inetsrv \ Config \ ApplicationHost.config file and configures protocol listeners through the appropriate adapters in accordance with the information specified in it. Recall that for HTTP / HTTPS protocols, the adapter is the W3SVC service, and the listener is the HTTP.sys driver. When the listener intercepts the request, it calls the WAS service through its adapter to receive the application workflow, to which the request will be sent for processing and forming a response to the client.

When activating a user request for processing a user request, the following components are involved:

• Listener adapters are special Windows services that work with a specific protocol and interact with WAS to direct requests to the correct workflow.
• Actually WAS. She is responsible for creating workflows and managing their lifetime.
• The w3wp.exe executable file is a workflow template.
• The application manager manages the creation and utilization of application domains (application domains) that are hosted within the workflow.
• Protocol handlers are protocol-dependent components within a workflow, responsible for exchanging data between a specific adapter and a workflow. There are 2 types of protocol handlers: the process (process protocol handler - PPH) and the application domain (AppDomain protocol handlers - ADPH).

The figure below shows an example of the component schema inside an instance of an application's workflow. When the WAS service starts the workflow, it loads the required process protocol handlers (PPH) into it according to the application configuration and, through the application manager, creates an application domain within the workflow that will host the application. The application manager loads the application code into the application domain and the required application level protocol handlers (ADPH) to process messages using the appropriate network protocols.


Fig. 4. Components w3wp.exe to interact with external components.

As noted above, the .NET Framework carries the implementation of components for the HTTP / HTTPS protocols (our favorite ASP.NET), net.tcp, net.pipe, and MSMQ. The HTTP / HTTPS and FTP protocol stacks are nevertheless more closely integrated into IIS and OS, therefore, the setting for the new protocol is better demonstrated by the example of less popular dotnet protocols. So, after installing the framework, the following entries appear in the IIS ApplicationHost.config configuration file:

<system.applicationHost> <listenerAdapters> <add name="http" /> <add name="net.tcp" identity="S-1-5-80-3579033775-2824656752-1522793541-1960352512-462907086" /> <add name="net.pipe" identity="S-1-5-80-2943419899-937267781-4189664001-1229628381-3982115073" /> <add name="net.msmq" identity="S-1-5-80-89244771-1762554971-1007993102-348796144-2203111529" /> <add name="msmq.formatname" identity="S-1-5-80-89244771-1762554971-1007993102-348796144-2203111529" /> </listenerAdapters> </system.applicationHost> 

And the corresponding PPH and ADPH components are configured in the dotnet machine machine.config:

 <system.web> <protocols> <add name="net.tcp" processHandlerType="System.ServiceModel.WasHosting.TcpProcessProtocolHandler" appDomainHandlerType="System.ServiceModel.WasHosting.TcpAppDomainProtocolHandler" validate="false" /> <add name="net.pipe" processHandlerType="System.ServiceModel.WasHosting.NamedPipeProcessProtocolHandler" appDomainHandlerType="System.ServiceModel.WasHosting.NamedPipeAppDomainProtocolHandler”/> <add name="net.msmq" processHandlerType="System.ServiceModel.WasHosting.MsmqProcessProtocolHandler" appDomainHandlerType="System.ServiceModel.WasHosting.MsmqAppDomainProtocolHandler" validate="false" /> </protocols> </system.web> 

Along with the application settings, bindings are stored in the applicationHost.config web server configuration file, which define the parameters of incoming requests that will be sent to this application. These parameters are the name of the network protocol, the IP address of the server, the domain name and the port of the site. These parameters must be unique among running applications to uniquely identify the target application. The WAS service monitors this restriction and will not allow you to start the site for which this condition is not met, or will offer to stop the site with the same bundle.

  <bindings> <binding protocol="http" bindingInformation="109.120.157.xxx:80:dddddd.info" /> <binding protocol="http" bindingInformation="109.120.157.xxx:80:www.ddddd.info" /> </bindings> 

Please note that in the standard IIS operation mode, the WAS service, the adapter service for each protocol listener (including W3SVC) and the drivers / listeners of each of the protocols (including HTTP.sys) are running in the OS an instance of. But individual requests can be sent to different applications in different workflows. On the other hand, requests for different protocols can be sent to a single application via the appropriate adapters. Apparently, for the correct implementation of this behavior, an architectural combination of the protocol driver — the protocol driver adapter — the activation service (a kind of traffic controller, more precisely, the router) —the workflow — was invented.

2.4. Application pool

When configuring a web application, in addition to binding (binding) to request parameters and other settings, the application pool is indicated. The application pool has become an innovation in IIS 6 and was designed to provide isolation of web applications from each other and thereby increase the stability of the web server as a whole. The bottom line is that the application code is executed within a special Windows process - w3wp.exe. Therefore, an exception within a web application will only crash this process and will not affect the availability of web applications in other pools or the operation of IIS. Moreover, the WAS service will try to restart the downed site, and external clients may not even notice problems with the server.

To manage certain parameters of a single w3wp.exe workflow in IIS, an application pool is used. The most commonly used of these are the account under which the process will run, restrictions for the request queue, various timers and counters for automatically restarting the process, the x86 / x64 architecture (in IIS 7.x) and some others (Figure 5), What a curious reader can easily read in MSDN and your favorite search engine. So You can talk (with certain reservations, see also the last paragraph in 2.5) about the identity of the w3wp.exe process and the application pool.


Fig. 5 Advanced Application Pool Settings

The key innovation in the concept of application pools in IIS 7.x was a new parameter - a container management model, which can take 2 values: classic (Classic mode) and integrated model (Integrated mode).
To explain the difference between these modes of operation, familiarity with the concept of “module” in IIS 6 / 7.x and the event-based model for processing requests in conjunction with IIS + ASP.NET is required. This topic is worthy of a separate article, but, alas, it is not enough for me, apparently. Here I present to your attention only general, key points.

So, when processing a request, IIS passes it inside the workflow through a sequence of special component modules. For example filtering, redirection, caching, authentication, authorization. Each such module is associated with a specific event, and their sequence constitutes an event-based model for processing requests. Modules are divided into Native and Managed. Native modules are shipped with IIS, and managed modules are included as part of the .NET Framework (ASP.NET). In general, you can manage them to a certain extent at the configuration level of a web application, but you can interact with the code of your ASP.NET site only with managed modules.


Fig. 6. The ideology of the modules in IIS.

The classic container management model provides backward compatibility with the isolation process of working processes in IIS 6 — requests to an ASP.NET site are first passed through native modules and then transferred to Aspnet_isapi.dll for processing by modules in a managed environment. This separation between IIS and ASP.NET leads to the duplication of certain functions, such as authentication and authorization. And you do not have the ability to control programmatically the behavior of native modules (an example, though not the most burning, but still - the section "Remove the Server header" in this article ).

The embedded model assumes a closer interaction between IIS and ASP.NET. A request in such a processing architecture is passed through an established sequence of events, in each of which the request is passed through the native and managed modules. In this mode, IIS and ASP.NET request processing models are combined into a single model, which allows avoiding duplication of functions and gaining more control over processing the request.

In practice, the most important thing to consider when developing and deploying web applications is the partial incompatibility of these two modes. Those. when translating a site (more precisely, the application pool in which the site operates) from the classical model to the embedded one, an adjustment of the code (although perhaps not significant) will always be required, as well as thorough testing.

2.5. Application Domain, Application

The actual containers for the web application are the application and the application domain (Application Domain, AppDomain). Often, these two concepts are identified, but still these are slightly different things. An application is an IIS concept, and an application domain is from ASP.NET. Moreover, in the general case there may be several domains in the application. You can manage the application from the IIS console, and the application domain - mainly programmatically. For example, the application is restarted from the console. And when we re-save the web.config, then the application domain is reloaded, not touching the IIS application.

More important from a practical point of view is that the application / application domain is a sandbox for the code of your ASP.NET site (not with such reliable isolation as in the case of the pool, but still). I will cite one of my favorite questions that I asked the job seekers. Suppose there are a website-1 and a website-2, as well as a certain library MyLib.dll, in which the class MyClass1 with the static field Field1 is defined. So, both sites are running the same application pool and use the same MyLib.dll library. Website-1 writes to MyClass1.Field1 = 16 (Fig. 7). Question: Will website 2 see the changes made? The answer is “No”. But why? Because, for IIS applications, non-overlapping address spaces are allocated, even if they work within a single workflow, therefore, copies of assemblies are loaded into the memory of web applications (please do not pick on possible inaccuracies in terms of working with memory in the .NET Framework).

Fig. 7. Drawing for a puzzle.

Another important point that I would like to mention here. By default, each individual workflow can use all of the processors / cores on the server, and the application pool runs on the same workflow and, therefore, the web application runs inside one IIS application. However, you can configure the web garden by increasing the number of worker processes per pool and, consequently, the number of IIS applications per web application. You can easily find information on the web garden on the Internet, so I omit the details here. The only thing I would like to warn is that this tool is not a tool to increase productivity, because by default, and so all the computing power of the server. On the contrary, the synchronization of the work of 2+ working processes took “extra” CPU time. This is done mainly to increase the availability of the web application. It is also impossible not to mention the web farm (web farm) as the simplest load balancing tool in IIS - there are also enough articles on the Web. This is another example of a distributed web application. However, with the same nginx, the built-in load balancing in IIS does not withstand the competition, and in real high-load systems you will have to reinvent your bike or use third-party products.

3. What's next?

Next you need to understand the work of the modules (in terms of IIS) and the event model, in which the actual processing of the request already takes place, as mentioned in Section 2.4. Generally speaking, this topic deserves a separate article for which, I am afraid, I will not be enough. But without this, we cannot say that we considered the entire request processing pipeline. Therefore, we will briefly go through here on the main points that a curious reader can work through on their own.

As noted above in section 2.4, IIS modules are contained within a workflow. Through them the request is sequentially passed (unlike HttpHandler). Their set and order is determined by the configuration of the server and / or a specific web application. The modules are designed for individual, focused tasks, such as authorization, caching, custom logging, compression, return of static content and, of course, the formation of HTML-pages at a given URL.

As we already know, modules in IIS come in 2 types: native and managed. You can find the exact list of modules in MSDN or in the article Regan Templin. You can always write your own module, for example, for redirects. Most often, of course, make managed modules, because they are easiest to implement. By the way, ASP.NET WebForms and MVC work in the form of such managed modules. T.ch. I personally have WebForms vs. Holivars MVC cause a smile and a hard pent-up desire to troll. Knowing the principles of IIS and ASP.NET, you can implement any pattern you like.

At the next level of consideration, we will encounter ASP.NET components, such as HttpHandlers and page processing events. About this written mass of articles, t.ch. I do not see any reason to linger on this. The only thing I will allow myself to advise those going to the interviews to score in the ASP.NET page lifecycle search engine before the meeting is precisely my deep conviction that it is embarrassing for experts who consider themselves to be ASP.NET developers.

Sources

1. – Introduction to IIS Architectures
2. Application Domains Overview
3. Application vs. AppDomain
4. ASP.NET Application Life Cycle Overview for IIS 5.0 and 6.0
5. ASP.NET Application Life Cycle Overview for IIS 7.0
6. Beginner's Guide: How IIS Process ASP.NET Request
7. Exchange Server 2003 Services Dependencies: Internet Information Services
8. HTTP Request Lifecycle Events in IIS Pipeline that every ASP.NET Developer Should Know
9. Http.sys registry settings for Windows
10. Understanding IIS 7.0 Architecture: Non-HTTP Request Processing
11. Windows Server 2008 Events and Errors: IIS Windows Process Activation Service (WAS)
12. Windows Server 2008 Events and Errors: IIS World Wide Web Publishing Service (W3SVC)
13. Windows Server 2008 R2: Windows Process Activation Service (WAS) Overview