IIS as an edge web server (now haproxy)
In this article I want to describe not the most typical scenario, which, however, has the right to life.
The fact is that we use IIS as a proxy for other web servers of the company. I will tell you how this is implemented and what difficulties have been encountered.
Formulation of the problem
Let us analyze the example of the YouTrack server. It is represented by unsightly srv-youtrack-01.local.domain and is located on a web server inside the company. The task is to provide access to it from the Internet using the beautiful name yt.company.ru. In this case, https must be used.
Implementation
To get started, we need to install the URL Rewrite component. This can be done using the web platform installer, as well as manually . After installing it, we’ll see a new shortcut in IIS Manager.
URL redefinition ".
With this tool, you can create a reverse proxy address rewrite rule.
')
When creating a rule, you need to specify the server URL (without the http: // prefix - IIS will add it automatically) to which proxying will occur. As a result, we get a rule that is available for editing. It does not apply to all requests, but only to those that fit the criteria that we can customize. To begin with, the URL is checked for compliance with the pattern, after which other criteria are checked.
I’ll just make a reservation that there are two ways: the first way is to create a set of rules with different URL patterns for different resources on the same IIS site; and the second is to create a site for each proxied resource and in each of them make one rule. Understanding that the first path is more Jedi, I nevertheless chose the path of the second - albeit not so beautiful, but I did not venture to write the wrong regular expression for one site to break all routing. Therefore, the URL pattern I have everywhere default "(. *)".
So, I create a website yt.company.ru with binders for port 80 and port 443 and a mandatory indication of the host name so that IIS knows which site I am accessing. About obtaining and installing a certificate for 443 I allow myself not to mention. I’ll only note that you don’t need to configure the service to use https - inside the network, no one will be encrypted, and external requests will be connected via ssl to the border server, which will already proxy requests inside the network via an unprotected channel.
While the mandatory requirement is the use of https, we will only proxy those requests that come to port 443, for which we will create a simple condition. When you create it, a drop-down list of possible options is offered.
Great, now all yt.company.ru requests are proxied to the internal server with the unsightly name srv-youtrack-01.local.domain transparent to the user.
However, all requests yt.company.ru cut off with an error of 403, which is not very beautiful. To solve this problem, you can either create index.html with a redirect, or another URL Rewrite rule, in which in the "action" field we select a permanent redirect to the URL we need.
It should be noted that the rules for the site are applied in order, so you must first place the rule with the condition, and then the rule without conditions. At the same time, since the second rule applies to all URLs without exception, for the first rule it is necessary to tick (leave checked) “Stop processing subsequent rules”.
After manipulating the graphical interface, a web.config is created in the root of our site, which contains all the rules created. Therefore, if you need to proxy some other site, you do not need to repeat these manipulations, you can simply copy the web.config and change the required URL in it or, after copying, use the graphical interface to change the rules. Moreover, it is possible not to use the interface at all, but to write right there - as you like.
Underwater rocks
When you click the Agile Boards tab, YouTrack generates a URL like yt.company.ru/rest/agile/Overview-0/sprint/Iteration+24 . Further, when switching between sprints yt.company.ru/rest/agile/Overview-0/sprint/Iteration%252023?q= . When switching to these URLs, IIS began to return me a 404 error. This indicated that requests were not being proxied. In this case, the transitions between the saved queries of the form yt.company.ru/issues/IT?q=%23 {Current + work} + Assigned + to% 3A + me + updated% 3A + {This + week} were completely correctly worked out.
Experiments with the addition of a question mark in the middle of the problematic URL ended up in that I began to receive a 404 error already from the YouTrack server, and not IIS. This gave me the idea that IIS interprets the URL for some reason (hello, Microsoft) and this needs to be fixed.
The problem with the plus sign in the middle of the address was solved by adding the requestFiltering parameter to allowDoubleEscaping = “true” :
But after that, switching between sprints still didn't work. It turned out that IIS considers such requests unsafe. This check also had to be disabled:
This is how web.config turned out after all the manipulations:
Results
Probably, the solutions that I found are not optimal, and instead of resolving everything in a row, it was necessary to carefully write down the rules appropriate in a particular case. But now it works. I will be glad to hear your thoughts and suggestions.
Thus, in our organization, absolutely all web servers that need external access are proxied, among which are nginx, and apache, and svn, and gitlab, and exchange web access.
The main problem that makes me look for a solution is that NTLM authentication does not work through a proxy, which is so necessary for many Microsoft services. I don’t want to use the dead TMG product, so now I’m trying to figure out the new Windows Server 2012 R2 service called Web Application Proxy while simultaneously glancing at nginx and apache, which, it seems, are not yet able to proxy NTLM either.
http://www.ifinity.com.au/Blog/EntryId/60/404-Error-in-IIS-7-when-using-a-Url-with-a-plus-sign-in-the-path
stackoverflow.com/questions/2831142/asp-net-4-url-limitations-why-url-cannot-contain-any-3f-characters
Big update:
In the comments I was advised to try haproxy. Logging on to the site, I made a search on the “ntlm” page and found “full HTTP”, which gave me confidence.
After a few days of active fuss in the console, I still mastered this wonderful tool and now I no longer need IIS as a proxy server. I think it’s not worth writing a separate article about it, so I decided to update the topic.
The whole thing works quite simply out of the box:
1. Install from backports using apt-get (prefer debian)
2. The config is written. Slightly corrected settings proxied applications
3. Switches iptables to new proxy
On the second point I will focus in more detail.
Configured in the defaults section
The last item is needed so that the host name is passed to the bend, the rest are “like all”.
Next, create frontends for 80 and 443 ports, which will listen and decide on which backend to send a request, depending on certain conditions. And I have only one condition - the incoming host name.
With https is somewhat more complicated. A neighboring topic came to the rescue, in the comments of which they recommended using SNI. He used it
It turned out to be very simple! First of all, certificates are generated for all backends - they will be given to customers. I use PKI from Microsoft, so I had to tinker a little with generating requests, issuing and transferring these certificates to the proxy. By the way, it is allowed to use * .company.name, but I decided that it was somehow not very solid, especially with such a small number of backends. After the certificates are ready, you need to write them stupidly in the line as in the example above, and then write the rules for backends - the certificates will be pushed in order.
The construction using sni is so simple that you don’t even need to explain. True, it turned out that most android mail clients do not know how (or do not want) sni, and send requests to port 443 without specifying the host name. No problem! For such cases there is
(I, by the way, did not check which certificate slips in this case)
Well, then described bekandy. With http, everything is trivial:
Here it.company.name is the host name that will be transmitted to srv-web-01. I need this by the fact that IIS on this server uses authentication by the host name.
For https, these are the constructions.
Here you can unload SSL by specifying port 80 - then the traffic between the client and the proxy will be encrypted, but it will not be inside the network. And you can continue to use https ( verify none means "not finding fault with the certificate"). However, it should be understood that the client still receives the certificate that we entered when creating the frontend. If you need to shove the certificate of the destination server, then you can use the method described in the topic, which I have indicated above.
One more thing: I want to beautifully redirect http to https for some servers. For this, I created special backends with the _r postfix, which neatly threw the unsuspecting user onto https.
I did not deliberately remove the commented line - this option was originally used, but it redirected me to the root of the site, which is very inconvenient if the user clicked on the long link http: //site.company.name/lib/doc/%20 letter of 2020% 20.Name.docx, and it was thrown onto the main page without any hope of finding your document. Chances are good that he will close it and try to follow the link again, but again he will not get anything and will be very upset. To prevent this from happening, we are helped by the construction of the redirect scheme https , which neatly redirects the user, substituting the entire URL.
Details on all the subtleties of configuration on the documentation page cbonte.imtqy.com/haproxy-dconv/configuration-1.5.html#4.2
Thanks for attention. I hope someone my experience will be useful.
The fact is that we use IIS as a proxy for other web servers of the company. I will tell you how this is implemented and what difficulties have been encountered.
Formulation of the problem
Let us analyze the example of the YouTrack server. It is represented by unsightly srv-youtrack-01.local.domain and is located on a web server inside the company. The task is to provide access to it from the Internet using the beautiful name yt.company.ru. In this case, https must be used.
Implementation
To get started, we need to install the URL Rewrite component. This can be done using the web platform installer, as well as manually . After installing it, we’ll see a new shortcut in IIS Manager.
URL redefinition ".
With this tool, you can create a reverse proxy address rewrite rule.
')
When creating a rule, you need to specify the server URL (without the http: // prefix - IIS will add it automatically) to which proxying will occur. As a result, we get a rule that is available for editing. It does not apply to all requests, but only to those that fit the criteria that we can customize. To begin with, the URL is checked for compliance with the pattern, after which other criteria are checked.
I’ll just make a reservation that there are two ways: the first way is to create a set of rules with different URL patterns for different resources on the same IIS site; and the second is to create a site for each proxied resource and in each of them make one rule. Understanding that the first path is more Jedi, I nevertheless chose the path of the second - albeit not so beautiful, but I did not venture to write the wrong regular expression for one site to break all routing. Therefore, the URL pattern I have everywhere default "(. *)".
So, I create a website yt.company.ru with binders for port 80 and port 443 and a mandatory indication of the host name so that IIS knows which site I am accessing. About obtaining and installing a certificate for 443 I allow myself not to mention. I’ll only note that you don’t need to configure the service to use https - inside the network, no one will be encrypted, and external requests will be connected via ssl to the border server, which will already proxy requests inside the network via an unprotected channel.
While the mandatory requirement is the use of https, we will only proxy those requests that come to port 443, for which we will create a simple condition. When you create it, a drop-down list of possible options is offered.
Great, now all yt.company.ru requests are proxied to the internal server with the unsightly name srv-youtrack-01.local.domain transparent to the user.
However, all requests yt.company.ru cut off with an error of 403, which is not very beautiful. To solve this problem, you can either create index.html with a redirect, or another URL Rewrite rule, in which in the "action" field we select a permanent redirect to the URL we need.
It should be noted that the rules for the site are applied in order, so you must first place the rule with the condition, and then the rule without conditions. At the same time, since the second rule applies to all URLs without exception, for the first rule it is necessary to tick (leave checked) “Stop processing subsequent rules”.
After manipulating the graphical interface, a web.config is created in the root of our site, which contains all the rules created. Therefore, if you need to proxy some other site, you do not need to repeat these manipulations, you can simply copy the web.config and change the required URL in it or, after copying, use the graphical interface to change the rules. Moreover, it is possible not to use the interface at all, but to write right there - as you like.
Underwater rocks
When you click the Agile Boards tab, YouTrack generates a URL like yt.company.ru/rest/agile/Overview-0/sprint/Iteration+24 . Further, when switching between sprints yt.company.ru/rest/agile/Overview-0/sprint/Iteration%252023?q= . When switching to these URLs, IIS began to return me a 404 error. This indicated that requests were not being proxied. In this case, the transitions between the saved queries of the form yt.company.ru/issues/IT?q=%23 {Current + work} + Assigned + to% 3A + me + updated% 3A + {This + week} were completely correctly worked out.
Experiments with the addition of a question mark in the middle of the problematic URL ended up in that I began to receive a 404 error already from the YouTrack server, and not IIS. This gave me the idea that IIS interprets the URL for some reason (hello, Microsoft) and this needs to be fixed.
The problem with the plus sign in the middle of the address was solved by adding the requestFiltering parameter to allowDoubleEscaping = “true” :
<system.webServer> <security> <requestFiltering allowDoubleEscaping="true" /> </security> </system.webServer> But after that, switching between sprints still didn't work. It turned out that IIS considers such requests unsafe. This check also had to be disabled:
<system.web> <httpRuntime requestPathInvalidCharacters="" /> </system.web> This is how web.config turned out after all the manipulations:
<?xml version="1.0" encoding="UTF-8"?> <configuration> <system.webServer> <rewrite> <rules> <rule name="ProxyToYouTrack" patternSyntax="ECMAScript" stopProcessing="true"> <match url="(.*)" negate="false" /> <action type="Rewrite" url="http://srv-youtrack-01.local.domain/{R:1}" appendQueryString="true" logRewrittenUrl="true" /> <conditions> <add input="{SERVER_PORT}" pattern="443" /> </conditions> </rule> <rule name="redir to ssl" enabled="true" stopProcessing="true"> <match url="(.*)" /> <action type="Redirect" url="https://yt.company.ru" /> </rule> </rules> <outboundRules> <preConditions> <preCondition name="ResponseIsHtml1"> <add input="{RESPONSE_CONTENT_TYPE}" pattern="^text/html" /> </preCondition> </preConditions> </outboundRules> </rewrite> <security> <requestFiltering allowDoubleEscaping="true" /> </security> </system.webServer> <system.web> <httpRuntime requestPathInvalidCharacters="" /> </system.web> </configuration> Results
Probably, the solutions that I found are not optimal, and instead of resolving everything in a row, it was necessary to carefully write down the rules appropriate in a particular case. But now it works. I will be glad to hear your thoughts and suggestions.
Thus, in our organization, absolutely all web servers that need external access are proxied, among which are nginx, and apache, and svn, and gitlab, and exchange web access.
The main problem that makes me look for a solution is that NTLM authentication does not work through a proxy, which is so necessary for many Microsoft services. I don’t want to use the dead TMG product, so now I’m trying to figure out the new Windows Server 2012 R2 service called Web Application Proxy while simultaneously glancing at nginx and apache, which, it seems, are not yet able to proxy NTLM either.
Links
http://www.ifinity.com.au/Blog/EntryId/60/404-Error-in-IIS-7-when-using-a-Url-with-a-plus-sign-in-the-path
stackoverflow.com/questions/2831142/asp-net-4-url-limitations-why-url-cannot-contain-any-3f-characters
Big update:
In the comments I was advised to try haproxy. Logging on to the site, I made a search on the “ntlm” page and found “full HTTP”, which gave me confidence.
After a few days of active fuss in the console, I still mastered this wonderful tool and now I no longer need IIS as a proxy server. I think it’s not worth writing a separate article about it, so I decided to update the topic.
The whole thing works quite simply out of the box:
1. Install from backports using apt-get (prefer debian)
2. The config is written. Slightly corrected settings proxied applications
3. Switches iptables to new proxy
On the second point I will focus in more detail.
Configured in the defaults section
mode http balance roundrobin option redispatch http-send-name-header Host The last item is needed so that the host name is passed to the bend, the rest are “like all”.
Next, create frontends for 80 and 443 ports, which will listen and decide on which backend to send a request, depending on certain conditions. And I have only one condition - the incoming host name.
frontend http bind *:80 #Define hosts acl host_yt hdr(host) -i yt.company.name acl host_ar hdr(host) -i ar.company.name acl host_portal hdr(host) -i portal.company.name acl host_crm hdr(host) -i crm.company.name acl host_git hdr(host) -i git.company.name acl host_mail hdr(host) -i mail.company.name ... use_backend yt if host_yt use_backend ar if host_ar use_backend crm_r if host_crm use_backend git_r if host_git use_backend mail_r if host_mail With https is somewhat more complicated. A neighboring topic came to the rescue, in the comments of which they recommended using SNI. He used it
frontend https bind *:443 ssl crt /etc/ssl/tfs.cer.ipk.pem crt /etc/ssl/yt.cer.ipk.pem crt /etc/ssl/crm.cer.ipk.pem crt /etc/ssl/git.cer.ipk.pem crt /etc/ssl/mail.cer.ipk.pem use_backend tfs if { ssl_fc_sni tfs.company.name } use_backend yt if { ssl_fc_sni yt.company.name } use_backend crm if { ssl_fc_sni crm.company.name } use_backend git if { ssl_fc_sni git.company.name } use_backend mail if { ssl_fc_sni mail.company.name } It turned out to be very simple! First of all, certificates are generated for all backends - they will be given to customers. I use PKI from Microsoft, so I had to tinker a little with generating requests, issuing and transferring these certificates to the proxy. By the way, it is allowed to use * .company.name, but I decided that it was somehow not very solid, especially with such a small number of backends. After the certificates are ready, you need to write them stupidly in the line as in the example above, and then write the rules for backends - the certificates will be pushed in order.
The construction using sni is so simple that you don’t even need to explain. True, it turned out that most android mail clients do not know how (or do not want) sni, and send requests to port 443 without specifying the host name. No problem! For such cases there is
default_backend mail (I, by the way, did not check which certificate slips in this case)
Well, then described bekandy. With http, everything is trivial:
backend it server it.company.name srv-web-01 backend ar server ar.company.name srv-web-01 Here it.company.name is the host name that will be transmitted to srv-web-01. I need this by the fact that IIS on this server uses authentication by the host name.
For https, these are the constructions.
backend yt server yt.company.name srv-youtrack-01:80 backend tfs server tfs.company.name tfs:443 ssl verify none Here you can unload SSL by specifying port 80 - then the traffic between the client and the proxy will be encrypted, but it will not be inside the network. And you can continue to use https ( verify none means "not finding fault with the certificate"). However, it should be understood that the client still receives the certificate that we entered when creating the frontend. If you need to shove the certificate of the destination server, then you can use the method described in the topic, which I have indicated above.
One more thing: I want to beautifully redirect http to https for some servers. For this, I created special backends with the _r postfix, which neatly threw the unsuspecting user onto https.
backend tfs_r #redirect location https://tfs.company.name code 301 redirect scheme https I did not deliberately remove the commented line - this option was originally used, but it redirected me to the root of the site, which is very inconvenient if the user clicked on the long link http: //site.company.name/lib/doc/%20 letter of 2020% 20.Name.docx, and it was thrown onto the main page without any hope of finding your document. Chances are good that he will close it and try to follow the link again, but again he will not get anything and will be very upset. To prevent this from happening, we are helped by the construction of the redirect scheme https , which neatly redirects the user, substituting the entire URL.
Details on all the subtleties of configuration on the documentation page cbonte.imtqy.com/haproxy-dconv/configuration-1.5.html#4.2
Thanks for attention. I hope someone my experience will be useful.
Source: https://habr.com/ru/post/240495/
All Articles