Validation of ASP.NET requests: from“The ability to validate queries in ASP.NET is designed to perform basic input control. It is not intended to make decisions regarding the security of the developed web applications. Only the developers themselves can determine which content should process their code. Microsoft recommends checking all input data from any source. We strive to contribute to the development of secure applications by our customers, and the query validation functionality has been designed and implemented to help developers in this direction. For more information about our design recommendations, read the MSDN article: msdn.microsoft.com/en-us/library/ff649487.aspx#pagguidelines0001_inputdatavalidation . ”
Official status of the ASP.NET Request Validation under the Microsoft Security Response Center
Despite the sudden response of MSRC to the recent report by Quotium Research Center on the discovery of another way to bypass the validation of requests in ASP.NET, it is worth noting that it is designed specifically for making decisions regarding the security of a web application. This is supported by the name of the class that implements the main set of checks (System.Web.CrossSiteScriptingValidation) and its very essence, which is to prevent some subset of XSS Type-1 class attacks (reflected cross-site scripts), and the original article from the web stack developers . Another question is how effectively this functionality could be implemented and how to get a full-fledged web application firewall from the existing primitive regular filter protecting against any XSS Type-1 vectors?
To answer this question, it is necessary to understand the implementation details of query validation in different versions of the .NET Framework, its limitations, known workarounds, and the possibilities of extending its functionality.
1. The Evolution of ASP.NET Query Validation
In all versions of ASP.NET (coinciding with versions of the .NET Framework), starting with v1.1 and ending with v4.5, request validation is reduced to searching for various elements of the HTTP request for occurrences of regular set chains, which describes the black list of dangerous values. From the point of view of coding, it is performed by a recognizing automaton implemented for performance reasons manually, without the use of standard regular expressions. The set of dangerous values contains elements of the HTML language, which can violate the integrity of the output document, if used in it without sufficient preprocessing.
')
For the first time, the request validation mechanism was implemented in ASP.NET v1.1 and used a fairly wide black list. The query processing was blocked if any parameters of the query string or form field values matched any of the regular expressions:
- <? i [az! /]
- (? i: script) \ s? \:
- (? i: on [az]) * \ s * =
- (? i: ex) pression \ (
There is nothing surprising in the fact that developers preferred to completely disable this feature due to the large number of false positives. Therefore, in ASP.NET v2.0, the set of dangerous values was greatly reduced and reached v4.5 in an unchanged form:
- # &
- <? i [az! / \?]
At the stage of preparing ASP.NET v4.0, the developers also stated that the set (? I: script) \ s ?: will be returned to the list, but this did not happen either in v4.0 or v4.5.
From version to version, not only the set of dangerous values was changed, but also the area of validation and the possibilities for developers to control this process. So, in v2.0, it became possible to disable the validation of requests for individual pages, and in v4.0 a new so-called mode was introduced. deferred granular validation, in which each of the parameters is checked when accessing it from the code of the web application, and not at the stage of preliminary processing of the request. Starting with v4.0, in addition to the parameters of the query string and form field values, the validation area also includes
- values of all elements from Request.Cookies;
- names of downloaded files from Request.Files;
- Request.RawUrl, Request.Path, and Request.PathInfo values
2. Validation of requests in ASP.NET v4.x
In the latest versions of ASP.NET, as part of the validation of the request, a number of additional checks are also performed, which are performed at the earliest stages of its life cycle. Their complete list is given in the table:
CHECK SETTINGS AND VALUES Check Length Request.Path The maxUrlLength attribute in the <httpRuntime> section. It can be defined globally for the entire application, or for individual virtual paths or pages.
Blocks the processing of an HTTP request containing a path longer than 260 characters. This value can be increased to the limits defined in IIS or http.sys . Checking the length of the Request.RawUrl fragment containing the query string The maxQueryStringLength attribute in the <httpRuntime> section. It can be defined globally for the entire application, or for individual virtual paths or pages.
Blocks the processing of an HTTP request containing a query string longer than 2048 characters. This value can be increased to the limits, IIS or http.sys . Scanning Request.Path for the presence of characters defined in ASP.NET as potentially dangerous The requestPathInvalidCharacters attribute in the <httpRuntime> section. It can be defined globally for the entire application, or for individual virtual paths or pages.
Blocks the processing of an HTTP request if the path in it contains any of the characters:
- <(XSS attack)
- > (XSS attacks)
- * (attacks on the canonization of file names)
- % (URL decoder attacks)
- : (attacks on alternative NTFS data streams)
- & (query string parser attacks)
- \ (attacks on canonization of file paths)
- ? (attack on parser query string)
In the requestPathInvalidCharacters attribute, forbidden characters are enclosed in double quotes and are separated by commas.
The sequence of path characters "\ .." is not included in this list due to the fact that IIS v6 + canonizes URIs automatically, correctly processing such sequences. In practice, errors associated with the appearance of the forward slash characters in the path also do not occur, since in the process of canonization, they are replaced by reverse ones. Finding the right managed configuration for each Request.Path The relaxedUrlToFileSystemMapping attribute in the <httpRuntime> section. It can only be defined globally for the entire application.
By default, this attribute is set to false, which tells ASP.NET to treat the component path in the URL as a valid file path that conforms to NTFS rules. This restriction can be disabled by setting the attribute value to true. Checking Request.QueryString, Request.Form, Request.Files, Request.Cookies, Request.Path, Request.PathInfo, Request.RawUrl for potentially dangerous values The requestValidationMode attribute in the <httpRuntime> section. It can be defined globally for the entire application, or for individual virtual paths or pages.
Sets the mode in which requests for the web application will be validated. A value of 4.0 (default) includes a deferred granular validation, which is performed with the direct access of the web application code to the elements from the validation area. Setting this attribute to 2.0 returns the validation mode used in previous versions of ASP.NET.
The requestValidationType attribute in the <httpRuntime> section. It can only be defined globally for the entire application.
Sets the type of the successor to the RequestValidator class that implements the query validation functionality. The default class is System.Web.Util.RequestValidator.
The last check is exactly that visible part of the iceberg, called request validation, and available to web application developers to extend its functionality.
3. Internal device validation requests
The source code of the IsValidRequestString method of the System.Web.Util.RequestValidator class, which is used by default for validating requests in ASP.NET v2.0 +, looks like this:
protected internal virtual bool IsValidRequestString( HttpContext context, string value, RequestValidationSource requestValidationSource, string collectionKey, out int validationFailureIndex) { if (requestValidationSource == RequestValidationSource.Headers) { validationFailureIndex = 0; return true; } return !CrossSiteScriptingValidation.IsDangerousString(value, out validationFailureIndex); }
It should be noted that before calling the IsValidRequestString method from the string passed in the value parameter, all occurrences of the zero byte are cut. This behavior is implemented in the ValidateString method of the HttpRequest class and cannot be overridden by the developer.
As can be seen from the source code, the basic functionality of request validation is implemented in the IsDangerousString method of the CrossSiteScriptingValidation class:
internal static bool IsDangerousString(string s, out int matchIndex) { matchIndex = 0; int startIndex = 0; while (true) { int num2 = s.IndexOfAny(startingChars, startIndex); if (num2 < 0) { return false; } if (num2 == (s.Length - 1)) { return false; } matchIndex = num2; char ch = s[num2]; if (ch != '&') { if ((ch == '<') && ((IsAtoZ(s[num2 + 1]) || (s[num2 + 1] == '!')) || ((s[num2 + 1] == '/') || (s[num2 + 1] == '?')))) { return true; } } else if (s[num2 + 1] == '#') { return true; } startIndex = num2 + 1; } }
Obviously, this filter is an automaton that recognizes in the transmitted string the occurrence of chains of a regular set <? I [az! / \?] | & #. In addition, the CrossSiteScriptingValidation class also defines two auxiliary methods that are not available for extending or modifying their functionality:
internal static bool IsDangerousUrl(string s) { if (string.IsNullOrEmpty(s)) { return false; } s = s.Trim(); int length = s.Length; if (((((length > 4) && ((s[0] == 'h') || (s[0] == 'H'))) && ((s[1] == 't') || (s[1] == 'T'))) && (((s[2] == 't') || (s[2] == 'T')) && ((s[3] == 'p') || (s[3] == 'P')))) && ((s[4] == ':') || (((length > 5) && ((s[4] == 's') || (s[4] == 'S'))) && (s[5] == ':')))) { return false; } if (s.IndexOf(':') == -1) { return false; } return true; } internal static bool IsValidJavascriptId(string id) { if (!string.IsNullOrEmpty(id)) { return CodeGenerator.IsValidLanguageIndependentIdentifier(id); } return true; }
The first checks the URL and considers dangerous all values that do not satisfy the set ^ (? I: https? :) | [^:]. The second checks the value of the argument against the grammar rule for language identifiers: ^ (? I: [a-z _] [a-z0-9 _]) $. Both methods were called from IsDangerousString as part of the validation of requests in ASP.NET v1.1. In all other versions, they are used only in some ASP.NET WebForms controls as functional verification methods and do not raise a RequestValidException exception.
4. Disadvantages of the standard implementation and ways to circumvent it
Obviously, the standard implementation of the validation of requests has a number of drawbacks that make it really unsuitable for making decisions regarding the security of a web application.
First, the reviewed checks can only protect against a limited subset of XSS Type-1 class attacks that require opening a tag to conduct them. In the event that an attack of a reflected XSS is possible as a result of the implementation of the parameter values inside the tag, attribute, or code of the client script, standard validation of requests will no longer be able to prevent it.
Secondly, blacklist control, by itself, is not a sufficient measure to ensure security. This is due to the presence of several well-known ways to bypass the standard query validation:
- The limitation on <? I [az! / \?] Can be bypassed using the percent sign between the opening angle bracket and the tag name (<% img src = # onerror = alert (1) />). In this case, the IE v9-HTML parser will consider this the correct tag definition. In some cases, if the web application implements Unicode-canonization of request parameters, it is also possible to bypass using Unicode-wide values (% uff1c img% 20src% 3D% 23% 20onerror% 3Dalert% 281% 29% 2f% uff1e).
- Restriction on (? I: script) \ s? \: And (? I: ex) pression \ (costs using white space characters inside the script and between the expression and the opening bracket (java% 09script: alert (1) and expression% 09 ( alert (1))).
- The restriction on # & does not take into account the existence of named references to HTML entities that can also be used in a number of vectors (javascript% 26Tab;: alert (1)). It should also be noted here that the standard implementation of the ASP.NET HTML decoder (HttpUtility.HtmlDecode) “knows” only about the existence of 253 named references to HTML entities, while in the HTML standard there are significantly more of them . This allows multiple HTML entities to be forwarded to the output document, even if the web application performs HTML decoding of the parameter values during the preprocessing of input data.
But the main drawback of the standard implementation is the request processing stage, at which it is validated. Even with the delayed mode enabled, without having information about the contents of an already generated response document, it is impossible to make correct assumptions about the danger of a particular parameter for a particular class of attacks. For example, if a parameter containing HTML markup elements does not fall into the server’s response, it is rather strange to argue about its potential danger from the point of view of XSS Type-1. This is about the same as stating the dangers of a “SELECT” value without having information about whether it ends up in the SQL query. Following this logic, ASP.NET developers would also need to include in its request validation the search in its parameters for elements of SQL syntax, paths, XPath expressions and other character sequences that are typical for injections into any languages, and not limit yourself to only a small subset of attacks XSS specific type. Of course, this approach generates a lot of false positives, which leads to both a complete shutdown of validation for the entire application, and the appearance of tools that allow you to do this without much effort (for example, nuget.org/packages/DisableRequestValidation ).
However, all these shortcomings can be eliminated by taking advantage of the opportunity discussed in the next section.
5. Extension of query validation functionality
Starting with ASP.NET v4.0, developers have the opportunity to extend the functionality of query validation, incl. completely overriding the standard implementation. In order to accomplish this, it is enough to create a descendant of the class System.Web.Util.RequestValidator by redefining the IsValidRequestString method in it. This method is called when it is necessary to check the next query parameter and takes the following arguments:
- HttpContext context - the context of the HTTP request, within which the check is performed;
- string value - the value to be checked;
- RequestValidationSource requestValidationSource - the source to which the value to be checked belongs;
- string collectionKey - the name of the checked value in the source;
- out int validationFailureIndex - output parameter containing an offset within value from which the dangerous symbol was detected or -1 otherwise;
For example, to eliminate the possibility of bypassing validation using the <% character combination, you can implement the following extension:
using System; using System.Web; using System.Web.Util; namespace RequestValidationExtension { public class BypassRequestValidator : RequestValidator { public BypassRequestValidator() { } protected override bool IsValidRequestString( HttpContext context, string value, RequestValidationSource requestValidationSource, string collectionKey, out int validationFailureIndex) { validationFailureIndex = -1; for (var i = 0; i < value.Length; i++) { if (value[i] == '<' && (i < value.Length - 1) && value[i + 1] == '%') { validationFailureIndex = i; return false; } } return base.IsValidRequestString( context, value, requestValidationSource, collectionKey, out validationFailureIndex); } } }
After that, by setting the value of the requestValidationType = "RequestValidationExtension.BypassRequestValidator" attribute in the httpRuntime section of the web application configuration, get the web application protected from this validation method.
6. Improved validation of requests
Using the ability to expand the functionality of query validation, it is realistic to eliminate all existing problems of the current implementation and get a full-fledged XSS Type-1 WAF. To do this, it is necessary to check the values of the request parameters immediately before sending a response when it is already formed and its contents are known. This is a necessary condition for a reliable assessment of the impact of request parameters on the integrity of the response. However, the ASP.NET query validation architecture does not provide the ability to execute it within its own framework, which makes it necessary to split the whole procedure into three steps.
The functionality of the first stage is implemented directly in the expansion of the standard query validation. At this stage, each checked parameter is mapped to the set ^? I [A-Za-z0-9 _] + $ and, in the event that the mapping failed, the parameter is marked as to be tested later. Thus, information is collected about all potentially dangerous request parameters for which validation was requested (that is, the parameters actually used in the web application when processing the request). This provides full integration with the existing query validation architecture, and also eliminates the need to expose non-dangerous parameters to additional checks.
The second stage is an ASP.NET output filter and implements observation of the stream to which all response fragments generated at different stages of the request life cycle are recorded. The content of the entire response is also saved for processing in the third stage.
The third stage, implemented as an HTTP module that processes the EndRequest event, evaluates the effect of all parameters collected in the first stage on the integrity of the response received in the second. In case of detection of violation of the integrity of the response, the check is considered failed. The evaluation of the effect of the integrity request parameters is based on a fuzzy search in the response of the so-called insertion points - places in which the response fragment approximately coincides with the value of any of the parameters. The set of insertion points forms an insertion map, using which one can carry out a much more substantiated check for the presence of forbidden characters in the parameter values, and also reveal the nature of their influence on the integrity of the response.
The last problem is solved using parsers of all languages that may occur in the response (HTML, JavaScript, CSS). Comparison of parse trees obtained as a result of parsing various fragments of the answer with the data from the insert map gives full information about which of the nodes of one or another tree were embedded in the response with the value of one or another parameter.
Detailed description of the verification algorithm- If the output document contains zero bytes, then the check is failed.
- For each element of the parameter list P, a fuzzy search is performed with a threshold of 0.75 of all occurrences of its value in the text of the answer R. The boundaries of each found entry determine the insertion area. A set of all insertion areas map inserts M.
- If M is empty, then the check is considered passed.
- For each element M, a check is performed to ensure that its value matches the regular set <? I [az! / \?%]. If a match is found, the check is failed.
- The output text R is parsed by the HTML parser into the tree R '.
- If, as a result of the parsing, any errors occurred and their places of intersection with the elements of M, then the check failed.
- All subsequent steps are repeated for each node N of the tree R ', describing the HTML tag or comment.
- If the initial position N in R has intersections with elements of M, then the check is considered failed.
- If N describes an HTML tag, then for each of its attributes, the position of which in R has an intersection with the elements of M, the verification is carried out according to the algorithm described below.
- If N describes a tag, then for its value innerText (script code), the check is performed using the algorithm described below.
- If N describes a tag, then for its innerText value (style definition code), the algorithm is tested using the algorithm described below.
- A check is considered passed if it was not failed in the previous steps.
The algorithm for checking the attributes of HTML elements (accepts the value of the attribute A, the insertion map M and the response text R):
- If the initial position A in R has an intersection with the elements of M, then the check is considered failed.
- If the position of the value of A in R does not intersect with the elements of M, then the check is considered to be passed.
- If the name A is contained in the list of event handler attributes, then its value is checked according to the algorithm described below.
- If the name A matches an element of the attribute type list of the reference type, then the following steps are taken:
- If the value of A contains a substring "& #" or a named reference to an HTML entity, then the check is considered failed.
- If the value of A does not contain ":", then the check is considered to be passed.
- The value of A is parsed by the URI parser into a U object.
- If errors occurred during parsing, the check is considered failed.
- If U does not describe an absolute path, then the check is considered failed.
- If U describes a path with a scheme that is on the list of dangerous, then the check is considered failed.
- If the name A = "style", then its value is checked according to the algorithm described below.
Algorithm for checking client script code and event handler attribute values (accepts a Vs value containing the code of the script being checked and the value Vm of the element of the set M with which the intersection was detected):
- If the largest common substring of L from Vs and Vm is less than 7, then the check is considered to be passed.
- Vs value is parsed by the JavaScript parser in the Vs' tree
- If errors occurred during parsing, the check is considered failed.
- If the number of tokens in Vs 'is less than 5 or the number of nodes in Vs' is less than 2, then the check is considered to be passed.
- If the entire value of L is the value of one token Vt of the tree Vs', then the check is considered to be passed.
- The JavaScript decoded Vt value is subjected to a recursive check, as if it were the answer text, fully formed from the Vm parameter.
The algorithm for checking the style definition code is absolutely similar to the previous one, with the exception of using the CSS parser and other threshold values for the elements of the parse tree and the largest common substring.
Proof-of-Concept implementation of the described algorithm is available on GitHub . At the time of preparation of the article there are no known ways to bypass this filter. The tests performed showed no tangible impact on the performance of web applications in cases where the request does not contain dangerous values and a 7-15% slowdown in the formation of a response otherwise. Considering the fact that the Proof-of-Concept version uses third-party parsers that solve a much more general task than is required by the response validation algorithm, an optimal implementation of these components will allow to achieve performance sufficient for confident application of the solution in production environments.
7. Conclusions
The implementation of the query validation functional in current versions of ASP.NET is ineffective and does not solve the problem of protection against XSS Type-1 class attacks. Nonetheless, its current architecture and expansion options allow itself to solve this problem using the response validation method described in this article.
However, we should not forget that the most effective protection against such attacks are not third-party mounted solutions, but the correct implementation of processing input and output data by the developers themselves. And using Irv or more complex products (such as mod-security for IIS or .NetIDS ) does not save a developer from having to follow basic rules for developing secure code (for example, www.troyhunt.com/2011/12/free-ebook-owasp-top -10-for-net.html or wiki.mozilla.org/WebAppSec/Secure_Coding_Guidelines ).
“The ability to validate queries in ASP.NET is designed to perform basic input control. It is not intended to make decisions regarding the security of the developed web applications. Only the developers themselves can determine which content should process their code. Microsoft recommends checking all input data from any source. We strive to contribute to the development of secure applications by our customers, and the query validation functionality has been designed and implemented to help developers in this direction. For more information about our design recommendations, read the MSDN article: msdn.microsoft.com/en-us/library/ff649487.aspx#pagguidelines0001_inputdatavalidation . ”
Official status of the ASP.NET Request Validation under the Microsoft Security Response Center
Despite the sudden response of MSRC to the recent report by Quotium Research Center on the discovery of another way to bypass the validation of requests in ASP.NET, it is worth noting that it is designed specifically for making decisions regarding the security of a web application. This is supported by the name of the class that implements the main set of checks (System.Web.CrossSiteScriptingValidation) and its very essence, which is to prevent some subset of XSS Type-1 class attacks (reflected cross-site scripts), and the original article from the web stack developers . Another question is how effectively this functionality could be implemented and how to get a full-fledged web application firewall from the existing primitive regular filter protecting against any XSS Type-1 vectors?
To answer this question, it is necessary to understand the implementation details of query validation in different versions of the .NET Framework, its limitations, known workarounds, and the possibilities of extending its functionality.
1. The Evolution of ASP.NET Query Validation
In all versions of ASP.NET (coinciding with versions of the .NET Framework), starting with v1.1 and ending with v4.5, request validation is reduced to searching for various elements of the HTTP request for occurrences of regular set chains, which describes the black list of dangerous values. From the point of view of coding, it is performed by a recognizing automaton implemented for performance reasons manually, without the use of standard regular expressions. The set of dangerous values contains elements of the HTML language, which can violate the integrity of the output document, if used in it without sufficient preprocessing.
')
For the first time, the request validation mechanism was implemented in ASP.NET v1.1 and used a fairly wide black list. The query processing was blocked if any parameters of the query string or form field values matched any of the regular expressions:
- <? i [az! /]
- (? i: script) \ s? \:
- (? i: on [az]) * \ s * =
- (? i: ex) pression \ (
There is nothing surprising in the fact that developers preferred to completely disable this feature due to the large number of false positives. Therefore, in ASP.NET v2.0, the set of dangerous values was greatly reduced and reached v4.5 in an unchanged form:
- # &
- <? i [az! / \?]
At the stage of preparing ASP.NET v4.0, the developers also stated that the set (? I: script) \ s ?: will be returned to the list, but this did not happen either in v4.0 or v4.5.
From version to version, not only the set of dangerous values was changed, but also the area of validation and the possibilities for developers to control this process. So, in v2.0, it became possible to disable the validation of requests for individual pages, and in v4.0 a new so-called mode was introduced. deferred granular validation, in which each of the parameters is checked when accessing it from the code of the web application, and not at the stage of preliminary processing of the request. Starting with v4.0, in addition to the parameters of the query string and form field values, the validation area also includes
- values of all elements from Request.Cookies;
- names of downloaded files from Request.Files;
- Request.RawUrl, Request.Path, and Request.PathInfo values
2. Validation of requests in ASP.NET v4.x
In the latest versions of ASP.NET, as part of the validation of the request, a number of additional checks are also performed, which are performed at the earliest stages of its life cycle. Their complete list is given in the table:
| CHECK | SETTINGS AND VALUES |
|---|---|
| The maxUrlLength attribute in the <httpRuntime> section. It can be defined globally for the entire application, or for individual virtual paths or pages. Blocks the processing of an HTTP request containing a path longer than 260 characters. This value can be increased to the limits defined in IIS or http.sys . | |
| Checking the length of the Request.RawUrl fragment containing the query string | The maxQueryStringLength attribute in the <httpRuntime> section. It can be defined globally for the entire application, or for individual virtual paths or pages. Blocks the processing of an HTTP request containing a query string longer than 2048 characters. This value can be increased to the limits, IIS or http.sys . |
| Scanning Request.Path for the presence of characters defined in ASP.NET as potentially dangerous | The requestPathInvalidCharacters attribute in the <httpRuntime> section. It can be defined globally for the entire application, or for individual virtual paths or pages. Blocks the processing of an HTTP request if the path in it contains any of the characters:
In the requestPathInvalidCharacters attribute, forbidden characters are enclosed in double quotes and are separated by commas. The sequence of path characters "\ .." is not included in this list due to the fact that IIS v6 + canonizes URIs automatically, correctly processing such sequences. In practice, errors associated with the appearance of the forward slash characters in the path also do not occur, since in the process of canonization, they are replaced by reverse ones. |
| Finding the right managed configuration for each Request.Path | The relaxedUrlToFileSystemMapping attribute in the <httpRuntime> section. It can only be defined globally for the entire application. By default, this attribute is set to false, which tells ASP.NET to treat the component path in the URL as a valid file path that conforms to NTFS rules. This restriction can be disabled by setting the attribute value to true. |
| Checking Request.QueryString, Request.Form, Request.Files, Request.Cookies, Request.Path, Request.PathInfo, Request.RawUrl for potentially dangerous values | The requestValidationMode attribute in the <httpRuntime> section. It can be defined globally for the entire application, or for individual virtual paths or pages. Sets the mode in which requests for the web application will be validated. A value of 4.0 (default) includes a deferred granular validation, which is performed with the direct access of the web application code to the elements from the validation area. Setting this attribute to 2.0 returns the validation mode used in previous versions of ASP.NET. The requestValidationType attribute in the <httpRuntime> section. It can only be defined globally for the entire application. Sets the type of the successor to the RequestValidator class that implements the query validation functionality. The default class is System.Web.Util.RequestValidator. |
3. Internal device validation requests
The source code of the IsValidRequestString method of the System.Web.Util.RequestValidator class, which is used by default for validating requests in ASP.NET v2.0 +, looks like this:
protected internal virtual bool IsValidRequestString( HttpContext context, string value, RequestValidationSource requestValidationSource, string collectionKey, out int validationFailureIndex) { if (requestValidationSource == RequestValidationSource.Headers) { validationFailureIndex = 0; return true; } return !CrossSiteScriptingValidation.IsDangerousString(value, out validationFailureIndex); } It should be noted that before calling the IsValidRequestString method from the string passed in the value parameter, all occurrences of the zero byte are cut. This behavior is implemented in the ValidateString method of the HttpRequest class and cannot be overridden by the developer.
As can be seen from the source code, the basic functionality of request validation is implemented in the IsDangerousString method of the CrossSiteScriptingValidation class:
internal static bool IsDangerousString(string s, out int matchIndex) { matchIndex = 0; int startIndex = 0; while (true) { int num2 = s.IndexOfAny(startingChars, startIndex); if (num2 < 0) { return false; } if (num2 == (s.Length - 1)) { return false; } matchIndex = num2; char ch = s[num2]; if (ch != '&') { if ((ch == '<') && ((IsAtoZ(s[num2 + 1]) || (s[num2 + 1] == '!')) || ((s[num2 + 1] == '/') || (s[num2 + 1] == '?')))) { return true; } } else if (s[num2 + 1] == '#') { return true; } startIndex = num2 + 1; } } Obviously, this filter is an automaton that recognizes in the transmitted string the occurrence of chains of a regular set <? I [az! / \?] | & #. In addition, the CrossSiteScriptingValidation class also defines two auxiliary methods that are not available for extending or modifying their functionality:
internal static bool IsDangerousUrl(string s) { if (string.IsNullOrEmpty(s)) { return false; } s = s.Trim(); int length = s.Length; if (((((length > 4) && ((s[0] == 'h') || (s[0] == 'H'))) && ((s[1] == 't') || (s[1] == 'T'))) && (((s[2] == 't') || (s[2] == 'T')) && ((s[3] == 'p') || (s[3] == 'P')))) && ((s[4] == ':') || (((length > 5) && ((s[4] == 's') || (s[4] == 'S'))) && (s[5] == ':')))) { return false; } if (s.IndexOf(':') == -1) { return false; } return true; } internal static bool IsValidJavascriptId(string id) { if (!string.IsNullOrEmpty(id)) { return CodeGenerator.IsValidLanguageIndependentIdentifier(id); } return true; } The first checks the URL and considers dangerous all values that do not satisfy the set ^ (? I: https? :) | [^:]. The second checks the value of the argument against the grammar rule for language identifiers: ^ (? I: [a-z _] [a-z0-9 _]) $. Both methods were called from IsDangerousString as part of the validation of requests in ASP.NET v1.1. In all other versions, they are used only in some ASP.NET WebForms controls as functional verification methods and do not raise a RequestValidException exception.
4. Disadvantages of the standard implementation and ways to circumvent it
Obviously, the standard implementation of the validation of requests has a number of drawbacks that make it really unsuitable for making decisions regarding the security of a web application.
First, the reviewed checks can only protect against a limited subset of XSS Type-1 class attacks that require opening a tag to conduct them. In the event that an attack of a reflected XSS is possible as a result of the implementation of the parameter values inside the tag, attribute, or code of the client script, standard validation of requests will no longer be able to prevent it.
Secondly, blacklist control, by itself, is not a sufficient measure to ensure security. This is due to the presence of several well-known ways to bypass the standard query validation:
- The limitation on <? I [az! / \?] Can be bypassed using the percent sign between the opening angle bracket and the tag name (<% img src = # onerror = alert (1) />). In this case, the IE v9-HTML parser will consider this the correct tag definition. In some cases, if the web application implements Unicode-canonization of request parameters, it is also possible to bypass using Unicode-wide values (% uff1c img% 20src% 3D% 23% 20onerror% 3Dalert% 281% 29% 2f% uff1e).
- Restriction on (? I: script) \ s? \: And (? I: ex) pression \ (costs using white space characters inside the script and between the expression and the opening bracket (java% 09script: alert (1) and expression% 09 ( alert (1))).
- The restriction on # & does not take into account the existence of named references to HTML entities that can also be used in a number of vectors (javascript% 26Tab;: alert (1)). It should also be noted here that the standard implementation of the ASP.NET HTML decoder (HttpUtility.HtmlDecode) “knows” only about the existence of 253 named references to HTML entities, while in the HTML standard there are significantly more of them . This allows multiple HTML entities to be forwarded to the output document, even if the web application performs HTML decoding of the parameter values during the preprocessing of input data.
But the main drawback of the standard implementation is the request processing stage, at which it is validated. Even with the delayed mode enabled, without having information about the contents of an already generated response document, it is impossible to make correct assumptions about the danger of a particular parameter for a particular class of attacks. For example, if a parameter containing HTML markup elements does not fall into the server’s response, it is rather strange to argue about its potential danger from the point of view of XSS Type-1. This is about the same as stating the dangers of a “SELECT” value without having information about whether it ends up in the SQL query. Following this logic, ASP.NET developers would also need to include in its request validation the search in its parameters for elements of SQL syntax, paths, XPath expressions and other character sequences that are typical for injections into any languages, and not limit yourself to only a small subset of attacks XSS specific type. Of course, this approach generates a lot of false positives, which leads to both a complete shutdown of validation for the entire application, and the appearance of tools that allow you to do this without much effort (for example, nuget.org/packages/DisableRequestValidation ).
However, all these shortcomings can be eliminated by taking advantage of the opportunity discussed in the next section.
5. Extension of query validation functionality
Starting with ASP.NET v4.0, developers have the opportunity to extend the functionality of query validation, incl. completely overriding the standard implementation. In order to accomplish this, it is enough to create a descendant of the class System.Web.Util.RequestValidator by redefining the IsValidRequestString method in it. This method is called when it is necessary to check the next query parameter and takes the following arguments:
- HttpContext context - the context of the HTTP request, within which the check is performed;
- string value - the value to be checked;
- RequestValidationSource requestValidationSource - the source to which the value to be checked belongs;
- string collectionKey - the name of the checked value in the source;
- out int validationFailureIndex - output parameter containing an offset within value from which the dangerous symbol was detected or -1 otherwise;
For example, to eliminate the possibility of bypassing validation using the <% character combination, you can implement the following extension:
using System; using System.Web; using System.Web.Util; namespace RequestValidationExtension { public class BypassRequestValidator : RequestValidator { public BypassRequestValidator() { } protected override bool IsValidRequestString( HttpContext context, string value, RequestValidationSource requestValidationSource, string collectionKey, out int validationFailureIndex) { validationFailureIndex = -1; for (var i = 0; i < value.Length; i++) { if (value[i] == '<' && (i < value.Length - 1) && value[i + 1] == '%') { validationFailureIndex = i; return false; } } return base.IsValidRequestString( context, value, requestValidationSource, collectionKey, out validationFailureIndex); } } } After that, by setting the value of the requestValidationType = "RequestValidationExtension.BypassRequestValidator" attribute in the httpRuntime section of the web application configuration, get the web application protected from this validation method.
6. Improved validation of requests
Using the ability to expand the functionality of query validation, it is realistic to eliminate all existing problems of the current implementation and get a full-fledged XSS Type-1 WAF. To do this, it is necessary to check the values of the request parameters immediately before sending a response when it is already formed and its contents are known. This is a necessary condition for a reliable assessment of the impact of request parameters on the integrity of the response. However, the ASP.NET query validation architecture does not provide the ability to execute it within its own framework, which makes it necessary to split the whole procedure into three steps.
The functionality of the first stage is implemented directly in the expansion of the standard query validation. At this stage, each checked parameter is mapped to the set ^? I [A-Za-z0-9 _] + $ and, in the event that the mapping failed, the parameter is marked as to be tested later. Thus, information is collected about all potentially dangerous request parameters for which validation was requested (that is, the parameters actually used in the web application when processing the request). This provides full integration with the existing query validation architecture, and also eliminates the need to expose non-dangerous parameters to additional checks.
The second stage is an ASP.NET output filter and implements observation of the stream to which all response fragments generated at different stages of the request life cycle are recorded. The content of the entire response is also saved for processing in the third stage.
The third stage, implemented as an HTTP module that processes the EndRequest event, evaluates the effect of all parameters collected in the first stage on the integrity of the response received in the second. In case of detection of violation of the integrity of the response, the check is considered failed. The evaluation of the effect of the integrity request parameters is based on a fuzzy search in the response of the so-called insertion points - places in which the response fragment approximately coincides with the value of any of the parameters. The set of insertion points forms an insertion map, using which one can carry out a much more substantiated check for the presence of forbidden characters in the parameter values, and also reveal the nature of their influence on the integrity of the response.
The last problem is solved using parsers of all languages that may occur in the response (HTML, JavaScript, CSS). Comparison of parse trees obtained as a result of parsing various fragments of the answer with the data from the insert map gives full information about which of the nodes of one or another tree were embedded in the response with the value of one or another parameter.
Detailed description of the verification algorithm
The algorithm for checking the attributes of HTML elements (accepts the value of the attribute A, the insertion map M and the response text R):
Algorithm for checking client script code and event handler attribute values (accepts a Vs value containing the code of the script being checked and the value Vm of the element of the set M with which the intersection was detected):
The algorithm for checking the style definition code is absolutely similar to the previous one, with the exception of using the CSS parser and other threshold values for the elements of the parse tree and the largest common substring.
- If the output document contains zero bytes, then the check is failed.
- For each element of the parameter list P, a fuzzy search is performed with a threshold of 0.75 of all occurrences of its value in the text of the answer R. The boundaries of each found entry determine the insertion area. A set of all insertion areas map inserts M.
- If M is empty, then the check is considered passed.
- For each element M, a check is performed to ensure that its value matches the regular set <? I [az! / \?%]. If a match is found, the check is failed.
- The output text R is parsed by the HTML parser into the tree R '.
- If, as a result of the parsing, any errors occurred and their places of intersection with the elements of M, then the check failed.
- All subsequent steps are repeated for each node N of the tree R ', describing the HTML tag or comment.
- If the initial position N in R has intersections with elements of M, then the check is considered failed.
- If N describes an HTML tag, then for each of its attributes, the position of which in R has an intersection with the elements of M, the verification is carried out according to the algorithm described below.
- If N describes a tag, then for its value innerText (script code), the check is performed using the algorithm described below.
- If N describes a tag, then for its innerText value (style definition code), the algorithm is tested using the algorithm described below.
- A check is considered passed if it was not failed in the previous steps.
The algorithm for checking the attributes of HTML elements (accepts the value of the attribute A, the insertion map M and the response text R):
- If the initial position A in R has an intersection with the elements of M, then the check is considered failed.
- If the position of the value of A in R does not intersect with the elements of M, then the check is considered to be passed.
- If the name A is contained in the list of event handler attributes, then its value is checked according to the algorithm described below.
- If the name A matches an element of the attribute type list of the reference type, then the following steps are taken:
- If the value of A contains a substring "& #" or a named reference to an HTML entity, then the check is considered failed.
- If the value of A does not contain ":", then the check is considered to be passed.
- The value of A is parsed by the URI parser into a U object.
- If errors occurred during parsing, the check is considered failed.
- If U does not describe an absolute path, then the check is considered failed.
- If U describes a path with a scheme that is on the list of dangerous, then the check is considered failed.
- If the name A = "style", then its value is checked according to the algorithm described below.
Algorithm for checking client script code and event handler attribute values (accepts a Vs value containing the code of the script being checked and the value Vm of the element of the set M with which the intersection was detected):
- If the largest common substring of L from Vs and Vm is less than 7, then the check is considered to be passed.
- Vs value is parsed by the JavaScript parser in the Vs' tree
- If errors occurred during parsing, the check is considered failed.
- If the number of tokens in Vs 'is less than 5 or the number of nodes in Vs' is less than 2, then the check is considered to be passed.
- If the entire value of L is the value of one token Vt of the tree Vs', then the check is considered to be passed.
- The JavaScript decoded Vt value is subjected to a recursive check, as if it were the answer text, fully formed from the Vm parameter.
The algorithm for checking the style definition code is absolutely similar to the previous one, with the exception of using the CSS parser and other threshold values for the elements of the parse tree and the largest common substring.
Proof-of-Concept implementation of the described algorithm is available on GitHub . At the time of preparation of the article there are no known ways to bypass this filter. The tests performed showed no tangible impact on the performance of web applications in cases where the request does not contain dangerous values and a 7-15% slowdown in the formation of a response otherwise. Considering the fact that the Proof-of-Concept version uses third-party parsers that solve a much more general task than is required by the response validation algorithm, an optimal implementation of these components will allow to achieve performance sufficient for confident application of the solution in production environments.
7. Conclusions
The implementation of the query validation functional in current versions of ASP.NET is ineffective and does not solve the problem of protection against XSS Type-1 class attacks. Nonetheless, its current architecture and expansion options allow itself to solve this problem using the response validation method described in this article.
However, we should not forget that the most effective protection against such attacks are not third-party mounted solutions, but the correct implementation of processing input and output data by the developers themselves. And using Irv or more complex products (such as mod-security for IIS or .NetIDS ) does not save a developer from having to follow basic rules for developing secure code (for example, www.troyhunt.com/2011/12/free-ebook-owasp-top -10-for-net.html or wiki.mozilla.org/WebAppSec/Secure_Coding_Guidelines ).
Source: https://habr.com/ru/post/178357/
All Articles