Geekly Articles each Day
📜 ⬆️ ⬇️

Anatomy of .NET Core: how we set up NTLM for Linux

We continue to talk about the migration of the mobile service in ASP.NET Core and Docker. This article will deal with the WCF client module mentioned in the previous article , NTLM authorization and other problems during its migration. Now we will tell why we had to study anatomy for a bit and touch the .NET Core from the inside.



Soft way. Windows container


First of all, we configured debag to the docker image and started the service locally in a windows container.


When I tried to send a request to the WCF service, I got a rather ornate error:


System.ServiceModel.Security.MessageSecurityException: The HTTP request is unauthorized with client authentication scheme 'Negotiate'. The authentication header received from the server was 'Negotiate TlRMTVNTUAACAAAAEAAQADgAA...


By the method of sampling we came to the conclusion that it is required to specify the Domain in the credit of the service. It's funny that you can specify any value, just not null - then it works.


 static partial void ConfigureEndpoint(ServiceEndpoint serviceEndpoint, ClientCredentials clientCredentials) { ... clientCredentials.Windows.ClientCredential.Domain = ""; }

Everything, now requests go, now in the Windows container affairs ok. We go further.


We try. NET Core under Linux


Switching to the Linux container assembly, the Domain value was removed for the sake of interest - and it works.


The first problem when sending requests to WCF is related to SSL. Swears like this:


System.Net.Http.CurlException SSL peer certificate or SSH remote key was not OK


Which means: there is no trust in the certificate. If the WCF service sent not only the final certificate, but also all the intermediate ones, there would be no problem.


As decided:


1. We extort intermediate certificates.


For example, in Chrome, open the link and go to F12 in the Security tab. Further View Certificate → Certification Path. For each certificate, open the View Certificate and on the Details tab of the Copy To File button, save the Base-64 encoded certificates to the project directory. File extension needs to be changed to .crt.


2. Add a new layer to the Dockerfile.


 FROM microsoft/aspnetcore:latest AS base WORKDIR /app EXPOSE 80 FROM base AS cert COPY Certificates/*.crt /usr/local/share/ca-certificates/ RUN update-ca-certificates || exit 0

For debugging and experimenting, you can simply temporarily disable SSL validation:


 clientCredentials.ServiceCertificate.SslCertificateAuthentication = new X509ServiceCertificateAuthentication() CertificateValidationMode = X509CertificateValidationMode.None, RevocationMode = X509RevocationMode.NoCheck };

The most useful and important thing we learned when we received a CurlException is that libcurl is used for network requests.


The tasty part was waiting for us to come.


Linux + WCF + NTLM = love, but after dinner


Now the road was blocked by such an exception.


MessageSecurityException The HTTP request is unauthorized with client authentication scheme 'Negotiate'. The authentication header received from the server was 'Negotiate, NTLM'.


Change Security.Transport.ClientCredentialType = HttpClientCredentialType.Windows to HttpClientCredentialType.Ntlm


The error has changed somewhat, but it has not become easier:


MessageSecurityException The HTTP request is unauthorized with client authentication scheme 'Ntlm'. The authentication header received from the server was 'Negotiate, NTLM'.


Let's make sure that we are not faced with another docker-related feature like the Domain value.


We start service in a virtualka with Ubuntu LTS.


Lyrical digression: Docker for Windows loves Hyper-V and may refuse to work when installing other virtual machines. Therefore, this time I had to raise Ubuntu under Hyper-V, in which copy-paste does not work between the host and guest machines, which is good news.

By the way, Microsoft, how is the friendship with Apple?

Next lay a Mac with Visual Studio installed. Hands themselves itch. When launching with SslCertificateAuthentication disabled, the The handler does not support custom handling of certificates with this combination of libcurl (7.54.0) and its SSL backend ("SecureTransport") . If you return to the place of validation of certificates, there will be the same error with NTLM. Still, the first error casts suspicion that the differences from Linux can be significant.
')
What other ways are there to return?

Ubuntu on Windows - when the service was started, they got into the error System.DllNotFoundException: Unable to load DLL 'System.Net.Http.Native' .

And in the case: on pure Linux, the error is exactly the same as in the container, which means that the problem lies in the implementation of the WCF client.


We try to come from the other side. From the command line, run:


1. curl -v --negotiate - fails


 * gss_init_sec_context() failed: SPNEGO cannot find mechanisms to negotiate.

2. curl -v --ntlm - everything is fine, the query works


Here is the time to remember the list of officially supported features in WCF . The search line says that Core on Linux does not know how to use NTLM. But it does not stick with the fact that curl is able, and it would be strange not to realize such a popular option.


From the commentary on the Internet, we learn that Negotiate Negotiate is different: in some implementations there is support for fallback Kerberos → NTLM (everywhere on Windows), and in others - not. Curl of the latter, and Negotiate becomes an obstacle.


All this suggests that HttpClient can not take into account this nuance, which means that there is hope for victory.


We watch source codes


And here it is impossible not to rejoice at the new Microsoft for their decision to open the code to the world. In the sorts, we find the key CURLHANDLER_DEBUG_VERBOSE=true , which will tell us what libcurl is doing at the time of the WCF requests.


In the logs we see the already familiar error gss_init_sec_context() failed for both HttpClientCredentialType.Windows and for HttpClientCredentialType.Ntlm.


Now it is clear that the WCF client does not respond to switching from Windows authorization to NTLM and tries to use Negotiate in both cases. This is most likely due to the WWW-Authentication 'Negotiate, NTLM' dual header, which sends the WCF service, and since Negotiate is a stronger authorization, it is used.


From the libcurl manual, it’s tasted that the authorization type is set via the CURLOPT_HTTPAUTH option. Following this trail, we went to the authorization selection table :


 private static readonly KeyValuePair<string,CURLAUTH>[] s_orderedAuthTypes = new KeyValuePair<string, CURLAUTH>[] { new KeyValuePair<string,CURLAUTH>("Negotiate", CURLAUTH.Negotiate), new KeyValuePair<string,CURLAUTH>("NTLM", CURLAUTH.NTLM), new KeyValuePair<string,CURLAUTH>("Digest", CURLAUTH.Digest), new KeyValuePair<string,CURLAUTH>("Basic", CURLAUTH.Basic), };

The static readonly attributes look especially tempting, since this means that it is enough to play around using Reflection with the values ​​in the table at the start of the service, and there will be no overhead in HTTP requests.


Added the following code to Program.cs :


 public static void Main(string[] args) { … // redirect Negotiate to NTLM (only for libcurl on Linux) var curlHandlerType = typeof(HttpClient).Assembly.GetTypes() .FirstOrDefault(type => type.Name == "CurlHandler"); if (curlHandlerType != null) { var authTypesField = urlHandlerType.GetField("s_orderedAuthTypes", BindingFlags.Static | BindingFlags.NonPublic); var authTypes = authTypesField.GetValue(null); var authTypesGetByIndex = authTypes.GetType().GetMethod("Get"); var ntlmKeyValuePair = authTypesGetByIndex.Invoke(authTypes, new object[] { 1 }); var ntlmValue = ntlmKeyValuePair.GetType().GetProperty("Value"); var CURLAUTH = ntlmValue.GetMethod.ReturnType; var CURLAUTH_NTLM = ntlmValue.GetValue(ntlmKeyValuePair); var authTypeKeyValuePairBuilder = typeof(KeyValuePairBuilder<,>) .MakeGenericType(new[] { typeof(string), CURLAUTH }); var builder = Activator.CreateInstance(authTypeKeyValuePairBuilder); var negotiateToNtlmKeyValuePair = authTypeKeyValuePairBuilder .GetMethod("Build") .Invoke(builder, new object[] { "", CURLAUTH_NTLM }); var authTypesSetByIndex = authTypes.GetType().GetMethod("Set"); authTypesSetByIndex.Invoke(authTypes, new object[] { 0, negotiateToNtlmKeyValuePair }); } } // makes it possible to call Activator.CreateInstance on KeyValuePair struct public class KeyValuePairBuilder<K, V> { public KeyValuePair<K, V> Build(K k, V v) { return new KeyValuePair<K, V>(k, v); } }

Here we nail the correspondence between "Negotiate" and CURLAUTH.NTLM.


Voila, now requests work successfully.


Bonus track


We did not stop there. If you take a close look at the logs, you can see that one WCF request-response includes several HTTP request-responses, and one of the responses returns steadily with a Bad Request . What's the matter?


For an erroneous request, the HEAD method is used. Indeed, the same behavior is easily emulated with curl -I . In libcurl, this corresponds to the CURLOPTION_NOBODY option. In corefx, this option is used when sending HttpMethod.Head requests.
We go up the stack higher in WCF. We see that in the SendPreauthenticationHeadRequestIfNeeded method, a HEAD request is sent for authorization, and all errors are simply ignored:


 try { // There is a possibility that a HEAD pre-auth request might fail when the actual request // will succeed. For example, when the web service refuses HEAD requests. We don't want // to fail the actual request because of some subtlety which causes the HEAD request. await SendPreauthenticationHeadRequestIfNeeded(); } catch { /* ignored */ }

Here a flag is clearly suggested, similar to HttpClientHandler.PreAuthenticate , in order not to launch a request that is doomed to 400 in advance.


Since we are not taken care of, then we will cut.


The SendPreauthenticationHeadRequestIfNeeded method SendPreauthenticationHeadRequestIfNeeded asynchronous, so patching it can cause red eyes at a very early age. If you look around you can see the simple and unpretentious method AuthenticationSchemeMayRequireResend . Obviously, if it returns always false, then SendPreauthenticationHeadRequestIfNeeded will not run.


Getting to the operation.


We add to the solution a new project WcfPreauthPatch. Now we put Cecil, with which we will get into the IL-code. We need a beta version to work under .NET Core.


Install-Package Mono.Cecil -Version 0.10.0-beta7 -ProjectName WcfPreauthPatch


The code is:


 static void Main(string[] args) { var curlHandlerType = typeof(HttpClient).Assembly.GetTypes() .FirstOrDefault(type => type.Name == "CurlHandler"); if (curlHandlerType == null) return; // continue only when libcurl is used var wcfDllPath = typeof(System.ServiceModel.ClientBase<>) .Assembly.ManifestModule.FullyQualifiedName; var wcfAssembly = AssemblyDefinition.ReadAssembly(wcfDllPath); var requestType = wcfAssembly.MainModule.GetAllTypes() .FirstOrDefault(type => type.Name.Contains("HttpClientChannelAsyncRequest")); var authRequiredMethod = requestType.Methods .FirstOrDefault(method => method.Name.Contains("AuthenticationSchemeMayRequireResend")); authRequiredMethod.Body.Instructions.Insert(0, Instruction.Create(OpCodes.Ldc_I4_0)); // put false on stack authRequiredMethod.Body.Instructions.Insert(1, Instruction.Create(OpCodes.Ret)); wcfAssembly.Write(wcfDllPath + ".patched"); File.Delete(wcfDllPath); File.Move(wcfDllPath + ".patched", wcfDllPath); }

In Dockerfile we will add


 # for build image FROM build AS build-wcf-patch WORKDIR /src/WcfPreauthPatch/ RUN dotnet build -c Debug -o /app ... # for release image FROM base AS base-wcf-preauth-fixed COPY --from=publish /app . RUN dotnet WcfPreauthPatch.dll

We start service and we are convinced that in logs one request became less.


Epilogue


WCF client in .NET Core gave us a lot of trouble.


On github, there is already a discussion of the issues and questions raised in the article:


1. Negotiate / NTLM


https://github.com/dotnet/corefx/issues/9533
https://github.com/dotnet/corefx/issues/9234


2. Preauthentication-request


https://github.com/dotnet/wcf/issues/2433


However, as we have seen, these problems are not completely solved. We hope that our 5 kopecks in the discussion will add a new turn to the process.


Some ideas and facts for a snack


Source: https://habr.com/ru/post/350054/

More articles:


All Articles