Configure Azure Application Insights for diagnosing software with microservice architecture

Application Insights is a cool thing that allows you to diagnose, profile and analyze the use of deployed systems (including in production mode), and at the same time does not require any effort from the developer. Of course, all this becomes possible at the price of an agonizing initial setup.

Of course, there is no special setting in the promotional videos, but life is more complicated, especially if your software is microservice. Why? And everything is very simple.

What should the diagnostic system in the microservice architecture do first of all?
Correctly, correlate diagnostics from various microservices within one operation.
Tyrknul user in the UI button - you need to see the diagnostics from all N microservices that somehow processed this tyrk. An exception happened somewhere - it is necessary to see not only in which microservice it happened, but also in the framework of which operation it happened.
Only here, Application Insights, from the point of view of a specific microservice, is primarily the SDK. And there are several such SDKs - for JS, for .NET Core, .NET (with its own settings for MVC, WebAPI, WCF), for Java, etc.

Some of these SDKs are opensource, some are internal MS development. And to get everything started - they need to be friends.

This is the main difficulty.

Not to say that I have achieved 100% enlightenment in this matter.
But at least, I have already collected a few rakes and I have a working sample from the UI on ASP.NET MVC (not Core) + JS and two microservices (Asp.Net WebApi, WCF)

Who cares - I ask under the cat.

A little about Application Insights

If in a nutshell - it works like this:

Using the SDK, handles or automatically generates objects from the Application Insights data model for software events.
Generated objects are uploaded to Azure via the REST API
In Azure there are tools (very rich) for analyzing the generated information

How does the Application Insights model date map to microservice architecture?

As can be seen from the picture, each kick in the microservice \ UI \ API generates a Request on the side where they kicked and Dependency on the side they kicked.
In the course of work, trace events, error messages, custom events, as well as such specialized objects as pageView can also be generated.

How Application Insights Correlates Objects

The blue arrows in the picture show the connections between the objects. As seen,

All objects are associated with a specific operation (this is not a separate entity of the AI date-model, they simply have the same property. The operation is initialized when the first request is not within the scope of the operation)
Trace messages / error messages / custom events / microservice dependencies are connected in addition to the request object (also through a property)
Request objects are additionally associated with the Dependency object (unless this is the very first Request) also through a property.

Details can be found in the official Application Insights documentation.

What's next?

And then there will be a description of two scenarios of interaction between microservices with a detailed description of the method of implementation.

Scenario 1 - AJAX, MVC, WebApi

Or the same, but with the words:

To display the page, it is required in the MVC controller to obtain data from the WebAPI microservice (via HttpClient)
After displaying the page, she herself, using jQuery, climbs onto the server in the MVC controller and receives more data, which in turn is also taken from the WebAPI microservice.

What we want to get

We want to be able to

See page view
See that the server went to microservice to generate a page
See that the page itself was performing ajax request to the server
To see that the server also went to microservice to satisfy this request.
In case of an error somewhere, we want to immediately see the context within which it occurred.

Ideally, we should see all the information on one screen as a sequence of actions.
Of course, all of these goals will ultimately be achieved.
In order not to torment -

First the result:

The transition from the page to the diagnostic information on the operation

Diagnostic information itself

And now with an error in microservice:

Implementation details

In the Web API microservice:

Installing NuGets Microsoft.ApplicationInsights , Microsoft.ApplicationInsights.Web , Microsoft.AspNet.WebApi.Tracing
With the packages above comes TraceListener for ApplicationInsights, it is written in the web.config itself.
In WebApiConfig we specify config.EnableSystemDiagnosticsTracing ()
Implement an IExceptionFilter that logs all uncaught errors in ApplicationInsights, write it in FilterConfig.cs

Code

[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, Inherited = true, AllowMultiple = true)] public class WebApiAITrackingAttribute : FilterAttribute, IExceptionFilter { public void OnException(ExceptionContext filterContext) { if (filterContext != null && filterContext.HttpContext != null && filterContext.Exception != null) { ApplicationInsightsSettings.TelemetryClient.TrackException(filterContext.Exception); } } }

TelemetryClient - object from the ApplicationInsights SDK. It is not recommended to create it every time, so it is here singleton through ApplicationInsightsSettings

In ASP.Net MVC UI:

Install the Microsoft.ApplicationInsights NuGet, Microsoft.ApplicationInsights.Web
Implement an IExceptionFilter that logs all uncaught errors in ApplicationInsights, write it in FilterConfig.cs (the same as for the Web API microservice)
Implement HttpClientHandler to work with HttpClient, creating Dependency.
Here it is a little more detailed: in general, the documentation declares that System.Net.HttpClient, starting from which version it is able to create Dependency from which version.
And this is indeed true. But he does it not directly (he does not work directly with the AI SDK), but slightly through one place, so this Dependency does not always bind correctly to Request. So when I got tired of catching bugs with this - I wrote my HttpClientHandler

Code

 public class DependencyTrackingHandler : HttpClientHandler { public DependencyTrackingHandler() { } private static int _dependencyCounter = 0; protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) { using (var op = ApplicationInsightsSettings.TelemetryClient.StartOperation<DependencyTelemetry>(request.Method.Method + " " + request.RequestUri.AbsolutePath)) { op.Telemetry.Target = request.RequestUri.Authority; op.Telemetry.Type = request.Method.Method; op.Telemetry.Data = request.Method.Method + " " + request.RequestUri.ToString(); op.Telemetry.Id += "." + Interlocked.Increment(ref _dependencyCounter).ToString(); request.Headers.Add("Request-Id", op.Telemetry.Id); var result = base.SendAsync(request, cancellationToken).Result; op.Telemetry.ResultCode = result.StatusCode.ToString(); op.Telemetry.Success = result.IsSuccessStatusCode; return result; } } }

As you can see, the meaning of this handler is to wrap the HttpClient call in DependencyTelemetry, and also (and this is very important), set the correct Request_Id header.
About heders also a little more detail. Through them, the AI infrastructure transmits information about the operation id, as well as information about the id dependency when making calls via the HttpClient.
Headers for these purposes are used such: "Request-Id", "x-ms-request-id", "x-ms-request-root-id". For the correlation to be correctly initialized, the first one or (the second and third one) is enough.
More information about correlations and heders can be read in the documentation (although it is rather messy)

We recall that here we are dealing with two SDKs - for JS and for .NET, so somewhere in MVC views we are looking for code that initializes appInsight on the browser side, and then
We register there InstrumentationKey

Code

 var appInsights=window.appInsights||function(config) { function r(config){ t[config] = function(){ var i = arguments; t.queue.push(function(){ t[config].apply(t, i)})} } var t = { config:config},u=document,e=window,o='script',s=u.createElement(o),i,f;for(s.src=config.url||'//az416426.vo.msecnd.net/scripts/a/ai.0.js',u.getElementsByTagName(o)[0].parentNode.appendChild(s),t.cookie=u.cookie,t.queue=[],i=['Event','Exception','Metric','PageView','Trace','Ajax'];i.length;)r('track'+i.pop());return r('setAuthenticatedUserContext'),r('clearAuthenticatedUserContext'),config.disableExceptionTracking||(i='onerror',r('_'+i),f=e[i],e[i]=function(config, r, u, e, o) { var s = f && f(config, r, u, e, o); return s !== !0 && t['_' + i](config, r, u, e, o),s}),t }({ instrumentationKey: "@Microsoft.ApplicationInsights.Extensibility.TelemetryConfiguration.Active.InstrumentationKey" });

We synchronize the global operation between the server and the browser (so that in one global operation we can also display server diagnostics and diagnostics from the browser)

Code

 window.appInsights.queue.push(function () { var serverId ="@Context.GetRequestTelemetry().Context.Operation.Id"; appInsights.context.operation.id = serverId; });

     WA **Sergey Kanzhelev**    [http://apmtips.com](http://apmtips.com)

Scenario 2 - AJAX, WCF, WebApi

Or the same, but with the words:

To display the page does not require data from microservice
After displaying the page, she, using jQuery, climbs into WCF microservice and receives more data, which in turn is also taken from WebAPI microservice.

The objectives are the same as in the previous scenario.

First the result:

With an error in microservice:

Implementation details

In the Web API microservice - nothing new relative to the previous scenario.

In ASP.Net MVC UI:

Somewhere in the MVC view, we look for code that initializes appInsight on the browser side, and then redefines the appInsights function. _AjaxMonitor.sendHandler

Code

 window.appInsights.queue.push(function () { appInsights._ajaxMonitor.sendHandler = function (e, n) { e.ajaxData.requestSentTime = Date.now(); if (!this.appInsights.config.disableCorrelationHeaders) { var i = this.appInsights.context.operation.id; e.setRequestHeader("x-ms-request-root-id", i); e.setRequestHeader("x-ms-request-id", e.ajaxData.id); } e.ajaxData.xhrMonitoringState.sendDone = !0; }; });

With this a little more detail. As you know, requests via XmlHttpRequest to hosts that are different from the current one are subject to additional security, so-called. CORS
This is expressed in the fact that the http API can fly so-called. preflight requests before the main for the headers, method and host of the main request.
So, for some reason, the Application Insights SDK for JS seems to be very afraid of sending these preflight requests and therefore never sends correlation headers to hosts other than the current (taking into account the port).
On the Application Insights SDK command for JS, FR is already running on this.
In the code above, as a WA, the check for matching hosts is simply removed, and thus the headers are sent anyway.

In WcfApi:

Install the Microsoft.ApplicationInsights , Microsoft.ApplicationInsights.Web , Microsoft.ApplicationInsights.Wcf NuGet (from https://www.myget.org/F/applicationinsights-sdk-labs/ )

We delete from ApplicationInsights.config

 <Add Type="Microsoft.ApplicationInsights.Web.OperationNameTelemetryInitializer, Microsoft.AI.Web"/>

Mark the attribute [ ServiceTelemetry ] class of service
Do not forget to set the Method = "*" in the WebInvoke attribute for the methods of the service interface (since preflight requests with the OPTIONS method will arrive in it)
We implement the answer to the preflight request in the service methods

Code

 private bool CheckCorsPreFlight() { var cors = false; if (WebOperationContext.Current != null) { var request = WebOperationContext.Current.IncomingRequest; var response = WebOperationContext.Current.OutgoingResponse; if (request.Method == "OPTIONS") { response.Headers.Add("Access-Control-Allow-Methods", "GET, POST, OPTIONS"); response.Headers.Add("Access-Control-Allow-Headers", "Content-Type, X-Requested-With, x-ms-request-root-id, x-ms-request-id"); response.Headers.Add("Access-Control-Allow-Credentials", "true"); cors = true; } var origin = request.Headers.GetValues("Origin"); if (origin.Length > 0) response.Headers.Add("Access-Control-Allow-Origin", origin[0]); else response.Headers.Add("Access-Control-Allow-Origin", "*"); } return cors; }

As a bonus - requests for Application Insights Analytics

Getting the global operation id on the page view

 pageViews | order by timestamp desc | project timestamp, operation_Id, name

Getting the id of the global operation upon the occurrence of an error

 exceptions | order by timestamp desc | project timestamp, operation_Id, problemId, assembly

Also for these purposes it is convenient to use the new interface Failures.

Obtaining diagnostic data for global operation

 requests | union dependencies | union pageViews | union exceptions | where operation_Id == "<place operation id here>" | order by timestamp desc | project timestamp, itemType, data = iif(itemType == "exception", problemId, name)