Route graph for apache camel

In this article I will tell you about how you can build a route graph for applications with Apache Camel, monitor the status of these routes and collect metrics for them.

We use Apache Camel in spring applications and Apache ServiceMix. And if the routes in a separate service is a thing that is understandable and easily foreseeable, then within the data bus, where there are many such routes, it is not so simple.

<?xml version="1.0" encoding="UTF-8"?> <blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.osgi.org/xmlns/blueprint/v1.0.0 http://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd"> <camelContext xmlns="http://camel.apache.org/schema/blueprint"> <route> <from uri="file:camel/input"/> <log message="Moving ${file:name} to the output directory"/> <to uri="file:camel/output"/> </route> </camelContext> </blueprint> 

Over the years of use, the tire has accumulated more than a hundred different difficulty routes, and we have come to understand that it is becoming more and more difficult to remember all these connections. Drawing route maps with hands or describing them in the form of tables has ceased to seem a convenient and easily supported solution. But it began to seem that the automatic construction of the route graph would save everyone.

To construct a graph, we need vertices and edges. And from them we are blind to something beautiful!

Route elements

The entry point (it is one) for the route is described by the from operator with an endpoint specified. Those. for

 <from uri="file:camel/input"/> 

endpoint will be file:camel/input . He tells us that at the beginning of the route the files will be taken from the camel/input directory.

The exit points of the route (there are many of them, which is why I indicated the plural number) are determined differently - depending on the message exchange pattern, also with an indication of an endpoint. In the example above, this point is described by to . Those. for

 <to uri="file:camel/output"/> 

endpoint will be file:camel/output . He tells us that at the end of the route files will be saved to the camel/output directory.

Endpoint's are the vertices we need. The ribs will determine the routes themselves.

Getting route descriptions

Servicemix provides the ability to access various information using JMX and we will use jolokia to access this information via http.

As an example, take this description of the routes.

 <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd"> <camelContext xmlns="http://camel.apache.org/schema/spring"> <route id="the-clock-is-ticking"> <from uri="timer://foo?fixedRate=true&amp;period=1000"/> <to uri="jms:topic:timer?connectionFactory=demo"/> </route> <route id="service-a"> <from uri="jms:topic:timer?connectionFactory=demo"/> <to uri="jms:queue:service-a?connectionFactory=demo"/> </route> <route id="service-a-log"> <from uri="jms:queue:service-a?connectionFactory=demo"/> <to uri="log:service-a"/> </route> </camelContext> </beans> 

Route list

The http://host:8181/jolokia/read/org.apache.camel:type=routes,* method returns a list of routes with details.

Sample return data for the service-a route:

 "org.apache.camel:context=a.xml,name=\"service-a\",type=routes": { "StatisticsEnabled": true, "EndpointUri": "jms:\/\/topic:timer?connectionFactory=demo", "CamelManagementName": "a.xml", "ExchangesCompleted": 173, "LastProcessingTime": 2, "ExchangesFailed": 0, "Description": null, "FirstExchangeCompletedExchangeId": "ID-...", "StartTimestamp": "2018-12-17T07:01:12Z", "FirstExchangeCompletedTimestamp": "2018-12-17T07:01:13Z", "LastExchangeFailureTimestamp": null, "MaxProcessingTime": 35, "LastExchangeCompletedTimestamp": "2018-12-17T07:04:05Z", "Load15": "", "DeltaProcessingTime": -8, "OldestInflightDuration": null, "ExternalRedeliveries": 0, "ExchangesTotal": 173, "ResetTimestamp": "2018-12-17T07:01:12Z", "ExchangesInflight": 0, "MeanProcessingTime": 4, "LastExchangeFailureExchangeId": null, "FirstExchangeFailureExchangeId": null, "Uptime": "2 minutes", "CamelId": "camel-3", "TotalProcessingTime": 827, "FirstExchangeFailureTimestamp": null, "RouteId": "service-a", "RoutePolicyList": "", "FailuresHandled": 0, "MessageHistory": true, "Load05": "", "OldestInflightExchangeId": null, "State": "Started", "InflightExchanges": 0, "Redeliveries": 0, "MinProcessingTime": 0, "LastExchangeCompletedExchangeId": "ID-...", "Tracing": false, "Load01": "" } 

There are many RouteId , and among them, of particular interest for graph construction are RouteId , Context , EndpointUri , State , Uptime .

It is important to mention that the method returns and metrics along the route: ExchangesTotal , ExchangesCompleted , ExchangesFailed , ExchangesInflight , etc.

The above method covers 90% of our data needs, but what it does not return is information about exit points from the route. To obtain this information you need to use the method of obtaining details of the route and the method of obtaining the scheme. One of the methods is not enough, since in some cases the methods do not return all the data necessary to form a list of exit points.

Route details

The route details are obtained from the method.

http://host:8181/jolokia/exec/org.apache.camel:context=a.xml,type=routes,name="service-a"/createRouteStaticEndpointJson(boolean)/true

Sample return data:

 { "request": { "mbean": "org.apache.camel:context=a.xml,name=\"service-a\",type=routes", "arguments": ["true"], "type": "exec", "operation": "createRouteStaticEndpointJson(boolean)" }, "value": "{\"routes\": { \"service-a\": { \"inputs\": [ { \"uri\": \"jms:\/\/topic:timer?connectionFactory=demo\" } ], \"outputs\": [ { \"uri\": \"jms:\/\/queue:service-a?connectionFactory=demo\" } ] }}\n}\n", "timestamp": 1545040570, "status": 200 } 

Route map

The route map is obtained from the method

http://host:8181/jolokia/exec/org.apache.camel:context=a.xml,type=routes,name="service-a"/dumpRouteAsXml(boolean)/true .

The method returns the route map in xml form only if it has been framed. For example, if a route is described using org.apache.camel.builder.RouteBuilder (used when describing routes in a spring application), then the method will not return anything.

Sample return data:

 { "request": { "mbean": "org.apache.camel:context=a.xml,name=\"service-a\",type=routes", "arguments": ["true"], "type": "exec", "operation": "dumpRouteAsXml(boolean)" }, "value": "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<route customId=\"true\" id=\"service-a\" xmlns=\"http:\/\/camel.apache.org\/schema\/spring\">\n <from uri=\"jms:topic:timer?connectionFactory=demo\"\/>\n <to uri=\"jms:queue:service-a?connectionFactory=demo\" id=\"to5\"\/>\n<\/route>\n", "timestamp": 1545040727, "status": 200 } 

Draw graph

Combining all the information received, you can safely draw a graph, I used vis.js and got this result



The points are the points of entry and exit, the edges are the routes, and the gray numbers on the routes are the ExchangesTotal metric.

Graphing for several services

The described approach to the construction of a graph is also suitable for the case when camel is used not only in the data bus, but also in applications. For example, having described the route in the application as follows:

 @Component public class EventRoutes extends RouteBuilder { @Override public void configure() throws Exception { from("jms:topic:timer") .inOnly("bean:service?method=handle"); } } 

You can combine all the data on the routes from the servicemix and the application and draw a general graph



Notice that a new route appears in the diagram from jms:topic:timer to bean:service .

Conclusion

Having implemented the described approach to building a route graph, we were able to get a general picture for both the bus and integrated services. Actually the graph of our tires looks like



Proof of concept applications can be viewed here - github