Geekly Articles each Day
📜 ⬆️ ⬇️

What to do if you do not know how the software works


A source

We also do not like software that does not know how it works. If the program is a black box, for each incomprehensible situation, there are exactly two options: try to attach the plantain or contact the manufacturer. But, firstly, we do not know where so much of the plantain is growing, and, secondly, to address the vendor for various details, too, is somehow not atsatsa.

A few years ago, one of our customers found exactly this kind of software: requests at the entrance, answers at the exit, and inside it is unclear what. Called automated banking system or ABS. But in essence, this is a database that generally processes requests from processing and returns the result. The result is the answer: can the bank carry out this operation or not. There is no way to approach this database with an external query due to its architectural limitations. In one of the recent projects, we again encountered the same software in retail and thought that it would be time to share our experiences.
')
In the last article we told how listening to traffic helped one of Technoserv's clients to catch the internal problems of the software. This will be a technical analysis of that long-standing task (hands are remembered), and at the end - two important benifits of this type of monitoring.

The special product for analyzing MicroFocus RUM traffic (more about it in the previous article at the link above), of course, supports a variety of network protocols, but the ones we need are not. And it was necessary to listen to service RPC requests from intrabank systems to ABS and traffic using the ISO8583 protocol from external devices (ATMs and POS terminals) all to the same ABS.

Analyzing RPC Data


Setting the task from the customer looked like this:

“Guys, here’s the traffic for a few hours to analyze. Here, from various banking systems to the ABS, transactions run in the form of requests and responses. You need to know the execution time of each such request, get the values ​​of some fields and count the number of errors. At the output you need a report in tabular form. "

The most difficult in this task was the analysis of the received traffic. Finding sequences and splitting the flow into separate transactions required hard work. After analysis, it turned out that each transaction is an XML packet with the same identifier for the request and response, as well as a set of other fields. The screenshot shows an example of this traffic in the Wireshark interface and the corresponding XML.



To RUM learn to understand the various types of network data, you need to describe this data to him. For this we have developed a script in C ++. Below is the code for parsing RPC traffic.

Traffic parsing code
#include "./ProtocolRPCRQ.hpp" #include <stdlib.h> #include <stdio.h> #include <signal.h> #include <setjmp.h> #include <iostream> #include <fstream> #include <base/ProtocolConfig.hpp> #include <base/ProcessingState.hpp> #include <utils/LogHelper.hpp> using namespace rum::public_sdk; sigjmp_buf mark; void segfault_sigaction(int signal) { ; } RPCRQProcessor::RPCRQProcessor( const char* name, IProtocolProcessor* next ) : ProtocolProcessorBase( name, next, "protocols.RPCRQProcessor" ) { } void RPCRQProcessor::initialize( const ProtocolConfig& config ) { ; } /*override*/ void RPCRQProcessor::process( ProtocolEvent& event, ProcessingState& ps, ProcessingMode mode ) { bool isParsed; try { if (mode != PROCESSING_FULL ) { forwardToNextProcessor( event, ps, mode); return; } EventParsingContext eventContext( event ); isParsed = doAllParsingWork( ps.getRequest(), ps.getResponse(), eventContext); if (isParsed) { forwardToNextProcessor(event, ps, mode); } else return; } catch (...) { LOG4PROBE_ERROR( getDefaultLogger(), "logger1"); } } String RPCRQProcessor::search_substring(String data, String start, String stop) { String empty = ""; try { size_t start_pos = data.find(start); if (start_pos!=std::string::npos) data = data.substr(start_pos+start.length()); size_t stop_pos = data.find(stop); if (stop_pos!=std::string::npos) data = data.substr(0,stop_pos); if (data.find("xmlns")!=std::string::npos) return empty; if ((start_pos==std::string::npos) || (stop_pos==std::string::npos)) return empty; else return data; } catch (...) {return empty;} } String RPCRQProcessor::stream_to_string(DataStream& stream,int offset) { String rumstring = ""; String empty = ""; int i = 0; try { while ( (!stream.eof()) && (i<10000)) { i++; if (stream.peek()!=0xff) { unsigned char value = stream.get(); if ((value!=0) && (value!=0xff)) rumstring += value; } else { stream.skip(1); } } } catch (...) {return empty;} if (rumstring.length()>offset) return rumstring.substr(offset); else return empty; } bool RPCRQProcessor::doAllParsingWork( DataStream& request, DataStream& response, EventParsingContext &context ) { try { rum::public_sdk::ActionInfo iactionInfo = context.getActionInfo(); String schema = " xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">"; String strdata = stream_to_string(request,45); String message = search_substring(strdata,"<message type=\"RPC_","</message>"); if (message!="") { String msgType = search_substring(strdata,"<message type=\"","\">"); String origMsgID = search_substring(message,"<origMsgID>","</origMsgID>"); if (origMsgID=="") origMsgID = search_substring(message,"<origMsgID"+schema,"</origMsgID>"); String origSysID = search_substring(message,"<origSysID>","</origSysID>"); if (origSysID=="") origSysID = search_substring(message,"<origSysID"+schema,"</origSysID>"); String targSysID = search_substring(message,"<targSysID>","</targSysID>"); if (targSysID=="") targSysID = search_substring(message,"<targSysID"+schema,"</targSysID>"); String origPrcID = search_substring(message,"<origPrcID>","</origPrcID>"); if (origPrcID=="") origPrcID = search_substring(message,"<origPrcID"+schema,"</origPrcID>"); String timeStamp = search_substring(message,"<timeStamp>","</timeStamp>"); if (timeStamp=="") timeStamp = search_substring(message,"<timeStamp"+schema,"</timeStamp>"); String CmdName = search_substring(message,"<command name=\"","\">"); if (CmdName=="") CmdName = "PARSING_ERROR"; iactionInfo.addKeyValue("msgType" , msgType , false ); iactionInfo.addKeyValue("origMsgID", origMsgID, false ); iactionInfo.addKeyValue("origSysID", origSysID, false ); iactionInfo.addKeyValue("targSysID", targSysID, false ); iactionInfo.addKeyValue("origPrcID", origPrcID, false ); iactionInfo.addKeyValue("timeStamp", timeStamp, false ); iactionInfo.addKeyValue("CmdName" , CmdName, false ); iactionInfo.addKeyValue("x-action-descriptor" , CmdName, false ); iactionInfo.setDescriptor(CmdName); String respstrdata = stream_to_string(response,45); String resmessage = search_substring(respstrdata,"<message type=\"RPC_","</message>"); if (resmessage!="") { String retCode = search_substring(resmessage,"<retCode>","</retCode>"); if (retCode!="") { iactionInfo.addKeyValue("retCode" , retCode, false ); if (retCode.find("OK") != std::string::npos) { context.getEvent().setStatusCode(0); } else context.getEvent().setStatusCode(1); } else { iactionInfo.addKeyValue("retCode" , "NOTFOUND", false ); context.getEvent().setStatusCode(2); } } } if (message=="") return false; else return true; } catch (...) { LOG4PROBE_ERROR( getDefaultLogger(), "exception in RPCRQProcessor" ); context.getEvent().setStatusCode(3); } } ////////////////////////////////////////////////////////////// // class MyProtocolParser::EventParsingContext ////////////////////////////////////////////////////////////// RPCRQProcessor::EventParsingContext::EventParsingContext( ProtocolEvent& event ) : _event(event) { try { _connectionContext = _event.getSessionContext(); if ( NULL == _connectionContext.get() ) { _connectionContext = new ConnectionInfo(); _event.setSessionContext( _connectionContext.get() ); } } catch (...) { std::ofstream fileSTRINGTEST; fileSTRINGTEST.open("/home/rum/fileSTRINGTEST_error2.out",std::ios::out|std::ios::app|std::ios::binary); fileSTRINGTEST << "--------------------------------\n"; fileSTRINGTEST.close(); } } // Declaring Factory for creating processor RUM_PROBE_EXPORT_PROCESSOR( RPCRQProcessor, getRPCRQProcessor);


The output was such an elegant report:



The customer is satisfied, and we received new skills in the product.

Analyzing data using ISO8583 protocol


The task is similar to the previous one with minor changes:

“Guys, here's a new batch of traffic. I have an aggregator that collects transactions from my ATMs and POS-terminals. All transactions are flying in the ABS. I want to know how many such transactions I had, their status and the execution time of each request. At the output you need a report in the form of a graph. "

As in the previous task, the network traffic packets were XML. Only with significantly more fields. With the help of the new code, we eliminated all the unnecessary and got about the same XML for analyzing and counting transactions as in the case of RPC.
At the exit, the customer received a schedule with the number of transactions. The screenshot below shows the weekly interval. Now, any anomalous behavior of this graph is interpreted by the system as a failure. We have configured event correlation with other business process applications, and administrators can immediately see the possible cause of a deviation from the norm. Of course, on different days the behavior of the graphics may differ. For example, on Saturday and Sunday there are no subsidence during the daytime by the number of transactions - this is the norm for weekends and the event is not generated. Here is such an artificial intelligence.



While working with user traffic monitoring systems, we identified two important advantages (there may be more of them, but for the time being we stopped at two):

  1. No additional load is created on the business application. There is no need to build special agents into the application, ask the admins for permission to reload something and generally disturb people for nothing.
  2. The ability to control "black boxes". There are many such applications and it is not always clear how to follow them. An example is our situation with ABS.

We invite you to take a survey, share in the comments examples of applications-black boxes and tell how you control them.

The author of the article is Anton Kasimov , architect of monitoring systems, Tekhnoserv company.

Source: https://habr.com/ru/post/342752/

More articles:


All Articles