Geekly Articles each Day
📜 ⬆️ ⬇️

Business analytics. IT objects, components, tools

Business analytics. IT objects, components, tools


I had the happiness (creativity) and misfortune (recognition and implementation) to develop a project of banking analytics. Develop from idea to implementation and directly participate in all stages of development. Most of all I liked the stage of setting and designing. Turning chaos of ideas into a clear statement is a great pleasure. Then, to transform the narrative of the performance into the constructiveness of the project is also a pleasure. Well, in addition, he developed the software architecture and programmed. This also contained its small charms, although at this stage each programmer had his own opinion and it was not easy to resolve contradictory views.

I had to read a lot on banking, on financial instruments, on accounting, management and in-kind accounting, on business operations ... And, of course, on design and programming.

So I wanted to share my views on business analytics.
')
There is, of course, an even more important subject - business synthesis, dealing with the integration of analysis data and decision making. “But that's another story.”

1. Business objects analytics


1.1. Agents


These are relatively passive objects that make sense on their own. Their examples:


1.2. Operations, Actions


These are active objects to which the term “Run” is applicable and the execution of which leads to a change in the state of other objects. Their examples:


Through operations, agents interact.

1.3. Developments


These are signals of new information.

Event classification:


1.4. Relations


By and large, everything can be represented as a relationship. So the attributes of a business agent are the characteristics of his relationship to the physical world, to the economic, to the legal world ... For example, the gender of a person by birth is a person’s attitude to the biological world, citizenship is his attitude to the legal world. The relational approach is also used in physics. The mass of the fundamental particle is a consequence of the relationship of the interaction of the particle with the Higgs field. Even with respect to space = time, there are relational theories.

Relationship classification:


Both visually and realistically, it is convenient to depict relationships with a directed loaded graph. The orientation arises from the asymmetry of the relationship: A pays B does not mean that B pays A. The ribs are loaded, for example, with the following attributes:


Nodes are loaded, for example, with the following attributes:


2. Indicator


2.1. Definition


The indicator of an object is a certain characteristic of an object. It is through indicators that an economic object is presented to the world of economics. An indicator is an economic quantity, an analogue of a physical quantity. From an information point of view, an object is a collection of indicators. This combination is not absolute. For different purposes, an object can be described by a different set of indicators.

Having come from physics to programming economic problems and faced with the concept of an economic indicator, I was sure that there is a system of economic indicators. A system in the sense as in the immortal phrase “There is a system in his madness”. I understood this system like this:


I began to search for such a system on the Internet. However, no matter how many times I looked through textbooks and monographs, I did not find any system. I came across the term “balanced scorecard”, but only talked about it, but I did not find a single complete example. Therefore, I will give my understanding of the scorecard.

2.2. Characteristics of indicators


There can be potentially infinitely many indicators. The question is naturally about such a subset of indicators, on the basis of which any other indicator can be calculated. Such a subset is called a basis . It has the property of completeness : on the basis of it, any non-basic derivative indicator can be calculated. The indicators included in it are independent of each other. This property is called base redundancy .
The state S (t) of the object at time t is all the indicators of the object at time t.
The motion M (t) = {S (t1), ..., S (tn)} of an object is a change in the state of the object over time. This is a set of states of an object over a time interval.
Indicator dynamics - indicator values ​​sorted by time.

Dynamics example:



The indicator trajectory is a curve in several coordinate indicators, parameterized by time. For example, for two indicators P, Q, this is a curve defined parametrically: P (t), Q (t).

An example of a trajectory for indicators P, Q.



Mugs - the value of indicators. Arrows - the direction of increasing time.
Having a trajectory, you can try to get a direct dependence of one indicator on another. So, if at the moment t the exponent Q has the value q, and the exponent P has the value p, then the dependence of f of the exponent Q on P is such that f (p) = q. The matter is only complicated by the fact that for different t, the same p can correspond to different q. Then, most likely, there is no functional dependence.

Similarly to two indicators, one can construct trajectories for an arbitrary number of indicators and use them to construct Q (P1, P2, ..., PT) dependencies. But with the number of indicators more than three, visibility is lost.

State trajectory - a set of trajectories of state indicators.

2.3. Classification of indicators


Indicator-flow is a numerical indicator, the value of which is set on a time interval. Examples: profit for a month, number of layoffs for a month, change in capital for a year, taxes paid for a quarter.

Indicator-state - a numerical indicator, the value of which is set at a point in time. Example: human growth, company capital, number of employees. Examples: the number of employees at the beginning of the year, the capital of the enterprise at the beginning of the year, the assets of the enterprise at the beginning of the day.

A change in an indicator for a certain period is an indicator-flow correlated with this period. Examples of changes: change in capital for the quarter, change in price per day, ...
If we fix one moment of time in the flow indicator, then we can talk about the indicator-state. So we can talk about profit at the moment, bearing in mind the change in profit from the beginning of the year. There is a fixed point in time - the beginning of the year.

Aggregate indicator is an indicator whose value is an integral function of the values ​​of other indicators. Examples: maximum, minimum, amount, average
External indicator - an indicator of the external environment for the subject in question. For example, for an enterprise it will be a market rate, prices, macroeconomic indicators.
Internal indicator - an indicator of the subject in question.
Basic indicator - an indicator belonging to the basis, i.e. a set of indicators through which all others are expressed ...
Input stream - a stream that sets the motion of the object being taken into account in the direction of the subject in question. Example: interest received for a loan issued, payment for goods sold.
Output stream - a stream that sets the motion of the object being taken in the direction from the subject in question. Example: interest paid on the loan received, payment for the purchased goods.

Value type


Numeric differentiate by:


The indicator can be considered as an independent object. Then it is characterized by certain indicators - indicator indicators or meta-indicators. The first is inconvenient, the second is too learned, so we will call them the attributes of the indicator.

Attributes of a numeric metric :


Key attributes (coordinates) of the indicator

These are attributes that uniquely determine the value of an indicator in the set of all values ​​of all indicators.


Example: Income ('MAA', '947', '01/01/2012', '31 / 12/2012, '0,' 0 ',' 0 ') - the actual value of the 2012 income indicator of the MAA entity.

In this case, the coordinates of the indicator:

Income - code of the indicator “income”
'MAA' - subject code for which the indicator is calculated
'947' - currency code in which the income was generated
'01 / 01/2012 '- the beginning of the income generation period
'31 / 12/2012 '- the end of the income generation period
'0' - reality code "Fact"
'0' - script number “No simulation”
'0' - basic calculation method

I saw systems in which the subject code was driven into the indicator code: KrsVltNB - exchange rate in the National Bank. Extending this method, all coordinates can be driven into the indicator code. “There is a system in his madness,” isn't it?

Example of an indicator description form:



2.4. Receive rule syntax


For programming, the clear formal syntax of the receive rule is important. Then it is possible to build a rule interpreter that allows the user to enter new indicators with new rules himself. It is important that the calculation rules fit into the implemented syntactical framework. Otherwise, you have to modify the program. Common operations allowed by the syntax:


2.5. Codification of indicators


The codification problem is generated by a large number of codified objects. If there are many objects, then the problem of their convenient codification becomes essential. Small problems become big. We need a system that introduces order, simplicity, and gives scalability to codification.

Codification example. Consider ways to codify a number.

Roman way.
I, II, III, IV, V, VI, VII, VIII, IX, X, XI, XII, XIII, XIV, XV, XVI ...

The coding system is rather obscure. How to do arithmetic operations in this record? There is no zero. It is not clear how to code fractional numbers.

Indian way.
0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16 ...

Any number is encoded in one way using ten digits. The notation _n _ (n-1) ... _0 corresponds to the integer _n * 〖10〗 ^ n + _ (n-1) * 〖10〗 ^ (n-1) ... _0 * 〖10〗 ^ 0. This is clear and clear. In such a record, it is easy to add, subtract, multiply, divide, and extract the square root. All of this is taught at school. The Indian way of writing numbers is a great example of the importance of notation, the importance of form.

Quite natural requirements for codification of indicators :


Specific brevity . The more often the code is used, the shorter the code should be.
Continuity with generally accepted designations.
Unified codification system . There should be no discrepancy in the codification system for different indicators or different types of indicators.
One-level sameness . “One-level” in the sense of indicators should have the same length. If A is an asset code, then it is logical for a liability to have a P code, not Passive, for example.

2.6. Balanced Scorecard


Now the so-called Balanced Scorecard is well known. Here's what Wikipedia says about her:
Balanced scorecard (balanced scorecard, BSC) - a strategic performance management tool, a partially standardized reporting form that allows managers to track the performance of tasks by employees, as well as the consequences of performance or non-performance. The system is equipped with special design and automation methods.

The term can correspond to both the system as a whole and the individual implementation of indicators. The most important characteristics of the approach are:

  • focus on the organization’s strategic agenda;
  • selection of a small number of monitored data;
  • combination of financial and non-financial data.

A balanced scorecard is an example of a feedback controller whose action is aimed at implementing a corporate strategy. This type of controller involves measuring performance, comparing a metric with a reference value, and making adjustments if there is a gap. Such control requires three prerequisites:

  • selection of data to be measured;
  • setting reference values;
  • fundamental opportunity to make adjustments.

In the context of strategic management, all three elements must flow from corporate strategy. It is assumed that the control subject is able to both observe and intervene in the process. In practice, this is not always fully practicable.

There are two factors that make this kind of control easier. First, the selection of data for monitoring can be simplified. Secondly, only those data that correspond to the ability of the manager to intervene in the process should be selected.

The end of the quote.

But I have not seen a specific set of indicators of MTP in any book.

2.7. Time series


The time series of the indicator I is the set I (t) = {I (t1), I (t2), .... I (tn)} of its values ​​at discrete time instants. It is also called the dynamic series of indicator I. The time series can be considered as a statistical series. Without knowing the internal background of obtaining the values ​​of a series, it can be considered as values ​​of a random variable, the distribution law of which is unknown.

2.8. Indicator groups


Many indicators are heterogeneous. Indicators are grouped. Unification can be objective and subjective. Subjective groups - grouping according to the local interests of the user. Objective groups - grouping by economic criteria. Examples of groups: balance, taxes, assets, securities, condition of the subject name, income ... Many groups can be considered as a graph in which the top is a group and the edges are types of relations between groups, for example, a group joining another group.

3. Product Structure Business Intelligence


3.1. ETL module


This is a module for extracting data from adjacent systems. Data is transformed and stored in the data warehouse.

3.2. Data store


This is a repository of time series of values ​​of indicators of both basic and derivatives.

3.3. Interpreter


Its purpose is to calculate the value of a derived indicator based on the formula for calculating it.

3.4. Performance machine


This is a generator of values ​​for multiple indicators - a computational pool of indicators. In the limit, these are all indicators. So it’s easier to realize real time. The indicators are interconnected and when calculating the values, the generator bypasses the graph of relations.

3.5. Event monitor


The module monitors internal events, controlling the values ​​of logical indicators, the calculation formulas of which formalize the condition of the event. External events are activated by external agents who, in the event of an event, notify the event monitor of this. The monitor calls the event handler.

3.6. Simulator


This is a program that allows you to answer the question “What happens if”. “If” is usually called a script. Simulator is a script implementation apparatus. The scenario determines the changes in indicators, and the indicator machine processes these changes throughout the settlement pool. The end of the calculation generates the corresponding event.

The scenario may include:


The base scenario is the passive evolution scenario. This is an empty scenario: it is assumed that there will be no operations. The actions of the rest, active, non-empty scripts determine the increments to the passive script.

3.7. Comparator


This is a module that implements all kinds of comparisons.

3.8. Decomposer


This is a module that implements the decomposition of an indicator into its component parts according to different decomposition criteria.

3.9. Display


This is a data visualization module.

4. Possible services


Many business intelligence modules can, and sometimes need to be implemented as stand-alone services. This enables flexible configuration of the complex and the possibility of multi-user work.

Service is understood as a shared application. The functions of the service are provided through the API announced by the developer. The service can be placed on any computer on the network. In the client-server architecture, there was only one service - the DBMS. In SOA services can be any number and they can be placed on different computers on the network.
Possible functional services :


Possible technological services :


Access . Determines the degree of access of a specific subject to a specific object. Access can be determined at the level of types, methods of an object, specific objects. Access can be from a ban and from permission.
Annunciator . The user is notified of an event. The client subscribes to the event himself or the technologist signs it.
Event monitor . A user-generated set of events is monitored.
Object Turnover . Business objects are driven along the production line.

5. Analytics Tools


5.1. Comparators


“Everything is Compared”
Some types of comparisons:


General comparison request: compare indicators P1, P2, ... PN having coordinates K1, K2, ..., respectively.

5.2. Decomposers


Some types of decompositions:


Example decomposition of the formula for calculating the value of an indicator:



5.3. Data mappers


Reports

This data is printed in a format specified externally. The report can also be viewed on the screen.
Some analytics systems view reports as the end result of analytics. I think this is not true. This is a semi-finished product. They usually look at the report, analyze something and select something from it and draw a conclusion. So is it not more correct to immediately formulate what you want to receive and implement it as a request.

Of course there are reports for external users. This is tax reporting, statistical ... Well, you have to. And reports are transmitted precisely as a report. Although it begs the possibility of transmitting not reports, but sets of indicators. And there, display them as you want. (A set of indicators) + (report form) + (coordinates of indicators on the report form) - that’s all you need to visualize the report. For this, you can even make a universal program.

Inquiries

This display of data as a result of user requirements for the composition of the data and the conditions imposed on them. The result is usually a table.

Graphs

This is a graphical display of data dependency. This can be a graph of dynamics, a graph of functional dependence. It can be diagrams - linear, columnar, circular. Types of images of relationships and dependencies - open space for the designer.

Hypercubes

This is an apparatus that considers data values ​​as packaged in cells, the coordinates of which are set by measurements - attributes that play the role of coordinates. For two dimensions we have the usual table, for three - a three-dimensional cube, for n-dimensions - an n-dimensional cube.

I consider hypercubes not as a finite representation, but as an intermediate one. Well, whoever comes to mind to stare at a multidimensional table from a multitude of data, trying to see something there. So formulate this something in the form of a request. Hypercube is a well-structured warehouse. But you can wander around the warehouse for a long time if you do not know the ultimate goal.

Hypercubes are usually visualized in the form of pivot tables.

Pivot tables

These are two-dimensional tables, allowing you to somehow see multidimensional tables. This is achieved by the fact that the graphs and rows of the tables are hierarchical - they allow decomposition (and vice versa - convolution).

A certain level of hierarchy corresponds to its level of data integration. In addition, it is possible to make graphs in rows and vice versa. This is usually done by Excel.

But it’s better to see once:



Scoreboard

Scoreboard - a form of displaying a group of indicators. This is the output form, which is desirable to be able to see at any time.
Trajectory scoreboard
The trajectories of the indicator group are presented.
Status board
The status of the subject in question is presented.
Real-time
Scoreboard This is a scoreboard in which data is updated in a user-defined update mode. The real-time status of the business object is presented. The real-time step can be adjusted by the user. For each type of user, this may be its own scoreboard.
For real-time implementation, the execution of each business transaction should cause a recalculation of indicators and after recalculation, the new values ​​should be displayed in the real-time panel.
Scoreboard statistics
For a given indicator, statistical characteristics of a number of indicator values ​​are given. The calculated relationships of the indicator with other indicators are given. Regression formulas for the value of indicators are given. The hypotheses about the relationships of the indicator are evaluated.

Groupings

This is a dataset grouped by a user-defined grouping hierarchy. Each top of the hierarchy is associated with a rule of admission to the top.

The trees

This is a mapping of data hierarchies.

And now for entertainment I will give occasions from life related to the topic.

6. Cases from IT life


6.1. This is not a drain tank.


It was a year in 2003. I work in the company STC Atlas. I lead the development of the venture project Banking Analytics. One day the chef invites to an urgent meeting. We are coming.We are introduced to new people (This, by the way, will be repeated several times. Business is business and many times it flashes up and the hope of meeting new interesting ideas fades). They are represented by doctors of technical (it seems) sciences. He says that his St. Petersburg company has developed an expert system as a semantic network and that he is ready to implement it in Belarus at the municipal, regional, republican levels. The network was displayed on the PC screen. The doctor spoke beautifully and convincingly. Energetic, assertive man. Well, all right, words of words ... Therefore, please read the documentation. Surprisingly, he gives us the terms of reference (TOR) for the system. We take a day to get to know each other and leave. I read TK. Fathers! What is not promised there: an accurate forecast of the exchange rate (by the way, this is an obsession with all scientists starting to try their hand at a practical economy), optimization of financial flows,improving the medical system, designing the President’s decision-making room (or below), etc. A lot more. It becomes clear to an experienced person what kind of system it is. But the man spoke so fascinatingly. I asked to see the knowledge database. They gave me a pretty dbf format database. There are dozens of three rules of the type in it: if a> b then c = d + f. I ask if this is all. Yes, that’s all!? .. But we can also introduce new rules, if necessary?! .. So this is the main catch.But we can introduce new rules, if necessary?! .. So this is the main catch.But we can introduce new rules, if necessary?! .. So this is the main catch.

Okay.The next day we’re going again. Our chief engineer, an experienced developer, asked a couple of critical questions. I also. The answers were unintelligible. And all with links to Academician Pospelov (who created the so-called situational management), and that "this is not your drain tank." The chief engineer said out loud that all this dregs, rose and left. The doctor of sciences became enraged and said that the kritkans are old-timers and they don’t understand modern innovations. I, as an old man, got up and left as well. I will learn the next day that a group of young people has been formed under the Doctorate of Sciences (students and yesterday’s students) and that there will be almost an introduction in the Brest region. Okay, work. Under the doctor, a group was also created in St. Petersburg. Time passes. I ask youth how are you. It turns out that they are already occupied with another topic - Wi-Fi. “What about the semantic network?” They shrug in response.I have not seen more Petersburgers. Yes, indeed, automation is not your drain tank. I think that he nevertheless sawed off something on bread and butter.

6.2.


This case can be called “Former physicists frolic.” The case occurred in Belarusbank. Sberbank (later it became Belarusbank) agreed with the company Namerek to develop a system of bank card payments. The case for those times is new and very relevant. Several stages were already well paid by the previous management of the bank. Now is the due date for the next stage. The executing company submitted a preliminary report and asks to pay for the current stage. And the leadership is now different. Sberbank has become a Belarusbank. And the new leadership wants to analyze the existing agreements. Is everything fulfilled there, are these contracts needed, etc. It is quite reasonable. The department of automation is asking to rate the work. After all, the topic concerns the introduction of an automated card settlement system. In the department, the assessment is entrusted to me.There are already several volumes of the “Preliminary Report” stage on this topic. I take the report (I still have it) I take the contract and begin to analyze. I open the report (sheets 300)) and see my familiar familiar formulas and equations. There is the Langevin equation from physics (something was considered as a random process and the apparatus of stochastic series was used) and many other equations. Continuous econophysics. Amazing.In roundabout ways I find out what kind of guys are in this company. Of course, former physicists. Quantums have come to the economy! Good!But you need to figure it out and it takes a lot of time. The problem is not only in formulas, but also in text sentences. I have not met such complex proposals and four-story titles before (I even wrote out some of them and tried to give my acquaintances for examination. The result is a stupor). Here is an example. The name of the contract topic: “Study and development of new types of services, forms of income generation and methods of entering into broad market operations, development of a system for managing and regulating the economy of these services, development of a system of legal and regulatory and technical acts protecting the rights of people who accept payments when servicing clients Sberbank system BelCard. " This name has not been fixed in my memory. What does the part "..., the development of the management system ...", ".., development of the legal system ..." interfere with.Why accusative case? Maybe just "development ..."? Then at least the grammatical construction of the sentence becomes clear ... And further on like that.

What to do?Then I take the contract and start looking literally point by point for traces of their implementation in the report. The contract appears as the output of the program contract. Where are the programs? - No programs. There are no programs, but there are formulas (Langevin equation, Fokker-Planck equation ...). And this is only a preliminary report. I went through all the points and realized that the contract was not executed on any point and, apparently, could not be executed. Note that I am not saying anything about the importance of the formulas and the mathematical model itself. I am only talking about fulfilling the contract. But he is not executed. I invited the management to refuse payment and said that I was ready to motivate the court to refuse to pay (at least I gave this type of paper for signature to my direct management and I don’t know what happened next). And, of course, he cited the arguments of his point of view. And indicatedthat even the previous payment was upfront (in my opinion, about $ 100,000). But what has fallen is gone. And then, sorry ...
What happened next, I don’t know. Soon I went to work at the BMRC (Belarusian Interbank Settlement Center). It turns out that immigrants from the former CC of the former Sberbank, and the current Belarusbank, worked there. We had a translator in our office, who often went abroad through the development of the Republic of Belarus payment system (RTGS), which was sponsored by Europe, and through which a lot of money flowed (but that's another matter). This translator brought gin and tonic from London and arranged a treat for some office workers. It was the beginning of perestroika, and for us, travel abroad, gin and tonic were new. I really enjoyed the drink. Jin was very fresh. We treated ourselves, relaxed, and began to talk about ... work, of course. And here I hear something interesting. We are talking about an agreement with Sberbank, to which the narrator (guest from outside) was involved before moving to the BMRC. It was supposedlygrandiose work, drawing, perhaps, to the Nobel Prize in economics. But then there were types in Belarusbank that hacked this project. I ask, and who hacked, specifically. And he looks at me and says: “Why, you’ve hacked it!” And it says on the face: “Do not pretend to be an innocent sheep.” I was taken aback. Then he remembered the ill-fated contract. It turns out that the contract, nevertheless, was hacked. (Then, working further, I realized that the same physicists want to attach the same topic to the BMRC. But this did not work either). The narrator himself was a very decent person. He approached everything in good faith and died of a heart attack at work. Rest in peace."And on the face it says:" Do not pretend to be an innocent sheep. " I was taken aback. Then he remembered the ill-fated contract. It turns out that the contract, nevertheless, was hacked. (Then, working further, I realized that the same physicists want to attach the same topic to the BMRC. But this did not work either). The narrator himself was a very decent person. He approached everything in good faith and died of a heart attack at work. Rest in peace."And on the face it says:" Do not pretend to be an innocent sheep. " I was taken aback. Then he remembered the ill-fated contract. It turns out that the contract, nevertheless, was hacked. (Then, working further, I realized that the same physicists want to attach the same topic to the BMRC. But this did not work either). The narrator himself was a very decent person. He approached everything in good faith and died of a heart attack at work. Rest in peace.

, , . . .

Source: https://habr.com/ru/post/461075/

More articles:


All Articles