Cards, accounts, 2 balances
As the holders of debit bank cards of Raiffeisenbank know, the card is only a tool for access to the account: it does not have its own balance, only those funds that are in the current account of the client are available. In this article we will talk about the architecture of a technical solution that allows cards and accounts in our bank to use a single balance. And also you will learn about how the project was organized, which allowed us to implement this architecture.
To begin with, we will show by example how the “card to account” and “card by itself” schemes differ. For example, through the banking system of the Russian Federation, you received a non-cash transfer - compensation from the state for the birth of a child. In the “card to account” scheme, this amount becomes available for card account transactions when the transfer is credited to your bank account. In the “card alone” scheme, it is necessary to transfer a certain amount of money from a regular account to a card account, after which funds can be spent with a card.
This seemingly elementary and logical functionality is provided by a rather complex technical solution, which is far from being implemented in every bank. For example, until 2014, in our bank, debit card balances lived separately from account balances: cards in processing, accounts near ABS. Balances synchronized in the morning of working days. Needless to say, such a scheme led to problems for clients: if a person terminated a deposit ahead of time, then he could spend / withdraw money on the card only tomorrow. And if today is Saturday, then “tomorrow” turned into Monday. In those days, a good third of the complaints about banki.ru were somehow connected with this topic. Inside the bank, this implementation also led to difficulties: it was necessary to check for any account expenditures on the account whether the client had spent the same money on the card. As a result, the majority of systems used a software emulator of the POS terminal, through which the card balance was checked and a special lock was put in processing. This check did not work correctly in all cases, often it was required manual intervention of the operator. For example, the card is temporarily blocked, 100 rubles on it, the client removes 100 from the account, the POS terminal emulator for such a lock gives an error, the amount does not block. If this problem is ignored, then after the expenditure transaction on the account is completed, the client unlocks the card and withdraws another 100 rubles.
This is not a complete list of the problems that the bank encountered when using the old architecture. At some point, it came to the realization that this architecture was holding back the development of bank services, and we decided to remake the “card-to-account” technical implementation.
')
The key question that needed to be answered was: in which system will the current value of the available balance live? Usually, banks have two candidates for this role - the processing system and ABS. Traditionally, in the processing of live card balances in the paradigm of "the card itself", and in the ABS - the balance of current accounts. Also traditionally, both of these systems are purchased box solutions from large suppliers with limited customization options. We decided to go the third way - to make a separate system to work with an available balance. Several factors have led us to this decision.
First, most of the “accounting” banking products (cash, interbank payments, insurance, and so on) do not live in our ABS. These are completely independent systems that load ready-made accounting entries into the ABS.
Secondly, we wanted to make a minimum of improvements in processing and ABS by the suppliers of these systems.
Thirdly, to control the balance of expenditure transactions with the account by these external systems, the ABS mechanism itself was not used.
Historically, the transaction day in the bank was arranged so that the current value of the account balance was calculated by a special system based on the balance at the beginning of the operating day in the ABS, which have already been confirmed in the ABS transaction / transaction, and intermediate operational data in some transaction systems (there were three such systems ). But how does the balance control for operations that are initiated inside the ABS are arranged? After all, she can not go to the third system to check the balance? For credit products (scheduled write-offs according to the payment schedule, overdue work), the ABS uses its balance sheet at the end of the day, since in the process all intermediate movements from third transaction systems are recorded in the ABS until the end of the trading day. For transactional products (placement of deposits, sending currency payments and so on) the balance control occurs either in manual mode (if the operation is manually entered into the ABS), or in an external system, which, after all the checks through the API, forms the operation in the ABS.
An experienced eye will see several drawbacks in this scheme. This is the implementation of the calculation of the current balance outside the ABS, and accounting for the calculation of the balance of the third systems data (which are stored on a separate database from the ABS). We have tried to turn our disadvantages into advantages. The fact that the current value of the available account balance is calculated outside the ABS has several advantages. The bank can, at its own discretion (without modifications of the ABS) change the balance calculation algorithms, we can somehow learn how to count the balance in the process of closing the day in the ABS. Let's talk about this moment in more detail. Our ABS is based on a concept from the last century: it provides for the process of closing the day, which lasts several hours, and during which nothing can be done with the ABS at all - either to enter transactions or look at data. But if the balance is considered to be non-ABS, then this problem can be circumvented!
So, we have decided on the target system, which will be the source of truth according to the available balance - this will be neither ABS, nor processing, but some kind of new system. Let's call it code letters NI. Based on the above-mentioned features of the work of the bank's systems, for the correct implementation of the NI system, you must perform the following steps:
How did this work? For example, on the client's account at the beginning of the day in the ABS, the balance was 100 rubles. The client withdrew through the cashier 50 rubles at 10 am. In the cash system there is an operation confirmed by the cashier, but no transactions have yet been made on it in the ABS (remembering that the ABS from the past century, she likes to take wiring in large batches, and not one by one). The cash system, according to its plan, plans to unload this and other confirmed operations by posting to the ABS at 12 o'clock in the afternoon with a single bundle. How to take into account in the balance of the account that 50 rubles is already gone? In the early 2000s, an “elegant” solution was found in the bank - the balance calculation system will open the connection to the cash register system base and select all transactions confirmed by the cashier but not yet unloaded in the ABS. And so on three different systems.
It is obvious that taking into account in the available balance of the account of the operation “at the moment”, which will later be unloaded in the ABS by a wiring block, is quite a typical task. All online banking products of the last one and a half decades has been directed in this direction. It turns out that as new similar systems appear, our balance calculation system would have to be connected to each of them. This is a dead end in terms of development. We realized this when there were only three systems, and even with them it was difficult to cope.
It seemed to us that the solution was more correct when a single repository of all account movements, but not yet unloaded into the ABS, is created. And all systems that need to account for their operations in an available balance in real time should connect to this repository through a set of APIs that should allow both to calculate the current value of the available balance and also change it by creating account movements. Since it was decided to call the new system NI, we will call its API “NI API”. At the same time, our old system for calculating the available balance should learn to look into this new NI storage, and the storage itself, for speed and reliability, decided to place on the same system where the ABS lives. In our case, this is the IBM i platform.
We decided to split all movements on the account conditionally into two types: hold and wiring. Holds are blocking specific amounts in customer accounts, which is not a fact that they will actually be written off. Holds can either have an expiration date or be indefinite. This type of movements was planned to be used for card authorizations, arrests of amounts by the decision of various supervisory authorities, and so on. The transactions are account movements that have actually already occurred, but have not yet reached the ABS.
It is also reasonable to add options in the API for creating account movements, in which movement is created only in case of successful checking of the available balance. The external system, as it were, says: “Create a spending operation on a client’s account for 100 rubles, if the current value of its balance allows it to be done”.
Imagine how in our example the cash system should work with the NI API. The client withdraws through the cashier 50 rubles at 10 am. The cash system appeals to the NI API with the request: “Create a spending operation on a client’s account for 50 rubles, if the current value of its balance allows it”. NI calculates the balance, it allows you to perform the operation, NI creates a movement for 50 rubles in its intermediate storage and responds to the cash system: “All is well, give out.” If the balance on the client’s account was less than 50 rubles, NI would answer: “It’s bad, the current balance value is 40 rubles”.
It is 12 o'clock, it's time for the cash register system to unload its stack of completed transactions by posting to the ABS. In theory, at the time of unloading operations, it should inform the NI API that these movements have already reached the ABS, and it is time to remove them from the intermediate repository of movements. And here begins the difficulty. According to the logic of the process of unloading a bundle of postings in the ABS, one commit is made for the whole pack. This means that it is also necessary to remove these movements from the NI repository with a single commit. Wiring bundles can be quite large, the time interval between commits can be quite large (tens of seconds maximum, but for us this is a lot), during which the client will have an incorrect balance on the account. Technical failures are also possible when the commit of a pack in the ABS is successful, and for some reason not in the NI API. Because of this, we risk getting incorrect values ​​of current balances on customer accounts, and this is inappropriate in a bank.
Fortunately, we identified these risks at the design stage of the NI system. For their leveling, they decided to “stretch” the NI functional to the ABS. The Posting Creation API has added a required field with the identifier of the posting bundle into which the external system wants to combine these postings. A separate function was also added: “Write a wiring stack in the ABS”, at the call of which NI wrote the entries in the ABS, with a single commit in its storage and in the ABS. This was greatly helped by the fact that the NI and ABS databases were located, in fact, in the same DBMS. Thus, at 12 o'clock, the cash register system simply said to the NI API: “Unload a bundle of NI transactions in the ABS and let me know about the problems”.
Problems are very rare, but have arisen. For example, when a bailiff came to the bank between the addition of the spending movement on an account in NI and the unloading of this movement to the ABS. In such cases, the cash register system gave the user information about the problem transaction, and it was necessary to handle the situation manually.
The NI API was developed, which provided external systems with several logically grouped APIs:
Technically, the API was a set of programs written in the programming language Free format ILE RPG using SQL. To use APIs outside the IBM I platform, programs were wrapped in stored procedures based on DB2 for i, with which external systems could work using standard ODBC, JDBC, and so on. Access to the NI API was also implemented from an enterprise ESB integration bus, which, in turn, also accessed the ODBC / JDBC API.
Since we decided to refine the previous system of calculating the available balance so that she could work with the new intermediate repository of movements on the NI accounts, we did not need to refine all the bank's IT systems. But we needed to redo the three systems mentioned (cash, ruble payments, legal entities' conversions) so that they would start using the NI API for intermediate movements, and when calculating the balance we would no longer turn to the transactional data of these systems. In fact, this turned out to be a rather complicated refinement, since in each system it was necessary to remake the main logic for processing the life cycle of an operation. For the system of ruble payments, we had to redo the internal system architecture, moving from two-tier to three-tier. This was required in order to centralize the logic of working with the NI API on the application server.
But the main difficulties at this stage were connected not only with technical and architectural, but also with business aspects. These systems are highly demanded by customers in terms of improvements, the backlog always has a lot of ideas for development, optimization of products and processes. The need to perform such large-scale improvements without a tangible, from a business point of view, benefits was perceived by customers without enthusiasm. Recall that the essence of the changes was to ensure that the system for calculating the available balance ceased to go to the bases of these systems, and they themselves notified the centralized system NI of account movements through the API. For business, it has almost no value.
The successful combination of circumstances helped us find a way out of the current situation and persuade the business to our side. Just at that time, a module on off-balance accounting of plastic cards was introduced in the cash register system. There are many cards, the processes of their movement are quite complicated, which means a large number of transactions on off-balance sheet accounts. There is a significant flaw in the old mechanism for generating transactions from external systems in the ABS: the packages of transactions created in the ABS must be accepted manually using the user interface of the ABS itself. And this can be done by a user with special rights that are granted only to a limited circle of people. And in the above-mentioned project for out-of-balance, there was a fierce debate between various departments about who would accept these packages, to whom what rights can or should not be given. We have proposed a solution: in the NI API, to make an option that will allow forming the posting packages in the ABS in an accepted form. Of course, for this, the entire module must be transferred to a new API. Everybody really liked this idea, they gave us a go-ahead for rework.
Implementing the mechanism for creating already accepted posting packages has, technically, proved to be a rather difficult but interesting task. We analyzed the call stack of the standard ABS programs for acceptance of transactions, tracked the DBMS transaction logs to manipulate data. As a result, we learned to deceive the ABS and create accepted transactions. The cash module for off-balance started on the NI API, the postings flowed to the ABS without operator acceptance, and we got an understanding of how to encourage businesses to switch to NI. The secret turned out to be simple - it is necessary to offer the business during migration additional functionality that will simplify their processes or in some other way improve the product. And here the map went to us ...
As a result, NI was overwhelmed with the functionality of creating accounts for blocked accounts (relevant for write-offs by court decisions and so on, in the old process, accounts were temporarily unblocked manually), with calculations of a balance for priority, overdraft and other tricks. This allowed the business to receive its buns and significantly optimize the processes during the migration of functionality. So, cash and ruble payments - done, remained currency exchange transactions of corporate clients.
And with this system, we are not going well. It had very few operations, there was no need for the development of functionality, optimization did not promise significant benefits. As a result, we did not manage to convince partners of the need for improvements, and we waved to her hand. Until now, when calculating the balance on the accounts of corporate clients, a connection is opened with its DBMS. But then it did not upset us, because we achieved our main goal - on the accounts of individuals, all data for calculating the balance were concentrated in a single IBM i DBMS, the calculation worked quickly (about 5 milliseconds per account) and reliably.
On closer examination, this task turned out to be surprisingly simple. Since our ABS in the process of closing the day was not able to take in any movement on the accounts, the bank just froze at night. No account transactions were performed. Therefore, our changes to NI did not need to be synchronized with any systems other than the ABS itself. As a result, we made minor improvements in the ABS, inserting a call to the special NI control programs at the very beginning (day-> night) and at the very end (night-> day) of the process of closing the trading day.
The program “day-> night” simply copied the ABS tables needed for calculating the balance in a separate library in the same DBMS. The primitive INSERT SELECT construct was used. All copying took about 20 minutes. After copying was completed, NI understood that now it was necessary to go not to the ABS table, but to its copy.
The program “night-> day” worked out instantly, simply switching NI to work with the main base of the ABS.
This solution allowed moving between phases of the day without interrupting the service. All the movements on the bills that external systems wanted to make during the closure of the day of the ABS, accumulated in the NI. Further, after opening the ABS day, the external systems had to be given a command to unload the entries to the ABS through the NI API. This is not very convenient for external systems, so the STP mechanism (from straight through processing) was created later, in which the external system only creates the wiring in NI, and then he himself groups them into packages and unloads them into the ABS, when is available.
The implementation of the day-night scheme is not devoid of grace. We used the IBM i standard context handling mechanism — a list of libraries. Two libraries were created - day and night. They created aliases for day and night copies of the ABS tables, respectively. For each table and its night copy, the name of the alias is the same. In all SQL expressions, aliases were used to access the ABS data.
Before processing each NI API call, look into a special Data Area object (very fast read-write) of the IBM i system to see if the mode has changed from the last API call in the process. As you might guess, the mode in the Data Area changes the program day-> night and night-> day. If the mode is changed, the program simply replaces the library with aliases in the list of process libraries; this is an extremely fast operation.
As a result, all NI API client systems were able to calculate the balance and carry out operations 24 hours a day without interruption, not really worrying about the operating day in the ABS. For us it was a revolution.
At this point, we did not stop the development of the night / day transition logic. Those 20 minutes to copy tables using SQL did not give us peace of mind. A replication scheme was designed and implemented using the MIMIX high availability solution. The idea is that a night copy is created not at the beginning of the closure of the day of the ABS, but at the opening of the day. During the day, this copy, in near real-time mode, is synchronized with the ABS data. And in order to be ready for any blows of fate, we replicate on two IBM i systems at once: the main one and the backup one.
At the moment we turn day-> night and back almost instantly, the service is not interrupted for a second.
As you know, the life cycle of a bank card transaction consists of two main stages: authorization and clearing. However, in some cases, authorization may not be.
From the point of view of the available balance, upon authorization, it is necessary to check the available balance on the client's account, block the transaction amount with a hold and respond to the payment system about your decision: approved or not. It is also advisable to send a notification about the operation to the client, but this will be discussed in the chapter on Internet banking. When clearing, you need to try to find the hold that was delivered earlier during authorization, delete it and create a transaction on the client's account.
Our processing system at the start of the project was able to apply for verification of the available balance to external systems during authorization. This functionality was created earlier, when combined with Impexbank, since Impex had a card-to-account product. In the process of authorizing a transaction, after successfully completing pure card checks (cryptography, PIN, and so on), the authorization system sends an ISO-8583 message to an external system for financial authorization and waits a certain number of seconds for a response. We were required to create a special NI module that could communicate with the processing system using this protocol. Accept the message, parse the fields, call the NI API, form the response. A rather simple task, but only at first glance.
The first difficulty lurked us with online customer account replenishment operations with a bank card. These are payments to ATMs and incoming transfers from card to card. According to the logic of these operations, authorization and clearing can take place on different banking days. Suppose a client with a zero balance on the account receives a transfer from a third-party bank on April 1 at 10:00. At 10:05 the client starts these funds for early repayment of the loan. Clearing on this operation will come on April 2. If, on April 1 (during authorization), we do not make an accounting entry with the account replenishment, but simply raise the balance with a “positive hold”, then after the loan is repaid on the account in the accounting, there will be a negative balance, technical overdraft, which will be closed only after clearing. In the bank accounting, technical overdrafts are avoided in every way, and we have only one way out - to make postings on authorization for replenishment operations.
From the first difficulty follows the second - exchange differences. When a transfer comes from a foreign bank, the amounts for authorization and clearing may vary due to currency fluctuations. We illustrate with a real example. On June 11, our client received a transfer to a card from a Belarusian bank, the transfer amount was 15.00 Belarusian rubles. The bank cannot physically support conversion rates to all currencies of the world for card transactions, we conduct courses in three currencies - rubles, dollars, euros. The payment system for authorization recalculates the transfer amount at its own rate in the currency of the issuer, in this case it was the Russian rubles, the amount was 455.50. The client’s account is opened in rubles, so we conducted a deposit for him at 455.50 rubles. From this point on, the client can dispose of this money in any way.
On June 14th, we received clearing from the payment system, and for this operation we transferred 7.16 Euros to our bank. From the point of view of accounting, it turns out that we carried out a time-distributed conversion operation to the client: for 455.50 rubles we bought 7.16 Euros. , . 14 , 7,16 445,35 . , 10,15 . , , . . , , . . , . — « ». , , .
. . , , . , SMS , . , . , , . SMS, .
, . - , . , , - NI API ( NI - ). — NI , . . :
, NI API — . , , - . , , . - NI API . , . NI , , , , (, ). -, . NI .
- : / . - . , . . ? - !
. ! - , , . . , - . ; , .
NI - . , . - NI API . . , - - .
SMS- . NI SMS - . - . NI IBM WebSphere MQ. NI , : , , . , , . SMS- , . , , , SMS , .
- , .
, , . . «Business 24x7», /. - .
, , , . !
:
To begin with, we will show by example how the “card to account” and “card by itself” schemes differ. For example, through the banking system of the Russian Federation, you received a non-cash transfer - compensation from the state for the birth of a child. In the “card to account” scheme, this amount becomes available for card account transactions when the transfer is credited to your bank account. In the “card alone” scheme, it is necessary to transfer a certain amount of money from a regular account to a card account, after which funds can be spent with a card.
This seemingly elementary and logical functionality is provided by a rather complex technical solution, which is far from being implemented in every bank. For example, until 2014, in our bank, debit card balances lived separately from account balances: cards in processing, accounts near ABS. Balances synchronized in the morning of working days. Needless to say, such a scheme led to problems for clients: if a person terminated a deposit ahead of time, then he could spend / withdraw money on the card only tomorrow. And if today is Saturday, then “tomorrow” turned into Monday. In those days, a good third of the complaints about banki.ru were somehow connected with this topic. Inside the bank, this implementation also led to difficulties: it was necessary to check for any account expenditures on the account whether the client had spent the same money on the card. As a result, the majority of systems used a software emulator of the POS terminal, through which the card balance was checked and a special lock was put in processing. This check did not work correctly in all cases, often it was required manual intervention of the operator. For example, the card is temporarily blocked, 100 rubles on it, the client removes 100 from the account, the POS terminal emulator for such a lock gives an error, the amount does not block. If this problem is ignored, then after the expenditure transaction on the account is completed, the client unlocks the card and withdraws another 100 rubles.
This is not a complete list of the problems that the bank encountered when using the old architecture. At some point, it came to the realization that this architecture was holding back the development of bank services, and we decided to remake the “card-to-account” technical implementation.
')
The key question that needed to be answered was: in which system will the current value of the available balance live? Usually, banks have two candidates for this role - the processing system and ABS. Traditionally, in the processing of live card balances in the paradigm of "the card itself", and in the ABS - the balance of current accounts. Also traditionally, both of these systems are purchased box solutions from large suppliers with limited customization options. We decided to go the third way - to make a separate system to work with an available balance. Several factors have led us to this decision.
First, most of the “accounting” banking products (cash, interbank payments, insurance, and so on) do not live in our ABS. These are completely independent systems that load ready-made accounting entries into the ABS.
Secondly, we wanted to make a minimum of improvements in processing and ABS by the suppliers of these systems.
Thirdly, to control the balance of expenditure transactions with the account by these external systems, the ABS mechanism itself was not used.
Historically, the transaction day in the bank was arranged so that the current value of the account balance was calculated by a special system based on the balance at the beginning of the operating day in the ABS, which have already been confirmed in the ABS transaction / transaction, and intermediate operational data in some transaction systems (there were three such systems ). But how does the balance control for operations that are initiated inside the ABS are arranged? After all, she can not go to the third system to check the balance? For credit products (scheduled write-offs according to the payment schedule, overdue work), the ABS uses its balance sheet at the end of the day, since in the process all intermediate movements from third transaction systems are recorded in the ABS until the end of the trading day. For transactional products (placement of deposits, sending currency payments and so on) the balance control occurs either in manual mode (if the operation is manually entered into the ABS), or in an external system, which, after all the checks through the API, forms the operation in the ABS.
An experienced eye will see several drawbacks in this scheme. This is the implementation of the calculation of the current balance outside the ABS, and accounting for the calculation of the balance of the third systems data (which are stored on a separate database from the ABS). We have tried to turn our disadvantages into advantages. The fact that the current value of the available account balance is calculated outside the ABS has several advantages. The bank can, at its own discretion (without modifications of the ABS) change the balance calculation algorithms, we can somehow learn how to count the balance in the process of closing the day in the ABS. Let's talk about this moment in more detail. Our ABS is based on a concept from the last century: it provides for the process of closing the day, which lasts several hours, and during which nothing can be done with the ABS at all - either to enter transactions or look at data. But if the balance is considered to be non-ABS, then this problem can be circumvented!
So, we have decided on the target system, which will be the source of truth according to the available balance - this will be neither ABS, nor processing, but some kind of new system. Let's call it code letters NI. Based on the above-mentioned features of the work of the bank's systems, for the correct implementation of the NI system, you must perform the following steps:
- Refuse, when calculating the balance, from analyzing data from transactional systems (we mentioned three such systems - cash, ruble payments, legal entities' conversions). This is necessary for reliable and fast work of card authorizations, so as not to go to third-party DBMS.
- Switch to NI all systems that use the current specialized system to calculate the balance. Or connect the old system to NI, which is easier; This we finally chose.
- Teach NI to work somehow in the process of closing the ABS day.
- Integrate NI with card processing.
- See how all these changes affect online banking.
Waiver of data from third systems when calculating account balance
How did this work? For example, on the client's account at the beginning of the day in the ABS, the balance was 100 rubles. The client withdrew through the cashier 50 rubles at 10 am. In the cash system there is an operation confirmed by the cashier, but no transactions have yet been made on it in the ABS (remembering that the ABS from the past century, she likes to take wiring in large batches, and not one by one). The cash system, according to its plan, plans to unload this and other confirmed operations by posting to the ABS at 12 o'clock in the afternoon with a single bundle. How to take into account in the balance of the account that 50 rubles is already gone? In the early 2000s, an “elegant” solution was found in the bank - the balance calculation system will open the connection to the cash register system base and select all transactions confirmed by the cashier but not yet unloaded in the ABS. And so on three different systems.
It is obvious that taking into account in the available balance of the account of the operation “at the moment”, which will later be unloaded in the ABS by a wiring block, is quite a typical task. All online banking products of the last one and a half decades has been directed in this direction. It turns out that as new similar systems appear, our balance calculation system would have to be connected to each of them. This is a dead end in terms of development. We realized this when there were only three systems, and even with them it was difficult to cope.
It seemed to us that the solution was more correct when a single repository of all account movements, but not yet unloaded into the ABS, is created. And all systems that need to account for their operations in an available balance in real time should connect to this repository through a set of APIs that should allow both to calculate the current value of the available balance and also change it by creating account movements. Since it was decided to call the new system NI, we will call its API “NI API”. At the same time, our old system for calculating the available balance should learn to look into this new NI storage, and the storage itself, for speed and reliability, decided to place on the same system where the ABS lives. In our case, this is the IBM i platform.
We decided to split all movements on the account conditionally into two types: hold and wiring. Holds are blocking specific amounts in customer accounts, which is not a fact that they will actually be written off. Holds can either have an expiration date or be indefinite. This type of movements was planned to be used for card authorizations, arrests of amounts by the decision of various supervisory authorities, and so on. The transactions are account movements that have actually already occurred, but have not yet reached the ABS.
It is also reasonable to add options in the API for creating account movements, in which movement is created only in case of successful checking of the available balance. The external system, as it were, says: “Create a spending operation on a client’s account for 100 rubles, if the current value of its balance allows it to be done”.
Imagine how in our example the cash system should work with the NI API. The client withdraws through the cashier 50 rubles at 10 am. The cash system appeals to the NI API with the request: “Create a spending operation on a client’s account for 50 rubles, if the current value of its balance allows it”. NI calculates the balance, it allows you to perform the operation, NI creates a movement for 50 rubles in its intermediate storage and responds to the cash system: “All is well, give out.” If the balance on the client’s account was less than 50 rubles, NI would answer: “It’s bad, the current balance value is 40 rubles”.
It is 12 o'clock, it's time for the cash register system to unload its stack of completed transactions by posting to the ABS. In theory, at the time of unloading operations, it should inform the NI API that these movements have already reached the ABS, and it is time to remove them from the intermediate repository of movements. And here begins the difficulty. According to the logic of the process of unloading a bundle of postings in the ABS, one commit is made for the whole pack. This means that it is also necessary to remove these movements from the NI repository with a single commit. Wiring bundles can be quite large, the time interval between commits can be quite large (tens of seconds maximum, but for us this is a lot), during which the client will have an incorrect balance on the account. Technical failures are also possible when the commit of a pack in the ABS is successful, and for some reason not in the NI API. Because of this, we risk getting incorrect values ​​of current balances on customer accounts, and this is inappropriate in a bank.
Fortunately, we identified these risks at the design stage of the NI system. For their leveling, they decided to “stretch” the NI functional to the ABS. The Posting Creation API has added a required field with the identifier of the posting bundle into which the external system wants to combine these postings. A separate function was also added: “Write a wiring stack in the ABS”, at the call of which NI wrote the entries in the ABS, with a single commit in its storage and in the ABS. This was greatly helped by the fact that the NI and ABS databases were located, in fact, in the same DBMS. Thus, at 12 o'clock, the cash register system simply said to the NI API: “Unload a bundle of NI transactions in the ABS and let me know about the problems”.
Problems are very rare, but have arisen. For example, when a bailiff came to the bank between the addition of the spending movement on an account in NI and the unloading of this movement to the ABS. In such cases, the cash register system gave the user information about the problem transaction, and it was necessary to handle the situation manually.
The NI API was developed, which provided external systems with several logically grouped APIs:
- Receipt of information on the account (balance, statuses and so on).
- Work with packs (creation, removal, unloading in ABS).
- Work with separate postings in packs.
- Work with Holds.
Technically, the API was a set of programs written in the programming language Free format ILE RPG using SQL. To use APIs outside the IBM I platform, programs were wrapped in stored procedures based on DB2 for i, with which external systems could work using standard ODBC, JDBC, and so on. Access to the NI API was also implemented from an enterprise ESB integration bus, which, in turn, also accessed the ODBC / JDBC API.
Switch some systems to NI
Since we decided to refine the previous system of calculating the available balance so that she could work with the new intermediate repository of movements on the NI accounts, we did not need to refine all the bank's IT systems. But we needed to redo the three systems mentioned (cash, ruble payments, legal entities' conversions) so that they would start using the NI API for intermediate movements, and when calculating the balance we would no longer turn to the transactional data of these systems. In fact, this turned out to be a rather complicated refinement, since in each system it was necessary to remake the main logic for processing the life cycle of an operation. For the system of ruble payments, we had to redo the internal system architecture, moving from two-tier to three-tier. This was required in order to centralize the logic of working with the NI API on the application server.
But the main difficulties at this stage were connected not only with technical and architectural, but also with business aspects. These systems are highly demanded by customers in terms of improvements, the backlog always has a lot of ideas for development, optimization of products and processes. The need to perform such large-scale improvements without a tangible, from a business point of view, benefits was perceived by customers without enthusiasm. Recall that the essence of the changes was to ensure that the system for calculating the available balance ceased to go to the bases of these systems, and they themselves notified the centralized system NI of account movements through the API. For business, it has almost no value.
The successful combination of circumstances helped us find a way out of the current situation and persuade the business to our side. Just at that time, a module on off-balance accounting of plastic cards was introduced in the cash register system. There are many cards, the processes of their movement are quite complicated, which means a large number of transactions on off-balance sheet accounts. There is a significant flaw in the old mechanism for generating transactions from external systems in the ABS: the packages of transactions created in the ABS must be accepted manually using the user interface of the ABS itself. And this can be done by a user with special rights that are granted only to a limited circle of people. And in the above-mentioned project for out-of-balance, there was a fierce debate between various departments about who would accept these packages, to whom what rights can or should not be given. We have proposed a solution: in the NI API, to make an option that will allow forming the posting packages in the ABS in an accepted form. Of course, for this, the entire module must be transferred to a new API. Everybody really liked this idea, they gave us a go-ahead for rework.
Implementing the mechanism for creating already accepted posting packages has, technically, proved to be a rather difficult but interesting task. We analyzed the call stack of the standard ABS programs for acceptance of transactions, tracked the DBMS transaction logs to manipulate data. As a result, we learned to deceive the ABS and create accepted transactions. The cash module for off-balance started on the NI API, the postings flowed to the ABS without operator acceptance, and we got an understanding of how to encourage businesses to switch to NI. The secret turned out to be simple - it is necessary to offer the business during migration additional functionality that will simplify their processes or in some other way improve the product. And here the map went to us ...
As a result, NI was overwhelmed with the functionality of creating accounts for blocked accounts (relevant for write-offs by court decisions and so on, in the old process, accounts were temporarily unblocked manually), with calculations of a balance for priority, overdraft and other tricks. This allowed the business to receive its buns and significantly optimize the processes during the migration of functionality. So, cash and ruble payments - done, remained currency exchange transactions of corporate clients.
And with this system, we are not going well. It had very few operations, there was no need for the development of functionality, optimization did not promise significant benefits. As a result, we did not manage to convince partners of the need for improvements, and we waved to her hand. Until now, when calculating the balance on the accounts of corporate clients, a connection is opened with its DBMS. But then it did not upset us, because we achieved our main goal - on the accounts of individuals, all data for calculating the balance were concentrated in a single IBM i DBMS, the calculation worked quickly (about 5 milliseconds per account) and reliably.
Teach NI to work in the process of closing the trading day
On closer examination, this task turned out to be surprisingly simple. Since our ABS in the process of closing the day was not able to take in any movement on the accounts, the bank just froze at night. No account transactions were performed. Therefore, our changes to NI did not need to be synchronized with any systems other than the ABS itself. As a result, we made minor improvements in the ABS, inserting a call to the special NI control programs at the very beginning (day-> night) and at the very end (night-> day) of the process of closing the trading day.
The program “day-> night” simply copied the ABS tables needed for calculating the balance in a separate library in the same DBMS. The primitive INSERT SELECT construct was used. All copying took about 20 minutes. After copying was completed, NI understood that now it was necessary to go not to the ABS table, but to its copy.
The program “night-> day” worked out instantly, simply switching NI to work with the main base of the ABS.
This solution allowed moving between phases of the day without interrupting the service. All the movements on the bills that external systems wanted to make during the closure of the day of the ABS, accumulated in the NI. Further, after opening the ABS day, the external systems had to be given a command to unload the entries to the ABS through the NI API. This is not very convenient for external systems, so the STP mechanism (from straight through processing) was created later, in which the external system only creates the wiring in NI, and then he himself groups them into packages and unloads them into the ABS, when is available.
The implementation of the day-night scheme is not devoid of grace. We used the IBM i standard context handling mechanism — a list of libraries. Two libraries were created - day and night. They created aliases for day and night copies of the ABS tables, respectively. For each table and its night copy, the name of the alias is the same. In all SQL expressions, aliases were used to access the ABS data.
Before processing each NI API call, look into a special Data Area object (very fast read-write) of the IBM i system to see if the mode has changed from the last API call in the process. As you might guess, the mode in the Data Area changes the program day-> night and night-> day. If the mode is changed, the program simply replaces the library with aliases in the list of process libraries; this is an extremely fast operation.
As a result, all NI API client systems were able to calculate the balance and carry out operations 24 hours a day without interruption, not really worrying about the operating day in the ABS. For us it was a revolution.
At this point, we did not stop the development of the night / day transition logic. Those 20 minutes to copy tables using SQL did not give us peace of mind. A replication scheme was designed and implemented using the MIMIX high availability solution. The idea is that a night copy is created not at the beginning of the closure of the day of the ABS, but at the opening of the day. During the day, this copy, in near real-time mode, is synchronized with the ABS data. And in order to be ready for any blows of fate, we replicate on two IBM i systems at once: the main one and the backup one.
At the moment we turn day-> night and back almost instantly, the service is not interrupted for a second.
Integration with bank card processing
As you know, the life cycle of a bank card transaction consists of two main stages: authorization and clearing. However, in some cases, authorization may not be.
From the point of view of the available balance, upon authorization, it is necessary to check the available balance on the client's account, block the transaction amount with a hold and respond to the payment system about your decision: approved or not. It is also advisable to send a notification about the operation to the client, but this will be discussed in the chapter on Internet banking. When clearing, you need to try to find the hold that was delivered earlier during authorization, delete it and create a transaction on the client's account.
Our processing system at the start of the project was able to apply for verification of the available balance to external systems during authorization. This functionality was created earlier, when combined with Impexbank, since Impex had a card-to-account product. In the process of authorizing a transaction, after successfully completing pure card checks (cryptography, PIN, and so on), the authorization system sends an ISO-8583 message to an external system for financial authorization and waits a certain number of seconds for a response. We were required to create a special NI module that could communicate with the processing system using this protocol. Accept the message, parse the fields, call the NI API, form the response. A rather simple task, but only at first glance.
The first difficulty lurked us with online customer account replenishment operations with a bank card. These are payments to ATMs and incoming transfers from card to card. According to the logic of these operations, authorization and clearing can take place on different banking days. Suppose a client with a zero balance on the account receives a transfer from a third-party bank on April 1 at 10:00. At 10:05 the client starts these funds for early repayment of the loan. Clearing on this operation will come on April 2. If, on April 1 (during authorization), we do not make an accounting entry with the account replenishment, but simply raise the balance with a “positive hold”, then after the loan is repaid on the account in the accounting, there will be a negative balance, technical overdraft, which will be closed only after clearing. In the bank accounting, technical overdrafts are avoided in every way, and we have only one way out - to make postings on authorization for replenishment operations.
From the first difficulty follows the second - exchange differences. When a transfer comes from a foreign bank, the amounts for authorization and clearing may vary due to currency fluctuations. We illustrate with a real example. On June 11, our client received a transfer to a card from a Belarusian bank, the transfer amount was 15.00 Belarusian rubles. The bank cannot physically support conversion rates to all currencies of the world for card transactions, we conduct courses in three currencies - rubles, dollars, euros. The payment system for authorization recalculates the transfer amount at its own rate in the currency of the issuer, in this case it was the Russian rubles, the amount was 455.50. The client’s account is opened in rubles, so we conducted a deposit for him at 455.50 rubles. From this point on, the client can dispose of this money in any way.
On June 14th, we received clearing from the payment system, and for this operation we transferred 7.16 Euros to our bank. From the point of view of accounting, it turns out that we carried out a time-distributed conversion operation to the client: for 455.50 rubles we bought 7.16 Euros. , . 14 , 7,16 445,35 . , 10,15 . , , . . , , . . , . — « ». , , .
. . , , . , SMS , . , . , , . SMS, .
, . - , . , , - NI API ( NI - ). — NI , . . :
- API .
- «» API , , .
, NI API — . , , - . , , . - NI API . , . NI , , , , (, ). -, . NI .
-
- : / . - . , . . ? - !
. ! - , , . . , - . ; , .
NI - . , . - NI API . . , - - .
SMS- . NI SMS - . - . NI IBM WebSphere MQ. NI , : , , . , , . SMS- , . , , , SMS , .
- , .
Conclusion
, , . . «Business 24x7», /. - .
, , , . !
:
- 70% NI.
- API 5 , 8. ( ).
Source: https://habr.com/ru/post/334242/
All Articles