Secrets of the resource forest: how we gathered together the IT structure of VTB
Even if you combine networks and infrastructures of at least two grocery stores, this can easily turn into a difficult task. Suddenly, various nuances related to network organization will emerge - for example, D-Link bricked up in an unknown back room, which works by pretending that it is the core of the network. When it comes to companies of the scale of VTB and VTB24, everything goes to a new level - requirements from a huge number of departments, managers, security services are added ... We will tell about how we dealt with this under the cut.
So, we were faced with the task of combining the infrastructures of several large companies in the context of a decent set of legacy tools that are capricious for architecture.
')
The basis of the created infrastructure is the āresource forestā - a system that combines an instant messenger, a single address book, portal, file service and other key resources. Initially, the resource forest was conceived as a framework for the interaction of various legal entities in the same banking group. For the merger of a group of companies, this option turned out to be optimal.
Technically, it was necessary to solve two problems: to create a common network segment where the services would be located, and to provide a single domain authentication. And all this is painless and without disrupting the mode of operation of banks.
All network segments are routed in the territories of two data centers, and engineers had to combine resources. For each bank, they created a separate segment with transparent routing, and the firewalls were already placed on the border between these segments and the internal resources of the companies themselves.
Resource forest implies a separate domain and the presence of a trust relationship with the group members. Fortunately, we managed to find a pool of network addresses, which for all merged banks do not overlap with local ones. As a result, at the sites of two data centers, all segments of each of the banks interact transparently in a single address space. This multi-layered structure helps ensure the most transparent routing.
It happens that the application server needs to connect to the client, initiating a call-back in the direction of his machine. The client may be any workstation with its peripherals, for example, a video camera. The bank has a number of reliable, but not new legacy-applications that require such an opportunity. We laid transparent call-back access into the architecture - it gives painless compatibility with legacy systems and helps to gradually move away from legacy in stages, without breaking the current functionality.
The uniform address space also allowed us to avoid non-standard workarounds when working with Active Directory, due to the fact that Microsoft does not officially support the creation of trust relationships between independent servers in the presence of NAT. We tried to work in several address spaces in the test mode, but decided to abandon the officially not supported implementation variant.
The most interesting thing is how we organized the routing. Puzzled for a long time, but in the end everything turned out quite simple. The segments of each bank in the infrastructure core are VPNs spread over MPLS and present at two sites. We planned to route between them through MPLS label import / export policies. But together, these components refused to work harmoniously, because we have different equipment at different sites. If you write on each problem in technical support vendor, you can disrupt all possible terms of the project (and in our case they were very short). All this did not suit us at all. In the end, we just made separate physical links between VPNs.
Another difficulty was to set up rules for filtering routes between two sites - in the process of uniting VPN networks there is a possibility of unexpected āloopsā that are fraught with a fall in the network infrastructure.
To simplify access to services of companies from every corner of the āresource forestā, we have implemented the concept of ādata martsā.
Each of the banks publishes in its network segment a set of proxies to access their tasks. That is, in the showcase each bank publishes those applications that must be used together with other participants. For example, VTB24 delivered a gateway to its segment for access to their VDI.
After the gateway is activated on the firewall, the corresponding rules are opened, and users connected to the āforestā have the opportunity to work at VTB24 virtual desktops. We have identified "windows" for terminal and web applications. Terminal applications are published on the Citrix farm, and for the web, technology has been used for many years in the bank. About 70 applications are integrated with it. It was built on the basis of the Oracle Access Manager product.
With applications from other banks, everything is much more complicated, since they are not integrated with Access Manager. Therefore, it was decided to put the windows running in reverse proxy mode.
Another prerequisite was the domain authentication requested by the storefront. For correct forwarding through it, HAProxy was chosen - free software included in the Red Hat Enterprise Server package. This is a great finished product, with which you can do both reverse HTTP Proxy, and proxies of many other things. You can proxy not only HTTP, but also any other traffic, including private protocols, which are used in the application for data exchange.
Another important task that we are now solving is to bring the communication system to a single standard in our companies. With all its conservatism, the banking sector requires very fast decision-making and well-established communications. Now we began to use Skype for Business. Major deployment work is almost complete.
Between the individual servers with Skype there is a federated relationship, that is, a trusted relationship between independent peer servers. Thanks to the integration and synchronization of accounts, you can now easily find a contact for contact. Skype for Business is integrated with telephony and video conferencing. You can call a regular phone with Skype and back on the PSTN if the employee has got a number in his Skype account. If you have a headset and a video camera, you can call the meeting room from your workplace via Skype for Business. Naturally, all types of communication must comply with strict safety standards.
In addition to the specific tasks associated with the functioning of banks as commercial structures, in the new architecture, we simultaneously solved a lot of pressing problems. For example, synchronization of passwords, accounts and mail.
New architecture with a resource forest is a kind of framework, the foundation for further information exchange with the groupās daughters. This includes both applications (for example, mail) and, say, access to the portal with contacts. In the future, based on the new architecture, standardized sets of requirements will be formed for connecting to the platform, for example, subsidiary banks. This will help simplify the entire procedure.
Now we have tried to provide all possible nuances of the development of the architecture and thus ensure the absence of problems with integration in the future.
So, we were faced with the task of combining the infrastructures of several large companies in the context of a decent set of legacy tools that are capricious for architecture.
')
The basis of the created infrastructure is the āresource forestā - a system that combines an instant messenger, a single address book, portal, file service and other key resources. Initially, the resource forest was conceived as a framework for the interaction of various legal entities in the same banking group. For the merger of a group of companies, this option turned out to be optimal.
Technically, it was necessary to solve two problems: to create a common network segment where the services would be located, and to provide a single domain authentication. And all this is painless and without disrupting the mode of operation of banks.
Network infrastructure
All network segments are routed in the territories of two data centers, and engineers had to combine resources. For each bank, they created a separate segment with transparent routing, and the firewalls were already placed on the border between these segments and the internal resources of the companies themselves.
Resource forest implies a separate domain and the presence of a trust relationship with the group members. Fortunately, we managed to find a pool of network addresses, which for all merged banks do not overlap with local ones. As a result, at the sites of two data centers, all segments of each of the banks interact transparently in a single address space. This multi-layered structure helps ensure the most transparent routing.
It happens that the application server needs to connect to the client, initiating a call-back in the direction of his machine. The client may be any workstation with its peripherals, for example, a video camera. The bank has a number of reliable, but not new legacy-applications that require such an opportunity. We laid transparent call-back access into the architecture - it gives painless compatibility with legacy systems and helps to gradually move away from legacy in stages, without breaking the current functionality.
The uniform address space also allowed us to avoid non-standard workarounds when working with Active Directory, due to the fact that Microsoft does not officially support the creation of trust relationships between independent servers in the presence of NAT. We tried to work in several address spaces in the test mode, but decided to abandon the officially not supported implementation variant.
The most interesting thing is how we organized the routing. Puzzled for a long time, but in the end everything turned out quite simple. The segments of each bank in the infrastructure core are VPNs spread over MPLS and present at two sites. We planned to route between them through MPLS label import / export policies. But together, these components refused to work harmoniously, because we have different equipment at different sites. If you write on each problem in technical support vendor, you can disrupt all possible terms of the project (and in our case they were very short). All this did not suit us at all. In the end, we just made separate physical links between VPNs.
Another difficulty was to set up rules for filtering routes between two sites - in the process of uniting VPN networks there is a possibility of unexpected āloopsā that are fraught with a fall in the network infrastructure.
Application level
To simplify access to services of companies from every corner of the āresource forestā, we have implemented the concept of ādata martsā.
Each of the banks publishes in its network segment a set of proxies to access their tasks. That is, in the showcase each bank publishes those applications that must be used together with other participants. For example, VTB24 delivered a gateway to its segment for access to their VDI.
After the gateway is activated on the firewall, the corresponding rules are opened, and users connected to the āforestā have the opportunity to work at VTB24 virtual desktops. We have identified "windows" for terminal and web applications. Terminal applications are published on the Citrix farm, and for the web, technology has been used for many years in the bank. About 70 applications are integrated with it. It was built on the basis of the Oracle Access Manager product.
With applications from other banks, everything is much more complicated, since they are not integrated with Access Manager. Therefore, it was decided to put the windows running in reverse proxy mode.
Another prerequisite was the domain authentication requested by the storefront. For correct forwarding through it, HAProxy was chosen - free software included in the Red Hat Enterprise Server package. This is a great finished product, with which you can do both reverse HTTP Proxy, and proxies of many other things. You can proxy not only HTTP, but also any other traffic, including private protocols, which are used in the application for data exchange.
Another important task that we are now solving is to bring the communication system to a single standard in our companies. With all its conservatism, the banking sector requires very fast decision-making and well-established communications. Now we began to use Skype for Business. Major deployment work is almost complete.
Between the individual servers with Skype there is a federated relationship, that is, a trusted relationship between independent peer servers. Thanks to the integration and synchronization of accounts, you can now easily find a contact for contact. Skype for Business is integrated with telephony and video conferencing. You can call a regular phone with Skype and back on the PSTN if the employee has got a number in his Skype account. If you have a headset and a video camera, you can call the meeting room from your workplace via Skype for Business. Naturally, all types of communication must comply with strict safety standards.
In addition to the specific tasks associated with the functioning of banks as commercial structures, in the new architecture, we simultaneously solved a lot of pressing problems. For example, synchronization of passwords, accounts and mail.
Perspectives
New architecture with a resource forest is a kind of framework, the foundation for further information exchange with the groupās daughters. This includes both applications (for example, mail) and, say, access to the portal with contacts. In the future, based on the new architecture, standardized sets of requirements will be formed for connecting to the platform, for example, subsidiary banks. This will help simplify the entire procedure.
Now we have tried to provide all possible nuances of the development of the architecture and thus ensure the absence of problems with integration in the future.
Source: https://habr.com/ru/post/346650/
All Articles