“We did microservices before it became mainstream”: Sberbank-Technologies on development

Sberbank Technologies is engaged in a number of interesting tasks - from unifying the interface of a huge bank to blockchain prototyping. In most cases, development is conducted in Java: it is considered by the company to be the best choice for large corporate systems, appreciating the opportunity to build a solution from third-party products and internal developments.

At conferences, Sbertech’s speakers talk directly about their experiences, as well as the general conclusions that can be drawn from it. But not everything fits into the format of the reports, and on the eve of Joker, we separately asked questions to three employees of the company speaking at Java conferences. They answered to us:
')

• Sergey vlsergey Vladimirov (will perform at Joker 2016, previously spoke at the JPoint)
• Alexander Matorin (spoke earlier on Joker, JPoint and JBreak)
• Maxim Zelinsky (spoke earlier at JPoint and JBreak)

Sergey Vladimirov (head of development at the Central Committee of a business development support platform)

- What are you working at Sbertech?

- For about a year, he was engaged in the development of a new accounting system for card acceptance terminals at retail outlets. The project replaces the outdated system written in C, which worked with direct connection to the database and had a large number of manual and semi-manual interactions with other systems. Our application will be part of the new Sberbank 2018+ platform (target platform from 2018). It is written as an “honest” server-side Java application (with the browser as a client), using a client-oriented data storage architecture (based on Apache Ignite / GridGain technology) and a single corporate data model. There are a lot of clever words, but for each of them there is a cardinal change, if not in the way one should think, then certainly in the way one should develop business applications.

In this project, the first phase is completed, my colleagues will continue to work on it, and I myself am starting a new project on the organization of the user interface. Again, this will not be just a rewrite of the code from HTML to JavaScript, but the introduction of new approaches to the interaction of frontend and backend.

- Do you teach information security and cryptography - is it connected with work in Sbertech, or are you interested in them separately? Sberbank started prototyping the blockchain - do you have any relation to this?

- I started to get involved in information security from school (as well as programming), but it always went in parallel and was not directly connected. But they influence each other constantly. At a minimum, you catch yourself on the fact that you start to look at the same problem from two sides: “as I would do it as a developer” and “where this system has weak points, if I want to break it”. This is especially true for complex enterprise level systems. It’s very bad if the “security accounting” in the design of systems is that a ready-made solution is forced to change under pressure from security personnel, and not vice versa - when following the safety rules is laid down in the principles of the system, and whoever develops the system understands and “ feels these are principles. Very useful stuff.

Not to mention the fact that I, as a programmer, cannot be frightened by the terrible words “certificate”, “digital signature” or “hash sum”.

Regarding the blockchain, unfortunately (really sorry), Sberbank has so many interesting projects that it is not just impossible to have time to participate in all of your interests, but sometimes you even learn about them from news and media. About the same blockchain, I gladly went to a lecture organized by Sberbank for my employees, and I learned what a blockchain is from a business point of view, but I don’t have time to take part in this project myself in the coming year. As, unfortunately, I do not have time to take part in the project “Data Factory” (project of big data analysis).

- You love open source - and how actively do you use it when working in Sbertech?

- You can not love open source, but it is very difficult not to use it :-) If you are a Java programmer and are developing a solution for a business, it means that you have more than a dozen open source libraries in your classpath, ranging from logger to XML libraries or Json At Sberbank Technologies, we not only use open source, but also try to be an active part of the community. The most vivid example is, of course, participation in the work on Apache Ignite.

Open source is not just cheaper. It turns out to be faster in terms of “time to market”. Using a paid product with a closed code, you can wait two or three years from the developer of fixes or improvements on the functionality you need. And you can take an open source solution, fix some shortcomings in it (if they were there and no one noticed them), and when you need to add or change something, just take and change. Suppose that this requires qualification, even if it is not a bit peculiar to your employees, but it will work, roughly speaking, in a week already in production, and not in 2-3 years, when a new version of the closed product will be released. In Sberbank Technologies, this is not a theory, but real cases when one has to change some technology or solution simply because it is too expensive to modify an existing one or, which is usually more important, too long.

- You speak to the public in different formats: as a teacher at MIPT, with scientific lectures, and at conferences. What is the format of conferences for you different?

- If you compare lectures and speeches at conferences, the latter are always more difficult and interesting. The speaker does not have a way to get people to come to his story - he will not put a deuce on the exam. You can’t just go out in front of an audience and tell the material - people came not to take notes, not just to learn new and cognitive, but people came to get the knowledge that will help to become professionals in their field. And if a report does not provide something useful that a person can put into practice (even if in the future), the person will get up and leave.

That is the goal of the student - to pass the exam. The goal of a popular science lecturer is to learn something interesting in your free time. Go to the lecture as an excursion. But the goal of a professional conference listener is professional (and career) growth. And if the report does not contribute to this goal, they will not listen to it.

- What will you tell about the upcoming Joker?

- The presentation for Joker was long finalized with the organizers, there were several preliminary auditions, and in the end decided to shift the emphasis a bit. Previously, it was assumed that there would be several examples of optimization at the level of algorithms, when with simple actions (and precisely without new libraries and tools) we get a significant performance gain. But in the end they decided to take one of the brightest moments and consider it from several sides.

Therefore, the student edition is waiting for you to talk about a data structure such as trie, or a loaded (prefix) tree. And a real example from the working experience will be shown, in which with the help of this structure, the productivity of a single narrow task was achieved a thousand times.

Alexander Matorin (Chief Development Manager at the Central Committee for Development Risk Systems)

- You are engaged in a system for determining fraud, and immediately curious: can you tell us more (without giving out any secrets that would help fraudsters)?

I work in a department that develops risk management systems in financial markets. There are systems that collect trades made by traders on behalf of Sberbank. Other systems may subscribe to receive new deal events to calculate certain risk metrics, or check a deal for fraud.

The simplest example of tracking fraud is the search for “paired deals”. That is, a trader buys a certain financial instrument and after a while sells it at a lower price, possibly to the same counterparty. For example, a trader bought dollars at 70 rubles, and sold them in 60 days a day, hoping that part of the difference would be returned to him by another channel.

Other tests to determine fraud consider more complex metrics, for example, Implied Volatility for FX options using the Black-Scholes method. That is, there is an interesting subject area and a lot of financial mathematics.

- What tools / technologies do you use?

In our department, about 15 different systems are developed, which interact with each other using different protocols. Here, the microservice architecture is very well suited, which we started using about 4 years ago, even before it became mainstream. The calculation of each risk metric or transaction aggregator is a separate service. Now our systems are divided into about 100 microservices that run in several data centers. We write in Java 8, Spring is used everywhere. Different services use different technologies and bases. Part of what we use: Cassandra, MongoDB, Coherence, GridGain, RabbitMQ, Oracle DB, TomCat.

- Also, you participate in internal conferences of the company - can you talk about them?

- For two years in a row, Sberbank Technologies has held internal Dev & QA days conferences. About 150 people came to the last Dev Day in Moscow, about 300 watched the online broadcast. There were about 20 reports in 3 streams. I participated as a speaker and as a member of the program committee. We, like at Joker, anyone could apply for a report. After the application is reviewed by the program committee. If the application is accepted, one or more reports will be rehearsed with the speaker. I hope these conferences will continue to evolve.

- Do you teach at the Sbertech Department at MIPT - but for you this is an additional activity that does not have a strong effect on the main one, or do you feel that teaching others you end up doing the tasks better when working in Sbertech?

- I have been teaching in the magistracy SBT at MIPT for more than 2 years. Also, a Java school opened a couple of months ago at the SBT, where I also lecture there.

The main objective of my Java course, which I set for myself, is not just to ensure that the students remember what I have said, but to convey to them my enthusiasm for Java, so that they themselves can deepen their expertise with interest. For the best students, the department each year offers the opportunity to attend Joker and JPoint conferences.

In the second semester I teach a course on distributed systems. I tell CAP, FLP theorems, distributed transaction algorithms, consensus protocols, the concept of time in distributed systems, what can break and how to ensure fault tolerance, the NoSQL database architecture.

The second course helped me learn a lot of new things, structure my knowledge and apply some of this in my projects.

For students of the course for 2 years, competitions in algotrading. Students must implement an algorithm for algo trading on Java. Then download this source code to the game site. The strategy is compiled, launched in a separate JVM and starts trading in real time with solutions from other participants. You can upload new versions of strategies an unlimited number of times. Last year, one of the participants uploaded about 160 versions by the end of the game. But won another :)

Maxim Zelinsky (Head of Department in the Central Committee of Frontal Systems Development)

- What exactly are you doing?

- I participate in the creation of the Unified Frontal System (EFS), namely, I am responsible for the creation and development of the Platform for developers implementing directly business functionality within the EFS.

In the future, the Unified Frontal System will serve all of the Bank’s customers through all available channels (Internet banks, mobile applications, ATMs, call centers, branches, etc.). Its task is unification and reusability of solutions. Of course, different channels have their own specifics, but in general, the essence of the program is to invest once in the implementation of something, and make this implementation similar for different channels.

Given the enormous scale of the ESF, to create high-quality and reliable solutions, a single set of technologies and services is required to reduce the time to market of new solutions, as well as to guarantee a uniform level of service in terms of reliability and performance. These issues (single stack, services, architecture) are handled by my team in the framework of the ESF Platform.

- You specialize in things like distributed systems - why then do you do a frontal system, and not a backend?

- In our case, the frontend in terms of performance, fault tolerance and reliability is even more interesting than backing: there are requirements of the type “uptime 99.99%”. And if the backend does not work at some point, the client will not even notice. His payments will not be executed instantly, but it will be possible to bring them into the system, it will be possible to get extracts from the cache.

But there is another big program in backend at Sberbank-Technologies: the Business Development Support Program. This is an attempt to transform technological solutions using the In-Memory Data Grid. At first there were attempts to make automatic sharding of data based on Oracle or PostgreSQL. There was a pilot, he gave both positive results and negative ones. Following the results of the pilot, it was decided that “current technologies are not entirely relevant to the size of the bank, let's look to the future.” Chose IMDG, and colleagues are now engaged in the transfer of back-up data arrays into memory.

- With such a huge scale, like you, do situations arise when one technological solution leads to a very large resource gain?

- A vivid example is the choice of technologies for the JS SPA and React web-presentation level. The mere transfer of presentation logic from the application server to the client allowed us to save ~ 200 application servers, which were originally planned to be used for the classical server-side web-presentation level. The same goes for reducing the traffic between the client and the server, which is obviously reduced due to the fact that the exchange is only data, and not HTML.

- Do you have experience in reverse engineering - does it turn out to be practically useful when working at Sbertech?

- Very much. Especially often you have to engage in reverse engeneering when working with IBM WebSphere Application Server, starting from the IBM JVM, where the team I previously worked with before the ESA found a critical defect leading to deadlock inside the JVM when working with load annotations, ending with Java drivers for IBM WebSphere MQ, when I had to study the problem for a long time, why the IBM WAS + IBM MQ + Spring JMS Templates combination does not work with non-persistent load queues.

- Thank. Let's wait for the Joker report by Sergey Vladimirov, but for now let's recall the previous reports from “Sbertech” at Java conferences: