Fish, fish: how to use the "data lake" in the bank. VTB experience
You go to the bank for a loan to develop a business, buy a car or for other purposes. To give or not to give - in each case, the bank's specialists decide this issue individually, taking into account the client's credit history, the size of his income and other factors. It would seem that the credit system has long been set up and is working properly. Is it possible to come up with something new in this regard? We at VTB retail answer this question in the affirmative. Research confirms: the data on client behavior that is available to the bank is far from fully involved, and in this direction the use of IT has a very good effect!
How we integrate IT into a business and what benefits customers receive - read under the cut.
In 2016, for the retail business of VTB Group, we implemented the first stage of a large project for processing and analyzing customer information. Thanks to this project, our clients began to receive personalized offers based on an analysis of their behavior in the past. At the first stage, we collected and use up to 60% of the data, and the results exceeded all expectations. Most of the clients readily accepted individual offers and, most importantly, were satisfied. So, the idea of a selective approach worked, the system functions perfectly.
')
Now the second stage is on the stage - launching a new DataResearchPlatform platform based on DataLake (“data lake”), which in the future should cover 99.9% of all client activity data in the database.
Like all modern solutions in the field of Big Data, our new DataResearchPlatform platform is based on the “data lake”. Why did we choose this technology? DataLake is good because it allows you to store huge amounts of raw data in their original format. This data can be used as you please: to compare, mix, organize according to various criteria. Unlike the standard data storage, DataLake data is available to analysts at once in full and with all the original links. This gives more opportunities to find the most unexpected options for their use, but this requires appropriate technologies and tools.
Client information is processed using data mining. Due to this, bank specialists can test their hypotheses about client behavior and its impact on solvency, as well as develop new predictive models.
There are other "chips" that we plan to get when working with DataLake:
When the system is established, the bank can take the most modern fishing rods and go fishing on its "lake". And there is no doubt: every time the catch will be excellent, and they will want to share with customers. Due to the in-depth analysis of client behavior, the bank can offer borrowers special offers, better credit conditions and individual (more loyal) interest rates on loans.
Before the decision was made to switch to DataLake, there was already a data warehouse at VTB, so the first thing we did was integrate a new platform with it.
In addition, at the first stage, we worked on debugging the technological environment for modeling: the mechanisms for updating all installed software were developed and the Hadoop cluster was expanded. It was also important to develop new approaches to the work of users, since the new platform imposes certain requirements on the delimitation of access to data.
As a result, the current version of DataResearchPlatform is deployed on 12 BDA nodes with a capacity of up to 288 TB (plans to expand it to 18 nodes by the end of the year). The platform works on the basis of the Hadoop ecosystem, OpenSource technologies and industrial Enterprise solutions. It is based on the Oracle BigData Appliance software and hardware solution. Analytical tools SAS HPDM, SAS EG, Python, R are used to work with data.
DataArchitect and DataScientist users have completely secured access to data, and data volumes have been expanded. Now, the DataResearchPlatform already collects almost all the information about customer activity that is available to the bank. You can “catch” it at any time from the “lake” and use it for the benefit of the client.
The working team of the project: members of the management board of VTB24 - A.Sokolov and S.Rusanov.
How we integrate IT into a business and what benefits customers receive - read under the cut.
In 2016, for the retail business of VTB Group, we implemented the first stage of a large project for processing and analyzing customer information. Thanks to this project, our clients began to receive personalized offers based on an analysis of their behavior in the past. At the first stage, we collected and use up to 60% of the data, and the results exceeded all expectations. Most of the clients readily accepted individual offers and, most importantly, were satisfied. So, the idea of a selective approach worked, the system functions perfectly.
')
Now the second stage is on the stage - launching a new DataResearchPlatform platform based on DataLake (“data lake”), which in the future should cover 99.9% of all client activity data in the database.
Why DataLake?
Like all modern solutions in the field of Big Data, our new DataResearchPlatform platform is based on the “data lake”. Why did we choose this technology? DataLake is good because it allows you to store huge amounts of raw data in their original format. This data can be used as you please: to compare, mix, organize according to various criteria. Unlike the standard data storage, DataLake data is available to analysts at once in full and with all the original links. This gives more opportunities to find the most unexpected options for their use, but this requires appropriate technologies and tools.
Client information is processed using data mining. Due to this, bank specialists can test their hypotheses about client behavior and its impact on solvency, as well as develop new predictive models.
There are other "chips" that we plan to get when working with DataLake:
- grow your own DataArchitect and DataScientist user profiles in a corporate environment;
- get excellent experience in depth data analysis;
- completely revise and improve customer information management systems (CRM);
- learn to more accurately predict the risk for each specific client.
When the system is established, the bank can take the most modern fishing rods and go fishing on its "lake". And there is no doubt: every time the catch will be excellent, and they will want to share with customers. Due to the in-depth analysis of client behavior, the bank can offer borrowers special offers, better credit conditions and individual (more loyal) interest rates on loans.
How does DataResearchPlatform work?
Before the decision was made to switch to DataLake, there was already a data warehouse at VTB, so the first thing we did was integrate a new platform with it.
In addition, at the first stage, we worked on debugging the technological environment for modeling: the mechanisms for updating all installed software were developed and the Hadoop cluster was expanded. It was also important to develop new approaches to the work of users, since the new platform imposes certain requirements on the delimitation of access to data.
As a result, the current version of DataResearchPlatform is deployed on 12 BDA nodes with a capacity of up to 288 TB (plans to expand it to 18 nodes by the end of the year). The platform works on the basis of the Hadoop ecosystem, OpenSource technologies and industrial Enterprise solutions. It is based on the Oracle BigData Appliance software and hardware solution. Analytical tools SAS HPDM, SAS EG, Python, R are used to work with data.
DataArchitect and DataScientist users have completely secured access to data, and data volumes have been expanded. Now, the DataResearchPlatform already collects almost all the information about customer activity that is available to the bank. You can “catch” it at any time from the “lake” and use it for the benefit of the client.
The working team of the project: members of the management board of VTB24 - A.Sokolov and S.Rusanov.
Source: https://habr.com/ru/post/336332/
All Articles