Machine learning in microfinance: building a scoring model for customers with a blank credit history

No credit history - do not give loans, do not give loans - there is no credit history. A vicious circle of some kind. What to do? Let's figure it out.

Hello! My name is Mark, I am a data scientist at Devim. Recently, we launched a model for scoring borrowers of the “Before Salary” IFC, which have no credit history. I want to share the experience of data search, features of design and interpretation of signs.

This topic is divided into two publications, in the first I will talk about the process of searching and constructing features. In the second part, on comparing model architectures, analyzing results and interpreting scoring solutions.

Part one. Construction of signs

Machine learning models are based on data, the quality and completeness of which is the determining factor in the success or failure of the model. And what if the data is small? Or if the data is not informative enough or not accurate? Where to find additional information and how to use it when building a model? Let's tell you how I solved this problem.

Factors for assessing credit risk

Credit scoring is based on an analysis of the borrower's characteristics associated with the risk of loan default. They can be divided into general economic and individual.

General economic factors

The economic situation has a serious impact on the financial and psychological state of the borrower. More accurately assess the degree of influence, you can select the factors related to the borrower. Conventionally, they are divided into two levels:

• Macro level factors are factors that are external to the borrower. These usually include GDP, inflation, exchange rate, etc.
• The micro - level factors are those that characterize a particular borrower, for example, profession, industry, average wage, etc.
  Immediately it should be noted that general economic factors serve as additional. According to many researchers , the information that they contain is general and poorly characterizes a particular borrower.

Individual factors

Individual factors contain the most valuable information for the scoring model. They can also be divided into categories:

• Demographic - age, gender, marital status, etc.
• Financial - income and expenses, access to financial resources, availability of financial reserves.
• Psychological - one of the most informative. The best source of such data is credit history. The credit history characterizes the financial discipline of the client, contains information about the ability to repay specific amounts, shows the current interest in the loan. If the credit history is not formed, you have to look for other sources of information: social networks, behavior when filling out an application, etc.
• Contact information - its volume and composition affect the risk of loan default.

Dataset description

The set for training the model is 9,500 borrowers who first received a loan from May to December 2018. Data for testing - 1500 borrowers for the period from January to March 2019.

Temporary separation of borrowers is used for several reasons. First, this separation makes it unlikely to leak information from the future. Secondly, it allows us to estimate the stability of the model over time. In the PDL ( Payday loan ) microloans, the amounts and terms are small compared to other types of loans, therefore, the following is selected as the target indicator: overdue payments by more than 15 days.

Construction of signs

Construction of signs begin with the more general - economic, then move on to the individual.

From the general macroeconomic factors, only one was found, which is consistently available and regularly updated factor - the ruble exchange rate. It is available on the website of the Central Bank for a long period of time (it is possible to upload data in a convenient format), and most importantly, it is updated daily. The ruble exchange rate has a stable downward trend. In the raw form, this factor is better not to use. After a certain period of time, the values ​​of the feature will go beyond the data in the training sample and will be incorrectly interpreted by the model.

To avoid negative consequences, we will convert the ruble exchange rate in relation to the current rate (at the time of consideration of the application) to the median value for the previous 35 days. Now the feature characterizes not the absolute value of the ruble exchange rate, but the tendency (growth, decline, stable state) in the period under consideration. On chart 1 the data obtained. On chart 2 - the percentage of default customers by category (fall, stability, growth).

Chart 1. Change in the ruble exchange rate, relative to the median value for the last 35 days.

Chart 2. The number of default clients depending on the course change.

From economic micro-factors are available: the region in which the borrower works, the type of organization, the profession.

At first glance, the region of work relates to individual factors rather than to general economic ones. However, it is possible to add general economic information to the data through the grouping of regions. The site Rosstat available information on various economic indicators of a particular region. The data on the average level of wages in the region, the cost of a fixed set of products and the amount of overdue loan payments per capita have influenced the probability of default. For grouping the regions, the agglomerative clustering algorithm was chosen. The Ward method, which unites the clusters in such a way that the dispersion increment is minimal, was used as a criterion of communication. The resulting data clusters are on three-dimensional graphics.

Another important microeconomic factor is the profession. The figure below shows data on the share of default customers by profession from the training data set.

The graph clearly shows the dependence of the probability of default on the profession. To group borrowers, it is desirable to apply one of the generally accepted principles in the economic community. The breakdown into categories from the Rosstat site correlates well with the data presented in the graph.

Frequently encountered professions, such as driver, manager, accountant, etc., can characterize a borrower in different ways, depending on the specific area or type of organization. For example, a driver working in a taxi and a driver working in the city administration are completely different borrowers.

To add this information to the model, we divide borrowers by the type of organizations in which they work:

• Commercial organizations
• State organizations
• Individual entrepreneurs and self-employed
• Idle
• Organization type not specified

In order to check if the division of information adds, let's look at the chart “the proportion of default borrowers grouped by profession and type of organization”.

From the graph it is clear that for some professions the difference is significant in what type of organization the borrower works. Unexpected results are obtained when the borrower indicates that it does not work, but at the same time indicates the profession. Additional analysis of the data showed that this behavior is typical for pensioners.

And the last general economic factor used in the model is the day of the month on which the loan application was filed. This is probably due to generally accepted wage payment rules in Russia (for example, 10 and 25). The days of the month are divided into two periods from the 9th to the 21st day inclusive and the rest of the month.

Individual factors

Demographic

In my data, there are only four demographic characteristics:

• Borrower's age (number of full years)
• Experience at the last place of work (in months)
• Family status (single, married, married, civil marriage, divorced, single, widowed, not filled)
• Number of family members (co-residents with the borrower)

Financial

In the data on borrowers there is information about wages and additional income. The value of these factors is often overestimated by customers, so they do not contain accurate information about the financial position of the borrower, but allow to estimate it approximately.

Psychological

The selected set of borrowers has no loans, so we do not have basic psychological (behavioral) information. But 90% of clients have information on the number of requests for credit history per year, quarter, month, week, day, hour. Thus, it is possible to estimate the need for a loan at the moment and the need for a loan from a historical perspective. The number of applications for a loan submitted for a short period adds information about the psycho-type of the borrower. (whether he submitted one application and waits for the decision, and then submits the second if he refuses. In this case, there will be few loans in the last hour, but many in the last day. Or the borrower submits applications to different organizations and waits for the decision from everyone at once.)

Contact Information

When submitting an application, filling in your own contact information is required. It is also desirable to provide contact details of two closely acquainted people. What allows to form two additional binary signs:

• full or not contact 2
• full or not contact 3

As a result, we obtain the following signs:

1. Change in ruble exchange rate, numeric attribute
2. Region of employment, categorical feature (6 categories)
3. Profession, categorical feature (5 categories)
4. Type of organization in which the borrower works, categorical feature (5 categories)
5. The day of the month on which the application is filed, the binary sign - falls in the interval from the 9th to the 21st day or not
6. Number of credit history requests for:
   
   • hour
   • day
   • a week
   • month
   • quarter
   • year
7. Marital status, categorical feature (8 categories)
8. Number of family members, numeric attribute
9. Experience at the last place of work, numeric attribute
10. Borrower's age, numeric attribute
11. Monthly income, numeric attribute
12. Additional income, numeric attribute
13. Whether contact 2 is full or not, binary sign
14. Whether contact 3 is full or not, binary sign

All of the above data is economically sound and easy to collect. Despite the fact that they do not carry complete information about the borrower, on their basis it is possible to build a cost-effective and working model.

I will tell you about the architecture selection process and the resulting results in the next article.
I hope it was interesting and useful.

Panenko Mark, Devim