MDM system without primary normalization. Actual approach to solving old problems

Disorder in reference books that store company’s core data is a frequent problem that causes quite a few troubles, including confusion in source documents due to the incorrect establishment of a nomenclature or the purchase of goods that are actually in the company's warehouses. This can be avoided by implementing the MDM system and normalizing the data.

Normalization relieves reference books from superfluous information, makes the entries in them uniform and standard. This allows you to use high-quality and structured data, take them from a single source and be sure that they are correct.
')
Let's take a look at how to normalize directories that store the company's core data, and how to avoid an expensive and time-consuming process.
For example, take the directory of material and technical resources (nomenclature). Normalization of such a directory requires a large amount of time, money and experts to clean up the data.

What is normalization?

Normalization is the reduction of information in reference books to a standard form, excluding duplicates and outdated records, complementing (if possible) incomplete positions.

It is assumed that after normalization, your directories will store relevant and consistent information that company employees will be able to use together to perform their work.

How to carry out the normalization now?

Classification of reference books is classically carried out in two stages:

1. The contents of the source directories are processed before being loaded into the MDM system. This procedure is called primary normalization.
2. After loading into the MDM system, entries in the directories are added or changed according to the normalization method adopted by the company.

What are the disadvantages of the described approach?

The main disadvantage of this approach lies in the primary normalization.

For example, the company maintains a “Nomenclature” directory of 200,000 entries. To carry out the initial normalization of this amount of data before loading into the MDM-system, 20 experts are allocated.

Based on the conditional rate of processing records by one expert in 1,000 positions per month, we find that such a number of records can be normalized in about 10 months of work.
We take into account in our calculation that the experts are not robots, but people, and we will add to the processing period another month and a half for possible force majeure, illness, vacations and unforeseen situations. We obtain that the source directory will be processed by 20 experts in approximately one year.
It is important to note that this year the directory will be updated with new entries. It will also take time to normalize them. So, the period of primary processing of directory entries may stretch for months.

At the same time, users will be able to see the correct records only after the system is put into operation. At the same time, the shortcomings of the normalization will begin to come to light, which will require additional processing of the directory content.

How much will you pay for the initial normalization?

The average price of the primary normalization of one entry in the Nomenclature reference book in Moscow is 150 rubles. In our example, 200,000 entries. The initial normalization of the data of such a reference book will cost 30,000,000 rubles.

And here you need to remember that with the usual approach, you will pay for the initial normalization of all directory entries: current and not used by the company.

Why continue to conduct primary normalization?

Primary normalization is performed to obtain fully normalized records of source directories. However, often after the initial normalization, about 40% of the generalized and inaccurately described data remain in the source directory, which are recognized as non-normalizable and are loaded in their original form into the MDM system.

In the case of the “Nomenclature” reference book, such entries can be refined only when the nomenclature is checked by experts at the company's warehouses by appearance or technical passport (inventory). That will add work and take a lot of time.

How to normalize directories more profitable and efficient?

• Exclude primary normalization.
• Analyze the directory entries to determine groups of records according to the importance and relevance of the data.
• Identify possible peak moments of record updates. They can be centralized procurement campaigns, which are conducted at regular intervals.
• Download the source directory to the MDM system.
• Block from changing part or all of the directory data.
• Update blocked data gradually at the request of users.

With this approach, all the original data will be included in the MDM system. They will be normalized gradually after loading, which will allow:

• Put the system into operation in a short time, without waiting for the completion of the primary normalization of the directory.
• Normalize what employees really use in their work and do not process obsolete records.
• Do not spend money to pay for the work of experts on the primary normalization of the directory.

In this case, the percentage of incompletely described data is reduced to zero. Such accuracy is achieved by the fact that when updating the data of the reference book on user requests, the experts will be able to clarify all the data from the application author.

How does it work on the example of the “Nomenclature” reference book?

Remember that we have a “Nomenclature” directory with 200,000 entries. We do not carry out its initial normalization and, in its original form, load it into the MDM system. Already at this stage, we do not spend about a year on the initial processing of records and on paying for the work of 20 experts. In this case, we have several options for its normalization:

1. We analyze the records and divide the entire volume of data into groups by importance and price.
2. After loading, we block to change the most demanded and expensive part of the nomenclature. Now, for the purchase of these items, company employees must fill out an application.
3. When processing requests by full-time experts, the blocked group of directory entries will be normalized gradually, and then similarly - the other groups of the nomenclature. Locking directory records will not be extended to write-off operations, which will allow working with data in the normal mode.
4. Without the user's application, the blocked items of the nomenclature will not be updated. Such positions will be transferred to the archive according to the rules of the company: during the regular inventory or after the write-off of balances on them. Based on experience, out of the entire array of 200,000 records, only a part of the actually relevant and used positions will be normalized (as a rule, about 20% (40,000) of existing records).

The procedure can be carried out in another way - by blocking for changing all the records of the “Nomenclature” directory, and not by separate groups. Thus, only normalized entries will gradually appear in the directory, and irrelevant positions are also identified and transferred to the archive. But in this case there is a risk of uneven loading of experts in the processing of records during peak periods of the purchase of the nomenclature. This should be taken into account when conducting analytics and correctly determine the workload of experts.

Based on practice, when using the new approach, the Nomenclature reference book will be normalized on average over one year. This time is usually spent on the primary normalization of all directory entries, both necessary and obsolete. Therefore, the timing of normalization will remain the same, but it will be carried out without extra money, experts, and delaying the date of putting the MDM system into operation.

Thus, with the new approach, you will get all that you expect from primary normalization: the actual and correct data of reference books that are used in daily work, but with lower costs. At the same time, the money saved can be spent on the elaboration of classifiers, the templating of names, the development of the functionality of the system and other improvements, which I will discuss in the following articles.

Do not think out of pattern: study current approaches to solving old problems that will save your money and time!