Black and white lists. One of the implementation options

Some time ago we already talked about our project RosSpam - an Android application that allows you to automate the dispatch of a complaint to FAS for advertising SMS. Of course, receiving a large amount of information about the senders of advertising SMS, it was impossible not to use it. Because we implemented a centralized blacklist. However, the logic of our list is not easy at all.

Global blacklist

He is a centralized blacklist. It's all relatively simple. From users who have noted that the SMS they received is spam, we receive information about the sender of the SMS, as well as the text of this SMS. If several different users complained to the same sender, then it is included in the Global blacklist. The blacklist is updated daily on user devices. But here there are several problems.

Problem 1. Is it spam?

One sender can send different SMS - both useful and advertising. For example, two messages from the sender 900 (Sberbank):
')

ECMC1234: 12/16/13 22:00 enrollment operation in the amount of 5000.00 rubles. ITT 103233 2216 completed successfully. Available: 30721.32 rub.

Housing loan under special conditions until December 31, 13 General license of the Bank of Russia for banking operations No. 1481 dated 08.08.2012. Detailed information by phone: t. +79161234567 and on the Bank's website www.sberbank.ru

For some, the first SMS is useful, and the second is spam. For someone else - both SMS are useful.
On the server side, it is possible to add the sender to the ignore list. Then no matter how many users complain about the sender of the SMS, it will not be included in the Global Black List. But in the above case it will not help.

Problem 2. Time of appearance

In order for the sender to get on the Global List, it is necessary that several people complain about it. Accordingly, if the sender sends advertising SMS rarely or to people who do not use RosSpam, it will not appear in the Global Black List soon.

Problem 3. The required SMS will be skipped

For example, a certain number of people complained to the sender. The moderators didn’t track the “good” sender on time, he was put on the Global blacklist, the required SMS was blocked.

How is this solved?

Let's start with problem 3 - of course, the received SMS is not deleted completely. All blocked SMS are saved in the Application History. But this is not enough. We return to this issue below.

Local blacklist

Solves partially Problem 1 and completely Problem 2 . As soon as a user complains about an SMS message, in his opinion, the sender immediately enters the local blacklist. It does not matter if the sender is added to the ignore list on the server side, if other users complained about it - the user will not see the SMS from this sender anymore. But if the user made a mistake, then he, of course, has the opportunity to remove the sender from the Local black list (you cannot remove the sender from the Global list).

White list

By default, the white list is the user's address book. If the sender is in the address book, then his message will not be considered spam. But not everyone wants to add to the address book banks, airlines and other senders whose SMS is needed. Then the application has its own whitelist. Any sender from the Global List can be added to the White List. The whitelist has priority over two Blacklists. If the sender is found on the White List - this is definitely not spam.

But how to help the user identify the good in his opinion of the senders in the Global blacklist and add them to the White list in time ( problem 3 )? Especially when there are a lot of entries in the Global Blacklist (at the time of writing this text in the RosSpam database there are about 600 such senders)? To do this, in our application, new entries in the list are considered “unread” and highlighted in blue. As soon as the user sees the new record (more precisely, he will look at it for at least 5 seconds) - it is marked “read” (like letters in MS Outlook). It is enough to periodically go to the Global Black List to view new entries and transfer them to the White List, if necessary. A small bonus that facilitates the user's life - the senders in the list are sorted without taking into account the special characters in their names (for example, as in the picture above - L_Gorod. And .L.Gorod follow each other).
Yes, the whitelist is local. You can also delete records from it.

What do you think about this implementation of black and white lists?