How smart service SMS handler works (shows only important information)

This article contains a description of the internal device of the smart service SMS handler.
The application parses incoming SMS messages and shows only important information from them.
Shows beautiful, fast and convenient.

1. How it works

We register permission to receive and read SMS in the manifest.

<uses-permission android:name="android.permission.RECEIVE_SMS"/> <uses-permission android:name="android.permission.READ_SMS"/>` 

In the same place we register receiver
The permission action_sms_received_test needed for testing.
In order not to spend money on real sms during testing, I send Intent with this action from the application and catch it.

 <receiver android:name=".receivers.SmsReceiver"> <intent-filter android:priority="2147483647"> <action android:name="android.provider.Telephony.SMS_RECEIVED"/> <action android:name="action_sms_received_test"/> <action android:name="android.intent.action.BOOT_COMPLETED"/> </intent-filter> </receiver> 

The receiver will now receive all incoming messages.

 @Override public void onReceive(Context context, Intent intent) { switch (intent.getAction()) { case ACTION_SMS_RECEIVED: handleIncomingSms(context, intent); break; case ACTION_SMS_RECEIVED_TEST: // do test break; } } 

Now in the handleIncomingSms(context, intent); method handleIncomingSms(context, intent); you need to figure out what kind of SMS came to us, and decide what to do.
If it is service, we disassemble it, get useful information, and display it in a beautiful way.
How we understand whether it is service or not - I will describe later.

Roughly, it looks like this

 private void handleIncomingSms(Context context, Intent intent) { Li("handleIncomingSms"); Bundle bundle = intent.getExtras(); if (bundle == null) { return; } try { Object[] pdus = (Object[]) bundle.get(PDUS); String smsText = ""; for (Object pdu : pdus) { final SmsMessage message = SmsMessage.createFromPdu((byte[]) pdu); smsText += message.getMessageBody(); } checkTemplates(context, smsText); } catch (Exception e) { Li("handleIncomingSms - Exception", Log.getStackTraceString(e)); } } 

Method checkTemplates();

 private void checkTemplates(Context context, String smsText) { Li("checkTemplates", smsText); // get templates List<SmsTemplate> smsTemplates = DatabaseManager.getSmsTemplates(); if (smsTemplates == null) { return; } // check if sms text according to some template for (SmsTemplate smsTemplate : smsTemplates) { List<String> messageLines = SmsNewParser.getMessageLines(smsTemplate, smsText); if (messageLines != null) { Sender sender = DatabaseManager.getSender(smsTemplate.sender); showPopupDialog(context, messageLines, sender != null ? sender.iconUrl : ""); } } } 

showPopupDialog method

 private void showPopupDialog(Context context, List<String> message, String iconUrl) { Li("showPopupDialog", message, iconUrl); Intent popupIntent = new Intent(context, PopupActivity.class); popupIntent.putExtra(PopupActivity.ICON_URL, iconUrl); popupIntent.putExtra(PopupActivity.MESSAGE_0, message.get(0)); popupIntent.putExtra(PopupActivity.MESSAGE_1, message.get(1)); popupIntent.putExtra(PopupActivity.MESSAGE_2, message.get(2)); popupIntent.putExtra(PopupActivity.MESSAGE_3, message.get(3)); popupIntent.setFlags(Intent.FLAG_ACTIVITY_CLEAR_TASK | Intent.FLAG_ACTIVITY_NEW_TASK); context.startActivity(popupIntent); } 

After that, the user sees such a screen.
The point is to quickly see useful information.

2. Algorithm of recognition of SMS and the issuance of important information

2.1. Briefly

• The server has templates
• Each template indicates a) how SMS should look like b) what exactly to show for it
• The application synchronizes each time it starts.
• Each incoming message is run on all templates.
• If a template is found, to which it corresponds - important information is shown in the required form.

2.2. Details about the model

The template looks like this

 { "sender": "bank_alfa", "text": "3*8272; Pokupka; Uspeshno; Summa: 212,30 RUR; Ostatok: 20537,96 RUR; RU/MOSKVA/GETT; 15.04.2016 06:02:43", "mask": "~N~*~N4~; ~BANK_ACTION_0~; Uspeshno; Summa: ~SUM_0~ ~CURRENCY_0~; ~BANK_ACTION_1~: ~SUM_1~ ~CURRENCY_1~; ~WORD~; ~N2~.~N2~.~N4~ ~N2~:~N2~:~N2~", "lines": [ { "line": "EXTRA_PURCHASE" }, { "line": "SUM_0" }, { "line": "EXTRA_TOTAL" }, { "line": "SUM_1" } ] } 

• sender - the sender
• text - the initial text of this SMS. can be used for tests
• mask - the template itself. used official words like ~FOO~
• lines - lines of the message to be displayed on the screen. They can specify parts of the template, and you can use words that are not in the template.

Service words are divided into extra and ordinary.
Extra means that they are not in the template.
Examples:

~SUM~ is an ordinary service word. Indicates an expression with numbers separated by a full stop or comma.
Used to determine the amount of money. Regex is used to find it.

 { "name": "SUM", "regex": "\\d+[.,]{0,1}\\d+", "values": [], "is_extra": false } 

~CURRENCY~ is a common word that can take several meanings. For its search it is used enumeration of its values

 { "name": "CURRENCY", "regex": "", "values": [ { "value": "usd" }, { "value": "rur" }, { "value": "eur" }, { "value": "rub" } ], "is_extra": false } 

~EXTRA_CODE_WORD~ is a service word of type extra . Used to display the text "Codeword" in the result.

 { "name": "EXTRA_CODE_WORD", "regex": "", "values": [ { "value": " " } ], "is_extra": true } 

we also need pictures to show who sent the message.
This information is stored in sender objects.

Example:
This is Alpha Bank and its icon.

 { name: "bank_alfa", icon_url: "https://dl.dropboxusercontent.com/u/1816879/CaptainSms/logo_alfa.png" } 

As a result, no server is stored

• Templates
• Service words
• Senders

Full json can be seen here.

2.3. Details about the algorithm

We download the model, save it.
Then follows the procedure of parsing SMS and the creation of the resulting message.

To parse the text of a message, I use the SmsParser class with static methods.
The main method is getMessageLines(SmsTemplate smsTemplate, String realSmsText)
It returns message strings if everything is ok, or null if we have not found a suitable template.
This method is called from this place of the checkTemplates method above.

  // check if sms text according to some template for (SmsTemplate smsTemplate : smsTemplates) { List<String> messageLines = SmsNewParser.getMessageLines(smsTemplate, smsText); if (messageLines != null) { Sender sender = DatabaseManager.getSender(smsTemplate.sender); showPopupDialog(context, messageLines, sender != null ? sender.iconUrl : ""); } } 

We go through all the patterns from the database and try to take the message lines for everyone.
If it works, we show a screen with information.

Logic getMessageLines briefly
We run by the mask and compare it character by character with the text of the SMS, writing down the values ​​of the encountered service words in the array, or throwing out null if there are inconsistencies

Logic getMessageLines more:

• We run character by character mask
• If the symbol is the beginning of the service word ( ~ ), then:
  - We understand what this word is (for example, ~ SUM_0 ~)
  - Calculate its value in the text SMS (for example, 255.00 )
  - We cut off this word from the mask, and this value is from the text (in order to continue to run character by character)
• Otherwise, if it is a simple symbol, then:
  - If they coincide in max and text, then we cut them off from there and from there to further compare
  - If they are different, then throw out null - the text does not fit the template

Logic with code examples

As parameters, the template and sms text come to the method.

 public static List<String> getMessageLines(SmsTemplate smsTemplate, String smsText) 

At the beginning of the method, we initialize the list of service words. They got to base from regular updating with api.
We need a global variable, since The method is large and broken apart.

 private static void initReservedWords() { Li("initReservedWords"); mReservedWords.clear(); mReservedWords = DatabaseManager.getReservedWords(); } 

Then create a list of service words from the specified template.

  List<ReservedWord> reservedWords = new ArrayList<>(); for (SmsTemplateLine line : smsTemplate.lines) { reservedWords.add(getReservedWordByName(line.line)); } 

those. if we have a pattern

 { "sender": "bank_alfa", "text": "3*8272; Pokupka; Uspeshno; Summa: 212,30 RUR; Ostatok: 20537,96 RUR; RU/MOSKVA/GETT; 15.04.2016 06:02:43", "mask": "~N~*~N4~; ~BANK_ACTION_0~; Uspeshno; Summa: ~SUM_0~ ~CURRENCY_0~; ~BANK_ACTION_1~: ~SUM_1~ ~CURRENCY_1~; ~WORD~; ~N2~.~N2~.~N4~ ~N2~:~N2~:~N2~", "lines": [ { "line": "EXTRA_PURCHASE" }, { "line": "SUM_0" }, { "line": "EXTRA_TOTAL" }, { "line": "SUM_1" } ] } 

then we want to get a list

• EXTRA_PURCHASE
• SUM_0
• EXTRA_TOTAL
• SUM_1

further comes the basic logic

  // check match symbol by symbol try { do { String s = mask.substring(0, 1); if (s.equals(ReservedWord.SYMBOL)) { // found start of a reserved word ReservedWord currentReservedWord = getFirstReservedWord(mask); String valueOfCurrentReservedWord = getValueOfReservedWord(smsText, mask, currentReservedWord); // add value in the list, if reserved word is in the list if (reservedWords.contains(currentReservedWord) && valueOfCurrentReservedWord.length() > 0) { values.put(currentReservedWord.getForm(), valueOfCurrentReservedWord); } // cut text and mask to look next symbols smsText = smsText.substring(valueOfCurrentReservedWord.length()); mask = mask.substring(currentReservedWord.getForm().length()); } else if (s.equals(smsText.substring(0, 1))) { // that symbols matches, go to the next symbol smsText = smsText.substring(1); mask = mask.substring(1); } else { /* * that symbol does not match, so text not match that mask, so method fails * because we cannot return correct values according to that list of reserved word */ return null; } } while (mask.length() > 0); } catch (StringIndexOutOfBoundsException e) { /* * There is some error during parsing. * That mean text does not match mask. */ Li(TAG, "getMessageLines - Exception - " + Log.getStackTraceString(e)); return null; } 

It does exactly what is described above, as "The logic of getMessageLines more detailed:"

Next, we re-sort the list, because in the text it is found in a different order than our message lines

  // convert list to the right order List<String> valuesList = new ArrayList<>(); for (ReservedWord word : reservedWords) { LLog.e(TAG, "getMessageLines - return list - " + values.get(word.getForm())); if (values.get(word.getForm()) != null) { valuesList.add(values.get(word.getForm())); } } 

Next, we add extra words like we did not find them when passing through the text SMS.

  // add values of all the extra words for (int i = 0; i < reservedWords.size(); i++) { if (reservedWords.get(i).isExtra) { valuesList.add(i, reservedWords.get(i).values.iterator().next().value); } } 

This is why this is why.
At the entrance we were given smsTemplate . He has a set of messageLines. For example, there were 4 of them.

  "lines": [ { "line": "EXTRA_PURCHASE" }, { "line": "SUM_0" }, { "line": "EXTRA_TOTAL" }, { "line": "SUM_1" } ] } 

But in the process of checking the text for a match with the template, we found only SUM_0 and SUM_1
Since This data, which really is in the text of SMS.
Thus, after the first piece of logic, we have an array of two elements (in this case, 212,30 and 20537,96 ).
But at the output we need to submit 4 lines (to these two we need to add EXTRA_PURCHASE and EXTRA_TOTAL ), and in the necessary order.
Therefore, at the end of the method we add them.

As a result, at the output we get an array of four lines.

For example, if we had a pattern

 { "sender": "bank_alfa", "text": "3*8272; Pokupka; Uspeshno; Summa: 212,30 RUR; Ostatok: 20537,96 RUR; RU/MOSKVA/GETT; 15.04.2016 06:02:43", "mask": "~N~*~N4~; ~BANK_ACTION_0~; Uspeshno; Summa: ~SUM_0~ ~CURRENCY_0~; ~BANK_ACTION_1~: ~SUM_1~ ~CURRENCY_1~; ~WORD~; ~N2~.~N2~.~N4~ ~N2~:~N2~:~N2~", "lines": [ { "line": "EXTRA_PURCHASE" }, { "line": "SUM_0" }, { "line": "EXTRA_TOTAL" }, { "line": "SUM_1" } ] } 

then at the output we get

• Purchase
• 212.30
• Left
• 20537.96

This is where the main logic ends.
Then we just show it in our pop-up activ by this method.

 showPopupDialog(context, messageLines, sender != null ? sender.iconUrl : ""); 

The text messageLines simply displayed in the text of the views.
iconUrl loaded into the image view using Glide - everything is extremely simple.

Conclusion

Obviously, the algorithm is primitive and can be improved.
Of ideas

• split api into different json files (for example, one json for each sender)
• clever algorithm for running on patterns (first, all with codes - they are needed most quickly, then frequently used, then all the others)
• You can probably improve the parsing code itself (check for creating unnecessary objects, reduce the number of loops, etc.)

But the assigned task solves.

I attach the main class for parsing messages.
It is slightly different from the code above,
because The code above has been improved visually.