Direct work with the MTS SMS-service: the history of one integration

Many believe that SMS is one of the reasons that Twitter “shot” so well in the West. In the United States and several European countries, tweets can be both published and received via SMS, while the price of outgoing tweets is equal to the price of an ordinary SMS, and incoming messages are free. For users without smartphones (most of which), this significantly reduces the threshold for entry.

In this article, I will share the experience of direct integration with a large Russian mobile operator (note: it is directly, and not through gateways), as well as at an introductory level, I discuss about SMS technology and the SMPP protocol - without boring tables and specifications, in the style of short detective story.

Plot plot

Once we ( Rutvit ) addressed a proposal to the largest telecoms operator, MTS. We understood that in order to make the price of a tweet indistinguishable from the price of a regular SMS, we need direct integration with the PPSA (by the way, this word stands for “Cellular Operator”). MTS is a huge company, sharpened to cooperate with major partners, so I thought that our proposal would not be considered soon, and did not particularly count on success. What was our joy when MTS not only quickly responded, but also, as a result of constructive negotiations, decided to launch a joint project with Rutvit.
')
We all know that large companies are often slow and cumbersome, but this does not seem to apply to MTS. The working group we contacted during the integration process quickly responded to our questions, and we moved forward. This was especially valuable in the context that direct integration with the cellular operator turned out to be an order of magnitude more difficult than connecting some popular SMS gateway: I’ll also talk about technical details near the end of the article.

And here we first launched the alpha version of the SMS service on Rutvit (read the press release on the PRIME-TASS website together with MTS ), and today we replace the alpha icon with a beta in this service, which we report in this article . The service, among other things, allows you to send tweets via SMS at a regular cost (according to the subscriber’s tariff plan) not only to Rutvit, but also to big Twitter via the export mechanism.

So, you can send tweets via SMS for exactly the same money for which you send any other SMS. For the same purpose, we decided to use not a short number, but a more familiar, long one: +7 916 140-0-140 . People often do not trust short numbers, because they are not sure how much money will be taken from them for SMS: it’s as if there is no such problem with a long number.

How to start writing tweets by SMS?

To start writing tweets via SMS, enter your phone number on the Rutwit website:



Then send the word “yes” (the register is not important) to the Rutvit number +7 916 140-0-140 . As soon as the robot sees this message, it will link your number to your account, and all subsequent SMS messages to +7 916 140-0-140 will be perceived as tweets on your behalf.

And you can also write SMS at cost to Twitter, FriendFeed, Facebook, LiveJournal, Buzz

If you are a Twitter fan, set up Rutweet -> Twitter message export (or FriendFeed, Facebook, LiveJournal, Buzz) and tweet (FriendFeed, Facebook, LiveJournal, Buzz) SMS at cost using Ruth as a gateway:



(For justice, it is worth noting that you can send SMS to Twitter via foreign gateways, but in Russia it turns out to be much more expensive: after all, SMS to the British number + 44xxxx, provided by Twitter, is more expensive than SMS to the Russian number Ruthwit 9 916 140-0-140 .)

Now a spoon of tar present in the beta version. One of our goals is to make sure that the user can not only send SMS to Rootvit at the price of an ordinary SMS, but also receive free SMS tweets to a mobile phone. Some microblogging services simply buy a GSM modem (or a smartphone with Linux) and receive / send SMS through it, but this method works only on small amounts of traffic: it doesn’t scale well. Nevertheless, we want to save social network users from additional costs when activating and using this service. But only technical integration and conclusion of a contract directly with the operator will allow this to be achieved. In the meantime, negotiations are in progress, the beta version does not implement the ability to receive SMS on the phone: you can only send tweets to the system. In this issue, we are betting on the future.

Technical details of direct integration

We have never worked with SMS and SMPP before. And, when we signed an agreement with MTS, we were faced with a choice:

• or make an SMPP client manually (using modules for Perl, for example);
• or use a ready-made industrial-scale SMPP server (the most popular of them is Kannel ).

As a result, it was decided to dwell on the second version, and we did not regret it, because in the process of integration we stepped on a mass of underwater rakes, many of which were successfully resolved by Kannel.

A little bit about SMPP, SMS messaging protocol

The SMPP protocol (stands for “Short Message Peer to Peer”) is itself quite complex. It is very low-level (specifications occupy 170 pages). The difficulty is mainly due to historical reasons. The group of protocols for voice transmission was developed in 1975. At that time there were no SMS yet (there were not even LCD displays for cell phones, like the cell phones themselves). And so, in 1985, they decided to “shove” support of text messages into standard voice packets. But there was little useful space in these packets, so the messages were short, only 160 characters - Short Messages.

The useful amount of data that can be transmitted in an SMPP packet is 140 bytes. This is not a typo. How do they fit 160 characters? You probably already guessed: 1 ASCII character can be encoded with 7 bits. Thus, it turns out 160 * 7 = 1120 bits, or 140 bytes.

If we lived in the USA, where the ASCII encoding "should be enough for everyone", then we would calmly write tweets of 160 characters. However, when sending Russian letters via SMS, they are packaged in UTF-16 encoding, i.e. 2 bytes per 1 Russian letter. Therefore, in 1 message can fit no more than 140/2 = 70 characters of the Russian alphabet. If the text is longer, it is packed into 2 or more SMS messages, which are then “glued together” when displayed on the phone.

But that is not all. In 2 SMS messages, you can deliver not only 70 * 2 = 140 Russian letters, but only 134. Extra space is required to encode service information, in particular, information on the number of SMS in the chain and the number of the current SMS in it.

Kannel associates

The Kannel server we used actually has a wider range of applicability. It is intended for content provisioning, when the request is an SMS, and the answer is some SMS content: melodies, pictures, etc. We use only the part that organizes the reception and transmission of SMS: Kannel receives an SMPP message, unpacks / decrypts it and sends us a phone number and text using an HTTP request. In response, we do not send any content.

True, it was not without surprises. MTS has a very scrupulous testing scheme: we not only have to correctly receive messages, but also fit into all possible time-outs when placing requests in a queue, support throttling, send delivery confirmation messages in time with an explosive increase in load. On the test bench, we encountered a problem, to solve which we had to patch Kannel a bit (it is written in C). In short, it can be described as follows: suppose that 2 SMPP packets arrived on Kannel, and the third one ran into limiting the throughput capacity set by the MTS, and therefore got stuck in the queue. If the restriction has now been lifted, but the 4th message immediately arrived, then in the standard version, Kannel will deliver this 4th message to all previously stuck. Therefore, the 3rd message will again be at the end of the queue and will move further and further as new SMS arrives. It turns out that for MTS load tests this behavior matters - like many other aspects that had to be taken into account in direct integration.

We had no experience with SMPP before, but Kannel allowed us to scramble out. Fortunately for us, Kannel even in the base configuration handles a lot of what was needed by the PPSC. But, hand on heart, now, having already “eaten a dog” on SMPP, we would start to do everything using Perl and modules with CPAN, and not Kannel. But that would be possible only because of the experience we received with Kannel. Without this experience, programming direct integration with POPOS on Perl is the same as teaching a person to swim by throwing him off the cliff: maybe he will learn, or maybe he will drown. MTS is especially scrupulous when testing error handling, maintaining a variety of timeouts. For example, the time for sending acknowledgments * _resp to SMPP messages is clearly regulated: delays should fit into the specified MTS time intervals. If you decide to work with SMPP directly, study Kannel first: I would recommend making your own solution, repeating the Kannel architecture, especially the message buffering aspects.