Twitter is watching you, anonymous

Often you hear that large corporations like Google, Facebook or LG follow every bunch of unhappy users . And having researched some of their software, I have to admit that yes, they follow.

Yesterday I investigated the most recent (at the time of this writing) official iOS client Twitter version 5.13.1 . I, among other things, was interested to look at the traffic between the client and the server. Since the Twitter client uses certificate pinning , its traffic is quite difficult to intercept using Fiddler2 and similar programs. Therefore, I wrote a define command for GDB that clings to a breakpoint at [NSURLConnection start] and dumps the NSURLConnection fields of interest to me to the NSURLConnection just before calling this start . Such a simple, in general, focus allows you to see what exactly the program sends to the server before it is “what exactly” is encrypted for further transmission via HTTPs.

And you know what I saw? The Twitter client, besides its usual requests (user registration, authorization, tweet, etc.) often ... no, not so ... very often sent requests to a mysterious URL to the POST server api.twitter.com/1.1/jot/client_event api.twitter.com/1.1/jot/client_event . These requests went to the server for almost any reason. I started the client - the request went. I clicked the “Sign in” button (I have not logged in yet, just pressed the button) - the request went. I got out the login screen, the "Username" field received focus - the request went. I rolled the client - the request went ... Of course, I was surprised and helpful to understand.
')
Typical request for api.twitter.com/1.1/jot/client_event api.twitter.com/1.1/jot/client_event looked like this (for obvious reasons, I blurred some of the fields):



Having looked at a couple of dozens of such requests, I roughly understood the general structure:

  URL: https://api.twitter.com/1.1/jot/client_event
 Method: POST
 Headers:
   X-Twitter-API-Version: 5
   Authorization: OAuth ... la la la la, regular OAuth authorization, signed with a secret key (without quotes)
                  "GgDYlkSvaPxGxC4X8liwpUoqKwwr3lCADbz8A7ADU & fgFuHBbtKUk1ykYzwEKaYeNg28H0XEvztfVQCEu8GA"
   X-Twitter-Client: Twitter-iPhone
   X-Twitter-Client-Version: 5.13.1
   Geolocation: my coordinates, damn it (but if you turn off the Location Service - this field disappears from the requests)
   Accept-Language: en
   X-Twitter-Client-DeviceID: my device ID
   X-Client-UUID: a unique UUID (generated once when the client is first started, does not change further) 
   User-Agent: Twitter-iPhone / 5.13.1 here is the version of iOS (Apple; iPhone3,1 ;;;;; 1)
 Body (of course, URL-encoded):
   lang = en & log = <cite> ... there’s an interesting JSON (see below) ... </ cite> & send_error_codes = 1 

Of course the most interesting thing here was in JSON in the request body. In all requests it was an array of elements of the following form:

  [
   ...
   {
    "event_name": event name
    "ts": UNIX timestamp,
    "format_version": 2,
    "_category _": "client_event",
    "client_version": "Twitter-iPhone \ 5.13.1 here is the version of iOS (Apple; iPhone3,1 ;;;;; 1)"
   }
   ...
 ] 

For example, when I pressed the “Sign in” button (I hadn’t logged in yet, just clicked my button), the server would get a request with something like JSON:

  [
   {
    "event_name": "iphone: login ::: impression",
    "ts": 1385678390003,
    "format_version": 2,
    "_category _": "client_event",
    "client_version": "Twitter-iPhone \ 5.13.1 iOS \ 6.1.3 (Apple; iPhone3,1 ;;;;; 1)"
   }
 ] 

I looked at other requests (there were many, yes), and it looks like the event_name , ts , format_version , _category_ and client_version are required. They were at each element of JSON'a, in each request. But besides them, there were additional fields. For example, when the “Username” field on the login screen received focus, the following JSON was left in the request:

  [
   {
    "_category _": "client_event",
    "event_name": "iphone: ddg: digits_iphone_signup_1337 :: experiment",
    "client_version": "Twitter-iPhone \ 5.13.1 iOS \ 6.1.3 (Apple; iPhone3,1 ;;;;; 1)",
    "ts": 1385678389354,
    "experiment_key": "digits_iphone_signup_1337",
    "format_version": 2,
    "version": 12,
    "bucket": "uncheck_follows"
    }
 ] 

Probably, the experiment_key , version and bucket fields contain some additional information about what and how I'm doing. Well that would be clearer to the guys on Twitter. What information this is and what these magical 12 and uncheck_follows I don’t know yet, but I might be going to explore this question more carefully.

That's all. At the end of the article, it would be necessary to say something so clever, but at four in the morning nothing comes to mind. Unless I do not like how Twitter through its application almost follows my every move and knows in detail what, when and (given the Geolocation header) where I do with their program. Right up to which button I clicked and when. Perhaps this is some necessary and useful experiment to collect my user experience, not for nothing that in some requests there is an experiment_key field.

And I don’t like that this application eats away my precious (literally) GPRS traffic with its endless “spy” requests. Well, at least I'm not roaming, and then with these Twitter experiments for a couple of days you can stay without pants ... well, something I grumbled: “I don’t like, I don’t like ...” I’ll probably go to bed.

Good Friday everyone :)

PS Sorry if the title seemed to someone yellowish. Thank you all for the views in the comments, it turns out an interesting discussion. At the moment, the points of view of the commentators are divided: someone does not like such meticulous and detailed tracking by the official Twitter client. Others say that they say nothing terrible, for example, sites for a long time collect statistics on user actions, and applications already. Mmmm ... I don't even know. In the case of sites, the user has at least a theoretical opportunity to compete for their privacy (using the same Ghostery for example). And applications from the App Store are open source, HTTPs, some certificate pinning and there is no “don't follow me” checkbox. It turns out that there are almost no chances to prevent the collection of statistics on the actions of an ordinary user from this very ordinary user. Although it can really do not worry, it's just that I'm already old and behind the times, and I should get used to the wondrous new world. But still, I hope not.