A little more about Xiaomi phones and fighting with them. Updated

Honestly, I didn’t have plans to write and publish this article, but after two months I saw 5 colleagues in the closest circle of colleagues from Xiaomi’s newly purchased phones, and a recent article on Geektimes advertising smart home management from Xiaomi to me conscience came and, bitch , demanded to share knowledge with the rest.

For a start, a small introduction for those who are not in the subject. There is a company Xiaomi, which makes quite good stuff on the phone and floods them with customized Android. The business model, as it was officially announced recently, “In fact, we distribute our smartphones, without earning money from it. We care more about long-term sources of income. We could sell 10 billion smartphones and not earn a cent from them. ” Source one and two .

Glancing at the September article on Security lab and yet this complaint, I personally had the feeling that the Xiaomi phone is something like a leash on which the Big Brother drives the owner (I exaggerate, of course).
')
This was the main motive for conducting a study of the behavior of the phone Xiaomi redmi 3S
with firmware MIUI Global 8.1 Stable 8.1.1.0 (MALMIDI)

Test rabbit test and problem detection
I take a new phone out of the box. I turn it on and go through the initial setup wizard by first turning on the traffic recording on the Wi-Fi router. Exactly two seconds later, after the phone connected to the access point, the download of a file of about 8 MB in size from one of the Xiaomi servers began. It was a regular zip archive, inside of which lay a lot of everything, including the AnalyticsCore.apk file mentioned in the article on SecurityLab.

Further more. In total, for all the observation time, I counted a little less than eight dozen server names in different domains. I’ll just say that there are no Google and Facebook servers in this number, the applications of which are also preinstalled. Just because I counted them separately. With them, too, everything is "fun."

Most of the connections to the Xiaomi servers were via HTTPS, so it was not possible to figure out the details exactly what was transmitted directly. Disabling all sorts of logins, syncs, etc. to the disappearance of this traffic has not led.

Additionally, it was embarrassing that most of the requests were small (the amount of received transmitted TCP traffic of sessions did not exceed 1-2 Kb), but since Our mobile operators round up the traffic volume up (for example, Tele2 to 150Kb), then, if an unsuccessful coincidence occurs, you can “pump” substantial amounts of traffic in this way, and in roaming you can suddenly get money.

Those who are not confused by this fact can no longer read, because Further, there will be a description of the specifics of isolating traffic from applications embedded in the factory firmware.

Prerequisites

The first thing you need is to rant the phone. How this is done in the case of Xiaomi, I will not describe here, referring those who wish to go this way to the full version of this article (link at the end).
The second is to inject the firmware through the cable into the phone and erase ALL user data.
Third, the phone MUST NOT have access to the Internet after a fresh firmware download.
Update. Until the installation of the following limitations, of course.

Disclamier. All further manipulations on the phone you do at your own peril and risk.
Responsibility for any outcome lies with who exactly did the actions described below.

Small technical introduction

The servers accessed by the phone are mostly located in the Amazon cloud, so calls are made to them by names that are resolved via round-robin DNS to different IP addresses from different subnets / 16. Blocking them all by subnets makes no sense - you can filter out half of the Internet, which is not good. Blocking by name is good, but not the fact that host names from L3 domains are not dynamically generated. It would be ideal to nail all applications that access Xiaomi servers, but, as practice has shown, the depth of their integration into Android is such that after removing some of them, the phone may simply refuse to boot.

Further. External servers are accessed not by one process, but by many, and the task is complicated by the availability of Android UID sharing, when different processes (applications) can generate network traffic under one UID. Moreover, one of the useful processes (responsible for GPS) must be released to the outside world in order to download small updates, but at the same time it sat under the same UID as the eight pieces of processes rushing to Xiaomi servers.

It is also necessary to mention the limitations of the tools available for solving the above tasks, because Most of the applications with the name of the firewall available on the Play Market work through the so-called. VPN, i.e. they do not protect them from information plums to launching the application.

Much of what will be discussed later for professional Android developers is a banal truth, but for everyone else this will make it possible to understand why filtering is built in this way.

Unlike regular Linux, where there are configuration files and startup scripts in / etc, everything is different in Android. The overall network management is performed by the Connection Manager, which is pulled by the netd system daemon, which, in turn, calls iptables with certain command line parameters. Accordingly, there is no special reason to call IPtables from the initial boot script (init and others) - netd will still invoke iptables at the start, clear the rules and flood its own ones.

The only solution left by Google is to write the necessary iptables configuration commands in the /system/bin/oem-iptables-init.sh script. The path to this script and its name are hard-coded inside the source code of the netd daemon.

To filter static host names, you can edit the / etc / hosts file, but be aware of their number and the possibility of their dynamic generation.
Next will be the story of how it all was done.

Remove and freeze (if not sure) unnecessary programs

With the free version of Titanium Backup, you can see the correspondence between the program name shown in the system (Play Market), its codename (com.google.vending) and, if necessary, delete what is clearly not necessary.

The disadvantage of the free version is that it does not know how to freeze programs, therefore we freeze through the ADB shell using the package manager. Example:

root@land:/ # pm disable com.miui.analytics
pm disable com.miui.analytics
Package com.miui.analytics new state: disabled
root@land:/ # pm disable com.miui.systemAdSolution
pm disable com.miui.systemAdSolution
Package com.miui.systemAdSolution new state: disabled
root@land:/ # reboot
reboot

Disclamier 2. «» . — .

.

1. C — /etc/hosts c IP 127.0.0.1. Google Drive Files.
— L3/L4.

Update. Netfilter/IPtables. , . — . -, .
--wait, . , , IPtables , , , , , .

2. /16 /24 Netfilter/IPtables oem-iptables-init.sh. , , .
Update. — Amazon (round-robin DNS) IP . /16, . . ( ) .

3. DNS . , .

Updated. IPtables, Android , . , DNS (UID 0) :

$IPTABLES -A oem_out --protocol udp --dport 53 -m owner --uid-owner 0 -m string --algo bm --hex-string '|04|miui|03|com|00|' -m comment --comment "Deny UID 0 DNS queries for miui.com domain" -j DROP
#
$IPTABLES -A oem_out --protocol udp --dport 53 -m owner --uid-owner 0 -j ACCEPT

Updated. UDP , (UID 0) 53 UDP IP 046d69756903636f6d00 ( DNS .miui.com). IPtables |04|miui|03|com|00| 046d69756903636f6d00.
--hex-string — , IPtables . - DNS , . (00h).

DNS . , iptables -L -v .

4. Assited GPS QualComm UID 1000. , .. , DNS , — TCP c SYN, ACK , HTTP , HTTP . TCP , .
3-4 :

#    TCP   80      UID 1000.
$IPTABLES -A oem_out -m owner --uid-owner 1000 --protocol tcp --dport 80 -m conntrack --ctstate NEW -j ACCEPT
#   xtrapath   TCP     80    UID 1000        5555.
$IPTABLES -A oem_out -m owner --uid-owner 1000 --protocol tcp --dport 80 -m conntrack --ctstate ESTABLISHED -m string --algo bm --string 'xtrapath' -j CONNMARK --set-xmark 0x5555
#       UID 1000 TCP       5555
$IPTABLES -A oem_out -m owner --uid-owner 1000 --protocol tcp --dport 80 -m conntrack --ctstate ESTABLISHED -m connmark ! --mark 0x5555 -j DROP

5. ( Google Chrome UID 10060). Google Chrome .

$IPTABLES -A oem_out -m owner --uid-owner 10060 -m comment --comment "Permit Google Chrome internet access" -j ACCEPT
#
# Block all other processes
#
$IPTABLES -A oem_out -m owner --uid-owner 0-9999 -m comment --comment "Block all other system processes internet access" -j DROP
$IPTABLES -A oem_out -m owner --uid-owner 10000-99999 -m comment --comment "Block all other user processes internet access" -j DROP

UID Netfilter/IPtables. TCP Google, UID. , Google Captive portal login. — ADB shell:

root@land:/ # settings put global captive_portal_detection_enabled 0
root@land:/ # reboot

, ( Wi-Fi ), UID .

Update. . «» , 127.0.0.1, .
.
, , :

$IPTABLES -A oem_out --protocol all --source 127.0.0.0/8 --destination 127.0.0.0/8 -m comment --comment "Accept internal traffic" --jump ACCEPT
$IPTABLES -A oem_out --protocol all -m owner ! --uid-owner 0-99999 -m comment --comment "Drop any traffic which does not have UID." --jump DROP

Update. , UID , , / . , Netfilter/IPtables .
, , , , , UID ( ) Netfilter/IPtables.
— — . , , — . /dev . , .
, .

# Permit intenet access for the packages listed at the end of this file. White list mode. 
#
SU=`/system/bin/which su`
# changing reading file behavior (read whole file with \r\n into variable)
IFS=""
# reading first and second fields of every line of the system packages database into variable PACKAGESDB. 
# Escalating privileges via su because of filesystem packages database file access limitations.
PACKAGESDB=`$SU -c "/system/bin/cut -d' ' -f 1,2 /data/system/packages.list"`
#
# Reading last lines of current script form the end till "exit 0" line
# Filtering empty lines, lines started with # and all symbols after # (comments) in every line.
# 
# 's/#.*//' - remove all in every line after #
# '/^#/d' - remove lines staring with #
# '/./!d' - remove empty lines
# '/exit 0/,$ d' - remove all lines starting line with "exit 0"
# 's/ //g' - remove spaces from line
#
/system/bin/tac $0 | /system/bin/sed -e '/^#/d' -e 's/#.*//' -e '/exit 0/,$ d' -e '/./!d' -e 's/ //g'| while read line;
do
# Just in case 8-)
OUR_PACKAGE_NAME=$line
# Strict checking for existence of our package name in the system packages database. Checking first field.
PACKAGE_NAME_IN_DB=`echo $PACKAGESDB | /system/bin/cut -f 1 -d' ' | /system/bin/grep -Fx "$line"`
if
# Checking grep utility exit code. "0" means pattern found
test "$?" == "0"
then
#
# Looking for package UID in database. Checking second field. VERY important space after $line!!!
#
PACKAGE_UID=`echo $PACKAGESDB |  /system/bin/grep "$line " | /system/bin/cut -f 2 -d' '`
else
# All other exit codes return us to the beginning of the cycle.
# echo "Package $OUR_PACKAGE_NAME not found"
$IPTABLES -A $CHAIN -m comment --comment "Package name $OUR_PACKAGE_NAME not found. Check package name." --jump LOG
continue
fi
#
# Set the package right for Internet access
# 
$IPTABLES -A $CHAIN -m owner --uid-owner $PACKAGE_UID -m comment --comment "Permit $OUR_PACKAGE_NAME Internet access" -j ACCEPT
#
done
######
... skipped...
####
exit 0
#### 
####### Do NOT edit before this line #########
# Please add package names and comments after this line for granting them internet access.
#####
# Google Play Store and its companion processes
# 
com.google.android.gms # Google Services Framework Internet access
com.android.vending # Google Play Market internet access
com.android.providers.downloads # Download manager service internet access
#
# Other Google apps
com.google.android.youtube # Youtube application internet access
com.google.android.apps.maps # Google Maps application internet access
com.google.android.googlequicksearchbox # Google Assistant internet access
#
#
com.android.chrome # Google Chrome browser internet access

/ .

6. Netfilter/IPtables :

$IPTABLES -A oem_out --source 10.1.30.42 --protocol tcp --jump LOG --log-prefix "IPtables log:" --log-uid

IP (--source 10.1.30.42) , , 127.0.0.1 hosts. dmesg (dmesg | grep IPtables) ADB Shell.

, Xioami Redmi 3S Google Drive. - .

P.S. Android-, . , , — . .

P.P.S. Zyxel Keenetic Extra. Wi-Fi .