Geekly Articles each Day
⬆️ ⬇️

How Android works, part 2



In this article, I will talk about some of the ideas on which the high-level parts of Android are built, about several of its predecessors, and about the basic security mechanisms.



Articles series:







Speaking about Unix- and Linux-roots of Android, you need to remember about other projects of operating systems, the influence of which can be traced in Android, although they are not its direct ancestors.



I have already mentioned BeOS, a legacy from which Android got Binder.



Plan 9 from Bell Labs



Plan 9 - a descendant of Unix, a logical continuation, the development of his ideas and bring them to perfection. Plan 9 was developed at Bell Labs by the same team that created Unix and C — people like Ken Thompson, Rob Pike, Dennis Ritchie, Brian Kernighan, Tom Duff, Doug McIlroy, Bjarne Stroustrup, Bruce Ellis, and others worked on it.



In Plan 9, the interaction of processes with each other and with the system core is not realized through numerous system calls and IPC mechanisms, but through virtual text files and file systems (the development of the Unix principle “everything is a file” ). In addition, each process group “sees” the file system in its own way ( namespaces , namespaces), which allows you to run different parts of the system in a different environment.



For example, to get the position of the mouse cursor, applications read the text file /dev/mouse . The rio window system provides each application with its own version of this file, in which only events related to the window of this application are visible and local coordinates are used with respect to the window. Rio itself reads the events of a “real” mouse through the same file /dev/mouse - in the form in which it sees it. If it is launched directly, this file is provided by the kernel and actually describes the movements of a real mouse, but it can be completely transparently launched as an application under another copy of rio, without any special support from her.



Screenshot Plan 9



Plan 9 fully supports access to remote file systems (it uses its own 9P protocol, in addition, it supports FTP and SFTP), which allows programs to completely transparently access remote files, interfaces, and resources. Such “native” network transparency turns Plan 9 into a distributed operating system — the user can physically be behind one computer running rio, run applications on several others, use the files stored on the file server and perform calculations on the CPU server — All this is completely transparent and without special support from each part of the system.



Due to the beautifully designed architecture, Plan 9 is much simpler and smaller than Unix - in fact, the core of Plan 9 is even several times smaller than the well-known Mach microkernel.



It is not necessary to add away.

Despite the technical superiority and the presence of a layer of compatibility with Unix, Plan 9 is not widespread. However, many of the ideas and technologies from Plan 9 became widespread and were implemented in other systems. The most famous of them - the UTF-8 encoding, which was developed in Plan 9 to ensure full Unicode support while maintaining backward compatibility with ASCII - has become a generally accepted standard.



Most of the ideas and technologies from Plan 9 are implemented in Linux:





Much of this is used, including, in Android. In addition, Android has an intent mechanism, similar to the plumber from Plan 9; I will tell about it in the next article.



You can learn more about Plan 9 on plan9.bell-labs.com ( saved copy on the Wayback Machine ), or its mirror 9p.io



Inferno



Plan 9 was continued as an Inferno project, also developed at Bell Labs. To the properties of Plan 9, as simplicity and distribution, Inferno adds portability . Programs for Inferno are written in the high-level Limbo language and are executed — using just-in-time compilation — by the virtual machine built into the Inferno kernel.



Inferno is so portable that it can run.





In this case, applications running inside Inferno are provided with exactly the same environment.



Inferno has received even less distribution and fame than Plan 9. On the other hand, Inferno largely anticipated Android, the most popular operating system in the world.



Danger



Danger Research Inc. It was co-founded by Andy Rubin (Andy Rubin) in 1999, 4 years before the co-founding of the same Android Inc. in 2004.



In 2002, Danger released their smartphone Danger Hiptop. Many of the developers of Danger subsequently worked on Android, so it is not surprising that its operating system was in many ways similar to Android. For example, it implemented:





Danger hiptop



Read more about Danger in the article by Chris DeSalvo, one of the developers, called The future that everyone forgot .



Java



Although the use of high-level languages ​​for serious development is no longer surprising anyone, of the popular operating systems, only Android has a native language — high-level Java (on the other hand, you can recall the web with its JavaScript, .NET for Windows, and relatively high-level). compiled into native code and not using garbage collection - Swift).



Despite the seeming flaws (“Java combines the beauty of C ++ syntax with python execution speed”), Java has many advantages.



First, Java is the most popular (by a wide margin) programming language. Java has a huge ecosystem of libraries and development tools (including build systems and IDEs). About Java written many articles, books and documentation. Finally, there are many qualified Java developers.



Java programs, like many other high-level languages, are portable between operating systems and processor architectures (“Write once, run anywhere”). Practically, this is manifested, for example, in the fact that Android applications work without recompilation on devices of any architecture (Android supports ARM, ARM64, x86, x86–64 and MIPS).



Unlike low-level languages ​​like C and C ++, which use manual memory management, Java memory is automatically controlled by the runtime environment. The Java program does not even have direct memory access, which automatically prevents several classes of errors, often leading to crashes and vulnerabilities in programs written in low-level languages ​​- “hanging links” are impossible (due to which use-after-free occurs) null pointer dereferencing (when you try to do this, a NullPointerException thrown), reading of uninitialized memory and going beyond the array boundaries.



Using full garbage collection (compared to automatic reference counting) eliminates the programmer from all the problems and difficulties with circular references and allows you to implement even more advanced (advanced) dependencies between objects.



This makes development for Android more enjoyable than development using low-level languages, and Android applications much more reliable, including security.



Running Java is ART



Unlike most other high-level languages, Java programs are not distributed as source code, but compiled into an intermediate format (bytecode), which is an executable binary code for a special processor.



Although attempts are being made to create a physical processor that would directly execute Java bytecode, in most cases a Java virtual machine (JVM) is used as such a processor. A commonly used implementation from Oracle / OpenJDK called HotSpot .



Android uses a proprietary implementation called Android Runtime (ART), specially optimized for use on mobile devices. In older versions of Android (up to 5.0 Lollipop), instead of ART, another implementation called Dalvik was used .



Both Dalvik and ART use their own bytecode format and a proprietary file format that stores bytecode DEX ( Dalvik executable ). Unlike class files in “regular Java”, all Java code of an application is usually compiled into one classes.dex DEX file . When building an Android application, the Java code is first compiled with a regular Java compiler into class files , and then converted into a DEX file with a special utility (possibly an inverse transformation ).



Both HotSpot, Dalvik, and ART further optimize the executable code. All three use just-in-time compilation (JIT), that is, at run-time they compile bytecode into pieces of completely native code that is executed directly. In addition to the obvious gain in speed, it allows you to optimize the code for execution on a specific processor, without abandoning the full portability of the bytecode.



In addition, ART can compile bytecode to native code in advance rather than at run time ( ahead-of-time compilation ) - and the system automatically schedules this compilation while the device is not in use and connected to charging (for example, at night). In this case, ART takes into account the data collected by the profiler during previous launches of this code ( profile-guided optimization ). This approach allows you to further optimize the code for the specifics of a particular application, and even for the specifics of using this application by this particular user.



As a result of all these optimizations, the performance of Java code on Android is not much inferior to the performance of low-level code (in C / C ++), and in some cases it exceeds it .



Java bytecode, unlike regular executable code, uses the Java object model — that is, things like classes, methods, and signatures are explicitly written in bytecode. This makes it possible to compile other languages into Java bytecode, which allows programs written on them to run on a Java virtual machine and to be more or less compatible (interoperable) with Java.



There are both JVM implementations of independent languages ​​— for example, Jython for Python, JRuby for Ruby, Rhino for JavaScript, and the Lisp Clojure dialect — and languages ​​originally developed for compiling to Java bytecode and running on JVM, the most famous of which are Groovy , Scala and Kotlin .



The newest of them, Kotlin , specially designed for perfect compatibility with Java and with a much more pleasant syntax (similar to Swift), is supported by Google as the official development language for Android along with Java.



Kotlin on Android logo



Despite all the advantages of Java, in some cases it is still desirable to use a low-level language — for example, to implement a performance-critical component, such as a browser engine, or to use an existing native library. Java allows you to invoke native code through the Java Native Interface (JNI), and Android provides special tools for native development — the Native Development Kit (NDK), which includes header files, a compiler (Clang), a debugger (LLDB), and an assembly system .



Although NDK is mainly focused on using C / C ++, it can be written under Android and in other languages ​​- including Rust , Swift , Python , JavaScript, and even Haskell . Moreover, there is even the ability to port iOS applications (written in Objective-C or Swift) on Android with virtually no changes.



About security



Classic Unix



The security model in classic Unix is ​​based on the UID / GID system — special numbers that the kernel stores for each process. Processes with the same UID are allowed access to each other, processes with different UIDs are protected from each other. Similarly, access to files is limited.



In terms of meaning, each UID (user ID) corresponds to its user — during the time of Unix creation, the situation when one computer was simultaneously used by many people was normal. Thus, in Unix, processes and files of different people were protected from each other. To allow sharing of certain files, users were grouped into groups with which the GID (group ID) corresponded.



At the same time, all programs launched by the user are given full access to everything that this user has access to. Actually, since the user cannot communicate with the kernel directly, but interacts with the computer through the shell and other processes — the user 's rights are the rights of programs running on his behalf.



This model implies that the user completely trusts all the programs that he uses. At that time it was logical, because the programs most often either were part of the system, or were created (written and compiled) by the user.



In Unix, there is an exception to the access restrictions - UID 0, which is called root . He has access to everything in the system, and no restrictions apply to him. This account was used by the system administrator; in addition, many system services run under UID 0.



In modern Linux, this model has been significantly expanded and generalized, including the capabilities that allow "to get some root-rights" and the SELinux subsystem that implements mandatory access control (MAC), which allows you to further restrict rights (including root access).



Everything has changed



For several decades, past from the creation of Unix to the creation of Android, the practice of using computers ("calculators") has changed significantly.



Instead of machines designed for parallel use by many users (via terminals , what emulators of terminals are now emulating), personal computers designed for use by one person appeared. Computers have ceased to be only a working tool and have become the center of our digital life. With the advent of mobile devices - first a PDA, then smartphones, tablets, smart watches, etc. - this trend has only intensified (because working on mobile devices is relatively uncomfortable).



On such devices are stored gigabytes of personal information, access to which must be protected and limited. At the same time, the market for third-party applications flourished, which the user has no reason to trust.



Thus, in modern conditions, instead of protecting different users from each other, it is necessary to protect other applications, user data, and the system itself from applications. In addition, viruses that commonly use vulnerabilities in the system are widespread - to protect against them, you need to further protect parts of the system from each other, so that the use of a single vulnerability does not allow an attacker to access the entire system.



Android



Although some Android applications come with the system — for example, standard applications such as Calculator, Clock, and Camera — most users install applications from third-party sources. The most famous of them is the Google Play Store , but there are others, for example, F-Droid , Amazon Appstore , Yandex.Store , Chinese Baidu App Store , Xiaomi App Store , Huawei App Store , etc. In addition, Android allows you to manually install arbitrary applications from APK files (this is called sideloading).



Like other Unix-like systems, Android uses the existing UID / GID mechanism to restrict access. At the same time, unlike traditional use, when UIDs correspond to users, in Android different UIDs correspond to different applications . Since the processes of different applications run with different UIDs, already at the kernel level, applications are protected and isolated from each other and do not have access to the system and user data. This forms a sandbox (Application Sandbox) and allows the user to install any applications without having to trust them.



In order to gain access to user data, camera, making calls, etc., the application must receive permission from the user. Some of the permissions exist as GIDs to which the application is added when it receives this permission — for example, obtaining the ACCESS_FM_RADIO permission puts the application into the media group, which allows it to access the /dev/fm file. The rest exist only at a higher level (in the form of entries in the packages.xml file) and are checked by other components of the system when accessing the high-level API via Binder.





A small part of the system services in Android runs under UID 0, that is, root, but most use dedicated UID numbers, increasing their rights if necessary using Linux capabilities. In addition, Android uses SELinux — using SELinux on Android is called SEAndroid — to further restrict what actions applications and system services are allowed to perform.



Android usually does not provide the user with direct access to the root account, but in some cases he has the opportunity to get this access. How this happens, why it is needed and what dangers it threatens, I will tell later.





In the next article (which comes out in a week), I’ll talk about the components that make up Android applications, and about the ideas behind this architecture.



')

Source: https://habr.com/ru/post/338292/

More articles:



All Articles