Foreword

There are many ways to install Linux on your computer and start using it for specific tasks. The choice of distributions is extremely wide, for every taste and color - and “for housewives” and for advanced users, allowing any level of customization, including assembly from source codes for a specific hardware. Installation of the system is, in principle, accessible to anyone who is a little bit competent PC user. And, if you don’t dive into the popular today de-jing on the topic “Linux vs Other OS”, then using this system does not require knowledge that we had to say for a new Linux user 10 years ago. From my deeply subjective point of view, in more than ten years that I have been watching the development of this system, Linux has become more friendly to beginners and got rid of many problems inherent in it in the past. And this is good.

Handmade penguins ...


On Habré already flashed a couple of articles on the topic of LFS, for example, this one , or this one . Comments to the latter suggest a logical thought - if the set of options for installing Linux and studying it is already comprehensively wide, why do we need LFS?
')
I will not tell you the stories about "how space ships plow ... and when the Earth was a fireball ...". I will answer the question posed based on my position - I collect LFS because I’m just interested to do it. I see in this process a good opportunity to look under the hood of the system. I admit that this path can hardly be called optimal. However, this article, and the subsequent ones, will be devoted to the process of manually assembling a Linux system. These articles will not be a translation of the assembly documentation — there is no particular need for this. Emphasis will be placed on the specifics and nuances of the process, which the author had to face personally. We will consider this cycle something like a neophyte diary.

1. System Requirements

To build LFS, you need a computer with GNU / Linux OS installed on it. In principle, any distribution will work. Suppose I use Arch Linux on my machines. As for the Debian-based distributions (Linux Mint, Ubuntu, etc.), they also fit, of course, but I hasten to pay attention to the following - at one of the initial stages of the work, when the environment variables are initialized, a command shell warning appears

$ source ~/.bash_profile dircolors: no SHELL environment variable, and no shell type option given 

I received this warning in Linux Mint 17 with indirectly calling dircolors (the command that color the terminal) from / etc / profile when the user re-entered and updated the environment variables with the source command. It is not critical and can be ignored.

On the host Linux system, the packages listed in the official system requirements must be installed. Most of these packages are included in the system "out of the box" or are installed by additional developer tools. In any case, they are easily installed from the official repository of a specific distribution. A good option is to use a Knoppix LiveCD, then you can install LFS on a machine that does not contain any operating system. In this sense, the choice is yours.

In addition to iron and OS, patience and perseverance are also necessary. The process of assembly and configuration is quite lengthy and time consuming. You should not engage in copy-pasting to the terminal commands from the book: firstly, your case may be unique; secondly - our goal is to understand the principle of building a workable OS, assembled from source codes, which means we will have to delve into the meaning of the processes that are taking place. Each stage of the work must be approached intelligently and without haste.

2. Minimum system build documentation

All the necessary documentation is concentrated in the book Linux Form Scratch (the link leads to the online version of the most stable release). The first advice that I would like to give to a newbie is to take the latest stable version in English. There are Russian translations, but they always lag behind the current version. In addition, the link to the latest Russian version of LFS 6.0 leads to a void, and the only translation available from the official site concerns LFS 5.0, but, to put it mildly, it is outdated. The kernel of the system and the GNU project software undergo regular changes, moreover, those versions of packages that are described in older versions of LFS are not always available for download, and if they are available, they will have to be searched manually. So our choice is the current English version of LFS.

Note : while this article was being written, a link to the translation of LFS 7.3 was found in the wiki article from LFS , which is closer to the truth. However, this translation is not yet complete - most of the books are in English.

I also want to warn the reader that the use of Current Development in the edition of the book can also turn into unforeseen difficulties. This version can be obtained from the daily updated SVN repository. The book is downloaded by the team

 $ mkdir ~/LFS && cd ~/LFS $ svn co svn://svn.linuxfromscratch.org/LFS/trunk/BOOK/ 

And it is easy to build, for example:

 $ cd BOOK $ mkdir ../html $ make BASEDIR=../html 

This version is the latest, focused on the use of the recently released "skynet-version" of the Linux 4.0 kernel.


The first problem arises already when checking checksums of downloaded packages.

 $ md5sum -c md5sums . . lfs-bootscripts-20150222.tar.bz2:  . . md5sum: :   1    

The specified archive is unpacked, the file sizes are true. Perhaps in the md5sums file containing the CRS list the error just crept in. However, in order not to risk it, it is better to take a stable version of the book.

3. Soil preparation

First of all, to deploy LFS, we need disk space. This may be a single partition (or several partitions) on the HDD, or maybe a directory in the host file system. I stopped at the version with a partition on the HDD of the virtual machine, which I had previously installed 64-bit Arch Linux.

The minimum size of the required disk space - 4 GB. I created a logical partition of 40 GB in size by formatting it in EXT4. This section must be mounted on the host system. Mount point is convenient to save in a special environment variable.

 # export LFS=/mnt/lfs 

for along this path we will have to go very often. We mount the partition with commands (in my case this is the sda6 partition, yours will probably be different in yours)

 # mkdir -v $LFS # mount /dev/sda6 $LFS 

Now we need to create two directories: $ LFS / sources - for storing tarballs with source codes; $ LFS / tools - to accommodate temporary, rugged system assembly tools

 # mkdir -v $LFS/sources # mkdir -v $LFS/tools 

Set permissions on the $ LFS / sources directory: write (+ w) is allowed for all users (a) with the option of deletion only by the owner of the directory (t)

 # chmod -v a+wt $LFS/sources 

Download two files to the $ LFS / sources directory - wget-list downloads list and md5sums package checksums list

 # wget http://www.linuxfromscratch.org/lfs/view/stable/wget-list --directory-prefix=$LFS/sources # wget http://www.linuxfromscratch.org/lfs/view/stable/md5sums --directory-prefix=$LFS/sources 

The --directory-prefix switch specifies the path where the downloaded file should be placed. After that, we proceed to downloading the source code of everything and everything that we have to install

 # wget --input-file=$LFS/sources/wget-list --continue --directory-prefix=$LFS/sources 

the --input-file switch points to the file containing the list of URLs of the packages being loaded; --continue - enables resuming of partially downloaded files, in case the connection to the network was interrupted. About 325 MB of archives with the source code of the required software as well as the patches necessary for the build will be downloaded. After downloading, you must check the integrity of the downloaded packages.

 # pushd $LFS/source # md5sum -c md5sums # popd 

The test itself is a call to the md5sum utility, which calculates the checksums of all files contained in the md5sums list passed as a parameter and compares the result with the value specified in the same list. The result, if successful, will look like this.


I paid special attention to a couple of pushd / popd commands. Although I am a dedicated linuksoid, who has been using this system for a long time, these internal bash commands were new to me. However, they are very convenient when you often have to skip the various points of the file system and then go back. The pushd command temporarily transfers you to the directory you specified in the parameter. Return back happens the popd team. In this case, the route by which you jump is organized in the form of a stack and you do not need to re-enter the path that you should go back.

Now we have everything we need to start the assembly, but it remains to perform the last preparatory operations.

4. Setting up the working environment

First, create a symbolic link to the $ LFS / tools directory in the root of the host system.

 # ln -sv $LFS/tools / 

What is it for? Generally, why do we need symbolic links? They are created to ensure uniformity of access to certain points in the file system. At the moment, in the host system the path to this directory looks like / mnt / lfs / tools. When we move to a temporary system, the root of which will be located along the path / mnt / lfs, in this temporary system the contents of the directory will be available along the path / tools. Those primary tools that we collect will access this directory along the / tools path, regardless of where they are run - this path will be hard-wired into the binaries after compilation. Therefore, by creating the specified link, we provide the ability to access this directory along the / tools path, regardless of the current location of the system root. Otherwise, immediately after the transition to the temporary system, we are waited for by failure at the first compilation.

Now we will create a new group and a new user, on behalf of whom we will work at the first stage of building the temporary system. You should not work as root exactly - for example, when configuring an installation package, you can confuse or forget to set the prefix for the installation directory and install the assembled package in the host system to break it (or complicate the package manager's work in the future). Working from an unprivileged user, who is given write access only to certain directories, will allow you to avoid trouble.

 # groupadd lfs # useradd -s /bin/bash -g lfs -m -k /dev/null lfs 

The first command creates the lfs group. The second is creating user lfs. The meaning of the parameters is as follows:

• -s / bin / bash - command shell used in user session
• -g lfs - user group
• -m indicates that you must create the home directory of the user / home / lfs
• -k / dev / null - empty directory of templates. By default, when creating a home directory, it places files and folders located in / etc / skel. If desired, this template can be changed, but we want to create an empty directory / home / lfs, which is indicated by passing a zero file as a template

We set the password to the user lfs

 # passwd lfs 

and make it the owner of the $ LFS / sources and $ LFS / tools directories

 # chown -v lfs $LFS/sources # chown -v lfs $LFS/tools 

Login as lfs

 # su - lfs 

The line in the command parameters (short for the -l or --login switch ) indicates that a “clean” session must be created for this user that does not inherit the values ​​of the environment variables from the previous session. For this session, we will create our own set of environment variables.

Create files in the user's home directory

~ / .bash_profile

 [lfs@arch-guest ~]/$ cat > ~/.bash_profile << "EOF" > exec env -i HOME=$HOME TERM=$TERM PS1='\u:w\$ ' /bin/bash > EOF 

~ / .bashrc

 [lfs@arch-guest ~]/$ cat > ~/.bashrc << "EOF" > set +h > umask 022 > LFS=/mnt/lfs > LC_ALL=POSIX > LFS_TGT=$(uname -m)-lfs-linux-gnu > PATH=/tools/bin:/bin:/usr/bin > export LFS LC_ALL LFS_TGT PATH > EOF 

The ~ / .bash_profile file defines individual user-specific settings for environment variables. Team

 exec env -i HOME=$HOME TERM=$TERM PS1='\u:W\$ ' /bin/bash 

creates a “clean” user environment (the -i switch to ignore previous settings), assigning values ​​to environment variables

• HOME - home directory, assign the current value of the variable equal to / home / lfs
• TERM - the type of terminal used, just leave the current one
• PS1 is a command prompt format. In this case, the format is set to the form user name: current working directory $, where the special character \ u is the user name, \ W is the current working directory.
• / bin / bash - the command that starts a new instance of the shell

The ~ / .bashrc file determines what settings should be made when logging into this user session.

• set + h - disables hashing in bash. The command shell remembers the paths to executable files that were launched earlier, which certainly speeds up the system. However, we will call different versions of the same programs located in different paths, so hashing will only harm us.
• umask 022 - set permissions for new files and directories set by default. A value of 022 means that files will be created with the rights 644 (rw- r-- r--) and directories with the rights 755 (rwx rx rx). Each coded umask bit means a reset of the corresponding bit in the rights attributes
• LFS = / mnt / lfs - the root of the assembled system
• LC_ALL = POSIX - defines the localization of the programs being run. POSIX sets the display standard regardless of the character encoding method.
• LFS_TGT - sets the description of the target machine used when building temporary system assembly tools
• PATH - paths for searching executable files
• export - assigns the specified values ​​to the specified variables

Separately, you should pay attention to how to create files. Why do we use the cat command, why can't we use our favorite console editor? Well, well, while we are in the host system, a text editor is available to us. But when we move to a temporary system, so we will not have any editor. Therefore, you have to use the cat command, which redirects the output stream from one file to another. The input file we have not specified - then the standard input stream will be used. The output file is the file created by us, and the completion condition is the appearance in the input stream of the character sequence of the end of the file - EOF. After entering the command, we will be able to type the text of the file being created directly in the terminal. To terminate the process, enter the string EOF. In this way, we turn the command shell into a primitive text editor.

The created scripts will be executed at each login on behalf of lfs. Update the user environment with the command

 [lfs@arch-guest ~]/$ source ~/.bash_profile 

Make sure the necessary environment variables are set.

 [lfs@arch-guest~]/$ set 

5. Build a temporary toolkit (Temporary System)

Starting to build the system, we are faced with the classic problem of "chicken and eggs." To build the system we need a compiler, linker and library. But we have none of the above, only their sources. We have a compiler in the host system, and we’ll begin to compile the necessary software. However, if you simply build the same gcc with the default settings, it will not work in the LFS system at the initial stage of its construction - it uses the dynamic layout of the necessary libraries for the work. All the code needed for operation will need to be statically linked, that is, explicitly included in the executable file.

Thus, first of all, we will cross-compile some parts of the future system in order to be able to use them when building the final version.

All work on building a temporary system is done in the $ LFS / sources directory, so we go there

 [lfs@arch-guest ~]/$ cd $LFS/sources 

For all packages being assembled, the assembly order contains the following sequence of actions (unless otherwise specified!):

• Unpack the archive with the package
  

   $ tar -pxf <package-archive-file-name> 

  
  
• Go to the unpacked directory
  

   $ cd <package-dir-name> 

  
  
• We carry out assembly instructions (each package has nuances described in detail in the documentation)
  
• Go to the directory above, back to $ LFS / source, and delete all created during unpacking and building directories
  

Packing time is different - from almost instant, to two and a half hours (the gcc build after “chrut” with all the tests lasts about that). Since the speed of computers is different, the relative unit, called the Standard Build Unit (SBU), is chosen as the unit of measure for the build time. One SBU is equal to the build time of the binutils package, which is compiled the very first. The duration of 1.0 SBU over time can be roughly estimated for your system, but more on that below, but for now

Recommendation : do not collect the system in several streams. If you have a multi-core processor, then the -j key will speed up the work, but later it may turn into a failure to pass some important tests and the unstable operation of the assembled software. Build in one thread is not so long - much more time will be spent on raking up the problems accumulated by multithreaded assembly.

We will not consider building each package — there is the LFS book itself. However, we will focus on some of the assembly’s nuances, considering compiling the most important packages.

6. Build Binutils - the first pass. SBU measurement

Binutils is a package containing utility utilities for building software. This includes the assembler (as) and linker (ld). They are required by glibc and gcc to pass some tests, so this package is built first.

Unpack the archive and go to the directory with its contents

 $ tar -pxf binutils-2.25.tar.bz2 $ cd binutils-2.25 

To build this package, it is recommended to create a separate directory, create it and go there.

 $ mkdir -v ../binutils-build $ cd ../binutils-build 

Now you need to configure the package to generate the correct Makefile.

 $ ../binutils-2.25/configure \ > --prefix=/tools \ > --with-sysroot=$LFS \ > --with-lib-path=/tools/lib \ > --target=$LFS_TGT \ > --disable-nls \ > --disable-werror 

Special attention to the keys:

• --prefix = / tools - indicates that the built packages will be installed in the / tools directory where the temporary system will be located
  
• --with-sysroot = $ LFS is the root directory of the system, relative to which libraries and other resources needed for running programs in the temporary system will be searched
  
• --with-lib-path = / tools / lib - the library search path specified for the ld linker
  
• --target = $ LFS_TGT - set the description of the target machine
  
• --disable-nls - disables the internationalization of collected packages. At the moment, it is unnecessary, and its presence can cause a number of failures.
  
• --disable-werror - disables the compilation stop when receiving warnings from the host compiler
  

After running the configurator, we will get a Makefile at the output, intended for building and installing the package. Build the package and measure the SBU

 $ time { make; } 

Time is measured by the utility time {...}, which returns the execution time of commands enclosed in braces. This will provide an opportunity to estimate SBU in order to have an idea about the build time of the remaining packages.

Note : LFS recommends measuring build and installation configuration times. But this recommendation is extensive. The main time of the machine goes to make and tests. In addition, other actions that you have to go into can be performed between make and make install. So I made the decision to perform the assessment in the manner described above, taking into account the approximate nature of the measurement, which is still determined by the overall load of the system by other work.

— 4- Intel Core i7 c 4096 , Intel Core i7-2600K (4 ) c 16 — SBU 2 32 .

SBU, , ( gcc , 63 SBU — )

, . , 64-, , .

 $ case $(uname -m) in > x86_64) mkdir -v /tools/lib && ln -sv lib /tools/lib64 ;; > esac 

 $ make install 

 $ cd .. $ rm -r binutils-2.25 $ rm -r binutils-build 

7. GCC —

. , -. , .

-

 $ tar -pxf gcc-4.9.2.tar.bz2 $ cd ../gcc-4.9.2 

: GMP, MPFR MPC. , gcc

 $ tar -pxf ../gmp-6.0.0a.tar.xz $ tar -pxf ../mpfr-3.1.2.tar.xz $ tar -pxf ../mpc-1.0.2.tar.gz $ mv -v gmp-6.0.0a gmp $ mv -v mpfr-3.1.2 mpfr $ mv -v mpc-1.0.2 mpc 

In addition, you will have to edit the gcc sources. The fact is that the search path for the linker ld in them is defined as for a normal working system. In our case, the linker is located in the / tools directory (it was compiled as part of binutils). Some of the macros should also refer to / tools, so we transform the sources using the monstrous command

 $ for file in \ > $(find gcc/config -name linux64.h -o -name linux.h -o -name sysv4.h) > do > cp -uv $file{,.orig} > sed -e 's@/lib\(64\)\?\(32\)\?/ld@/tools&@g' \ > -e 's@/usr@/tools@g' $file.orig > $file > echo ' > #undef STANDARD_STARTFILE_PREFIX_1 > #undef STANDARD_STARTFILE_PREFIX_2 > #define STANDARD_STARTFILE_PREFIX_1 "/tools/lib/" > #define STANDARD_STARTFILE_PREFIX_2 ""' >> $file > touch $file.orig > done 

-… . inux64.h, linux.h, sysv4.h, gcc/config. *.orig ( — sed), /tools ld /usr /tools. STANDARD_STARTFILE_PREFIX_1 STANDARD_STARTFILE_PREFIX_2, , . touch , -u cp ( - ) .


, ( ) . , . , *nix-.

:

 $ sed -i '/k prot/agcc_cv_libc_provides_ssp=yes' gcc/configure 

, . gcc .

 $ ../gcc-4.9.2/configure \ > --target=$LFS_TGT \ > --prefix=/tools \ > --with-sysroot=$LFS \ > --with-newlib \ > --without-headers \ > --with-local-prefix=/tools \ > --with-native-system-header-dir=/tools/include \ > --disable-nls \ > --disable-shared \ > --disable-multilib \ > --disable-decimal-float \ > --disable-threads \ > --disable-libatomic \ > --disable-libgomp \ > --disable-libitm \ > --disable-libquadmath \ > --disable-libsanitizer \ > --disable-libssp \ > --disable-libvtv \ > --disable-libcilkrts \ > --disable-libstdc++-v3 \ > --enable-languages=c,c++ 

:

• --with-newlib — , libc — ,
  
• --without-headers — ,
  
• --with-local-prefix=/tools — GCC . /usr/local
  
• --with-native-system-header-dir=/tools/include —
  
• --disable-shared — ,
  
• --disable-decimal-float, --disable-threads, --disable-libatomic, --disable-libgomp, --disable-libitm, --disable-libquadmath, --disable-libsanitizer, --disable-libssp, --disable-libvtv, --disable-libcilkrts, --disable-libstdc++-v3 — — , , C++ .
  
• --disable-multilib — 32- 64-
  
• --enable-languages=c,c++ — C/C++
  

,

 $ make $ make install 

 $ cd .. $ rm -r gcc-build && rm -r gcc-4.9.2 

8.

LFS. — , . , , , .

. gcc

• Linux-3.19 — GLibc
  
• Glibc-2.21 — C, , , .
  
  Glibc .
  
  

   $ echo 'main(){}' > dummy.c $ $LFS_TGT-gcc dummy.c 

  
  
  
  
  

   $ readelf -f a.out | grep ': /tools' 

  
  :
  
  

    [Requesting program interpreter: /tools/lib64/ld-linux-x86-64.so.2] 

  
  . 32- , 64-, .
  
  , ,
  

   $ rm -v dummy.c a.out 

  
  
• Libstdc++-4.9.2 — C++ g++
  
• Binutils-2.25 — , . , ar ( binutils), , CC, AR RANLIB
  
  

   $ CC=$LFS_TGT-gcc \ > AR=$LFS_TGT-ar \ > RANLIB=$LFS_TGT-ranlib \ >../binutils-2.25/configure \ > --prefix=/tools \ > --disable-nls \ > --disable-werror \ > --with-lib-path=/tools/lib \ > --with-sysroot 

  
  
• GCC-4.9.2 — ,
  
  

   $ CC=$LFS_TGT-gcc \ > CXX=$LFS_TGT-g++ \ > AR=$LFS_TGT-ar \ > RANLIB=$LFS_TGT-ranlib \ > ../gcc-4.9.2/configure \ > --prefix=/tools \ > --with-local-prefix=/tools \ > --with-native-system-header-dir=/tools/include \ > --enable-languages=c,c++ \ > --disable-libstdcxx-pch \ > --disable-multilib \ > --disable-bootstrap \ > --disable-libgomp 

  
  
  , , -.
  
  

   $ echo 'main(){}' > dummy.c $ cc dummy.c $ readelf -l a.out | grep ': /tools' $ [Requesting program interpreter: /tools/lib64/ld-linux-x86-64.so.2] $ rm -r dummy.c a.out 

  
  
  
• Tcl-8.6.3 — Tcl, .
  
• Expect-5.45 — , Tcl
  
• DejaGNU-1.5.2 —
  
• Check-0.9.14 — unit- C-
  
• Ncurses-5.9 — UI, ( Linux) mc.
  
• Bash-4.3.30 — . No comments.
  
• Bzip2-1.0.6 — bzip2.
  
• Coreutils-8.23 — GNU-, , cat, ls, rm
  
• Diffutils-3.3 — .
  
• File-5.22 — .
  
• Findutils-4.4.2 — . , , find
  
• Gawk-4.1.1 — GNU- awk — .
  
• Gettext-0.19.4 — .
  
• Grep-2.21 - text search utility in the input stream
  
• Gzip-1.6 - another archiver
  
• M4-1.4.17 - the macro processor used by software configuration and build tools
  
• Make-4.1 - software automatic build utility
  
• Patch-2.7.4 - utility for working with source code patches
  
• Perl-5.20.2 - Perl language interpreter
  
• Sed-4.2.2 - Streaming Text Editor
  
• Tar-1.28 - where do without it
  
• Texinfo-5.2 - utilities for working with info pages of documentation
  
• Util-linux-2.26 is the main linux utility package. Contains, for example, mount
  
• Xz-5.2.0 is another archiver.
  

9. Trimming debug information

After everything has been collected, it is recommended to save debugging information from the collected binaries to save space. Make it teams

 $ strip --strip-debug /tools/lib/* $ /usr/bin/strip --strip-unneeded /tools/{,s}bin/* 

, —

 $ rm -rf /tools/{,share}/{info,man,doc} 

Total

, $LFS/tools , GNU- . .

!

To be continued...