Equip a hamster or “OpenWrt does firmware from TP-LINK?”. Part 1 - preparatory

“With this half-chair, Master Gambs begins a new batch of furniture. 1865 St. Petersburg "(I. Ilf, E. Petrov).
You were not mistaken, this is another article about building alternative firmware for home routers, these hot masters of our small local area networks in some places. On a reasonable question, why not immediately write where the material is the place I answer: I want to describe the process starting with preparing a virtual machine for assembly (yes, it will be Ubuntu on VirtualBox), going through the firmware build, connecting an external hard drive and setting up the boot from connect the printer, install and configure packages like a media server and a torrent client, and try to end with a description of such a StrongSwan setting that allows you to connect to it from an IPhone, iPad or Windows Phone without installing third-party clients. At the same time, I would like not to ignore important things like safety, as well as useful trifles like a script that will turn off the blue LEDs of my TL-WDR3600 at night. And I am not at all sure of the fullness of my knowledge on many of the issues that have been voiced now.
In short, at this stage I have a lot of plans and confusion. Therefore, any tips and suggestions are important. The text of the article will be exactly changed and supplemented over time, and the entire article may be hidden in drafts by a vote so as not to litter Habr. If you are interested, you want to help with advice, or you just wanted to throw in the author with a sneaker - welcome. Further on, I will try to describe in detail this promising interesting pastime action, the essence of which was depicted by a medieval Japanese artist.

“Weapons - To Inspect!”

First, briefly about the features of the standard firmware TL-WDR3600 from the review of the device stated on the manufacturer's website:

• Simultaneous connections — up to 300 Mbit / s at 2.4 GHz and up to 300 Mbit / s at 5 GHz allow you to achieve a total bandwidth of 600 Mbit / s.
• Two USB 2.0 ports allow users to easily share a printer and files on a local network or via the Internet
• Gigabit ports allow maximum data transfer rates
• With hardware NAT, the maximum bandwidth from WAN to LAN reaches 800 Mbps
• IGMP Proxy Support and Bridge Mode for IPTV Multicast

From all this magnificence, I would like to separately mention the “ease” of sharing printers, implemented through a scary-looking proprietary utility. Offhand, most of the above is easily implemented using OpenWrt. But let's not forget, our goal is not to catch up with TP-LINK firmware. We want to find out how OpenWrt can bypass it.

The transition to Openwrt can be done in several ways. For supported devices, you can download the finished image, upload it directly from the factory firmware. Further, if there is a USB port, a pre-partitioned and formatted USB flash drive is connected to the router. The boot from the flash drive is configured, the necessary packages are installed. As the saying goes: “Madame! Faustpatron is easy. "
You can build your own image from source with the help of OpenWrt Buildroot , this thing builds the firmware as a Frankenstein monster , collecting pieces of source codes from multiple addresses. The process is often interrupted, the more often, the more time has passed since the release date. Then the newly appeared Victor Frankenstein with the help of adhesive plaster in the form of endless patches and symlinks should make his miracle machine work. This method opens up many possibilities, for example, you can force BusyBox to display Cyrillic characters instead of question marks in file names. But most likely you will have to build all the necessary packages yourself, most of the packages from the OpenWrt repository will become unavailable.
We will try to stay on the middle ground and use Image Builder , which promises us a relatively easy build of firmware from pre-compiled packages and compatibility of our firmware with packages from the OpenWrt repository.

What we will definitely lose, not counting the time spent:

• The warranty on the device, the warranty states that the user loses the warranty if:

  Malfunction of the goods caused by the use of embedded software, which was not approved by the manufacturer and was not published on the official website of the manufacturer. Also for malfunctions caused by a violation of the product software update technique or a failure in the update process, in particular, power off or pressing the reset button during the program code update procedure.

• Hardware NAT will not work either, and there is an opinion that it will never work in the foreseeable future. Although there also claim that it really does not work and is not needed in native firmware.

"To the distributor of ammunition, give the change of three live ammunition!"

• Download and install Oracle VM VirtualBox and Oracle VM VirtualBox Extension Pack.
• Download the installation disk image of your favorite familiar Linux distribution. If not, you can download the current LTS version of Ubuntu Linux. In this case, I recommend downloading the network installer from the alternative downloads page - it is smaller, and during installation you will be able to select the necessary packages, for example, the desktop environment.
  I, when working in VirtualBox, have a default shell - Unity has braked severely even after installing the add-ons of the guest system. I did not try to figure out why and if it can be cured, but simply at the stage of selecting packages, noted the Xubuntu desktop , based on Xfce . Regarding the capacity of the guest system, I think that if you are not going to give the virtual machine more than 4 GB of RAM, i386 will be a logical choice.
• We create a virtual machine, choose Ubuntu Linux with the same bitness as the downloaded image as a guest operating system, 1 GB will suffice for the start, a hard disk of about 20 GB will also suffice. After creating the machine, we reconfigure the network, instead of NAT, we select a network bridge. For a laptop, you need to choose a network card instead of a wifi adapter, if you have one. In the optical drive, mount the image of the installation disk.
• Install the guest operating system. At this stage, everything is trivial, except that at the stage of disk partitioning, I recommend not agreeing to automatic partitioning, but manually create two primary partitions. The first - 2 GB under the swap, the rest - at the root. This disk layout will allow, if necessary, to quickly change the size of the root partition after increasing the virtual hard disk.
• After installing the operating system on the virtual machine in VirtualBox, several standard screen resolutions will be available. Among them you most likely will not find suitable for the laptop or the monitor with the big diagonal. We will correct this misunderstanding and install VirtualBox add-ons for the guest system. At the same time put the console file manager - Midnight Commander .
  To begin, add the missing screen resolution. As far as I know , you can add up to 16 user-defined screen resolutions. In the example below, “Test” is the name of the virtual machine, and “1366x768x32” is the added screen resolution. Open the command line of the host system, go to the VirtualBox directory and add the desired screen resolution:
  

  cd "C:\Program Files\Oracle\VirtualBox" VBoxManage setextradata "Test" "CustomVideoMode1" "1366x768x32" 

• Go to the guest system. Open the terminal emulator, update the package lists, install the Midnight Commander and the packages needed to build the kernel modules - dkms and build-essential . Hereinafter, I assume that you are working with Ubuntu or another Debian-like system, and aptitude and nano are already installed. When installing from the network installer image, it will. If this is not the case, correct the commands from the examples for yourself:
  

   sudo aptitude update && sudo aptitude upgrade sudo aptitude install dkms build-essential mc 

  We connect the guest's add-ons disk image to the virtual machine, run the add-on installation script from it:
  

   cd /media/%username%/VBOXADDITIONS_4.3.20_96996/ sudo ./VBoxLinuxAdditions.run 

  After the script completes, close the terminal, disable the add-ons image of the guest system, and reboot the virtual machine. If everything went well, then when you go to full-screen mode, the resolution of the virtual machine screen will correspond to the normal resolution of your monitor. But so far only in graphical mode, when you go to the console, you still see that it occupies a small part in the center of the screen. To fix this, you need to tweak the bootloader settings a bit. Open the terminal emulator, edit the bootloader settings:
  

   sudo nano /etc/default/grub 

  Customize the display of the Grub menu at boot:
  

   #GRUB_HIDDEN_TIMEOUT=0 

  Set the resolution of the Grub graphic menu, save the resolution set by the bootloader for the console:
  

   # The resolution used on graphical terminal # note that you can use only modes which your graphic card supports via VBE # you can see them in real GRUB with the command 'vbeinfo' GRUB_GFXMODE=1366x768 GRUB_GFXPAYLOAD_LINUX=keep 

  Save the settings, update the boot loader configuration and reboot:
  

   sudo update-grub && sudo reboot 

  After updating the bootloader configuration, the menu did not appear when I restarted. If you have the same, turn off and then turn on the virtual machine. When launched in full-screen mode, the bootloader menu should appear in full screen, and when switching to the first console using Ctrl + Alt + F1, it also takes up the entire screen. If everything is the same, you can enter your login, password and customize fonts. Everything here is of course individual, but for me the standard font usually seems a bit small. If you, too, do this:
  

   sudo dpkg-reconfigure console-setup 

  If you like bold fonts - choose TerminusBold, if not - Terminus. Size - according to your vision, you can try a few to decide.
  I'm afraid to insult you with petty care, but in Ubuntu you can return from the console to the graphics mode by using Ctrl + Alt + F7.
• Download and unpack your version of Image Builder. I collected the firmware for the TL-WDR3600 , for it you need to select the ar71xx generic architecture. Refine the architecture for your device and find out if it is supported by OpenWrt in the table of supported devices . We unpack the archive into the home directory and, for convenience, shorten the name to OpenWrt-ImageBuilder. Go to OpenWrt-ImageBuilder and create a directory of files for the files that we will add to the firmware.
• To make the firmware build work properly, I had to tweak the script a bit. The essence of the problem lies in the fact that the “make clean” command does not clear the dl directory in which the files downloaded during the build are saved, but deletes it. Then I even found a ticket dedicated to this error, marked as fixed. To make it work, open the terminal emulator and edit the Makefile, we find the lines 161-162:
  

   clean: rm -rf $(TOPDIR)/tmp $(TOPDIR)/dl $(TARGET_DIR) $(BIN_DIR) 

  And we lead to this form:
  

   clean: rm -rf $(TOPDIR)/tmp $(TOPDIR)/dl/* $(TARGET_DIR) $(BIN_DIR) 

  After this small revision with a file, you can carry out a trial build, it is done with the command:
  

   make image PROFILE=WNDR3700 PACKAGES="pkg1 pkg2 pkg3 -pkg4 -pkg5 -pkg6" FILES=files/ 

  Where: WNDR3700 - your device profile; pkg1 - package to be added to the firmware; -pkg4 - package that will be removed from the firmware. The list of profiles can be viewed in the output of the “make info” command. You can read more about working with Image Builder in the OpenWrt Wiki . Before you start building, you need to add repositories to the repositories.conf file. Open it in the editor and bring it to this form:
  

   src/gz barrier_breaker_base http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/base src/gz barrier_breaker_luci http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/luci src/gz barrier_breaker_management http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/management src/gz barrier_breaker_oldpackages http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/oldpackages src/gz barrier_breaker_packages http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/packages src/gz barrier_breaker_routing http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/routing src/gz barrier_breaker_telephony http://downloads.openwrt.org/barrier_breaker/14.07/ar71xx/generic/packages/telephony ## This is the local package repository, do not remove! src imagebuilder file:packages 

  We have added the official Openwrt repositories. Considering the bulk of our plans and to facilitate the launch of the firmware collector, create a small script in the OpenWrt-ImageBuilder directory:
  

   #!/bin/bash # Build OpenWRT 14.07 for TP-LINK TL-WDR3500/3600/4300/4310/MW4350R echo "Prepare OpenWRT images for Tp-Link TL-WDR3500/3600/4300/4310/MW4350R \n" make image PROFILE="TLWDR4300" PACKAGES="luci openssh-sftp-server nano" FILES="files/" 

  Save the script, for example, as tl-wdr3600.sh, allow its execution, and run the firmware build:
  

   chmod +x tl-wdr3600.sh ./tl-wdr3600.sh 

  At the end of the build, several files should appear in the bin directory. You can determine which of them is your firmware by the device model in the file name. The word factory in the name means that such a file can be uploaded to your device over the factory firmware. In more detail about it, once again about a difference between JFFS2 and SquashFS, and also I am going to tell about many other things further.
  In the meantime, “Private Hamster is ready for battle!”, To be continued ...