Hello!

Today we would like to share our list of materials on reverse engineering (RE). This list is very extensive, because our research department is primarily concerned with the tasks of RE. In our opinion, the selection of materials on the topic is good for the start, while it may be relevant for a long time.

We have been sending this list of links, resources, books for five years to people who would like to get into our research department, but they don’t yet pass by the level of knowledge or are just starting their way in the field of information security. Naturally, this list, like most of the materials / collections, will need updating and updating in some time.

Funny fact: we were shown how some companies send out our list of materials from themselves, but only in a very old edition. And after this publication, they will finally be able to use its updated version with a clear conscience;)

So, go to the list of materials!

1. Topics
   a. Reverse
   b. Search for vulnerabilities (fuzzing)
   c. Vulnerability Exploitation
   d. Malicious code analysis
2. Instruments
   a. IDA Pro
   b. Radare2
   c. WinDBG (Ollydbg / Immunity Debugger / x64dbg)
   d. Gdb
   e. DBI
   f. SMT
   g. Python to automate
   h. BAF (Binary Analysis Frameworks)
3. Architecture
   a. x86-x86_64
   b. ARM
4. OS
   a. Windows
   b. Linux
   c. Mac OS (OSX) / iOS
   d. Android
5. File formats
   a. PE
   b. ELF
   c. Mach-o
6. Programming
   a. C / C ++
   b. Assembler
7. Practice
   a. War games

1. Topics

In this section, we will look at the main areas of RE application. Let's start directly from the reverse development process itself, move on to finding vulnerabilities and developing exploits, and, of course, let's get to malware analysis.

1.a Reverse engineering

• Chris Kaspersky’s “The Art of Disassembling ” is not new, but a very good and up-to-date book from Chris with good systematization of knowledge and excellent material;
• " Practical Reverse Engineering: x86, x64, ARM, Windows Kernel, Reversing Tools, and Obfuscation " - a "new" book from several well-known information security specialists covering some new moments and topics that are missing from Chris's book;
• Reversal for Beginners by Denis Yuryevich is a completely free book, already translated into many languages ​​of the world. Here, probably, the most remarkable thing is the presence of interesting tasks after each chapter, while for several architectures at once;
• " Practical RE tips " - an excellent webinar in English from Gynvael Coldwind, containing many useful tips and scripts about RE;
• The resource " OPENSECURITYTRAINING.INFO " contains good educational lectures and videos on RE in English;
• " Digging Through the Firmware " is a good series of Practical Reverse Engineering articles - useful articles for those who are just about to dive into the world of device firmware reversal;
• " Training: Security of BIOS / UEFI System Firmware from Attacker and Defender Perspectives " - if you want to plunge into the world of firmware security, UEFI BIOS, then you definitely need to familiarize yourself with these slides that were previously in paid training at leading security conferences;
• CRYPTO101 - a little introduction to cryptography, without which it can not do.

1.b Vulnerability Scan

• " Fuzzing: Brute Force Vulnerability Discovery " - although not a new book, it’s just right for understanding the basics of fuzzing. There is a translation into Russian, but it contains rather funny blunders;
• " Automatic search for vulnerabilities in programs without source texts " - a good introductory material in Russian, presented at PHDays 2011;
• " The Evolving Art of Fuzzing " - an article about the development of fuzzing;
• " Modern Security Vulnerability Discovery " - a compilation of different techniques for finding vulnerabilities in one document;
• " (State of) The Art of War: Offensive Techniques in Binary Analysis " - an all-in-one document on all existing vulnerability scan techniques;
• " The Art of Software Security Assessment: Identifying and Preventing Software Vulnerabilities " is far from being a new, but still relevant, book on various approaches to finding vulnerabilities.

1.c Examples of exploitation of found vulnerabilities

• " Exploit Writing Tutorials by Corelan Team " ( translation ) - a famous series of posts on writing exploits and shellcodes, starting with the basics;
• " Exploit Development Community " ( partial translation ) - a series of articles on writing a combat exploit for IE 10 and 11 versions;
• " Modern Binary Exploitation " - materials from the RPISEC team from the training course they conducted at the Rensselaer Polytechnic Institute;
• " Web-archive of the blog company Vupen " - submerged blog with examples of exploiting complex vulnerabilities in VirualBox, XEN, Firefox, IE10, Windows Kernel, Adobe Flash, Adobe Reader;
• " Project Zero " - a blog from the Google research team, where their experts often share interesting stories on the exploitation of various cool vulnerabilities;
• " Browser mitigations against memory corruption vulnerabilities " - protection technologies used in popular browsers:
  
  • " Browsers and app specific security mitigation. Part 1 "
  • " Browsers and app specific security mitigation. Part 2. Internet Explorer and Edge "
  • " Browsers and app specific security mitigation. Part 3. Google Chrome "
• " SoK: Eternal War in Memory " is an excellent document that shows the attack model and describes various mechanisms to prevent exploitation at different stages for different types of vulnerabilities associated with memory corruption;
• " Writing Exploits for Win32 Systems from Scratch " - a detailed article on writing an exploit from scratch for a vulnerability in the SLMAIL program;
• Phrack - the famous hacker magazine Phrack. We recommend reading, first of all, the articles of the category "The Art of Exploitation";
• " The Shellcoder's Handbook: Discovering and Exploiting Security Holes " is a legendary book on shellcode writing.

1.d Malware Analysis

• " Practical Malware Labs " - source for the book " Practical Malware Analysis ";
• " Malware Analyst's Cookbook and DVD: Tools and Techniques for Fighting Malicious Code " - we recommend this and the previous book with one set to those interested in this topic;
• " Malware Analysis Tutorials: a Reverse Engineering Approach " ( translation ) is a rather long series of articles devoted to setting up an environment with subsequent analysis of malware in it;
• " Course materials for Malware Analysis by RPISEC " - another course from RPISEC, only now about malware;
• " Computer viruses and antiviruses. Programmer's view " - even though the book examines malicious programs starting from the DOS times, it will still be useful, because in addition to analyzing the code of such programs, the author shows examples of writing antiviruses for each specific case.

2. Necessary tools

Below are the popular tools used in RE.

2.a IDA Pro

• " The IDA Pro Book: The Unofficial Guide to the World Popular Disassembler " is a book that will make your acquaintance with IDA Pro easy and relaxed :)
• " TiGa's Video Tutorial Series on IDA Pro " - a selection of small HOW-TO videos using IDA Pro;
• " Open Analysis Live " - in contrast to the previous selection on the use of IDA Pro, this newer and updated. Mostly, malware analysis is considered.

2.b Radare2

• " The radare2 book " - the main book on the use of the Radare2 framework for reverse;
• " Radare2 Cheatsheet " - "cheat sheet" for the main teams;
• " Radare Today - the blog of radare2 " - a framework blog. There are not only news, but also practical examples.

2.c WinDBG (Ollydbg / Immunity Debugger / x64dbg)

Without knowledge of the principles of the debugger and the ability to use it, too, can not do. Below we look at the debuggers for Windows, and in the next paragraph we will pay attention to the famous GDB. So, let's go:

• Advanced Windows Debugging: Developing and Administering Reliable, Robust, and Secure Software - first of all, this book is useful for understanding and “catching” errors like heap damage;
• " Inside Windows Debugging: A Practical Guide to Debugging and Tracing Strategies in Windows " - this edition will well complement the previous book;
• "Introduction to cracking from scratch using OllyDbg" - unfortunately, the oldest resource wasm.ru was closed, but such a compilation is easily searched because it has been duplicated on many resources. In addition, "forks" began to appear on the network, only they are already using x64dbg or IDA.

2.d gdb

• " gdb Debugging Full Example (Tutorial): ncurses " - a guide for using GDB;
• " GEF - Multi-Architecture GDB Enhanced Features for Exploiters & Reverse-Engineers " - add-on GDB over the Python language, adds a lot of new useful commands that are useful for developing exploits;
• " GEF Tutorials " is a series of screencasts on using GEF.

2.e DBI

Programmable debugging is today an essential approach in the arsenal of any reverser. And DBI is one of the tools. More details:

• " Dynamic Binary Instrumentation in Information Security " - this article has already collected some generalized information about DBI;
• " Light And Dark Side Of Code Instrumentation " - this presentation will help you navigate in the varieties of various code tools and in what and when you can help with the analysis of programs.

2.f SMT

What is SMT solver? In short, an SMT solver is a program that can solve logical formulas.

The basic idea of ​​using SMT in the field of software security is to translate a program code or algorithm into a logical formula, and then use a SMT solver to check one or another property of this code.

In other words, SMT provides a mathematical tool for semantic code analysis.
SMT solvers have been used in our field for quite some time. They are well established for the following tasks:

• search for bugs (static analysis / fuzzing);
• deobfuscation;
• "home" cryptanalysis;
• character execution (as an "engine");
• There are also some successes in the field of automatic exploit generation (for example, ROP generation).

During this time, SMT lost the aura of mystery, more or less working tools for “ordinary” people appeared.

Below are sources that will help to plunge into the topic:

• " SMT Solvers for Software Security, Sean Heelan, Rolf Rolles " - perhaps the first scientific work in which the application of SMT was proposed for solving software security problems. Gives an idea of ​​where and how SMT can find its place in this area;
• Z3 is one of the most popular and effective SMT solvers;
  
  • Z3 wiki - project repository;
  • " Getting Started with Z3: A Guide " - online tutorial, SMT solver for experiments;
  • Z3Py - binding in Python for Z3;
  • " Experimenting with Z3 - Dead code elimination ";
  • " Experimenting with Z3 - Proving opaque predicates ";
• " Theorem prover, symbolic execution and practical reverse-engineering " - a good overview presentation, with examples of solving real-world problems and using Z3Py;
• " Quick introduction into SAT / SMT solvers and symbolic execution " ( Russian version ) is a good book with interesting practical examples.
• " An introduction to the use of SMT solvers " - review material.

2.g Python for Automation

Today, without basic knowledge of Python, it will be very difficult, because this programming language is considered the most popular means for automating various tasks in the field of information security (and not only). In addition, it is used in various utilities (for example, all the above utilities allow you to add functionality with the help of this PL):

• " Gray Hat Python " ( translation ) is a great book that will tell you how useful Python is in reverse;
• " The Beginner's Guide to IDAPython " - a free book on IDAPython;
• " Python Arsenal for Reverse Engineering " is a resource dedicated to various utilities and libraries for reverse engineering using Python.

2.h BAF (Binary Analysis Frameworks)

For a bit more advanced, we recommend paying attention to whole frameworks, which in their composition use the previously mentioned mechanisms and analysis tools for solving more complex problems. So, here they are:

• " Overview and Usage of Binary Analysis Frameworks " - a small overview of BAF;

Some interesting frameworks / tools:

• Triton
  
  • Developer Usage Examples
  • " Dynamic Binary Analysis and Obfuscated Codes "
  • How can Triton help virtual machine based software protections
• Angr
  
  • Solving kao's toy project with symbolic execution and angr
• Ponce
• Binary Analysis Platform .

3. Architecture

We will cover only a few popular architectures. At the end of the article in the section with additional materials you will find information on many others (MIPS, PowerPC, etc.).

3.a x86-x86_64

• " Intel 64 and IA-32 Architectures Software Developer Developers " - previously, such manuals were sent by mail, but because of the large amount of material in them, printing became expensive. Recommended as a desktop reference.

3.b ARM

• Azeria Labs (ARM Assembly Basics & ARM Exploit Development) - a site with articles on the basics of ARM-assembler and the development of exploits for this architecture;
• The course " Introduction to ARM " - a two-day video course dedicated to ARM-development and operation;
• VisUAL - visualization of the work of ARM-commands.

4. OS

Knowledge of the principles of work of popular Operating Systems.

4.a Windows

• " Windows Internals " - the fundamental book for understanding the work of Windows.
  The following points, although mainly related to the exploitation of vulnerabilities in this OS, but allow you to learn more about the insides of Windows:
  
  • " Windows exploits, mostly precompiled "
  • " Exploit Development Environment "
  • " Windows Breakout from Defcon24 "
  • " Part 10: Kernel Exploitation -> Stack Overflow "
  • " Kernel and Driver explotation ".

4.b Linux

• " Linux insides " is an analogue of the Windows Internals book, but only for OS such as Linux.
  As in the case of Windows, the following topics are related to the development of exploits:
  
  • " Heap Exploitation into Linux "
  • " A series of tutorial for linux exploit development to newbie "
  • " Linux Kernel Exploitation "
  • " Programming Linux Anti-Reversing Techniques "

4.c Mac OS (OSX) / iOS

• " Reverse Engineering Resources Mac and iOS " - a selection of materials on this topic.

4.d Android

• " Android Hacker's Handbook " - probably the most popular book dedicated to the safety of the Android OS;
• " Android Internals :: Power User's View " - a book that tells about the internal mechanisms of this OS. Due to recent leaks, the material appeared in the public domain, about which the author himself writes on his website and provides the opportunity to download the previous version.

5. Executable file formats

This section provides links explaining the details of popular executable file formats.

5.a PE

• " PE sections ";
• " PE Title ";
• " Windows executable file format. PE32 and PE64 ";
• " Computer viruses inside and out ."

5.b ELF

• " Linux x64 Infection for Lamers (by a Lamer) ."

5.c mach-o

• " Parsing mach-o files "

The famous researcher corkami makes very useful and interesting "posters" with the scheme of various file formats, including those mentioned above. We recommend using them as a cheat sheet. A utility Kaitai Sctruct will help in the analysis.

6. Programming

One of our friends once said that a good reverser is 80% a good programmer. The ability to program and understand what is being done and why simplifies the process of researching someone else's program. Therefore, without programming in the reverse nowhere. And of course, the automation of routine tasks, as you probably already understood, is a very useful thing;)

6.a C / C ++

• Modern Memory Safety: C / C ++ Vulnerability Discovery, Exploitation, Hardening is a great course with excellent examples. Just must have stuff for everyone.

6.b ASM

• " A Crash Course in x86 Assembly for Reverse Engineers " - an "accelerated course" for diving in x86 Assembler, positioned as special for RE;
• " Assembly Programming Tutorial " - assembly programming manual, with the ability to run examples online in the course of study;
• " Assembler. 2nd edition " - it is recommended to use as a reference;
• " x86 Assembly Guide " - online version.

7. Practice

This section provides links to virtual machines and online resources to practice.

7.a War Games

• SmashTheStack Wargaming Network - this multi-wargame network is maintained by volunteers and is available online. We recommend starting with it;
• BinTut - local wargame;
• Reversing Workshop - a master class in solving tasks from the annual competition "The Flare On Challenge" for 2016;
• Exploit-Challenges - a selection of vulnerable ARM binary files;
• ARM Reverse Engineering Exercises - the original repository "disappeared", but one of the forks was found in the github;
• CTF Time - here you can find out the schedule of future CTF-events and read the solutions of the past.

And finally, a few links with a large number of materials on the above topics:

• Selection, generally devoted to the field of information security
• About exploitation of vulnerabilities
• About reverse engineering:
  
  • Awesome-reversing
  • REMath Resource Overview
• About the exploitation of vulnerabilities in Windows
• About phasing
• Pro Malware Analysis
• And a lot of different awesome collections.