The Binwalk Firmware Analysis Tool

I’ve recently been experimenting with a wicked-fun tool you may find useful called Binwalk:  a “fast, easy to use tool for analyzing and extracting firmware images” including, but not limited to, UEFI images.  Binwalk is written in Python and is a project of Craig Heffner and /dev/ttyS0, the fine folks who describe themselves as a “collection of hackers, professionals and hobbyists with a passionate interest in embedded systems”.

Binwalk scans through firmware images and identifies signatures matching various industry-standard file systems and file types.  Alternatively, the user can supply his own list of proprietary signatures to scan for.  Binwalk can extract the various components of the firmware image, and supports many forms of compression, and so therefore can even extract compressed file contents.  Finally, the tool can generate graphs mapping the firmware image’s “entropy”, which is a graphical representation of real data in the image, as opposed to blank or unused space.

 

Installation

The latest version of Binwalk as of this writing is v2.1.1.  Support for Windows is still experimental and under development.  So, I used Ubuntu 15.10 and Python 2.7.9.  Installation was a snap.  After downloading Binwalk from Github, installing it is a matter of the following commands.  For the most basic operation, only steps 1 and 2 are required, but if you want the advanced functionality of graphing and extracting compressed firmware volumes you should perform all five steps:

1. sudo python setup.py install
2. sudo apt-get install python-lzma
3. sudo apt-get install libqt4-opengl python-opengl python-qt4 python-qt4-gl python-numpy python-scipy python-pip
4. sudo pip install pyqtgraph
5. sudo apt-get install mtd-utils gzip bzip2 tar arj lhasa p7zip p7zip-full cabextract cramfsprogs cramfsswap squashfs-tools

(alternatively, see the Quick Start Guide)

 

What Can Binwalk Do?

To give you an overview of what this tool is all about, here is the Binwalk help:

Usage: binwalk [OPTIONS] [FILE1] [FILE2] [FILE3] ...

Signature Scan Options:
    -B, --signature              Scan target file(s) for common file signatures
    -R, --raw=<str>              Scan target file(s) for the specified sequence of bytes
    -A, --opcodes                Scan target file(s) for common executable opcode signatures
    -m, --magic=<file>           Specify a custom magic file to use
    -b, --dumb                   Disable smart signature keywords
    -I, --invalid                Show results marked as invalid
    -x, --exclude=<str>          Exclude results that match <str>
    -y, --include=<str>          Only show results that match <str>

Extraction Options:
    -e, --extract                Automatically extract known file types
    -D, --dd=<type:ext:cmd>      Extract <type> signatures, give the files an extension of <ext>, and execute <cmd>
    -M, --matryoshka             Recursively scan extracted files
    -d, --depth=<int>            Limit matryoshka recursion depth (default: 8 levels deep)
    -C, --directory=<str>        Extract files/folders to a custom directory (default: current working directory)
    -j, --size=<int>             Limit the size of each extracted file
    -n, --count=<int>            Limit the number of extracted files
    -r, --rm                     Delete carved files after extraction
    -z, --carve                  Carve data from files, but don't execute extraction utilities

Entropy Analysis Options:
    -E, --entropy                Calculate file entropy
    -F, --fast                   Use faster, but less detailed, entropy analysis
    -J, --save                   Save plot as a PNG
    -Q, --nlegend                Omit the legend from the entropy plot graph
    -N, --nplot                  Do not generate an entropy plot graph
    -H, --high=<float>           Set the rising edge entropy trigger threshold (default: 0.95)
    -L, --low=<float>            Set the falling edge entropy trigger threshold (default: 0.85)

Binary Diffing Options:
    -W, --hexdump                Perform a hexdump / diff of a file or files
    -G, --green                  Only show lines containing bytes that are the same among all files
    -i, --red                    Only show lines containing bytes that are different among all files
    -U, --blue                   Only show lines containing bytes that are different among some files
    -w, --terse                  Diff all files, but only display a hex dump of the first file

Raw Compression Options:
    -X, --deflate                Scan for raw deflate compression streams
    -Z, --lzma                   Scan for raw LZMA compression streams
    -P, --partial                Perform a superficial, but faster, scan
    -S, --stop                   Stop after the first result

General Options:
    -l, --length=<int>           Number of bytes to scan
    -o, --offset=<int>           Start scan at this file offset
    -O, --base=<int>             Add a base address to all printed offsets
    -K, --block=<int>            Set file block size
    -g, --swap=<int>             Reverse every n bytes before scanning
    -f, --log=<file>             Log results to file
    -c, --csv                    Log results to file in CSV format
    -t, --term                   Format output to fit the terminal window
    -q, --quiet                  Suppress output to stdout
    -v, --verbose                Enable verbose output
    -h, --help                   Show help output
    -a, --finclude=<str>         Only scan files whose names match this regex
    -p, --fexclude=<str>         Do not scan files whose names match this regex
    -s, --status=<int>           Enable the status server on the specified port

 

Furthermore, Binwalk has a great wiki located at:

https://github.com/devttys0/binwalk/wiki

 

Tour of Binwalk

In the following subsections I’ll test drive the tool and give an overview of what it looks like to use this tool.  Hopefully this will give you an understanding of how the tool might assist your UEFI development.  Note that in each case these commands are highly configurable, so you can adjust the start/stop offsets in the image, the depth of the directory structure to act upon, and many other parameters.

 

Search for File and File System Signatures

This is the automatic scanning that the tool does.

 

Search for a Custom String

Binwalk can search for strings in an image.  Your search string can even include escaped octal and/or hexadecimal values.

 

Diff Firmware Images

Binwalk can diff multiple binary images simultaneously and report the bytes that are a) the same in all files; b) different in all files; and c) different in some of the files.  An example from the Binwalk website:

 

Recursively Extract Files

Binwalk can traverse into an image’s file system structure and recursively extract and decompress the files onto your hard drive.

 

Generate an Entropy graph

An entropy analysis is important to discover important data that may not get caught by a scan for industry-standard signatures.

 

Advanced Usage

 

Scripting With Binwalk’s Python API

The Binwalk Python module can be used by any Python script to programmatically perform Binwalk scans and obtain the results of those scans.  More info.

 

Using the Binwalk IDA Plugin

Binwalk nicely integrates into the popular IDA disassembler tool.  After specifying your IDA installation directory, Binwalk will add a couple menu items which will allow IDA to use Binwalk to search an image’s signatures and opcodes and display the results directly in IDA’s user interface.  More info.

 

Creating Custom Plugins

Users can customize and extend Binwalk through Python plugins. Plugins are useful when validating complex data structures which may be difficult or impossible to create full signatures for, or when Binwalk's actions need to be modified on a conditional basis.  More info.

 

Conclusion

I was blown away by the maturity and comprehensiveness of this tool.  The documentation is top-notch.  Every command worked as described in the various help facilities.  These qualities are NOT found in every open-source project out there, so it was a joy to experiment with Binwalk.

Binwalk shares many characteristics with the equally-wonderful UEFI Tool utility.  I think determining which to use comes down to whether you prioritize working interactively with one image (maybe UEFI Tool is the better of the two) versus working in batch mode from a command line (maybe Binwalk is the better choice).  UEFI Tool works in Windows, whereas Binwalk doesn’t; however, Binwalk has an API and extensibility features that UEFI Tool doesn’t—these are the kinds of factors to take into consideration when determining which tool to use.  They’re both great—try them both out and see which one best meets your needs.

Do you use Binwalk in your UEFI development?  If so, leave a comment and explain how.  Thanks!