I’ve recently been experimenting with a wicked-fun tool you may find useful called Binwalk: a “fast, easy to use tool for analyzing and extracting firmware images” including, but not limited to, UEFI images. Binwalk is written in Python and is a project of Craig Heffner and /dev/ttyS0, the fine folks who describe themselves as a “collection of hackers, professionals and hobbyists with a passionate interest in embedded systems”.
Binwalk scans through firmware images and identifies signatures matching various industry-standard file systems and file types. Alternatively, the user can supply his own list of proprietary signatures to scan for. Binwalk can extract the various components of the firmware image, and supports many forms of compression, and so therefore can even extract compressed file contents. Finally, the tool can generate graphs mapping the firmware image’s “entropy”, which is a graphical representation of real data in the image, as opposed to blank or unused space.
The latest version of Binwalk as of this writing is v2.1.1. Support for Windows is still experimental and under development. So, I used Ubuntu 15.10 and Python 2.7.9. Installation was a snap. After downloading Binwalk from Github, installing it is a matter of the following commands. For the most basic operation, only steps 1 and 2 are required, but if you want the advanced functionality of graphing and extracting compressed firmware volumes you should perform all five steps:
- sudo python setup.py install
- sudo apt-get install python-lzma
- sudo apt-get install libqt4-opengl python-opengl python-qt4 python-qt4-gl python-numpy python-scipy python-pip
- sudo pip install pyqtgraph
- sudo apt-get install mtd-utils gzip bzip2 tar arj lhasa p7zip p7zip-full cabextract cramfsprogs cramfsswap squashfs-tools
(alternatively, see the Quick Start Guide)
What Can Binwalk Do?
To give you an overview of what this tool is all about, here is the Binwalk help:
Usage: binwalk [OPTIONS] [FILE1] [FILE2] [FILE3] ...
Signature Scan Options:
-B, --signature Scan target file(s) for common file signatures
-R, --raw=<str> Scan target file(s) for the specified sequence of bytes
-A, --opcodes Scan target file(s) for common executable opcode signatures
-m, --magic=<file> Specify a custom magic file to use
-b, --dumb Disable smart signature keywords
-I, --invalid Show results marked as invalid
-x, --exclude=<str> Exclude results that match <str>
-y, --include=<str> Only show results that match <str>
-e, --extract Automatically extract known file types
-D, --dd=<type:ext:cmd> Extract <type> signatures, give the files an extension of <ext>, and execute <cmd>
-M, --matryoshka Recursively scan extracted files
-d, --depth=<int> Limit matryoshka recursion depth (default: 8 levels deep)
-C, --directory=<str> Extract files/folders to a custom directory (default: current working directory)
-j, --size=<int> Limit the size of each extracted file
-n, --count=<int> Limit the number of extracted files
-r, --rm Delete carved files after extraction
-z, --carve Carve data from files, but don't execute extraction utilities
Entropy Analysis Options:
-E, --entropy Calculate file entropy
-F, --fast Use faster, but less detailed, entropy analysis
-J, --save Save plot as a PNG
-Q, --nlegend Omit the legend from the entropy plot graph
-N, --nplot Do not generate an entropy plot graph
-H, --high=<float> Set the rising edge entropy trigger threshold (default: 0.95)
-L, --low=<float> Set the falling edge entropy trigger threshold (default: 0.85)
Binary Diffing Options:
-W, --hexdump Perform a hexdump / diff of a file or files
-G, --green Only show lines containing bytes that are the same among all files
-i, --red Only show lines containing bytes that are different among all files
-U, --blue Only show lines containing bytes that are different among some files
-w, --terse Diff all files, but only display a hex dump of the first file
Raw Compression Options:
-X, --deflate Scan for raw deflate compression streams
-Z, --lzma Scan for raw LZMA compression streams
-P, --partial Perform a superficial, but faster, scan
-S, --stop Stop after the first result
-l, --length=<int> Number of bytes to scan
-o, --offset=<int> Start scan at this file offset
-O, --base=<int> Add a base address to all printed offsets
-K, --block=<int> Set file block size
-g, --swap=<int> Reverse every n bytes before scanning
-f, --log=<file> Log results to file
-c, --csv Log results to file in CSV format
-t, --term Format output to fit the terminal window
-q, --quiet Suppress output to stdout
-v, --verbose Enable verbose output
-h, --help Show help output
-a, --finclude=<str> Only scan files whose names match this regex
-p, --fexclude=<str> Do not scan files whose names match this regex
-s, --status=<int> Enable the status server on the specified port
Furthermore, Binwalk has a great wiki located at:
Tour of Binwalk
In the following subsections I’ll test drive the tool and give an overview of what it looks like to use this tool. Hopefully this will give you an understanding of how the tool might assist your UEFI development. Note that in each case these commands are highly configurable, so you can adjust the start/stop offsets in the image, the depth of the directory structure to act upon, and many other parameters.
Search for File and File System Signatures
This is the automatic scanning that the tool does.
Search for a Custom String
Binwalk can search for strings in an image. Your search string can even include escaped octal and/or hexadecimal values.
Diff Firmware Images
Binwalk can diff multiple binary images simultaneously and report the bytes that are a) the same in all files; b) different in all files; and c) different in some of the files. An example from the Binwalk website:
Recursively Extract Files
Binwalk can traverse into an image’s file system structure and recursively extract and decompress the files onto your hard drive.
Generate an Entropy graph
An entropy analysis is important to discover important data that may not get caught by a scan for industry-standard signatures.
Scripting With Binwalk’s Python API
The Binwalk Python module can be used by any Python script to programmatically perform Binwalk scans and obtain the results of those scans. More info.
Using the Binwalk IDA Plugin
Binwalk nicely integrates into the popular IDA disassembler tool. After specifying your IDA installation directory, Binwalk will add a couple menu items which will allow IDA to use Binwalk to search an image’s signatures and opcodes and display the results directly in IDA’s user interface. More info.
Creating Custom Plugins
Users can customize and extend Binwalk through Python plugins. Plugins are useful when validating complex data structures which may be difficult or impossible to create full signatures for, or when Binwalk's actions need to be modified on a conditional basis. More info.
I was blown away by the maturity and comprehensiveness of this tool. The documentation is top-notch. Every command worked as described in the various help facilities. These qualities are NOT found in every open-source project out there, so it was a joy to experiment with Binwalk.
Binwalk shares many characteristics with the equally-wonderful UEFI Tool utility. I think determining which to use comes down to whether you prioritize working interactively with one image (maybe UEFI Tool is the better of the two) versus working in batch mode from a command line (maybe Binwalk is the better choice). UEFI Tool works in Windows, whereas Binwalk doesn’t; however, Binwalk has an API and extensibility features that UEFI Tool doesn’t—these are the kinds of factors to take into consideration when determining which tool to use. They’re both great—try them both out and see which one best meets your needs.
Do you use Binwalk in your UEFI development? If so, leave a comment and explain how. Thanks!