There is a wealth of validation utilities and frameworks available to UEFI developers—some might say too many. Here’s a summary of those I am aware of:
- UEFI Self-Certification Test (SCT)
- Intel BIOS Implementation Test Suite (BITS)
- Linux UEFI Validation project (luvOS)
- Firmware Test Suite (FWTS)
- Platform Initialization Self-Certification Test (pi-sct)
I have recently been playing around with the SCT version 2.4 and wanted to share my results.
Description
The SCT is a toolset for BIOS developers to validate UEFI implementations. It supports IA32, x86-64, and ARM architectures. The SCT tries to make sure that a UEFI implementation properly implements all the UEFI interfaces. It is an automated test, and runs in the same spirit as Microsoft’s Windows Logo Kit tests, albeit with a different emphasis.
Package
The SCT packages can be downloaded from the Test Tools section of UEFI.org. The .ZIP file package is a collection of smaller .ZIP files:
- Documents.zip – documentation
- EMS.zip – an application to be run in an OS on a separate host machine to facilitate network testing
- IHV-Sct_Binary.zip – for IHVs to test their UEFI drivers
- UEFI-Sct_Binary.zip – the SCT tests
Documentation
The documentation is pretty thorough. Here’s what you get:
UEFI_2_4B_SCT_II_GettingStarted.pdf
This document is not so much “Getting Started Running the SCT”, as it is “Getting Started Building the SCT”. If you’re looking to build the test suite yourself, start with this document. Otherwise, you can just use the pre-built binaries available in the download from UEFI.org.
UEFI_2_4B_SCT_II_CaseSpec.pdf
This is a reference manual for all the possible test assertions and what conditions trigger assertion. If you’re familiar with a unit test framework like JUnit, this is like an English-language description of all the unit tests. You DO NOT read this document cover-to-cover. (it’s over 1800 pages!) It’s only a reference manual for shedding light on tests that fail during a test run.
UEFI_2_4B_SCT_II_UserGuide.pdf
This is by far the most meaningful document. The User Guide is the document that, after a brief introduction, proceeds to tell the reader step-by-step how to run the various tests. (there are even screenshots!) The User Guide discusses stand-alone running of the SCT, the networked “passive” mode, plus the IHV tests.
Running the Tests
The tests are run from the UEFI shell. I copied the X64 version of the SCT to a USB flash drive that also contained UEFI shell version 2.40. Then I configured the system to boot to the UEFI shell. I configured the shell to be the first entry in the UEFI boot order since the SCT will reboot several times during testing and needs to get back to the shell.
The first thing you need to do is install the tests. I ran InstallX64.efi, which copies the 100MB worth of tests to \SCT on the selected drive.
Next, from the shell I switched to \SCT, the directory containing SCT.efi, the test binary. There are command-line options for SCT.efi, and there is also a text-mode graphical interface for that resembles a text-mode BIOS Setup program. I chose the GUI by running SCT.efi –u, which opens with:
For an extra dose of fun, I selected ALL the tests by going to the Test Case Management screen and highlighting each test and pressing Space to select it:
Selecting all the tests took a long time to run; I’m not exactly sure, but it was over five hours to run all the tests. Keep that in mind when planning your own test run.
Test Results
The SCT generated a 26MB log file in the directory \Overall called Summary.log. This is useful for digging into the details of the test runs.
For a more user-friendly test report, I selected “Test Report Generator …” from the SCT main menu, which created a .csv-formatted report suitable to opening in Microsoft Excel or other spreadsheet application. This results file had over 21000 rows in it. Here are some screenshots of what the report looks like:
Conclusion
Of course, running the SCT is fun and straight-forward enough. The hard part comes in parsing the tool’s output, investigating the failures, debugging, and implementing solutions. This is left as an exercise for the reader.
The SCT is a fun and easy-to-use tool to help validate UEFI implementations. How do you use the SCT in your organization? Leave a comment below!
Thanks, This is a very useful info
ReplyDelete