Donjon | 08/10/2022

Integrating fault injection in development workflows

Fault injection vulnerabilities can be tedious to evaluate even with the right tools and experts. Could we improve this situation by giving appropriate open-source tools to developers?

As a first step toward mitigating fault injection attacks, we introduce a new open-source evaluation tool that painlessly integrates with an IDE or continuous integration testing pipelines. We illustrate the usage of this tool using the Rust programming language.

Table of contents

Fault injection simulation with Rainbow

Fault injection effects are usually simulated as corruptions during register write (bit stuck-at model) or as instructions skips¹. Security-critical embedded devices such as smart cards and hardware wallets need to be hardened using these models to guarantee that these effects do not introduce vulnerabilities in their processing.

Electromagnetic fault injection setup using a Scaffold board and a SiliconToaster.

Ledger Donjon has been developing the open-source Python side-channel and fault injection simulator called Rainbow since 2019. We recently added first-class support for simulating fault injection attacks by providing fault models:

fault_skip models fault attacks causing one instruction to be skipped during execution,
fault_stuck_at models fault attacks causing the destination register of one instruction to be overridden with a faulty value during execution. A “stuck-at zeros” model is often referred to as a bit-reset attack, and a “stuck-at ones” model is often referred to as a bit-set attack.

Using these models with Rainbow makes it possible to quickly simulate the effects of different fault injection attacks without needing access to expensive equipment, as seen above, but also removing uncertainties and measure effects like jitter from the equation. It also opens up the ability to check exhaustively that a given fault model cannot be applied to a given code.

Let’s illustrate the usage of the fault_stuck_at model on the third instruction of a PIN verification process taken from an older Trezor firmware:

We successfully faulted the output of the PIN code comparison function: rather than returning 0 as expected (1874 != 0000), it returned 1.

We can find all instructions vulnerable to a single-fault attack with these fault models if we iterate this fault simulation on every instruction. However, this method cannot find vulnerabilities caused by fault injection effects that are not modelled. Another shortcoming is that we do not expect that firmware developers will write Python code for each piece of critical code they need to harden anytime soon.

Integration in Rust development workflows

Developers are accustomed to using code style checking and testing pipelines in their daily workflows. Taking inspiration from how these tools are used, we propose a new tool called fi_check that checks for potential fault injection vulnerabilities. This tool was designed to be easily embeddable in an IDE or continuous testing pipelines, enabling developers to be alerted by code modifications that introduce single-fault injection vulnerabilities.

We only considered single-fault injection attacks to simplify the problem. A naive generalization to N�-fault injection attacks would exponentially increase the evaluation time. Protecting code against single-fault injections is still an important goal as it makes potential attacks much harder.

Writing fault injection evaluation tests in Rust

Let’s consider that compare_pin is a security-critical function that needs to be hardened against single-fault injection attacks. To define the expected behavior of this function and prepare it for automatic fault injection evaluation, one may append to their Rust source code:

This structure looks like classical Rust tests asserting that the compare_pin function returns false as the PIN codes do not match. fi_check can recognize these tests and evaluates whether it can make compare_pin return true by faulting its instructions. We do not use #[test] macro as we just need the function symbol to exist in the compiled binary to execute it later with Rainbow.

Thanks to this tool, Rust crates can be quickly evaluated for potential vulnerability to single-fault injection by:

Adding the rust_fi crate to their project dev-dependencies,
Writing fault injections robustness tests using the above structure,
Running fi_check.py on the crate.

By default, fi_check.py instantiates a Rainbow emulator configured for ARM targets, but this can be easily changed to target other architectures.

How does it work?

Successful fault injection detection:

We consider a function taking no arguments and returning one Boolean value (true or false). This function logic is written to always returns false by checking an invalid condition². In theory, this function should always return false. However, if we execute this function on real hardware, it can result in 3 different states:

Nominal behavior: the code returned false as expected,
Faulted behavior: the code returned true, meaning the disrupted execution caused the check to be skipped,
Panicked or crashed: an exception was raised during the execution, such as an out of bounds, or the device got an unexpected instruction and crashed.

Healthy hardware not under extreme conditions should always behave in the nominal behavior. In our case, we want to detect if an attack creating a single fault in the processing would be able to get a faulted behavior without raising an exception or crashing the device.

assert_eq! is a macro that raises a panic if operands differ. We can distinguish between these 3 states by using a modified assert_eq! macro in Rust.

Proposed evaluation algorithm:

We choose one of the proposed fault models, then:

We execute the function multiple times, but in each run, we apply the chosen fault model on the i�-th instruction. i� starts from the first instruction and increments until we reach the end of the function.
When the function returns true without panicking or crashing, we know which instruction makes the function vulnerable to this fault model.

If the developer is not directly working on assembly code, we use addr2line tool³, which can retrieve which line of code generated the problematic assembly instruction. This requires to compiling the code with debug symbols⁴.

Examples of code evaluation and mitigation

We will illustrate the usage of this tool with some pieces of code that are vulnerable to single-fault injection attacks once compiled to ARM Cortex-M3 assembly (ARM Thumb). A commonly used function for this kind of benchmark is the critical PIN code comparison.

Example 1: imperative-style PIN code comparison

Let’s consider the following PIN code comparison function written in an imperative-style Rust code:

The compiler outputs the following assembly code:

As expected by the calling convention used by Rust, user_pin array is represented by a pointer in r0 and a size in r1, ref_pin array is represented by a pointer in r2 and a size in r3 and the returned value is represented by r0.

We run ./fi_check.py --cli test_fi_simple to check for any interesting faults:

The output indicates vulnerable instructions in the test_fi_simple function, which is the test function calling compare_pin, so we can ignore these. It also indicates that this function is vulnerable to a bit-set fault attack. When looking at the source code, we understand that this is due to the developer initializing the returned value to true, then setting it false during comparison. This vulnerability exploitation consists in setting good=0xFFFFFFFF in the last iteration of the loop, which Rust considers to be equivalent to true.

On a side note, we also observe that the Rust compiler makes the code panic if user_pin array is accessed out of bounds (checked at 0x90) as expected from a memory-safe language.

Hardening through double call and inlining:

This compare_pin function is vulnerable to a simple fault attack. A common mitigation is to simply execute the test twice.

Running an evaluation with fi_check on this function confirms that we successfully hardened it:

Hardening using a protected Boolean type:

Boolean values are usually encoded on the first bit of a register, meaning that “stuck-at” fault injection attacks can flip its value. A method to harden these values against fault injection vulnerabilities is to change the representation of “true” and “false”. We choose the following representation on 32-bit:

This enables us to use the 31 extra bits to do error checking. We implemented these checks as a Bool Rust type.

We can then use it in our PIN verification function:

fi_check confirms that this method works:

Example 2: functional-style PIN code comparison

Sometimes it can be difficult to predict how a function will be assembled. For illustration purposes, let’s switch to functional-style code:

The compiler outputs the following assembly code:

Our tool can find 9 vulnerable points, 4 vulnerabilities with the fault_skip model, 3 with the stuck_at_0x0 model and 2 with the stuck_at_0xFFFFFFFF model:

Hardening using a protected Boolean type:

Let’s use the protected Boolean type that we described earlier:

Using the protected Boolean type, we are now down to 2 vulnerable instructions. These last two vulnerabilities are due to an early size check on the input that makes the function return true if one array is empty. In our context, we should handle these cases manually.

Now our tool no longer finds any vulnerable instructions, voilà!

Conclusion

We published the evaluation script and associated Rust crates at https://github.com/Ledger-Donjon/fault_injection_checks_demo/.

We show that we are able to simulate the effect of modelled fault injection attacks using Rainbow. Then we tightly integrate this simulator with the Rust ecosystem to demonstrate a scenario where these evaluations are relatively easy to set up for developers.

To demonstrate the integration of such tools in workflows, we opened a pull request that introduces a vulnerability: https://github.com/Ledger-Donjon/fault_injection_checks_demo/pull/13. The automated checks fail due to fi_check finding a vulnerability.

Such a tool enables developers to design new Rust types hardened against fault injection attacks. We propose an early design of a protected Boolean type and a Protected struct that hardens PartialEq traits.

M. Otto, “Fault attacks and countermeasures.” Ph.D. dissertation, University of Paderborn, 2005 ↩
We consider that the compiler does not optimize the condition. This can always be enforced with a few tricks if needed, for example with https://doc.rust-lang.org/std/hint/fn.black_box.html ↩
From GNU Binutils, available in most GNU/Linux distributions. A cross-platform version can also be installed from https://github.com/gimli-rs/addr2line. ↩
We use the release profile in Rust with debug=true. This does not increase the final binary size on flash for embedded binaries. ↩