Recon Montreal 2019

»Vectorized Emulation: Putting it all together«
2019-06-28, 14:00–15:00, Grand Salon

Take a peek into arguably the fastest fuzzer in the world, with Vectorized Emulation. I'll take about how it works, how I made it fast, and talk about how we found over 10 DHCP vulnerabilities in Windows with it.

Summary

Vectorized emulation leverages AVX-512 to run 8 64-bit (or 16 32-bit) VMs in parallel per core. By running 8 VMs in lock step, we can determine very cheaply when a VM diverged due to the input. This allows us to track what aspects of the fuzz input caused a change for almost zero cost. With this information, we are able to gather more than just code coverage information. Instead we track register, memory, and stack state in addition to traditional code coverage.

Since we are running 8 VMs per core, we're also able to fuzz, with full coverage, above native x86 execution speeds. While the performance is awesome, the ability to do hardware accelerated taint tracking and feedback allows solver-like abilities in the fuzzer. We'll focus more on these aspects than the actual performance and design.

Intro

AVX-512 offers a new CPU feature called kmask registers. These kmask registers are fundamental to allowing this technology to exist, so I'll cover the basics of how this technology works, and what AVX-512 is.

There's a brief intro blog on this:

https://gamozolabs.github.io/fuzzing/2018/10/14/vectorized_emulation.html

MMU Design

A subject of it's own blog, getting the performance of a JIT-based MMU fast enough to work in this environment required some exotic design. We'll talk briefly about what we did here to achieve the performance we did.

MMU design blog:

https://gamozolabs.github.io/fuzzing/2018/11/19/vectorized_emulation_mmu.html

Coverage, Feedback, and "Dynamic Solving"

I've yet to talk publicly about the coverage mechanisms of vectorized emulation, and that will be the focus of this talk. We've used it successfully to find over 10 bugs in DHCP on Windows, one DHCP bug in OpenBSD, a default config remote bluescreen in Windows Firewall, and various others. Due to the feedback going beyond just code coverage, we were able to find bugs without needing overly complex fuzzers. The feedback allows the fuzzer to learn and grow to even vendor-specific extensions of protocols!

I'll talk about how we found these bugs, and the impact of more-than-code-coverage has on fuzzing. All while covering how to gather this coverage in a meaningful way without a huge performance loss (about 30% slowdown for all coverage enabled).

See also: Slides