I had so many questions.. Panicking is nothing out of the ordinary for my Rust project, but today I saw something new:

panicked while panicking. aborting.
illegal hardware instruction (core dumped)

New and interesting is not something you want to see in error messages.

First off, which illegal hardware instruction did we hit? ud2, which is intentionally placed by the compiler in situations when the program encounters a panic during the clean up instigated by a previous panic.

Reduced down to merely panic, inside Drop
thread 'main' panicked at:
segment_bounds: accessing index = 0, segment capacity = 0.

This error message was really scary for me, as allocated segments shouldn’t have zero capacity and so it seemed like I might be dealing with a use-after-free bug. The scary thing? I have a guard against this, which should have reported it. Something was funky, and as you may have guessed, it only happens in release mode.

The debugger was unfortunately not enlightening. Ultimately the problem was in my dynamic dispatch (virtual function) code.

This code takes the base address of a table of virtual functions, and constructs a Rust trait object (a vtable address and the address of an in-memory structure):

Coersion of vtable to trait object
pub fn as_dispatch<'a>(prism: &'a Unit) -> &'a dyn Dispatch {
    let ptr_and_table: [Unit; 2] = [Unit::from(0), *prism];
    unsafe {
        transmute::<[Unit; 2], &dyn Dispatch>(ptr_and_table)
    }
}

Unit is a type def for usize, and prism is a base address of a vtable. Notice the 0 literal. The mechanism I use for heap layout uses arrays instead of structs, so there is no suitable address-of-a-struct to use. Instead, I use an arbitrarily picked meaningless bit pattern. In this case I picked 0. That’s where my problems started.

Described in my previous post, checking for this zero on the receiving side of a virtual call is perilous. The compiler understands it to be a non-zero address, and makes it difficult to compare it to zero (because it assumes it can’t be zero). The solution was to remove all code that received the zero, one line in total.

But, as of March 4, the Rust nightly compiler optimizations turn code that sends the zero into a dead end! It seems that the code path leading up to the dead end is pruned, up until an IO point. The resulting program is difficult to understand with the debugger.

The solution? Use a non-zero abitrary bit pattern. 1 works just fine. So the fix is a one character change.