Development:Debugging FEX with Signals

Signals and you

FEX uses signals in quite a few ways that may cause debugging to be a bit of a pain. This is necessary for correct operation so it is an evil we must live with.

Signals that FEX uses

• SIGBUS - The AArch64 JIT generates unaligned loadstore instructions then captures and backpatches these when they fault
  • This is necessary for emulating x86-TSO on AArch64 and can't be dropped without massive performance loses
• SIGILL - All of the CPU backends generate this fault and is necessary for the emulator to return from a guest signal
  • Going in to a signal happens frequently, gdb will usually ignore the entry when it is SIGALRM or SIGUSR*
  • We could choose a different signal, but this one can be generated in one instruction which makes it nice to use
  • Using a signal to punch back through to the guest return path is necessary since we need to save and restore guest registers when guest signals occur.
• SIG63 - We use this for various ancillary functions
  • This signal currently tells a thread that it should stop running, to pause its execution, or to resume from its paused frame
  • This is another necessary signal since if we want to safely shutdown the emulator and a thread is in a syscall, there is no way for it to leave without a signal.
  • The stop signal specifically will have the CPU backend long jump from the signal handler to the exit stage of the ASM dispatcher for safe shutdown
  • This slightly increases performance because then there aren't atomic checks per iteration of the execution loop for if we should stop executing

Debugging FEX

Read more here Development:Debugging_Crash

Profiling FEX code

You'll want to enable a style of JIT code mark-up for perf to annotate with

• FEX currently has three options for this
• Depending on what you're trying to profile you will want to enable one of these options
  • FEX_GLOBALJITNAMING
    • Useful if you want to throw all of FEX JIT space in to a global-ish namespace
  • FEX_LIBRARYJITNAMING
    • Useful if you want to clump FEX JIT space in to library specific code segments
    • Can be nice for finding libraries to thunk
  • FEX_BLOCKJITNAMING
    • This is likely the one you want if you're profiling your application's code
    • This will give you hot blocks in `perf top`

Enabling one of these will give you some symbols when capturing FEX with `perf top`. The block naming is useful for seeing hot blocks in program space so if something ends up being particularly slow in FEX, it can get more isolated.

• We don't currently have an option to relate JIT code directly to function symbols in the ELF at this point.

Debugging the guest application

When FEX is started with the -G argument it will open a gdbstub socket listening on port 8086. This can be connected to with gdbserver, ida, ghidra, or binary ninja.

This is currently only useful for inspection, gdbstub doesn't yet send guest signals over through gdbstub!