Development:Setting up FEX
Build dependencies
- git
- cmake
- ninja-build
- ccache (Recommended if you're rebuilding constantly)
- clang
- lld (For ThinLTO)
- binfmt-support (For AArch64 binfmt_misc support)
- libsdl2-dev (For GUI)
- libepoxy-dev (For GUI)
- g++-x86-64-linux-gnu (For building thunks)
- nasm (only if building tests)
- python3-clang (for struct verifier)
- libstdc++-10-dev-i386-cross (for struct verifier)
- libstdc++-10-dev-amd64-cross (for struct verifier)
Build Configuration
- Ensure release mode is enabled
- Enable static pie
- This is necessary for chroot and pressure-vessel
- If using LLD then you need at least LLD 13.0
- If using LLD then you need at least glibc 2.35
- Recommendation: Disable LLD, use binutils ld
- Enable LTO - Currently bugged in some configurations
- Disable if enabling static-pie
- Disable test building
git clone --recurse-submodules https://github.com/FEX-Emu/FEX.git
cd FEX
mkdir Build
cd Build
CC=clang CXX=clang++ cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Release -DENABLE_LTO=False -DENABLE_STATIC_PIE=True -DBUILD_TESTS=False -DENABLE_ASSERTIONS=False -G Ninja ..
ninja
Installation
sudo ninja install
On AArch64 hosts
You can install a binfmt_misc handler for both 32bit and 64bit x86 execution directly from the environment. If you already have box86's 32bit binfmt_misc handler installed then I don't recommend installing FEX's until it is useful. Make sure to have run install prior to this, otherwise binfmt_misc will install an old handler even if the executable has been updated.
sudo ninja binfmt_misc_32
sudo ninja binfmt_misc_64
binfmt_misc problems
- Double check that binfmt_misc has worked
- `ls /usr/share/binfmts/FEX*` should return files
- `ls /var/lib/binfmts/FEX*` should return files
- `/proc/sys/fs/binfmt_misc/status` should exist
- If it doesn't exist then make sure your kernel has `CONFIG_BINFMT_MISC=y`
- If it still doesn't exist then make sure systemd isn't disabling it `systemctl enable binfmt-support`
Runtime Configuration
FEXConfig is the application to use for runtime configuration setup.
CTRL+SHIFT+O will open default "global" configuration.
CTLR+S saves it.
Things to ensure to set
On AArch64 host, a rootfs path is mandatory, while it is optional on x86-64 host
- Core: JIT
- RootFS
- Silent Logging (Useful to get FEX information spam out of the way)
- Block Size (500 is a good default. Optimization passes may break things though)
- Multiblock: enabled (Can cause more stuttering)
- Emulated CPU Cores: 1 (threading bugs cause issues with more)
RootFS generation
AArch64 hosts require a rootfs for running applications Follow the instructions here https://wiki.fex-emu.org/index.php/Development:Setting_up_RootFS
Installing Thunks
Only if you want
Follow the instructions here: https://wiki.fex-emu.org/index.php/Development:Setting_up_Thunks
Additional details here: https://github.com/FEX-Emu/FEX/blob/main/ThunkLibs/README.md
Binaries
- FEXLoader
- This is useful for development purposes but is generally recommended against using.
- Allows passing arguments to FEX for configuration purposes, which can break under some execve situations
- FEXInterpreter
- This is a hardlink to FEXLoader which refuses to treat arguments as configuration options
- Is recommended to use over FEXLoader; Using FEXConfig to set configuration
- Requires being a hardlink or executable copy, execve logic sees through a softlink
- Is the executable used for binfmt_misc for interpreters
- FEXConfig
- Used for setting configuration options from a GUI
- Written using Dear ImGui
- Can load/save global configuration and application configuration files
- Application configuration files are compared against the executable filename
- FEXMountDaemon
- This is a mount daemon that will run in the background when you are using a squashfs based rootfs
- Usually one one instance open at any given moment
- It usually closes when the FEXInterpreter exits, but not always. See the problems section in setting up a rootfs
- Has a ten second timeout window in case a another instance of FEX opens and uses it.
Debug binaries
- TestHarnessRunner
- Used for running FEX's ASM unit tests
- Runs some binary in a host x86_64 host environment or FEX and checks results
- Useful for quickly creating ASM based unit tests
- IRLoader
- Used for running FEX's IR based unit tests
- Only runs through FEX and checks the results
- Minimal number of tests
- UnitTestGenerator
- This is supposed to be an x86-64 instruction generator
- Has an understanding of x86-64 instruction encoding format
- Generates hundreds of thousands of instructions
- Used to be used with a lockstep runner to ensure correct behaviour
- Largely unsupported, avoiding undefined behaviour is difficult and FEX doesn't need to match undefined behaviour
- Lockstep hostrunner wasn't ever made to be bug free, hard to support
- Doesn't ensure correct behaviour on AArch64 device