FEX-Emu Wiki - User contributions [en]

Development:Setting up FEX

2024-09-24T15:20:16Z

D: Update version of libstdc++ from 10 to 12, as 10 is no longer available in the Debian repositories. Additionally, limit ninja to 4 threads, as any more may lead to a "Too many open files" error.

[[Category:Development]]
== Build dependencies ==
=== Debian/Ubuntu dependencies ===
* git
* cmake
* ninja-build
* pkgconf
* ccache (Recommended if you're rebuilding constantly)
* clang
* llvm
* lld (For ThinLTO)
* binfmt-support (For AArch64 binfmt_misc support)
* libssl-dev
* python-setuptools (For build scripts)
* g++-x86-64-linux-gnu (For building thunks)
* libgcc-XX-dev-i386-cross (For building thunks, version should match your g++-x86-64-linux-gnu package)
* libgcc-XX-dev-amd64-cross (For building thunks, version should match your g++-x86-64-linux-gnu package)
* nasm (only if building tests)
* python3-clang (for struct verifier)
* libstdc++-12-dev-i386-cross (for struct verifier)
* libstdc++-12-dev-amd64-cross (for struct verifier)
* libstdc++-12-dev-arm64-cross (for struct verifier)
* squashfs-tools (For squashfs rootfs support)
* squashfuse (For mounting the squashfs)
* libc-bin or glibc-tools (For unittests, needs catchsegv from this)
* libc6-dev-i386-amd64-cross (For FEXLinuxTests)
* lib32stdc++-XX-dev-amd64-cross (For FEXLinuxTests, version should match your g++-x86-64-linux-gnu package)
* qtdeclarative5-dev (For GUI)
* qml-module-qtquick-controls (For GUI)
* qml-module-qtquick-controls2 (For GUI)
* qml-module-qtquick-dialogs (For GUI)

=== Gentoo dependencies ===
* dev-vcs/git
* cmake
* ninja
* ccache
* clang
* lld
* nasm

'''Use gubbins'''
* >=media-libs/libglvnd-1.3.4 X

=== Fedora dependencies ===
* git
* cmake
* ninja-build
* pkg-config
* ccache
* clang
* lld
* llvm
* llvm-devel
* openssl-devel
* nasm
* python3-clang
* python3-setuptools
* squashfs-tools
* squashfuse
* erofs-fuse
* erofs-utils

== Build Configuration ==
* Ensure release mode is enabled
* Disable test building
* Enable LTO in a release build
* Enable LLD to make LTO builds faster

git clone --recurse-submodules https://github.com/FEX-Emu/FEX.git
cd FEX
mkdir Build
cd Build
CC=clang CXX=clang++ cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Release -DUSE_LINKER=lld -DENABLE_LTO=True -DBUILD_TESTS=False -DENABLE_ASSERTIONS=False -G Ninja ..
ninja

== Installation ==
sudo ninja install -j4

You can install a binfmt_misc handler for both 32bit and 64bit x86 execution directly from the environment.
If you already have box86's 32bit binfmt_misc handler installed then make sure to uninstall their's first.
Make sure to have run install prior to this, otherwise binfmt_misc will install an old handler even if the executable has been updated.

sudo ninja binfmt_misc_32
sudo ninja binfmt_misc_64

=== Quirks ===

==== binfmt_misc problems ====
* Double check that binfmt_misc has worked
** `ls /usr/share/binfmts/FEX*` should return files
** `ls /var/lib/binfmts/FEX*` should return files
** `/proc/sys/fs/binfmt_misc/status` should exist
*** If it doesn't exist then make sure your kernel has `CONFIG_BINFMT_MISC=y`
*** If it still doesn't exist then make sure systemd isn't disabling it `systemctl enable binfmt-support`

==== DESTDIR quirks ====
* When using DESTDIR, make sure the path is absolute instead of relative
** FEX's thunks use subprojects because it needs to cross-compile
** If DESTDIR is relative then the install paths of the subprojects go to the wrong location
** Using $(pwd)/install works well

== Runtime Configuration ==
FEXConfig is the application to use for runtime configuration setup.

CTLR+S saves the configuration.

=== Things to ensure to set ===
On AArch64 host, a rootfs path is mandatory, while it is optional on x86-64 host

* Core: JIT
* RootFS
* Silent Logging (Useful to get FEX information spam out of the way)
* Block Size (500 is a good default. Optimization passes may break things though)

More Details at: Follow the instructions [[Development:Configuring_FEX|here]]

== RootFS generation ==
AArch64 hosts require a rootfs for running applications. FEX provides various x86 images that can be installed through [[Development:Setting_up_RootFS#Quick_Setup_with_FEXRootFSFetcher|FEXRootFSFetcher]]

For more detailed instructions see [[Development:Setting_up_RootFS]]

== Installing Thunks ==
Only if you want

Follow the instructions [[Development:Setting_up_Thunks|here]]

Additional details here: https://github.com/FEX-Emu/FEX/blob/main/ThunkLibs/README.md

== Binaries ==
* FEXLoader
** This is useful for development purposes but is generally recommended against using.
** Allows passing arguments to FEX for configuration purposes, which can break under some execve situations
* FEXInterpreter
** This is a hardlink to FEXLoader which refuses to treat arguments as configuration options
** Is recommended to use over FEXLoader; Using FEXConfig to set configuration
** Requires being a hardlink or executable copy, execve logic sees through a softlink
** Is the executable used for binfmt_misc for interpreters
* FEXConfig
** Used for setting configuration options from a GUI
** Written using Dear ImGui
** Can load/save global configuration and application configuration files
** Application configuration files are compared against the executable filename
* FEXBash
** Can be useful for starting a bash instance that is running under emulation
** It's not a chroot! Don't use sudo inside of this environment!
* FEXMountDaemon
** This is a mount daemon that will run in the background when you are using a squashfs based rootfs
** Usually one one instance open at any given moment
** It usually closes when the FEXInterpreter exits, but not always. See the problems section in setting up a rootfs
*** Has a ten second timeout window in case a another instance of FEX opens and uses it.

=== Debug binaries ===
* TestHarnessRunner
** Used for running FEX's ASM unit tests
** Runs some binary in a host x86_64 host environment or FEX and checks results
** Useful for quickly creating ASM based unit tests
* IRLoader
** Used for running FEX's IR based unit tests
** Only runs through FEX and checks the results
** Minimal number of tests
* UnitTestGenerator
** This is supposed to be an x86-64 instruction generator
** Has an understanding of x86-64 instruction encoding format
** Generates hundreds of thousands of instructions
** Used to be used with a lockstep runner to ensure correct behaviour
** Largely unsupported, avoiding undefined behaviour is difficult and FEX doesn't need to match undefined behaviour
** Lockstep hostrunner wasn't ever made to be bug free, hard to support
** Doesn't ensure correct behaviour on AArch64 device

Development:Setting up FEX

2024-09-24T03:26:50Z

D: Update build dependencies to add Qt5.

[[Category:Development]]
== Build dependencies ==
=== Debian/Ubuntu dependencies ===
* git
* cmake
* ninja-build
* pkgconf
* ccache (Recommended if you're rebuilding constantly)
* clang
* llvm
* lld (For ThinLTO)
* binfmt-support (For AArch64 binfmt_misc support)
* qtdeclarative5-dev (For GUI)
* qml-module-qtquick-controls (For GUI)
* qml-module-qtquick-controls2 (For GUI)
* qml-module-qtquick-dialogs (For GUI)
* libssl-dev
* python-setuptools (For build scripts)
* g++-x86-64-linux-gnu (For building thunks)
* libgcc-XX-dev-i386-cross (For building thunks, version should match your g++-x86-64-linux-gnu package)
* libgcc-XX-dev-amd64-cross (For building thunks, version should match your g++-x86-64-linux-gnu package)
* nasm (only if building tests)
* python3-clang (for struct verifier)
* libstdc++-10-dev-i386-cross (for struct verifier)
* libstdc++-10-dev-amd64-cross (for struct verifier)
* libstdc++-10-dev-arm64-cross (for struct verifier)
* squashfs-tools (For squashfs rootfs support)
* squashfuse (For mounting the squashfs)
* libc-bin or glibc-tools (For unittests, needs catchsegv from this)
* libc6-dev-i386-amd64-cross (For FEXLinuxTests)
* lib32stdc++-XX-dev-amd64-cross (For FEXLinuxTests, version should match your g++-x86-64-linux-gnu package)
* qtdeclarative5-dev
* qml-module-qtquick-controls
* qml-module-qtquick-controls2
* qml-module-qtquick-dialogs

=== Gentoo dependencies ===
* dev-vcs/git
* cmake
* ninja
* ccache
* clang
* lld
* nasm

'''Use gubbins'''
* >=media-libs/libglvnd-1.3.4 X

=== Fedora dependencies ===
* git
* cmake
* ninja-build
* pkg-config
* ccache
* clang
* lld
* llvm
* llvm-devel
* openssl-devel
* nasm
* python3-clang
* python3-setuptools
* squashfs-tools
* squashfuse
* erofs-fuse
* erofs-utils

== Build Configuration ==
* Ensure release mode is enabled
* Disable test building
* Enable LTO in a release build
* Enable LLD to make LTO builds faster

git clone --recurse-submodules https://github.com/FEX-Emu/FEX.git
cd FEX
mkdir Build
cd Build
CC=clang CXX=clang++ cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_BUILD_TYPE=Release -DUSE_LINKER=lld -DENABLE_LTO=True -DBUILD_TESTS=False -DENABLE_ASSERTIONS=False -G Ninja ..
ninja

== Installation ==
sudo ninja install

You can install a binfmt_misc handler for both 32bit and 64bit x86 execution directly from the environment.
If you already have box86's 32bit binfmt_misc handler installed then make sure to uninstall their's first.
Make sure to have run install prior to this, otherwise binfmt_misc will install an old handler even if the executable has been updated.

sudo ninja binfmt_misc_32
sudo ninja binfmt_misc_64

=== Quirks ===

==== binfmt_misc problems ====
* Double check that binfmt_misc has worked
** `ls /usr/share/binfmts/FEX*` should return files
** `ls /var/lib/binfmts/FEX*` should return files
** `/proc/sys/fs/binfmt_misc/status` should exist
*** If it doesn't exist then make sure your kernel has `CONFIG_BINFMT_MISC=y`
*** If it still doesn't exist then make sure systemd isn't disabling it `systemctl enable binfmt-support`

==== DESTDIR quirks ====
* When using DESTDIR, make sure the path is absolute instead of relative
** FEX's thunks use subprojects because it needs to cross-compile
** If DESTDIR is relative then the install paths of the subprojects go to the wrong location
** Using $(pwd)/install works well

== Runtime Configuration ==
FEXConfig is the application to use for runtime configuration setup.

CTLR+S saves the configuration.

=== Things to ensure to set ===
On AArch64 host, a rootfs path is mandatory, while it is optional on x86-64 host

* Core: JIT
* RootFS
* Silent Logging (Useful to get FEX information spam out of the way)
* Block Size (500 is a good default. Optimization passes may break things though)

More Details at: Follow the instructions [[Development:Configuring_FEX|here]]

== RootFS generation ==
AArch64 hosts require a rootfs for running applications. FEX provides various x86 images that can be installed through [[Development:Setting_up_RootFS#Quick_Setup_with_FEXRootFSFetcher|FEXRootFSFetcher]]

For more detailed instructions see [[Development:Setting_up_RootFS]]

== Installing Thunks ==
Only if you want

Follow the instructions [[Development:Setting_up_Thunks|here]]

Additional details here: https://github.com/FEX-Emu/FEX/blob/main/ThunkLibs/README.md

== Binaries ==
* FEXLoader
** This is useful for development purposes but is generally recommended against using.
** Allows passing arguments to FEX for configuration purposes, which can break under some execve situations
* FEXInterpreter
** This is a hardlink to FEXLoader which refuses to treat arguments as configuration options
** Is recommended to use over FEXLoader; Using FEXConfig to set configuration
** Requires being a hardlink or executable copy, execve logic sees through a softlink
** Is the executable used for binfmt_misc for interpreters
* FEXConfig
** Used for setting configuration options from a GUI
** Written using Dear ImGui
** Can load/save global configuration and application configuration files
** Application configuration files are compared against the executable filename
* FEXBash
** Can be useful for starting a bash instance that is running under emulation
** It's not a chroot! Don't use sudo inside of this environment!
* FEXMountDaemon
** This is a mount daemon that will run in the background when you are using a squashfs based rootfs
** Usually one one instance open at any given moment
** It usually closes when the FEXInterpreter exits, but not always. See the problems section in setting up a rootfs
*** Has a ten second timeout window in case a another instance of FEX opens and uses it.

=== Debug binaries ===
* TestHarnessRunner
** Used for running FEX's ASM unit tests
** Runs some binary in a host x86_64 host environment or FEX and checks results
** Useful for quickly creating ASM based unit tests
* IRLoader
** Used for running FEX's IR based unit tests
** Only runs through FEX and checks the results
** Minimal number of tests
* UnitTestGenerator
** This is supposed to be an x86-64 instruction generator
** Has an understanding of x86-64 instruction encoding format
** Generates hundreds of thousands of instructions
** Used to be used with a lockstep runner to ensure correct behaviour
** Largely unsupported, avoiding undefined behaviour is difficult and FEX doesn't need to match undefined behaviour
** Lockstep hostrunner wasn't ever made to be bug free, hard to support
** Doesn't ensure correct behaviour on AArch64 device