Annotate Artifact Blame Check-ins Using Doc Download Text

File lisp-build-instructions.md artifact 4068c899c8 on branch trunk

Building the Lisp Version

Dependencies

There are four conceptual sets of dependencies:

  1. A Common Lisp implementation.
  2. C/C++ libraries.
  3. Common Lisp libraries that you can get via Quicklisp
  4. Common Lisp libraries you have to get manually. These are mostly the ones that Remilia (the author of Benben) wrote.

You'll need SBCL, preferably a recent version. SBCL 2.3.11 or later is required, and SBCL 2.5.x or later is recommended. Older SBCL versions may work, but are not officially supported. Other Lisp implementations will NOT work, are NOT supported, and will NOT be supported in the future.

C/C++ Dependencies

Required C/C++ libraries:

  • libzstd
  • S-Lang (2.3.2 or higher)
  • libmpg123 (1.29 or higher)
  • libxmp (4.6.0 or higher recommended)
  • libsidplayfp (2.1.0 or higher recommended)
  • libwavpack (5.0 or higher recommended)
  • libopus (1.3.0 or higher recommended)
  • libao (1.2.0 or higher recommended)
  • libremicsid

Optional C libraries:

  • libasound (only when the ALSA driver is enabled)
  • libout123 (only when the out123 driver is enabled)

Some of the Quicklisp-installable Common Lisp dependencies may have additional C/C++ library requirements and are not listed here.

Important: static libraries are not supported.

Dependencies You Can (Probably) Get via Quicklisp or Similar

These dependencies should be available via Quicklisp, or if they aren't they should be easy to find.

If you don't want to use QuickLisp (Remilia doesn't), then the Ruby script that's used to build the official AppImages may be useful, as it has an updated list of URLs for these dependencies.

A large chunk of these dependencies are for the FXML library, or are subdependencies.

  • uiop (part of ASDF)
  • alexandria
  • babel
  • cffi
  • closer-mop
  • cl-ppcre
  • cl-cpus
  • trivial-indent
  • computable-reals
  • st-json
  • lparallel
  • atomics
  • documentation-utils
  • trivial-garbage
  • zstd
  • quri
  • fxml
  • chipz
  • trivial-features
  • trivial-gray-streams
  • parse-number
  • bordeaux-threads
  • global-vars
  • cl-octet-streams
  • split-sequence
  • cl-utilities
  • idna
  • flexi-streams
  • named-readtables
  • mgl-pax-bootstrap (part of mgl-pax)
  • cl-who
  • serapeum
  • trivia
  • type-i
  • iterate
  • introspect-environment
  • lisp-namespace
  • trivial-cltl2
  • string-case
  • parse-declarations
  • trivial-file-size
  • trivial-macroexpand-all

Dependencies You Cannot Yet Get via Quicklisp

Follow these links, then clone the repositories somewhere that ASDF can find them. See the section of the ASDF manual on where ASDF looks for dependencies for more information.

Optional Dependencies Needed for Building

These programs are only needed if you want to build the manual.

  • rst2man
  • gzip
  • texi2any

Pre-building Tasks

The CL-RemiChips dependency requires you to run make within its toplevel directory before it can be used. This is to build a small C++ file that contains C bindings for libsidplayfp (which is a C++ library, and therefore Benben can't bind to it directly).

You may want to copy the resulting .so file somewhere that the resulting binary can find it. It will be located in cl-remichips/src/sid/libremisid.so.

A Note for Package Maintainers

It's highly recommended you clear out your fasl cache prior to building Benben. This will ensure Benben gets built with the correct compiler policies applied. A common way to do this is to run rm -rf ~/.cache/common-lisp/ prior to running the build script.

Getting Benben's Sources

$ fossil clone https://chiselapp.com/user/MistressRemilia/repository/benben/
$ cd benben
$ fossil checkout lisp-rewrite

Be sure you clone the repository so that ASDF can locate it.

Building With the Build Script

Using the provided build script, build.sh, is the expected way to build Benben. For a default build, you can just run ./build.sh from the root repository directory. The binary will be in the bin/ directory.

Important: Do NOT strip the resulting binary. It will not function if you do.

You can run ./build.sh --help to see a list of available options. In most cases you won't need to use any of them, though you may want to do ./build.sh --manual to build Benben's man page, Info page, and manual files.

If your copy of SBCL does not include the SB-SIMD package, you're on a non-x86-64 platform (e.g. ARM AArch64), or you're on a CPU older than Intel's Broadwell microarchitecture, then you will have to pass the --no-simd option to the build script. This will result in a slightly slower binary.

Building Without the Build Script

The first step is to set up some environment variables:

  • BENBEN_OPT_MODE: This controls the build type. It can be one of: normal, unsafe, or debug.
  • BENBEN_NO_SIMD: If this is set to 1, then various blocks of SIMD code are disabled.
  • BENBEN_NO_OUT123: If this is set to 1, then the out123 driver is disabled.
  • BENBEN_NO_ALSA: If this is set to 1, then the ALSA driver is disabled. Note that the ALSA driver is always disabled on non-Linux systems anyway.
  • BENBEN_COMPRESS: If this is set to 1, then a compressed binary is built. Setting this to 0 produces an uncompressed binary.
  • BENBEN_DONT_RELOAD_LIBRARIES: Normally the build script will rebuild a large chunk of Common Lisp libraries to ensure that they are built with the same compiler policies as Benben. Setting this to 1 will tell the build script to NOT rebuild these libraries. This saves a large chunk of time, especially if you're rebuilding Benben quite a bit and you know you haven't changed things in Slime or elsewhere. It's recommended you leave this unset or set to 0, however.

Note: this environment variable can also be used with the Rakefile.

Next, you'll want to determine the maximum heap size you want. The Rakefile defaults to 8192 (that is, 8 gigabytes), and is the recommended size. Benben won't use this much, but it basically puts a cap on how much the Lisp runtime is allowed to allocate. Do not set this too low or Benben won't even start.

Finally, start SBCL like this from the root of the repository so that it loads the build file:

$ sbcl --dynamic-space-size <MAX HEAP SIZE HERE> --disable-ldb --merge-core-pages --disable-debugger --load build-scripts/build-benben.lisp

The finished binary will be in the bin/ directory. Important: Do NOT strip the resulting binary. It will not function if you do.

Additional Debug Info

Benben contains various internal debugging instruments that may be useful if you're hacking on the code. Here are some tips for debugging:

  • Benben's Minimal UI is great for debugging most of the code. All messages get logged to your REPL's output. In fact, that's why and how the Minimal UI started life.
  • To debug Benben's Original UI code, you'll want to start your Lisp process with the start-with-swank.sh file. This will load Benben, then start up a new Swank listener that you can connect to via Slime or similar. The script takes a few options as well, so check ./start-with-swank.sh --help.
  • Benben (and SatouSynth) redefines CL-SDM's DEFINE-TYPED-FN function and MUFFLING macro to better aid in debugging. Look at src/lib/redefines.lisp to see how they differ.
  • If you want to always invoke the Lisp debugger every time a player has an error, push :BENBEN-DEBUG-PLAYER-ERRORS onto *FEATURES* and recompile Benben. This will occur within a HANDLER-BIND that exists within the main PLAYER-MANAGER's SUBSYSTEM-START method.
  • Unless you're debugging the Original UI, you'll almost certainly want to load the startup-scripts/benben-debug.lisp into your REPL. This sets up most of the debugging options by pushing these symbols onto *FEATURES*: :REMIAUDIO-EXTRA-CHECKS, :NEVER-APPLY-BLOCK-COMPILATION, :SATOU-NEVER-INLINE, :SATOU-DEBUG, :SATOU-VERBOSE-DEBUG, :BENBEN-NEVER-INLINE, :BENBEN-DEBUG, :BENBEN-VERBOSE-DEBUG.

Building the AppImage

Building an AppImage of Benben is, in general, very easy. It does assume that you are on an x86-64 CPU that supports the FMA3 SIMD instructions, however. This means a CPU that's at least as new as Intel's Broadwell architecture, released in 2014.

If you want to build the AppImage from scratch, then you should use the build-benben-lisp-x86-64-appimage.rb script from the dev-scripts/ directory. As indicated by the name of the script, it only builds an x86-64 binary. Using this script won't require you to download any of the Common Lisp dependencies (Quicklisp or otherwise) listed above since this will automatically be handled by the script. You will still need to install the C/C++ dependencies yourself, however.

This script is specifically built to be run on a Slackware Linux 14.2 system. This was chosen for two reasons:

  1. Remilia is extremely familiar with Slackware, having used it almost exclusively since 2002.
  2. It uses an old glibc version, which is important for producing AppImages that run on a wide variety of systems.

Newer versions of Slackware are known to work as well. Other Linux distributions may work are but are not supported by this script. So you may need to modify it to suit your needs.

The script clones repos and builds things in the system's /tmp directory. You'll need a couple hundred megabytes of free space for this.

To use the script, simply run it:

$ ./build-benben-lisp-x86-64-appimage.rb

The resulting AppImage will be at /tmp/benben-<version>-x86-64.Appimage.