File lisp-build-instructions.md from the latest check-in


# Building the Lisp Version

## Dependencies

There are four conceptual sets of dependencies:

1. A Common Lisp implementation.
2. C/C++ libraries.
3. Common Lisp libraries that you can get via Quicklisp
4. Common Lisp libraries you have to get manually.  These are mostly the ones
   that Remilia (the author of Benben) wrote.

You'll need [SBCL](https://www.sbcl.org/), preferably a recent version.  SBCL
2.3.11 or later is required, and SBCL 2.5.x or later is recommended.  Older SBCL
versions *may* work, but are not officially supported.  Other Lisp
implementations will **NOT** work, are **NOT** supported, and will **NOT** be
supported in the future.

### C/C++ Dependencies

Required C/C++ libraries:

* S-Lang (2.3.2 or higher)
* libmpg123 (1.29 or higher)
* libxmp (4.6.0 or higher recommended)
* libsidplayfp (2.1.0 or higher recommended)
* libwavpack (5.0 or higher recommended)
* libao
* [libremicsid](https://nanako.mooo.com/fossil/libremicsid/)

Some of the Quicklisp-installable Common Lisp dependencies may have additional
C/C++ library requirements and are not listed here.

Important: static libraries are not supported.

### Dependencies You Can (Probably) Get via Quicklisp or Similar

These dependencies should be available via Quicklisp, or if they aren't they
should be easy to find.

If you don't want to use QuickLisp (Remilia doesn't), then the Ruby script
that's used to build the official AppImages may be useful, as it has an updated
list of URLs for these dependencies.

A large chunk of these dependencies are for the FXML library, or are
subdependencies.

* uiop (part of ASDF)
* alexandria
* babel
* cffi
* closer-mop
* monotonic-clock
* cl-ppcre
* cl-cpus
* trivial-indent
* st-json
* lparallel
* trivial-garbage
* zstd
* quri
* fxml
* chipz
* trivial-features
* trivial-gray-streams
* parse-number
* bordeaux-threads
* global-vars
* cl-octet-streams
* split-sequence
* cl-utilities
* idna
* flexi-streams
* named-readtables
* mgl-pax-bootstrap (part of mgl-pax)
* cl-who
* serapeum
* trivia
* type-i
* iterate
* introspect-environment
* lisp-namespace
* trivial-cltl2
* string-case
* parse-declarations
* trivial-file-size
* trivial-macroexpand-all

### Dependencies You Cannot Yet Get via Quicklisp

Follow these links, then clone the repositories somewhere that ASDF can find
them.  See the section of the [ASDF
manual](https://asdf.common-lisp.dev/asdf.html#Configuring-ASDF-to-find-your-systems)
on where ASDF looks for dependencies for more information.

* [cl-sdm](https://chiselapp.com/user/MistressRemilia/repository/cl-sdm/)
* [cl-remimarshal](https://chiselapp.com/user/MistressRemilia/repository/cl-remimarshal/)
* [cl-remiyaml](https://nanako.mooo.com/fossil/cl-remiyaml/)
* [cl-remiaudio](https://chiselapp.com/user/MistressRemilia/repository/cl-remiaudio/)
* [cl-remichips](https://nanako.mooo.com/fossil/cl-remichips/)
* [cl-remi-slang](https://chiselapp.com/user/MistressRemilia/repository/cl-remi-slang/)
* [satou](https://nanako.mooo.com/fossil/satousynth/)

### Optional Dependencies Needed for Building

These programs are only needed if you want to build the manual.

* rst2man
* gzip
* texi2any

## Pre-building Tasks

The CL-RemiChips dependency requires you to run `make` within its toplevel
directory before it can be used.  This is to build a small C++ file that
contains C bindings for libsidplayfp (which is a C++ library, and therefore
Benben can't bind to it directly).

You may want to copy the resulting .so file somewhere that the resulting binary
can find it.  It will be located in `cl-remichips/src/sid/libremisid.so`.

## Getting Benben's Sources

```
$ fossil clone https://chiselapp.com/user/MistressRemilia/repository/benben/
$ cd benben
$ fossil checkout lisp-rewrite
```

Be sure you clone the repository so that ASDF can locate it.

## Building With the Build Script

Using the provided build script, `build.sh`, is the expected way to build
Benben.  For a default build, you can just run `./build.sh` from the root
repository directory.  The binary will be in the `bin/` directory.

Important: Do **NOT** strip the resulting binary.  It will not function if you
do.

You can run `./build.sh --help` to see a list of available options.  In most
cases you won't need to use any of them, though you may want to do
`./build.sh --manual` to build Benben's man page, Info page, and manual files.

If your copy of SBCL does not include the SB-SIMD package, you're on a
non-x86-64 platform (e.g. ARM AArch64), or you're on a CPU older than Intel's
Broadwell microarchitecture, then you will have to pass the `--no-simd` option
to the build script.  This will result in a slightly slower binary.

## Building Without the Build Script

The first step is to set up some environment variables:

* `BENBEN_OPT_MODE`: This controls the build type.  It can be one of: `normal`,
  `unsafe`, or `debug`.
* `BENBEN_NO_SIMD`: If this is set to `1`, then various blocks of SIMD code are
  disabled.
* `BENBEN_COMPRESS`: If this is set to `1`, then a compressed binary is built.
  Setting this to `0` produces an uncompressed binary.
* `BENBEN_DONT_RELOAD_LIBRARIES`: Normally the build script will rebuild a large
  chunk of Common Lisp libraries to ensure that they are built with the same
  compiler policies as Benben.  Setting this to `1` will tell the build script
  to **NOT** rebuild these libraries.  This saves a large chunk of time,
  especially if you're rebuilding Benben quite a bit and you know you haven't
  changed things in Slime or elsewhere.  It's recommended you leave this unset
  or set to `0`, however.
  
  Note: this environment variable can also be used with the Rakefile.

Next, you'll want to determine the maximum heap size you want.  The Rakefile
defaults to `8192` (that is, 8 gigabytes), and is the recommended size.  Benben
won't _use_ this much, but it basically puts a cap on how much the Lisp runtime
is allowed to allocate.  Do not set this too low or Benben won't even start.

Finally, start SBCL like this from the root of the repository so that it loads
the build file:

```
$ sbcl --dynamic-space-size <MAX HEAP SIZE HERE> --disable-ldb --merge-core-pages --disable-debugger --load build-scripts/build-benben.lisp
```

The finished binary will be in the `bin/` directory.  Important: Do **NOT**
strip the resulting binary.  It will not function if you do.

## Building the AppImage

Building an AppImage of Benben is, in general, very easy.  It does assume that
you are on an x86-64 CPU that supports the FMA3 SIMD instructions, however.
This means a CPU that's at least as new as Intel's Broadwell architecture,
released in 2014.

If you want to build the AppImage from scratch, then you should use the
`build-benben-lisp-x86-64-appimage.rb` script from the `dev-scripts/` directory.
As indicated by the name of the script, it only builds an x86-64 binary.  Using
this script won't require you to download any of the Common Lisp dependencies
(Quicklisp or otherwise) listed above since this will automatically be handled
by the script.  You will still need to install the C/C++ dependencies yourself,
however.

This script is specifically built to be run on a Slackware Linux 14.2 system.
This was chosen for two reasons:

1. Remilia is extremely familiar with Slackware, having used it almost
   exclusively since 2002.
2. It uses an old glibc version, which is important for producing AppImages that
   run on a wide variety of systems.

Newer versions of Slackware are known to work as well.  Other Linux
distributions *may* work are but are not supported by this script.  So you may
need to modify it to suit your needs.

The script clones repos and builds things in the system's `/tmp` directory.
You'll need a couple hundred megabytes of free space for this.

To use the script, simply run it:

```
$ ./build-benben-lisp-x86-64-appimage.rb
```

The resulting AppImage will be at `/tmp/benben-<version>-x86-64.Appimage`.