CL-MeltySynth: CL-MeltySynth API Overview

CL-MeltySynth's API can be conceptually divided up into three pieces: SoundFont handling, MIDI, and audio synthesis. The SoundFont-side of things handles SoundFont loading, presets, instruments, and sample data. The MIDI part of the API allows you to load MIDI files and send MIDI events to the synthesizer. The audio synthesis part of the API is for rendering audio to buffers for playback.

The CL-MeltySynth library can be accessed via the CL-MELTYSYNTH package (or its nickname, CL-MSYNTH).

Full reference is available online, and via docstrings in the code.

SoundFont API

SoundFont loading is accomplished using the LOAD-SOUNDFONT function. This accepts either a string or a pathname, and returns a new SOUNDFONT instance. A SOUNDFONT has accessor functions for its instruments (SOUNDFONT-INSTRUMENTS), presets (SOUNDFONT-PRESETS), and sample data (SOUNDFONT-WAVE-DATA and SOUNDFONT-SAMPLE-HEADERS).

The sample data for a SoundFont is stored as a single block in memory. You can use the SAMPLE-HEADERs of the SOUNDFONT to locate individual samples within this block. These headers can also be accessed from each individual INSTRUMENT instance.

Example of loading a SoundFont and iterating through its instruments:

(let ((sf (cl-msynth:load-soundfont #P"/path/to/soundfont.sf2")))
  (loop for inst across (cl-msynth:soundfont-instruments sf) do
    (format t "Instrument name: ~a~%" (cl-msynth:inst-name inst))))

Example of loading a SoundFont and iterating through its presets:

(let ((sf (cl-msynth:load-soundfont #P"/path/to/soundfont.sf2")))
  (loop for preset across (cl-msynth:soundfont-presets sf) do
    (format t "Bank ~3,'0d Patch: ~3,'0d - ~a~%"
            (cl-msynth:preset-bank-number preset)
            (cl-msynth:preset-patch-number preset)
            (cl-msynth:preset-name preset))))

MIDI API

MIDI files can be loaded using the MAKE-MIDI-FILE generic function. This takes a string, pathname, or open stream (which must have an element type of (UNSIGNED-BYTE 8)), and returns a new MIDI-FILE instance. MIDI files are represented in RAM as a set of messages and a set of corresponding timings for those messages.

The length of the MIDI file can be retrieved with the MIDI-FILE-LENGTH function, which returns the final message timestamp in the MIDI. Dividing this by the standard [INTERNAL-TIME-UNITS-PER-SECOND](http://www.lispworks.com/documentation/HyperSpec/Body/v_intern.htm#internal-time-units-per-second) value will give you the length of the MIDI in seconds.

MIDI files are played using the MIDI-FILE-SEQUENCER class. This not only handles MIDI playback, but is also the usual entry point for rendering audio, and thus is closely tied with the audio synthesis API as well. You will want to read the following section on the audio synthesis API.

A MIDI-FILE-SEQUENCER instance can be created with the usual [MAKE-INSTANCE](http://www.lispworks.com/documentation/HyperSpec/Body/f_mk_ins.htm#make-instance) call, though you should take care and always provide a synthesizer along with it:

(let ((synth (cl-msynth:make-synthesizer my-soundfont my-synth-settings)))
  (make-instance 'cl-msynth:midi-file-sequencer :synthesizer synth)
  ;; ...more code here...
  )

See the Audio Synthesis API section below for info on the SYNTHESIZER class.

Following this, you can start playback with the PLAY function, which takes the MIDI-FILE-SEQUENCER and a MIDI-FILE, and the STOP function, which just takes the MIDI-FILE-SEQUENCER.

Audio Synthesis API

Audio rendering is done by associating a MIDI-FILE-SEQUENCER (described above) with a SYNTHESIZER instance, then calling the RENDER method repeatedly to fill a buffer.

The SYNTHESIZER class is an opaque class that is configured entirely through the SYNTHESIZER-SETTINGS class. To create a SYNTHESIZER-SETTINGS instance, use the MAKE-SYNTHESIZER-SETTINGS function, which takes multiple keyword arguments for the settings. These can be changed later by SETF-ing the various SYNTH-SETTING-* methods.

Once you have a SYNTHESIZER-SETTINGS instance, you can create a SYNTHESIZER instance using the MAKE-SYNTHESIZER function (you should never create a SYNTHESIZER directly using MAKE-INSTANCE). This function takes a string pathname for a SoundFont (or a SOUNDFONT instance) and the SYNTHESIZER-SETTINGS instance.

Once you have a SYNTHESIZER, you create a MIDI-FILE-SEQUENCER (see above) using the synthesizer, start playback using PLAY, then repeatedly call RENDER, passing the MIDI-FILE-SEQUENCER instance to it each time. This method will automatically advance the sequencer, "run" the synthesizer, and render the resulting audio to a buffer.

Example showing how to change the patch, then synthesizing a chord using CL-PortAudio:

(let* ((sample-rate 44100)
       (synth (cl-msynth:make-synthesizer #P"/path/to/soundfont.sf2" sample-rate)))
  ;; Change the patch to number 30, distorted guitar
  (cl-msynth:synth-process-midi-message synth 0 #xC0 30 0)

  ;; Play a C-minor chord one octave below middle C
  (cl-msynth:synth-note-on synth 0 48 100)
  (cl-msynth:synth-note-on synth 0 51 100)
  (cl-msynth:synth-note-on synth 0 55 100)

  ;; Create an output buffer.  * 3 for three seconds, and * 2 for two channels.
  (let ((buf (make-array (* 3 sample-rate 2) :element-type 'single-float
                                             :initial-element 0.0)))
    ;; Render the audio to the buffer
    (cl-msynth:render synth :stereo-interleaved :float32 buf)

    ;; Play buffer using CL-PortAudio
    (pa:with-audio
      (pa:with-default-audio-stream (strm 0 2 :frames-per-buffer (truncate (length buf) 2)
                                              :sample-rate (coerce sample-rate 'double-float))
        (pa:write-stream strm buf)
        (sleep 3)))))

Rendering MIDI files to a certain format is controlled using a MIDI-FILE-SEQUENCER and by passing the information to RENDER:

(let ((destination-buffer (make-array buffer-size :element-type 'single-float :initial-element 0.0)))
  ;; Render stereo 32-bit floating point samples to a buffer
  (cl-msynth:render my-sequencer :stereo-interleaved :float32 destination-buffer)
  ;; ...more code...
  )

;; Render stereo 16-bit signed integer samples to a buffer
(let ((destination-buffer (make-array buffer-size :element-type '(signed-byte 16) :initial-element 0)))
  (cl-msynth:render my-sequencer :stereo-interleaved :sint16-le destination-buffer)
  ;; ...more code...
  )

;; Render monaural 16-bit signed integer samples to a buffer
(let ((destination-buffer (make-array buffer-size :element-type '(signed-byte 16) :initial-element 0)))
  (cl-msynth:render my-sequencer :mono :sint16-le destination-buffer)
  ;; ...more code...
  )

You can also call RENDER on a SYNTHESIZER instance directly, sending MIDI messages to the synthesizer using SYNTH-PROCESS-MIDI-MESSAGE, SYNTH-NOTE-ON, SYNTH-NOTE-OFF, etc. When calling it with a SYNTHESIZER instance, the synth processes any pending messages and "runs" its voices.

Controlling Synthesis

CL-MeltySynth comes with built-in chorus and reverb effects. These are implemented as channel inserts, where each channel "sends" a certain amount of its signal to be added to the effect, which is then mixed in at the final stage of rendering. This is the same as an insert effect on a mixer, where the effect gets its own bus channel that is then mixed into the master output channel.

The reverb and chorus can be turned on and off individually. This is done using the :EFFECTS keyword parameter when calling MAKE-SYNTHESIZER-SETTINGS. This parameter takes a list of keywords:

;; Enable both chorus and reverb
(cl-msynth:make-synthesizer-settings :effects '(:chorus :reverb))

;; Enable only the reverb
(cl-msynth:make-synthesizer-settings :effects '(:reverb))

;; Enable only the chorus
(cl-msynth:make-synthesizer-settings :effects '(:chorus))

;; Disable both the chorus and reverb
(cl-msynth:make-synthesizer-settings :effects nil)

The default send levels can be set using the :REVERB-SEND and :CHORUS-SEND keyword parameters when calling MAKE-SYNTHESIZER-SETTINGS, both of which take a value between 0 and 255. These values may still be overridden by the MIDI file itself if controllers 91 (reverb send amount) or 93 (chorus send amount) are used, but you can at least set the default levels using these parameters. This is especially useful for MIDIs that do not use these controllers at all.

There are four reverbs available in CL-MeltySynth: three Schroeder-style plate reverbs, and one hall(-ish) reverb:

Freeverb: This is the original Freeverb, tweaked slightly to correct some of its calculations, as mentioned on the Freeverb3 website. This is a plate reverb, and thus sounds metallic. This is the "lightest" reverb available and can be created with the MAKE-FREEVERB function.
Schroeder: This is a reverb based on Freeverb, but using the original "incorrect" calculations, and a different value for the all-pass filter feedback. This is slightly fuller sounding than the Freeverb option, but still metallic. This can be created with the MAKE-SCHROEDER-REVERB function.
Remiverb: My own tweaked version of Freeverb. This is still a plate reverb (and thus somewhat metallic sounding), but is much more full and aggressive sounding. This is created with the MAKE-REMIVERB function.
Zita-Rev1: This is a port of Zita-Rev1, a reverb that is more like a hall reverb. It is very rich and warm, and has the most options. It includes a configurable pre-delay and parametric EQ. Create this with the MAKE-ZITA-REVERB function.

The reverb effect has many additional configuration options. To access these, first get a handle on the reverb unit itself by calling SYNTH-REVERB-UNIT (this is also SETF-able, which is how you change the reverb type). The reverb can then be configured with the following methods:

REVERB-ROOM-SIZE: Gets or sets the size of the virtual room the reverb simulates. This is ignored by the ZITA-REVERB class.
REVERB-DAMP: Gets or sets the amount of dampening applied to the reverb. This controls how quickly the high frequencies diminish, thereby emulating how 'hard' the walls of the virtual room are that the reverb is in. The more damp the reverb is, the less sparkle the reverberation sound has. Implemented by all reverbs.
REVERB-WET: Gets or sets the gain of processed signal that is returned by the reverb unit. This is ignored by the ZITA-REVERB class.
REVERB-WIDTH: Gets or sets the stereo spread amount of the reverb. This is ignored by the ZITA-REVERB class.
REVERB-PRE-DELAY: Gets or sets the pre-delay time of the reverb. This is the amount of time, in seconds, between the original dry sound and the onset of the reverb echo. Increasing this value gives the impression of a larger room. This is only implemented by then ZITA-REVERB class and is ignored by all other reverb classes.
REVERB-TIME and REVERB-SET-TIME: These control the reverberation time (RT60). This is implemented by all reverb classes.

The ZITA-REVERB class has two processing bands, low and high, which can be adjusted individually. To do this, pass the keyword parameter :BAND along with either :LOW or :HIGH. This keyword parameter is ignored by other reverb classes.

REVERB-XOVER: Gets or sets the frequency that separates the low and high bands of the reverb. This is only implemented by the ZITA-REVERB class and is ignored by all other reverb classes.
REVERB-EQ and REVERB-SET-EQ: These control the parametric EQ of the ZITA-REVERB class. The EQ is applied only to the wet reverberation signal, not the dry signal. These are ignored by all other reverb classes.