RemiZstd

RemiZstd is a set of bindings for Nim of the ZStandard C library. They are essentially a port of the Crystal bindings by Didactic Drunk. The original Crystal bindings can be found here.

If you want an alternative set of bindings, check out nim_zstd by wltsmrz.

Current API documentation can be found here.

Features

• Two APIs: a simple context-based one, and one for (de)compressing to/from streams.
• API calls allow reusable buffers to reduce GC overhead.
• export ZSTD_CLEVEL=1 sets the default compression level just like the zstd command line utilities.
• Probably pretty fast
• Custom dictionaries (not well tested)

To import all of the library (minus the low-level FFI stuff), use import remizstd. Or, you can import just what you need:

• remizstd/common: Custom dictionaries and some common types/methods/values
• remizstd/compress: Context-based compression
• remizstd/decompress: Context-based decompression
• remizstd/compstream: Compressing streams
• remizstd/decompstream: Decompression streams
• remizstd/libffi: Low-level C bindings

TODO

• ☐ Support automatic downloading/compiling of libzstd (but not as the default).
• ☐ Implement more of the std/streams API.
• ☐ Implement additional convenience functions.
• ☐ Example (de)compression program.

How do I get set up?

You will need the ZStandard development files on your system. Something like one of the following should work on Linux:

• slackpkg install zstd
• pacman -S zstd
• apt-get install libzstd-dev
• yum install libzstd-devel

After getting ZStandard:

1. Clone the GitLab mirror or this repository manually using Fossil.
2. Change into the cloned directory.
3. Run nimble install to install the library.

Usage

There are two APIs: a stream-based API, and a context-based API. More examples can be found in the documentation.

Streaming API

import std/streams
import remizstd/[compstream]

# Compression to a string stream
var dest = newStringStream()
withCompressStream(dest, cio):
  cio.write("This is some test data")

Context API

import remizstd/[compress, decompress]

let buf = "this is a test buffer"

# Compression using a context
var
  cctx = newCompressCtx(1) # Compression level of 1
  cbuf = cctx.compress(buf)

# Decompression using a context
var
  dctx = newDecompressCtx()
  dbuf = dctx.decompress(cbuf)

assert(dbuf == buf)

Development

Style info

Keep lines 118 characters or shorter. Obviously sometimes you can't, but please try.

Documentation

Please keep the code comments up to date and compatible with NimDoc. Also be sure to update the documentation in www/remizstd.md. The docs generated from www/remizstd.md are what's hosted in the Fossil repository. The reason for having two sets of documentation is due to the security policy of Fossil (which I'd rather not adjust) not allowing some of the techniques that nim doc uses when it generates documentation. However, I also don't want to prevent other users from generating Nim-style documentation by running nim doc --project src/remizstd.nim as usual.

How do I contribute?

I do not use Git - if you're reading this on Gitlab, you're looking at a mirror.

1. Go to https://chiselapp.com/user/MistressRemilia/repository/RemiZstd/ and clone the Fossil repository.
2. Create a new branch for your feature.
3. Push locally to the new branch.
4. Create a bundle with your changes.
5. Send me a message so I can get the bundle from you.

Contributors

Remilia Scarlet - creator and maintainer (remilia (<at>) posteo .jp)