06 January 2016

LLVM and its attendant compiler project Clang are powerhouses in F/OSS. Unfortunately its attendant documentation is not such a powerhouse. Indeed its documentation for getting started with it is laughably terrible. Consider the "getting started" guide for LLVM. Now consider the equivalent document for Clang.

The problem…

The problem is that the docs are:

  • inaccurate;

  • irrelevant;

  • incomplete.

That’s pretty much the Holy Trinity of bad documentation right there. Let’s address these one at a time (using the Clang guide as the basis):

Inaccurate

Clang is released as part of regular LLVM releases. You can download the release versions from http://llvm.org/releases/.

OK. Let’s follow that link. Let’s look at the available options. OK, there’s "Current" which is version "SVN". The bleeding edge alternative downloaded via SVN. Let’s not do that one. Next is "01 Sep 2015" which is version "3.7.0". That’s the one, right? Let’s follow it now.

Sure enough, that’s the one. Let’s start downloading the even dozen tarballs (not including documentation). Hmmm… While I’m waiting for the downloads I happen to see that this is not at the top of the download page. Scrolling up I find that there’s a version 3.7.1. You know, the one I probably should have been downloading in the first place. Woohoo! Time to kill the downloads and waste more bandwidth and time downloading the proper tarballs!

This degree of inaccuracy (albeit not specifically documentation alone involved) is the kind of thing that is maddening to a project newcomer. When you consider how little attention to detail it requires to link to the right places, just how good is the coding part that requires intense attention to detail?

Irrelevant

Note that all the instructions are for people using SVN to download the bleeding edge. Quick show of hands here: how many of you, when faced with a new language or compiler or library automatically reach for the trunk of the source control system instead of the latest release? … That many, eh? What is that? Three hands showing? That’s two more than I was expecting.

Most newcomers are not going to reach for the bleeding edge. This is DOUBLY true when the newcomers have come to your documentation via a link from the very place they downloaded your stuff from in the first place. Here’s the work flow of a newcomer to LLVM:

  • "Huh. This sounds neat. Let’s check it out."

  • OK, download link here. Wow. That’s a lot of tarballs.

  • OK, downloaded all the tarballs finally. What’s next?

  • OK, these tarballs don’t make a lot of sense. What do the docs say?

  • Why are these docs all about the SVN check-outs? What do I do with these tarballs?

  • Huh. OK, I see. I can map the tarball names to the SVN check-outs. That’s a bit of a pain though.

Really, having the existing instructions is a bad way to communicate anything. They do not document anything that most users are actually going to do. Instead they act as a hurdle you have to jump to even use the project’s output.

Incomplete

Note that in the previous section I mentioned there are a dozen tarballs. Can you guess how many of those are represented (even indirectly) in the "getting started" docs? Five. Fewer than half. LLVM, Clang, CLang’s extra tools, the compiler runtime, and libcxx. Missing are any references to or explanations of:

  • libcxxabi

  • libunwind

  • lld

  • lldb

  • openmp

  • polly

  • test-suite

It takes digging around a half-dozen different pages on the llvm.org site, along with some guesswork, to place these properly. (To be honest I’m not actually sure I have all of them in the right place.)

Of course some will say that these aren’t necessary. But … why not mention them?! Make a list of the packages (there’s only twelve for cryin' out loud!) and say "these are the necessary ones, these are the optional ones, these are the experimental ones, …" and so on. Your newcomers don’t know the damned project. Expecting them to suddenly become experts in which pieces are needed when is idiotic!

The solution

The solution is simple. I’m going to write (in summary form) the docs that the LLVM and Clang projects should have written in the first place.

Before you do anything, however, make sure that you have all of the prerequisites. After that:

  1. Go to llvm.org/releases/download.html and download all the sources except the LLVM test suite. (You can download it if you like, actually, but if you do you’ll have to figure out where to place it on your own. Probably in the projects directory.)

  2. Untar the tarballs. Make a note of where the llvm tarball expands to. That directory will be referred to as llvm in the instructions below. Also the version you downloaded will be referred to as version.

  3. Move the cfe-version.src directory to llvm/tools/clang

  4. Move the lld-version.src directory to llvm/tools/lld

  5. Move the lldb-version.src directory to llvm/tools/lldb

  6. Move the polly-version.src directory to llvm/tools/polly

  7. Move the clang-tools-extra-version.src directory to llvm/tools/clang/tools/extra

  8. Move the compiler-rt-version.src llvm_root/projects/compiler-rt

  9. Move the libcxx-version.src llvm_root/projects/libcxx

  10. Move the libcxxabi-version.src llvm_root/projects/libcxxabi

  11. Move the libunwind-version.src llvm_root/projects/libunwind

  12. Move the openmp-version.src llvm_root/projects/openmp

Now change your working directory to llvm and issue:

  • mkdir build

  • cd build

Now here you have to decide if you want to make a debug build or a release build. For a debug build:

  • cmake ..

For a release build:

  • cmake -DCMAKE_BUILD_TYPE=Release ..

Note

I personally want a release build and I want to install to /usr/local. I thus use:

  • cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=Release ..

You’ll have to read the detailed docs if you want to do anything even more customized.

Now you just use normal make and make install commands.

Easy peasy!

Note I think I just spent more time documenting the process for the first-time user than the project maintainers did!

It gets easier, of course. You could have this in a script. Indeed the LLVM and Clang projects could even supply you with one. If they did it might look something like…

Let’s automate this!

getandbuild.sh
#! /usr/bin/bash

version=3.7.1
bases="llvm-${version}.src cfe-${version}.src compiler-rt-${version}.src"
bases="${bases} libcxx-${version}.src libcxxabi-${version}.src"
bases="${bases} libunwind-${version}.src lld-${version}.src lldb-${version}.src"              \
bases="${bases} openmp-${version}.src polly-${version}.src"
bases="${bases} clang-tools-extra-${version}.src"

# "test-suite-${version}.src" is not included because reasons.  It's easy enough
# to add if you want it.

for base in ${bases}
do
  wget -t inf -c http://llvm.org/releases/${version}/${base}.tar.xz
  tar xvf ${base}.tar.xz
done

llvm_root=llvm-${version}.src
mv -v cfe-${version}.src ${llvm_root}/tools/clang
mv -v clang-tools-extra-${version}.src ${llvm_root}/tools/clang/tools/extra
mv -v compiler-rt-${version}.src ${llvm_root}/projects/compiler-rt
mv -v libcxx-${version}.src ${llvm_root}/projects/libcxx
mv -v libcxxabi-${version}.src ${llvm_root}/projects/libcxxabi
mv -v libunwind-${version}.src ${llvm_root}/projects/libunwind
mv -v lld-${version}.src ${llvm_root}/tools/lld
mv -v lldb-${version}.src ${llvm_root}/tools/lldb
mv -v openmp-${version}.src ${llvm_root}/projects/openmp
mv -v polly-${version}.src ${llvm_root}/tools/polly

mkdir ${llvm_root}/build
cd ${llvm_root}/build
cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=Release ..
make -j3
sudo make install

Of course such a script made for the actual release would have to be a bit more bulletproof, but the concepts are simple enough. All it takes is sufficient empathy for the first-time user to spend the hour or so it takes to code such a script.