Documenting SWI-Prolog Projects with PlDoc

Introduction

Beginning PlDoc

Generating Documentation

Project Documentation

Conclusion

Introduction

Target Audience

Motivation

Requirements

License

Code

Target Audience

This tutorial is for anybody who wishes to use the powerful facilities that SWI-Prolog provides for generating project documentation. The student is expected to know Prolog in general and the SWI-Prolog dialect in specific to a self-assessed level of at least advanced beginner. The student should also be sufficiently motivated to document code that learning a comprehensive and sometimes complicated documentation system is not a problem.

Motivation

Any source code in any language, when needing to be maintained for an extended time or when being worked upon by multiple coders, requires good documentation. This documentation, to be complete must take three forms: tutorial, overview or guide, and reference. It is this third kind of documentation that PlDoc is chiefly designed for, although it can assist in the first two as well.

Through the use of so-called structured comments, PlDoc, like JavaDoc, Doxygen, EDoc and other such source code documentation facilities, generates high quality documentation in LaTeX format. PlDoc also provides a web server facility which can be used by the developer to browse documentation in situ; this is a valuable tool for library and package documentation. Finally PlDoc also provides a standalone web server facility for putting documentation into the public eye.

PlDoc is designed to be simple to start using and to be rewarding to the developers who use it (individual or group) from the start. While the whole package put together can seem overwhelming, the individual pieces are simple and can be used quickly with little fuss or bother.

Requirements

To go through this tutorial you will require the following:

To get the most out of the tutorial you will need to:

Note that the samples are in subdirectories of the project root named according to the headings of the tutorial. The names will be all in lower case and will have whitespace replaced with underscores. Thus, for example, if there were sample files for this section, they would be found in the introduction/requirements subdirectory. The final version of the samples is contained in the src directory for those eager to see the end instead of paying attention to the journey.

License

This tutorial is Copyright (c) 2012 Michael T. Richter. It comes without any warranty of any kind, including, but not limited to, that of accuracy, of utility, or even of good taste to the extent permitted by applicable law. You can redistribute it and/or modify it under the terms of the Do What The Fuck You Want To Public License, Version 2.

Code

All of the code in this tutorial stems from section 2.8 of the Creating Web Applications in SWI-Prolog tutorial by Anne Ogborn. The files have been renamed and commensurate changes have been made in the source. No other changes have been made. The source code has been used with permission.

Beginning PlDoc

Comment Format

Comment Syntax

Module Comments

Comment Format

(The files for this section can be found in the beginning_pldoc/comment_format directory.)

So we've been busy little beavers and we've made ourselves a little web application. This web app is going to be bigger than Facebook by far, however, so we need to get ready to add more team workers to the mix. After careful consultation with the oracle of our choice (the author prefers chicken intestines) we've decided that, given that our target platform is SWI-Prolog, the supplied PlDoc facility is how to begin.

But how do we start?

We know (because we've been told a dozen times by now) that PlDoc works through something called "structured comments" which sounds exactly like what we want: keep the documentation together with the source it documents.

Let's crack open the server.pl file and look at the first predicate listed:

server(Port) :- http_server(http_dispatch, [port(Port)]).

That's not a very friendly predicate! Any newcomer to the code base is almost certainly going to find this bewildering. There's nothing there to guide understanding! Also, there's nothing there we can use to build our external documentation.

Structured comments to the rescue! There are two variants of structured comments based upon the /* . . . */ C-style comments or the % Prolog-style comments depending on your preferences. In either case, however, the internal format is the same. A structured comment consists of three sections in the following order:

  1. A semi-formal type and mode description.
  2. A TWiki-style documentation body.
  3. JavaDoc/Doxygen-style tags.

Let's concoct a pair of structured comments for the predicate we've listed above.

Prolog-style comments

%% server(+Port:int) is nondet
%
%  The server/1 predicate launches the server main loop at the provided port.
%  If launched, for example with Port set to 8000, the server can be browsed at
%  http://localhost:8000.
%  
%  @param Port The port the server should listen to.

In a Prolog-style comment, the type and mode descriptions are flagged by use of the %% commenting convention. The types and modes are read from the first line that uses the double-comment symbol and end with the first line that uses only a single comment symbol. PlDoc will then read and process the documentation body until the first line beginning with the @ symbol. It will then process the rest of the comment block as tags. Blank lines are ignored while processing (which is convenient for readable formatting of the comments in source). So in the above example, the following is the type and mode description block:

server(+Port:int) is nondet

The following is the documentation body:

The server/1 predicate launches the server main loop at the provided port.
If launched, for example with Port set to 8000, the server can be browsed at
http://localhost:8000.

The following is the tags block:

@param Port The port the server should listen to.

C-style comments

/**
 * server(+Port:int) is nondet
 *
 * The server/1 predicate launches the server main loop at the provided
 * port.  If launched, for example with Port set to 8000, the server can be
 * browsed at http://localhost:8000.
 *  
 * @param Port The port the server should listen to.  */

The above comment is exactly the same in meaning to the Prolog-style block we did above and generates exactly the same type/mode block, documentation block and tags block. The double ** symbol identifies the beginning of a PlDoc comment. The type/mode block begins with the first non-blank line (which is why we could start it on a line by itself above). It ends with the first blank line afterwards which signals the beginning of the documentation body. This itself ends with the first line beginning with @, signalling the beginning of the tags block.

A note on future style

Given that the C-style blocks and the Prolog-style blocks are identical in semantics to PlDoc, all future comment blocks we do in this tutorial will follow the Prolog style of commenting except for the module comment block (which we will address later). This is done to avoid unnecessary replication of code.

Completing the predicate comments

The files server_out.pl and module_out.pl contain the results of our first attempt to document the predicates of our source code in preparation for the incoming massive team that will have to work with our code. Read the files preferably side by side with the originals and see how much easier it is to figure out the module even without any documentation generation!

Exercises
  1. Using the server_out.pl file provided, convert the Prolog-style comments into C-style comments.
  2. Then convert some of them back.
  3. Check the structured comments provided in that file and see what other kinds of information would be useful. (Use the online PlDoc reference for types, tags etc. to get ideas.) Add them to the file.

Comment Syntax

(The files for this section can be found in the beginning_pldoc/comment_syntax directory.)

The three sections of a PlDoc comment have three different syntax conventions. None of them is particularly difficult, but it is important to get them in the right place.

Types and Modes

Types and modes are not formally a part of Prolog. They are, however, commonly referenced in the literature and documentation about Prolog systems. As concepts they are vital to understanding predicates, even if they're not enforced by the compiler (and, indeed, have no standing at all even in the runtime system.

The full documentation for the type and mode declarations can be found in the SWI-Prolog documentation. To help explain the (rather terse) documentation we will here replicate the type and mode declarations from the previous section and explain them.

server(+Port:int)

Reading this in English one would say something like "the server/1 predicate takes an input parameter named Port which is an integer." Note that nothing in the language enforces any aspect of that description. You could try to pass in an uninstantiated variable. You could pass in an atom like booger. You could name the parameter IckyThing in the actual implementation. All of this will do nothing until runtime at which point you have to contend with SWI-Prolog's interpreter getting ticked off with you.

Even worse, unlike the instantiation symbols (+, -, ?, :, @, !) there are no standard types. I used int in this one. It could just as easily have been integer or 'this here is pretty weird as a type'. Of course any sane project will have a set of standards for these types written up in an easily available document. (We'll be talking more about this later.)

Note, also, when you look at the official documents, that the instantiation pattern and the type are optional. As is the determinism mode (a fact I made use of here because I'm not actually sure what the determinism of server/1 is).

say_hi(?Request) is det

This predicate is as simple as the above one. I have chosen to drop the type here (as I'm not familiar enough with the HTTP library in use to know for sure what to call it). In this case, however, I have added the words is det to the end. This is because I know what the determinism mode of this predicate is supposed to be. It is not supposed to be able to fail and it can only generate one value. This makes it, in the terminology of Mercury, the programming language PlDoc borrows the terminology from, "deterministic" and is thus annotated with is det.

Also of interest in this one is that the instantiation symbol is ?. This is because the HTTP Request sent in is both an input parameter and an output one. Parts of it are instantiated and used to help direct the predicate calls throughout the system and parts of it are unified behind the scenes to fill out the response.

page_content(?Request)// is det

This predicate works like say_hi/1 but has the added // annotation. This means that the predicate in question is a DCG clause. Discussion of DCG clauses is out of scope for this tutorial, however, except for mentioning how they're annotated.

nav(?Name:atom, ?HREF:atom) is nondet

Well, we're back with types here and we've got the is nondet annotation added. To explain why this predicate is non-deterministic, let's first look at its implementation:

nav('Home', '/home').
nav('About', '/about').
nav('Recipes', '/recipes').

What happens if we call this with nav(X, Y)? The runtime leaves choice points; we can get up to three results back from this. That means it has to be non-deterministic or multiple-return (is nondet or is multi). To know which it is, consider what happens if we call nav('YAY TEAM!', X) or nav(X, 'YAY TEAM!'). We would get a failure. The is multi indicator specifies that it cannot fail; it must succeed at least one time. Thus nav/2 is non-deterministic.

This brings us to our final example:

as_top_nav(+Name, -Pred) is semidet

Straightforward enough: as_top_nav/2 takes a name as an input parameter and returns a predicate term as an output parameter. But why is it semi-deterministic?

Semi-deterministic predicates will either succeed precisely once or they will fail. Let's look at the implementation:

as_top_nav(Name, a([href=HREF, class=topnav], Name)) :-
        nav(Name, HREF).

Now since Name is an input parameter, it must be instantiated. This means that nav/2 will either return a value in its HREF parameter or it will not. There will be no choice points. One value or no values. It's semi-deterministic.

With this explanation you should be ready to read and understand the SWI-Prolog reference for type and mode declarations. The only thing left to note is that it is perfectly acceptable to have multiple type and mode declarations for a single predicate. Prolog is a remarkably flexible language and, PlDoc being a documentation tool (as opposed to a compiler), it is probably best to document the expected use cases in the type and mode declarations block. Like this:

length(+List:list, -Length:int) is det.     % 1.
length(?List:list, -Length:int) is nondet.  % 2.
length(?List:list, +Length:int) is det.     % 3.

These declarations, taken from the PlDoc reference, illustrate the three common use cases for the list length operation:

  1. Return in Length the number of elements in List.

    e.g. length([1,2,3], L).

  2. Generate any number of lists in List (filled with uninstantiated variables) and return the list generated plus the length in Length.

    e.g. length(L, M).

  3. Generate a list with Length uninstantiated variable and return it in List.

    e.g. length(L, 5).

Exercises
  1. Visit the SWI-Prolog Library documentation and look over the various modules. For those with type and mode declarations (not all of them have this!) try to figure out what is meant by each and, importantly, try to work out why the determinism is what it's listed as.

Documentation Body

The syntax of the documentation body is a Wiki notation based upon the venerable TWiki web application. Details of the conventional structuring notation are left for exploration of the documentation. The Prolog-specific extensions, however, warrant some further exploration.

First, TWiki

TWiki is itself a very large, very complicated Wiki syntax. It is far outside of the scope of this tutorial to show every nook and cranny of it. Instead the alterations of and enhancements to the TWiki format are discussed here, along with some notes on how to use the more common markup tags. For more details on the TWiki format itself, consult the following sources:

The last link, in particular, is likely of most direct use, especially when combined with the PlDoc documentation page.

Term lists

When describing parameters or various use cases of a predicate, it is common to provide example terms to explain the various kinds of behaviour. Consider, for example, the length/2 predicate shown above. The type declaration provided shows three use cases. The body of the description should show the provided examples:

* length([1,2,3], Size)
  Return in `Size` the length of the list `[1,2,3]`.
  (`Size = 3.`)
* length(List, Size)
  Starting at `Size=0`, generate lists `List` of uninstantiated variables.
  ( `List = [], Size = 0; List = [_G11], Size = 1; List = [_G11, _G12], Size = 2,` ... )
* length(List, 2)
  Generate a list of 2 uninstantiated variables.
  ( `List = [_G11, _G12].` )

Note when using these that the closing . character should not be provided. The Wiki processor will supply it on your behalf (a questionable approach in this author's humble opinion).

Link markup

The usual TWiki rules for links apply with little change. There are three Prolog-specific enhancements of TWiki's notation: predicate links, file links and images. In addition there is an extra form of hyperlink notation.

Extra link notation

Any construct in the form of <url> is converted into a hyperlink. It is chiefly used when making complex urls through www_browser:expand_url_path/2 aliasing in which case the construct is <alias>:<local>. (You will have to consult documentation for this feature should you choose to use it.)

Predicate links

Predicates can be linked by simple use of functor/arity (regular) or functor//arity (DCG) in the documentation body. Any such predicates which are documented in anything else processed by PlDoc will be linked in the ensuing documentation. Note, this will not link to documentation that PlDoc hasn't been informed about!

File links

Any name.ext will be converted to a link to the named file if the file exists and ext is any of:

Images

Images can be linked to as stated above for file links, but to have images displayed instead they must be wrapped in double square braces: [[my_image.png]].

Predicate lists

Unlike link markup of predicate names (like length/2), predicate lists are used to bring in the content of a predicate's description inline in your documentation. This is usually most useful for file-level documentation, but can be useful even in predicate-level documentation. The following two examples will illustrate the differences:

The following predicates are useful for manipulating lists in Prolog:

* member/2
* append/3
* prefix/2
...

This code will provide the user with a list of links to the named predicates. Compare and contrast with:

The following predicates are useful for manipulating lists in Prolog:

* [[member/2]]
* [[append/3]]
* [[prefix/2]]
...

This code will instead bring in the documentation for the cited predicates. This permits replication of information without the dangers of being required to change source in multiple locations.

Font markup

Font markup works as normal for TWiki-style markup with one very important exception: font markup will only work as documented for one word at a time! This is because of the prevalence of =, * and _ in code. Should a larger stream of text be desired in an alternative font, it must be wrapped in additional vertical bar (|) characters:

Blocks of code are preceded and followed by == as follows:

==
append([], A, A).
append([A|B], C, [A|D]) :-
        append(B, C, D).
==
Exercises
  1. Visit the SWI-Prolog Library documentation and look over the various modules. See if you can find examples of all of the above syntax constructs in the real documentation.
  2. (Optional) If you have the SWI-Prolog source installed on your system, inspect it for each of the above syntax elements and compare it to the results in the online documentation.

Tags

As with the documentation body, the details of the tags are best left to the reference documentation. The following tags are best-suited to documenting predicates, however, with the remaining tags more useful for file-level documentation (which is addressed later):

(As a point of style it would probably be better to fix bugs than to document them!)

Exercises
  1. Visit the SWI-Prolog Library documentation and look over the various modules. See if you can find examples of all of the above tags in the real documentation.
  2. (Optional) If you have the SWI-Prolog source installed on your system, inspect it for each of the above tags and compare it to the results in the online documentation.

Module Comments

(The files for this section can be found in the beginning_pldoc/module_comments directory.)

So far we have focused on documenting predicates, but a good reference document should also provide information at the module level as well. Documenting a complex module by only explaining individual predicates is rather like documenting how to build a house by explaining the component bricks and beams and nails and such in isolation: confusing and pointless.

The files module.pl and server.pl are the output of the previous chapter unaltered. We will be adding module level comments to them to illustrate how this is accomplished; the ensuing files will be module_out.pl and server_out.pl. Keep in mind that just as an example of a coding style the module comment will be done as a C-style comment (/* ... */) instead of a Prolog-style one (% ...).

IMPORTANT NOTE

It appears that as a bug or by design, PlDoc requires the module comment to be a C-style comment! It is only by good fortune that the author of this piece didn't stumble headlong into this problem by selecting, at random, C-style module comments for illustrative purposes.

The syntax of module comments is identical to that of the predicate comments except that the type and mode declaration is replaced by <module> followed by the title of the module. Here is the module comment for the server, by way of example:

/** <module> Sample HTTP Server
 *
 *  This module is part of Anne Ogborn's [[Creating Web Applications in
 *  SWI-Prolog][http://www.pathwayslms.com/swipltuts/html/index.html]]
 *  tutorial.  It illustrates the
 *  [[Mailman][http://www.swi-prolog.org/pldoc/doc_for?object=section%284,%273.17.2%27,swi%28%27/doc/packages/http.html%27%29%29]]
 *  facility of SWI-Prolog's web application library.
 *
 *  @author Anne Ogborn (Prolog code)
 *  @author Michael T. Richter (PlDoc markup)
 *  @version ch2.8
 *  @see http://www.pathwayslms.com/swipltuts/html/index.html
 *  @copyright (c)2012 Anne Ogborn.
 *  @license All rights reserved.  Used with permission.
 */

Aside from the "type" header, the only real difference between a module-scoped comment and a predicate-scoped one is the tags which apply. The tags most likely to apply to the module scope are:

One thing to beware of, the :- module(...). declaration must precede the module comment. Failure to do this will result in the module documentation not being generated!

Generating Documentation

Accomplishments So Far

During Development

HTTP Server and Browser

LaTeX Documentation

Accomplishments So Far

While it may not seem like it, we have actually accomplished a lot so far. Indeed the most important part of documentation is behind us: the content. By using a structured tool for writing the reference documentation we have source files that completely document the module in a way that aids comprehension; this is vital to any team-oriented task work.

Still, that being said, it would be nice, wouldn't it, if we could reference this documentation in ways other than scrolling through our source.

During Development

(The files for this section can be found in the generating_documentation/during_development directory.)

One of the almost-unique uses of PlDoc lies in its use as an online documentation server in development. The work cycle goes roughly like this:

  1. Write code, complete with PlDoc comments.
  2. Consult the documentation of existing predicates and modules (including third-party ones) whilst coding the new ones.
  3. Lather. Rinse. Repeat.

Any module anywhere that you use can be referenced in this documentation provided the source is available and suitably commented. (This includes the SWI-Prolog system libraries themselves, along with any packs you may have installed.) This feature makes PlDoc a valuable tool in your toolbox.

To make use of this, however, you'll need to modify your load/debug/whatever scripts. (You do use these, right?) In this tutorial, we will be making use of load.pl and debug.pl.

load.pl

:- load_files([server, module], []).
:- unload_file(debug).
:- unload_file(load).

Our load script is dirt simple (to go along with the dirt simple web application underlying it). It merely loads our server.pl and module.pl files and then unloads the debug.pl and load.pl file to avoid clutter in the documentation server.

debug.pl

:- doc_server(4040).
:- portray_text(true).
[load].

Our debug script, on the other hand, does some interesting stuff. The first thing it does is it starts up the documentation server on port 4040. You can consult the full docs for details, but in general your debug script will fire up the documentation server with doc_server/1 or doc_server/2 as we did. You will probably also want to call portray_text/1 to ensure that strings are displayed as, well, strings (instead of lists of integers). Once all that has been done, the debug script simply consults the load script (which we've already seen above).

Putting it together

From within our project directory we fire up SWI-Prolog and call the debug script as follows:

$ swipl
% /home/michael/.plrc compiled 0.03 sec, 1,501 clauses
Welcome to SWI-Prolog (Multi-threaded, 64 bits, Version 6.3.0)
Copyright (c) 1990-2012 University of Amsterdam, VU Amsterdam
SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to redistribute it under certain conditions.
Please visit http://www.swi-prolog.org for details.

For help, use ?- help(Topic). or ?- apropos(Word).

1 ?- [debug].
% Started Prolog Documentation server at port 4040
% You may access the server at http://localhost:4040/
%     library(prolog_stack) compiled into prolog_stack 0.01 sec, 72 clauses
%    library(http/http_error) compiled into http_error 0.02 sec, 80 clauses
%    module compiled into module 0.00 sec, 10 clauses
%   server compiled 0.03 sec, 127 clauses
%   module compiled into module 0.00 sec, 1 clauses
%  load compiled 0.03 sec, 133 clauses
% debug compiled 1.23 sec, 11,556 clauses
true.

2 ?-

At this point the documentation server is running and the project code has been loaded. We should be good to go. Open a browser at http://localhost:4040 and see the glory!

...

Waitwhat?! This isn't the documentation server! There's nothing document-like about this! What's going on?

Well, one of the weaknesses of PlDoc is one we wouldn't have seen if it weren't for the fact that our sample project is a web server. PlDoc and our web server are running at the same time in the same instance of SWI-Prolog and they're clashing. Our web app wins because its mapping of the root happens after the one the documentation server made.

The solution to this problem is fairly trivial. We have to pick an url we want to serve our help from and modify the debug script (now called debug2.pl for purposes of this tutorial) to take this into account.

debug2.pl

:- use_module(library(http/http_path)).
http:location(pldoc, root('help/source'), [priority(10)]).

:- doc_server(4040).
:- portray_text(true).

:- [load2].

The change here is to define very early in the proceedings an http:location/3 predicate that redirects all PlDoc urls to, in this case, <base_url>/help/source. Now instead of accessing http://localhost:4040 we instead access http://localhost:4040/help/source. Do that now and you'll be rewarded with our documentation.

One very important item to note is that the full contents of the module comment will not be displayed unless the source file is a proper Prolog module. Only the name of the file will be displayed on its reference page, for example. To have all of your documentation information displayed (as detailed later), you will have to add a line like the following (which we've added for our example) to your code:

:- module(server, [http_handler/3, server/1]).

Public IconPrivate IconAn additional point to note is that by default only the public (i.e. exported) predicates will initially show in the documentation. To see the private (i.e. non-exported) predicates, you will have to click on the icon that looks like the image to the left. To switch back to the public-only view you will have to click on the icon that looks like the image to the right.

Exercises
  1. Remove one or both of the unload_file declarations from load.pl and see what the results are.
  2. Remove the portray_text declaration in debug.pl or debug2.pl and see what the results are.

The point of all this

There are several reasons why being able to access the documentation live while coding is useful:

That last point needs some expansion. As an experiment, change some of the markup in the structured comments in server.pl while the documentation server is running. Now hit the refresh in your browser. Watch as nothing whatsoever changes! Now go to your SWI-Prolog console and type [load]. followed by the return key. Refresh your browser again. Note the changed markup! The magic is that PlDoc's server will reconstruct its information every time you load a module. (We just reloaded everything by using [load]., but we could just as easily have used [server]. instead.) When working on shared code bases, on a live development session you could update your code base from an SCM repository, reload and have all your partners' working documentation updated on your screen!

Exercises
  1. Add a new module to the project. Ensure that it contains:

    • a module declaration
    • a module-level structured comment
    • at least one predicate
    • one predicate-level structured comment per predicate added

    Add the new module to the load.pl script and reload with [load]. Check the results in your browser after reloading the page.

HTTP Server and Browser

The same ability to do documentation reading and updating while working live on the code can be used to serve up the documentation as a straight-up documentation server. (This is, indeed, how the SWI-Prolog documentation itself is provided to the web.) To do this you use the doc_server/2 predicate with the allow(...) option to permit the documentation server to work on more than localhost. You would then, instead of loading your code, use the doc_load_library/0 predicate.

Full details on this (ridiculously simple) process are available in the PlDoc Reference Manual.

LaTeX Documentation

(The files for this section can be found in the generating_documentation/latex_documentation directory.)

Good documentation needs to be available offline as well. PlDoc helps in this with its LaTeX back-end which can easily and quickly generate high-quality reference documentation with a quick command line at the shell:

swipl -g "doc_latex(['server.pl','module.pl'], 'doc.tex', [public_only(false)]),halt" -t "halt(1)"

Obviously this will have to be put into a script file or a Makefile goal because you won't want to type this each time, but the operation of this could not be simpler. The doc_latex/3 predicate here is being used with these arguments:

The option selected specifies that we want all predicates documented, not only those exported from the module. (The reason for this is that our source files aren't done in modules.)

Full details of using doc_latex/3 are available in the PlDoc reference documentation. Other, finer-grained predicates are also documented there.

One important point to note is that you need the pldoc.sty LaTeX style sheet on your command line for latex or in your working directory. This file can be found in <swipl-library-root>/library/pldoc.

Exercises
  1. Using the extra module you added from previous exercises, generate a TeX document that includes your new module.
  2. Experiment with the command line to reorder the modules in the document.
  3. Write a PrologScript (or equivalent) that gives convenient command line access to creating TeX documents based on arguments.
  4. Create a Makefile target called docs that automatically bundles all of a project's Prolog source files into a TeX document. (This may or may not involve the script you created previously.)

Project Documentation

How Project Documentation Differs

Text Files

Building a Better Reference

Building a User Guide

Other Documents

How Project Documentation Differs

So far we have very good documentation, but only for the developers on the team. It's purely reference documentation and it's not tied well together. For people who know the project already this isn't a problem, but we don't just document for our current team, do we? We'll have newcomers to bring up to speed and we'll have third-party users to contend with. Even our reference documentation nees a bit more of a punch.

To tie the documentation together into a more coherent whole, we need to have some way of organizing our documentation within a broader framework like an opening section that talks about the project as a whole and what its components are. This section could, for example, talk about how the various source files relate to each other or could assemble the predicates into related groups with interspersed commentary instead of merely dumping them in source code order.

In addition there are document types that go beyond simple references like user guides (which offer architectural overview, typical work flows, hook points, expected use cases and "recipes" for use) or tutorials (which, like this document, give detailed instructions for basic use for first-time users). Both of these can be written using PlDoc's markup (although there will be decreasing value the farther you move from the structured code comments).

Text Files

(The files for this section can be found in the project_documentation/txt_files directory.)

The first tool available for documentation purposes is the humble text file. Several such files are treated specially by PlDoc and are used to help structure documentation into something meaningful. All of the Wiki markup cited above and in the SWI-Prolog documentation apply here and, indeed, a wider subset of the markup is likely to be used in text files (including, for example, horizontal rules or heading markup).

README and TODO

README, which may also be named README.TXT or README.txt, is used to help build the so-called directory index. The directory index is a landing page built by the PlDoc document server. The contents of README are first displayed, followed by a table of all the project source files (plus their descriptive summary) and their public predicates.

TODO, which may also be named with the .TXT or .txt suffix as above, is appended to the directory index.

A note on naming

PlDoc will serve up README and TODO over the versions with .txt or .TXT suffixes. That is to say if you have a README file and a README.txt file, PlDoc will display the contents of the former in the landing page, not the latter.

This is unfortunate because for maximum portability across platforms you should probably use the *.txt naming convention; in Windows, for example, it's impossible to associate the name README with a program while README.txt is already configured out of the box to open with a text editor.

Exercises
  1. Create a README, a README.txt, and a README.TXT file and figure out which is higher priority than which when PlDoc loads.
  2. Do the same for TODO.

.txt files

Whenever the text README, TODO, or any name ending in .txt or .TXT is encountered inside Wiki source, a link is generated. (Other files share this property.) Any such referenced text files are processed as Wiki documents, allowing for rich mark-up. This is one of the primary techniques for documenting more than just references.

An example

The following files have been added to the project to help rein in the documentation chaos:

Run SWI-Prolog from that directory and enter the following:

?- [debug].

Now fire up the browser and point at http://localhost:4040/help/source. Note how the better use of structured commenting in README and TODO make the landing page much more accessible. It's also nice having links to the tutorial and user guide immediately visible even before the module reference. (Note also how the module reference had a header added beforehand to make it stand out more in the formatting.)

If you clink through to the tutorial or to the user guide you'll also see documentation formatted from whole cloth using the TWiki syntax.

But ... what's this oddity in userguide.txt?

Building a Better Reference

The entire contents of userguide.txt are shown below:

---+ "So You Want to Make Prolog Web Apps?"

---++ A user guide.

---+++ Devoid of any meaningful content.

Because this is just an example after all.

---++++ Well, here's a bit of content.

* [[http_handler/3]]

If, however, you click through on it, you will see a whole lot more than just these lines (formatted by the wiki processor). Indeed you'll see a lot of information for the http_handler/3 predicate including details that aren't in our source code.

This is the magic of the predicate description list format. By using that notation we tell the document processor that we want it to replicate the full documentation for the named predicate at the given point in the document.

This is a powerful capability that allows better reference manuals to be made. Instead of relying on the (admittedly not bad) landing page with its alphabetic list of modules and the underlying alphabetic list of public predicates, it is quite possible to instead have a richer reference document that pulls in predicate references (including from the whole SWI-Prolog system, not just our own code!) in an order that makes more sense to the user.

As an example, the modules could be introduced in an order that is more likely to be seen by someone actually using the software. Related predicates could be grouped in such a way as to be together, even if one predicate is called anteater/2 and the other is called zebra/1. Explanations on how to use predicates (e.g. sample code) can be interspersed where they make the most sense. You'll still have your landing page and straightforward module and predicate dictionary, but you'll also have better, more usefully-arranged reference material without code replication.

Exercises
  1. Build a better reference using TWiki syntax and predicate description lists.

Building a User Guide

Building a user guide is just like building a better reference, only moreso. When building the user guide you'll be doing less wholesale importing of predicate documentation and making more use of code examples, term lists, and links to predicate documentation.

When building a user guide, you may find yourself having to repeatedly introduce things like sample code verbatim in several parts of your document. In such circumstances putting the snippet of code into its own file and referencing it in your documentation like [[snippet.txt]] will import it verbatim. If you have to change your sample code, you only change it in one place and it will be updated in your documentation wherever it is referenced.

Other Documents

User guides and tutorials are the obvious kinds of extra documentation that a project is going to need. Other documents, however, are also possible and, indeed, for any project of any decent size, required. These can include:

The imagination of the users is about the only limit to the kinds of uses to which PlDoc can be put in a team environment.

One document that's probably really needed

Remember the part where we talked about the types and modes? Remember how it was mentioned that a sane team would probably have a set of standards for the type names written up? Remember how we then hand-waved over that? The waving stops here. It is almost mandatory, in the opinion of this author, to have a rigorous list of types documented in a single place. The file name could even be something difficult to remember like types.txt. Linking to (or flat-out including with double square brackets!) this document in the README file would give a convenient way to reference the available types while coding or reading the project's reference documentation. Such a document could be very easily written like this:

---## Standard Types
   $ int32:   a 32-bit signed integer
   $ int64:   a 64-bit signed integer
   $ request: a composite value containing an HTTP request
   $ str:     a list of integers interpreted as a string (encoding not
              specified)
   $ utf8str: a list of integers interpreted as a string (UTF-8 encoded)
   $ name:    an atom interpreted as a name

Note here that the "types" are intentional types: they document the intent of the passed-in type. For example in SWI-Prolog there is no such thing as a "32-bit integer" or a "64-bit integer". In a given piece of software, however, there may be reason to be aware of intended restrictions; the PlDoc types are for documentation purposes, not for enforcement.

Rather than documenting the literal type (which, in Prolog, is a rather restricted set!), it is, in the opinion of this author, more important to document how the type is intended to be used with implicit limitations.

Exercises
  1. Make a document that documents the types used in server.pl and module.pl.

Conclusion

Credits

Contact Information

Credits

The author of this piece would like to thank Anne Ogborn for the impetus to write this tutorial, as well as for the sample code used to illustrate it.

Contact Information

The author of this piece can be reached via email at ttmrichter@gmail.com. This is also his GoogleTalk ID. He has been known to frequent the ##prolog channel on the Freenode IRC service as ttmrichter. He also has a Google+ Page upon which he vents his spleen to his ever-shrinking set of so-called "circles". He also maintains a blog of sorts.