Eric's Literate Programming

Contents

Introduction
Chapter 1Basic Issues of Implementation
1.1  Language
1.2  Ancestry
1.3  Tangle and Weave
1.4  Filter
1.5  Tcl Procedures and Namespaces
Chapter 2Start and Finish
Chapter 3Namespaces
Chapter 4Another Markup System
4.1  The Structure of elp Input
4.2  The Special Brackets
4.3  Semantic Intent
4.4  Embedding Code
4.5  Advanced Macros
Chapter 5Implementing Markup
5.1  Finding the Paragraphs
5.2  Finding and Processing Macros
5.2.1    The Primary Context
5.2.2    The Control and Type Contexts
5.2.3    The Inline Context
5.3  Extracting and Expanding a Macro from the Text
5.4  Finding and Processing Code Blocks
5.5  Error Handling
Chapter 6Built-in Macros
6.1  Macros in the Primary Context
6.1.1    Configuration Macros
6.1.2    Section and Other Headings (Primary)
6.1.3    Macros to Create Lists (Primary)
6.1.4    Formatting Shortcut Macros
6.1.5    Defining New Macros
6.2  Macros in the Control Context
6.2.1    Section and Other Headings (Control)
6.2.2    Grouping Macros
6.2.3    Macros to Create Lists (Control)
6.3  Macros in the Type Context
6.4  Macros in the Inline Context
Chapter 7The Internal Format
Chapter 8Reading Input Files
Chapter 9Nested Things
Chapter 10IDs for Code Blocks and Headings
Chapter 11Generating Code Files
Chapter 12Generating HTML
12.1  Processing the Internal Format
12.2  HTML Utilities
12.3  HTML Header and Footer
12.4  HTML TOC Generation
Chapter 13The Main Procedure
Appendix ACommand Line Option Processing
Appendix BAssembly Line
Appendix CAlphabetical List of Built-in Macros
Appendix DLicensing
D.1  The elp License
D.2  Licenses for Adapted Code Fragments
D.2.1    License for W.H.Duquette's Expand
D.2.2    License for W.H.Duquette's Notebook App
D.2.3    License for TclLib textutil::expander
Appendix ERule One
Appendix FKnown Issues

Introduction

Eric's Literate Programming (elp) is a system for doing Literate Programming. In other words, from a single source file (or set of files) it can produce both a formatted document describing a program and the actual source code files for that program. The aim is for the document to explain the program to humans, with each piece of code being kept with the text that explains it, in an order chosen to make the explanation understandable rather than in an order determined by the rules of the source code language.

This document describes elp in a Literate Programming style and therefore includes the code for elp itself. Obviously in this case a lot of elp needed to be written before this document could be properly constructed. This also means that too much of the code was written before the document text, which is somewhat contrary to the spirit of Literate Programming. Improving this document will remain a work in progress.

Chapter 1   Basic Issues of Implementation

1.1  Language

The choice of Tcl as the implementation language was mostly a mere personal preference. Choosing anything else was likely to mean that elp would never get to the point of actually working, let alone being finished in any sense of the word.

The only external Tcl package used by elp is the option handler from W.H.Duquette's expand. This is only one of many command-line option handlers for Tcl, and in some sense the choice is arbitrary.

1.2  Ancestry

Any previous Literate Programming system is obviously a conceptual ancestor of elp. In the same way, any system producing a formatted document from marked-up text is also a conceptual ancestor (see Chapter 4 (Another Markup System)).

Specifically on the Tcl level, the most obvious ancestors are textutil::expander in Tcllib and its own ancestor W.H.Duquette's expand, as well as W.H.Duquette's Notebook App. A number of ideas have been taken from those, as well as a small amount of code (see Appendix D (Licensing)).

1.3  Tangle and Weave

The original Literate Programming system created the program file(s) from the single file with tangle and the formatted document from the same file with weave, but elp does both with a single program though either can be omitted depending on program options.

1.4  Filter

The elp program will be a typical Unix filter, with every non-option command-line argument specifying an input file, to be processed in the order given. If none are specified, standard input will be read. Normally, the results of processing will be written to standard output.

1.5  Tcl Procedures and Namespaces

Since elp is a Tcl program (or script), almost all logic will be in procedures (procs), with the side-effect that the order of the procedures in the file is not important to Tcl itself. It is normally recommended for Tcl packages that all procedures and shared variables should be in namespaces so that there is no interference with other packages or with global procedures and variables which would belong to the application. elp extends this to avoid all use of anything global so that it is safer to use in Androwish where multiple applications may run in the same instance.

Chapter 2   Start and Finish

Because elp is written in Tcl it is (normally) a single-file Tcl script, which will be executable on Unix/Linux if it has the execute permission and starts with something to tell the system that it should be run by a Tcl interpreter (commonly tclsh). Also, because of the decisions noted in 1.5 (Tcl Procedures and Namespaces), almost the only part of the script file that is not in a proc definition will be the few lines needed to call the main proc that controls all the processing. Those lines must be at the end of the script file so that all the procedures have been defined when it runs.

Firstly, the beginning of the script file, which is one of the well-known recipes for getting the system (Unix/Linux) to execute the file with an appropriate interpreter.

Tcl Script Heading
#!/bin/sh
# vim: set ft=tcl
# -*- tcl -*-
# The next line is executed by /bin/sh, but not tcl \
exec tclsh "$0" ${1+"$@"}

Actually the second and third lines are not part of the recipe, but are instructions for the vim and emacs editors respectively to tell them that this is a Tcl file. (Unix/Linux executables should not depend on a file extension such as .tcl to specify their type.)

Now, the end of the script file. The condition here is another well-known recipe, and is true only if the script file is executed, not if it is loaded with source.

The call to the main proc has an error trap which displays the input location then just re-raises the error. This will allow the source of an unexpected error to be found in the input file.

Run the Main Proc
if { [info script] eq $argv0 } {
    try {
        ${::elpns}::elp {*}$argv
    } on error { res opt } {
        puts stderr [string cat "In [set ${::elpns}::currfile]" \
                            " at line [set ${::elpns}::startline] :"]
        return -options $opt $res
    }
} else {
    ${::elpns}::elp -u
}

unset ::elpns

The else path uses an undocumented option to display a brief usage message.

The final line removes the only global variable created by the entire script. See Chapter 3 (Namespaces) for details.

Chapter 3   Namespaces

The main other exception to not in a proc definition is the definition of namespaces (see 1.5 (Tcl Procedures and Namespaces)).

Set up the main namespace. We keep the full namespace name of the main namespace in a global variable for use at the top level of the script file, and in a namespace variable for using in other places. The global variable will be removed at the end of the script file. It will not matter how many ancestors the main namespace has.

Namespace Creation
namespace eval elp {
    namespace export *
    set ::elpns [namespace current]
    variable elpns $::elpns

Set up an array variable for miscellaneous common data.

    variable Data

Create a counter for remembering the order of headers ( [C], [H], [S], [A], [U] ) for use when generating the table of contents.

    variable HdgOrder

Create a variable for the HTML output file descriptor.

    variable Outfd

Create a list of HTML void tags, used to avoid stacking them during HTML generation.

    variable HtmlVList

Create a list of HTML semantic formatting tags, so they can be used as Intents.

    variable HtmlSMList

Location variables for error reporting.

    variable currfile
    variable startline
}

Create a procedure to initialise (or re-initialise) the namespace variables.

proc elp::elpInit {} {
    variable Data

Clear the miscellaneous common data and set the default values. The variables shown may be changed using the vset macro. The first four are the special characters, and changing these is not expected to be necessary for most target languages.

    array unset Data
    set Data(lb) "\["
    set Data(rb) "\]"
    set Data(ecb) "@@"
    set Data(refmark) "@/"

The next two will need to be changed for languages other than English, and also if the names are not wanted in the final document (set them to ""). Other levels and types of headings do not have names.

    set Data(chapter) "Chapter "
    set Data(appendix) "Appendix "

Initialise the heading counter.

    variable HdgOrder
    set HdgOrder 0

Initialise the variable for the HTML output file descriptor.

    variable Outfd
    set Outfd "X"

Initialise the list of void HTML tags.

    variable HtmlVList
    set HtmlVList [list area base br col embed hr img input \
                            link meta param source track wbr]

Initialise the list of semantic markup HTML tags.

    variable HtmlSMList
    set HtmlSMList [list code del em ins kbd mark q samp small strong \
                        sub sup var]

Initialise section etc. numbers.

    variable ChapNum
    set ChapNum 0
    variable HeadNum
    set HeadNum 0
    variable SectNum
    set SectNum 0
    variable AppNum
    set AppNum 0

Initialise the location variables.

    variable currfile
    set currfile "INIT"
    variable startline
    set startline 0

Clear heading and codeblock information.

    array unset HdgList
    array unset CBs
}

The purpose of the following group of namespaces is to keep macros in groups so that the correct set of macros for the context may be aliased into the safe interpreter.

namespace eval ${::elpns}::primary { }
namespace eval ${::elpns}::control { }
namespace eval ${::elpns}::type { }
namespace eval ${::elpns}::inline { }

Set up an alias to allow a relatively simple means for the four namespaces above to refer to commands and variables in the parent namespace. This alias is global in that both source and target commands are in the global namespace, and it may need to be deleted in some situations. Other aliases will be created but all will refer only to commands defined in namespaces.

interp alias {} np {} namespace parent

Procs to retrieve some of the common data. These are not macros.

Since these are the first proc definitions we have seen, we should note that they are created in the main elp namespace, using the global variable created above. All proc definitions (except for macros) will use the main namespace.

proc ${::elpns}::apdx {} { variable Data ; return $Data(appendix) }
proc ${::elpns}::chap {} { variable Data ; return $Data(chapter) }
proc ${::elpns}::ecb {} { variable Data ; return $Data(ecb) }
proc ${::elpns}::lb {} { variable Data ; return $Data(lb) }
proc ${::elpns}::rb {} { variable Data ; return $Data(rb) }
proc ${::elpns}::refmark {} { variable Data ; return $Data(refmark) }

Chapter 4   Another Markup System

Any attempt to use an existing markup system has one or more of the following problems:

Of course it is possible to claim that almost all of the above points are subjective, but see Appendix E (Rule One).

Therefore, elp has its own markup language.

The rest of this chapter describes the basics of this markup, needed to make much sense of the details described in Chapter 5 (Implementing Markup)

4.1  The Structure of elp Input

The only physical structure in an elp input file is the paragraph, and the definition of a paragraph is

Any amount of text not including any blank lines, which is preceded and followed by blank lines.

For the purpose of the definition, both the beginning of a file and the end of a file are equivalent to blank lines, and any number of blank lines (except zero!) are equivalent to a single blank line. This means that if elp is reading more than one input file, a paragraph can not begin in one file and end in the next.

Lines in a paragraph may be of any length (except zero!).

There is no such thing as an empty paragraph.

Indentation is not significant at all. Tabs are not significant at all.

All of the above needs an exception to deal with the need to embed pieces of program code, each of which will be in a pseudo-paragraph called a code block. A code block starts with a specific marker, and ends with a different specific marker, and may contain blank lines. For details of what a code block looks like, see 4.4 (Embedding Code).

Logical structure is entirely within paragraphs. Headings and (for example) bullet points are just special kinds of paragraphs. Paragraphs may be grouped (to affect how they are output) merely by using special start group and end group paragraphs. Paragraphs are made special using macros.

4.2  The Special Brackets

Give elp a file as described in 4.1 (The Structure of elp Input) and the output will be a lot of neatly formatted paragraphs. To do anything more interesting, markup is needed. For markup, elp uses macros in brackets (square brackets [ and ]). This means that brackets are the only special characters - except in code blocks, and as described in Advanced Macros.

The purpose of a macro is to replace itself (including the brackets) with something else, which may be text, or formatting, or both, or even nothing at all (in which case the purpose of the macro is to tell the elp program something). Processing a macro to produce its replacement is often referred to as expanding the macro.

Inside the brackets, the first thing is the name of the macro. It is usually immediately after the left bracket, but may be preceded by spaces. If the macro does not need any further information, the right bracket follows the name, though spaces are allowed here too. If the macro does need further information, that follows the name, with at least one intervening space. The format of that information depends on the actual macro specified.

Macros are actually expanded in one of four contexts, see 5.2 (Finding and Processing Macros).

4.3  Semantic Intent

Within a paragraph, there may be many reasons to display a character, or word, or words, in some distinctive way for emphasis or to denote a file name or a program name, or many other things. Here, this is referred as the semantic intention of the piece of text, and the macros used specify, for example, format this as a file name rather than format this in a blue fixed-width font. The basic way to do this is with the [as] and [/as] macros, though other macros are available (see 6.4 (Macros in the Inline Context) and 6.1.4 (Formatting Shortcut Macros)).

4.4  Embedding Code

Embedded code is placed in pseudo-paragraphs called code blocks. A code block starts with

[codeblock] ...

and ends with

@@

and both of those lines must start at the left margin with no leading spaces or tabs.

The first line looks like a macro, but is a special case to allow elp to change from normal paragraph and macro processing to processing a code block. Once in a code block, nothing is special except for

On the starting line of a code block, [codeblock] may be followed on the same line by options for the code block:

-name blockname -file codefile -continue blockname -extend blockname

All of these are optional, but only one of -continue and -extend may be present, and if either of them is present, neither -file nor -name may be present.

-name gives the code block a name which appears as a caption above the block in the formatted document, and which may be used to refer to the block in another code block (with a @/-> line) or within the document text (with the [cblink blockname] macro).

-file specifies the name of a code file to which the block will be written if the -code command line option is given to elp. All code blocks which specify the same name will be written to that file in the order that they occur in the elp input file. This option may be used more that once, so that the code block will be written to more than one file.

-continue makes the code block a continuation of an earlier (in the elp input) code block. The purpose of this is to have a formatted text explanation in the middle of a what is otherwise a single code block.

-extend makes the code block an extension of an earlier (in the elp input) code block. Unlike -continue this block will be labelled and marked as an extension. The purpose of this is to allow a block of code which will end up in one piece to have pieces added next to the part of the document that describes their use.

A code block with none of these options is useful only as example code in the formatted document, will not be written to a code file, and can not be referenced.

4.5  Advanced Macros

WARNING: Knowledge of the Tcl language would be very useful for this part of the document.

Since elp in written in Tcl, macros are, in fact, Tcl procedures. A macro normally looks like

[name arg arg arg ... ]

and calls the Tcl procedure name with arguments arg ...

The brackets around the macro are elp macro markers (which may be changed), and not Tcl command substitution brackets, even though they (if not changed) look the same. However, between the elp brackets, the syntax rules are those of Tcl, so a macro in the elp input may look like

[sample "quoted argument" {braced argument} [string reverse olleh]]

Variable substition is not very useful, as there are normally no variables available.

Command substitution in the macro arguments, and the macro itself, are run in a Safe Interpreter

Chapter 5   Implementing Markup

5.1  Finding the Paragraphs

Scan the text to split into lines then re-form into paragraphs separated by blank lines. Some paragraphs are code blocks, which we have to detect here since they may contain blank lines.

Collecting Paragraphs
proc ${::elpns}::text2paras {txt outp} {
    variable currfile
    variable startline
    upvar $txt text
    upvar $outp output
    set offset 0
    set para ""
    set re {\n}
    set incodeblock 0
    set linenum 0
    while { [regexp -start $offset -indices $re $text pos] } {
        incr linenum
        set eol [expr {[lindex $pos 0] - 1}]
        set line [string range $text $offset $eol]
        set offset [expr {[lindex $pos 1] + 1}]

We now have a line, and have advanced the offset ready to get the next line the next time round the loop.

Now, if we are at the beginning of a paragraph and this line looks like the start of a code block, make a note that we are in a code block.

        if { $para eq "" && [string match "\\[lb]codeblock\\[rb]*" $line] } {
            set incodeblock 1
        }

Note the line number if we are starting a paragraph or code block. This will be used in error reporting.

        if { $para eq "" } {
            set startline $linenum
        }
        if { ( [string trim $line] eq "" && !$incodeblock ) 
                || ( [string trim $line] eq "[ecb]" && $incodeblock ) } {

This line is either the end of a paragraph (blank line) or the end of a code block (special marker).

If we have somehow ended up with an empty paragraph, throw it away, otherwise append the paragraph to the output, marked with P for a paragraph or CB for a code block. Then get ready for the next paragraph.

            set para [string trim $para]
            if { $para eq "" } {
                # don't have empty paragraphs
            } else {
                if { $incodeblock } {
                    lappend output CB [string trim $para] $currfile $startline
                } else {
                    lappend output P [string trim $para] $currfile $startline
                }
            }
            set para ""
            set incodeblock 0
        } else {

This line is just another line in the current paragraph.

            append para "$line\n"
        }
    }

No more lines, so deal with the last paragraph.

    set para [string trim $para]
    if { $para eq "" } {
        # don't have empty paragraphs
    } else {
        if { $incodeblock } {
            lappend output CB [string trim $para] $currfile $startline
        } else {
            lappend output P [string trim $para] $currfile $startline
        }
    }
    return ""
}

5.2  Finding and Processing Macros

This is something that is actually done three times while processing elp input:

5.2.1  The Primary Context

See 6.1 (Macros in the Primary Context) for details of the built-in macros available in this context. It is also possible to define macros for this context (see 6.1.5 (Defining New Macros)).

Primary Context
proc ${::elpns}::replacement { text } {
    variable elpns
    upvar $text txt
    set rslt ""
    set nmacros 0
    set offset 0
    set ll [string length [lb]]

Keep looking for a left bracket [, which starts a macro, until there aren't any more.

    while { [regexp -start $offset -indices "(\\[lb])" $txt ==> pos] } {
        lassign $pos s e

Copy the text (if any) up to just before the bracket.

        set this [string range $txt $offset [expr {$e - $ll}]]
        if { $this ne "" } {
            append rslt $this
        }
        set som [expr {$e + 1}]
        incr nmacros

Extract the macro which starts at the bracket.

        try {
            lassign [${elpns}::getmacro txt $som] macro eom
        } trap {ELP GETM INCOMPLETE} {res opt} {

For an incomplete macro, put a left bracket in the output, then restart scanning from the character after the bracket. This will leave the problem in the text. If some following macro does not complete it, the incomplete macro will be reported in the inline context.

            append rslt [lb]
            set offset $som
            continue
        } on error {res opt} {
            return -options $opt $res
        }
        if { $macro eq "codeblock" && \
                [string index $txt [expr {$som-2}]] eq "\n" } {

This is a [codeblock] macro at the beginning of a line so just copy everything up to and including the end marker @@ to the output without looking at it. Note the need for newline-sensitive matching to ensure the marker is on a line by itself, and non-greedy matching to find the next marker, not a later one.

            regexp -start $offset -indices "(?n).*?^([ecb])$" $txt ==> bpos
            lassign $bpos bs be
            append rslt [string range $txt [expr {$som - $ll}] $be]
            set offset [expr {$be + 1}]
        } else {

Expand the macro and add the result (which may be nothing) to the end of the output.

            try {
                set mret [${elpns}::expmacro $macro primary]
            } trap {ELP EXPM MERR} {res opt} {
                append rslt "[lb]ERROR {$res}[rb]"
                errrpt $res
                set offset [expr {$eom + 1}]
                continue
            } on error {res opt} {
                return -options $opt $res
            }
            append rslt $mret
            set offset [expr {$eom + 1}]
        }
    }

Copy any text after the last macro to the output.

    append rslt [string range $txt $offset end]
    return $rslt
}

5.2.2  The Control and Type Contexts

These contexts apply only to macros at the start of a paragraph.

The control context is for macros by themselves in a paragraph. They provide information to elp and often return nothing, but can also generate output, such as headings. Note that some macros fitting that description are in the primary context either because elp needs to know about them as early as possible, or because the are actually processed twice (e.g. headings). See 6.2 (Macros in the Control Context) for details of the built-in macros available in this context.

The type context is for macros at the start of a paragraph which identify the type of paragraph, e.g. a bullet point, or a block quote. See 6.3 (Macros in the Type Context) for details of the built-in macros available in this context.

This proc is called with the contents of one paragraph.

Control and Type Contexts
proc ${::elpns}::scanpara { content } {
    variable currfile
    variable startline
    variable elpns
    variable safeinterp
    set ll [string length [lb]]

Check whether there is a macro at the start of the paragraph.

    set c_or_t [expr { [string range $content 0 \
                [expr {$ll - 1}]] eq [lb] }]
    if { $c_or_t } {
        try {
            lassign [getmacro content $ll] macro eom
        } trap {ELP GETM INCOMPLETE} {res opt} {

The macro at the start of the paragraph is incomplete, so pretend it is not there. The problem will be reported in the inline context.

            set c_or_t 0
        } on error {res opt} {
            return -options $opt $res
        }
    }
    if { $c_or_t } {

Get the remaining text of the paragraph (if any) after the macro. Although it may contain macros, we are ignoring them.

        set rest [string range $content [expr {$eom + 1}] end]
        set rest [string trim $rest]

Decide which context this is, and expand the macro in that context.

        set ctl [expr {[expr {$eom + 1}] == [string length $content]}]
        set tag [expr {$ctl ? "control" : "type"}]
        setsafeinterp $tag
        set rslt "NONE"
        try {
            set rslt [${elpns}::expmacro $macro $tag]
        } trap {ELP EXPM UNKNOWN} {res opt} {

The macro is unknown in the context, so ignore it.

            set c_or_t 0
        } trap {ELP EXPM MERR} {res opt} {

The macro has failed, so insert the error into the output, and also report it immediately (perhaps, see 5.5 (Error Handling)).

            set rslt "P"
            lappend rslt [list PM [list ERROR $res]]
            errrpt $res
        } on error {res opt} {
            return -options $opt $res
        }
        if { $c_or_t && $ctl } {

Return the result of the control macro (with location information).

            lappend rslt $currfile $startline
            set retval $rslt
        } elseif { $c_or_t } {

Run the inline context on the rest of the paragraph, with the result of the type macro as the paragraph type.

            setsafeinterp "inline"
            set expp [do_inlines rest $rslt]
            set retval $expp
        }
    }
    if { !$c_or_t } {

No control or type macro, so just a normal paragraph, run the inline context.

        setsafeinterp "inline"
        set expp [do_inlines content "P"]
        set retval [list {*}$expp $currfile $startline]
    }
    return $retval
}

5.2.3  The Inline Context

This context is for macros that mostly provide inline formatting for part of a paragraph.

See 6.4 (Macros in the Inline Context) for details of the built-in macros available in this context.

Inline Context
proc ${::elpns}::do_inlines { text {type NONE} } {
    variable elpns
    upvar $text txt
    set rslt ""
    set nmacros 0
    set offset 0
    set ll [string length [lb]]

The rest of this is very similar to <<Primary Context>>, but different enough that merging the code would be awkward.

    while { [regexp -start $offset -indices "(\\[lb])" $txt  ==> pos] } {
        lassign $pos s e

Copy the text (if any) up to just before the bracket.

        set this [string range $txt $offset [expr {$e - $ll}]]
        if { $this ne "" } {
            lappend rslt PT $this
        }
        set som [expr {$e + 1}]
        incr nmacros

Extract the macro which starts at the bracket. If it is incomplete, report it and skip it to go on to the next.

        try {
            lassign [${elpns}::getmacro txt $som] macro eom
        } trap {ELP GETM INCOMPLETE} {res opt} {
            lappend rslt PM
            lappend rslt [list ERROR $res]
            errrpt $res
            set offset $som
            continue
        } on error {res opt} {
            return -options $opt $res
        }

If the macro is unknown or results in an error, report it and go on to the next macro. Otherwise add it as a macro result.

        try {
            set mret [${elpns}::expmacro $macro inline]
        } trap {ELP EXPM UNKNOWN} {res opt} {
            lappend rslt PM [list ERROR $res]
            errrpt $res
            set offset [expr {$eom + 1}]
            continue
        } trap {ELP EXPM MERR} {res opt} {
            lappend rslt PM [list ERROR $res]
            errrpt $res
            set offset [expr {$eom + 1}]
            continue
        } on error {res opt} {
            return -options $opt $res
        }
        lappend rslt PM $mret
        set offset [expr {$eom + 1}]
    }
    if { $nmacros == 0 } {

There were no macros, so just return what was received.

        return [list {*}$type $txt]
    }

Add any text after the last macro to the output.

    lappend rslt PT [string range $txt $offset end]
    return [list {*}$type $rslt]
}

5.3  Extracting and Expanding a Macro from the Text

To avoid the expansion of macros or even evaluation of their arguments affecting anything outside elp and its output files, all macros are run in a Safe Interpreter. The interpreter is created in Chapter 13 (The Main Procedure) but setting it up for a specified context is done when required in this procedure.

Safe Interpreter Setup
proc ${::elpns}::setsafeinterp { ctx } {
    variable safeinterp

First, clean up any existing aliases.

    foreach a [$safeinterp aliases] {
        $safeinterp alias $a {}
    }

Create aliases for all commands in the namespace corresponding to the specified context.

    foreach c [info commands ${ctx}::*] {
        $safeinterp alias [namespace tail $c] $c
    }

Create aliases for macro brackets if needed.

    if { $ctx eq "primary" } {
        $safeinterp alias lb [namespace current]::lb
        $safeinterp alias rb [namespace current]::rb
    }
}

In all three instances of processing macros described in 5.2 (Finding and Processing Macros), when left bracket [ has been found, the macro must be separated from the surrounding text before it can be expanded.

Extract Macro
proc ${::elpns}::getmacro {content offset} {
    upvar $content text
    set start $offset
    set complete 0
    while { [regexp -start $offset -indices "\\[rb]" $text pos] } {
        set macro [string range $text \
            [expr {$start}] \
            [expr {[lindex $pos 0] - 1}]]

Rather than trying to count nested brackets while looking for the matching right bracket ], we let the Tcl interpreter do it for us using the info complete command.

        if { [info complete "puts \[$macro\]"] } {
            set complete 1
            break;
        }
        set offset [expr {[lindex $pos 1] + 1}]
    }
    if { $complete } {
        return [list $macro [lindex $pos 1]]
    } else {
        set msg [string range $text [expr {$start - 4}] [expr {$start + 4}]]
        throw {ELP GETM INCOMPLETE {incomplete macro}}  \
                        "Incomplete macro in $msg"
    }
}
Expand Macro
proc ${::elpns}::expmacro { macro ctx } {
    variable elpns
    variable safeinterp
    regexp {^\s*(\S+)(\s|$)} $macro ==> mname

Make sure we have the name of the macro.

    if { ![info exists mname] } {
        throw {ELP EXPM NONAME {no valid name}} \
                    "No name for $ctx macro [lb]$macro[rb]"
    }

Check to see if the macro name exists in the passed context (command, not proc since some are aliases). If it does not exist and this is the Primary context, return the unchanged macro string surrounded by brackets, which leaves it to the later contexts. Otherwise, throw an error.

    if { [info command ${elpns}::${ctx}::$mname] eq "" } {
        if { $ctx eq "primary" } {
            return [string cat [lb] $macro [rb]]
        } else {
            throw {ELP EXPM UNKNOWN {unknown macro}} \
                    "Unknown $ctx macro [lb]$macro[rb]"
        }
    }

Try to evaluate the command which is the macro in the passed context namespace, either returning the result or throwing an error.

    #set macro [string map { {[} {[[np]::lb]} {]} {[[np]::rb]} } $macro]
    try {
        set rslt ""
        set rslt [$safeinterp eval $macro]
    } on error { res opt } {
        throw {ELP EXPM MERR {error in macro}} \
                "Error in $macro : $res"
    }
    return $rslt
}

5.4  Finding and Processing Code Blocks

This proc is called with the contents of one paragraph, already identified as a code block in <<Collecting Paragraphs>>

Process Code Block
proc ${::elpns}::scancb { para } {
    variable currfile
    variable startline
    variable elpns
    set CBdata [cbpass1 $para]
    cbpass2 $CBdata
    return [list CB $CBdata $currfile $startline]
}

Proc cbpass1 reads the code block line by line to pick out anything of interest.

proc ${::elpns}::cbpass1 {content} {
    set offset 0
    set part ""
    set rslt ""
    set re {\n|\Z}
    set l1re "^\\[lb]codeblock\\[rb]"
    set refre "^\\[refmark]-+>"
    set atatre "^\\[refmark][ecb]"
    set incodeblock 0
    set linenum 0
    set indent 0
    while { [regexp -start $offset -indices $re $content pos] } {

The use of {\n|Z} to separate lines means that the following check is necessary, but also that the last line need not be handled separately after the end of the loop.

        lassign $pos s e
        if { $offset == $s && $s > $e } {
            break
        }
        incr linenum
        set eol [expr {[lindex $pos 0] - 1}]
        set line [string range $content $offset $eol]
        if { $linenum == 1 } {
            if { [regexp -indices $l1re $line l1pos] } {

Extract the parameters from the end of the first line and make sure that they form a list with an even number of members. The bad first line error should be impossible, except that changing the special characters from brackets to some other string(s) may cause problems.

                set params [string range $line [expr {[lindex $l1pos 1] + 1}] end]
                if { [expr {[llength $params] % 2}] != 0 } {
                    set params [list error "codeblock params must be key value pairs"]
                    errrpt "codeblock params must be key value pairs"
                }
            } else {
                set params [list error "codeblock bad first line"]
                errrpt "codeblock bad first line"
            }
        } elseif { [regexp -indices $refre $line rpos] } {

This is a reference line, so end the current part (collected lines from lines that are not special (see below)) and start a new part. Collections of ordinary lines (part) and reference lines (ref) are both appended in their turn to the params list started when reading the first line of the block.

The optional indentation parameter will only be used if it is less than the actual indentation of the preceding ordinary line.

            lappend params part [join $part \n]
            set part ""
            # save the reference (including checking the parameters)
            set r [string range $line [expr {[lindex $rpos 1] + 1}] end]
            switch -- [llength $r] {
                0 {
                    lappend params error {no parameters on ref}
                    errrpt {no parameters on ref}
                }
                1 {
                    set t [lindex $r 0]
                    set r [list $t $indent]
                }
                2 {
                    lassign $r t i
                    if { $i < $indent } {
                        set r [list $t $indent]
                    }
                }
                default {
                    lappend params error {extra parameters on ref}
                    errrpt {extra parameters on ref}
                }
            }
            lappend params ref $r
        } elseif { [regexp -indices $atatre $line rpos] } {

This is a line that allows the end-of-code-block marker to appear within a code block.

            lappend params part [ecb]
        } else {

This is an ordinary line which is just added on to the current code part. The indentation of this line is remembered for use with a reference as above.

            set indent [expr {[string length $line]
                            -[string length [string trimleft $line]] }]
            lappend part $line
        }
        set offset [expr {[lindex $pos 1] + 1}]
    }

Save the last part if it is not empty.

    if { $part ne "" } {
        # end and save the last part
        set part [join $part "\n"]
        lappend params part $part
    }
    return $params
}

Proc cbpass2 validates and processes the code block options and saves them and the content of the block for use in Generating Code Files.

proc ${::elpns}::cbpass2 {cb} {
    variable currfile
    variable startline
    variable elpns
    variable MacroNum
    variable CBs
    variable CBnum
    namespace import ${elpns}::eopt::getoptions
    getoptions cb -strict {
                    {-file Opt(file) multi}
                    {-name Opt(name) string ""}
                    {-continue Opt(continue) string ""}
                    {-extend Opt(extend) string ""}
    }

    if { $Opt(continue) ne "" && $Opt(extend) ne "" } {
        return -code error [string cat "May not have -continue with -extend" \
                        " in $currfile at line $startline"]
    }
    if { $Opt(continue) ne "" } {
        set cbname cb_[textToID $Opt(continue)]
        if { ![info exists CBs($cbname-content)] } {
            return -code error [string cat "Invalid code block continuation" \
                            " in $currfile at line $startline"]
        }
        if { $Opt(name) ne "" || [llength $Opt(file)] > 0 } {
            return -code error \
                [string cat "May not have -name or -file with -continue" \
                            " in $currfile at line $startline"]
        }
        append CBs($cbname-content) "\n" $cb
    } elseif { $Opt(extend) ne "" } {
        set cbname cb_[textToID $Opt(extend)]
        if { ![info exists CBs($cbname-content)] } {
            return -code error [string cat "Invalid code block extension" \
                            " in $currfile at line $startline"]
        }
        if { $Opt(name) ne "" || [llength $Opt(file)] > 0 } {
            return -code error \
                [string cat "May not have -name or -file with -extend" \
                            " in $currfile at line $startline"]
        }
        append CBs($cbname-content) "\n" $cb
    } else {
        if { $Opt(name) ne "" } {
            set cbname cb_[textToID $Opt(name)]
            set CBs($cbname-lname) $Opt(name)
        } else {
            set cbname cb_m[incr MacroNum]
            set CBs($cbname-lname) $cbname
        }
        set CBs($cbname-content) $cb
        if { [llength Opt(file)] > 0 } { 
            set CBs($cbname-file) $Opt(file)
        }
        set CBs($cbname-order) [incr CBnum]
    }
}

5.5  Error Handling

Errors in low-level procedures may be turned into Tcl errors using throw, and may then be caught in higher-level procedures and, if necessary, reported to the user. Tcl errors, if not otherwise handled (with try or catch), will be detected and reported in <<Run the Main Proc>>. Any Tcl error that is due to a bug in elp will be reported at that point.

Reporting errors to the user is, if possible, done by embedding in the formatted document in such a way that back-end processing can make them as obvious as possible. Errors, especially those that can not be embedded in the document, are reported to stderr.

Reporting errors to stderr is done with this procedure. Note that currfile will be INIT if no files have yet been read, and that startline will be zero during processing of the Primary context.

Error Report
proc ${::elpns}::errrpt { msg } {
    variable currfile
    variable startline
    puts stderr "ERROR $msg in $currfile at line $startline"
}

Chapter 6   Built-in Macros

Macros are listed here by context. See Appendix C (Alphabetical List of Built-in Macros).

6.1  Macros in the Primary Context

These macros are processed first, even before the separation into paragraphs. They may also generate text including other macros which will be processed in one of the other contexts.

Primary Macros

<< Primary Macro A >>

<< Primary Macro C >>

<< Primary Macro cmd >>

<< Primary Macro code >>

<< Primary Macro em >>

<< Primary Macro H >>

<< Primary Macro macro >>

<< Primary Macro prog >>

<< Primary Macro q >>

<< Primary Macro replace >>

<< Primary Macro S >>

<< Primary Macro sethlink >>

<< Primary Macro strong >>

<< Primary Macro title >>

<< Primary Macro TOC >>

<< Primary Macro U >>

<< Primary Macro var >>

<< Primary Macro vget >>

<< Primary Macro vset >>

6.1.1  Configuration Macros

These macros exist to provide information to elp and are normally placed together at the beginning of the first elp input file, except for [vget] which is often used elsewhere. Using [vset] elsewhere may have complications, and is not currently recommended.

[sethlink] defines a shortcut for a hyperlink (intended to be external) so that it can be referenced in text as [hlink name]. As well as the URL, a default display text may be defined, which may be over-ridden using [hlink name new_text].

proc ${::elpns}::primary::sethlink { name url {ntext ""} } {
    variable [np]::Links
    if { $ntext eq "" } {
        set ntext $url
    }
    set [np]::Links($name) [list $ntext $url]
    return ""
}

[title] provides a title to be displayed at the top of the formatted document. The title can be used elsewhere in text with [vget title].

Primary Macro title
proc ${::elpns}::primary::title {title} {
    set [np]::Data(title) $title
    return ""
}

[vget] retrieves the value of a variable from elp's common data store. These variables can be set using [vset]. Some variables are pre-defined (in <<Namespace Creation>>) with default values.

Primary Macro vget
proc ${::elpns}::primary::vget { name } {
    return [set [np]::Data($name)]
}

[vset] sets the value of a variable in elp's common data store. Some variables are pre-defined (in <<Namespace Creation>>) with default values.

Primary Macro vset
proc ${::elpns}::primary::vset { name value } {
    set [np]::Data($name) $value
    return {}
}

6.1.2  Section and Other Headings (Primary)

These macros provide a hierarchy of Chapter, Heading, and Section headings. There is also Appendix which is on the same level as Chapter but which is lettered, not numbered. An unnumbered heading which otherwise looks like a Chapter heading is also available.

The naming of these reflects that in the (rather obscure) document preparation system Halibut. There no lower levels below Section at this stage, it should be possible to survive without them.

In this context the headings save information about themselves to allow automatic numbering and Table of Contents generation. They also leave themselves in the output of this context to be processed again in the Control context.

Primary Macro C
proc ${::elpns}::primary::C {title} {
    set tagName [[np]::textToID [[np]::striplink $title]]
    set [np]::TopStr [incr [np]::ChapNum]
    set [np]::HdgList($tagName-num) "[[np]::chap][set [np]::TopStr]"
    set [np]::HdgList($tagName-title) $title
    set [np]::HdgList($tagName-order) [incr [np]::HdgOrder]
    set [np]::HdgList($tagName-depth) 1
    set [np]::HeadNum 0
    return "[[np]::lb]C \{$title\}[[np]::rb]"
}
Primary Macro H
proc ${::elpns}::primary::H {title} {
    set tagName [[np]::textToID [[np]::striplink $title]]
    set num [incr [np]::HeadNum]
    set [np]::HdgList($tagName-num) "[set [np]::TopStr].$num"
    set [np]::HdgList($tagName-title) $title
    set [np]::HdgList($tagName-order) [incr [np]::HdgOrder]
    set [np]::HdgList($tagName-depth) 2
    set [np]::SectNum 0
    return "[[np]::lb]H \{$title\}[[np]::rb]"
}
Primary Macro S
proc ${::elpns}::primary::S {title} {
    set tagName [[np]::textToID [[np]::striplink $title]]
    set num [incr [np]::SectNum]
    set [np]::HdgList($tagName-num) "[set [np]::TopStr].[set [np]::HeadNum].$num"
    set [np]::HdgList($tagName-title) $title
    set [np]::HdgList($tagName-order) [incr [np]::HdgOrder]
    set [np]::HdgList($tagName-depth) 3
    return "[[np]::lb]S \{$title\}[[np]::rb]"
}
Primary Macro A
proc ${::elpns}::primary::A {title} {
    set tagName [[np]::textToID [[np]::striplink $title]]
    set num [incr [np]::AppNum]
    if { $num == 1 } {
        set num [scan A %c]
        set [np]::AppNum $num
    }
    set [np]::TopStr [format %c $num]
    set [np]::HdgList($tagName-num) "[[np]::apdx][set [np]::TopStr]"
    set [np]::HdgList($tagName-title) $title
    set [np]::HdgList($tagName-order) [incr [np]::HdgOrder]
    set [np]::HdgList($tagName-depth) 1
    set [np]::HeadNum 0
    return "[[np]::lb]A \{$title\}[[np]::rb]"
}
Primary Macro U
proc ${::elpns}::primary::U {title} {
    variable HdgOrder
    set tagName [[np]::textToID [[np]::striplink $title]]
    set [np]::HdgList($tagName-title) $title
    set [np]::HdgList($tagName-order) [incr [np]::HdgOrder]
    set [np]::HdgList($tagName-depth) 0
    return "[[np]::lb]U \{$title\}[[np]::rb]"
}

6.1.3  Macros to Create Lists (Primary)

[TOC] creates a heading for the Table of Contents, and also a call to [gentoc] which will be used in the Control context to actually generate the table.

The U macro generated here is never executed in the Primary context, so it is not saved in the heading list, so it doesn't appear in the generated contents, which is, of course, the way it should be.

Primary Macro TOC
proc ${::elpns}::primary::TOC {} {
    set ret "[[np]::lb]U Contents[[np]::rb]"
    append ret "\n\n[[np]::lb]gentoc[[np]::rb]"
    return $ret
}

6.1.4  Formatting Shortcut Macros

These macros provide semantic intent shortcuts, and mostly replace themselves with the provided name surrounded by [as] and [/as] macros to be processed in the Inline context.

Semantic intent for macro names.

Primary Macro macro
proc ${::elpns}::primary::macro { name } {
    set ret "[[np]::lb]as code[[np]::rb]"
    append ret "[[np]::lb]lb[[np]::rb]"
    append ret $name
    append ret "[[np]::lb]rb[[np]::rb]"
    append ret "[[np]::lb]/as[[np]::rb]"
    return $ret
}

Semantic intent for command names.

Primary Macro cmd
proc ${::elpns}::primary::cmd { args } {
    set ret "[[np]::lb]as command[[np]::rb]"
    append ret [join $args]
    append ret "[[np]::lb]/as[[np]::rb]"
    return $ret
}

Semantic intent for program code.

Primary Macro code
proc ${::elpns}::primary::code { } {
    return "[[np]::lb]as code[[np]::rb]"
}
proc ${::elpns}::primary::/code { } {
    return "[[np]::lb]/as[[np]::rb]"
}

Semantic intent for text emphasis.

Primary Macro em
proc ${::elpns}::primary::em { } {
    return "[[np]::lb]as em[[np]::rb]"
}
proc ${::elpns}::primary::/em { } {
    return "[[np]::lb]/as[[np]::rb]"
}

Semantic markup for program names.

Primary Macro prog
proc ${::elpns}::primary::prog { args } {
    set ret "[[np]::lb]as prog[[np]::rb]"
    append ret [join $args]
    append ret "[[np]::lb]/as[[np]::rb]"
    return $ret
}

Semantic intent for quoted text.

Primary Macro q
proc ${::elpns}::primary::q { } {
    return "[[np]::lb]as q[[np]::rb]"
}
proc ${::elpns}::primary::/q { } {
    return "[[np]::lb]/as[[np]::rb]"
}

Semantic intent for strong text emphasis.

Primary Macro strong
proc ${::elpns}::primary::strong { } {
    return "[[np]::lb]as strong[[np]::rb]"
}
proc ${::elpns}::primary::/strong { } {
    return "[[np]::lb]/as[[np]::rb]"
}

Semantic intent for a variable name.

Primary Macro var
proc ${::elpns}::primary::var { } {
    return "[[np]::lb]as var[[np]::rb]"
}
proc ${::elpns}::primary::/var { } {
    return "[[np]::lb]/as[[np]::rb]"
}

6.1.5  Defining New Macros

[replace shortcut "expansion"] creates a macro [shortcut] which will be replaced by its expansion whenever it is used. The expansion must be in quotes. The created macro is also in the Primary context. The original use of [replace] removes itself from the text.

In other words, this is an abbreviation!

Primary Macro replace
proc ${::elpns}::primary::replace { shorthand by } {
    set cmd [list proc $shorthand {}]
    lappend cmd [list return $by]
    {*}$cmd
    [set [np]::safeinterp] alias $shorthand [namespace current]::$shorthand
    return ""
}

6.2  Macros in the Control Context

Each of these macros must be in a paragraph by itself and is processed in the Control context.

Control Macros

<< Control Macro A >>

<< Control Macro blist >>

<< Control Macro /blist >>

<< Control Macro C >>

<< Control Macro cbgrp >>

<< Control Macro /cbgrp >>

<< Control Macro gentoc >>

<< Control Macro H >>

<< Control Macro mcol >>

<< Control Macro /mcol >>

<< Control Macro nlist >>

<< Control Macro /nlist >>

<< Control Macro R >>

<< Control Macro S >>

<< Control Macro U >>

6.2.1  Section and Other Headings (Control)

These macros generate headings in the Internal Format. Those which are numbered have had their numbers generated by the equivalent macro in the Primary context.

Control Macro C
proc ${::elpns}::control::C {title} {
    set tagName [[np]::textToID [[np]::striplink $title]]
    set top [set [np]::HdgList($tagName-num)]
    return [list CHAP $tagName $top $title]
}
Control Macro H
proc ${::elpns}::control::H {title} {
    set tagName [[np]::textToID [[np]::striplink $title]]
    set top [set [np]::HdgList($tagName-num)]
    return [list HDG $tagName $top $title]
}
Control Macro S
proc ${::elpns}::control::S {title} {
    set tagName [[np]::textToID [[np]::striplink $title]]
    set top [set [np]::HdgList($tagName-num)]
    return [list SECT $tagName $top $title]
}
Control Macro U
proc ${::elpns}::control::U {title} {
    set tagName [[np]::textToID [[np]::striplink $title]]
    return [list UCHAP $tagName {} $title]
}
Control Macro A
proc ${::elpns}::control::A {title} {
    set tagName [[np]::textToID [[np]::striplink $title]]
    set top [set [np]::HdgList($tagName-num)]
    return [list APDX $tagName $top $title]
}

This macro generates a rubric - it looks like a heading but it is un-numbered, never appears in a Table of Contents, is centred, and the text is usually coloured maroon. It is meant for author notes to self and is expected to be removed before the formatted document is considered complete.

Control Macro R
proc ${::elpns}::control::R { args } {
    return [list RUBRIC [join $args]]
}

6.2.2  Grouping Macros

These macros exist in pairs, starting and ending a group of paragraphs. The two macros of a pair have the same name except that the end macro's name starts with a / (just like HTML tags).

[blist] begins a group of bullet points ([bullet] or [b]) which will be terminated by [/blist]. Any normal paragraphs within the group will be formatted as a continuation of the preceding bullet point. These groups may be nested. Nesting is managed using the startlist and stoplist procedures in <<Nested List Stack>>.

Control Macro blist
proc ${::elpns}::control::blist {args} {
    set depth [[np]::startlist blist]
    return [list BLIST $depth $args]
}
Control Macro /blist
proc ${::elpns}::control::/blist {args} {
    set depth [[np]::stoplist blist]
    return [list /BLIST $depth $args]
}

[nlist] begins a group of numbered points ([numbered] or [n]) which will be terminated by [/nlist]. Any normal paragraphs within the group will be formatted as a continuation of the preceding numbered point. These groups may be nested and may include or be included by [nlist] groups. Nesting is managed using the startlist and stoplist procedures in <<Nested List Stack>>.

Control Macro nlist
proc ${::elpns}::control::nlist {args} {
    set depth [[np]::startlist nlist]
    return [list NLIST $depth $args]
}
Control Macro /nlist
proc ${::elpns}::control::/nlist {args} {
    set depth [[np]::stoplist nlist]
    return [list /NLIST $depth $args]
}

[mcol] and [/mcol] begin and end a section of multi-column output. The number of columns is determined by a stylesheet or other formatting mechanism in the back-end.

Control Macro mcol
proc ${::elpns}::control::mcol {args} {
    return [list MCOL $args]
}
Control Macro /mcol
proc ${::elpns}::control::/mcol {args} {
    return [list /MCOL $args]
}

[cbgrp] and [/cbgrp] begin and end group of code blocks. Normally any number (one or more) of consecutive related code blocks should be grouped. Any paragraph inside such a group will be formatted (indent etc.) to match the code blocks.

Control Macro cbgrp
proc ${::elpns}::control::cbgrp {args} {
    return [list CBGRP $args]
}
Control Macro /cbgrp
proc ${::elpns}::control::/cbgrp {args} {
    return [list /CBGRP $args]
}

6.2.3  Macros to Create Lists (Control)

[gentoc] generates a table of contents in the Internal Format from information about headings collected in the Primary context.

Control Macro gentoc
proc ${::elpns}::control::gentoc {} {
    foreach hdg [array names [np]::HdgList "*-order"] {
        set tl [list]
        set hdgname [regsub -- {-order$} $hdg ""]
        lappend toclist [set [np]::HdgList($hdg)]
        lappend tl $hdgname
        if { [info exists [np]::HdgList($hdgname-num)] } {
            lappend tl [set [np]::HdgList($hdgname-num)]
        } else {
            lappend tl {}
        }
        lappend tl [set [np]::HdgList($hdgname-title)]
        lappend tl [set [np]::HdgList($hdgname-depth)]
        lappend toclist $tl
    }
    set toclist [lsort -integer -stride 2 -index 0 $toclist]
    return [list TOC $toclist]
}

6.3  Macros in the Type Context

These macros must be at the beginning of a paragraph and define the type of the paragraph. If none of them are found, the paragraph is a standard paragraph.

Type Macros

<< Type Macro b >>

<< Type Macro bullet >>

<< Type Macro lp >>

<< Type Macro Labelledpara >>

<< Type Macro note >>

<< Type Macro n >>

<< Type Macro numbered >>

<< Type Macro quote >>

The following macro (with a shortcut alias) makes the paragraph a bullet point. Groups of bullet points (even just one) must be surrounded by the control macros [blist] and [/blist].

Type Macro b
interp alias {} ${::elpns}::type::b {} ${::elpns}::type::bullet
Type Macro bullet
proc ${::elpns}::type::bullet {args} {
    [np]::checklist blist
    return [list BI $args]
}

The following macro (with a shortcut alias) makes the paragraph a numbered point. Groups of numbered points (even just one) must be surrounded by the control macros [nlist] and [/nlist].

Type Macro n
interp alias {} ${::elpns}::type::n {} ${::elpns}::type::numbered
Type Macro numbered
proc ${::elpns}::type::numbered {args} {
    [np]::checklist nlist
    return [list NI $args]
}

The following macro (with shortcut aliases) makes the paragraph a labelled paragraph. This could be used for admonitions (notes, warnings, ...) but may have other uses. The label is meant to be displayed prominently by the backend formatters.

Type Macro lp
interp alias {} ${::elpns}::type::lp {} ${::elpns}::type::labelledpara
Type Macro note
interp alias {} ${::elpns}::type::note {} ${::elpns}::type::labelledpara note
Type Macro labelledpara
proc ${::elpns}::type::labelledpara { {type note} args} {
    return [list LP $type $args]
}

The following macro makes the paragraph a quote (usually indented and perhaps with other visual cues).

Type Macro quote
proc ${::elpns}::type::quote {args} {
    return [list QUOTE $args]
}

6.4  Macros in the Inline Context

These macros generate text and/or format information in the Internal Format.

Inline Macros

<< Inline Macro as >>

<< Inline Macro /as >>

<< Inline Macro cblink >>

<< Inline Macro ERROR >>

<< Inline Macro hlink >>

<< Inline Macro hlinkto >>

<< Inline Macro /hlinkto >>

<< Inline Macro ilink >>

<< Inline Macro lb >>

<< Inline Macro rb >>

<< Inline Macro space >>

<< Inline Macro TEX >>

Macro pair to provide a semantic intent for the intervening text (normally tranlated into formatting).

Inline Macro as
proc ${::elpns}::inline::as {intent} {
    variable IntentStack
    if { $intent eq "" } {
        [np]::errrpt [set err "[[np]::lb]as[[np]::rb] must have an intent"]
        return [list ERROR $err]
    }
    [np]::StkPush IntentStack $intent
    return [list INTENT $intent]
}
Inline Macro /as
proc ${::elpns}::inline::/as {} {
    variable IntentStack
    if { ![info exists IntentStack] } {
        set IntentStack ""
    }
    set intent [[np]::StkPop IntentStack]
    if { $intent eq "" } {
        [np]::errrpt \
            [set err "Unbalanced [[np]::lb]as[[np]::rb][[np]::lb]/as[[np]::rb]"]
        return [list ERROR $err]
    }
    return [list /INTENT $intent]
}

[cblink] generates a link to a code block in normal text. It is normally formatted as <<cb_link>>.

proc ${::elpns}::inline::cblink { name } {
    set id cb_[[np]::textToID $name]
    return [list CBLINK $id]
}

[hlink] generates a hyperlink to an external URL previously saved using the Primary macro [sethlink]. The optional second argument replaces the stored display text for the link for this use only.

proc ${::elpns}::inline::hlink { name {ntext ""} } {
    lassign [set [namespace parent]::Links($name)] txt url
    if { $ntext eq "" } {
        set ntext $txt
    }
    return [list HLINK $url $ntext]
}

[hlinkto] and [/hlinkto] are an alternative to [hlink] that allows any other formatting in the link text because it does not have to be an argument to the macro.

Inline Macro hlinkto
proc ${::elpns}::inline::hlinkto { name args} {
    lassign [set [namespace parent]::Links($name)] txt url
    return [list A $url $args]
}
Inline Macro /hlinkto
proc ${::elpns}::inline::/hlinkto {} {
    return "/A"
}

[ilink] generates a link to a heading in the document. The display text of the link is normally the text of the heading but this can be over-ridden by supplying a second argument to the macro.

proc ${::elpns}::inline::ilink { name { txt "" } } {
    set id [[np]::textToID $name]
    if { [info exists [np]::HdgList($id-num)] } {
        set num [set [np]::HdgList($id-num)]
    } else {
        set num {}
    }
    try {
        set ret [list ILINK $id $num [set [np]::HdgList($id-title)] $txt]
    } trap {TCL READ VARNAME} {res opt} {
        error "reference to missing element"
    }
    return $ret
}

[lb] and [rb] insert a left or right bracket respectively so that it will appear in the output text.

Inline Macro lb
proc ${::elpns}::inline::lb {} {
    return [list LB]
}
Inline Macro rb
proc ${::elpns}::inline::rb {} {
    return [list RB]
}

[ERROR] may be inserted into the text by the Primary context.

Inline Macro ERROR
proc ${::elpns}::inline::ERROR { text } {
    return [list ERROR $text]
}

[space] inserts a guaranteed space in the output text. This should not really be necessary, but instances of spaces being lost in web browsers have been noticed.

Inline Macro space
proc ${::elpns}::inline::space {} {
    return [list SPACE]
}

[TEX] inserts the name of the document-formatting system TeX with the correct appearance (as far as possible depending on the output format).

Inline Macro TEX
proc ${::elpns}::inline::TEX {} {
    return [list TEX]
}

Chapter 7   The Internal Format

Although currently the primary output format for the formatted document is HTML, other formats may be added in the future, so elp translates its input into an internal intermediate format and then translates that to the output format. The internal format is produced in <<Control and Type Contexts>> and <<Inline Context>>, although much of it actually comes from the built-in macros expanded in those contexts.

The internal format is a Tcl list of paragraphs. Each element of the list is itself a list, the first element of which indicates the type of paragraph, and subsequent elements (if any) are parameters and/or contents for the paragraph. The number and nature of those subsequent elements depends on the type of paragraph. This is how the HTML backend handles the elements (i.e. paragraph types) of the top-level list:

Top-level Internal Format List
foreach elt $plist {
    set t [lindex $elt 0]
    switch -exact -- $t {

Ordinary untyped paragraphs with text content.

        P {
            para2html $t [lindex $elt 1]
        }

Paragraphs with text content which have been assigned a type. Unlike untyped paragraphs, the sub-list has one or more extra leading elements the last of which is a list of options for the paragraph (not currently processed). The number of extra elements depends on the type.

        BI {
            para2html $t [lindex $elt 2]
        }
        NI {
            para2html $t [lindex $elt 2]
        }
        LP {
            para2html $t [lindex $elt 3] [lindex $elt 1]
        }
        QUOTE {
            para2html $t [lindex $elt 2]
        }

All of the above are processed in the same procedure para2html.

Code Blocks are obviously a special case.

        CB {
            cb2html [lindex $elt 1]
        }

The various types of heading just need to be formatted from the list. Output formats other than HTML may need something more complicated.

        APDX {
            lassign $elt c tag num title
            htmlout "[tag h2 class appendix id $tag]"
            htmlout "$num&nbsp;&nbsp;&nbsp;$title[tag /h2]"
        }
        CHAP {
            lassign $elt c tag num title
            htmlout "[tag h2 class chapter id $tag]"
            htmlout "$num&nbsp;&nbsp;&nbsp;$title[tag /h2]"
        }
        HDG {
            lassign $elt c tag num title
            htmlout "[tag h3 class heading id $tag]"
            htmlout "$num&nbsp;&nbsp;$title[tag /h3]"
        }
        SECT {
            lassign $elt c tag num title
            htmlout "[tag h4 class section id $tag]"
            htmlout "$num&nbsp;&nbsp;$title[tag /h4]"
        }
        UCHAP {
            lassign $elt c tag num title
            htmlout "[tag h2 class unnumbered id $tag]$title[tag /h2]"
        }

Format the Table of Contents which has already been generated.

        TOC {
            lassign $elt c tlist
            htmltoc $tlist
        }

A rubric is very much like a heading.

        RUBRIC {
            lassign $elt t rbr
            htmlout "[tag p class rubric]$rbr[tag /p]"
        }

Begin or end a group of paragraphs for some kind of special formatting.

        BLIST {
            htmlout [tag ul]
        }
        /BLIST {
            htmlout [tag /ul]
        }
        NLIST {
            htmlout [tag ol]
        }
        /NLIST {
            htmlout [tag /ol]
        }
        CBGRP {
            htmlout [tag div class cbgrp]
        }
        /CBGRP {
            htmlout [tag /div class cbgrp]
        }
        MCOL {
            htmlout [tag div class mcol]
        }
        /MCOL {
            htmlout [tag /div]
        }

Format an elp error report included in the document.

        ERROR {
            lassign $elt c tag err
            htmlout "[tag span class elperr]$c $err[tag /span]"
        }

An unknown paragraph type has been specified. Report the error as well as including it in the document.

        default {
            errrpt "Control type $t unhandled!!"
            htmlout \
                "[tag span class elperr]INTERNAL ERROR $t unhandled!![tag /span]"
        }
    }
}

A paragraph with text content and no embedded macros merely needs to be formatted according to its type, and has no structure in the internal format.

A paragraph with text content and embedded macros is, in the internal format, a sequential list of pieces, each element of which is either a list of a tag PT and the text content or a list of a tag PM and the result of expanding the macro. The macro result is a list of a type identifier and any necessary parameters, depending on the type. This is how the HTML backend handles the macro expansions:

Inline Internal Format List
switch -exact $typ {

Format a link to a URL.

    HLINK {
        lassign $text h url txt
        htmlout "[tag a href $url]$txt[tag /a]"
    }
    A {
        lassign $text h url args
        htmlout "[tag a href $url {*}$args]"
    }
    /A {
        htmlout "[tag /a]"
    }

Format a link to a Code Block using << and >> as in a code block reference.

    CBLINK {
        lassign $text i id
        if { [info exists CBs($id-lname)] } {
            set title [set CBs($id-lname)]
            htmlout "&lt;&lt;[tag a href #$id]$title[tag /a]&gt;&gt;"
        } else {
            errrpt "reference to missing code block $id"
            htmlout \
                "[tag span class elperr]ERROR reference to missing code block $id[tag /span]"
        }
    }

Format a link to a heading in the document.

    ILINK {
        lassign $text i id num title tx
        if { $tx ne "" } {
            htmlout "[tag a href #$id]$tx[tag /a]"
        } elseif { $num eq "" } {
            htmlout "[tag a href #$id]$title[tag /a]"
        } else {
            htmlout "[tag a href #$id]$num ($title)[tag /a]"
        }
    }

Format an elp error report included in the document.

    ERROR {
        htmlout "[tag span class elperr][join $text][tag /span]"
    }

Semantic markup of text pieces with Intents.

    INTENT {
        lassign $text f t
        if { $t in $HtmlSMList } {
            htmlout [tag $t]
        } else {
            htmlout [tag span class $t]
        }
    }
    /INTENT {
        lassign $text f t
        if { $t in $HtmlSMList } {
            htmlout [tag "/$t"]
        } else {
            htmlout [tag /span]
        }
    }

Output the special brackets.

    LB {
        htmlout "\["
    }
    RB {
        htmlout "\]"
    }

Output an explicitly requested space.

    SPACE {
        htmlout "&nbsp;"
    }

Output a properly formatted TeX.

    TEX {
        htmlout [tag span class "texhtml" style "font-family: 'CMU Serif', cmr10, LMRoman10-Regular, 'Nimbus Roman No9 L', 'Times New Roman', Times, serif;"]T[tag span style "text-transform: uppercase; vertical-align: -0.5ex; margin-left: -0.1667em; margin-right: -0.125em;"]e[tag /span]X[tag /span]
    }

We have an unknown type.

    default {
        errrpt "unknown Inline type $typ"
        htmlout \
            "[tag span class elperr]ERROR in inline $typ[tag /span]"
    }
}

For more details of how the internal format is handled by the HTML backend, see <<elp to HTML>>, <<Paragraph to HTML>> and <<Code Block to HTML>>.

The internal format is deliberately NOT referred to as a document tree, it is always created and processed sequentially. Headings, group start and end macros, and some other things, are merely markers at some point in the top-level list.

The internal format may be written to a specified file by using the command-line option --intfmt file_name. This is written by Tcl itself as the string representation of the list.

Chapter 8   Reading Input Files

As a Unix/Linux filter, elp reads all files named on the command line in the order given. If none are given, stdin is read, but it can also be read at any point in the list of files by using - on the command line.

Each file is read into memory with a single read, but only one file at a time is read and processed.

Read Files
proc ${::elpns}::readfile {fileName} {
    variable currfile
    if { $fileName eq "-" } {
        set fin stdin
        set currfile "stdin"
    } else {
        if {[catch "open $fileName" fin]} {
            error "Could not read file '$fileName': $fin"
        }
        set currfile $fileName
    }

    set contents [read $fin]
    if { $fileName ne "-" } {
        close $fin
    }

    return $contents
}

Chapter 9   Nested Things

Since elp needs to keep track of Grouping Macros and HTML tags and other things which can be nested, some form of stack will be useful.

This one comes from The Tclers' Wiki.

Stack Handling
interp alias {} ${::elpns}::StkPush {} lappend

proc ${::elpns}::StkPeek { name { lvl 0 } } {
    upvar 1 $name stack
    set res [lindex $stack end-$lvl]
}

proc ${::elpns}::StkPop name {
    upvar 1 $name stack
    set res [lindex $stack end]
    set stack [lreplace $stack [set stack end] end]
                                # TRICKY: Making sure list is not shared.
    set res
}

The above stack is used in tag in <<HTML Utilities>>, and also here for use by nested grouping macros.

Nested List Stack
proc ${::elpns}::startlist {macro} {
    variable NestListStk
    StkPush NestListStk $macro
    return [llength $NestListStk]
}

proc ${::elpns}::stoplist {macro} {
    variable NestListStk
    set depth [llength $NestListStk]
    if { $macro eq [StkPeek NestListStk] } {
        StkPop NestListStk
    } elseif { [llength $NestListStk] == 0 } {
        throw {ELP NSTK UFLOW} "Unable to pop $macro - no stack"
    } else {
        throw {ELP NSTK MISMATCH} \
              "Expected $macro, got [StkPeek NestListStk]"
    }
    return $depth
}

proc ${::elpns}::checklist {macro} {
    variable NestListStk
    if { ![info exists NestListStk] || $macro ne [StkPeek NestListStk] } {
        throw {ELP NSTK NOLIST} \
              "Not contained in the expected $macro.../$macro"
    }
}

Chapter 10   IDs for Code Blocks and Headings

The idea is to produce a normal form for a name or heading. This is done by the following steps:

ID from Name
proc ${::elpns}::textToID {text} {
    set text [string trim [string tolower $text]]
    regsub -all {[ ]+} $text "_" text
    regsub -all {\/} $text "X" text
    regsub -all {[^a-zX0-9_]} $text "" text
    return $text
}

Chapter 11   Generating Code Files

Code files, as specified by -file options on code blocks, are written out to the specified files by using the --code command-line option for elp.

Code Files
proc ${::elpns}::elpwritefiles {} {
    variable CBs

Find all -file options on code blocks and, for each file found, make a list of the code blocks that specify it, including a record of the order they appear in in the source file.

    foreach fl [array names CBs "*-file"] {
        set cbname [regsub -- {-file$} $fl ""]
        foreach f $CBs($fl) {
            lappend file_cbs($f) $cbname $CBs($cbname-order)
        }
    }

For every file, sort its list of code blocks into the right order, and write them to the file.

    foreach f [array names file_cbs] {
        set of [open $f {WRONLY CREAT TRUNC}]
        foreach {cb n} [lsort -stride 2 -index 1 $file_cbs($f)] {
            writecb $cb $of
        }
        close $of
    }
}

The content of code blocks was saved in proc cbpass2 in <<Process Code Block>> as a tagged list of part and ref entries.

proc ${::elpns}::writecb { cb of {prefix 0} } {
    variable CBs
    set re {\n|\Z}
    set cbid $cb
    if { ![info exists CBs($cbid-content)] } {
        errrpt "UNKNOWN CODE BLOCK $cb"
        return
    }

Loop over the tagged list for this code block.

    foreach {tag content} $CBs($cbid-content) {
        switch -exact $tag {
            error {
                errrpt "IN CODE BLOCK $cb : $content"
            }
            part {
                set indent 0
                set offset 0

Process the part, line by line, recording the indent and writing to the file.

                while { [regexp -start $offset -indices $re $content pos] } {
                    lassign $pos s e
                    if { $offset == $s && $s > $e } {
                        break
                    }
                    set eol [expr {[lindex $pos 0] - 1}]
                    set line [string range $content $offset $eol]
                    set indent [expr {[string length $line]
                                -[string length [string trimleft $line]] }]
                    set pref [string repeat " " $prefix]
                    puts $of "$pref$line"
                    set offset [expr {[lindex $pos 1] + 1}]
                }
            }
            ref {

For a reference, get the referred-to name, set the correct indent, and call this proc recursively to write the referred-to block to the file.

                lassign $content name ind
                set indent [expr {$ind > $indent ? $ind : $indent}]
                writecb cb_[textToID $name] $of $indent
            }
        }
    }
}

Chapter 12   Generating HTML

TBP - CSS

12.1  Processing the Internal Format

This procedure prepares the output file (command-line option --outfile filename, otherwise stdout), writes the HTML header, uses the tag of the outermost list to choose the right action, writes the HTML footer, and closes the output file.

elp to HTML
proc ${::elpns}::elp2html { paralist { fname "" } } {
    upvar $paralist plist
    variable Outfd
    if { $fname eq "" } {
        set Outfd stdout
    } else {
        set Outfd [open $fname {WRONLY CREAT TRUNC}]
    }
    htmlout [htmlhead]

    << Top-level Internal Format List >>

    htmlout [htmlfoot]
    close $Outfd
}
Paragraph to HTML
proc ${::elpns}::para2html { type defn args } {
    variable Links
    variable CBs
    variable HtmlSMList

    set label ""
    if { [llength $args] == 1 } {
        set label [lindex $args 0]
    }

Set the starting and ending HTML tags for this paragraph.

    switch -exact $type {
        BI {
            set tag1 "li"
            set tag2 "/li"
        }
        NI {
            set tag1 "li"
            set tag2 "/li"
        }
        P {
            set tag1 "p"
            set tag2 "/p"
        }
        LP {
            set tag1 "p"

This paragraph has a label, so first tidy up the label, then turn tag1 into a two-element list with the label as the second element.

            switch -glob $label {
                "" {
                    #
                }
                default {
                    set label "[string totitle $label]: "
                }
            }
            if { $label ne "" } {
                lappend tag1 $label
            }
            set tag2 "/p"
        }
        QUOTE {
            set tag1 "blockquote"
            set tag2 "/blockquote"
        }
    }
    if { [regexp -indices {\s*P(T|M) } $defn] } {

This is a paragraph with embedded macros.

        << Start HTML Paragraph >>

        foreach { tag text } $defn {
            if { $tag eq "PT" } {

This is just a piece of text.

                if { [string trim $text] eq "" } {
                    continue
                }
                set txt [htmlencode $text]
                htmlout "$txt"
            } elseif { $tag eq "PM" } {

This is just a macro result.

                set typ [lindex $text 0]

                << Inline Internal Format List >>

            } else {
                errrpt "unknown paragraph tag $tag"
                htmlout \
                    "[tag span class elperr]INTERNAL ERROR unknown tag $tag[tag /span]"
            }
        }
        htmlout [tag $tag2]
    } else {

This is a paragraph with no embedded macros.

        set txt [htmlencode $defn]

        << Start HTML Paragraph >>

        htmlout "$txt"
        htmlout [tag $tag2]
    }
}

In <<Paragraph to HTML>> we might have left tag1 as a two-element list. This is where we deal with that (for a labelled paragraph). This is separate because it is used more than once.

Start HTML Paragraph
        if { [llength $tag1] == 2 } {
            htmlout [tag [lindex $tag1 0]]
            htmlout [tag span class plabel]
            htmlout [lindex $tag1 1]
            htmlout [tag /span]
        } else {
            htmlout [tag $tag1]
        }
Code Block to HTML
proc ${::elpns}::cb2html {cb} {
    set cbtext ""
    set lasttag "NONE"
    foreach {tag val} $cb {

Process each piece of the code block contents. This depends on all options being before any parts or refs. If there is a error piece, there may not be any options.

        switch -exact -- $tag {
            -continue {
                set cbname $val
            }
            -extend {
                set cbname $val
            }
            -name {
                set cbname $val
            }
            -file {
                # ignore here
            }
            error {
                htmlout "[tag span class elperr]ERROR $val[tag /span]"
            }
            part {

Insert the title if the code block has a name and this is not a continuation. Also switch pre-formatting on if necessary.

                if { [info exists cbname] } {
                    if { ( $lasttag eq "NONE" || $lasttag in [list -name -file] ) } {
                        htmlout \
                            "[tag h5 id cb_[textToID $cbname] class cbcap]$cbname[tag /h5]"
                    } elseif { $lasttag eq "-extend" } {
                        htmlout \
                            "[tag h5 id cb_[textToID $cbname] class cbcap]$cbname : Extend[tag /h5]"
                    }
                }
                if { $lasttag ne "part" } {
                    htmlout [tag pre class codeblock]
                    htmlout [htmlencode $val]
                } else {
                    htmlout [htmlencode $val]
                }
            }
            ref {

Insert the title if the code block has a name and this is not a continuation. Also switch pre-formatting off if necessary. Then insert the reference link.

                if { [info exists cbname] } {
                    if { ( $lasttag eq "NONE" || $lasttag in [list -name -file] ) } {
                        htmlout \
                            "[tag h5 id cb_[textToID $cbname] class cbcap]$cbname[tag /h5]"

                    } elseif { $lasttag eq "-extend" } {
                        htmlout \
                            "[tag h5 id cb_[textToID $cbname] class cbcap]$cbname : Extend[tag /h5]"
                    }
                }
                lassign $val title indent
                if { $lasttag eq "part" } {
                    htmlout "[tag /pre]"
                }
                htmlout "[tag p class cbref]"
                set i [expr {$indent}]
                if { $i } {
                    htmlout "[tag code]"
                    htmlout "[string repeat "&nbsp;" $i]"
                    htmlout "[tag /code]"
                }
                htmlout "&lt;&lt; "
                htmlout "[tag a href #cb_[textToID $title]]"
                htmlout "$title[tag /a] &gt;&gt;[tag /p]"
            }
            default {
                if { [info exists cbname] } {
                    errrpt "codeblock format $cbname"
                    htmlout \
                        "[tag span class elperr]INTERNAL ERROR in codeblock format $cbname[tag /span]"
                } else {
                    errrpt "codeblock format (unnamed)"
                    htmlout \
                        "[tag span class elperr]INTERNAL ERROR in codeblock format (unnamed)[tag /span]"
                }
            }
        }
        set lasttag $tag
    }
    if { $lasttag ni [list -name -file ref] } {

Switch pre-formatting off if necessary.

        htmlout [tag /pre]
    }
}

12.2  HTML Utilities

Replace HTML-special characters by their character entities.

HTML Utilities
proc ${::elpns}::htmlencode {txt} {
  return [string map {& &amp; < &lt; > &gt; \" &quot; \\ &#92;} $txt]
}

The simplest possible output procedure, for the moment.

proc ${::elpns}::htmlout { text } {
    variable Outfd
    puts -nonewline $Outfd $text
    flush stdout
}

This is useless in elp at the moment, but is kept as a placeholder for something that will probably be quite different.

proc ${::elpns}::striplink {name} {
    return [regsub -all -- {</?a[^>]*>} $name ""]
}

Return an HTML tag. name is the name of the tag and args is one or more pairs of attribute names and their values. A stack is used to ensure that all HTML tags are closed. Self-closing and void tags are not placed on the stack.

In order to produce correct nested HTML5 lists, look-ahead is needed, which elp itself does not do. Here we simulate this with a pseudo-tag CLI (Continuing List Item) and look-behind, though only in as few special cases as possible.

Note: This approach means that list items that are not in a declared list will cause an error, but elp itself catches this at an earlier stage.

proc ${::elpns}::tag {name args} {
    variable HTagStack
    variable HtmlVList
    switch -- $name {

A /li tag should not actually be generated until we know what comes next, since that may need to be nested, so we push the pseudo-tag CLI onto the tag stack and return nothing instead, and any tag that needs the list item to be closed will look for that and do the right thing. Only the special cases below this will close a list item, so most things will be nested in the current list item.

        "/li" {
            if { "li" eq [StkPeek HTagStack] } {
                StkPop HTagStack
                StkPush HTagStack "CLI"
                return ""
            }
        }

An li tag should close a preceding li tag so it checks for CLI and pops it if it is there, pushes itself, then returns </li><li>. If there is no CLI, the special case does nothing, and the tag is handled normally below.

        "li" {
            if { "CLI" eq [StkPeek HTagStack] } {
                StkPop HTagStack
                StkPush HTagStack $name
                return "</li><li>"
            }
        }

A /ol or /ul tag should close a preceding list item, so the CLI (if present) should be popped from the tag stack, the corresponding ol or ol should be popped (for it to be missing is an error), and </li></ol> or </li></ol> should be returned.

        "/ol" -
        "/ul" {
            if { "CLI" eq [StkPeek HTagStack] } {
                StkPop HTagStack
                set e [string range $name 1 end]
                if { $e eq [StkPeek HTagStack] } {
                    StkPop HTagStack
                } elseif { [llength $HTagStack] == 0 } {
                    throw {ELP HSTK UFLOW} "Unable to pop $e - no stack"
                } else {
                    throw {ELP HSTK MISMATCH} "Expected $e, got [StkPeek HTagStack]"
                }
                return "</li><$name>"
            }
        }
    }
    set f [string index $name 0]
    if { $f ne "/" && [string index $name end] ne "/" &&
                $name ni $HtmlVList } {
        StkPush HTagStack $name
    } elseif { $f eq "/" } {
        set e [string range $name 1 end]
        if { $e eq [StkPeek HTagStack] } {
            StkPop HTagStack
        } elseif { [llength $HTagStack] == 0 } {
            throw {ELP HSTK UFLOW} "Unable to pop $e - no stack"
        } else {
            throw {ELP HSTK MISMATCH} "Expected $e, got [StkPeek HTagStack]"
        }
    }
    set result "<$name"
    foreach {attr val} $args {
        append result " $attr=\"$val\""
    }
    append result ">"
}

The HTML header is standard boilerplate, with a CSS file specified, and the title (if any) specified in the document used for the HTML page title and as a h1 heading in the document.

proc ${::elpns}::htmlhead {} {
    variable Data
    set ret "<!DOCTYPE html>
        [tag html lang en]
        [tag head]
        [tag meta charset utf-8]
        [tag meta name viewport content "width=device-width, initial-scale=1.0"]
        [tag link rel stylesheet href elp.css]"
    if { [info exists Data(title)] } {
        append ret [tag title]$Data(title)[tag /title]
    }
    append ret "[tag /head]
        [tag body]"
    if { [info exists Data(title)] } {
        append ret "[tag h1 class title]
        $Data(title)
        [tag /h1]"
    }
    return $ret
}

Just close the body and html tags. The stack of tags is not used to close open tags, there should not be any at this point and any error or missing tag should be reported.

proc ${::elpns}::htmlfoot {} {
    return "[tag /body][tag /html]"
}

12.4  HTML TOC Generation

Format the Table of Contents from the previously prepared list.

HTML TOC
proc ${::elpns}::htmltoc { tlist } {
    set pdepth 1
    htmlout [tag table class toc]
    foreach { n ti } $tlist {
        lassign $ti tag num name depth
        if { $depth == 0 } {
            set depth $pdepth
        }
        set pdepth $depth
        set l [string repeat "&nbsp;" [expr {( $depth - 1 ) * 2}]]
        htmlout [tag tr][tag td class tnum$depth]$num[tag /td]
        htmlout [tag td class tname]$l[tag a href #$tag]$name[tag /a]
        htmlout [tag /td][tag /tr]
    }
    htmlout [tag /table]
}

Chapter 13   The Main Procedure

The main procedure is run from <<Run the Main Proc>> if elp is run as an executable file, and may be run from the tclsh prompt after sourcing the elp script.

Main Procedure
proc ${::elpns}::elp {args} {
    variable argv $args
    variable elpns
    variable safeinterp
    variable currfile
    variable startline

Initialise (or re-initialise) the namespace variables.

    elpInit

Set up and process the command-line options.

    namespace import eopt::getoptions

    set uexit 0
    if { [catch {
                getoptions argv -strict {
                    {-code Opt(code) flag}
                    {-help Opt(h) flag}
                    {-html Opt(html) flag}
                    {-intfmt Opt(intfmt) string ""}
                    {-outfile Opt(outfile) string ""}
                    {-u Opt(u) flag}
                }
            } res opt] } {
        set uexit 1
        set Opt(h) 1
        puts $res
    }

    if { [info script] eq $::argv0 } {
        set pname $::argv0
    } else {
        set pname [set elpns]::elp
    }
    if { [info exists Opt(u)] && $Opt(u) } {
        puts "Usage:"
        puts "    $pname \[options\] \[--\] \[infile ...\]"
        puts "\n  Use $pname -help for further information."
        return
    }
    if { [info exists Opt(h)] && $Opt(h) } {
        puts "Usage:"
        puts "    $pname \[options\] \[--\] \[infile...\]"
        puts "\nParameters:"
        puts {  Other than options, all parameters will be taken as input}
        puts {  files, and processed in the order given. If no files are}
        puts {  specified, standard input will be read. Output will be}
        puts {  to standard output unless -outfile is given, when the}
        puts {  specified file will be used.}
        puts {  }
        puts {  If any input filename begins with a "-", the special}
        puts {  parameter "--" must be specified before the first filename.}
        puts "\nOptions:"
        puts "  -code           Write code to the files specified in the"
        puts "                  code blocks in the input file(s)"
        puts "  -html           Write html to standard output or to the"
        puts "                  file given by -outfile"
        puts "  -intfmt fname   Specify a file for saving the"
        puts "                  intermediate format"
        puts "  -outfile fname  Specify the single output file (use"
        puts "                  with -html)"
        puts "\n  If none of -intfmt, -html, and -code are specified,"
        puts "  then -html is assumed."
        if { [info script] eq $::argv0 } {
            exit $uexit
        } else {
            return
        }
    }
    if { $Opt(intfmt) == "" &&
         !$Opt(html) &&
         !$Opt(code) } {
        set Opt(html) 1
    }

    if { [llength $argv] == 0 } {
        lappend argv "-"
    }

If no options are given, assume -html. If no files are given, assume standard input.

    if { $Opt(intfmt) == "" &&
         !$Opt(html) &&
         !$Opt(code) } {
        set Opt(html) 1
    }

    if { [llength $argv] == 0 } {
        lappend argv "-"
    }

Create a safe interpreter for running macros.

    set safeinterp [interp create -safe]
    interp share {} stderr $safeinterp

For each input file in turn, set the primary context, read the file, do replacements in the primary context, split into paragraphs, and append the paragraph list to those from preceding files (if any).

    set paralist ""
    foreach file $argv {
        setsafeinterp "primary"
        set filedata0 [${elpns}::readfile $file]
        set filedata [${elpns}::replacement filedata0]
        unset filedata0
        ${elpns}::text2paras filedata paralist0
        unset filedata
        if { [info exists paralist0] } {
            lappend paralist1 {*}$paralist0
            unset paralist0
        }
    }

Apply the remaining contexts to the paragraph list, creating a new list which is the internal format.

    if { [info exists paralist1] } {
        foreach { flag content currfile startline } $paralist1 {
            switch -exact $flag {
                P {
                    lappend paralist [scanpara $content]
                }
                CB {
                    lappend paralist [scancb $content]
                }
            }
        }
        unset paralist1
    }
    interp delete $safeinterp

Produce output depending on the options.

    if { $Opt(intfmt) ne "" } {
        set of [open $Opt(intfmt) {WRONLY CREAT TRUNC}]
        foreach obj $paralist {
            puts $of $obj
        }
        close $of
    }
    if { $Opt(html) } {
        if { $Opt(outfile) ne "" } {
            elp2html paralist $Opt(outfile)
        } else {
            elp2html paralist
        }
    }
    if { $Opt(code) } {
        elpwritefiles
    }
}

Appendix A   Command Line Option Processing

This is a modified version of the option handler from W.H.Duquette's expand. This version has

option Package
#-------------------------------------------------------------------------------
namespace eval ${::elpns}::eopt {
    namespace export getoptions popArg
}

# Utility routine: pops an arg off of the front of an arglist.
proc ${::elpns}::eopt::popArg {arglist} {
    upvar $arglist args

    if {[llength $args] == 0} {
        set arg ""
    } elseif {[llength $args] == 1} {
        set arg [lindex $args 0]
        set args ""
    } else {
        set arg [lindex $args 0]
        set args [lrange $args 1 end]
    }

    return $arg
}

proc ${::elpns}::eopt::getoptions {arglist strictOrDefs {defsOrNil ""}} {
    # First, the arglist is called by name.
    upvar $arglist args

    # Next, strictOrDefs is either the "-strict" option or the 
    # definition list.
    if {$strictOrDefs == "-strict"} {
        set strictFlag 1
        set defList $defsOrNil
    } else {
        set strictFlag 0
        set defList $strictOrDefs
    }

    # Next, get names of the options
    set optNames {}
    set optTypes {flag enum string multi}
    set optLens {3 5 4 3}
    foreach def $defList {
        if {[llength $def] < 3} {
            error "Error in option definition: $def"
        }
        lappend optNames [lindex $def 0]
        set varName [lindex $def 1]
        set optType [lindex $def 2]
        set i [lsearch -exact $optTypes $optType]

        if {$i == -1} {
            error "Unknown option type: $optType"
        }

        if {[llength $def] < [lindex $optLens $i]} {
            error "Error in option definition: $def"
        }

        upvar $varName theVar
        switch $optType {
            flag {set theVar 0}
            enum -
            string {set theVar [lindex $def 3]}
            multi { set theVar [list] }
        }
    }

    # Next, process the options on the command line.
    set errorCount 0
    set newList {}
    set blocked 0
    while {[llength $args] > 0} {
        set arg [popArg args]

        # First, is it the special option "--"? If so, set the
        # flag to ignore any more, and don't put the "--" in the
        # revised list
        if { $arg eq "--" } {
            set blocked 1
            continue
        }

        # Next, does it look like an option?  If not, add it to the
        # output list.
        if { [string index $arg 0] != "-" || $blocked } {
            lappend newList $arg
            continue
        }

        # Next, Is the argument unknown?  Flag an error or just skip it.
        set i [lsearch -exact $optNames $arg] 
        if {$i == -1} {
            if {$strictFlag} {
                error "Unknown option: $arg"
            } else {
                lappend newList $arg
            }

            continue
        }

        # Next, process the argument
        set def [lindex $defList $i]
        set varName [lindex $def 1]
        set optType [lindex $def 2]

        upvar $varName theVar
        switch $optType {
            flag {
                set theVar 1
            }

            enum {
                set vals [lreplace $def 0 2]
                if {[llength $args] == 0 || 
                    [string index [lindex $args 0] 0] == "-"} {
                    error "Missing option value: $arg"
                }

                set theVar [popArg args]

                if {[lsearch -exact $vals $theVar] == -1} {
                    error "Invalid option value: $arg $theVar"
                }
            }

            string {
                if {[llength $args] == 0 || 
                    [string index [lindex $args 0] 0] == "-"} {
                    error "Missing option value: $arg"
                }
                set theVar [popArg args]
            }

            multi {
                if {[llength $args] == 0 || 
                    [string index [lindex $args 0] 0] == "-"} {
                    error "Missing option value: $arg"
                }
                lappend theVar [popArg args]
            }
        }
    }

    # Next, return the new argument list.
    set args $newList
    return
}
#-------------------------------------------------------------------------------

Appendix B   Assembly Line

Here, all the code blocks are assembled into the final Tcl script.

elpscript

<< Tcl Script Heading >>

<< License >>

<< Namespace Creation >>

#

<< option Package >>

#

<< Stack Handling >>

<< Nested List Stack >>

<< ID from Name >>

<< Error Report >>

#

<< Primary Macros >>

<< Control Macros >>

<< Type Macros >>

<< Inline Macros >>

#

<< Collecting Paragraphs >>

<< Primary Context >>

<< Control and Type Contexts >>

<< Inline Context >>

#

<< Safe Interpreter Setup >>

<< Extract Macro >>

<< Expand Macro >>

#

<< Process Code Block >>

<< Code Files >>

#

<< Read Files >>

#

<< elp to HTML >>

<< Paragraph to HTML >>

<< Code Block to HTML >>

<< HTML Utilities >>

<< HTML Header and Footer >>

<< HTML TOC >>

#

<< Main Procedure >>

<< Run the Main Proc >>

Appendix C   Alphabetical List of Built-in Macros

For details of each macro look at the context(s) : Primary or Control or Type or Inline.

A             control
A             primary
as            inline
/as           inline
b             type
blist         control
/blist        control
bullet        type
C             control
C             primary
cbgrp         control
/cbgrp        control
cblink        inline
cmd           primary
code          primary
/code         primary
em            primary
/em           primary
ERROR         inline
gentoc        control
H             control
H             primary
hlink         inline
hlinkto       inline
/hlinkto      inline
ilink         inline
labelledpara  type
lb            inline
lp            type
macro         primary
mcol          control
/mcol         control
n             type
nlist         control
/nlist        control
note          type
numbered      type
prog          primary
q             primary
/q            primary
quote         type
R             control
rb            inline
replace       primary
S             control
S             primary
sethlink      primary
space         inline
strong        primary
/strong       primary
TEX           inline
title         primary
TOC           primary
U             control
U             primary
var           primary
/var          primary
vget          primary
vset          primary

Appendix D   Licensing

D.1  The elp License

License
#-------------------------------------------------------------------------------
# Copyright 2018 Eric Junkermann
# 
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 
# 1. Redistributions of source code must retain the above copyright notice,
# this list of conditions and the following disclaimer.
# 
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# 
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
# HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
# For other relevant licenses, see elp.html or elpdoc.txt in the distribution.
#-------------------------------------------------------------------------------

D.2  Licenses for Adapted Code Fragments

This section is for software not used as supplied, but from which some code has been copied, adapted, or used for inspiration.

D.2.1  License for W.H.Duquette's Expand

This includes the command-line option handler, sightly modified here as eopt.

This software is copyrighted by William H. Duquette.  The following
terms apply to all files associated with the software unless
explicitly disclaimed in individual files.

The authors hereby grant permission to use, copy, modify, distribute,
and license this software and its documentation for any purpose, provided
that existing copyright notices are retained in all copies and that this
notice is included verbatim in any distributions. No written agreement,
license, or royalty fee is required for any of the authorized uses.
Modifications to this software may be copyrighted by their authors
and need not follow the licensing terms described here, provided that
the new terms are clearly indicated on the first page of each file where
they apply.

IN NO EVENT SHALL THE AUTHORS OR DISTRIBUTORS BE LIABLE TO ANY PARTY
FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES
ARISING OUT OF THE USE OF THIS SOFTWARE, ITS DOCUMENTATION, OR ANY
DERIVATIVES THEREOF, EVEN IF THE AUTHORS HAVE BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

THE AUTHORS AND DISTRIBUTORS SPECIFICALLY DISCLAIM ANY WARRANTIES,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.  THIS SOFTWARE
IS PROVIDED ON AN "AS IS" BASIS, AND THE AUTHORS AND DISTRIBUTORS HAVE
NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR
MODIFICATIONS.

GOVERNMENT USE: If you are acquiring this software on behalf of the
U.S. government, the Government shall have only "Restricted Rights"
in the software and related documentation as defined in the Federal 
Acquisition Regulations (FARs) in Clause 52.227.19 (c) (2).  If you
are acquiring the software on behalf of the Department of Defense, the
software shall be classified as "Commercial Computer Software" and the
Government shall have only "Restricted Rights" as defined in Clause
252.227-7013 (c) (1) of DFARs.  Notwithstanding the foregoing, the
authors grant the U.S. Government and others acting in its behalf
permission to use and distribute the software in accordance with the
terms specified in this license.

D.2.2  License for W.H.Duquette's Notebook App

Copyright (c) 2014, Will Duquette
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice, this
  list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
  this list of conditions and the following disclaimer in the documentation
  and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

D.2.3  License for TclLib textutil::expander

This software is copyrighted by Ajuba Solutions and other parties.
The following terms apply to all files associated with the software unless
explicitly disclaimed in individual files.

The authors hereby grant permission to use, copy, modify, distribute,
and license this software and its documentation for any purpose, provided
that existing copyright notices are retained in all copies and that this
notice is included verbatim in any distributions. No written agreement,
license, or royalty fee is required for any of the authorized uses.
Modifications to this software may be copyrighted by their authors
and need not follow the licensing terms described here, provided that
the new terms are clearly indicated on the first page of each file where
they apply.

IN NO EVENT SHALL THE AUTHORS OR DISTRIBUTORS BE LIABLE TO ANY PARTY
FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES
ARISING OUT OF THE USE OF THIS SOFTWARE, ITS DOCUMENTATION, OR ANY
DERIVATIVES THEREOF, EVEN IF THE AUTHORS HAVE BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

THE AUTHORS AND DISTRIBUTORS SPECIFICALLY DISCLAIM ANY WARRANTIES,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT.  THIS SOFTWARE
IS PROVIDED ON AN "AS IS" BASIS, AND THE AUTHORS AND DISTRIBUTORS HAVE
NO OBLIGATION TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR
MODIFICATIONS.

GOVERNMENT USE: If you are acquiring this software on behalf of the
U.S. government, the Government shall have only "Restricted Rights"
in the software and related documentation as defined in the Federal 
Acquisition Regulations (FARs) in Clause 52.227.19 (c) (2).  If you
are acquiring the software on behalf of the Department of Defense, the
software shall be classified as "Commercial Computer Software" and the
Government shall have only "Restricted Rights" as defined in Clause
252.227-7013 (c) (1) of DFARs.  Notwithstanding the foregoing, the
authors grant the U.S. Government and others acting in its behalf
permission to use and distribute the software in accordance with the
terms specified in this license.

Appendix E   Rule One

This is my program and I will write it however I want.

(which does not actually mean that suggestions will not be seriously considered)

Appendix F   Known Issues