Fossil

Check-in [c1b62c3260]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Merge in documentation enhancements from trunk.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | vdiff-improvements
Files: files | file ages | folders
SHA3-256: c1b62c32602179d24892a454d92e4ec5a8dbcda912769d5c791473afcec2311e
User & Date: drh 2019-08-27 00:29:40.125
Context
2019-08-27
01:47
Add the "merge-in:NAME" name type, similar to "root:NAME" except that it finds the youngest anscestor of NAME that is in the branch from which the branch of NAME derived. check-in: dcd8f1d8f4 user: drh tags: vdiff-improvements
00:29
Merge in documentation enhancements from trunk. check-in: c1b62c3260 user: drh tags: vdiff-improvements
00:29
Remove an unused subroutine. Fix a minor CSS problem. check-in: 2078c746a5 user: drh tags: vdiff-improvements
2019-08-25
13:24
Added HTTP proxying info to Debian nginx server setup guide. check-in: c6a033cea6 user: wyoung tags: trunk
Changes
Unified Diff Ignore Whitespace Patch
Changes to www/server/any/scgi.md.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20








21



22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

45

46
47

48
49
50
51
52


53
54
55
56


# Serving via SCGI

There is an alternative to running Fossil as a [standalone HTTP
server](./none.md), which is to run it in SimpleCGI (a.k.a. SCGI) mode,
which uses the same [`fossil server`](/help/server) command as for HTTP
service. Simply add the `--scgi` command-line option and the stand-alone
server will speak the SCGI protocol rather than raw HTTP.

This can be used with a web server such as [nginx](http://nginx.org)
which does not support [Fossil’s CGI mode](./cgi.md).

A basic nginx configuration to support SCGI with Fossil looks like this:

        location /example/ {
            include scgi_params;
            scgi_pass localhost:9000;
            scgi_param SCRIPT_NAME "/example";
            scgi_param HTTPS "on";
        }









Start Fossil so that it will respond to nginx’s SCGI calls like this:




        fossil server /path/to/repo.fossil --scgi --localhost --port 9000

The `--scgi` option switches Fossil into SCGI mode from its default,
which is [stand-alone HTTP server mode](./none.md). All of the other
options discussed in that linked document — such as the ability to serve
a directory full of Fossil repositories rather than just a single
repository — work the same way in SCGI mode.

The `--localhost` option is simply good security: we’re using nginx to
expose Fossil service to the outside world, so there is no good reason
to allow outsiders to contact this Fossil SCGI server directly.

Giving an explicit non-default TCP port number via `--port` is a good
idea to avoid conflicts with use of Fossil’s default TCP service port,
8080, which may conflict with local uses of `fossil ui` and such.

Fossil requires the `SCRIPT_NAME` environment variable in order to
function properly, but nginx does not provide this variable by default,
so it is necessary to provide it in the configuration.  Failure to do
this will cause Fossil to return an error.

The [example `fslsrv` script](/file/tools/fslsrv) shows off these same

concepts in a more complicated setting. You might want to mine that

script for ideas.


You might want to next read one of the platform-specific versions of this
document, which goes into more detail:

*   [Debian/Ubuntu](../debian/nginx.md)



There is a [separate article](../../tls-nginx.md) showing how to add TLS
encryption to this basic SCGI + nginx setup.

*[Return to the top-level Fossil server article.](../)*















|

<
|
|


>
>
>
>
>
>
>
>
|
>
>
>

|















|
<
<
|
|
<
>
|
>
|

>
|
|
|
|

>
>
|
|


>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49


50
51

52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
# Serving via SCGI

There is an alternative to running Fossil as a [standalone HTTP
server](./none.md), which is to run it in SimpleCGI (a.k.a. SCGI) mode,
which uses the same [`fossil server`](/help/server) command as for HTTP
service. Simply add the `--scgi` command-line option and the stand-alone
server will speak the SCGI protocol rather than raw HTTP.

This can be used with a web server such as [nginx](http://nginx.org)
which does not support [Fossil’s CGI mode](./cgi.md).

A basic nginx configuration to support SCGI with Fossil looks like this:

        location /code/ {
            include scgi_params;

            scgi_param SCRIPT_NAME "/code";
            scgi_pass localhost:9000;
        }

The `scgi_params` file comes with nginx, and it simply translates nginx
internal variables to `scgi_param` directives to create SCGI environment
variables for the proxied program; in this case, Fossil. Our explicit
`scgi_param` call to define `SCRIPT_NAME` adds one more variable to this
set, which is necessary for this configuration to work properly, because
our repo isn’t at the root of the URL hierarchy. Without it, when Fossil
generates absolute URLs, they’ll be missing the `/code` part at the
start, which will typically cause [404 errors][404].

The final directive simply tells nginx to proxy all calls to URLs under
`/code` down to an SCGI program on TCP port 9000. We can temporarily
set Fossil up as a server on that port like so:

        $ fossil server /path/to/repo.fossil --scgi --localhost --port 9000 &

The `--scgi` option switches Fossil into SCGI mode from its default,
which is [stand-alone HTTP server mode](./none.md). All of the other
options discussed in that linked document — such as the ability to serve
a directory full of Fossil repositories rather than just a single
repository — work the same way in SCGI mode.

The `--localhost` option is simply good security: we’re using nginx to
expose Fossil service to the outside world, so there is no good reason
to allow outsiders to contact this Fossil SCGI server directly.

Giving an explicit non-default TCP port number via `--port` is a good
idea to avoid conflicts with use of Fossil’s default TCP service port,
8080, which may conflict with local uses of `fossil ui` and such.

We characterized the SCGI service start command above as “temporary”


because running Fossil in the background like that means it won’t start
back up on a reboot of the server. A simple solution to that is to add

that command to `/etc/rc.local` on systems that have it. However, you
might want to consider setting Fossil up as an OS service instead, so
that you get the benefits of the platform’s service management
framework:

*   [Linux (systemd)](../debian/service.md)
*   [Windows service](../windows/service.md)
*   [macOS (launchd)](../macos/service.md)
*   [xinetd](../any/xinetd.md)
*   [inetd](../any/inetd.md)

We go into more detail on nginx service setup with Fossil in our
[Debian/Ubuntu specific guide](../debian/nginx.md). Then in [a later
article](../../tls-nginx.md) that builds upon that, we show how to add
TLS encryption to this basic SCGI + nginx setup on Debian type OSes.

*[Return to the top-level Fossil server article.](../)*

[404]: https://en.wikipedia.org/wiki/HTTP_404
Changes to www/server/debian/nginx.md.
64
65
66
67
68
69
70
71
72
73
74
75

76
77
78
79
80
81
82

## <a name="modes"></a>Fossil Service Modes

Fossil provides four major ways to access a repository it’s serving
remotely, three of which are straightforward to use with nginx:

*   **HTTP** — Fossil has a built-in HTTP server: [`fossil
    server`](/help/server).  While this method is efficient and it’s
    possible to use nginx to proxy access to another HTTP server, this
    option is overkill for our purposes.  nginx is itself a fully
    featured HTTP server, so we will choose in this guide not to make
    nginx reinterpret Fossil’s implementation of HTTP.


*   **CGI** — This method is simple but inefficient, because it launches
    a separate Fossil instance on every HTTP hit.

    Since Fossil is a relatively small self-contained program, and it’s
    designed to start up quickly, this method can work well in a
    surprisingly large number of cases.







|
|
|
<
|
>







64
65
66
67
68
69
70
71
72
73

74
75
76
77
78
79
80
81
82

## <a name="modes"></a>Fossil Service Modes

Fossil provides four major ways to access a repository it’s serving
remotely, three of which are straightforward to use with nginx:

*   **HTTP** — Fossil has a built-in HTTP server: [`fossil
    server`](../any/none.md).  While this method is efficient and it’s
    possible to use nginx to proxy access to another HTTP server, we
    don’t see any particularly good reason to make nginx reinterpret

    Fossil’s own implementation of HTTP when we have a better option.
    (But see [below](#http).)

*   **CGI** — This method is simple but inefficient, because it launches
    a separate Fossil instance on every HTTP hit.

    Since Fossil is a relatively small self-contained program, and it’s
    designed to start up quickly, this method can work well in a
    surprisingly large number of cases.
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156

157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184





185
186
187
188
189
190
191
192
193
194
195
196
197

198
199
200
201
202
203
204
205
206
207
208
209
210
211
your server, then say:

       $ sudo apt install fossil nginx


## <a name="scgi"></a>Running Fossil in SCGI Mode

I run my Fossil SCGI server instances with a variant of [the `fslsrv`
shell script](/file/tools/fslsrv) currently hosted in the Fossil source
code repository. You’ll want to download that and make a copy of it, so
you can customize it to your particular needs.

This script allows running multiple Fossil SCGI servers, one per
repository, each bound to a different high-numbered `localhost` port, so
that only nginx can see and proxy them out to the public.  The
“`example`” repo is on TCP port localhost:12345, and the “`foo`” repo is
on localhost:12346.

As written, the `fslsrv` script expects repositories to be stored in the
calling user’s home directory under `~/museum`, because where else do
you keep Fossils?

That home directory also needs to have a directory to hold log files,
`~/log/fossil/*.log`. Fossil doesn’t put out much logging, but when it
does, it’s better to have it captured than to need to re-create the
problem after the fact.

The use of `--baseurl` in this script lets us have each Fossil
repository mounted in a different location in the URL scheme.  Here, for
example, we’re saying that the “`example`” repository is hosted under
the `/code` URI on its domains, but that the “`foo`” repo is hosted at
the top level of its domain.  You’ll want to do something like the
former for a Fossil repo that’s just one piece of a larger site, but the
latter for a repo that is basically the whole point of the site.

You might also want another script to automate the update, build, and
deployment steps for new Fossil versions:

       #!/bin/sh
       cd $HOME/src/fossil/trunk
       fossil up
       make -j11
       killall fossil
       sudo make install
       fslsrv

The `killall fossil` step is needed only on OSes that refuse to let you
replace a running binary on disk.

As written, the `fslsrv` script assumes a Linux environment.  It expects
`/bin/bash` to exist, and it depends on non-POSIX tools like `pgrep`.
It should not be difficult to port to systems like macOS or the BSDs.



## <a name="config"></a>Configuration

On Debian and Ubuntu systems the primary user-level configuration file
for nginx is `/etc/nginx/sites-enabled/default`. I recommend that this
file contain only a list of include statements, one for each site that
server hosts:

      include local/example
      include local/foo

Those files then each define one domain’s configuration.  Here,
`/etc/nginx/local/example` contains the configuration for
`*.example.com` and `*.example.net`; and `local/foo` contains the
configuration for `*.foo.net`.

The configuration for our `foo.net` web site, stored in
`/etc/nginx/sites-enabled/local/foo` is:

      server {
          server_name .foo.net;
          include local/generic;

          access_log /var/log/nginx/foo.net-https-access.log;
           error_log /var/log/nginx/foo.net-https-error.log;

          # Bypass Fossil for the static Doxygen docs





          location /doc/html {
              root /var/www/foo.net;

              location ~* \.(html|ico|css|js|gif|jpg|png)$ {
                  expires 7d;
                  add_header Vary Accept-Encoding;
                  access_log off;
              }
          }

          # Redirect everything else to the Fossil instance
          location / {
              include scgi_params;

              scgi_pass 127.0.0.1:12345;
              scgi_param HTTPS "on";
              scgi_param SCRIPT_NAME "";
          }
      }

As you can see, this is a simple extension of [the basic nginx service
configuration for SCGI][scgii], showing off a few ideas you might want to
try on your own site, such as static asset proxying.

The `local/generic` file referenced above helps us reduce unnecessary
repetition among the multiple sites this configuration hosts:

      root /var/www/$host;







<
<
<
<
|
<
<
<
<
<
|
<
<
<
|
<
<
<
<
|
<
<
<
<
<
<
<

<
<
|
<
<
|
<
|
<
|

<
<
|
<
<
<
>









|
|


|
|
|

|
|


|


|
|

|
>
>
>
>
>
|
|









|

>

<
<



|







105
106
107
108
109
110
111




112





113



114




115







116


117


118

119

120
121


122



123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171


172
173
174
175
176
177
178
179
180
181
182
your server, then say:

       $ sudo apt install fossil nginx


## <a name="scgi"></a>Running Fossil in SCGI Mode





For the following nginx configuration to work, it needs to contact a





Fossil instance speaking the SCGI protocol. There are [many ways](../)



to set that up. For Debian type systems, we primarily recommend




following [our systemd user service guide](service.md).










Another option would be to customize [the `fslsrv` shell


script](/file/tools/fslsrv) that ships with Fossil as an example of

launching multiple Fossil instances in the background to serve multiple

URLs.



However you do it, you need to match up the TCP port numbers between it



and those in the nginx configuration below.


## <a name="config"></a>Configuration

On Debian and Ubuntu systems the primary user-level configuration file
for nginx is `/etc/nginx/sites-enabled/default`. I recommend that this
file contain only a list of include statements, one for each site that
server hosts:

      include local/example.com
      include local/foo.net

Those files then each define one domain’s configuration.  Here,
`/etc/nginx/local/example.com` contains the configuration for
`*.example.com` and its alias `*.example.net`; and `local/foo.net`
contains the configuration for `*.foo.net`.

The configuration for our `example.com` web site, stored in
`/etc/nginx/sites-enabled/local/example.com` is:

      server {
          server_name .example.com .example.net "";
          include local/generic;

          access_log /var/log/nginx/example.com-https-access.log;
           error_log /var/log/nginx/example.com-https-error.log;

          # Bypass Fossil for the static documentation generated from
          # our source code by Doxygen, so it merges into the embedded
          # doc URL hierarchy at Fossil’s $ROOT/doc without requiring that
          # these generated files actually be stored in the repo.  This
          # also lets us set aggressive caching on these docs, since
          # they rarely change.
          location /code/doc/html {
              root /var/www/example.com/code/doc/html;

              location ~* \.(html|ico|css|js|gif|jpg|png)$ {
                  expires 7d;
                  add_header Vary Accept-Encoding;
                  access_log off;
              }
          }

          # Redirect everything else to the Fossil instance
          location /code {
              include scgi_params;
              scgi_param SCRIPT_NAME "/code";
              scgi_pass 127.0.0.1:12345;


          }
      }

As you can see, this is a pure extension of [the basic nginx service
configuration for SCGI][scgii], showing off a few ideas you might want to
try on your own site, such as static asset proxying.

The `local/generic` file referenced above helps us reduce unnecessary
repetition among the multiple sites this configuration hosts:

      root /var/www/$host;
219
220
221
222
223
224
225
226
227
228
229
























230
variables into, citing performance considerations, so there is a limit
to how much repetition you can squeeze out this way. One such example is
the `access_log` and `error_log` directives, which follow an obvious
pattern from one host to the next. Sadly, you must tolerate some
repetition across `server { }` blocks when setting up multiple domains
on a single server.

The configuration for `example.com` and `example.net` is similar.

See [the nginx docs](http://nginx.org/en/docs/) for more ideas.

























*[Return to the top-level Fossil server article.](../)*







|



>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
variables into, citing performance considerations, so there is a limit
to how much repetition you can squeeze out this way. One such example is
the `access_log` and `error_log` directives, which follow an obvious
pattern from one host to the next. Sadly, you must tolerate some
repetition across `server { }` blocks when setting up multiple domains
on a single server.

The configuration for `foo.net` is similar.

See [the nginx docs](http://nginx.org/en/docs/) for more ideas.


## <a name="http"></a>Proxying HTTP Anyway

[Above](#modes), we argued that proxying SCGI is a better option than
making nginx reinterpret Fossil’s own implementation of HTTP.  If you
want Fossil to speak HTTP, just [set Fossil up as a standalone
server](../any/none.md). And if you want nginx to [provide TLS
encryption for Fossil][tls], proxying HTTP instead of SCGI provides no
benefit.

However, it is still worth showing the proper method of proxying
Fossil’s HTTP server through nginx if only to make reading nginx
documentation on other sites easier:

        location /code {
            rewrite ^/code(/.*) $1 break;
            proxy_pass http://127.0.0.1:12345;
        }

The most common thing people get wrong when hand-rolling a configuration
like this is to get the slashes wrong. Fossil is senstitive to this. For
instance, Fossil will not collapse double slashes down to a single
slash, as some other HTTP servers will.

*[Return to the top-level Fossil server article.](../)*