Changes On Branch glob-docs
Not logged in

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Changes In Branch glob-docs Excluding Merge-Ins

This is equivalent to a diff from 8466a15722 to bb292b64ae

2020-03-18
17:25
Merged glob-docs branch down to trunk. check-in: b45a985c86 user: wyoung tags: trunk
17:25
Assorted improvements to globs.md, especially to the discussion of * expansion on Windows vs POSIX. Closed-Leaf check-in: bb292b64ae user: wyoung tags: glob-docs
17:03
Hoist the list of POSIX subsystem features for Windows up to the top of the "Windows" section in globs.md to wave readers off to the "POSIX" section of the same doc before we get into truly Windows-specific details. check-in: 21929d37a3 user: wyoung tags: glob-docs
15:09
Correct a statement that contradicts the first paragraph in the same section. check-in: ce5925c202 user: florian tags: glob-docs
15:03
Repeated clicks on the audio-captcha button cause it to start over. Change the title of 206 HTTP replies to "Partial Content". check-in: 8466a15722 user: drh tags: trunk
14:56
Try to make it easier for people to find the documentation on how to add new commands to the Fossil core. check-in: 29c3e99d73 user: drh tags: trunk
14:14
Change the HTTP reply status for range requests from "206 OK" to "206 Partial Content". Closed-Leaf check-in: 8a1e1c9096 user: florian tags: audio-captcha

Changes to www/globs.md.

186
187
188
189
190
191
192

193
194
195
196
197
198
199
 *  [`changes`][]
 *  [`clean`][]
 *  [`commit`][]
 *  [`extras`][]
 *  [`merge`][]
 *  [`settings`][]
 *  [`status`][]

 *  [`unset`][]

The commands [`tarball`][] and [`zip`][] produce compressed archives of a
specific checkin. They may be further restricted by options that
specify glob patterns that name files to include or exclude rather
than archiving the entire checkin.








>







186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
 *  [`changes`][]
 *  [`clean`][]
 *  [`commit`][]
 *  [`extras`][]
 *  [`merge`][]
 *  [`settings`][]
 *  [`status`][]
 *  [`touch`][]
 *  [`unset`][]

The commands [`tarball`][] and [`zip`][] produce compressed archives of a
specific checkin. They may be further restricted by options that
specify glob patterns that name files to include or exclude rather
than archiving the entire checkin.

207
208
209
210
211
212
213

214
215
216
217
218
219
220
[`changes`]: /help?cmd=changes
[`clean`]: /help?cmd=clean
[`commit`]: /help?cmd=commit
[`extras`]: /help?cmd=extras
[`merge`]: /help?cmd=merge
[`settings`]: /help?cmd=settings
[`status`]: /help?cmd=status

[`unset`]: /help?cmd=unset

[`tarball`]: /help?cmd=tarball
[`zip`]: /help?cmd=zip

[`http`]: /help?cmd=http
[`cgi`]: /help?cmd=cgi







>







208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
[`changes`]: /help?cmd=changes
[`clean`]: /help?cmd=clean
[`commit`]: /help?cmd=commit
[`extras`]: /help?cmd=extras
[`merge`]: /help?cmd=merge
[`settings`]: /help?cmd=settings
[`status`]: /help?cmd=status
[`touch`]: /help?cmd=touch
[`unset`]: /help?cmd=unset

[`tarball`]: /help?cmd=tarball
[`zip`]: /help?cmd=zip

[`http`]: /help?cmd=http
[`cgi`]: /help?cmd=cgi
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
settings` commands.

That advice does not help you when you are giving one-off glob patterns
in `fossil` commands. The remainder of this section gives remedies and
workarounds for these problems.


## POSIX Systems

If you are using Fossil on a system with a POSIX-compatible shell
— Linux, macOS, the BSDs, Unix, Cygwin, WSL etc. — the shell
may expand the glob patterns before passing the result to the `fossil`
executable.

Sometimes this is exactly what you want.  Consider this command for







|







259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
settings` commands.

That advice does not help you when you are giving one-off glob patterns
in `fossil` commands. The remainder of this section gives remedies and
workarounds for these problems.


### <a name="posix"></a>POSIX Systems

If you are using Fossil on a system with a POSIX-compatible shell
&mdash; Linux, macOS, the BSDs, Unix, Cygwin, WSL etc. &mdash; the shell
may expand the glob patterns before passing the result to the `fossil`
executable.

Sometimes this is exactly what you want.  Consider this command for
344
345
346
347
348
349
350

















351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367


368
369

370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391

392
393
394
395
396
397
398
399


400
401
402
403
404

405







406
407

408
409
410
411
412
413
414
415
416
417

418







419

420
421
422
423
424
425
426
427
428


429
















430
431
432
433
434
435
436
above to make sure the right set of files were scheduled for insertion
into the repository before checking the changes in. You never want to
accidentally check something like a password, an API key, or the
private half of a public cryptographic key into Fossil repository that
can be read by people who should not have such secrets.



















## Windows

Neither standard Windows command shell &mdash; `cmd.exe` or PowerShell
&mdash; expands glob patterns the way POSIX shells do. Windows command
shells rely on the command itself to do the glob pattern expansion. The
way this works depends on several factors:

 *  the version of Windows you are using
 *  which OS upgrades have been applied to it
 *  the compiler that built your Fossil executable
 *  whether you are running the command interactively
 *  whether the command is built against a runtime system that does this
    at all
 *  whether the Fossil command is being run from a file named `*.BAT` vs
    being named `*.CMD`

These factors also affect how a program like `fossil.exe` interprets


quotation marks on its command line.


The fifth item above does not apply to `fossil.exe` when built with
typical tool chains, but we will see an example below where the exception
applies in a way that affects how Fossil interprets the glob pattern.

The most common problem is figuring out how to get a glob pattern passed
on the command line into `fossil.exe` without it being expanded by the C
runtime library that your particular Fossil executable is linked to,
which tries to act like the POSIX systems described above. Windows is
not strongly governed by POSIX, so it has not historically hewed closely
to its strictures.

(This section does not cover the [Microsoft POSIX
subsystem](https://en.wikipedia.org/wiki/Microsoft_POSIX_subsystem),
Windows' obsolete [Services for Unix
3.*x*](https://en.wikipedia.org/wiki/Windows_Services_for_UNIX) feature,
or the [Windows Subsystem for
Linux](https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux). (The
latter is sometimes incorrectly called "Bash on Windows" or "Ubuntu on
Windows.") See the POSIX Systems section above for those cases.)

For example, consider how you would set `crlf-glob` to `*` in order to
disable Fossil's "looks like a binary file" checks. The na&iuml;ve

approach will not work:

    C:\...> fossil setting crlf-glob *

The C runtime library will expand that to the list of all files in the
current directory, which will probably cause a Fossil error because
Fossil expects either nothing or option flags after the setting's new
value.



Let's try again:

    C:\...> fossil setting crlf-glob '*'


That may or may not work. Either `'*'` or `*` needs to be passed through







to Fossil untouched for this to do what you expect, which may or may not
happen, depending on the factors listed above.


An approach that *will* work reliably is:

    C:\...> echo * | fossil setting crlf-glob --args -

This works because the built-in command `echo` does not expand its
arguments, and the `--args -` option makes it read further command
arguments from Fossil's standard input, which is connected to the output
of `echo` by the pipe. (`-` is a common Unix convention meaning
"standard input.")









Another (usually) correct approach is:


    C:\...> fossil setting crlf-glob *,

This works because the trailing comma prevents the command shell from
matching any files, unless you happen to have files named with a
trailing comma in the current directory. If the pattern matches no
files, it is passed into Fossil's `main()` function as-is by the C
runtime system. Since Fossil uses commas to separate multiple glob
patterns, this means "all files at the root of the Fossil checkout


directory and nothing else."


















## Converting `.gitignore` to `ignore-glob`

Many other version control systems handle the specific case of
ignoring certain files differently from fossil: they have you create
individual "ignore" files in each folder, which specify things ignored







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|















|
>
>
|
|
>
|
<
<




|



<
<
<
<
<
<
<
<
<

<
>
|






|
>
>





>
|
>
>
>
>
>
>
>
|
<
>





|
|
|

|
>

>
>
>
>
>
>
>
|
>



|




|
>
>
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392


393
394
395
396
397
398
399
400









401

402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427

428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
above to make sure the right set of files were scheduled for insertion
into the repository before checking the changes in. You never want to
accidentally check something like a password, an API key, or the
private half of a public cryptographic key into Fossil repository that
can be read by people who should not have such secrets.


### <a name="windows"></a>Windows

Before we get into Windows-specific details here, beware that this
section does not apply to the several Microsoft Windows extensions that
provide POSIX semantics to Windows, for which you want to use the advice
in [the POSIX section above](#posix) instead:

  *  the ancient and rarely-used [Microsoft POSIX subsystem][mps];
  *  its now-discontinued replacement feature, [Services for Unix][sfu]; or
  *  their modern replacement, the [Windows Subsystem for Linux][wsl]

[mps]: https://en.wikipedia.org/wiki/Microsoft_POSIX_subsystem
[sfu]: https://en.wikipedia.org/wiki/Windows_Services_for_UNIX
[wsl]: https://en.wikipedia.org/wiki/Windows_Subsystem_for_Linux

(The latter is sometimes incorrectly called "Bash on Windows" or "Ubuntu
on Windows," but the feature provides much more than just Bash or Ubuntu
for Windows.)

Neither standard Windows command shell &mdash; `cmd.exe` or PowerShell
&mdash; expands glob patterns the way POSIX shells do. Windows command
shells rely on the command itself to do the glob pattern expansion. The
way this works depends on several factors:

 *  the version of Windows you are using
 *  which OS upgrades have been applied to it
 *  the compiler that built your Fossil executable
 *  whether you are running the command interactively
 *  whether the command is built against a runtime system that does this
    at all
 *  whether the Fossil command is being run from a file named `*.BAT` vs
    being named `*.CMD`

Usually (but not always!) the C runtime library that your `fossil.exe`
executable is built against does this glob expansion on Windows so the
program proper does not have to. This may then interact with the way the
Windows command shell you’re using handles argument quoting. Because of
these differences, it is common to find perfectly valid Fossil command
examples that were written and tested on a POSIX system which then fail
when tried on Windows.



The most common problem is figuring out how to get a glob pattern passed
on the command line into `fossil.exe` without it being expanded by the C
runtime library that your particular Fossil executable is linked to,
which tries to act like [the POSIX systems described above](#posix). Windows is
not strongly governed by POSIX, so it has not historically hewed closely
to its strictures.










For example, consider how you would set `crlf-glob` to `*` in order to

get normal Windows text files with CR+LF line endings past Fossil's
"looks like a binary file" check. The na&iuml;ve approach will not work:

    C:\...> fossil setting crlf-glob *

The C runtime library will expand that to the list of all files in the
current directory, which will probably cause a Fossil error because
Fossil expects either nothing or option flags after the setting's new
value, not a list of file names. (To be fair, the same thing will happen
on POSIX systems, only at the shell level, before `.../bin/fossil` even
gets run by the shell.)

Let's try again:

    C:\...> fossil setting crlf-glob '*'

Quoting the argument like that will work reliably on POSIX, but it may
or may not work on Windows. If your Windows command shell interprets the
quotes, it means `fossil.exe` will see only the bare `*` so the C
runtime library it is linked to will likely expand the list of files in
the current directory before the `setting` command gets a chance to
parse the command line arguments, causing the same failure as above.
This alternative only works if you’re using a Windows command shell that
passes the quotes through to the executable *and* you have linked Fossil
to a C runtime library that interprets the quotes properly itself,
resulting in a bare `*` getting clear down to Fossils `setting` command

parser.

An approach that *will* work reliably is:

    C:\...> echo * | fossil setting crlf-glob --args -

This works because the built-in Windows command `echo` does not expand its
arguments, and the `--args -` option makes Fossil read further command
arguments from its standard input, which is connected to the output
of `echo` by the pipe. (`-` is a common Unix convention meaning
"standard input," which Fossil obeys.) A [batch script][fng.cmd] to automate this trick was
posted on the now-inactive Fossil Mailing List.

[fng.cmd]: https://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg25099.html

(Ironically, this method will *not* work on POSIX systems because it is
not up to the command to expand globs. The shell will expand the `*` in
the `echo` command, so the list of file names will be passed to the
`fossil` standard input, just as with the first example above!)

Another (usually) correct approach which will work on both Windows and
POSIX systems:

    C:\...> fossil setting crlf-glob *,

This works because the trailing comma prevents the glob pattern from
matching any files, unless you happen to have files named with a
trailing comma in the current directory. If the pattern matches no
files, it is passed into Fossil's `main()` function as-is by the C
runtime system. Since Fossil uses commas to separate multiple glob
patterns, this means "all files from the root of the Fossil checkout
directory downward and nothing else," which is of course equivalent to
"all managed files in this repository," our original goal.


## Experimenting

To preview the effects of command line glob pattern expansion for
various glob patterns (unquoted, quoted, comma-terminated), for any
combination of command shell, OS, C run time, and Fossil version,
preceed the command you want to test with [`test-echo`][] like so:

    $ fossil test-echo setting crlf-glob "*"
    C:\> echo * | fossil test-echo setting crlf-glob --args -

The [`test-glob`][] command is also handy to test if a string
matches a glob pattern.

[`test-echo`]: /help?cmd=test-echo
[`test-glob`]: /help?cmd=test-glob


## Converting `.gitignore` to `ignore-glob`

Many other version control systems handle the specific case of
ignoring certain files differently from fossil: they have you create
individual "ignore" files in each folder, which specify things ignored