Fossil

Things to work on:

1. Automated self-test

2. Add a diff option similar to --tk that shows the graphical diff in a web-browser instead of a Tk window. Implemented in 2.17

3. Improvements to the "grep" command:

   • Search filenames given on the command-line or all managed files if no filesnames are supplied
   • Search the check-in named on the command-line or the current check-out if no check-in is named
   • --from VERSION and --to VERSION options to search a range of check-ins
   • --timeline, --tickets, or --wiki PATTERN to search things other than check-ins
   • --before DATE and --after DATE to limit the timespan of a search.
   • Always output filename and line number
   • Only show matches not found in adjacent check-ins, unless --all is used.
   • --earliest shows only the first match and --latest shows only the most recent match when grepping a range of check-ins.
   • -l just lists matching files
   • --diff-only only search the difference between check-ins

4. Change the PHANTOM table into a view on BLOB using a partial index where BLOB.SIZE<0.

5. Prohibit database writes if any query parameters have been decoded but the request is not from the same origin. This effort would benefit from an SQLite enhancement that allows "PRAGMA query_only" to be applied to individual database files, so that the repository can be made read-only while still allowing TEMP writes, as TEMP writes are used to compose intermediate results even on pages that are technically read-only.

6. Implement a "fossil backup" command using VACUUM INTO. Implemented in 2.12. Docs.

7. Multiple remote repositories. Running "fossil push" pushes to them all, as does autosync. Partially implemented as of 2020-08-12: Multiple remotes remembered but can only sync to one at a time. Docs

8. Sync relay → On a server, when another repository pushes to the server (or edits a wiki page on the server) the server automatically schedules a push to peer repos. Should be able to do this with hooks, but more testing is needed. Also, need a way to configure relay hooks in the Admin web interface. Consider also providing the ability to do an automatic GitHub relay via the same mechanism.

9. Backoffice daemon → Instead of backoffice running in response to a web request, have a separate process that monitors multiple repositories and runs backoffice after "mtime" changes on the repository file, or periodically (every hour? every day?) in the absence of "mtime" changes. Implemented in 2.12

10. Update preview using XMLHttpRequest instead of reloading the entire page. Implemented in /wikiedit and /fileedit as of 2.12 and /forum/... is pending.

11. Allow help text to be in markup, either Fossil-Wiki or Markdown. Implemented in 2.12

12. Search on help-text and/or on unversioned files

    • The "helptext" virtual table added by check-in b2dacfcd735d4b1c is a step toward providing search on built-in help text, but has not yet been integrated into the search subsystem.
    • There are so many configuration pages in the web interface now that it can be difficult to find the right page to change a setting. One possible solution: Enhance the help text on all of the various setup web pages, and then add a search box at the top of the main /setup page. For maximum effectiveness, it might be necessary to add a new "Keywords" section to help pages that is not normally displayed but which is used for search.
    • It would also be good to add a search box at the top of the /help webpage, perhaps

13. Documentation on sync-via-sneaker-net.

14. Macros or other mechanisms for embedding a last-update timestamp in the middle of text for wiki pages and/or embedded documentation.

15. Add a command-line variant of the /secaudit0 page and make that command accessible using "fossil all".

16. Improved transaction control:

    • Better detection of potential SQLITE_BUSY errors when promoting from a read to a write txn. This will require SQLite enhancements.
    • On /xfer, only start a write transaction if the login has write permissions, thus allowing parallel clones.

17. On the wiki page list, omit wiki pages that are associated with check-ins and branches by default, but provide a button to show associated wiki pages if desired.

18. Add the ability to associate a forum thread with a check-in or branch.

    • Perhaps the linkage is based on the forum thread title, as is done for wiki pages. But a fast lookup mechanism will need to be devised, as forum thread titles are not currently stored in the TAG table as are wiki page names.
    • Perhaps also provide forum-like threading to tickets. Maybe merge the functionality of forum-post artifacts and ticket-change artifacts to allow both features within the same artifact.
    • Consider mechanisms to identifying check-ins or branches that include forum discussion when those check-ins/branches are displayed on the timeline, or on other pages.

19. Provide SSL capabilities for the "fossil server" and "fossil http" commands.

    • Because the website is not served from individual files on disk, standard tools for obtaining a LetsEncrypt cert won't work. Some sort of mechanism to do this will need to be built into Fossil. Or, a minimum, a mechanism should be in place to redirect requests to ".well-known" to files on disk.

20. Provide a setting that determines whether HTML content files are displayed as HTML or as plain text when browsing repository files. See the forum thread: https://www.fossil-scm.org/forum/forumpost/cc9d20228d

21. Client/Server mode or Shallow Clones. Allow a remote repository to be opened without having to clone all history.

22. Consider adding support for interwiki link syntax. Implemented by f4dc114a780fea41

23. When entering a check-in comment using $EDITOR, there is no way to preview the comment. This is particular frustrating when there are hyperlinks or Wiki escape codes (like "<" or "["). Errors result. For example on check-in 5244a5484a103065 the comment was originally entered using a Markdown-style hyperlink. Only after the commit completed was the error seen, and the check-in comment was fixed with a tag.

24. More ability to customize the /sitemap page. Docs

25. More "diff" links associated with Wiki.

    • With each wiki edit entry of the timeline.
    • On the submenu for Wiki display
    • On the wiki history display, provide more than current single-change diff. (Maybe the /whistory needs to be shown as a timeline graph rather than a simple list, so we can click on two nodes to get a diff.)
    • Diff links on editted Forum posts.
    • Semi-related: loading of additional context for /wikiedit diffs, analog to the context loading in the /vinfo (and similar) pages. This requires adding some medadata to the diff output for those diffs.

26. Add the ability to provide change comments on Wiki-Page edits. The existing artifact format already supports this, but the code does not provide the user with an option to enter a change comment with a wiki edit, and any change comment that is entered is silently ignored, rather than being displayed in the timeline or on the /whistory page.

27. Timeline graph improvement opportunities:

    • /timeline?p=3814c9ff58e2f4ae&y=ci&n=20
    • /timeline?b=20200608&n=22
    • /timeline?p=2c9aa1b9391fa8c5&n=30
    • /timeline?b=20200513&n=25
    • /timeline?p=11d1233d305c0fd6&n=23
    • https://sqlite.org/src/timeline?d=572f1ed59d29e74f&n=20

28. New manifest setting options.

    • manifest.h → a C/C++ header containing macros like FOSSIL_MANIFEST_UUID and FOSSIL_MANIFEST_DATE. Programs can #include this header to gain easy access to version information.
    • How long after the previous will it be before there are requests for manifest.json and manifest.tcl and manifest.py? Where do we draw the line?
    • Maybe instead of the previous two, we just add manifest.date. That in combination with manifest.uuid provides most of the versioning information that most programs will need.

29. For the purpose of regression testing when changing the markup language formatters, provide test commands that will scan an entire repository for Wiki or Markdown-formatted artifacts (embedded documentation, Wiki, Ticket comments, Forum posts) and run them through the formatter. Then, after making changes to formatters, we can run this command on various large repos both in the old and new version and look for unexpected differences. We could also maybe run this test prior to each release.

30. New email notifications for administrators:

    • Alerts to any configuration change.
    • Periodic security audit reports. (Dependency of ToDo #15.)

31. The /timeline page has many options, only a few of which are selectable from the submenu bar. Perhaps the "Advanced" submenu option should expand to a much larger "submenu" (using client-side javascript) that includes options to:

    • Specify a range of check-ins
    • Select forks
    • Select name changes
    • View timelines related to a branch
    • Show only timewarps
    • Show a path between two check-ins.

    An alternative to this idea is to have a submenu off of /sitemap that provides links to many of the specialized timelines. See item 24 above.

32. The passwords stored on behalf of fossil remote are obfuscated, but are still accessible to an attacker who gains unrestricted access to a local repository clone. Perhaps it would be better to store a security token (a 64-digit random hex value). This security token could only be used to sync, not to login. If the local repository is compromised, the attacker could push content, but could not perform administrative actions. And they wouldn't learn the password which might be shared by other repositories and/or services.

    The sync protocol might be enhanced so that after a successful login using the password, over a TLS link, the server includes a pragma in the reply that passes the security token to the client with the instruction to use that token for all subsequent logins. In this way, the change is completely transparent to the user and the user never has to even know that the security token exists.

33. Add the ability to import 3rd-party skins and include them in the /skins selection list. The ability to edit such skins would be a big plus, e.g. to include any site-specific JS. Perhaps skins could be provided as "plain" format (the same file structure used by the existing skins), or perhaps in a format suitable for (fossil config import) (see tools/skintxt2config.c), or perhaps as sqlar and/or zip files. We'd need to be able to export skins as well. Motivating use case: it would be really nice to be able to host multiple skins generated by Inskinerator.

34. The "fossil patch create" command might include deltas against private artifacts. This needs to be fixed. Perhaps "fossil patch create" could be enhanced with a "--from VERSION" option that created a patch with a specific baseline. Perhaps also a "--branch BRANCHNAME" that creates a patch for the (presumably private) branch call BRANCHNAME.

35. Add a password reset mechanism. Turned off by default - enabled by a setting and configurable under the Setup/Access control panel. There should be a warning that it is turned on in the security scan. All password resets should be logged. Password resets should be disabled for any user with Admin, Setup, or UV-Push privileges. Perhaps password-reset should be a three-level setting: (1) Off (the default). (2) On. (3) Message sent to moderators who much approve the reset before it is accomplished.