Fossil

# Is Fossil A Blockchain?

The Fossil version control system shares a lot of similarities with
blockchain based technologies, but it also differs from the more common
sorts of blockchains. This document will discuss the term’s
applicability, so you can decide whether applying the term to Fossil
makes sense to you.


## The Dictionary Argument

[Wikipedia defines "blockchain"][bcwp] in part as

>
  "…a growing list of records, called blocks, which are linked using
   cryptography. Each block contains a cryptographic hash of the previous
   block, a timestamp, and transaction data…"


By that partial definition, Fossil is indeed a blockchain.
The blocks are ["manifests" artifacts](./fileformat.wiki#manifest).
Each manifest has a SHA1 or SHA3 hash of its parent or parents,
a timestamp, and other transactional data.  The repository grows by
adding new manifests onto the list.

Nevertheless, there are many reasons to regard Fossil as *not* a
blockchain.

[bcwp]: https://en.wikipedia.org/wiki/Blockchain



## Cryptocurrency

Because blockchain technology was first popularized as Bitcoin, many
people associate the term with cryptocurrency.  Since Fossil has nothing
to do with cryptocurrency, someone using the term “blockchain” to refer
to Fossil is likely to fail to communicate their ideas clearly.

Cryptocurrency also has unfortunate implications in certain circles, its
anonymity and lack of regulation leading it to become associated with
bad actors. Even if we ignore all of the other criticisms in this
document, our unwillingness to be so associated may be enough of a
reason for us to avoid using it.



## Marketing Capture

The fact that blockchain technology has become a hot marketing buzzword
should affect your choice of whether to use the term “blockchain” to
refer to Fossil. Your choice may well vary based on the audience:

*   **Executive Board:** At the quarterly all-hands meeting, the big
    boss — who just read about blockchains in [PHB] Weekly — asks if
    your development organization “has a blockchain.” With Fossil and a
    suitably narrow definition of the term “blockchain” in mind, you
    could answer “Yes,” except that you know they’re then going to go to
    the shareholders and happily report, “Our development organization
    has been using blockchain technology for years!” You may decide that
    this makes you responsible for a public deception, putting the
    organization at risk of an SEC investigation for making false
    statements.

    Yet if you answer “No,” knowing you’ll be punished for not being on
    top of the latest whiz-bang as the technologically gormless PHB sees
    it, are you advancing the organization’s actual interests? If the
    organization has no actual need for a proper blockchain tech base,
    isn’t it better to just say “Yes” and point at Fossil so you can get
    back to useful work?

*   **Middle Management:** Your project leader asks the same question,
    so you point them at this document, which tells them the truth:
    kinda yes, but mostly no.

*   **Developer Lunch:** A peer asks if you’re doing anything with
    blockchains. Knowing the contents of this document, you decide you
    can’t justify using that term to refer to Fossil at a deep technical
    level, so you admit that you are not.

[PHB]:   https://en.wikipedia.org/wiki/Pointy-haired_Boss


## Distributed Ledgers

Cryptocurrencies are a type of [distributed ledger technology][dlt]. Is
Fossil a distributed ledger?

A key tenet of DLT is that records be unmodifiable after they’re
committed to the ledger, which matches quite well with Fossil’s design
and everyday use cases.

Yet, Fossil also has [purge] and [shunning][shun]. Doesn’t that mean
Fossil cannot be a distributed ledger?

What if you removed those features from Fossil, creating an append-only
variant? Is it a DLT then? Arguably still not, because [today’s Fossil
is an AP-mode system][fapm] in the [CAP theorem][cap] sense, which means
there can be no guaranteed consensus on the content of the ledger at any
given time. If you had an AP-mode accounts receivable system, it could
have different bottom-line totals at different sites, because you’ve
cast away “C” to get AP-mode operation.

What are the prospects for CA-mode or CP-mode Fossil? [We don’t want
CA-mode Fossil, but CP-mode could be useful.][fapm] Until the latter
exists, this author believes Fossil is not a distributed ledger in a
technologically defensible sense. If you restrict your definition’s
scope to cover only the most common uses of “blockchain,” which are all
DLTs, that means Fossil is not a blockchain.

[fapm]:   ./cap-theorem.md
[cap]:    https://en.wikipedia.org/wiki/CAP_theorem
[dlt]:    https://en.wikipedia.org/wiki/Distributed_ledger
[DVCS]:   https://en.wikipedia.org/wiki/Distributed_version_control
[purge]:  /help?cmd=purge
[shun]:   ./shunning.wiki


## Distributed Partial Consensus

If we can’t get DLT, can we at least get some kind of distributed
consensus at the level of individual Fossil’s commits?

Many blockchain based technologies have this property: given some
element of the blockchain, you can make certain proofs that it either is
a legitimate part of the whole blockchain, or it is not.

Unfortunately, this author doesn’t see a way to do that with Fossil.
Given only one “block” in Fossil’s putative “blockchain” — a commit, in
Fossil terminology — all you can prove is whether it is internally
consistent, not corrupt. That then points you at the parent(s) of that
commit, which you can repeat the exercise on, back to the root of the
DAG. This is what the enabled-by-default [`repo-cksum` setting][rcks]
does.

If cryptocurrencies worked this way, you wouldn’t be able to prove that
a given cryptocoin was legitimate without repeating the proof-of-work
calculations for the entire cryptocurrency scheme!

What would it even mean to prove that a given Fossil commit “*belongs*”
to the repository you’ve extracted it from? For a software project,
isn’t that tantamount to automatic code review, where the server would
be able to reliably accept or reject a commit based solely on its
content? That sounds nice, but this author believes we’ll need to invent
[AGI] first.

A better method to provide distributed consensus for Fossil would be to
rely on the *natural* intelligence of its users: that is, distributed
commit signing, so that a commit is accepted into the blockchain only
once some number of users countersign it. This amounts to a code review
feature, which Fossil doesn’t currently have.

Solving that problem basically requires solving the [PKI] problem first,
since you can’t verify the proofs of these signatures if you can’t first
prove that the provided signatures belong to people you trust. This is a
notoriously hard problem in its own right.

A future version of Fossil could instead provide consensus [in the CAP
sense][fapm]. For instance, you could say that if a quorum of servers
all have a given commit, it “belongs.” Fossil’s strong hashing tech
would mean that querying whether a given commit is part of the
“blockchain” would be as simple as going down the list of servers and
sending it an HTTP GET `/info` query for the artifact ID, returning
“Yes” once you get enough HTTP 200 status codes back. All of this is
hypothetical, because Fossil doesn’t do this today.

Even with all of the above solved, you’d still have another problem:
Fossil currently has no way to do partial cloning of a repository. The
only way to remotely extract individual “blocks” — commits — from a
remote repository is to make `/artifact`, `/info`, or `/raw` queries to
its HTTP interface. For Fossil to be a true blockchain, we’d want a way
to send around as little as one commit which could be individually
verified as being “part of the blockchain” using only intra-block
consistency checks.

[AGI]:  https://en.wikipedia.org/wiki/Artificial_general_intelligence
[PKI]:  https://en.wikipedia.org/wiki/Public_key_infrastructure
[rcks]: https://fossil-scm.org/home/help?cmd=repo-cksum


# Conclusion

This author believes it is technologically indefensible to call Fossil a
“blockchain” in any sense likely to be understood by a majority of those
you’re communicating with.

Within a certain narrow scope, you can defend this usage, but if you do
that, you’ve failed any goal that requires clear communication: it
doesn’t work to use a term in a nonstandard way just because you can
defend it.  The people you’re communicating your ideas to must have the
same concept of the terms you use.

What term should you use instead? A blockchain is a type of [Merkle
tree][mt], also called a hash tree, and Fossil is certainly that.

Fossil and “blockchain” are technological peers. They are related
technologies, but neither is a subset or instance of the other in any
useful sense.

[mt]: https://en.wikipedia.org/wiki/Merkle_tree

things that the Setup user has not changed from the stock configuration.
In this way, an Admin-only user can avoid overriding the Setup user's
choices.

You can also look at the role of Admin from the other direction, up
through the [user power hierarchy][ucap] rather than down from Setup. An
Admin user is usually a “super-developer” role, given full control over
the repository’s managed content: versioned artifacts in [the hash tree][bc],
[unversioned content][uv], forum posts, wiki articles,
tickets, etc.

We’ll explore these distinctions in the rest of this document.

[bc]:   ../blockchain.md
[ucap]: ./index.md#ucap
[uv]:   ../unvers.wiki

    Shunned page to Admin users rather than reserve it to Setup users
    because one of the primary purposes of [the Fossil shunning
    system][shun] is to clean up after a spammer, and that's
    exactly the sort of administrivia we wish to delegate to Admin users.

    Coupled with the Rebuild button on the same page, an Admin user has
    the power to delete the repository's entire
    [hash tree][bc]! This makes this feature a pretty good
    razor in deciding whether to grant someone Admin capability: do you
    trust that user to shun Fossil artifacts responsibly?

    Realize that shunning is cooperative in Fossil. As long as there are
    surviving repository clones, an Admin-only user who deletes the
    whole hash tree has merely caused a nuisance. An Admin-only user
    cannot permanently destroy the repository unless the Setup user has
    been so silly as to have no up-to-date clones.

*   **Moderation**: According to the power hierarchy laid out at the top
    of this article, Admins are greater than Moderators, so control over
    what Moderators can do clearly belongs to both Admins and to the
    Setup user(s).

        restriction.  (chroot, jails, SELinux, VMs, etc.) Since it makes
        no sense to trust Admin-only users with <tt>root</tt> level
        access on the host system, we almost certainly don't want to
        allow them to change such settings.</p>

*   **SQL**: The Admin → SQL feature allows the Setup user to enter raw
    SQL queries against the Fossil repository via Fossil UI. This not
    only allows arbitrary ability to modify the repository hash tree
    and its backing data tables, it can probably also be used to damage
    the host such as via `PRAGMA temp_store = FILE`.

*   **Tickets**: This section allows input of aribtrary TH1 code that
    runs on the server, affecting the way the Fossil ticketing system
    works. The justification in the **TH1** section below therefore
    applies.

    check-ins appear to go back in time and other bad effects.

3.  You can purposely overwrite good timestamps with bad ones and push
    those changes up to the remote with no interference, even though
    Fossil tries to make that a Setup-only operation.

All of this falls out of two of Fossil’s design choices: sync is
all-or-nothing, and [the Fossil hash tree][bc] is immutable. Fossil
would have to violate one or both of these principles to filter such
problems out of incoming syncs.

We have considered auto-[shunning][shun] “bad” content on sync, but this
is [difficult][asd] due to [the design of the sync protocol][dsp]. This
is not an impossible set of circumstances, but implementing a robust
filter on this input path would be roughly as difficult as writing a

  <tr id="d">
    <th>d</th>
    <th>n/a</th>
    <td>
      Legacy capability letter from Fossil's forebear <a
      href="http://cvstrac.org/">CVSTrac</a>, which has no useful
      meaning in Fossil due to the nature of its durable Merkle tree design. This
      letter was assigned by default to Developer in repos created with
      Fossil 2.10 or earlier, but it has no effect in current or past
      versions of Fossil; we recommend that you remove it in case we
      ever reuse this letter for another purpose. See <a
      href="https://fossil-scm.org/forum/forumpost/43c78f4bef">this
      post</a> for details.
    </td>

  #  [#timestamps | Timestamps], with preference to ISO8601 forms
  #  [#tagpfx | tag:TAGNAME]
  #  [#root | root:BRANCH]
  #  [#merge-in | merge-in:BRANCH]
  #  [#tag-ts | TAG:timestamp]
  #  Full artifact hash or hash prefix.
  #  Any other type of symbolic name that Fossil extracts from
     artifacts.

<div style="height:40em" id="this-space-intentionally-left-blank"></div>

[referer]: https://en.wikipedia.org/wiki/HTTP_referer
[csrf]: https://en.wikipedia.org/wiki/Cross-site_request_forgery
[xhr]: https://en.wikipedia.org/wiki/XMLHttpRequest

## `/fileedit` **Works by Creating Commits**

Thus any edits made via that page become a normal part of the
repository.

## `/fileedit` is *Intended* for use with Embedded Docs

... and similar text files, and is most certainly
**not intended for editing code**.

Editing files with unusual syntax requirements, e.g. hard tabs in

<h3 id="durable" name="database">2.3 Durable</h3>

The baseline data structures for Fossil and Git are the same, modulo
formatting details. Both systems manage a
[https://en.wikipedia.org/wiki/Directed_acyclic_graph | directed acyclic
graph] (DAG) of [https://en.wikipedia.org/wiki/Merkle_tree | Merkle
tree] structured check-in objects.
Check-ins are identified by a cryptographic hash of the check-in
contents, and each check-in refers to its parent via <i>its</i> hash.

The difference is that Git stores its objects as individual files in the
<tt>.git</tt> folder or compressed into bespoke
[https://git-scm.com/book/en/v2/Git-Internals-Packfiles|pack-files],
whereas Fossil stores its objects in a [https://www.sqlite.org/|SQLite]

The prime example in Git is rebasing: the change happens to the local
repository immediately if successful, even though you haven't tested the
change yet. It's possible to argue for such a design in a tool like Git
which doesn't automatically push the change up to its parent, because
you can still test the change before pushing local changes to the parent
repo, but in the meantime you've made a durable change to your local Git
repository. You must do something drastic like <tt>git
reset --hard</tt> to revert that rebase if it causes a problem. If you
push your rebased local repo up to the parent without testing first,
you've now committed the error on a public branch, effectively a
violation of
[https://www.atlassian.com/git/tutorials/merging-vs-rebasing#the-golden-rule-of-rebasing
| the golden rule of rebasing].

Lesser examples are the Git <tt>merge</tt>, <tt>cherry-pick</tt>, and
<tt>revert</tt> commands, all of which apply work from one branch onto
another, and all of which do their work immediately without giving you
an opportunity to test the change first locally unless you give the
<tt>--no-commit</tt> option.

Fossil cannot sensibly work that way because of its default-enabled
autosync feature. Instead of jumping straight to the commit step, Fossil
applies the proposed merge to the local working directory only,
requiring a separate check-in step before the change is committed to the
repository. This gives you a chance to test the change first,
either manually or by running your software's automatic tests. (Ideally,
both!)

Another difference is that because Fossil requires an explicit commit
for a merge, it makes you give an explicit commit <i>message</i> for
each merge, whereas Git writes that commit message itself by default
unless you give the optional <tt>--edit</tt> flag to override it.

Git only supports version control. The additional features of Fossil such
as Wiki, Tickets, Technotes, and the Forum are not supported in Git,
so those features are not included in an export.

Third-party Git based tooling may add some of these features (e.g.
GitHub, GitLab) but because their data are not stored in the Git
repository, there is no single destination for Fossil to convert its
equivalent data *to*. For instance, Fossil tickets do not become GitHub
issues, because that is a proprietary feature of GitHub separate from
Git proper, stored outside the repository on the GitHub servers.

You can also see the problem in its inverse case: you do not get a copy
of your GitHub issues when cloning the Git repository. You *do* get the
Fossil tickets, wiki, forum posts, etc. when cloning a remote Fossil
repo.

## (2) Cherrypick Merges

The Git client supports cherrypick merges but does not record the
cherrypick parent(s).

Fossil tracks cherrypick merges in its repository and displays
cherrypicks in its timeline. (As an example, the dashed lines
[here](/timeline?c=0a9f12ce6655b7a5) are cherrypicks.) Because Git does
not have a way to represent this same information in its repository, the
history of Fossil cherrypicks cannot be exported to Git, only their
direct effects on the managed file data.

## (3) Named Branches

Git has only limited support for named branches.  Git identifies the head
check-in of each branch.  Depending on the check-in graph topology, this

[ghrtv]: https://github.com/drhsqlite/fossil-mirror/tree/release

## (5) Amendments To Check-ins

Check-ins are immutable in both Fossil and Git.
However, Fossil has a mechanism by which tags can be added to
its repository to provide after-the-fact corrections to prior check-ins.

For example, tags can be added to check-ins that correct typos in the
check-in comment.  The original check-in is immutable and so the
original comment is preserved in addition to the correction. But
software that displays the check-ins knows to look for the comment-change
tag and if present displays the corrected comment rather than the original.
([Example](/info/8ed91bbe44d0d383) changing the typo "os" into "so".)

  adding_code.wiki {Adding New Features To Fossil}
  adding_code.wiki {Hacking Fossil}
  alerts.md {Email Alerts And Notifications}
  antibot.wiki {Defense against Spiders and Bots}
  backoffice.md {The "Backoffice" mechanism of Fossil}
  backup.md {Backing Up a Remote Fossil Repository}
  blame.wiki {The Annotate/Blame Algorithm Of Fossil}
  blockchain.md {Is Fossil A Blockchain?}
  branching.wiki {Branching, Forking, Merging, and Tagging}
  bugtheory.wiki {Bug Tracking In Fossil}
  build.wiki {Compiling and Installing Fossil}
  cap-theorem.md {Fossil and the CAP Theorem}
  caps/ {Administering User Capabilities}
  caps/admin-v-setup.md {Differences Between Setup and Admin Users}
  caps/ref.html {User Capability Reference}

<li><a href="password.wiki">Authentication &mdash; Password Management And</a></li>
<li><a href="backup.md"><b>Backing Up a Remote Fossil Repository</b></a></li>
<li><a href="backoffice.md">Backoffice mechanism of Fossil &mdash; The</a></li>
<li><a href="fossil_prompt.wiki">Bash Prompt &mdash; Fossilized</a></li>
<li><a href="whyusefossil.wiki"><b>Benefits Of Version Control</b></a></li>
<li><a href="caps/admin-v-setup.md">Between Setup and Admin Users &mdash; Differences</a></li>
<li><a href="hashpolicy.wiki">Between SHA1 and SHA3-256 &mdash; Hash Policy: Choosing</a></li>
<li><a href="blockchain.md">Blockchain? &mdash; Is Fossil A</a></li>
<li><a href="antibot.wiki">Bots &mdash; Defense against Spiders and</a></li>
<li><a href="private.wiki">Branches &mdash; Creating, Syncing, and Deleting Private</a></li>
<li><a href="branching.wiki"><b>Branching, Forking, Merging, and Tagging</b></a></li>
<li><a href="bugtheory.wiki"><b>Bug Tracking In Fossil</b></a></li>
<li><a href="makefile.wiki">Build Process &mdash; The Fossil</a></li>
<li><a href="cap-theorem.md">CAP Theorem &mdash; Fossil and the</a></li>
<li><a href="caps/">Capabilities &mdash; Administering User</a></li>

<li><a href="delta_format.wiki">Format &mdash; Fossil Delta</a></li>
<li><a href="fileformat.wiki">Format &mdash; Fossil File</a></li>
<li><a href="image-format-vs-repo-size.md">Format vs Fossil Repo Size &mdash; Image</a></li>
<li><a href="../../../md_rules">Formatting Rules &mdash; Markdown</a></li>
<li><a href="../../../wiki_rules">Formatting Rules &mdash; Wiki</a></li>
<li><a href="forum.wiki">Forums &mdash; Fossil</a></li>
<li><a href="cap-theorem.md"><b>Fossil and the CAP Theorem</b></a></li>

<li><a href="changes.wiki"><b>Fossil Changelog</b></a></li>
<li><a href="concepts.wiki"><b>Fossil Core Concepts</b></a></li>
<li><a href="css-tricks.md"><b>Fossil CSS Tips and Tricks</b></a></li>
<li><a href="delta_encoder_algorithm.wiki"><b>Fossil Delta Encoding Algorithm</b></a></li>
<li><a href="delta_format.wiki"><b>Fossil Delta Format</b></a></li>
<li><a href="hacker-howto.wiki"><b>Fossil Developers Guide</b></a></li>
<li><a href="fileformat.wiki"><b>Fossil File Format</b></a></li>

<li><a href="tech_overview.wiki">Implementation Of Fossil &mdash; A Technical Overview Of The Design And</a></li>
<li><a href="inout.wiki"><b>Import And Export To And From Git</b></a></li>
<li><a href="build.wiki">Installing Fossil &mdash; Compiling and</a></li>
<li><a href="fossil-from-msvc.wiki"><b>Integrating Fossil in the Microsoft Express 2010 IDE</b></a></li>
<li><a href="selfcheck.wiki">Integrity Self Checks &mdash; Fossil Repository</a></li>
<li><a href="webui.wiki">Interface &mdash; The Fossil Web</a></li>
<li><a href="interwiki.md"><b>Interwiki Links</b></a></li>
<li><a href="blockchain.md"><b>Is Fossil A Blockchain?</b></a></li>
<li><a href="javascript.md">JavaScript in Fossil &mdash; Use of</a></li>
<li><a href="th1.md">Language &mdash; The TH1 Scripting</a></li>
<li><a href="copyright-release.html">License Agreement &mdash; Contributor</a></li>
<li><a href="mirrorlimitations.md"><b>Limitations On Git Mirrors</b></a></li>
<li><a href="interwiki.md">Links &mdash; Interwiki</a></li>
<li><a href="../../../help"><b>Lists of Commands and Webpages</b></a></li>
<li><a href="password.wiki">Management And Authentication &mdash; Password</a></li>

Fossil to insert bad control artifacts. Therefore, before we get to
methods of permanently deleting content from a Fossil repos, let's give
some alternatives that usually suffice, which don't damage the project's
fossil record:

<ul>
    <li><p>When a forum post or wiki article is "deleted," what actually
    happens is that a new empty version is added to the Fossil repository.
    The web interface interprets this
    as "deleted," but the prior version remains available if you go
    digging for it.</p></li>

    <li><p>When you close a ticket, it's marked in a way that causes it
    to not show up in the normal ticket reports. You usually want to
    give it a Resolution such as "Rejected" when this happens, plus
    possibly a comment explaining why you're closing it. This is all new