<title>Technical Overview</title>
<h2 align="center">
A Technical Overview<br>Of The Design And Implementation<br>Of Fossil
</h2>

<h2>1.0 Introduction</h2>

At its lowest level, a Fossil repository consists of an unordered set
of immutable "artifacts".  You might think of these artifacts as "files",
since in many cases the artifacts exactly correspond to source code files
that are stored in the Fossil repository.  But other "control artifacts" 
are also included in the mix.  These control artifacts define the relationships
between artifacts - which files go together to form a particular
version of the project, who checked in that version and when, what was
the check-in comment, what wiki pages are included with the project, what
are the edit histories of each wiki page, what bug reports or tickets are
included, who contributed to the evolution of each ticket, and so forth,
and so on.  This low-level file format is called the "global state" of
the repository, since this is the information that is synced to peer
repositories using push and pull operations.   The low-level file format
is also called "enduring" since it is intended to last for many years.
The details of the low-level, enduring, global file format 
are [./fileformat.wiki | described separately].

This article is about how Fossil is currently implemented.  Instead of
dealing with vague abstractions of "enduring file formats" as the
[./fileformat.wiki | that other document] does, this article provides
some detail on how Fossil actually stores information on disk.  

<h2>2.0 Three Databases</h2>

Fossil stores state information in 
[http://www.sqlite.org/ | SQLite] database files.
SQLite keeps an entire relational database, including multiple tables and
indices, in a single disk file.  The SQLite library allows the database
files to be efficiently queried and updated using the industry-standard
SQL language.  And SQLite makes updates to these database files atomic,
even if a system crashes or power failure occurs in the middle of the
update, meaning that repository content is protected even during severe
malfunctions.

Fossil uses three separate classes of SQLite databases:

<ol>
<li>The configuration database
<li>Repository databases
<li>Checkout databases

of the artifacts that comprise the 
[./fileformat.wiki | enduring, global, shared state] of the project.
The artifacts are stored as BLOBs, compressed using
[http://www.zlib.net/ | zlib compression] and, where applicable,
using [./delta_encoder_algorithm.wiki | delta compression].
The combination of zlib and delta compression results in a considerable
space savings.  For the SQLite project, at the time of this writing,
the total size of all artifacts is over 1.7 GB but thanks to the
combined zlib and delta compression, that content only takes up
51.4 MB of space in the repository database, for a compression ratio
of about 33:1.


Note that the zlib and delta compression is not an inherent part of the
Fossil file format; it is just an optimization.  
The enduring file format for Fossil is the unordered
set of artifacts. The compression techniques are just a detail of
how the current implementation of Fossil happens to store these artifacts
efficiently on disk.

works by reading the input git-fast-export stream and using it to construct
corresponding artifacts which are then written into the repository database.

<h4>2.2.2 Project Metadata</h4>

The global project state information in the repository database is
supplemented by computed metadata that makes querying the project state
more efficient.  Metadata includes but information such as the following:

  *  The names for all files found in any checkin.
  *  All check-ins that modify a given file
  *  Parents and children of each checkin.
  *  Potential timeline rows.
  *  The names of all symbolic tags and the checkins they apply to.
  *  The names of all wiki pages and the artifacts that comprise each
     wiki page.
  *  Attachments and the wiki pages or tickets they apply to.
  *  Current content of each ticket.
  *  Cross-references between tickets, checkins, and wiki pages.

The metadata is held in various SQL tables in the repository database.
The metadata is designed to facilitate queries for the various timelines and
reports that Fossil generates.  
As the functionality of Fossil evolves,
the schema for the metadata can and does change from time to time.
But schema changes do no invalidate the repository.  Remember that the
metadata contains no new information - only information that has been
extracted from the canonical artifacts and saved in a more useful form.
Hence, when the metadata schema changes, the prior metadata can be discarded
and the entire metadata corpus can be recomputed from the canonical
artifacts.  That is what the
[/help/rebuild | fossil rebuild] command does.

privileges on the remote repository.

<h4>2.2.5 Shunned Artifact List</h4>

The set of canonical artifacts for a project - the global state for the
project - is intended to be an append-only database.  In other words,
new artifacts can be added but artifacts can never be removed.  But
it sometimes happens that inappropriate content can be mistakenly or
maliciously added to a repository.  When that happens, the only way
to get rid of the content is to [./shunning.wiki | "shun"] it.
The "shun" table in the repository database records the SHA1 hash of 
all shunned artifacts.

The shun table can be pushed or pulled using
the [/help/config | fossil config] command with the "shun" AREA argument.
The shun table is also copied during a [/help/clone | clone].