Fossil

Changes On Branch delta-format-pikchr
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Changes In Branch delta-format-pikchr Excluding Merge-Ins

This is equivalent to a diff from ec816c919e to b72cc79526

2020-09-30
00:08
Replaced old GIF diagrams from the delta_format.wiki doc with Pikchrs. Also removed delta?.gif from test/commit-warning.test. ... (check-in: d7c074e7fa user: wyoung tags: trunk)
00:05
Tweak to the "\n" text inside 2.1 Header of the new delta_format.wiki doc to make use of new Pikchr handling of backslashes. ... (Closed-Leaf check-in: b72cc79526 user: wyoung tags: delta-format-pikchr)
00:02
Merged in trunk ... (check-in: 45ff53dd6f user: wyoung tags: delta-format-pikchr)
2020-09-29
23:33
Removed unused www/CollRev?.gif and the references to it from test/commit-warning.test. Also removed obsolete references to already-removed www/branch??.gif. ... (check-in: ec816c919e user: wyoung tags: trunk)
11:44
Fix backslash processing in pikchr. ... (check-in: 7a9bf394b1 user: drh tags: trunk)

Deleted www/delta1.gif.

cannot compute difference between binary files

Deleted www/delta2.gif.

cannot compute difference between binary files

Deleted www/delta3.gif.

cannot compute difference between binary files

Deleted www/delta4.gif.

cannot compute difference between binary files

Deleted www/delta5.gif.

cannot compute difference between binary files

Deleted www/delta6.gif.

cannot compute difference between binary files

Changes to www/delta_format.wiki.
59
60
61
62
63
64
65

66




67
68
69
70
71
72
73
74
75

76



77
78
79
80
81
82
83
84
85
86
87

88



89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104

105




















106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121






122
123
124
125
126
127
128
129
130
131
132







133

134
135
136
137
138
139
140

   *   <b>delta_parse(</b><i>D</i>)</b> &rarr; This is a table-valued function
       that returns one row for the header, for the trailer, and for each segment
       in delta D.


<a name="structure"></a><h1>2.0 Structure</h1>

<img src="delta1.gif" align="left" hspace="10">





<p>A delta consists of three parts, a "header", a "trailer", and a
"segment-list" between them.</p>

<p>Both header and trailer provide information about the target
helping the decoder, and the segment-list describes how the target can
be constructed from the original.</p>

<a name="header"></a><h2>2.1 Header</h2>

<img src="delta6.gif" align="left" hspace="10">




<p>The header consists of a single number followed by a newline
character (ASCII 0x0a). The number is the length of the target in
bytes.</p>

<p>This means that, given a delta, the decoder can compute the size of
the target (and allocate any necessary memory based on that) by simply
reading the first line of the delta and decoding the number found
there. In other words, before it has to decode everything else.</p>

<a name="trailer"></a><h2>2.2 Trailer</h2>

<img src="delta5.gif" align="left" hspace="10">




<p>The trailer consists of a single number followed by a semicolon (ASCII
0x3b). This number is a checksum of the target and can be used by a
decoder to verify that the delta applied correctly, reconstructing the
target from the original.</p>

<p>The checksum is computed by treating the target as a series of
32-bit integer numbers (MSB first), and summing these up, modulo
2^32-1. A target whose length is not a multiple of 4 is padded with
0-bytes (ASCII 0x00) at the end.</p>

<p>By putting this information at the end of the delta a decoder has
it available immediately after the target has been reconstructed
fully.</p>

<a name="slist"></a><h2>2.3 Segment-List</h2>

<img src="delta2.gif" align="left" hspace="10">





















<p>The segment-list of a delta describes how to create the target from
the original by a combination of inserting literal byte-sequences and
copying ranges of bytes from the original. This is where the
compression takes place, by encoding the large common parts of
original and target in small copy instructions.</p>

<p>The target is constructed from beginning to end, with the data
generated by each instruction appended after the data of all previous
instructions, with no gaps.</p>

<a name="insertlit"></a><h3>2.3.1 Insert Literal</h3>

<p>A literal is specified by two elements, the size of the literal in
bytes, and the bytes of the literal itself.</p>







<img src="delta4.gif" align="left" hspace="10">
<p>The length is written first, followed by a colon character (ASCII
0x3a), followed by the bytes of the literal.</p>

<a name="copyrange"></a><h3>2.3.2 Copy Range</h3>

<p>A range to copy is specified by two numbers, the offset of the
first byte in the original to copy, and the size of the range, in
bytes. The size zero is special, its usage indicates that the range
extends to the end of the original.</p>








<img src="delta3.gif" align="left" hspace="10">

<p>The length is written first, followed by an "at" character (ASCII
0x40), then the offset, followed by a comma (ASCII 0x2c).</p>

<a name="intcoding"></a><h1>3.0 Encoding of integers</h1>

<p>
The format currently handles only 32 bit integer numbers. They are







>
|
>
>
>
>









>
|
>
>
>











>
|
>
>
>
















>
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
















>
>
>
>
>
>
|










>
>
>
>
>
>
>
|
>







59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188

   *   <b>delta_parse(</b><i>D</i>)</b> &rarr; This is a table-valued function
       that returns one row for the header, for the trailer, and for each segment
       in delta D.


<a name="structure"></a><h1>2.0 Structure</h1>
<verbatim type="pikchr">
    leftmargin = 0.1
    box height 50% "Header"
    box same "Segments"
    box same "Trailer"
</verbatim>

<p>A delta consists of three parts, a "header", a "trailer", and a
"segment-list" between them.</p>

<p>Both header and trailer provide information about the target
helping the decoder, and the segment-list describes how the target can
be constructed from the original.</p>

<a name="header"></a><h2>2.1 Header</h2>
<verbatim type="pikchr">
    leftmargin = 0.1
    box height 50% "Size"
    box same "\"\\n\""
</verbatim>

<p>The header consists of a single number followed by a newline
character (ASCII 0x0a). The number is the length of the target in
bytes.</p>

<p>This means that, given a delta, the decoder can compute the size of
the target (and allocate any necessary memory based on that) by simply
reading the first line of the delta and decoding the number found
there. In other words, before it has to decode everything else.</p>

<a name="trailer"></a><h2>2.2 Trailer</h2>
<verbatim type="pikchr">
    leftmargin = 0.1
    box height 50% "Checksum"
    box same "\":\""
</verbatim>

<p>The trailer consists of a single number followed by a semicolon (ASCII
0x3b). This number is a checksum of the target and can be used by a
decoder to verify that the delta applied correctly, reconstructing the
target from the original.</p>

<p>The checksum is computed by treating the target as a series of
32-bit integer numbers (MSB first), and summing these up, modulo
2^32-1. A target whose length is not a multiple of 4 is padded with
0-bytes (ASCII 0x00) at the end.</p>

<p>By putting this information at the end of the delta a decoder has
it available immediately after the target has been reconstructed
fully.</p>

<a name="slist"></a><h2>2.3 Segment-List</h2>
<verbatim type="pikchr">
    leftmargin = 0.1
    PART1: [
        B1: box height 50% width 15% ""
        B2: box same ""
        B3: box same ""
            move 0.1 ; box same invisible "***" ; move 0.1
            box height 50% width 15% ""
        I1: line down 50% from B2 .s
            arrow right until even with B3.e
            box "Insert Literal" height 50%
            line down 75% from I1 .s
            arrow right until even with B3.e
            box "Copy Range" height 50%
    ]
    down
    box invisible "" height 0.1
    box "Length" height 50% with .nw at (PART1.w, previous.s)
    right
    box "\":\"" same
    box "Bytes" same
</verbatim>

<p>The segment-list of a delta describes how to create the target from
the original by a combination of inserting literal byte-sequences and
copying ranges of bytes from the original. This is where the
compression takes place, by encoding the large common parts of
original and target in small copy instructions.</p>

<p>The target is constructed from beginning to end, with the data
generated by each instruction appended after the data of all previous
instructions, with no gaps.</p>

<a name="insertlit"></a><h3>2.3.1 Insert Literal</h3>

<p>A literal is specified by two elements, the size of the literal in
bytes, and the bytes of the literal itself.</p>

<verbatim type="pikchr">
    leftmargin = 0.1
    box "Length" height 50%
    box "\":\"" same
    box "Bytes" same
</verbatim>

<p>The length is written first, followed by a colon character (ASCII
0x3a), followed by the bytes of the literal.</p>

<a name="copyrange"></a><h3>2.3.2 Copy Range</h3>

<p>A range to copy is specified by two numbers, the offset of the
first byte in the original to copy, and the size of the range, in
bytes. The size zero is special, its usage indicates that the range
extends to the end of the original.</p>

<verbatim type="pikchr">
    leftmargin = 0.1
    box "Length" height 50%
    box "\"@\"" same
    box "Offset" same
    box "\",\"" same
</verbatim>


<p>The length is written first, followed by an "at" character (ASCII
0x40), then the offset, followed by a comma (ASCII 0x2c).</p>

<a name="intcoding"></a><h1>3.0 Encoding of integers</h1>

<p>
The format currently handles only 32 bit integer numbers. They are
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
4E@0,2:thFN@4C,6:scenda1B@Jd,6:scenda5x@Kt,6:pieces79@Qt,F: Example: eskil~E@Y0,2zMM3E;</pre>
</td></tr></table>

<p>This can be taken apart into the following parts:</p>

<table border=1>
<tr><th>What  </th> <th>Encoding         </th><th>Meaning </th><th>Details</th></tr>
<tr><td>Header</td> <td>1Xb              </td><td>Size    </td><td> 6246	     </td></tr>
<tr><td>S-List</td> <td>4E@0,	         </td><td>Copy    </td><td> 270 @ 0	     </td></tr>
<tr><td>&nbsp;</td> <td>2:th	         </td><td>Literal </td><td> 2 'th'	     </td></tr>
<tr><td>&nbsp;</td> <td>FN@4C,	         </td><td>Copy    </td><td> 983 @ 268	     </td></tr>
<tr><td>&nbsp;</td> <td>6:scenda         </td><td>Literal </td><td> 6 'scenda'	     </td></tr>
<tr><td>&nbsp;</td> <td>1B@Jd,	         </td><td>Copy    </td><td> 75 @ 1256	     </td></tr>
<tr><td>&nbsp;</td> <td>6:scenda         </td><td>Literal </td><td> 6 'scenda'	     </td></tr>
<tr><td>&nbsp;</td> <td>5x@Kt,	         </td><td>Copy    </td><td> 380 @ 1336	     </td></tr>
<tr><td>&nbsp;</td> <td>6:pieces	 </td><td>Literal </td><td> 6 'pieces'	     </td></tr>
<tr><td>&nbsp;</td> <td>79@Qt,	         </td><td>Copy    </td><td> 457 @ 1720     </td></tr>
<tr><td>&nbsp;</td> <td>F: Example: eskil</td><td>Literal </td><td> 15 ' Example: eskil'</td></tr>
<tr><td>&nbsp;</td> <td>~E@Y0,           </td><td>Copy    </td><td>  4046 @ 2176        </td></tr>
<tr><td>Trailer</td><td>2zMM3E           </td><td>Checksum</td><td> -1101438770         </td></tr>
</table>

<p>The unified diff behind the above delta is</p>








|
|
|
|
|
|
|
|
|
|







227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
4E@0,2:thFN@4C,6:scenda1B@Jd,6:scenda5x@Kt,6:pieces79@Qt,F: Example: eskil~E@Y0,2zMM3E;</pre>
</td></tr></table>

<p>This can be taken apart into the following parts:</p>

<table border=1>
<tr><th>What  </th> <th>Encoding         </th><th>Meaning </th><th>Details</th></tr>
<tr><td>Header</td> <td>1Xb              </td><td>Size    </td><td> 6246         </td></tr>
<tr><td>S-List</td> <td>4E@0,            </td><td>Copy    </td><td> 270 @ 0      </td></tr>
<tr><td>&nbsp;</td> <td>2:th             </td><td>Literal </td><td> 2 'th'       </td></tr>
<tr><td>&nbsp;</td> <td>FN@4C,           </td><td>Copy    </td><td> 983 @ 268        </td></tr>
<tr><td>&nbsp;</td> <td>6:scenda         </td><td>Literal </td><td> 6 'scenda'       </td></tr>
<tr><td>&nbsp;</td> <td>1B@Jd,           </td><td>Copy    </td><td> 75 @ 1256        </td></tr>
<tr><td>&nbsp;</td> <td>6:scenda         </td><td>Literal </td><td> 6 'scenda'       </td></tr>
<tr><td>&nbsp;</td> <td>5x@Kt,           </td><td>Copy    </td><td> 380 @ 1336       </td></tr>
<tr><td>&nbsp;</td> <td>6:pieces     </td><td>Literal </td><td> 6 'pieces'       </td></tr>
<tr><td>&nbsp;</td> <td>79@Qt,           </td><td>Copy    </td><td> 457 @ 1720     </td></tr>
<tr><td>&nbsp;</td> <td>F: Example: eskil</td><td>Literal </td><td> 15 ' Example: eskil'</td></tr>
<tr><td>&nbsp;</td> <td>~E@Y0,           </td><td>Copy    </td><td>  4046 @ 2176        </td></tr>
<tr><td>Trailer</td><td>2zMM3E           </td><td>Checksum</td><td> -1101438770         </td></tr>
</table>

<p>The unified diff behind the above delta is</p>