SQLite Archiver (Windows)
Check-in [f563beec00]
Not logged in

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Rename the program from "SAR" to "SFA" to avoid a name collision with the System Activity Report (sar) utility from Solaris.
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1:f563beec009360d32a9e18685f60908f387528b7
User & Date: drh 2014-05-15 11:52:23
Context
2014-05-15 11:55
Change the title on the README.md page. check-in: 3d0e7db437 user: drh tags: trunk
2014-05-15 11:52
Rename the program from "SAR" to "SFA" to avoid a name collision with the System Activity Report (sar) utility from Solaris. check-in: f563beec00 user: drh tags: trunk
2014-04-07 13:49
Fix typos in the documentation. check-in: d6387bdf60 user: drh tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to Makefile.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
#!/bin/make

CC = gcc -g -I.
ZLIB = -lz
SQLITE_OPT = -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION

all: sar
	

sar:	sar.c sqlite3.o
	$(CC) -o sar sar.c sqlite3.o $(ZLIB)

sqlite3.o:	sqlite3.c sqlite3.h
	$(CC) -c sqlite3.c $(SQLITE_OPT) sqlite3.c






|


|
|



1
2
3
4
5
6
7
8
9
10
11
12
13
14
#!/bin/make

CC = gcc -g -I.
ZLIB = -lz
SQLITE_OPT = -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION

all: sfa
	

sfa:	sfa.c sqlite3.o
	$(CC) -o sfa sfa.c sqlite3.o $(ZLIB)

sqlite3.o:	sqlite3.c sqlite3.h
	$(CC) -c sqlite3.c $(SQLITE_OPT) sqlite3.c

Changes to README.md.

1
2
3
4
5
6
7
8
9
10
11
..
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
<h1 align="center">SAR - SQLite Archiver</h1>

This repository contains sources for a proof-of-concept "SQLite Archiver"
program.  This program (named "sar") operates much like "zip", except that
the compressed archive it builds is stored in an SQLite database instead
of a ZIP archive.

The motivation for this is to see how much larger an SQLite database file
is compared to a ZIP archive containing the same content.  The answer depends
on the filenames, but 2% seems to be a reasonable guess.  In other words,
storing files as compressed blobs in an SQLite database file results in a 
................................................................................
On unix, just type "make".  The SQLite sources are included.  The zlib
compression library is needed to build.

## Usage

To create an archive:

        sar ARCHIVE FILES...

All files named in FILES... will be added to the archive.  If another file
with the same name already exists in the archive, it is replaced.  If any
of the named FILES is a directory, that directory is scanned recursively.

To see the contents of an archive:

        sar -l ARCHIVE

To extract the contents of an archive:

        sar -x ARCHIVE [FILES...]

If a FILES argument is provided, then only the named files are extracted.
Without a FILES argument, all files are extracted.

All commands can be supplemented with -v for verbose output. For example:

        sar -v ARCHIVE FILES..
        sar -lv ARCHIVE
        sar -xv ARCHIVE

File are normally compressed using zlib prior to being stored as BLOBs in
the database.  However, if the file is incompressible or if the -n option
is used on the command-line, then the file is stored in the database exactly
as it appears on disk, without compression.
    
## Storage

The database schema looks like this:

        CREATE TABLE sar(
          name TEXT PRIMARY KEY,  -- name of the file
          mode INT,               -- access permissions
          mtime INT,              -- last modification time
          sz INT,                 -- original file size
          data BLOB               -- compressed content
        );
        
Both directories and empty files have sar.sz==0.  Directories can be
distinguished from empty files because directories have sar.data IS NULL.
The file is compressed if length(sar.blob)<sar.sz and is stored
as plaintext if length(sar.blob)==sar.sz.


|
|







 







|







|



|






|
|
|










|







|
|
|
|
1
2
3
4
5
6
7
8
9
10
11
..
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
<h1 align="center">SAR - SQLite Archiver</h1>

This repository contains sources for a proof-of-concept "SQLite File Archiver"
program.  This program (named "sfa") operates much like "zip", except that
the compressed archive it builds is stored in an SQLite database instead
of a ZIP archive.

The motivation for this is to see how much larger an SQLite database file
is compared to a ZIP archive containing the same content.  The answer depends
on the filenames, but 2% seems to be a reasonable guess.  In other words,
storing files as compressed blobs in an SQLite database file results in a 
................................................................................
On unix, just type "make".  The SQLite sources are included.  The zlib
compression library is needed to build.

## Usage

To create an archive:

        sfa ARCHIVE FILES...

All files named in FILES... will be added to the archive.  If another file
with the same name already exists in the archive, it is replaced.  If any
of the named FILES is a directory, that directory is scanned recursively.

To see the contents of an archive:

        sfa -l ARCHIVE

To extract the contents of an archive:

        sfa -x ARCHIVE [FILES...]

If a FILES argument is provided, then only the named files are extracted.
Without a FILES argument, all files are extracted.

All commands can be supplemented with -v for verbose output. For example:

        sfa -v ARCHIVE FILES..
        sfa -lv ARCHIVE
        sfa -xv ARCHIVE

File are normally compressed using zlib prior to being stored as BLOBs in
the database.  However, if the file is incompressible or if the -n option
is used on the command-line, then the file is stored in the database exactly
as it appears on disk, without compression.
    
## Storage

The database schema looks like this:

        CREATE TABLE sfa(
          name TEXT PRIMARY KEY,  -- name of the file
          mode INT,               -- access permissions
          mtime INT,              -- last modification time
          sz INT,                 -- original file size
          data BLOB               -- compressed content
        );
        
Both directories and empty files have sfa.sz==0.  Directories can be
distinguished from empty files because directories have sfa.data IS NULL.
The file is compressed if length(sfa.blob)<sfa.sz and is stored
as plaintext if length(sfa.blob)==sfa.sz.

Name change from sar.c to sfa.c.

36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
...
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
...
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
...
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
...
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
  exit(1);
}

/*
** The database schema:
*/
static const char zSchema[] = 
  "CREATE TABLE IF NOT EXISTS sar(\n"
  "  name TEXT PRIMARY KEY,\n"
  "  mode INT,\n"
  "  mtime INT,\n"
  "  sz INT,\n"
  "  data BLOB\n"
  ");"
;
................................................................................
  check_filename(zFilename);
  rc = stat(zFilename, &x);
  if( rc ) errorMsg("no such file or directory: %s\n", zFilename);
  if( x.st_size>1000000000 ){
    errorMsg("file too big: %s\n", zFilename);
  }
  if( pStmt==0 ){
    db_prepare("REPLACE INTO sar(name,mode,mtime,sz,data)"
               " VALUES(?1,?2,?3,?4,?5)");
  }
  zName = zFilename;
  while( zName[0]=='/' ) zName++;
  sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC);
  sqlite3_bind_int(pStmt, 2, x.st_mode);
  sqlite3_bind_int64(pStmt, 3, x.st_mtime);
................................................................................
  int nFiles = 0;
  int listFlag = 0;
  int extractFlag = 0;
  int verboseFlag = 0;
  int noCompress = 0;
  int i, j;

  if( sqlite3_strglob("*/unsar", argv[0])==0 ){
    extractFlag = 1;
  }
  for(i=1; i<argc; i++){
    if( argv[i][0]=='-' ){
      for(j=1; argv[i][j]; j++){
        switch( argv[i][j] ){
          case 'l':   listFlag = 1;    break;
................................................................................
  if( zArchive==0 ) showHelp(argv[0]);
  if( listFlag ){
    int rc;
    db_open(zArchive, 0);
    if( verboseFlag ){
      db_prepare(
          "SELECT name, sz, length(data), mode, datetime(mtime,'unixepoch')"
          " FROM sar ORDER BY name"
      );
      while( sqlite3_step(pStmt)==SQLITE_ROW ){
        printf("%10d %10d %03o %s %s\n", 
               sqlite3_column_int(pStmt, 1),
               sqlite3_column_int(pStmt, 2),
               sqlite3_column_int(pStmt, 3)&0777,
               sqlite3_column_text(pStmt, 4),
               sqlite3_column_text(pStmt, 0));
      }
    }else{
      db_prepare(
          "SELECT name FROM sar ORDER BY name"
      );
      while( sqlite3_step(pStmt)==SQLITE_ROW ){
        printf("%s\n", sqlite3_column_text(pStmt,0));
      }
    }
    db_close(1);
  }else if( extractFlag ){
................................................................................
    db_open(zArchive, 0);
    if( nFiles ){
      NameList x;
      x.azName = azFiles;
      x.nName = nFiles;
      sqlite3_create_function(db, "name_on_list", 1, SQLITE_UTF8,
                              (char*)&x, name_on_list, 0, 0);
      zSql = "SELECT name, mode, mtime, sz, data FROM sar"
             " WHERE name_on_list(filename)";
    }else{
      zSql = "SELECT name, mode, mtime, sz, data FROM sar";
    }
    db_prepare(zSql);
    while( sqlite3_step(pStmt)==SQLITE_ROW ){
      const char *zFN = (const char*)sqlite3_column_text(pStmt, 0);
      check_filename(zFN);
      if( zFN[0]=='/' ){
        errorMsg("absolute pathname: %s\n", zFN);







|







 







|







 







|







 







|











|







 







|


|







36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
...
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
...
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
...
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
...
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
  exit(1);
}

/*
** The database schema:
*/
static const char zSchema[] = 
  "CREATE TABLE IF NOT EXISTS sfa(\n"
  "  name TEXT PRIMARY KEY,\n"
  "  mode INT,\n"
  "  mtime INT,\n"
  "  sz INT,\n"
  "  data BLOB\n"
  ");"
;
................................................................................
  check_filename(zFilename);
  rc = stat(zFilename, &x);
  if( rc ) errorMsg("no such file or directory: %s\n", zFilename);
  if( x.st_size>1000000000 ){
    errorMsg("file too big: %s\n", zFilename);
  }
  if( pStmt==0 ){
    db_prepare("REPLACE INTO sfa(name,mode,mtime,sz,data)"
               " VALUES(?1,?2,?3,?4,?5)");
  }
  zName = zFilename;
  while( zName[0]=='/' ) zName++;
  sqlite3_bind_text(pStmt, 1, zName, -1, SQLITE_STATIC);
  sqlite3_bind_int(pStmt, 2, x.st_mode);
  sqlite3_bind_int64(pStmt, 3, x.st_mtime);
................................................................................
  int nFiles = 0;
  int listFlag = 0;
  int extractFlag = 0;
  int verboseFlag = 0;
  int noCompress = 0;
  int i, j;

  if( sqlite3_strglob("*/unsfa", argv[0])==0 ){
    extractFlag = 1;
  }
  for(i=1; i<argc; i++){
    if( argv[i][0]=='-' ){
      for(j=1; argv[i][j]; j++){
        switch( argv[i][j] ){
          case 'l':   listFlag = 1;    break;
................................................................................
  if( zArchive==0 ) showHelp(argv[0]);
  if( listFlag ){
    int rc;
    db_open(zArchive, 0);
    if( verboseFlag ){
      db_prepare(
          "SELECT name, sz, length(data), mode, datetime(mtime,'unixepoch')"
          " FROM sfa ORDER BY name"
      );
      while( sqlite3_step(pStmt)==SQLITE_ROW ){
        printf("%10d %10d %03o %s %s\n", 
               sqlite3_column_int(pStmt, 1),
               sqlite3_column_int(pStmt, 2),
               sqlite3_column_int(pStmt, 3)&0777,
               sqlite3_column_text(pStmt, 4),
               sqlite3_column_text(pStmt, 0));
      }
    }else{
      db_prepare(
          "SELECT name FROM sfa ORDER BY name"
      );
      while( sqlite3_step(pStmt)==SQLITE_ROW ){
        printf("%s\n", sqlite3_column_text(pStmt,0));
      }
    }
    db_close(1);
  }else if( extractFlag ){
................................................................................
    db_open(zArchive, 0);
    if( nFiles ){
      NameList x;
      x.azName = azFiles;
      x.nName = nFiles;
      sqlite3_create_function(db, "name_on_list", 1, SQLITE_UTF8,
                              (char*)&x, name_on_list, 0, 0);
      zSql = "SELECT name, mode, mtime, sz, data FROM sfa"
             " WHERE name_on_list(filename)";
    }else{
      zSql = "SELECT name, mode, mtime, sz, data FROM sfa";
    }
    db_prepare(zSql);
    while( sqlite3_step(pStmt)==SQLITE_ROW ){
      const char *zFN = (const char*)sqlite3_column_text(pStmt, 0);
      check_filename(zFN);
      if( zFN[0]=='/' ){
        errorMsg("absolute pathname: %s\n", zFN);