Fossil

Check-in [f5f4471323]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Rework the "fossil grep" command so that it shows both the file and check-in hash for matching files, and so that it can scan multiple files all at once.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: f5f4471323d44a82a73a506f4ecd277f7ff8a937547d9bc2a023f07243b82d3b
User & Date: drh 2019-11-30 13:53:45
References
2022-04-11
09:33
Brought the "Fossil grep vs POSIX grep" doc up to date relative to the [/info/f5f4471323d44a82 | merged grep-enhancements branch]. ... (check-in: caba4b0188 user: wyoung tags: trunk)
Context
2019-12-02
13:45
Add the ability to have a C-card on a wiki page. The current implementation does not use or generate wiki page artifacts with a C-card. ... (check-in: cad57db96e user: drh tags: trunk)
2019-12-01
20:18
Special UI controls added onto the /whistory webpage for easy comparison of arbitrary versions of a wiki page. ... (check-in: ec04c8b4a8 user: george tags: wiki-history)
2019-11-30
13:53
Rework the "fossil grep" command so that it shows both the file and check-in hash for matching files, and so that it can scan multiple files all at once. ... (check-in: f5f4471323 user: drh tags: trunk)
13:38
Completely rework the "fossil grep" command. Omit the -H option. Instead, print a header line that includes both the file hash and the check-in hash and the timestamp for every file that contains any match. Scan all files together, in reverse chronological order. ... (Closed-Leaf check-in: 9c2080a360 user: drh tags: grep-enhancements)
2019-11-28
10:31
Changes to support CGI on IIS web servers. ... (check-in: c06e0b2d0a user: drh tags: trunk)
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to src/regexp.c.

717
718
719
720
721
722
723

724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744


745



746
747
748
749
750
751
752
  }
}

/*
** Flags for grep_buffer()
*/
#define GREP_EXISTS    0x001    /* If any match, print only the name and stop */


/*
** Run a "grep" over a text file
*/
static int grep_buffer(
  ReCompiled *pRe,
  const char *zName,
  const char *z,
  u32 flags
){
  int i, j, n, ln, cnt;
  for(i=j=ln=cnt=0; z[i]; i=j+1){
    for(j=i; z[j] && z[j]!='\n'; j++){}
    n = j - i;
    ln++;
    if( re_match(pRe, (const unsigned char*)(z+i), j-i) ){
      cnt++;
      if( flags & GREP_EXISTS ){
        fossil_print("%S\n", zName);
        break;
      }


      fossil_print("%S:%d:%.*s\n", zName, ln, n, z+i);



    }
  }
  return cnt;
}

/*
** COMMAND: test-grep







>


















|


>
>
|
>
>
>







717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
  }
}

/*
** Flags for grep_buffer()
*/
#define GREP_EXISTS    0x001    /* If any match, print only the name and stop */
#define GREP_QUIET     0x002    /* Return code only */

/*
** Run a "grep" over a text file
*/
static int grep_buffer(
  ReCompiled *pRe,
  const char *zName,
  const char *z,
  u32 flags
){
  int i, j, n, ln, cnt;
  for(i=j=ln=cnt=0; z[i]; i=j+1){
    for(j=i; z[j] && z[j]!='\n'; j++){}
    n = j - i;
    ln++;
    if( re_match(pRe, (const unsigned char*)(z+i), j-i) ){
      cnt++;
      if( flags & GREP_EXISTS ){
        if( (flags & GREP_QUIET)==0 && zName ) fossil_print("%s\n", zName);
        break;
      }
      if( (flags & GREP_QUIET)==0 ){
        if( cnt==1 && zName ){
          fossil_print("== %s\n", zName);
        }
        fossil_print("%d:%.*s\n", ln, n, z+i);
      }
    }
  }
  return cnt;
}

/*
** COMMAND: test-grep
785
786
787
788
789
790
791
792
793
794
795





796
797
798
799


800
801





802
803
804
805
806
807
808
809
810









811
812
813
814











815
816
817
818
819
820
821
822




823
824
825
826
827
828












829
830



831

832
833
834

835
836
837
838
839
840









841

842







843







844






845
846
847
  }
  re_free(pRe);
}

/*
** COMMAND: grep
**
** Usage: %fossil grep [OPTIONS] PATTERN FILENAME
**
** Attempt to match the given POSIX extended regular expression PATTERN
** over all historic versions of FILENAME.  For details of the supported





** RE dialect, see https://fossil-scm.org/fossil/doc/trunk/www/grep.md
**
** Options:
**


**     -i|--ignore-case         Ignore case
**     -l|--files-with-matches  List only checkin ID for versions that match





**     -v|--verbose             Show each file as it is analyzed
*/
void re_grep_cmd(void){
  u32 flags = 0;
  int bVerbose = 0;
  ReCompiled *pRe;
  const char *zErr;
  int ignoreCase = 0;
  Blob fullName;










  if( find_option("ignore-case","i",0)!=0 ) ignoreCase = 1;
  if( find_option("files-with-matches","l",0)!=0 ) flags |= GREP_EXISTS;
  if( find_option("verbose","v",0)!=0 ) bVerbose = 1;











  db_find_and_open_repository(0, 0);
  verify_all_options();
  if( g.argc<4 ){
    usage("REGEXP FILENAME");
  }
  zErr = re_compile(&pRe, g.argv[2], ignoreCase);
  if( zErr ) fossil_fatal("%s", zErr);





  if( file_tree_name(g.argv[3], &fullName, 0, 0) ){
    int fnid = db_int(0, "SELECT fnid FROM filename WHERE name=%Q",
                      blob_str(&fullName));
    if( fnid ){
      Stmt q;
      add_content_sql_commands(g.db);












      db_prepare(&q,
        "SELECT content(ux), ux FROM ("



        "  SELECT blob.uuid AS ux, min(event.mtime) AS mx"

        "    FROM mlink, blob, event"
        "   WHERE mlink.mid=event.objid"
        "     AND mlink.fid=blob.rid"

        "     AND mlink.fnid=%d"
        "   GROUP BY blob.uuid"
        ") ORDER BY mx DESC;",
        fnid
      );
      while( db_step(&q)==SQLITE_ROW ){









        if( bVerbose ) fossil_print("%S:\n", db_column_text(&q,1));

        grep_buffer(pRe, db_column_text(&q,1), db_column_text(&q,0), flags);







      }







      db_finalize(&q);






    }
  }
}







|


|
>
>
>
>
>
|



>
>
|
|
>
>
>
>
>
|








>
>
>
>
>
>
>
>
>



|
>
>
>
>
>
>
>
>
>
>
>



|




>
>
>
>
|
|
|
|
<
|
>
>
>
>
>
>
>
>
>
>
>
>
|
|
>
>
>
|
>
|
|
|
>
|
|
|
<
|
|
>
>
>
>
>
>
>
>
>
|
>
|
>
>
>
>
>
>
>

>
>
>
>
>
>
>
|
>
>
>
>
>
>



791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868

869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895

896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
  }
  re_free(pRe);
}

/*
** COMMAND: grep
**
** Usage: %fossil grep [OPTIONS] PATTERN FILENAME ...
**
** Attempt to match the given POSIX extended regular expression PATTERN
** historic versions of FILENAME.  The search begins with the most recent
** version of the file and moves backwards in time.  Multiple FILENAMEs can
** be specified, in which case all named files are searched in reverse
** chronological order.
**
** For details of the supported regular expression dialect, see
** https://fossil-scm.org/fossil/doc/trunk/www/grep.md
**
** Options:
**
**     -c|--count                 Suppress normal output; instead print a count
**                                of the number of matching files
**     -i|--ignore-case           Ignore case
**     -l|--files-with-matches    List only hash for each match
**     --once                     Stop searching after the first match
**     -s|--no-messages           Suppress error messages about nonexistant
**                                or unreadable files
**     -v|--invert-match          Invert the sense of matching.  Show only
**                                files that have no matches. Implies -l
**     --verbose                  Show each file as it is analyzed
*/
void re_grep_cmd(void){
  u32 flags = 0;
  int bVerbose = 0;
  ReCompiled *pRe;
  const char *zErr;
  int ignoreCase = 0;
  Blob fullName;
  int ii;
  int nMatch = 0;
  int bNoMsg;
  int cntFlag;
  int bOnce;
  int bInvert;
  int nSearch = 0;
  Stmt q;


  if( find_option("ignore-case","i",0)!=0 ) ignoreCase = 1;
  if( find_option("files-with-matches","l",0)!=0 ) flags |= GREP_EXISTS;
  if( find_option("verbose",0,0)!=0 ) bVerbose = 1;
  if( find_option("quiet","q",0) ) flags |= GREP_QUIET|GREP_EXISTS;
  bNoMsg = find_option("no-messages","s",0)!=0;
  bOnce = find_option("once",0,0)!=0;
  bInvert = find_option("invert-match","v",0)!=0;
  if( bInvert ){
    flags |= GREP_QUIET|GREP_EXISTS;
  }
  cntFlag = find_option("count","c",0)!=0;
  if( cntFlag ){
    flags |= GREP_QUIET|GREP_EXISTS;
  }
  db_find_and_open_repository(0, 0);
  verify_all_options();
  if( g.argc<4 ){
    usage("REGEXP FILENAME ...");
  }
  zErr = re_compile(&pRe, g.argv[2], ignoreCase);
  if( zErr ) fossil_fatal("%s", zErr);

  add_content_sql_commands(g.db);
  db_multi_exec("CREATE TEMP TABLE arglist(iname,fname,fnid);");
  for(ii=3; ii<g.argc; ii++){
    const char *zTarget = g.argv[ii];
    if( file_tree_name(zTarget, &fullName, 0, 1) ){
      int fnid = db_int(0, "SELECT fnid FROM filename WHERE name=%Q",
                        blob_str(&fullName));
      if( !fnid ){

        if( bNoMsg ) continue;
        if( file_size(zTarget, ExtFILE)<0 ){
          fossil_fatal("no such file: %s", zTarget);
        }
        fossil_fatal("not a managed file: %s", zTarget);
      }else{
        db_multi_exec(
          "INSERT INTO arglist(iname,fname,fnid) VALUES(%Q,%Q,%d)",
          zTarget, blob_str(&fullName), fnid);
      }
    }
    blob_reset(&fullName);
  }
  db_prepare(&q,
    " SELECT"
    "   A.uuid,"       /* file hash */
    "   A.rid,"        /* file rid */
    "   B.uuid,"       /* check-in hash */
    "   datetime(min(event.mtime)),"  /* check-in time */
    "   arglist.iname"                /* file name */
    " FROM arglist, mlink, blob A, blob B, event"
    " WHERE mlink.mid=event.objid"
    "   AND mlink.fid=A.rid"
    "   AND mlink.mid=B.rid"
    "   AND mlink.fnid=arglist.fnid"
    " GROUP BY A.uuid"
    " ORDER BY min(event.mtime) DESC;"

  );
  while( db_step(&q)==SQLITE_ROW ){
    const char *zFileHash = db_column_text(&q,0);
    int rid = db_column_int(&q,1);
    const char *zCkinHash = db_column_text(&q,2);
    const char *zDate = db_column_text(&q,3);
    const char *zFN = db_column_text(&q,4);
    char *zLabel;
    Blob cx;
    content_get(rid, &cx);
    zLabel = mprintf("%.16s %s %S checkin %S", zDate, zFN,zFileHash,zCkinHash);
    if( bVerbose ) fossil_print("Scanning: %s\n", zLabel);
    nSearch++;
    nMatch += grep_buffer(pRe, zLabel, blob_str(&cx), flags);
    blob_reset(&cx);
    if( bInvert && cntFlag==0 ){
      if( nMatch==0 ){
        fossil_print("== %s\n", zLabel);
        if( bOnce ) nMatch = 1;
      }else{
        nMatch = 0;
      }
    }
    fossil_free(zLabel);
    if( nMatch ){
      if( (flags & GREP_QUIET)!=0 ) break;
      if( bOnce ) break;
    }
  }
  db_finalize(&q);
  re_free(pRe);
  if( cntFlag ){
    if( bInvert ){
      fossil_print("%d\n", nSearch-nMatch);
    }else{
      fossil_print("%d\n", nMatch);
    }
  }
}

Changes to www/changes.wiki.

1
2






3
4
5
6
7
8
9
<title>Change Log</title>







<a name='v2_10'></a>
<h2>Changes for Version 2.10 (2019-10-04)</h2>

  *  Added support for [./serverext.wiki|CGI-based Server Extensions].
  *  Added the [/help?cmd=repolist-skin|repolist-skin] setting used to
     add style to repository list pages.
  *  Enhance the hierarchical display of Forum threads to do less


>
>
>
>
>
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
<title>Change Log</title>

<a name='v2_11'></a>
<h2>Changes for Version 2.11 (pending)</h2>

  *  Support Markdown in the default ticket configuration
  *  Rework the "fossil grep" command to be more useful

<a name='v2_10'></a>
<h2>Changes for Version 2.10 (2019-10-04)</h2>

  *  Added support for [./serverext.wiki|CGI-based Server Extensions].
  *  Added the [/help?cmd=repolist-skin|repolist-skin] setting used to
     add style to repository list pages.
  *  Enhance the hierarchical display of Forum threads to do less