Fossil

Check-in [00bf8c198e]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:The performance was still not satisfying, even with faster recomputing of successors. Doing it multiple times (Building the graph in each breaker and sort passes) eats time. Caching in memory blows the memory. Chosen solution: Cache this information in the database. Created a new pass 'CsetDeps' which is run between 'InitCsets' and 'BreakRevCsetCycles' (i.e. changeset creation and first breaker pass). It computes the changeset dependencies from the file-level dependencies once and saves the result in the state, in the new table 'cssuccessor'. Now the breaker and sort passes can get the information quickly, with virtually no effort. The dependencies are recomputed incrementally when a changeset is split by one of the breaker passes, for its fragments and its predecessors. The loop check is now trivial, and integrated into the successor computation, with the heavy lifting for the detailed analysis and reporting moved down into the type-dependent SQL queries. The relevant new method is 'loops'. Now that the loop check is incremental the pass based checks have been removed from the integrity module, and the option '--loopcheck' has been eliminated. For paranoia the graph setup and modification code got its loop check reinstated as an assert, redusing the changeset report code. Renumbered the breaker and sort passes. A number of places, like graph setup and traversal, loading of changesets, etc. got feedback indicators to show their progress. The selection of revision and symbol changesets for the associated breaker passes was a bit on the slow side. We now keep changeset lists sorted by type (during loading or general construction) and access them directly.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 00bf8c198ee6db85e9ec3868df1b6649695ae0e3
User & Date: aku 2007-12-02 20:04:40.000
Context
2007-12-02
20:06
Importer Status... Speed. This is now mostly acceptable. The one exception is pass 'BreakAllCsetCycles'. The reason is that the limit computation it does for backward branches still uses the inefficient file-level dependency computation. This will be tackled in short order. For the other passes the file spent is 'CsetsDeps' is recouped by the much faster graph setup. Memory. This seems to be mostly acceptable as well, with the exceptions of 'BreakAllCsetCycles' (again, for reasons see above), and 'InitCsets'. It seems to happen while the pass breaks internal dependencies, but there is no hard data. I have to measure using a memory-debug enabled tclsh. I suspect either the actual internal dependencies, or the pseudo-dependencies. Maybe combined with a bad choice of data structures. Well, measuring first. check-in: e8c374f670 user: aku tags: trunk
20:04
The performance was still not satisfying, even with faster recomputing of successors. Doing it multiple times (Building the graph in each breaker and sort passes) eats time. Caching in memory blows the memory. Chosen solution: Cache this information in the database. Created a new pass 'CsetDeps' which is run between 'InitCsets' and 'BreakRevCsetCycles' (i.e. changeset creation and first breaker pass). It computes the changeset dependencies from the file-level dependencies once and saves the result in the state, in the new table 'cssuccessor'. Now the breaker and sort passes can get the information quickly, with virtually no effort. The dependencies are recomputed incrementally when a changeset is split by one of the breaker passes, for its fragments and its predecessors. The loop check is now trivial, and integrated into the successor computation, with the heavy lifting for the detailed analysis and reporting moved down into the type-dependent SQL queries. The relevant new method is 'loops'. Now that the loop check is incremental the pass based checks have been removed from the integrity module, and the option '--loopcheck' has been eliminated. For paranoia the graph setup and modification code got its loop check reinstated as an assert, redusing the changeset report code. Renumbered the breaker and sort passes. A number of p... check-in: 00bf8c198e user: aku tags: trunk
06:58
Added progress output to the breaking of backward branches. check-in: a437da486d user: aku tags: trunk
Changes
Unified Diff Ignore Whitespace Patch
Changes to tools/cvs2fossil/lib/c2f_cyclebreaker.tcl.
100
101
102
103
104
105
106



107
108

109
110
111
112
113
114
115
	#    oldest to youngest, saving and removing dependencies. If
	#    we find no nodes without predecessors we have a cycle,
	#    and work on breaking it.

	log write 3 cyclebreaker {Traverse changesets}

	InitializeCandidates $dg



	while {1} {
	    while {[WithoutPredecessor $dg n]} {

		MarkWatch $dg
		ProcessedHook $dg $n $myat
		$dg node delete $n
		incr myat
		ShowPendingNodes
	    }








>
>
>


>







100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
	#    oldest to youngest, saving and removing dependencies. If
	#    we find no nodes without predecessors we have a cycle,
	#    and work on breaking it.

	log write 3 cyclebreaker {Traverse changesets}

	InitializeCandidates $dg

	set k   0
	set max [llength [$dg nodes]]
	while {1} {
	    while {[WithoutPredecessor $dg n]} {
		log progress 2 cyclebreaker $k $max ; incr k
		MarkWatch $dg
		ProcessedHook $dg $n $myat
		$dg node delete $n
		incr myat
		ShowPendingNodes
	    }

203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
	foreach cset $changesets {
	    log progress 2 cyclebreaker $n $max
	    foreach succ [$cset successors] {
		# Changesets may have dependencies outside of the
		# chosen set. These are ignored
		if {![$dg node exists $succ]} continue
		$dg arc insert $cset $succ
		if {$succ eq $cset} {
		    $cset loopcheck
		    trouble internal "[$cset str] depends on itself"
		}
	    }
	    incr n
	}

	if {$log} {
	    log write 3 cyclebreaker "Has [nsp [llength [$dg arcs]] dependency dependencies]"
	}







|
|
|
<







207
208
209
210
211
212
213
214
215
216

217
218
219
220
221
222
223
	foreach cset $changesets {
	    log progress 2 cyclebreaker $n $max
	    foreach succ [$cset successors] {
		# Changesets may have dependencies outside of the
		# chosen set. These are ignored
		if {![$dg node exists $succ]} continue
		$dg arc insert $cset $succ
		integrity assert {
		    $succ ne $cset
		} {[$cset reportloop 0]Changeset loop was not detected during creation}

	    }
	    incr n
	}

	if {$log} {
	    log write 3 cyclebreaker "Has [nsp [llength [$dg arcs]] dependency dependencies]"
	}
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444

	foreach cset $replacements {
	    foreach succ [$cset successors] {
		# The new changesets may have dependencies outside of
		# the chosen set. These are ignored
		if {![$dg node exists $succ]} continue
		$dg arc insert $cset $succ
		if {$succ eq $cset} {
		    $cset loopcheck
		    trouble internal "[$cset str] depends on itself"
		}
	    }
	}
	foreach cset $pre {
	    foreach succ [$cset successors] {
		# Note that the arc may already exist in the graph. If
		# so ignore it. The new changesets may have
		# dependencies outside of the chosen set. These are







|
|
|
<







430
431
432
433
434
435
436
437
438
439

440
441
442
443
444
445
446

	foreach cset $replacements {
	    foreach succ [$cset successors] {
		# The new changesets may have dependencies outside of
		# the chosen set. These are ignored
		if {![$dg node exists $succ]} continue
		$dg arc insert $cset $succ
		integrity assert {
		    $succ ne $cset
		} {[$cset reportloop 0]Changeset loop was not detected during creation}

	    }
	}
	foreach cset $pre {
	    foreach succ [$cset successors] {
		# Note that the arc may already exist in the graph. If
		# so ignore it. The new changesets may have
		# dependencies outside of the chosen set. These are
553
554
555
556
557
558
559
560
561

562
563
564
565
566
567
568

    # # ## ### ##### ######## #############
}

namespace eval ::vc::fossil::import::cvs {
    namespace export cyclebreaker
    namespace eval cyclebreaker {
	namespace eval project {
	    namespace import ::vc::fossil::import::cvs::integrity

	    namespace import ::vc::fossil::import::cvs::project::rev
	    namespace import ::vc::fossil::import::cvs::project::revlink
	}
	namespace import ::vc::tools::misc::*
	namespace import ::vc::tools::log
	namespace import ::vc::tools::trouble
	namespace import ::vc::tools::dot







<
|
>







555
556
557
558
559
560
561

562
563
564
565
566
567
568
569
570

    # # ## ### ##### ######## #############
}

namespace eval ::vc::fossil::import::cvs {
    namespace export cyclebreaker
    namespace eval cyclebreaker {

	namespace import ::vc::fossil::import::cvs::integrity
	namespace eval project {
	    namespace import ::vc::fossil::import::cvs::project::rev
	    namespace import ::vc::fossil::import::cvs::project::revlink
	}
	namespace import ::vc::tools::misc::*
	namespace import ::vc::tools::log
	namespace import ::vc::tools::trouble
	namespace import ::vc::tools::dot
Changes to tools/cvs2fossil/lib/c2f_integrity.tcl.
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
	log write 4 integrity {Check database consistency}

	set n 0
	AllButMeta
	return
    }

    typemethod changesets {csets} {
	log write 4 integrity {Check database consistency}

	set n 0
	RevisionChangesets
	TagChangesets
	BranchChangesets
	trouble abort? ; # Avoid expensive check if anything found before

	LoopCheck $csets
	return
    }

    typemethod loopcheckon {} {
	set myloopcheck 1
	return
    }

    # # ## ### ##### ######## #############
    ## Internal methods

    proc AllButMeta {} {







|






<
<
<
<
<
<
<
<







49
50
51
52
53
54
55
56
57
58
59
60
61
62








63
64
65
66
67
68
69
	log write 4 integrity {Check database consistency}

	set n 0
	AllButMeta
	return
    }

    typemethod changesets {} {
	log write 4 integrity {Check database consistency}

	set n 0
	RevisionChangesets
	TagChangesets
	BranchChangesets








	return
    }

    # # ## ### ##### ######## #############
    ## Internal methods

    proc AllButMeta {} {
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
				 WHERE VV.cid = UU.cid
				 AND   UU.fcount < VV.rcount)
		AND    T.tid = C.type
	    }
	return
    }

    proc LoopCheck {csets} {
	::variable myloopcheck
	if {!$myloopcheck} return

	log write 4 integrity {Checking changesets for self-references}

	foreach cset $csets {
	    if {[$cset loopcheck]} {
		trouble fatal "[$cset str] depends on itself"
	    }
	}
	return
    }

    proc ___UnusedChangesetChecks___ {} {
	# This code performs a number of paranoid checks of the
	# database, searching for inconsistent changeset/revision
	# information.

	return ; # Disabled for now, bottlenecks ...








<
<
<
<
<
<
<
<
<
<
<
<
<
<







759
760
761
762
763
764
765














766
767
768
769
770
771
772
				 WHERE VV.cid = UU.cid
				 AND   UU.fcount < VV.rcount)
		AND    T.tid = C.type
	    }
	return
    }















    proc ___UnusedChangesetChecks___ {} {
	# This code performs a number of paranoid checks of the
	# database, searching for inconsistent changeset/revision
	# information.

	return ; # Disabled for now, bottlenecks ...

901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
	    set b "<$cstype $csid>"
	    trouble fatal "$fname <$revnr> [string map [list @ $b] $label]"
	}
	log write 5 integrity {\[[format %02d [incr n]]\] [expr {$ok ? "Ok    " : "Failed"}] ... $header}
	return
    }

    # # ## ### ##### ######## #############

    typevariable myloopcheck 0 ; # Boolean flag. Controls whether
				 # 'integrity changesets' looks for
				 # changesets with loops or not.
				 # Default is to not look for them.

    # # ## ### ##### ######## #############
    ## Configuration

    pragma -hasinstances   no ; # singleton
    pragma -hastypeinfo    no ; # no introspection
    pragma -hastypedestroy no ; # immortal








<
<
<
<
<
<
<







879
880
881
882
883
884
885







886
887
888
889
890
891
892
	    set b "<$cstype $csid>"
	    trouble fatal "$fname <$revnr> [string map [list @ $b] $label]"
	}
	log write 5 integrity {\[[format %02d [incr n]]\] [expr {$ok ? "Ok    " : "Failed"}] ... $header}
	return
    }








    # # ## ### ##### ######## #############
    ## Configuration

    pragma -hasinstances   no ; # singleton
    pragma -hastypeinfo    no ; # no introspection
    pragma -hastypedestroy no ; # immortal

Changes to tools/cvs2fossil/lib/c2f_option.tcl.
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
    # --project
    # -v, --verbose
    # -q, --quiet
    # --state (conversion status, ala config.cache)
    # --trunk-only
    # --exclude, --force-tag, --force-branch
    # --batch
    # --loopcheck

    # -o, --output
    # --dry-run
    # --symbol-transform RE:XX

    # # ## ### ##### ######## #############
    ## Public API, Methods







<







45
46
47
48
49
50
51

52
53
54
55
56
57
58
    # --project
    # -v, --verbose
    # -q, --quiet
    # --state (conversion status, ala config.cache)
    # --trunk-only
    # --exclude, --force-tag, --force-branch
    # --batch


    # -o, --output
    # --dry-run
    # --symbol-transform RE:XX

    # # ## ### ##### ######## #############
    ## Public API, Methods
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
		--trunk-only                { repository trunkonly! }
		--exclude                   { project::sym exclude     [Value arguments] }
		--force-tag                 { project::sym forcetag    [Value arguments] }
		--force-branch              { project::sym forcebranch [Value arguments] }
		--batch                     { log noprogress }
		--dots                      { cyclebreaker dotsto [Value arguments] }
		--watch                     { cyclebreaker watch  [Value arguments] }
		--loopcheck                 { integrity loopcheckon }
		default {
		    Usage $badoption$option\n$gethelp
		}
	    }
	}

	if {[llength $arguments] > 1} Usage







<







79
80
81
82
83
84
85

86
87
88
89
90
91
92
		--trunk-only                { repository trunkonly! }
		--exclude                   { project::sym exclude     [Value arguments] }
		--force-tag                 { project::sym forcetag    [Value arguments] }
		--force-branch              { project::sym forcebranch [Value arguments] }
		--batch                     { log noprogress }
		--dots                      { cyclebreaker dotsto [Value arguments] }
		--watch                     { cyclebreaker watch  [Value arguments] }

		default {
		    Usage $badoption$option\n$gethelp
		}
	    }
	}

	if {[llength $arguments] > 1} Usage
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
	trouble info "                               branch. Both project and symbol names"
	trouble info "                               are glob patterns."
	trouble info ""
	trouble info "    --dots PATH                Write the changeset graphs before, after,"
	trouble info "                               and during breaking the of cycles to the"
	trouble info "                               direcotry PATH, using GraphViz's dot format"
	trouble info ""
	trouble info "    --loopcheck                Activate the expensive search for change-"
	trouble info "                               with loops, i.e. depending on themselves."

	# --project, --cache
	# ...
	return
    }

    proc PrintVersion {} {







<
<







145
146
147
148
149
150
151


152
153
154
155
156
157
158
	trouble info "                               branch. Both project and symbol names"
	trouble info "                               are glob patterns."
	trouble info ""
	trouble info "    --dots PATH                Write the changeset graphs before, after,"
	trouble info "                               and during breaking the of cycles to the"
	trouble info "                               direcotry PATH, using GraphViz's dot format"
	trouble info ""



	# --project, --cache
	# ...
	return
    }

    proc PrintVersion {} {
Changes to tools/cvs2fossil/lib/c2f_patopsort.tcl.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
## -*- tcl -*-
# # ## ### ##### ######## ############# #####################
## Copyright (c) 2007 Andreas Kupries.
#
# This software is licensed as described in the file LICENSE, which
# you should have received as part of this distribution.
#
# This software consists of voluntary contributions made by many
# individuals.  For exact contribution history, see the revision
# history and logs, available at http://fossil-scm.hwaci.com/fossil
# # ## ### ##### ######## ############# #####################

## Pass X. This pass goes over all changesets and sorts them
## topologically. It assumes that there are no cycles which could
## impede it, any remaining having been broken by the previous two
## passes, and aborts if that condition doesn't hold.

# # ## ### ##### ######## ############# #####################
## Requirements













|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
## -*- tcl -*-
# # ## ### ##### ######## ############# #####################
## Copyright (c) 2007 Andreas Kupries.
#
# This software is licensed as described in the file LICENSE, which
# you should have received as part of this distribution.
#
# This software consists of voluntary contributions made by many
# individuals.  For exact contribution history, see the revision
# history and logs, available at http://fossil-scm.hwaci.com/fossil
# # ## ### ##### ######## ############# #####################

## Pass XI. This pass goes over all changesets and sorts them
## topologically. It assumes that there are no cycles which could
## impede it, any remaining having been broken by the previous two
## passes, and aborts if that condition doesn't hold.

# # ## ### ##### ######## ############# #####################
## Requirements

Changes to tools/cvs2fossil/lib/c2f_pbreakacycle.tcl.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
## -*- tcl -*-
# # ## ### ##### ######## ############# #####################
## Copyright (c) 2007 Andreas Kupries.
#
# This software is licensed as described in the file LICENSE, which
# you should have received as part of this distribution.
#
# This software consists of voluntary contributions made by many
# individuals.  For exact contribution history, see the revision
# history and logs, available at http://fossil-scm.hwaci.com/fossil
# # ## ### ##### ######## ############# #####################

## Pass IX. This is the final pass for breaking changeset dependency
## cycles. The previous breaker passes (6 and 8) broke cycles covering
## revision and symbol changesets, respectively. This pass now breaks
## any remaining cycles, each of which has to contain at least one
## revision and at least one symbol changeset.

# # ## ### ##### ######## ############# #####################
## Requirements













|
|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
## -*- tcl -*-
# # ## ### ##### ######## ############# #####################
## Copyright (c) 2007 Andreas Kupries.
#
# This software is licensed as described in the file LICENSE, which
# you should have received as part of this distribution.
#
# This software consists of voluntary contributions made by many
# individuals.  For exact contribution history, see the revision
# history and logs, available at http://fossil-scm.hwaci.com/fossil
# # ## ### ##### ######## ############# #####################

## Pass X. This is the final pass for breaking changeset dependency
## cycles. The previous breaker passes (7 and 9) broke cycles covering
## revision and symbol changesets, respectively. This pass now breaks
## any remaining cycles, each of which has to contain at least one
## revision and at least one symbol changeset.

# # ## ### ##### ######## ############# #####################
## Requirements

78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99



100
101
102
103
104
105
106

	state transaction {
	    LoadCommitOrder
	    cyclebreaker run break-all [myproc Changesets]
	}

	repository printcsetstatistics
	integrity changesets [project::rev all]
	return
    }

    typemethod discard {} {
	# Pass manager interface. Executed for all passes after the
	# run passes, to remove all data of this pass from the state,
	# as being out of date.
	return
    }

    # # ## ### ##### ######## #############
    ## Internal methods

    proc Changesets {} { project::rev all }




    proc LoadCommitOrder {} {
	::variable mycset
	::variable myrevisionchangesets

	state transaction {
	    foreach {cid pos} [state run { SELECT cid, pos FROM csorder }] {







|













|
>
>
>







78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109

	state transaction {
	    LoadCommitOrder
	    cyclebreaker run break-all [myproc Changesets]
	}

	repository printcsetstatistics
	integrity changesets
	return
    }

    typemethod discard {} {
	# Pass manager interface. Executed for all passes after the
	# run passes, to remove all data of this pass from the state,
	# as being out of date.
	return
    }

    # # ## ### ##### ######## #############
    ## Internal methods

    proc Changesets {} {
	log write 2 breakrcycle {Selecting all changesets}
	return [project::rev all]
    }

    proc LoadCommitOrder {} {
	::variable mycset
	::variable myrevisionchangesets

	state transaction {
	    foreach {cid pos} [state run { SELECT cid, pos FROM csorder }] {
Changes to tools/cvs2fossil/lib/c2f_pbreakrcycle.tcl.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
## -*- tcl -*-
# # ## ### ##### ######## ############# #####################
## Copyright (c) 2007 Andreas Kupries.
#
# This software is licensed as described in the file LICENSE, which
# you should have received as part of this distribution.
#
# This software consists of voluntary contributions made by many
# individuals.  For exact contribution history, see the revision
# history and logs, available at http://fossil-scm.hwaci.com/fossil
# # ## ### ##### ######## ############# #####################

## Pass VI. This pass goes over the set of revision based changesets
## and breaks all dependency cycles they may be in. We need a
## dependency tree. Identical to pass VII, except for the selection of
## the changesets.

# # ## ### ##### ######## ############# #####################
## Requirements

package require Tcl 8.4                                   ; # Required runtime.
package require snit                                      ; # OO system.












|

|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
## -*- tcl -*-
# # ## ### ##### ######## ############# #####################
## Copyright (c) 2007 Andreas Kupries.
#
# This software is licensed as described in the file LICENSE, which
# you should have received as part of this distribution.
#
# This software consists of voluntary contributions made by many
# individuals.  For exact contribution history, see the revision
# history and logs, available at http://fossil-scm.hwaci.com/fossil
# # ## ### ##### ######## ############# #####################

## Pass VII. This pass goes over the set of revision based changesets
## and breaks all dependency cycles they may be in. We need a
## dependency tree. Identical to pass IX, except for the selection of
## the changesets.

# # ## ### ##### ######## ############# #####################
## Requirements

package require Tcl 8.4                                   ; # Required runtime.
package require snit                                      ; # OO system.
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88

89
90
91
92
93
94
95
96
97
98
99
100
	cyclebreaker breakcmd {::vc::fossil::import::cvs::cyclebreaker break}

	state transaction {
	    cyclebreaker run break-rev [myproc Changesets]
	}

	repository printcsetstatistics
	integrity changesets [project::rev all]
	return
    }

    typemethod discard {} {
	# Pass manager interface. Executed for all passes after the
	# run passes, to remove all data of this pass from the state,
	# as being out of date.
	return
    }

    # # ## ### ##### ######## #############
    ## Internal methods

    proc Changesets {} {

	return [struct::list filter [project::rev all] [myproc IsByRevision]]
    }

    proc IsByRevision {cset} { $cset byrevision }

    # # ## ### ##### ######## #############
    ## Configuration

    pragma -hasinstances   no ; # singleton
    pragma -hastypeinfo    no ; # no introspection
    pragma -hastypedestroy no ; # immortal








|














>
|


<
<







67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92


93
94
95
96
97
98
99
	cyclebreaker breakcmd {::vc::fossil::import::cvs::cyclebreaker break}

	state transaction {
	    cyclebreaker run break-rev [myproc Changesets]
	}

	repository printcsetstatistics
	integrity changesets
	return
    }

    typemethod discard {} {
	# Pass manager interface. Executed for all passes after the
	# run passes, to remove all data of this pass from the state,
	# as being out of date.
	return
    }

    # # ## ### ##### ######## #############
    ## Internal methods

    proc Changesets {} {
	log write 2 breakrcycle {Selecting the revision changesets}
	return [project::rev rev]
    }



    # # ## ### ##### ######## #############
    ## Configuration

    pragma -hasinstances   no ; # singleton
    pragma -hastypeinfo    no ; # no introspection
    pragma -hastypedestroy no ; # immortal

Changes to tools/cvs2fossil/lib/c2f_pbreakscycle.tcl.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

24
25
26
27
28
29
30
## -*- tcl -*-
# # ## ### ##### ######## ############# #####################
## Copyright (c) 2007 Andreas Kupries.
#
# This software is licensed as described in the file LICENSE, which
# you should have received as part of this distribution.
#
# This software consists of voluntary contributions made by many
# individuals.  For exact contribution history, see the revision
# history and logs, available at http://fossil-scm.hwaci.com/fossil
# # ## ### ##### ######## ############# #####################

## Pass VIII. This pass goes over the set of symbol based changesets
## and breaks all dependency cycles they may be in. We need a
## dependency tree. Identical to pass VI, except for the selection of
## the changesets.

# # ## ### ##### ######## ############# #####################
## Requirements

package require Tcl 8.4                                   ; # Required runtime.
package require snit                                      ; # OO system.
package require struct::list                              ; # Higher order list operations.

package require vc::fossil::import::cvs::cyclebreaker     ; # Breaking dependency cycles.
package require vc::fossil::import::cvs::repository       ; # Repository management.
package require vc::fossil::import::cvs::state            ; # State storage.
package require vc::fossil::import::cvs::integrity        ; # State integrity checks.
package require vc::fossil::import::cvs::project::rev     ; # Project level changesets

# # ## ### ##### ######## ############# #####################












|
|
|
|







>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
## -*- tcl -*-
# # ## ### ##### ######## ############# #####################
## Copyright (c) 2007 Andreas Kupries.
#
# This software is licensed as described in the file LICENSE, which
# you should have received as part of this distribution.
#
# This software consists of voluntary contributions made by many
# individuals.  For exact contribution history, see the revision
# history and logs, available at http://fossil-scm.hwaci.com/fossil
# # ## ### ##### ######## ############# #####################

## Pass IX. This pass goes over the set of symbol based changesets and
## breaks all dependency cycles they may be in. We need a dependency
## tree. Identical to pass VII, except for the selection of the
## changesets.

# # ## ### ##### ######## ############# #####################
## Requirements

package require Tcl 8.4                                   ; # Required runtime.
package require snit                                      ; # OO system.
package require struct::list                              ; # Higher order list operations.
package require vc::tools::log                            ; # User feedback.
package require vc::fossil::import::cvs::cyclebreaker     ; # Breaking dependency cycles.
package require vc::fossil::import::cvs::repository       ; # Repository management.
package require vc::fossil::import::cvs::state            ; # State storage.
package require vc::fossil::import::cvs::integrity        ; # State integrity checks.
package require vc::fossil::import::cvs::project::rev     ; # Project level changesets

# # ## ### ##### ######## ############# #####################
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87

88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112


113
114
115
116
117
118
119
120
	cyclebreaker breakcmd {::vc::fossil::import::cvs::cyclebreaker break}

	state transaction {
	    cyclebreaker run break-sym [myproc Changesets]
	}

	repository printcsetstatistics
	integrity changesets [project::rev all]
	return
    }

    typemethod discard {} {
	# Pass manager interface. Executed for all passes after the
	# run passes, to remove all data of this pass from the state,
	# as being out of date.
	return
    }

    # # ## ### ##### ######## #############
    ## Internal methods

    proc Changesets {} {

	return [struct::list filter [project::rev all] [myproc IsBySymbol]]
    }

    proc IsBySymbol {cset} { $cset bysymbol }

    # # ## ### ##### ######## #############
    ## Configuration

    pragma -hasinstances   no ; # singleton
    pragma -hastypeinfo    no ; # no introspection
    pragma -hastypedestroy no ; # immortal

    # # ## ### ##### ######## #############
}

namespace eval ::vc::fossil::import::cvs::pass {
    namespace export breakscycle
    namespace eval breakscycle {
	namespace import ::vc::fossil::import::cvs::cyclebreaker
	namespace import ::vc::fossil::import::cvs::repository
	namespace import ::vc::fossil::import::cvs::state
	namespace import ::vc::fossil::import::cvs::integrity
	namespace eval project {
	    namespace import ::vc::fossil::import::cvs::project::rev
	}


    }
}

# # ## ### ##### ######## ############# #####################
## Ready

package provide vc::fossil::import::cvs::pass::breakscycle 1.0
return







|














>
|


<
<




















>
>








67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92


93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
	cyclebreaker breakcmd {::vc::fossil::import::cvs::cyclebreaker break}

	state transaction {
	    cyclebreaker run break-sym [myproc Changesets]
	}

	repository printcsetstatistics
	integrity changesets
	return
    }

    typemethod discard {} {
	# Pass manager interface. Executed for all passes after the
	# run passes, to remove all data of this pass from the state,
	# as being out of date.
	return
    }

    # # ## ### ##### ######## #############
    ## Internal methods

    proc Changesets {} {
	log write 2 breakscycle {Selecting the symbol changesets}
	return [project::rev sym]
    }



    # # ## ### ##### ######## #############
    ## Configuration

    pragma -hasinstances   no ; # singleton
    pragma -hastypeinfo    no ; # no introspection
    pragma -hastypedestroy no ; # immortal

    # # ## ### ##### ######## #############
}

namespace eval ::vc::fossil::import::cvs::pass {
    namespace export breakscycle
    namespace eval breakscycle {
	namespace import ::vc::fossil::import::cvs::cyclebreaker
	namespace import ::vc::fossil::import::cvs::repository
	namespace import ::vc::fossil::import::cvs::state
	namespace import ::vc::fossil::import::cvs::integrity
	namespace eval project {
	    namespace import ::vc::fossil::import::cvs::project::rev
	}
	namespace import ::vc::tools::log
	log register breakscycle
    }
}

# # ## ### ##### ######## ############# #####################
## Ready

package provide vc::fossil::import::cvs::pass::breakscycle 1.0
return
Changes to tools/cvs2fossil/lib/c2f_pinitcsets.tcl.
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
package require Tcl 8.4                               ; # Required runtime.
package require snit                                  ; # OO system.
package require vc::tools::misc                       ; # Text formatting.
package require vc::tools::log                        ; # User feedback.
package require vc::fossil::import::cvs::repository   ; # Repository management.
package require vc::fossil::import::cvs::state        ; # State storage.
package require vc::fossil::import::cvs::integrity    ; # State integrity checks.
package require vc::fossil::import::cvs::project::sym ; # Project level symbols
package require vc::fossil::import::cvs::project::rev ; # Project level changesets

# # ## ### ##### ######## ############# #####################
## Register the pass with the management

vc::fossil::import::cvs::pass define \
    InitCsets \







<







20
21
22
23
24
25
26

27
28
29
30
31
32
33
package require Tcl 8.4                               ; # Required runtime.
package require snit                                  ; # OO system.
package require vc::tools::misc                       ; # Text formatting.
package require vc::tools::log                        ; # User feedback.
package require vc::fossil::import::cvs::repository   ; # Repository management.
package require vc::fossil::import::cvs::state        ; # State storage.
package require vc::fossil::import::cvs::integrity    ; # State integrity checks.

package require vc::fossil::import::cvs::project::rev ; # Project level changesets

# # ## ### ##### ######## ############# #####################
## Register the pass with the management

vc::fossil::import::cvs::pass define \
    InitCsets \
111
112
113
114
115
116
117



118
119
120
121
122
123

124
125
126
127
128
129

130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
	state reading csitem
	state reading cstype

	# Need the types first, the constructor in the loop below uses
	# them to assert the correctness of type names.
	project::rev getcstypes




	foreach {id pid cstype srcid} [state run {
	    SELECT C.cid, C.pid, CS.name, C.src
	    FROM   changeset C, cstype CS
	    WHERE  C.type = CS.tid
	    ORDER BY C.cid
	}] {

	    set r [project::rev %AUTO% [repository projectof $pid] $cstype $srcid [state run {
		SELECT C.iid
		FROM   csitem C
		WHERE  C.cid = $id
		ORDER BY C.pos
	    }] $id]

	}

	project::rev loadcounter
	return
    }

    typemethod run {} {
	# Pass manager interface. Executed to perform the
	# functionality of the pass.

	state transaction {
	    CreateRevisionChangesets  ; # Group file revisions into csets.
	    BreakInternalDependencies ; # Split the csets based on internal conflicts.
	    CreateSymbolChangesets    ; # Create csets for tags and branches.
	    PersistTheChangesets
	}

	repository printcsetstatistics
	integrity changesets [project::rev all]
	return
    }

    typemethod discard {} {
	# Pass manager interface. Executed for all passes after the
	# run passes, to remove all data of this pass from the state,
	# as being out of date.







>
>
>






>






>


















|







110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
	state reading csitem
	state reading cstype

	# Need the types first, the constructor in the loop below uses
	# them to assert the correctness of type names.
	project::rev getcstypes

	# TODO: Move to project::rev
	set n 0
	log write 2 initcsets {Loading the changesets}
	foreach {id pid cstype srcid} [state run {
	    SELECT C.cid, C.pid, CS.name, C.src
	    FROM   changeset C, cstype CS
	    WHERE  C.type = CS.tid
	    ORDER BY C.cid
	}] {
	    log progress 2 initcsets $n {}
	    set r [project::rev %AUTO% [repository projectof $pid] $cstype $srcid [state run {
		SELECT C.iid
		FROM   csitem C
		WHERE  C.cid = $id
		ORDER BY C.pos
	    }] $id]
	    incr n
	}

	project::rev loadcounter
	return
    }

    typemethod run {} {
	# Pass manager interface. Executed to perform the
	# functionality of the pass.

	state transaction {
	    CreateRevisionChangesets  ; # Group file revisions into csets.
	    BreakInternalDependencies ; # Split the csets based on internal conflicts.
	    CreateSymbolChangesets    ; # Create csets for tags and branches.
	    PersistTheChangesets
	}

	repository printcsetstatistics
	integrity changesets
	return
    }

    typemethod discard {} {
	# Pass manager interface. Executed for all passes after the
	# run passes, to remove all data of this pass from the state,
	# as being out of date.
Changes to tools/cvs2fossil/lib/c2f_prev.tcl.
49
50
51
52
53
54
55

56
57
58
59
60
61
62
	set mysrcid	$srcid
	set myitems     $items
	set mypos       {} ; # Commit location is not known yet.

	# Keep track of the generated changesets and of the inverse
	# mapping from items to them.
	lappend mychangesets   $self

	set     myidmap($myid) $self
	foreach iid $items {
	    set key [list $cstype $iid]
	    set myitemmap($key) $self
	    lappend mytitems $key
	    log write 8 csets {MAP+ item <$key> $self = [$self str]}
	}







>







49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
	set mysrcid	$srcid
	set myitems     $items
	set mypos       {} ; # Commit location is not known yet.

	# Keep track of the generated changesets and of the inverse
	# mapping from items to them.
	lappend mychangesets   $self
	lappend mytchangesets($cstype) $self
	set     myidmap($myid) $self
	foreach iid $items {
	    set key [list $cstype $iid]
	    set myitemmap($key) $self
	    lappend mytitems $key
	    log write 8 csets {MAP+ item <$key> $self = [$self str]}
	}
84
85
86
87
88
89
90
91
























92
93

94

95

96
97
98
99
100
101
102
103
    delegate method bysymbol   to mytypeobj
    delegate method byrevision to mytypeobj
    delegate method isbranch   to mytypeobj
    delegate method istag      to mytypeobj

    method setpos {p} { set mypos $p ; return }
    method pos    {}  { return $mypos }

























    # result = list (changeset)
    method successors {} {

	return [struct::list map \

		    [$mytypeobj cs_successors $myitems] \

		    [mytypemethod of]]
    }

    # result = dict (item -> list (changeset))
    method successormap {} {
	# NOTE / FUTURE: Definitive bottleneck (can be millions of pairs).
	#
	# Only user is pass 9, computing the limits of backward








>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>


>
|
>
|
>
|







85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
    delegate method bysymbol   to mytypeobj
    delegate method byrevision to mytypeobj
    delegate method isbranch   to mytypeobj
    delegate method istag      to mytypeobj

    method setpos {p} { set mypos $p ; return }
    method pos    {}  { return $mypos }

    method determinesuccessors {} {
	# Pass 6 operation. Compute project-level dependencies from
	# the file-level data and save it back to the state. This may
	# be called during the cycle breaker passes as well, to adjust
	# the successor information of changesets which are the
	# predecessors of dropped changesets. For them we have to
	# remove their existing information first before inserting the
	# new data.
	state run {
	    DELETE FROM cssuccessor WHERE cid = $myid;
	}
	set loop 0
	foreach nid [$mytypeobj cs_successors $myitems] {
	    state run {
		INSERT INTO cssuccessor (cid,  nid)
		VALUES                  ($myid,$nid)
	    }
	    if {$nid == $myid} { set loop 1 }
	}
	# Report after the complete structure has been saved.
	if {$loop} { $self reportloop }
	return
    }

    # result = list (changeset)
    method successors {} {
	# Use the data saved by pass 6.
	return [struct::list map [state run {
	    SELECT S.nid
	    FROM   cssuccessor S
	    WHERE  S.cid = $myid
	}] [mytypemethod of]]
    }

    # result = dict (item -> list (changeset))
    method successormap {} {
	# NOTE / FUTURE: Definitive bottleneck (can be millions of pairs).
	#
	# Only user is pass 9, computing the limits of backward
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184

	# We perform all necessary splits in one go, instead of only
	# one. The previous algorithm, adapted from cvs2svn, computed
	# a lot of state which was thrown away and then computed again
	# for each of the fragments. It should be easier to update and
	# reuse that state.

	# The code checks only sucessor dependencies, as this
	# automatically covers the predecessor dependencies as well (A
	# successor dependency a -> b is also a predecessor dependency
	# b -> a).

	# Array of dependencies (parent -> child). This is pulled from
	# the state, and limited to successors within the changeset.








|







198
199
200
201
202
203
204
205
206
207
208
209
210
211
212

	# We perform all necessary splits in one go, instead of only
	# one. The previous algorithm, adapted from cvs2svn, computed
	# a lot of state which was thrown away and then computed again
	# for each of the fragments. It should be easier to update and
	# reuse that state.

	# The code checks only successor dependencies, as this
	# automatically covers the predecessor dependencies as well (A
	# successor dependency a -> b is also a predecessor dependency
	# b -> a).

	# Array of dependencies (parent -> child). This is pulled from
	# the state, and limited to successors within the changeset.

354
355
356
357
358
359
360
361
362

363
364
365
366
367
368
369
370
371
372

373
374
375
376
377
378

379
380
381
382

383
384
385
386
387
388
389
390
391

392
393
394
395
396
397

398
399

400
401


402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437





438
439
440
441
442
443
444
445
446
    method timerange {} { return [$mytypeobj timerange $myitems] }

    method drop {} {
	log write 8 csets {Dropping $self = [$self str]}

	state transaction {
	    state run {
		DELETE FROM changeset WHERE cid = $myid;
		DELETE FROM csitem    WHERE cid = $myid;

	    }
	}
	foreach iid $myitems {
	    set key [list $mytype $iid]
	    unset myitemmap($key)
	    log write 8 csets {MAP- item <$key> $self = [$self str]}
	}
	set pos          [lsearch -exact $mychangesets $self]
	set mychangesets [lreplace $mychangesets $pos $pos]
	return

    }

    method loopcheck {} {
	log write 7 csets {Checking [$self str] for loops /[llength $myitems]}

	if {![struct::set contains [$self successors] $self]} {

	    return 0
	}
	if {[log verbosity?] < 8} { return 1 }


	# Print the detailed successor structure of the self-
	# referential changeset, if the verbosity of the log is dialed
	# high enough.

	log write 8 csets [set hdr {Self-referential changeset [$self str] __________________}]
	array set nmap [$self nextmap]
	foreach item [lsort -dict [array names nmap]] {
	    foreach succitem $nmap($item) {
		set succcs $myitemmap($succitem)

		set hint [expr {($succcs eq $self)
				? "LOOP"
				: "    "}]
		set i   "<$item [$type itemstr $item]>"
		set s   "<$succitem [$type itemstr $succitem]>"
		set scs [$succcs str]

		log write 8 csets {$hint * $i --> $s --> cs $scs}
	    }

	}
	log write 8 csets [regsub -all {[^ 	]} $hdr {_}]


	return 1
    }

    typemethod split {cset args} {
	# As part of the creation of the new changesets specified in
	# ARGS as sets of items, all subsets of CSET's item set, CSET
	# will be dropped from all databases, in and out of memory,
	# and then destroyed.
	#
	# Note: The item lists found in args are tagged items. They
	# have to have the same type as the changeset, being subsets
	# of its items. This is checked in Untag1.

	log write 8 csets {OLD: [lsort [$cset items]]}
	ValidateFragments $cset $args

	# All checks pass, actually perform the split.

	struct::list assign [$cset data] project cstype cssrc

	$cset drop
	$cset destroy

	set newcsets {}
	foreach fragmentitems $args {
	    log write 8 csets {MAKE: [lsort $fragmentitems]}

	    set fragment [$type %AUTO% $project $cstype $cssrc \
			      [Untag $fragmentitems $cstype]]
	    lappend newcsets $fragment
	    $fragment persist

	    if {[$fragment loopcheck]} {
		trouble fatal "[$fragment str] depends on itself"
	    }
	}






	trouble abort?
	return $newcsets
    }

    typemethod itemstr {item} {
	struct::list assign $item itype iid
	return [$itype str $iid]
    }







|
|
>









|
>
|
|
<
|
|
|
>
|
|
<

>
|
<
<

|
|
|
|
|
>
|
|
|
|
|
|
>
|
|
>
|
<
>
>
|



















|









<

|
|
|
|
>
>
>
>
>
|
|







382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404

405
406
407
408
409
410

411
412
413


414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431

432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463

464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
    method timerange {} { return [$mytypeobj timerange $myitems] }

    method drop {} {
	log write 8 csets {Dropping $self = [$self str]}

	state transaction {
	    state run {
		DELETE FROM changeset   WHERE cid = $myid;
		DELETE FROM csitem      WHERE cid = $myid;
		DELETE FROM cssuccessor WHERE cid = $myid;
	    }
	}
	foreach iid $myitems {
	    set key [list $mytype $iid]
	    unset myitemmap($key)
	    log write 8 csets {MAP- item <$key> $self = [$self str]}
	}
	set pos          [lsearch -exact $mychangesets $self]
	set mychangesets [lreplace $mychangesets $pos $pos]
	set pos                    [lsearch -exact $mytchangesets($mytype) $self]
	set mytchangesets($mytype) [lreplace $mytchangesets($mytype) $pos $pos]

	# Return the list of predecessors so that they can be adjusted.

	return [struct::list map [state run {
	    SELECT cid
	    FROM   cssuccessor
	    WHERE  nid = $myid
	}] [mytypemethod of]]
    }


    method reportloop {{kill 1}} {
	# We print the items which are producing the loop, and how.



	set hdr "Self-referential changeset [$self str] __________________"
	set ftr [regsub -all {[^ 	]} $hdr {_}]

	log write 0 csets $hdr
	foreach {item nextitem} [$mytypeobj loops $myitems] {
	    # Create tagged items from the id and our type.
	    set item     [list $mytype  $item]
	    set nextitem [list $mytype $nextitem]
	    # Printable labels.
	    set i  "<[$type itemstr $item]>"
	    set n  "<[$type itemstr $nextitem]>"
	    set ncs $myitemmap($nextitem)
	    # Print
	    log write 0 csets {* $i --> $n --> cs [$ncs str]}
	}
	log write 0 csets $ftr


	if {!$kill} return
	trouble internal "[$self str] depends on itself"
	return
    }

    typemethod split {cset args} {
	# As part of the creation of the new changesets specified in
	# ARGS as sets of items, all subsets of CSET's item set, CSET
	# will be dropped from all databases, in and out of memory,
	# and then destroyed.
	#
	# Note: The item lists found in args are tagged items. They
	# have to have the same type as the changeset, being subsets
	# of its items. This is checked in Untag1.

	log write 8 csets {OLD: [lsort [$cset items]]}
	ValidateFragments $cset $args

	# All checks pass, actually perform the split.

	struct::list assign [$cset data] project cstype cssrc

	set predecessors [$cset drop]
	$cset destroy

	set newcsets {}
	foreach fragmentitems $args {
	    log write 8 csets {MAKE: [lsort $fragmentitems]}

	    set fragment [$type %AUTO% $project $cstype $cssrc \
			      [Untag $fragmentitems $cstype]]
	    lappend newcsets $fragment


	    $fragment persist
	    $fragment determinesuccessors
	}

	# The predecessors have to recompute their successors, i.e.
	# remove the dropped changeset and put one of the fragments
	# into its place.
	foreach p $predecessors {
	    $p determinesuccessors
	}

	return $newcsets
    }

    typemethod itemstr {item} {
	struct::list assign $item itype iid
	return [$itype str $iid]
    }
558
559
560
561
562
563
564

565
566
567
568
569
570
571
	    SELECT tid, name FROM cstype;
	}] { set mycstype($name) $tid }
	return
    }

    typemethod loadcounter {} {
	# Initialize the counter from the state

	set mycounter [state one { SELECT MAX(cid) FROM changeset }]
	return
    }

    typemethod num {} { return $mycounter }

    proc InitializeBreakState {revisions} {







>







594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
	    SELECT tid, name FROM cstype;
	}] { set mycstype($name) $tid }
	return
    }

    typemethod loadcounter {} {
	# Initialize the counter from the state
	log write 2 initcsets {Loading changeset counter}
	set mycounter [state one { SELECT MAX(cid) FROM changeset }]
	return
    }

    typemethod num {} { return $mycounter }

    proc InitializeBreakState {revisions} {
776
777
778
779
780
781
782
783



784
785
786
787
788
789
790
791
792
793
794





795
796
797
798
799
800
801
	set s [lindex $range 0]
	set e [lindex $range end]
	return
    }

    # # ## ### ##### ######## #############

    typevariable mychangesets     {} ; # List of all known changesets.



    typevariable myitemmap -array {} ; # Map from items (tagged) to
				       # the list of changesets
				       # containing it. Each item can
				       # be used by only one
				       # changeset.
    typevariable myidmap   -array {} ; # Map from changeset id to
				       # changeset.

    typemethod all    {}    { return $mychangesets }
    typemethod of     {cid} { return $myidmap($cid) }
    typemethod ofitem {iid} { return $myitemmap($iid) }






    # # ## ### ##### ######## #############
    ## Configuration

    pragma -hastypeinfo    no  ; # no type introspection
    pragma -hasinfo        no  ; # no object introspection








|
>
>
>
|
|
|
|
|






>
>
>
>
>







813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
	set s [lindex $range 0]
	set e [lindex $range end]
	return
    }

    # # ## ### ##### ######## #############

    typevariable mychangesets         {} ; # List of all known
					   # changesets.
    typevariable mytchangesets -array {} ; # List of all known
					   # changesets of a type.
    typevariable myitemmap     -array {} ; # Map from items (tagged)
					   # to the list of changesets
					   # containing it. Each item
					   # can be used by only one
					   # changeset.
    typevariable myidmap   -array {} ; # Map from changeset id to
				       # changeset.

    typemethod all    {}    { return $mychangesets }
    typemethod of     {cid} { return $myidmap($cid) }
    typemethod ofitem {iid} { return $myitemmap($iid) }

    typemethod rev    {}    { return $mytchangesets(rev) }
    typemethod sym    {}    { return [concat \
					  ${mytchangesets(sym::branch)} \
					  ${mytchangesets(sym::tag)}] }

    # # ## ### ##### ######## #############
    ## Configuration

    pragma -hastypeinfo    no  ; # no type introspection
    pragma -hasinfo        no  ; # no object introspection

922
923
924
925
926
927
928




































929
930
931
932
933
934
935
		    set dep($a,$b) .
		    set dep($b,$a) .
		}
	    }
	}
	return
    }





































    # var(dv) = dict (item -> list (item)), item  = list (type id)
    typemethod successors {dv revisions} {
	upvar 1 $dv dependencies
	set theset ('[join $revisions {','}]')

	# The following cases specify when a revision S is a successor







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
		    set dep($a,$b) .
		    set dep($b,$a) .
		}
	    }
	}
	return
    }

    # result = 4-list (itemtype itemid nextitemtype nextitemid ...)
    typemethod loops {revisions} {
	# Note: Tags and branches cannot cause the loop. Their id's,
	# bein of a fundamentally different type than the revisions
	# coming in cannot be in the set.

	set theset ('[join $revisions {','}]')
	return [state run [subst -nocommands -nobackslashes {
	    -- (1) Primary child
	    SELECT R.rid, R.child
	    FROM   revision R
	    WHERE  R.rid   IN $theset     -- Restrict to revisions of interest
	    AND    R.child IS NOT NULL    -- Has primary child
	    AND    R.child IN $theset     -- Loop
	    --
	    UNION
	    -- (2) Secondary (branch) children
	    SELECT R.rid, B.brid
	    FROM   revision R, revisionbranchchildren B
	    WHERE  R.rid   IN $theset     -- Restrict to revisions of interest
	    AND    R.rid = B.rid          -- Select subset of branch children
	    AND    B.rid   IN $theset     -- Loop
	    --
	    UNION
	    -- (4) Child of trunk root successor of last NTDB on trunk.
	    SELECT R.rid, RA.child
	    FROM   revision R, revision RA
	    WHERE  R.rid    IN $theset     -- Restrict to revisions of interest
	    AND    R.isdefault             -- Restrict to NTDB
	    AND    R.dbchild IS NOT NULL   -- and last NTDB belonging to trunk
	    AND    RA.rid = R.dbchild      -- Go directly to trunk root
	    AND    RA.child IS NOT NULL    -- Has primary child.
	    AND    RA.child IN $theset     -- Loop
	}]]
    }

    # var(dv) = dict (item -> list (item)), item  = list (type id)
    typemethod successors {dv revisions} {
	upvar 1 $dv dependencies
	set theset ('[join $revisions {','}]')

	# The following cases specify when a revision S is a successor
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
	    # Consider moving this to the integrity module.
	    integrity assert {$rid != $child} {Revision $rid depends on itself.}
	    lappend dependencies([list rev $rid]) [list rev $child]
	}
	foreach {rid child} [state run "
	    SELECT R.rid, T.tid
	    FROM   revision R, tag T
	    WHERE  R.rid in $theset
	    AND    T.rev = R.rid
	"] {
	    lappend dependencies([list rev $rid]) [list sym::tag $child]
	}
	foreach {rid child} [state run "
	    SELECT R.rid, B.bid
	    FROM   revision R, branch B
	    WHERE  R.rid in $theset
	    AND    B.root = R.rid
	"] {
	    lappend dependencies([list rev $rid]) [list sym::branch $child]
	}
	return
    }








|







|







1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
	    # Consider moving this to the integrity module.
	    integrity assert {$rid != $child} {Revision $rid depends on itself.}
	    lappend dependencies([list rev $rid]) [list rev $child]
	}
	foreach {rid child} [state run "
	    SELECT R.rid, T.tid
	    FROM   revision R, tag T
	    WHERE  R.rid IN $theset
	    AND    T.rev = R.rid
	"] {
	    lappend dependencies([list rev $rid]) [list sym::tag $child]
	}
	foreach {rid child} [state run "
	    SELECT R.rid, B.bid
	    FROM   revision R, branch B
	    WHERE  R.rid IN $theset
	    AND    B.root = R.rid
	"] {
	    lappend dependencies([list rev $rid]) [list sym::branch $child]
	}
	return
    }

1157
1158
1159
1160
1161
1162
1163






1164
1165
1166
1167
1168
1169
1170
    }

    # var(dv) = dict (item -> list (item)), item  = list (type id)
    typemethod successors {dv tags} {
	# Tags have no successors.
	return
    }







    # var(dv) = dict (item -> list (item)), item  = list (type id)
    typemethod predecessors {dv tags} {
	upvar 1 $dv dependencies
	# The predecessors of a tag are all the revisions the tags are
	# attached to, as well as all the branches or tags which are
	# their prefered parents.







>
>
>
>
>
>







1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
    }

    # var(dv) = dict (item -> list (item)), item  = list (type id)
    typemethod successors {dv tags} {
	# Tags have no successors.
	return
    }

    # result = 4-list (itemtype itemid nextitemtype nextitemid ...)
    typemethod loops {tags} {
	# Tags have no successors, therefore cannot cause loops
	return {}
    }

    # var(dv) = dict (item -> list (item)), item  = list (type id)
    typemethod predecessors {dv tags} {
	upvar 1 $dv dependencies
	# The predecessors of a tag are all the revisions the tags are
	# attached to, as well as all the branches or tags which are
	# their prefered parents.
1242
1243
1244
1245
1246
1247
1248

















1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
	return [state run "
	    SELECT IFNULL(MIN(R.date),0), IFNULL(MAX(R.date),0)
	    FROM  branch B, revision R
	    WHERE B.bid IN $theset
            AND   R.rid = B.root
	"]
    }


















    # var(dv) = dict (item -> list (item)), item  = list (type id)
    typemethod successors {dv branches} {
	upvar 1 $dv dependencies
	# The first revision committed on a branch, and all branches
	# and tags which have it as their prefered parent are the
	# successors of a branch.

	set theset ('[join $branches {','}]')
	foreach {bid child} [state run "
	    SELECT B.bid, R.rid
	    FROM   branch B, revision R
	    WHERE  B.bid IN $theset
	    AND    B.first = R.rid
	"] {
	    lappend dependencies([list sym::tag $bid]) [list rev $child]
	}
	foreach {bid child} [state run "
	    SELECT B.bid, BX.bid
	    FROM   branch B, preferedparent P, branch BX
	    WHERE  B.bid IN $theset
	    AND    B.sid = P.pid
	    AND    BX.sid = P.sid
	"] {
	    lappend dependencies([list sym::tag $bid]) [list sym::branch $child]
	}
	foreach {bid child} [state run "
	    SELECT B.bid, T.tid
	    FROM   branch B, preferedparent P, tag T
	    WHERE  B.bid IN $theset
	    AND    B.sid = P.pid
	    AND    T.sid = P.sid
	"] {
	    lappend dependencies([list sym::tag $bid]) [list sym::tag $child]
	}
	return
    }

    # var(dv) = dict (item -> list (item)), item  = list (type id)
    typemethod predecessors {dv branches} {
	upvar 1 $dv dependencies







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>















|








|








|







1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
	return [state run "
	    SELECT IFNULL(MIN(R.date),0), IFNULL(MAX(R.date),0)
	    FROM  branch B, revision R
	    WHERE B.bid IN $theset
            AND   R.rid = B.root
	"]
    }

    # result = 4-list (itemtype itemid nextitemtype nextitemid ...)
    typemethod loops {branches} {
	# Note: Revisions and tags cannot cause the loop. Being of a
	# fundamentally different type they cannot be in the incoming
	# set of ids.

	set theset ('[join $branches {','}]')
	return [state run [subst -nocommands -nobackslashes {
	    SELECT B.bid, BX.bid
	    FROM   branch B, preferedparent P, branch BX
	    WHERE  B.bid IN $theset
	    AND    B.sid = P.pid
	    AND    BX.sid = P.sid
	    AND    BX.bid IN $theset
	}]]
    }

    # var(dv) = dict (item -> list (item)), item  = list (type id)
    typemethod successors {dv branches} {
	upvar 1 $dv dependencies
	# The first revision committed on a branch, and all branches
	# and tags which have it as their prefered parent are the
	# successors of a branch.

	set theset ('[join $branches {','}]')
	foreach {bid child} [state run "
	    SELECT B.bid, R.rid
	    FROM   branch B, revision R
	    WHERE  B.bid IN $theset
	    AND    B.first = R.rid
	"] {
	    lappend dependencies([list sym::branch $bid]) [list rev $child]
	}
	foreach {bid child} [state run "
	    SELECT B.bid, BX.bid
	    FROM   branch B, preferedparent P, branch BX
	    WHERE  B.bid IN $theset
	    AND    B.sid = P.pid
	    AND    BX.sid = P.sid
	"] {
	    lappend dependencies([list sym::branch $bid]) [list sym::branch $child]
	}
	foreach {bid child} [state run "
	    SELECT B.bid, T.tid
	    FROM   branch B, preferedparent P, tag T
	    WHERE  B.bid IN $theset
	    AND    B.sid = P.pid
	    AND    T.sid = P.sid
	"] {
	    lappend dependencies([list sym::branch $bid]) [list sym::tag $child]
	}
	return
    }

    # var(dv) = dict (item -> list (item)), item  = list (type id)
    typemethod predecessors {dv branches} {
	upvar 1 $dv dependencies
Changes to tools/cvs2fossil/lib/c2f_prtopsort.tcl.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
## -*- tcl -*-
# # ## ### ##### ######## ############# #####################
## Copyright (c) 2007 Andreas Kupries.
#
# This software is licensed as described in the file LICENSE, which
# you should have received as part of this distribution.
#
# This software consists of voluntary contributions made by many
# individuals.  For exact contribution history, see the revision
# history and logs, available at http://fossil-scm.hwaci.com/fossil
# # ## ### ##### ######## ############# #####################

## Pass VII. This pass goes over the set of revision based changesets
## and sorts them topologically. It assumes that there are no cycles
## which could impede it, any having been broken by the previous pass,
## and aborts if that condition doesn't hold.

# # ## ### ##### ######## ############# #####################
## Requirements













|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
## -*- tcl -*-
# # ## ### ##### ######## ############# #####################
## Copyright (c) 2007 Andreas Kupries.
#
# This software is licensed as described in the file LICENSE, which
# you should have received as part of this distribution.
#
# This software consists of voluntary contributions made by many
# individuals.  For exact contribution history, see the revision
# history and logs, available at http://fossil-scm.hwaci.com/fossil
# # ## ### ##### ######## ############# #####################

## Pass VIII. This pass goes over the set of revision based changesets
## and sorts them topologically. It assumes that there are no cycles
## which could impede it, any having been broken by the previous pass,
## and aborts if that condition doesn't hold.

# # ## ### ##### ######## ############# #####################
## Requirements

97
98
99
100
101
102
103

104
105
106
107
108
109
110
111
112
113
114
115
	return
    }

    # # ## ### ##### ######## #############
    ## Internal methods

    proc Changesets {} {

	return [struct::list filter [project::rev all] [myproc IsByRevision]]
    }

    proc IsByRevision {cset} { $cset byrevision }

    proc SaveOrder {graph at cset} {
	::variable myatfmt
	::variable mycsfmt

	set cid [$cset id]

	log write 4 rtopsort "Changeset @ [format $myatfmt $at]: [format $mycsfmt [$cset str]] <<[FormatTR $graph $cset]>>"







>
|


<
<







97
98
99
100
101
102
103
104
105
106
107


108
109
110
111
112
113
114
	return
    }

    # # ## ### ##### ######## #############
    ## Internal methods

    proc Changesets {} {
	log write 2 breakscycle {Selecting the revision changesets}
	return [project::rev rev]
    }



    proc SaveOrder {graph at cset} {
	::variable myatfmt
	::variable mycsfmt

	set cid [$cset id]

	log write 4 rtopsort "Changeset @ [format $myatfmt $at]: [format $mycsfmt [$cset str]] <<[FormatTR $graph $cset]>>"
Changes to tools/cvs2fossil/lib/cvs2fossil.tcl.
31
32
33
34
35
36
37

38
39
40
41
42
43
44

# Note: cvs2svn's SortRevisionSummaryPass and SortSymbolSummaryPass
#       are not implemented by us. They are irrelevant due to our use
#       of a relational database proper for the persistent state,
#       allowing us to sort the data on the fly as we need it.

package require vc::fossil::import::cvs::pass::initcsets   ; # Init'ialize C'hange'Sets

package require vc::fossil::import::cvs::pass::breakrcycle ; # Break' R'evision Cycle's
package require vc::fossil::import::cvs::pass::rtopsort    ; # R'evision Top'ological Sort'
package require vc::fossil::import::cvs::pass::breakscycle ; # Break' S'ymbol Cycle's
package require vc::fossil::import::cvs::pass::breakacycle ; # Break' A'll Cycle's
package require vc::fossil::import::cvs::pass::atopsort    ; # A'll Top'ological Sort'

# # ## ### ##### ######## ############# #####################







>







31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

# Note: cvs2svn's SortRevisionSummaryPass and SortSymbolSummaryPass
#       are not implemented by us. They are irrelevant due to our use
#       of a relational database proper for the persistent state,
#       allowing us to sort the data on the fly as we need it.

package require vc::fossil::import::cvs::pass::initcsets   ; # Init'ialize C'hange'Sets
package require vc::fossil::import::cvs::pass::csetdeps    ; # C'hange'Set Dep'endencies
package require vc::fossil::import::cvs::pass::breakrcycle ; # Break' R'evision Cycle's
package require vc::fossil::import::cvs::pass::rtopsort    ; # R'evision Top'ological Sort'
package require vc::fossil::import::cvs::pass::breakscycle ; # Break' S'ymbol Cycle's
package require vc::fossil::import::cvs::pass::breakacycle ; # Break' A'll Cycle's
package require vc::fossil::import::cvs::pass::atopsort    ; # A'll Top'ological Sort'

# # ## ### ##### ######## ############# #####################
Changes to tools/cvs2fossil/lib/pkgIndex.tcl.
13
14
15
16
17
18
19

20
21
22
23
24
25
26
package ifneeded vc::fossil::import::cvs::integrity         1.0 [list source [file join $dir c2f_integrity.tcl]]
package ifneeded vc::fossil::import::cvs::pass              1.0 [list source [file join $dir c2f_pass.tcl]]
package ifneeded vc::fossil::import::cvs::pass::collar      1.0 [list source [file join $dir c2f_pcollar.tcl]]
package ifneeded vc::fossil::import::cvs::pass::collrev     1.0 [list source [file join $dir c2f_pcollrev.tcl]]
package ifneeded vc::fossil::import::cvs::pass::collsym     1.0 [list source [file join $dir c2f_pcollsym.tcl]]
package ifneeded vc::fossil::import::cvs::pass::filtersym   1.0 [list source [file join $dir c2f_pfiltersym.tcl]]
package ifneeded vc::fossil::import::cvs::pass::initcsets   1.0 [list source [file join $dir c2f_pinitcsets.tcl]]

package ifneeded vc::fossil::import::cvs::pass::breakrcycle 1.0 [list source [file join $dir c2f_pbreakrcycle.tcl]]
package ifneeded vc::fossil::import::cvs::pass::rtopsort    1.0 [list source [file join $dir c2f_prtopsort.tcl]]
package ifneeded vc::fossil::import::cvs::pass::breakscycle 1.0 [list source [file join $dir c2f_pbreakscycle.tcl]]
package ifneeded vc::fossil::import::cvs::pass::breakacycle 1.0 [list source [file join $dir c2f_pbreakacycle.tcl]]
package ifneeded vc::fossil::import::cvs::pass::atopsort    1.0 [list source [file join $dir c2f_patopsort.tcl]]
package ifneeded vc::fossil::import::cvs::cyclebreaker      1.0 [list source [file join $dir c2f_cyclebreaker.tcl]]
package ifneeded vc::fossil::import::cvs::project           1.0 [list source [file join $dir c2f_project.tcl]]







>







13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
package ifneeded vc::fossil::import::cvs::integrity         1.0 [list source [file join $dir c2f_integrity.tcl]]
package ifneeded vc::fossil::import::cvs::pass              1.0 [list source [file join $dir c2f_pass.tcl]]
package ifneeded vc::fossil::import::cvs::pass::collar      1.0 [list source [file join $dir c2f_pcollar.tcl]]
package ifneeded vc::fossil::import::cvs::pass::collrev     1.0 [list source [file join $dir c2f_pcollrev.tcl]]
package ifneeded vc::fossil::import::cvs::pass::collsym     1.0 [list source [file join $dir c2f_pcollsym.tcl]]
package ifneeded vc::fossil::import::cvs::pass::filtersym   1.0 [list source [file join $dir c2f_pfiltersym.tcl]]
package ifneeded vc::fossil::import::cvs::pass::initcsets   1.0 [list source [file join $dir c2f_pinitcsets.tcl]]
package ifneeded vc::fossil::import::cvs::pass::csetdeps    1.0 [list source [file join $dir c2f_pcsetdeps.tcl]]
package ifneeded vc::fossil::import::cvs::pass::breakrcycle 1.0 [list source [file join $dir c2f_pbreakrcycle.tcl]]
package ifneeded vc::fossil::import::cvs::pass::rtopsort    1.0 [list source [file join $dir c2f_prtopsort.tcl]]
package ifneeded vc::fossil::import::cvs::pass::breakscycle 1.0 [list source [file join $dir c2f_pbreakscycle.tcl]]
package ifneeded vc::fossil::import::cvs::pass::breakacycle 1.0 [list source [file join $dir c2f_pbreakacycle.tcl]]
package ifneeded vc::fossil::import::cvs::pass::atopsort    1.0 [list source [file join $dir c2f_patopsort.tcl]]
package ifneeded vc::fossil::import::cvs::cyclebreaker      1.0 [list source [file join $dir c2f_cyclebreaker.tcl]]
package ifneeded vc::fossil::import::cvs::project           1.0 [list source [file join $dir c2f_project.tcl]]