Marpa

Changes On Branch bad-build-order-trouble
Login

Changes On Branch bad-build-order-trouble

Patch Side-by-Side Diff
Tcl 2016 Conference, Houston/TX, US, Nov 14-18
Send your abstracts to tclconference@googlegroups.com by Sep 12.

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Changes In Branch bad-build-order-trouble Excluding Merge-Ins

This is equivalent to a diff from 08e6e9634d to 514199b8ed

2017-10-15
16:50
Fixed problems in the handling of charclass as set of code-points and -ranges. Range validation was incomplete, allowing bad input to crash. Fixed, and tests added. Tracing as well, plus more notes when certain code paths will be reached. check-in: ac18987fd3 user: aku tags: trunk
04:46
Moving critcl after tcl solves OSX issue with install dependency order. Check if this breaks linux. Leaf check-in: 243e280f60 user: aku tags: build-order-trouble
04:42
Moving critcl after tcl solves OSX issue with install dependency order. Check if this breaks linux. Leaf check-in: 514199b8ed user: aku tags: bad-build-order-trouble
2017-10-12
06:59
Tcl runtime. Fix flush issue where partial flush and redo needs recognition check-in: f26d4f328e user: aku tags: flush-fix
2017-10-11
05:28
Mark recognizer cons/dest points better check-in: 08e6e9634d user: aku tags: trunk
2017-10-06
22:01
Equivalent changes in the C runtime. 1. The C runtime already inter-twined tree extraction, valuation and hand-over which was added to the Tcl runtime in the previous commit. 2. Fixed same issue with possible L0 discards after G1 end. 3. Fixed bad assertions in symset, byteset, exposed by 2. check-in: 32c320340a user: aku tags: trunk

Changes to build.tcl. 
1
2
3
4


5

1
2


3
4
5



-
-
+
+


#!/usr/bin/env kettle
# -*- tcl -*-
kettle critcl3
kettle tcl
kettle tcl
kettle critcl3
kettle tclapp bin/marpa-gen
Changes to runtime-c/byteset.c.
︙
36
37
38
39
40
41
42

43

44
45
46
47
48
49
50

36
37
38
39
40
41
42
43

44
45
46
47
48
49
50
51








+
-
+








}

int 
marpatcl_rtc_byteset_contains (marpatcl_rtc_byteset* s, unsigned char c)
{
    // sizeof (unsigned char) == 8 --> max(c) = 255, no assertion required
    TRACE_FUNC ("((byteset*) %p, byte %d)", s, c);
    // ATT: assert not required for (unsigned char) == byte, always true
    ASSERT (c < MARPATCL_RTC_BSMAX, "Symbol beyond set capacity");
    //ASSERT (c < MARPATCL_RTC_BSMAX, "Symbol beyond set capacity");
    TRACE_RETURN ("%d", (XL [c] < SZ) && (DE [XL [c]] == c));
}

Marpa_Symbol_ID*
marpatcl_rtc_byteset_dense (marpatcl_rtc_byteset* s)
{
    TRACE_FUNC ("((byteset*) %p)", s);
︙
Changes to runtime-c/spec.c.
︙
308
309
310
311
312
313
314

315
316
317
318
319
320
321

308
309
310
311
312
313
314
315
316
317
318
319
320
321
322








+








	    TRACE_RETURN ("(sym*) %p (len 0)", NULL);
	}
	TRACE ("(len*) %p = %d", len, *len);
	ASSERT_BOUNDS (1+offset+1+length-1, coding->size);
	TRACE_RETURN ("(sym*) %p", coding->data + 1 + offset + 1);
    }
    ASSERT (0, "Unsupported type of g1 coding");
    TRACE_RETURN ("(sym*) %p FAIL", 0);
}

marpatcl_rtc_stack_p
marpatcl_rtc_spec_setup_rd (marpatcl_rtc_rules* s)
{
#define PUSH(detail)					\
    marpatcl_rtc_stack_push (rule_data, pc - s->rcode); \
︙
Changes to tests/unicode.test.
︙
144
145
146
147
148
149
150




















151
152
153
154
155
156
157

144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177








+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+








} {
    incr k
    test marpa-unicode-norm-class-2.$k "norm-class, $charclass" -body {
	marpa unicode norm-class $charclass
    } -result $normalized
}


set k 0
foreach {charclass message notes} {
    {-1}          {Point out of range (0...*): -1} {out of range, under}
    {500000000}   {Point out of range (0...*): 500000000} {out of range, over, regardless of mode}
    {2 {3 3 4} 4} {Expected 2-element list for range} {too many elements for interval/range}
    {2 {} 4}      {Expected 2-element list for range} {not enough element for interval/range}
    {2 {-1 3} 4}  {Range (start) out of range (0...*): -1} {out of range, under}
    {2 {3 500000000} 4} {Range (end) out of range (0...*): 500000000} {out of range, over, regardless of mode}
    {2 {5 2} 4} {Range empty (end (5) before start (2))} {}
    {2 {{} 3} 4} {expected integer but got ""} {}
    {2 {3 {}} 4} {expected integer but got ""} {}
    {bogus} {Expected 2-element list for range} {}
} {
    incr k
    test marpa-unicode-norm-class-3.$k "norm-class, bad $charclass" -body {
	marpa unicode norm-class $charclass
    } -returnCodes error -result $message -match glob
}

# # ## ### ##### ######## ############# #####################
## negate-class - Complement a char class.

test marpa-unicode-negate-class-1.0 {negate-class, wrong args, not enough} -body {
    marpa unicode negate-class
} -returnCodes error -result {wrong # args: should be "marpa unicode negate-class charclass"}
︙
Changes to unicode/cc_objtype.tcl.
︙
259
260
261
262
263
264
265

266

267
268
269
270
271
272

273
274
275


276
277
278
279
280
281
282
283
284
285

286
287
288









289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307

308
309
310
311

312
313
314

315
316
317
318
319
320
321

259
260
261
262
263
264
265
266

267
268
269
270
271
272
273
274
275
276

277
278
279
280
281
282
283
284
285
286
287
288
289
290
291

292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336








+
-
+






+


-
+
+










+


-
+
+
+
+
+
+
+
+
+



















+




+



+








	    goto fail;
	}

	scr = marpatcl_scr_new (objc);
	TRACE ("CAP %d", objc);
	for (i = 0; i < objc; i++) {
	    Tcl_Obj* elt = objv[i];
	    TRACE ("PROCESS. [%02d] %p", i, elt);

	    
	    /*
	    // First handle objects which already have a suitable type.
	    // No conversions required, only data extraction and validation.
	    */

	    if (elt->typePtr == ctx->intType) {
		TRACE ("INT. ... [%02d] %p", i, elt);
		Tcl_GetIntFromObj(ip, elt, &start);

	    process_int:
		process_int:
		TRACE ("INT. CHK [%02d] %p", i, elt);
		if (marpatcl_scr_bad_codepoint (ip, "Point", start)) {
		    goto fail;
		}
		TRACE ("++ (%d)", start);
		marpatcl_scr_add_range(scr, start, start);
		continue;

	    }

	    if (elt->typePtr == ctx->listType) {
		TRACE ("LIST ... [%02d] %p", i, elt);
		Tcl_ListObjGetElements(ip, elt, &robjc, &robjv);

	    process_list:
		process_list:
		TRACE ("LIST CHK [%02d] %p", i, elt);
		if (robjc != 2) {
	    #define MSG "Expected 2-element list for range"
		    Tcl_SetErrorCode (ip, "MARPA", NULL);
		    Tcl_SetObjResult (ip, Tcl_NewStringObj(MSG,-1));
		    goto fail;
	    #undef MSG
		}
		if ((Tcl_GetIntFromObj (ip, robjv[0], &start) != TCL_OK) ||
		    marpatcl_scr_bad_codepoint (ip, "Range (start)", start) ||
		    (Tcl_GetIntFromObj (ip, robjv[1], &end) != TCL_OK) ||
		    marpatcl_scr_bad_codepoint (ip, "Range (end)", end) ||
		    marpatcl_scr_bad_range(ip, start, end)) {
		    goto fail;
		}
		TRACE ("++ (%d...%d)", start, end);
		marpatcl_scr_add_range(scr, start, end);
		continue;
	    }

	    /*
	    // While object has no suitable type, it may be
	    // convertible to such. Those which are convertable get
	    // dispatched to the handlers above.
	    */

	    if (Tcl_GetIntFromObj(ip, elt, &start) == TCL_OK) {
		TRACE ("INT. CVT [%02d] %p", i, elt);
		goto process_int;
	    }

	    if (Tcl_ListObjGetElements(ip, elt, &robjc, &robjv) == TCL_OK) {
		TRACE ("LIST CVT [%02d] %p", i, elt);
		goto process_list;
	    }

	    TRACE ("NO.. CVT [%02d] %p", i, elt);
	    /*
	    // No suitable type, and not convertible to such either.
	    */

	    Tcl_SetErrorCode (ip, "MARPA", NULL);
	    Tcl_SetObjResult (ip, Tcl_NewStringObj("Expected codepoint or range, got neither",-1));
	    goto fail;
︙
Changes to unicode/marpa_unicode.tcl.
︙
70
71
72
73
74
75
76




77
78
79
80
81
82
83

70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87








+
+
+
+









critcl::subject marpa unicode {character class} charclass case-folding

# # ## ### ##### ######## #############
## Implementation.

critcl::tcl 8.5

critcl::ccode {
    TRACE_OFF;
}

# # ## ### ##### ######## #############
## Declare the Tcl layer

critcl::tsources tcl.tcl          ; # Tcl-level operations
critcl::source   unicode.tcl      ; # C-level support functions.
critcl::source   cc_objtype.tcl   ; # ObjType for uni-char classes (SCR).
︙