Marpa

Timeline
Login
Tcl 2016 Conference, Houston/TX, US, Nov 14-18
Send your abstracts to tclconference@googlegroups.com by Sep 12.

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

50 most recent check-ins

2018-09-22
00:01
[3dbe995b99] Leaf: Added larger doctools examples from Tcllib. Modified dt helper to show new statistics. Untested. (user: andreask tags: io-statistics)
2018-09-21
22:26
[aa1749989d] Completed IO statistics for rt-C. Fixed name clash in the new code (existing type vs new function, function renamed). Updated tests to match generator changes (cparse, clex). (user: andreask tags: io-statistics)
00:00
[23a1ea88eb] Added state and methods to collect and return IO statistics (#streams, #processed, #input). Implemented for rt-Tcl. Added tests. rt-C placeholders. (user: andreask tags: io-statistics)
2018-09-20
01:54
[7337a35e08] Leaf: Merge staging into main stream. Basic parse events (io, lexer) done. Basic input extension done. Doctools example working (macros, includes) json blocked out (issues with partial utf, non-utf, non-standard codings - Tcl vs. C pre-handling, strings vs. file) (user: aku tags: trunk)
2018-09-18
17:18
[15feed5264] Closed-Leaf: Extended multi-byte char counting test to full unicode range. (user: aku tags: trunk-stage)
05:01
[d336725a4a] Fix trailing whitespace (user: aku tags: trunk-stage)
05:00
[44fee72091] Fixed miscounting of multi-byte characters in rtC. Reworked `inbound_step()`. Changec to test a character for correctness on the first byte, then arrange to (not) step the character location on the trailer bytes following, if any. Always steps the character location on the first byte. Updates the byte/char index the same time. Further reworked and fixed the interface from `gate` to `inbound` for rewinding after a rejection (straight end, or when the matched lexeme is actually shorter than the number of bytes consumed before stopping). The used API (`moveby`) was character based, which caused us to rewind to far, namely to the previous character. No trouble when it was a single-byte character. Bad for a multi-byte however. This could then trip the lexer if the first byte of the previous character is not acceptable at the given point. (user: aku tags: trunk-stage)
2018-09-17
23:43
[9337c00616] Added test demonstrating that rtC miscounts multi-byte character locations (because counting at end). Added tools to help with generation of test input (user: aku tags: trunk-stage)
23:41
[1694246f20] Tweaked test parser gen to leave a log file behind on grammar parse failure (user: aku tags: trunk-stage)
2018-09-15
04:46
[e64b244e12] Trim trailing whitespace (user: aku tags: trunk-stage)
2018-09-10
23:05
[8b8a1cedf6] Leaf: Added runtime tests. Note: These look to need some of the fixes on trunk-stage. (user: andreask tags: non-utf-bytes)
22:23
[2d87d5c92b] Started implementation of non-utf byte support. (user: andreask tags: non-utf-bytes)
2018-09-09
06:00
[cd54178989] Fixed mindt|doctools crashes. Two for asserts. Problem with `mindt` was in the new `gather_events` function. It was not fully adapted to the changed data structures, with the split trigger and event arrays. It saved the loop index as event id, which was correct for the old structures. With the new structures OTOH the event id is a field in the array we iterate over instead. Problem with `doctools` was as above, plus also an unknown mismanagement of the `always` array by the shared rtC generator code for LTM-mode lexemes, i.e LATM=false. It triggered a similar assertion on lexer setup after the fix above was applied. `mindt` avoided that issue because it did not use LTM-mode lexemes for its special forms. Reactivated the disabled tests. Passing now. Note: json still disabled. (user: aku tags: trunk-stage)
00:02
[cba29d80fc] Integrated doctools example - tests - remeta Regenerated doctools example parser to match runtime changes. Disabled tests with known problems - json (char mislocation) - mindt (crash! (rtc, mis-capacity for some symset)) - doctools (crash! (rtc, mis-capacity for some symset)) (user: aku tags: trunk-stage)
2018-09-08
22:16
[ecf881268e] Integrate base event work. Stage before true merge. (user: aku tags: trunk-stage)
22:15
[77d4610d6a] Closed-Leaf: Integrate base doctools example. (user: aku tags: parse-events)
22:11
[d912ded2f3] Closed-Leaf: Brought trunk and event work. (user: aku tags: language-doctools)
22:07
[6ceda1b56e] Brought trunk work (user: aku tags: parse-events)
21:38
[f97a8147f6] Tkt [d9b47bbb78] completed (user: aku tags: parse-events)
09:04
[a48220cd05] Bring in the extended test suite which runs the example tests too. (user: aku tags: trunk)
04:23
[26a81da805] Closed-Leaf: More integration of the language examples into the main - Run example tests with the main - Regenerate example parsers with the main parsers Updated a few tests. Note: json tests have issues. Parse location differences. Investigate. Tracked the location differences to commit [dd2dfa9d07530d8d] where definition of lexeme start switched from byte-based LEX_START to character-based LEX_START_C. This commit is on branch [parse-events-rtc]. I suspect that the stepping of character locations is incorrect. Looking at the `step()` function I see the character increment done for the last byte of the character. This is ok for the singletons. For multi-byte characters OTOH it looks to be too late. We should step at the first byte of a character. Complicating things, partial characters, i.e. missing trailers, unexpected trailers. json is good to find these issues, its test suite contains many of these cases. Fix will be done on new branch. This commit is not responsible, can be merged. (user: aku tags: example-integration)
2018-09-07
23:12
[67d5a8317b] Leaf: Integrated the language example tests into the main. Added the language example parsers to the `remeta` tool to ensure that they are updated after runtime API changes too, not just the system parsers. Dropped the replicated support files, redirected test suites to the main support as the only support. Note: The json example has issues, major testsuite fail. Bisect where this started and fix. - Looks to be lexeme location information differences between rtC and rtTcl. - Might be fixed in the parse-event branch. (user: andreask tags: mistake, example-test-integration)
19:05
[7dbbfd79f5] Closed-Leaf: Fixed handling of `args` parameter in wrong#args error messages. The text is dependent on core version. -- There is more going on -- Critcl differences, Core differences -- Try later again -- (user: andreask tags: mistake)
04:42
[e71840b49a] The code generating symbol names for literals was still subject on the exact content of the [:control:] charclass, causing differences in output depending on the version of Tcl used (seen between 8.5.18 and 8.5.19). Now ensured version-independent quoting in all the relevant places. Fixed order of conditions too. Cross-referenced these places. Regenerated the internal parsers (slif, literals). Updated tests. (user: aku tags: trunk)
2018-09-06
19:29
[3b692749d7] Closed-Leaf: Fix missed bogus `static const` in test results. (user: andreask tags: tkt-d9b47bbb78-event-management)
16:32
[e2021e3ef5] Ticket [d9b47bbb78]. Fix event management in the generators. C runtime and generator updated for the new data structures. Tests updated. (user: aku tags: tkt-d9b47bbb78-event-management)
2018-09-04
19:53
[69f8e6628c] Ticket [d9b47bbb78]. Fix event management in the generators. Tcl runtime and generator updated for the new data structures. Tests updated. C runtime tests broken. Next: C runtime and generator. (user: aku tags: tkt-d9b47bbb78-event-management)
2018-08-31
23:12
[5b0bdf9591] Ticket [d9b47bbb78]. Fix event management in the generators. Extended to fix the event data structures in general. - Modified container, grammar base, grammar classes - Event information split into declaration with state (events), and usage information (triggers). Events are managed in the main container class. Triggers are managed in the grammar base class. Code is common to L0 and G1. API used by the semantics is unchanged. Serialization structure changes. Event section is global. Trigger sections under L0, G1. Used a purpose-written tool to rewrite the old serializations in tests results (gcstate, gcr_*) to the new form. Reworked the SLIF generator to use the new form of serialization. Tests pass, no results changed. Fixed testsuite oops (generatec_gc(c|tcl)) not seen in trunk because everything with gc also had gc_c, etc. GC generators are ok as well. Extended testsuite to cover grammars with events. (user: andreask tags: tkt-d9b47bbb78-event-management)
19:46
[5bf97382b1] Closed-Leaf: Ticket [d9b47bbb78]. Fix event management in the generators. rt-C generator ... scrap ... Working on this makes me realize that my data structures are still not right. Re-reading the documentation at https://metacpan.org/pod/distribution/Marpa-R2/pod/Event.pod I came back with the impression that the user-facing event names are indeed in a single scope. It is not spelled out explicitly, but I can't see it work if not. Especially with runtime (de)activation of events it must be. Which means that having separate structures for G1 and L0 is wrong. The core event information (name -> activation) status must be shared. The trigger tables can be separate, should be even. Saving this for posterity, and restarting (from the container outward) (user: andreask)
19:36
[83c23635b6] Ticket [d9b47bbb78]. Fix event management in the generators. rt-Tcl generator updated. Tests updated. Split the more complex internal formatting commands into more composable pieces, and re-composed them. (user: andreask)
07:39
[d4619268cc] Ticket [d9b47bbb78]. Fix event management in the generators. Tests fully updated. Started update of generators. - slif ok - tparse in the works (user: aku)
2018-08-30
23:32
[8c77fd896a] Ticket [d9b47bbb78]. Fix event management in the generators. Changed container data structures. Separated event definition from event usage by symbols (triggers). Attention: This changes the grammar serialization. Started update of tests. TODO: Update generators. (user: andreask)
21:21
[4cd32b9b44] Ticket [d9b47bbb78]. Fix event management in the generators. Prep work, a test. As part of the expected results this also contains an implied draft spec for various changed data structures (engine, slif container), and semantic processing. (user: andreask tags: mistake)
06:03
[783110305c] Merge latest to doctools work (user: aku tags: language-doctools)
06:02
[783f0feb8b] Merge latest to event work (user: aku tags: parse-events)
06:01
[cf5be67f4a] Merge work back to main. (user: aku tags: trunk)
06:00
[ef8ad60e6d] Closed-Leaf: Completed the pass. (user: aku tags: test-cleanup)
00:07
[4f1c1e6f02] Start some cleanup of testsuite, some simplifications for var handling of the loop helpers ... (user: andreask tags: test-cleanup)
2018-08-29
06:59
[36533140c7] Updated main line with continued event work. (user: aku tags: trunk)
06:56
[c5b2707fc3] Updated example (user: aku tags: language-doctools)
06:54
[16d33a63ac] Completed work on the ticket, integrated with base. (user: aku tags: parse-events)
06:53
[31becc141c] Closed-Leaf: Added test for parser reset. (user: aku tags: tkt-a5abdaacd4-shift-test-reset)
2018-08-28
23:57
[d8bfe7d142] Document i-gen a bit more. Additional helpers for one-shot grammar tests outside of the big loops. (user: andreask tags: tkt-a5abdaacd4-shift-test-reset)
22:37
[230aebacc3] Ticket [a5abdaacd4]. Shift engine reset to start of parsing. Keep state for introspection (statistics, progress reports, etc.) post parse. Needs testcases, before/after introduction of reset. (user: andreask tags: tkt-a5abdaacd4-shift-test-reset)
05:33
[6563481eae] Get trunk and doc work into current example. (user: aku tags: language-doctools)
05:32
[1e2d209efa] Pull doc work into official code. (user: aku tags: trunk)
04:40
[4b364c62cf] Final grammar fixes based on processing the Tcllib manpages. 100% pass (429 pages. Note, three down in the total, had include files caught in the simple find) - Allow backslash in general text (and examples). - Allow continuation lines in markup commands. - Allow whitespace after the last element of `tclwords` sequences. - Fixed oops in the include file handling, failed to process the last character of the included file. This was ok when it was a closing newline. Bad for files without such, i.e. ending in the closing bracket of a command. - Allow section command with two arguments. Same as subsection. - Allow 'args' as alias for 'arguments' (list type). - Tweak character escapes, recognize double-backslash. - Allow empty list elements (No text between element markers) - Paragraph sequences are improper (allow tailing separator) Fixed ca 20 tcllib manpages which had issues the older doctools system is not catching (para before first list item, bad closing brackets, silent swallowing of bogus markup commands, ...). (user: aku tags: language-doctools)
2018-08-26
04:52
[d8022c427d] Leaf: Added documentation for package `marpa::util`. Timing/Sequencing missing. Regenerated embedded docs. (user: aku tags: more-documentation)
2018-08-25
20:39
[2f04a2d613] Completed documentation for `marpa::unicode. Regenerated embedded docs to pull in this and previous changes. Completed ticket [a82dc50647]. (user: aku tags: more-documentation)
05:05
[ac96c4984d] More filling in of the manpage for `marpa::unicode`. (user: aku tags: more-documentation)