View Ticket
Not logged in
Ticket UUID: bfe0f34e6ca558c525d3d649f93970e2267b5ff1
Title: catch and yieldto
Type: Bug Version: 8.7
Submitter: pooryorick Created on: 2017-06-05 03:09:20
Subsystem: 16. Commands A-H Assigned To: nobody
Priority: 5 Medium Severity: Minor
Status: Closed Last Modified: 2017-06-06 16:49:42
Resolution: None Closed By: sebres
    Closed on: 2017-06-06 16:49:42
Description:

In a coroutine, when there's a [yieldto] inside a [catch] script, and the thing yielded to produces an error, that error isn't propagated back to the first coroutine:

coroutine c2 ::apply {{} {
    yield
    error "this isn't caught"
}}

coroutine c1 ::apply {{} {
    catch {yieldto c2}
    # Control never gets to this point.
    puts hello
}}

User Comments: sebres added on 2017-06-06 16:49:42:

Below an amend-example for my last comment (as remainder for me)...

% test {yieldto c2} {out "** c2 .. back to c1"; yieldto c1}
  ++ test enter
    ++ c1 enter
      ++ c2 enter
    ** c1
      ** c2
      ** c2 .. back to c1
    -- c1 leave
  ** test
  -- test leave
%
% namespace which -command c2
::c2
% c2
      -- c2 leave
%


sebres added on 2017-06-06 16:41:47:

> Somebody with the NRE "in his cache" might give some insight here.

> For this reason, an error in a coroutine "pops up" back to the closest [coroutine] or resumption

The coroutine is scope-less execution with anchoring on the execution level (without possibility to upvar/uplevel etc).

Sometimes it is just good to imagine "how coroutines work", to take a look on example like this:

% proc out {args} {::puts {*}[lrange $args 0 end-1] [string repeat "  " [expr {[info level]-1}]][lindex $args end]}
%
% out 0:[info level]; apply {{} { 
    out 1:[info level]; apply {{} {
      out 2:[info level] 
    }}
  }}
0:0
  1:1
    2:2
% out 0:[info level]; coroutine c1 apply {{} {
    out 1:[info level]; coroutine c2 apply {{} { 
      out 2:[info level] 
    }}
  }}
0:0
  1:1
  2:1

Another example is good to explain "yieldto":

% proc out {args} {
    set lev [set rlev [expr {[info level]-1}]]
    if {[regexp {^::\D*(\d+)} [info coroutine] _ lev]} {incr lev $rlev}
    ::puts {*}[lrange $args 0 end-1] [string repeat "  " $lev][lindex $args end]
  }
%
% proc test {c1body c2body} {
    out "++ test enter";
    if {[catch {
      coroutine c1 ::apply [list [list [list c1body $c1body] [list c2body $c2body]] {
        out "++ c1 enter"
        coroutine c2 ::apply [list [list [list c2body $c2body]] {
          out "++ c2 enter"
          yield
          out "** c2"
          if 1 $c2body
          out "-- c2 leave"
        }]
        out "** c1"
        if {[catch $c1body r]} {
          out "** c1 !! ERROR: $r"
        }
        out "-- c1 leave"
      }]
    } errMsg]} {
      out "** test !! ERROR: $errMsg";
    }
    out "** test"
    if {[namespace which -command c1] ne ""} {
      out "** test .. continue ::c1"
      c1; # finish c1 if expected
    }
    out "-- test leave";
  }
%
% test {yieldto c2} {out "** c2 .. BOOM"; error "BOOM"}
  ++ test enter
    ++ c1 enter
      ++ c2 enter
    ** c1
      ** c2
      ** c2 .. BOOM
  ** test !! ERROR: BOOM
  ** test
  ** test .. continue ::c1
    -- c1 leave
  -- test leave
% test {yieldto c2} {out "** c2 .. OK"; return OK}
  ++ test enter
    ++ c1 enter
      ++ c2 enter
    ** c1
      ** c2
      ** c2 .. OK
  ** test
  ** test .. continue ::c1
    -- c1 leave
  -- test leave
% test {yieldto c2} {out "** c2 .. back to c1"; yieldto c1}
  ++ test enter
    ++ c1 enter
      ++ c2 enter
    ** c1
      ** c2
      ** c2 .. back to c1
    -- c1 leave
  ** test
  -- test leave
%
% test {c2} {out "** c2 .. BOOM"; error "BOOM"}
  ++ test enter
    ++ c1 enter
      ++ c2 enter
    ** c1
      ** c2
      ** c2 .. BOOM
    ** c1 !! ERROR: BOOM
    -- c1 leave
  ** test
  -- test leave
% test {c2} {out "** c2 .. OK"; return OK}
  ++ test enter
    ++ c1 enter
      ++ c2 enter
    ** c1
      ** c2
      ** c2 .. OK
    -- c1 leave
  ** test
  -- test leave
% test {c2} {out "** c2 .. back to c1"; yieldto c1}
  ++ test enter
    ++ c1 enter
      ++ c2 enter
    ** c1
      ** c2
      ** c2 .. back to c1
    ** c1 !! ERROR: coroutine "c1" is already running
    -- c1 leave
  ** test
  -- test leave
%

Note that the correct way to use "yieldto" explained above in 3th example:

  test {yieldto c2} {out "** c2 .. back to c1"; yieldto c1}

My conclusion also: the subject is indeed not a bug (as you already correct said).
But IMHO the logic of "yieldto" can be extended a bit. I'll try to specify my idea resp. to describe or explain what I want to do later (too many things are currently open).


ferrieux added on 2017-06-05 22:37:24:
Here is a notation that might be illustrative:

 . is normal calling (callstack nesting)
 ; is coro-floor in active stack (defines the left boundary of what gets "put aside" by [yield])
 & is coro-floor in inactive stack

(1) f1 calls f2 calls f3

 f1.f2.f3

(2) f1 calls f2 calls f3 starts coro f4 calls f5

 f1.f2.f3;f4.f5

(3) f1 calls f2 calls f3 starts coro f4 calls f5 yields 

 f1.f2.f3
 &f4.f5

(4) f1 calls f2 calls f3 resumes f4+f5

 f1.f2.f3;f4.f5

(5) f1 calls f2 calls f3 starts coro f6 (after f5 yields again)

 f1.f2.f3;f6
 &f4.f5

(6) f6 then [yieldsto] f4 : essentially a "branch swap":

 f1.f2.f3;f4.f5
 &f6

(7) f5 raises an error: it can be caught in f3, f2, or f1. Note that any number of [yieldto] steps can be inserted, we'll always end up with the same active stack f1.f2.f3;f4.f5 , and all the intermediary yielded-tos will be there as inactive stacks: &f6 , &f7, ...


Bottom line: the "non-propagation" through f6, f7, etc. is just a consequence of the fact that they are "no longer on the path", since [yieldto]s branch-swapping surgically bolts the error-generating branch f4.f5 on top of the active stack f1.f2.f3.

So, in addition to not being a bug, it would seem pretty hard to do otherwise ;-)

pooryorick added on 2017-06-05 16:24:40:
Hmm, tying [catch] and [yieldto] together like that equates to "yieldto, but not really," in which case the programmer can just choose to call the distal coroutine instead of yielding to it.

So I agree, there's no bug here.

pooryorick added on 2017-06-05 16:16:13:
In other words, I'm stumping for [catch] and [try] to be treated as explicit requests from a script that a script that the intepreter treat a return from the distal coroutine as [yieldto] back to the catcher.

pooryorick added on 2017-06-05 16:08:55:

The "rigid connector like [tailcall]" interpretation is valid, but the ability to [catch] an error in the distal coroutine is so useful that I think it should be implemented. If the code author doesn't want that, it's easy enough to keep the [yieldto] out of a [catch] script. Perhaps there is a convincing argument that arranging for catch and yieldto to work this way would upset current semantics or be harmful in some other way, but I haven't come up with any such scenarios yet.


ferrieux added on 2017-06-05 09:47:33:
In yet other words: [yieldto] is a "rigid connector" like [tailcall], keeping the current branch of the current stack tree, while [coroutine/yield] forks a branch off.
For this reason, an error in a coroutine "pops up" back to the closest [coroutine] or resumption, ie the coro floor of the current branch.
So, not a bug, after all.
Agreed ?

ferrieux added on 2017-06-05 09:29:52:
Hum, misread the output. Resumption of the coro floor doesn't really restore the error context, it just gets a stale errorstack. My bad. 

So the workaround is different: to get that distal error you must catch the distal coro floor from the "synchronous invocation" that starts the yieldto chain, which is either the initial one or a resumption:

% coroutine coro3 apply {{} { yield;error CORO3}}
% coroutine coro2 apply {{} { yield;yieldto coro3}}
% catch {coroutine coro1 apply {{} { yieldto coro2}}} e;puts CAUGHT:$e
CAUGHT:CORO3

% coroutine coro3 apply {{} { yield;error CORO3}}
% coroutine coro2 apply {{} { yield;yieldto coro3}}
% coroutine coro1 apply {{} {yield;yieldto coro2}}
% catch coro1 e;puts CAUGHT:$e
CAUGHT:CORO3

ferrieux added on 2017-06-05 09:20:35:
Note: it may well be that the current "distal" behavior is not a bug but a consequence of design choices made to have the simplest implementation. But I ack the fact that it's bad new for (e.g.) a state-machine implementer who would prefer a "proximal" semantics where the closest [yieldto] gets the error...

Somebody with the NRE "in his cache" might give some insight here.
Donal ? Donald ? Kevin ? Sergey ?

ferrieux added on 2017-06-05 09:09:54:
Argh indeed.

I wondered whether the propagation was affected by th enumber of [yieldto] hops, so I tried the following. Same result. But see what happens on resumption of the toplevel coro:

% coroutine coro3 apply {{} { yield;error CORO3}}
% coroutine coro2 apply {{} { yield;yieldto coro3}}
% coroutine coro1 apply {{} { catch {yieldto coro2} e;puts "CORO1-CAUGHT:$e\nCORO1-ERRORSTACK:[info errorstack]"}}
CORO3
% puts [info errorstack]
INNER {returnImm CORO3 {}} CALL {apply {{} { yield;error CORO3}}}

# at this point we've just reproduced the bug with a chain of 2 yieldtos

% coro1
CORO1-CAUGHT:
CORO1-ERRORSTACK:INNER {returnImm CORO3 {}} CALL {apply {{} { yield;error CORO3}}}

# resuming the toplevel "coro floor" seems to restore the error context