Ticket Change Details
Not logged in
Overview

Artifact ID: 3228d41af0ad515f2a9e6ee378b73c155819ed406adb2d4b1444c5861da87b8b
Ticket: 578155d5a19b348dc1a9fe96cc2c067a59326a89
Very rare bug (segfault) if set variable (with error case) using self-releasable object as new value
User & Date: pooryorick 2023-03-30 19:01:35
Changes

  1. icomment:
    While diagnosing and fixing [6d4e9d1af5|memory leak: SetFsPathFromAny, assisted
    by the global literal table, causes a Tcl_Obj to reference itself], I spent
    days looking into this and similar reports.  I tried various experiments to
    come up with an example that would confirm this report, ultimately spending a
    fair chunk of time only to determine that it is baseless.  Now it's time to put
    this ticket to rest so that the next developer doesn't have to spend their time
    on it too.  [510663a99e3a096b] is a smart commit and a good commit:  It fixes
    the real bug, as described by msofer for [1334947]:  The real bug is that not
    all failure paths clear 0-ref objs.  It also made Tc_ObjSetVar2 do more so that
    the caller can do less.  The "TCL_OWN_OBJREF" invention proposed by sebres
    doesn't solve any known problem so there is no reason to adopt it.  The
    ad-hominem assertion, "someone doesn't understand something (or doesn't want
    spent more time to go deeper)" is simply another indicator that there is no
    substantial argument to be made here on behalf of this report.
    
    Now for a basic explanation of reference counting in Tcl: It is well documented
    and well understood that in order to preserve a Tcl_Obj a caller increments its
    reference count before passing it to another procedure, and later either
    decrements the reference count or documents that it has incremented the
    reference count so that its own caller can decrement it.  It's common for
    procedures to increment and then decrement a Tcl_Obj reference count, and
    Tcl_ObjSetVar2 does nothing more than the equivalent of that, and its not
    alone.  If a procedure knows that nothing else is going to decrement the
    reference count in the meantime, then the performant way to do the equivalent
    of incrementing and then decremening the reference count is to only increment
    it on success, and then on failure to handle the only case where decrementing
    it would do anything:  The case in which the reference count is zero.  That's
    all that Tcl_ObjSetVar2 does.  It's elegant and correct.
    
    Now to rebut the statements made by sebres so far:
    
    "we can be not sure what happens with reference in Tcl_ObjSetVar2"
    
    <blockquote>
    Yes, we can be sure:  The reference count of the Tcl_Obj is incremented, and in
    the failure case, it is decremented.
    </blockquote>
    
    "The function checks the refCount only AFTER trying of set the variable (which
    can change the refCount of the object by calling of the traces..."
    
    <blockquote>
    False.  As dgp explained, in the failure case Tcl_ObjSetVar2 decrements the reference count and returns before any trace runs.
    </blockquote>
    
    "Much worse it looks just like undefined behavior."
    
    <blockquote>
    False.  Tcl_ObjSetVar2 performs the equivalent of incrementing the variable,
    and then decrementing it in the error case.
    </blockquote>
    
    
    "For the developer using Tcl_ObjSetVar, it is very bad up to impossible to act plausible in error case, because he don't know when he should decrease refCount in error case or not..."
    
    <blockquote>
    False.  The caller either increments the Tcl_Obj prior to passing it to Tcl_ObjSetVar2 and arranges for it to be decremented later, or just doesn't increment it and allows Tcl_ObjSetVar2 to manage it.
    </blockquote>
    
    "Thus either he do this nevertheless and can get segfault, or he produces intentionally a memory leak (quasi accepts the risk of possible leak)."
    
    <blockquote>
    False.  If the caller wants to use the Tcl_Obj after passsing it to
    Tcl_ObjSetVar2, it simply increments the reference count first, and then
    decrements it later, or passes it back to its own caller, explaining that the
    reference count has been incremented.  Commit [510663a99e3a096b], which sebres
    is complaining about, is actually the commit that fixed this type of issue, but
    sebres doesn't seem to be aware of that.  </blockquote>
    
    "this behavior is undocumented,"
    
    <blockquote>
    This was not false at the time it was written, but in general developers should know to increment the reference count of a Tcl_Obj before passing it to a function they don't control.
    </blockquote>
    
    "You don't know what the object "objPtr" is (it is not yours, you'll get it from foreign code and from other people),
    so it can have refCount == 0, 1 and larger as 1.
    How you can set it to the variable varName, inside this function and what you'll do in error case.
    If you'll increment it, possibly you can (unwanted) destroy it in error case by the paired decrement.
    If you don't increment it, you can not be sure what do you want to do in the "
    
    <blockquote>
    If someone else passes you a Tcl_Obj with a refCount of 0, then they are
    expecting you to clean it up if needed, so incrementing it and decrementing it
    is not a problem.  If you want it to be unshared in order to efficiently modify
    the internal representation, then modify the Tcl_Obj and then, depending on
    your needs, either increment the reference count or don't before passing it to
    Tcl_ObjSetVar2, which handles the unshared object correctly in the error case.
    It's starting to sound like the bug you're complaining about is in the design
    of your own system.
    </blockquote>
    
    "Meanwhile I've implemented both things for my own tcl-fork, and it works fine and saves me 
    many time, work and code-lines (not to mention the readability)."
    
    <blockquote>
    You are welcome to commit high-quality work.  If it's good, people will be glad
    to have it in the implementation o Tcl.  No need for a bug report.  The code
    will suffice.  If it's not a good contribution it will be duly reverted.
    </blockquote>
    
    "or additionally the "obscure" hack with auto-decrement in the error case will remain"
    
    <blockquote>
    It's not an obscure hack.  It's simply the equivalent of incrementing the
    reference count and then decrementing it in the error case.
    </blockquote>
    
    
    The most recent code presented by sebres as an example of the issue he is
    reporting is actually just an example of code that is itself buggy: 
    
    <code><verbatim>
    Tcl_Obj * CacheSetObj(Tcl_Obj *varName, Tcl_Obj *objPtr)
    {
      Tcl_ObjSetVar2(interp, cache, varName, objPtr, 0);
      return objPtr;
    }
    
    ...
    
    if ((objPtr = CacheGetObj(someName)) != NULL)
      return objPtr;
    return CacheSetObj(someName, SomethingToObtainObj(...));
    </verbatim></code>
    
    The example failed to make use of the result of Tcl_ObjSetVar2.  As the
    documentation warns, it is not correct to simply return objPtr becuause the
    Tcl_Obj * returned by Tcl_ObjSetVar2 may be an entirely different object.  The
    correct code would look like this:
    
    <code><verbatim>
    Tcl_Obj * CacheSetObj(Tcl_Obj *varName, Tcl_Obj *objPtr)
    {
      return Tcl_ObjSetVar2(interp, cache, varName, objPtr, 0);
    }
    
    ...
    
    if ((objPtr = CacheGetObj(someName)) != NULL)
      return objPtr;
    return CacheSetObj(someName, SomethingToObtainObj(...));
    </verbatim></code>
    
    Notice that there is no reason for CacheSetObj to manipulate the reference
    count of objPtr at all, and that Tcl_ObjSetVar2 correctly cleans up objPtr if
    needed.
    
    sebres has had over five years to come up with some working code that
    demonstrates some memory leak, some segmentation fault, or some other
    unexpected behaviour that justifies this report, and has so far failed.   This
    report will now be closed as "invalid".  My opinion is that sebres is entirely
    wrong about this issue.  The most likely explanation is that he is misusing the
    Tcl API and then blaming problems on Tcl.  If he wishes to reopen this repor,
    sebres should supply a complete working example that reproduces an issue.
    
  2. login: "pooryorick"
  3. mimetype: "text/x-fossil-wiki"