tcl_wordchars and tcl_nonwordchars are unknown at Tcl startup.

They are currently auto-loaded along with the commands they control (tcl_wordBreakAfter, tcl_wordBreakBefore, tcl_endOfWord, tcl_startOfNextWord, tcl_startOfPreviousWord). Only when one of these commands are used the file word.tcl where all this is defined is sourced.

That means that ::tcl_wordchars and ::tcl_nonwordchars cannot be used before using one of these commands first.

Therefore, running one of the following commands (for instance) make Tcl aware of tcl_wordchars and tcl_nonwordchars:
    catch {tcl_endOfWord}
    tcl_endOfWord "Hello world" 2  ; # or any parameters
    source [file join [info library] word.tcl]

But this should not be necessary from a user point of view: tcl_wordchars and tcl_nonwordchars should be readily available at Tcl session startup.

Note: this ticket is a follow-up from the following Tk ticket:
    http://core.tcl.tk/tk/tktview/1517768fffffffffffffffffffffffffffffffff

See also this comp.lang.tcl thread:
    http://groups.google.fr/group/comp.lang.tcl/browse_thread/thread/659fd6c1f41d9a81/4a3c30d2c97604cc

Another reference is in the wiki:
    http://wiki.tcl.tk/1655

So long as compatibility demands require us to
continue auto-loading, then yes we should make
it work properly.  The changes to 8.5 and 8.6
are fine.  (The best fix for most problems with
auto-loading is to STOP AUTOLOADING! :))

The trunk change is fine with me too, but it
highlights that a Tk behavior change is getting
made in Tcl.  Anyone using the commands for
some other purpose is going to be affected, but
we can hope that there are few making that mistake,
and hitting them with it in a new minor is acceptable.

Remaining work is in the library.n documentation
which makes explicit claims about default behaviors
on platforms.  The documentation already lies, so now
it's just lying in a different way, but while we're
working here might as well tidy it all up.  I'll get to
that at some point if no one else does.

The other possible "fix" to consider is to establish
constraints so that $::tcl_wordchars and $::tcl_nonwordchars
are forced to remain complements.

Since I'm the one most annoyed by the legacy state of
affairs, I will at some point implement the Tk changes
on a branch for further consideration, and then pursue
the command purge for Tcl 9.

I regret any feelings of abuse that are coming out of this,
but I don't have a strong sense of what I could have done
differently.  When the topic came to my attention, I replied
with what I knew and what I thought best.  At this point,
developing Tcl and Tk are both coding in the presence of
significant constraints, and worst are the unknown constraints
of the expectations of unknown or silent partners in Tcl'ing.
Only thing I can think of to ease the rough bumps is to keep
in mind that we are all working toward the common end of better
Tcl and better Tk and we are getting there.

Well, I don't think DGP means it the way he poses it. A little bit polish doesn't hurt in my opinion.

I prefer to fix this ticket, give Windows the same behavior as UNIX (as Eric Brunel originally requested), and then start thinking what's the best to be done from here.

Closing this ticket as "fixed". I'll consider a TIP, moving all of this to Tk, but removing it from Tcl 8.x is out-of-the question IMHO. Making it work the way it's documented to work is the maximum that can be done in the 8.x line.

Thanks, François, for all your work on this. I appreciate it!

Changing the title of this ticket, as this was the title of the original bug report by Eric Brunel

Thank you for commenting.

As often, these comments pop out when people have already thought about solutions (not just myslef, for once!), implemented them in fossil branches and spent time on them. 

I at least will not spend an additional minute on this topic. I bet this ticket will sleep for another ten years until someone wakes up again being sufficiently annoyed to write the required TIP and loose lots of energy to support it. Good luck, sincerely.

We should stop polishing our mistakes and destroy them.

These commands have been a packaging error since day one
(7.5 / 4.1, I believe -- explains the primitive lack of namespacing).

Since they've been in Tcl, they've been the source of confusion
a number of times, with people believing they were the key
to configure the parser, or the regexp engine, or the string
matcher or any number of things that deal with "words".
They do none of those things.  They are not used by Tcl or
its set of built-in commands at all.

These are Tk utility commands.  They have no business being
part of Tcl.  They belong in Tk.

Actually, it sounds like they don't even belong in Tk anymore
because reportedly the era that required a platform-specific
UI notion of what a "word" is has passed from the scene.

The autoloading mess that appears to be the origin of these
tickets is another matter, but one which will disappear if
we simply require Tk to keep its own gear.

I will vigorously oppose all the ideas proposed on this ticket so far.
(The effort is appreciated, but wrong is wrong.)
I'll happily support any scheme that sets up the complete destruction
of these commands in Tcl 9, while keeping Tk compatibly functional.

Actually ... I would like to change the default for tcl_wordchars/tcl_nonwordchars to "\w" resp "\W" for all platforms: Nowaydays most Windows applications appear to work like that, so it would be better for Tcl/Tk 8.7 to adapt accordingly. At TIP is needed for that.

So, how about your solution for trunk and up, mine for core-8-6-branch and core-8-5-branch? That way we minimize the backwards compatibility risk for patch releases, but move in the right direction for trunk and up.

Well, not my opinion. An app could well need to know the value of tcl_wordchars at startup, not just set it at startup.

Alternative solution committed in bug-f1253530cd-alt branch.

The way I understand the original bug report (in comp.lang.tcl): The problem is not that the mentioned variables should be available at start-up. The real problem is that if a user modifies the value of those variables it has no effect (contrary to what's documented) because the value is overwritten as soon as word.tcl is autoloaded.

Advantage: This solution can be backported without compatibility risk to core-8-6-branch and core-8-5-branch too, I wouldn't do that with your solution. Sorry ;-)

The fix consists in moving the definition of tcl_wordchars / tcl_nonwordchars from word.tcl to the initialization code of the Tcl interp, in TclpSetVariables().

Patch committed in branch bug-f1253530cd for review.