It gets annoying and tiresome to hold down the mouse button for long periods of time.  We should instead make it so that you click to activate or deactivate the mouse joystick.  The only caveat is that this behavior is undesirable for touch surfaces as you already must continue touch the screen to move around.  So the behavior should not be changed for touch events, only for the actual mouse events.   This should not be that hard.