505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
|
@ <p><b>Exceptions to anti-robot restrictions</b><br>
@ The entry below is a list of regular expressions, one per line.
@ If any of these regular expressions match the input URL, then the
@ request is exempt from anti-robot defenses. Use this, for example,
@ to allow scripts to download release tarballs using a pattern
@ like:</p>
@ <p>
@   <tt>^/tarball\\b*\\b(version-|release)\\b</tt>
@ <p>The pattern should match against the REQUEST_URI with the
@ SCRIPT_NAME prefix removed, and with QUERY_STRING appended following
@ a "?" if QUERY_STRING exists. (Property: robot-exception)<br>
textarea_attribute("", 3, 80,
"robot-exception", "rbexcept", "", 0);
@ <hr>
|
|
|
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
|
@ <p><b>Exceptions to anti-robot restrictions</b><br>
@ The entry below is a list of regular expressions, one per line.
@ If any of these regular expressions match the input URL, then the
@ request is exempt from anti-robot defenses. Use this, for example,
@ to allow scripts to download release tarballs using a pattern
@ like:</p>
@ <p>
@   <tt>^/tarball\b*\b(version-|release)\b</tt>
@ <p>The pattern should match against the REQUEST_URI with the
@ SCRIPT_NAME prefix removed, and with QUERY_STRING appended following
@ a "?" if QUERY_STRING exists. (Property: robot-exception)<br>
textarea_attribute("", 3, 80,
"robot-exception", "rbexcept", "", 0);
@ <hr>
|