Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
32 check-ins
|
2013-12-21
| ||
| 00:29 | improving gui and responsiveness Leaf check-in: af7df126ca user: jrogers tags: trunk | |
|
2013-12-12
| ||
| 02:50 | add "qack" - "qa crawl keeper" - gui version of crawler. initial version check-in: 9677abf989 user: jrogers tags: trunk | |
| 02:50 | track link sources check-in: 533d3f7731 user: jrogers tags: trunk | |
| 02:49 | track source of links check-in: 3a5c1e1ed0 user: jrogers tags: trunk | |
| 02:49 | refactor for command-line options add port mapping ability check-in: 0613cc06e1 user: jrogers tags: trunk | |
| 02:47 | cleaning up validation check-in: 1a8c2fe6e4 user: jrogers tags: trunk | |
| 02:46 | add google-rank type thing check-in: 0201026d7a user: jrogers tags: trunk | |
|
2013-09-12
| ||
| 23:39 | removed vwait and update commands replaced jiffy accounting with a more simple scheme that should have about the same effect Leaf check-in: 1232ac487d user: pooryorick tags: pyk | |
|
2013-07-23
| ||
| 18:51 | added start of css parser check-in: c7e676d1d5 user: jrogers tags: trunk | |
|
2013-06-07
| ||
| 01:18 | millis -> micros for better resolution check-in: afbb61c5d9 user: jrogers tags: trunk | |
|
2013-06-06
| ||
| 22:31 | change "clock clicks" to "clock millis" - stop wraparound problems check-in: 645a77d908 user: jrogers tags: trunk | |
| 22:30 | update from other working copy check-in: 85181060bb user: jrogers tags: trunk | |
|
2013-01-03
| ||
| 19:56 | keep all references to a link, to report multiple sources for broken links check-in: 68b5b842f8 user: jrogers tags: trunk | |
|
2012-07-26
| ||
| 00:04 | improve robots loading check-in: d20be14de2 user: jrogers tags: trunk | |
|
2012-07-25
| ||
| 04:55 | add host column to db check-in: 492d5f6766 user: jrogers tags: trunk | |
| 03:20 | add crawlq main script check-in: fa773831f5 user: jrogers tags: trunk | |
| 01:21 | break apart db-queued multithread crawler check-in: c024a6f394 user: jrogers tags: trunk | |
|
2012-07-19
| ||
| 07:14 | initial add of crawlq (sqlite database backed crawler) Leaf check-in: 46ba98ec58 user: jeffr tags: trunk | |
|
2012-03-27
| ||
| 22:26 | add getattr proc for consistently parsing attributes check-in: 6fbc67eeb0 user: jrogers tags: trunk | |
|
2011-10-18
| ||
| 22:02 | added gradual increasing of rate/load check-in: 6db86c82d9 user: jrogers tags: trunk | |
|
2011-10-17
| ||
| 22:15 | micro-tuning for MAXIMUM speed. It's now SLIGHTLY faster. Sometimes. check-in: 674060936b user: jrogers tags: trunk | |
|
2011-10-13
| ||
| 19:24 | added ability to get paramters from file. Some other tweaks too. check-in: ebe1c999fc user: jrogers tags: trunk | |
|
2011-01-27
| ||
| 00:22 | improve usage message; trap case where no urls passed check-in: 5ab2331b23 user: jrogers tags: trunk | |
|
2010-10-28
| ||
| 22:35 | reorganizing check-in: 97f9221388 user: jrogers tags: trunk | |
| 22:25 | added webtest files check-in: 25755d4d64 user: jrogers tags: trunk | |
| 19:58 | imported various versions of redirecting proxy check-in: 45d0b16402 user: jrogers tags: trunk | |
| 19:37 | rearranging check-in: b22c5cad1f user: jrogers tags: trunk | |
| 19:15 | remove some junk check-in: 49d7692eca user: jrogers tags: trunk | |
| 18:41 | initial empty check-in Leaf check-in: d54447df75 user: evilotto tags: trunk | |
| 18:36 | added flag to write broken links to a file detect broken links (404 errors) handle get timeouts better handling of anchor-only links implemented 'exit after time' flag track link sources to command line and through redirects check-in: 91d87a70fd user: jrogers tags: trunk | |
|
2010-03-30
| ||
| 23:04 | initial add check-in: 0fd3433a50 user: jrogers tags: trunk | |
| 23:03 | initial empty check-in check-in: 87bea19dca user: jrogers tags: trunk | |