Misc notes
- To change the unix time stamp to put a bar between seconds and microseconds:
cat FILENAME | sed 's/\(^[0-9]*\)\./\1|/g'
- The run_bro.sh shell script replaces all timestamps with two fields - seconds and microseconds
- On a 10.1GB PCAP on machine with 6 proc, 8GB RAM, running the various bro policy scripts as separate processes takes 13:41, maxes out approx 3GB RAM. RAM usage seems to run up and down as different policy scripts finish. Running all of them in one instance takes 20:11, takes up max of approx 3GB RAM
- In the same test, it looks like ftp, dns and ssl all finish in approx 3:42, bt-tracker after 5:41, which should help free up some resources. conn-dv finishes after 10:27, the slowest one is http. This could just be a symptom of the type of traffic being examined
Output formats
- conn-dv.bro output format to conn-dv.log:
start time|duration|orig IP|resp IP|service|orig port|resp port|transport layer proto|orig bytes|resp bytes|content 1g entropy|connection state|flags
- dns-dv.bro output format to dns-dv.log
time|session_id|orig IP|orig port|resp IP|resp port|DNS details string
- dns-dv.bro output to dns_a_responses.log
time|session_id|orig IP|orig port|resp IP|resp port|A|host name|IP of host|Other DNS details
** NOTE: Need to do a grep -v "<query addl" to clean up the output of dns_a_responses.log
- ftp-dv.bro output format to ftp-dv.log:
start time|ftp session id|orig IP|orig port|resp IP|resp port|FTP message
- http-dv.bro output - http_req.log - REQUESTS (triple quotes used to escape any weird GET requests)
time~~~~~~session id~~~~~~orig IP~~~~~~orig port~~~~~~resp IP~~~~~~resp port~~~~~~method~~~~~~URI~~~~~~1g entropy
- http-dv.bro output - http_rep.log - REPLIES (triple quotes used to escape any weird GET requests)
time~~~~~~session id~~~~~~orig IP~~~~~~orig port~~~~~~resp IP~~~~~~resp port~~~~~~request~~~~~~request 1g entropy~~~~~~code~~~~~~reason~~~~~~content length~~~~~~body length~~~~~~interrupted?~~~~~~Content gap (bytes)
- http-dv.bro output - http_headers.log - HEADERS (triple quotes used to escape any weird header values)
time~~~~~~session id~~~~~~orig IP~~~~~~orig port~~~~~~resp IP~~~~~~resp port~~~~~~direction~~~~~~header name~~~~~~header name 1g entropy~~~~~~header value~~~~~~header value 1g entropy
- os-fingerprint-dv.bro ouput to software-dv.log
time|client IP|message about software being used
TODO: Right now, 'client IP' field also has the word 'client' at the end.. don't know why. Need to figure out a way to clean that up
- bt-tracker-dv.bro ouput to bt-tracker-dv.log
time|tracker ID|orig IP|orig port|resp IP|resp port|tag|direction|Bittorrent message
NOTE: The 'infohash' (in some of the bittorrent messages) is actually the hash of the torrent, you can google it and find out what was downloaded
- ssl-dv.bro ouput to ssl-dv-conns.log
time|session ID|orig IP|orig port|resp IP|resp port|start
- ssl-dv.bro ouput to ssl-dv.log
time session_ID SSL_msg
NOTE: need to do some post processing on this because bro's output only has spaces