[Otterhole]


Otterhole

Personal
 Wedding

Security
 List
 Tools

Career
 Tips
Otterhole :: pfrtg
 

(20050813 Slightly updated version of pfrtg, w/ support for ipf files tested on FreeBSD 4.8, and other types of firewall logs, CSV from a commercial firewall, and with the ability to report on a whole month and not just a day)
(20050622 Slightly updated version of this file, with grammar fixed up a bit and better explanations. Still, this note could be better!)
(20050719 added some notes about tcpdump/pcap, and information about where this has been referenced)


Introduction

I wrote a utility for ipfilter, and changed it to support some graphical-like output (textgraph, I like to call it). I then then changed it for pf, and then to the pf supported in OpenBSD 3.6 (or higher?). Some might like 'real' graphics, but this works for me.

PF . Report . with TextGraph = pfrtg

I must note that while this utility was designed to summarize my pf logs, these logs are held in a slightly modified pcap format, and since this script merely uses the tcpdump utility to extract that data from the log files, this pfrtg program can be used for summarizing any old tcpdump/pcap file.


Example
Date: 1115956848 - 1115999243
Date: 2005/05/13 04:00:48 S - 2005/05/13 15:47:23 S GMT
Date: 2005/05/13 00:00:48 D - 2005/05/13 11:47:23 D Eastern

Source and Port Plus  (1180680 bytes  8716 packets)
 count:          source 45678901n12345678901M123G  bytes  dstn  some ports
 -----:          ------ 012345678901n12345678901E ------  ----  -----
  1500: 0.0.0.0         C...C...C................  72000 252 N 1:  139
   314: 204.60.78.55    ...C.....................  14482 206 N 1:  135
   265: 218.66.104.133  .......C.................  93810 174 N 2:  1026 1027
   264: 222.77.185.228  .....C................... 115896 179 N 2:  1026 1027
   253: 82.161.114.92   ..........C..............  12144 253 N 1:  135
   253: 221.137.120.22  ....C....................  12144 253 N 1:  135
   251: 204.11.152.101  ...C.....................  13052 251 N 1:  135
   248: 84.177.228.28   .........C...............  11904 248 N 1:  135
   247: 12.214.47.89    ..C......................  11856 247 N 1:  135
   213: 218.94.1.226    .......C.................   8520 213 N 1:  1433
   199: 24.201.75.195   ...........C.............   9552  40 N 2:  12345 27374
   125: 222.34.5.45     4XXXXXXXXXX9.............  50500 125 N 1:  1434
   118: 218.56.135.186  ..C......................   4720 118 N 1:  1433
   115: 213.60.4.121    64XXX88XX7X1.............   5520  56 N 1:  135
   104: 208.34.232.79   7674X998XXX..............   4992  52 N 1:  135
    76: 62.206.238.146  X624686866X4.............   3648  36 N 1:  135
    75: 61.129.34.19    667676757675.............  30300  75 N 1:  1434
    72: 211.98.104.7    656576676765.............  29088  70 N 1:  1434
    71: 61.159.15.2     765766765664.............  28684  71 N 1:  1434
    70: 210.74.232.40   664666576765.............  28280  70 N 1:  1434
    69: 61.0.108.132    ........L................   5382  69 N 1:  137
    69: 202.109.140.213 666665666664.............  27876  69 N 1:  1434
    62: 217.166.233.52  .9785X6X5................   2976  47 N 1:  135
    58: 221.210.203.126 .45656565664.............  23432  58 N 1:  1434
    52: 82.52.35.111    ..XXX21.472..............   2496  28 N 1:  135
    49: 61.129.32.96    454554254362.............  19796  49 N 1:  1434
    45: 221.6.238.186   .265XX4..................   2160  24 N 1:  135
    38: 61.152.95.73    234444532223.............  15352  38 N 1:  1434
    38: 219.147.62.71   424333324343.............  15352  38 N 1:  1434
    37: 208.34.232.127  .26X2...293..............   1776  18 N 1:  135
       overflow

Explanation

One is supposed to observe, in this case, that the report was run at 2005/05/13 15:47:23 S GMT or 2005/05/13 11:47:23 D Eastern. Usually, 24 of the hour columns are filled, as I run it after midnight on yesterday's logs.

The first line of interest in the report shows that during the day someone, likely with a Microsoft windows box, sent me 1500 packets from a source address of 0.0.0.0. The packets came in three blocks of over 100 packets, but less than or equal to 1000. The blocks occurred between 12:00-1:00, 4:00-5:00, and 8:00-9:00 AM Eastern Daylight time. The packets were sent to 252 addresses in my class C, and all the packets were sent to port 139 (probably TCP).

The last line is the word 'overflow' which exists because the script was run to show only the top 30 entries by packet count. This is set by an argument.

Monthly examples reports to my DarkNet(HoneyPot?)


Legend

Header

  1. The first three lines are just time stamps displayed in different common formats.
  2. the next line includes the total number of bytes and packets represented by the logs, based on the filter used in the report.
  3. the next lines are column titles, an for the third column, the hour of the day represented with a single character.
    (
    This script assumes that the input log files have a 24hr period, thus, this script might be appropriate to include in the daily security report on BSD.
    crontab entry could be:
      1 2 * * * pfrtg -t 100 0 2>&1
    to get the top 100 count for yesterdays report, run at 1:02 AM.
    )
Columns
  1. The first column is the number of hits for this entry
  2. the second is the source address,
  3. the third column as a single char which represents the number of hits for each hour
  4. the fourth is the total byte count for the entry
  5. the fifth is the number of destination addresses probed
     (I've got a portable class C :-)
  6. the sixth is useless for this forum
  7. the seventh is the number of different ports hit
  8. the eighth is a list of some of the ports hit
Single Character number
  • . = no hits
  • 1-9 = the number of hits in an hour
  • X = 10 to 50 hits in an hour
  • L = 50 to 100 in an hour
  • C = 100 to 1000 hits in an hour
  • M = 1000 or more hits in an hour

Source

The source is ugly. But here it is if anyone has a desire to make it clean: pfrtg.pl
and an older version pfrtg.pl.old.

There are a few arguments, and different outputs that are possible. Another output is more verbose : pfrtgv.txt

If anyone likes this, improves it, wants an updated version with my improvements, has questions, please send a note to openbsd . at . otterhole.ca

20050622 added a copyright statement to the script, in case anyone wanted to make it useful.


References

Places where PFRTG has been referenced:

http://undeadly.org
This is an OpenBSD information site where I first posted information about pfrtg on 20050616. It was a comment for an announced utility called 'Hatchet'. http://undeadly.org/cgi?action=article&sid=20050613165150 Nobody added any comments.
http://www.furl.net/members/Nonesuch/Log Analysis
This site rates software, and on 20050623 gave pfrtg a 2, out of 5 I think.
http://www.geeklan.co.uk/
This is a diary style log where the author talks of PF Statistics items he has looked at on 20050703. He chose something else (Hatchet).

 Copyright Otterhole 2008    last updated: 2008/08/29 23:14  .