Some initial data from a real-time LaBrea data processing program

While I'm waiting a call back [1], some more on LaBrea [2].

Yesterday (from January 16 at 06:28:25 to January 17 08:54:50) LaBrea generated 1.1G (gigabytes) of log data, and it took full five minutes to run grep 'Initial Connect' daemon.log.0 | wc -l (255,344 new tarpitting connections by the way).

LaBrea was also running at full speed, maxed out at 64Kbps (Kilobits per second) bandwidth to keep all these connections tarpitted (the maximum I set LaBrea to use, by the way).

[NETWORK TARPIT bandwidth usage] [3]

That first large dip in the graph (the one around 6:30 in the morning) is probably due to the system attempting to rotate a 1.1G log file. The second dip, at the right (around 3:00 pm) is when I restarted LaBrea so its logging information would go through ltpstat [4]. After an hour of running:

>
```
Start: Tue Jan 17 14:55:59 2006 End: Tue Jan 17 15:55:59 2006 Running time: 1h
Pool-max: 1048576
Pool-num: 24322
Rec-max: 1048576
Rec-num: 24322
UIP-max: 1048576
UIP-num: 1282
Reported-bandwidth: 40 (Kb/sec)
```

And after two hours:

>
```
Start: Tue Jan 17 14:55:59 2006 End: Tue Jan 17 16:56:19 2006 Running time: 2h 20s
Pool-max: 1048576
Pool-num: 33326
Rec-max: 1048576
Rec-num: 33326
UIP-max: 1048576
UIP-num: 1632
Reported-bandwidth: 40 (Kb/sec)
```

And right this second:

>
```
Start: Tue Jan 17 14:55:59 2006 End: Tue Jan 17 18:37:19 2006 Running time: 3h 41m 20s
Pool-max: 1048576
Pool-num: 42931
Rec-max: 1048576
Rec-num: 42931
UIP-max: 1048576
UIP-num: 2148
Reported-bandwidth: 40 (Kb/sec)
```

Okay, pool-max and rec-max are the maximum sizes for the structure array and pointer array, and both should always be equal at all times (I'm displaying this number more for debugging purposes than anything else), while pool-num and rec-num (which should also be equal at all times) represent the current number of connectioned tarpitted. I also keep track of unique IP (Internet Protocol)s, which just now is currently 2,148 (out of 1,048,576 that I can store). I also just found out that IP address 195.130.152.85 has 4,809 connections currently tarpitted (and in the few seconds it took to do that query, five more connections were tarpitted).

I'll be releasing this code in the next few days, when I can write up some documentation and slap on a license.

[1] /boston/2006/01/17.3

[2] http://sourceforge.net/projects/labrea

[3] /boston/2006/01/17/tarpit.20060117.png

[4] /boston/2006/01/17.2

Gemini Mention this post

Contact the author