BjOG – Bjou’s Blog, that is!

Having to work with Netflow data for my Diploma Thesis I invested quite some time into the following challenge:
Our Routers export Cisco Netflow Data to HOST A, where we do accounting. I want to use HOST B for several Netflow-related tests. The Routers only support one target for their netflow export (as mentioned, this target is HOST A).

Problem: How is it possible to clone the incoming stream of packets at HOST A and forward one copy into HOST A’s userspace (for accounting applications) and the other copy to HOST B’s userspace (for testing purposes)?
The specific challenge is that I do not want a simple FORWARD to HOST B, but a FORWARD of a copy, so that I can work with the data on both machines. This leads to the next problem: Packets arriving at HOST B have the Destination IP address of HOST A in their IP header. We need to rewrite this IP at HOST B so that userspace applications are able to process these packets (which they are not, if the packets are not destined to HOST B’s address).

Note in advance: Despite all efforts this tutorial only works for connectionless udp traffic. A successful 3-way-handshake on HOST A prevents HOST B (despite IP-address rewriting) from accepting the packets in userspace. It just does not work, I appreciate any comments on that. Remember that tee is normally used to clone traffic to another host for passive sniffing and traffic analysis. Note as well, that even if you might want to keep this approach centralized and rewrite the packet’s IP addresses already at HOST A in the POSTROUTING chain, this will not work: Teed packets do not yet show up anywhere within the iptables structures to avoid interfering with the original packet’s table traversal. This is subject to change, though. Thanks to Jan Engelhardt for this information.

So here is how we achieve this goal (tested on Debian Etch stable):
History: There used to be a tee option for an experimental ROUTE target, patchable into iptables with patch-o-matic (pom). This will not work on recent kernels and is deprecated!

This is what we will do on HOST A: Get xtables-addons from http://dev.computergmbh.de:
wget http://dev.computergmbh.de/files/xtables/xtables-combined-1.5.4.1.tar.bz2
This includes a current snapshot of iptables.

Xtables-addons is the proclaimed successor to patch-o-matic(-ng). It
contains extensions that were not accepted in the main Xtables
package.
Xtables-addons is different from patch-o-matic in that you do not have
to patch or recompile either kernel or Xtables(iptables).

Untar, configure, make and make install. Should you run into problems of the kind
warning: #warning You need either CONFIG_NF_CONNTRACK or CONFIG_IP_NF_CONNTRACK
change into your kernel source directory and adapt your kernel. Therefore, look for the Networking option, find the Netfilter (formerly know as ipchains) framework entry and enable the appropriate options. I also ran into problems saying
warning: #warning You have CONFIG_IP_NF_CONNTRACK enabled, but CONFIG_IP_NF_CONNTRACK_MARK or CONFIG_IP_NF_CONNTRACK_SECMARK are not (please enable)
so be sure to enable these options as well in the IP: Netfilter Configuration section.
Save your config and build a new kernel. However, this is not topic of this tutorial.
Should there be other errors because of a special addon, deactivate it in the xtables-addons directory using the mconfig file. Be sure not to deactivate the TEE target, as this is the one we need. The installation success of the xtables-addons may largely depend on the kernel that is being used. If it just won’t work for you with your existing kernel, try another one. I had successful setups on 2.6.23.16 and 2.6.18.6

After successful installation fire the command
iptables -t mangle -A PREROUTING -p udp --dport 9996 -j TEE --gateway <IP of HOST B>.
This command will clone all incoming udp-packets to port 9996 in kernelspace and copy them to HOST B, where we will rewrite the IP addresses. Confirm by typing
iptables -t mangle -L
This will list your rules in the mangle table.
Should there be an error about an unknown table/target/chain, then xtables-addons did not build/install successful, probably because of some missing kernel options.

On HOST B: You do not need xtables-addons here, but only some standard iptables version, as you only need the default DNAT target (Your kernel needs to support it however. Therefore, make sure to have the IPv4 connection tracking support (required for NAT) option enabled in the IP: Netfilter Configuration section of your netfilter kernel category).
iptables -t nat -A PREROUTING -p udp -d <IP of HOST A> --dport 9996 -j DNAT --to-destination <IP of HOST B>:<Port>

That should be it. Now test your setup. You will need three hosts: HOST A and B and another HOST C where you will generate (UDP-) packets. Get a packet generator (I used IP-Packet) and download it to HOST C. Read its documentation, create a config file and fire up your packets to HOST A port 9996. But first, make sure you have a listening process on both, HOST A and B running and waiting for your packets on that specific port. The easiest way will be to use netcat in udp-mode:
nc -ulp9996 on HOST A and on HOST B respective with the port used there. Fire your packets and both netcat instances should receive the UDP payload data. If only HOST A gets them, your tee or DNAT is not working. Debug yourself That’s what I need to do now, as well, because teeing seems to work perfectly fine from one host, but not from the other… Same settings, though, this is just not fair :’(