Resolved

Hi, I believe I've located a problem when using advanced NAT and ticking the 'not' destination option!

I've manage to reproduce the bug on my test machine so at least I can make this happen at will.

My configuration is:

• Compaq Deskpro EN SFF PII 350MHz
• 128MB RAM
• 64MB CF card in a CF to IDE adapter
• On board Intel 82558 ethernet adapter as LAN (fxp0)
• 3com 3c905C ethernet adapter (I've also tried a 3c905B) as WAN (xl0)

I'm starting with a bare installation of 1.3b15, changing the LAN address range to 192.168.235.0/24 with a LAN address 192.168.235.1, changing the LAN DHCP range to match the new address range and then changing the WAN address to a spare IP address I have in a /29 address range.

With advanced NAT disabled performance is absolutely fine and as expected.

Enabling advanced outbound NAT and adding the following rule (which is exactly the same as advanced NAT being disabled) results in expected performance:

Interface       Source            Destination         Target
WAN             192.168.235.0/24  *                   *

ipnat -lv displays:

List of active MAP/Redirect filters:
map xl0 192.168.235.0/24 -> 0.0.0.0/32 proxy port ftp ftp/tcp
map xl0 192.168.235.0/24 -> 0.0.0.0/32 portmap tcp/udp 1024:64535
map xl0 192.168.235.0/24 -> 0.0.0.0/32
map xl0 from x.y.z.3/32 to any port = 53 -> 0.0.0.0/32 tcp/udp

However, adding the following rule shows the problem (which shouldn't even trigger for this test):

Interface       Source            Destination         Target
WAN             192.168.235.0/24  ! 192.168.0.0/24    *

When the above rule is in effect I'm seeing packets being dropped and the client doing a retransmit and eventually the request is completed but not until after a substantial delay (about 3 seconds!).

ipnat -lv displays:

List of active MAP/Redirect filters:
map xl0 from 192.168.235.0/24 ! to 192.168.0.0/24 -> 0.0.0.0/32 proxy port ftp ftp/tcp
map xl0 from 192.168.235.0/24 ! to 192.168.0.0/24 -> 0.0.0.0/32 portmap tcp/udp 1024:64535
map xl0 from 192.168.235.0/24 ! to 192.168.0.0/24 -> 0.0.0.0/32
map xl0 from x.y.z.3/32 to any port = 53 -> 0.0.0.0/32 tcp/udp


I'm testing an HTTP connection to my ADSL router which requests authentication immediately without the inverted NAT rule but take approximately 3 seconds with the inversion in place!.

This is repeatable and in fact I've tried it several times just to make sure I'm changing the bare minimum options to hopefully make the problem easier to diagnose!

I've searched for FreeBSD 6.3 bugs related to this issue but not discovered anything!  I've checked out the FreeBSD man page for ipnat and it doesn't actually list being able to invert the destination as an option!

Is anyone else able to confirm this bug?

I've done a fairly un-scientific test on all of the 1.3 beta releases!

I can report that this NAT bug appears to affect 1.3b1 thru 1.3b15 so it
looks to be a FreeBSD 6.2/6.3 bug!

My testing consisted of accessing one specific web site with and without
the advanced NAT enabled and measuring the total time to load the page. 
With advanced NAT enabled, page load times ranged from 13 to 47 seconds
whereas with advanced NAT disabled page load times were 2 to 7 seconds! 
Quite a difference!

The testing with the NAT rule was performed with a single NAT rule that
had a target of 'NOT 192.168.0.0/24' which should always hit as I've not
got a 192.168.0.0/24 network connected to the m0n0wall!

I was able to reproduce this bug - it's exactly as you describe it, and it's pretty odd. I also see ICMP Destination Host Unreachable packets sent from the system while the TCP connection hangs at the beginning. Definitely looks like a bug in ipnat/ipfilter to me; I've done some cursory looking at sys/contrib/ipfilter/netinet/ip_nat.c, but wasn't able to spot anything that could cause this yet. Looks like more in-depth debugging will be necessary to track this one down - could get nasty.

BTW - it seems to be related to specifying a destination in the NAT rule, not to whether it's inverted (I could also make it happen by specifiying 0.0.0.0/0 as the destination).

Another thing to do would be to report this on the ipfilter mailing list, but judging from prior experience, it only makes sense to post there if one already has a working patch...

Manuel,

Many thanks for confirming that it's not me doing something stupid!  I'm surprised no one else has noticed it!

I've not used FreeBSD before but I'll try to take a look at the source code to see if I can spot anything but if you can't see anything then it's unlikely I will!

I'll also check out the ipfilter mailing list and see if I can report this there.  Is there anything that I need to quote and will they want this tested on a stock FreeBSD 6.3 or do you think reporting it against the versions on m0n0wall will suffice?

Do you think this might be fixed in 6.4?

Many thanks once again (especially for m0n0wall itself).

Neil.

I'll also check out the ipfilter mailing list and see if I can report this there.  Is there anything that I need to quote and will they want this tested on a stock FreeBSD 6.3 or do you think reporting it against the versions on m0n0wall will suffice?

Giving the output from ipnat -l should be enough for anyone to reproduce it, but reproducing it under a stock FreeBSD 6.3 (and saying so ;) might make it more credible. m0n0wall has a few kernel patches that make it different from a stock FreeBSD kernel and that could theoretically play a role here, so it might be a good idea to do that, just in case.

Do you think this might be fixed in 6.4?

Unfortunately not, as it uses the same version of ipfilter (4.1.28) as 6.3. I've checked the change log of the latest ipfilter version (4.1.30), but the changes don't seem to reflect anything close to this bug. :(

I've now investigated this problem, and found the following:

- When dealing with NAT proxy rules that include from/to match statements, the destination port specified with "proxy port xxx" in the rule is not compared against the actual destination port in the packet. This caused the automatically generated FTP proxy rule to match TCP connections on any port (not just 21) when a destination address match was being used in an advanced outbound NAT rule. That in turn caused a huge slowdown and general weirdness attributed to the fact that everything was processed by the FTP proxy (even if it wasn't FTP).

The offending code is in /sys/contrib/ipfilter/netinet/ip_nat.c, around line 3848:

Code:

if (*np->in_plabel != '\0') {
    if (((np->in_flags & IPN_FILTER) == 0) &&
        (np->in_dport != tcp->th_dport))
            continue;
    if (appr_ok(fin, tcp, np) == 0)
        continue;
}

The IPN_FILTER flag is set whenever there is a from/to statement in the NAT rule; in_plabel contains a string if the NAT rule refers to a proxy.

I've considered filing an ipfilter bug report about this, but have decided against it, since one could argue that this is intended behavior. Instead, m0n0wall now adds a destination port match statement to FTP proxy rules with source/destination match statements.

To sum up - this bug will be fixed in 1.3b17. Thanks for the report, and sorry for taking so long to investigate and fix this.