Firewall/NAT

Hello all,

First of all, I am a m0n0wall newbie.  Great product from what I have seen already.  I am having a problem that is baffling me.  I am not a network newbie, and am familier with the PIX firewalls, however, I must be missing something really small.  Here is what I am trying to do.

I have 5 Internet static IP addresses (29 bit).  I have 3 SMTP/www servers and do not want to use 1:1 NAT.  I have configured my Server NATs with my external IP addresses.  I have set up the Inbound NAT to direct the traffic.  I let it automatically configure the Rule.  It all looks good and makes "logical" sense, however, from the Internet (my broadband card on laptop) I cannot access the service that I just allowed through the firewall. 

However, here is one thing I noticed, if I configure the external address to be the Interface address, then I can access my service.  If I change it to another on of my external IP address (already configured from NAT server), and it fails again not allowing any access from the outside.  Nothing is showing up in the log as well which has left me scratching my head.

Any ideas will be greatly appreciated.  I am hoping that it is something really tiny that I have overlooked (and overlooked and overlooked.....).

Thanks!
Russell

Ps. Here is my config.


<?xml version="1.0"?>
<m0n0wall>
    <version>1.6</version>
    <lastchange>1175921996</lastchange>
    <system>
        <hostname>firewall</hostname>
        <domain>abc.com</domain>
        <username>xxxxx</username>
        <password>xxxxx</password>
        <timezone>America/Chicago</timezone>
        <time-update-interval>300</time-update-interval>
        <timeservers>pool.ntp.org</timeservers>
        <webgui>
            <protocol>https</protocol>
            <port>443</port>
        </webgui>
        <dnsserver>151.164.1.8</dnsserver>
        <dnsserver>151.64.142.149</dnsserver>
        <dnsserver>38.8.82.2</dnsserver>
    </system>
    <interfaces>
        <lan>
            <if>fxp0</if>
            <ipaddr>192.168.1.1</ipaddr>
            <subnet>24</subnet>
            <media/>
            <mediaopt/>
        </lan>
        <wan>
            <if>fxp2</if>
            <mtu/>
            <media/>
            <mediaopt/>
            <spoofmac/>
            <blockpriv/>
            <ipaddr>1.2.3.149</ipaddr>
            <subnet>29</subnet>
            <gateway>1.2.3.150</gateway>
        </wan>
        <opt1>
            <if>fxp1</if>
            <descr>DMZ</descr>
            <ipaddr>10.10.10.1</ipaddr>
            <subnet>24</subnet>
            <bridge/>
            <enable/>
        </opt1>
    </interfaces>
    <staticroutes/>
    <pppoe/>
    <pptp/>
    <bigpond/>
    <dyndns>
        <type>dyndns</type>
        <username/>
        <password/>
        <host/>
        <mx/>
        <server/>
        <port/>
    </dyndns>
    <dnsupdate/>
    <dhcpd>
        <lan>
            <range>
                <from>192.168.1.100</from>
                <to>192.168.1.199</to>
            </range>
        </lan>
        <opt1>
            <range>
                <from>10.10.10.100</from>
                <to>10.10.10.199</to>
            </range>
            <defaultleasetime/>
            <maxleasetime/>
            <enable/>
        </opt1>
    </dhcpd>
    <pptpd>
        <mode/>
        <redir/>
        <localip/>
        <remoteip/>
    </pptpd>
    <dnsmasq>
        <enable/>
    </dnsmasq>
    <snmpd>
        <syslocation>Office</syslocation>
        <syscontact>xxxx xxxx</syscontact>
        <rocommunity>public</rocommunity>
        <enable/>
        <bindlan/>
    </snmpd>
    <diag>
        <ipv6nat>
            <ipaddr/>
        </ipv6nat>
    </diag>
    <bridge/>
    <syslog>
        <reverse/>
        <nentries>50</nentries>
        <remoteserver/>
        <nologdefaultblock/>
    </syslog>
    <nat>
        <servernat>
            <ipaddr>1.2.3.145</ipaddr>
            <descr>External x.145</descr>
        </servernat>
        <servernat>
            <ipaddr>1.2.3.146</ipaddr>
            <descr>External x.146</descr>
        </servernat>
        <servernat>
            <ipaddr>1.2.3.147</ipaddr>
            <descr>External x.147</descr>
        </servernat>
        <servernat>
            <ipaddr>1.2.3.148</ipaddr>
            <descr>External x.148</descr>
        </servernat>
        <advancedoutbound/>
        <rule>
            <external-address>1.2.3.146</external-address>
            <protocol>tcp</protocol>
            <external-port>80</external-port>
            <target>onc01</target>
            <local-port>80</local-port>
            <interface>wan</interface>
            <descr>WAN -&gt; onc01 http</descr>
        </rule>
    </nat>
    <filter>
        <rule>
            <type>pass</type>
            <interface>wan</interface>
            <protocol>tcp</protocol>
            <source>
                <any/>
            </source>
            <destination>
                <address>onc01</address>
                <port>80</port>
            </destination>
            <log/>
            <descr>NAT WAN -&gt; onc01 http</descr>
        </rule>
        <rule>
            <type>pass</type>
            <descr>Default LAN -&gt; any</descr>
            <interface>lan</interface>
            <source>
                <network>lan</network>
            </source>
            <destination>
                <any/>
            </destination>
        </rule>
    </filter>
    <shaper/>
    <ipsec/>
    <aliases>
        <alias>
            <name>onc01</name>
            <address>192.168.1.6</address>
            <descr>onc01</descr>
        </alias>
    </aliases>
    <proxyarp/>
    <wol/>
</m0n0wall>

Since your /29 is not routed through your m0n0wall (i.e. the WAN interface itself uses one of the IP addresses from that /29), you'll have to set up Proxy ARP for the additional 4 IP addresses on your m0n0wall. It even says so on the "Server NAT" page:

Note:
The external IP addresses defined on this page may be used in inbound NAT mappings. Depending on the way your WAN connection is setup, you may also need proxy ARP.

Great!  I will test and will post the results.  I thought I tested it out. 

Thanks again!  And I think I am going to love m0n0wall!  Compared to the PIX, it is so easy to setup....

Russell

Unfortunately it is still not working...  Any other ideas?   

Thanks!
Russell

Do you have access to whatever router is upstream of your m0n0wall, or is that your ISP's? You may have to clear the ARP cache on it, or power cycle it to accomplish the same end result. Also I would be curious if it shows the proxy ARP'ed IP/MAC pairs properly in its ARP cache, if you have access.

though in all likelihood if you still have the box plugged in it's probably timed out its cache by now, though it's still worth trying power cycling it.

I have powered off the DSL modem and left it off for about a minute.  That should be enough time to clear the cache.  It is still not working.  Any more suggestions?  I even tried a different PC to see if that had any effect on it.....   

What does the proxy ARP section of your config look like?

Hello cmb,

Thanks for your help.  Here is a copy of my Proxy ARP. 


    <proxyarp>
        <proxyarpnet>
            <interface>lan</interface>
            <network>192.168.1.1/24</network>
            <descr>Proxy ARP for LAN</descr>
        </proxyarpnet>
        <proxyarpnet>
            <interface>wan</interface>
            <network>xxx.xxx.xxx.149/29</network>
            <descr>Proxy ARP for Outside</descr>
        </proxyarpnet>
    </proxyarp>


I have read the documentation and am confused.  Do I need to have a Proxy ARP entry for each of my servers? 

Thanks again for your help!
Russell

Turn off proxy ARP for your LAN. Wow, I'm surprised anything works with that.    Actually, that might be part of your problem. Can those hosts even get out to the Internet and communicate properly within the network? With that configuration, any host that ARP queries on your LAN is going to get two responses - one from the actual machine and one from m0n0wall with its LAN MAC address.

The only thing you need proxy ARP for is IP's that don't have a machine that'll answer ARP requests and m0n0wall needs to handle the traffic on their behalf for those IP's. To simplify that, in your situation, you only need proxy ARP for public IP's.

First, remove the LAN proxy ARP. Then, I would suggest rebooting m0n0wall and all your machines, or clearing all their ARP caches manually. Otherwise you could have some interesting remnants of that config around that could cause problems. Then see if it works as you desire.

If that doesn't work, delete your WAN proxy ARP as well (even though it looks correct) and try setting up proxy ARP for just one of your additional public IP's (not your WAN IP). Does Server NAT for that particular IP then work?

Thanks for the information.  I will turn off the Proxy ARP for the LAN.  Since I could not make it work, I reverted back to my old firewalls, thus how I am able to work.

Thanks for the information about the Proxy ARP, I did not fully understand it from the documentation.  I do have a question, I am using the basic DSL modem, could it be possible that there is something that is blocking it?

I have not had a chance to test it out.  When I get to, I will let you all know the outcome.

Thanks!
Russell

I asked you to do it that way with the WAN IP's because I seem to recall someone reported a bug a while back where if your proxy ARP included your WAN IP (which your /29 setup does) then proxy ARP wouldn't work.

It's possible it's something in the DSL modem, but at this point from what you've said and what you've tried I doubt that to be the case.

Well, I finally got a chance to give it a try.  It still did not work.  I am beginning to feel that it is my little DSL modem that is causing the problem.  And, unfortunately, I cannot get into it to take a look at it.  I have another DSL modem, I am going to give it a try and see if I can make it work.

Any other suggestions is greatly appreciated.  Thanks!

Hello all,

I have spent hours on this with out any luck.  Currently I am using multiple (cheap) routers (linksys) to accomplish what I need to do.  However, I do not like that, and want to be able to use m0n0wall.  But for some reason, it is not working for me.  I have a 29 bit (5 static) public IP addresses.  And if it was not because I play around with different email servers, I could easily get by with 1:1 NAT.  What I want to use is Inbound NAT.  I am not sure what the deal is, but here is my configuration.


<?xml version="1.0"?>
<m0n0wall>
    <version>1.6</version>
    <lastchange>1176445243</lastchange>
    <system>
        <hostname>firewall</hostname>
        <domain>oncinc.com</domain>
        <username>russell</username>
        <password>xxxxx</password>
        <timezone>America/Chicago</timezone>
        <time-update-interval>300</time-update-interval>
        <timeservers>pool.ntp.org</timeservers>
        <webgui>
            <protocol>https</protocol>
            <port>443</port>
            <certificate/>
            <private-key/>
            <expanddiags/>
        </webgui>
        <dnsserver>151.164.1.8</dnsserver>
        <dnsserver>151.64.142.149</dnsserver>
        <dnsserver>38.8.82.2</dnsserver>
    </system>
    <interfaces>
        <lan>
            <if>fxp0</if>
            <ipaddr>192.168.1.1</ipaddr>
            <subnet>24</subnet>
            <media/>
            <mediaopt/>
        </lan>
        <wan>
            <if>fxp2</if>
            <mtu/>
            <media/>
            <mediaopt/>
            <spoofmac/>
            <blockpriv/>
            <ipaddr>xxx.xxx.xxx.149</ipaddr>
            <subnet>29</subnet>
            <gateway>xxx.xxx.xxx.150</gateway>
        </wan>
        <opt1>
            <if>fxp1</if>
            <descr>OPT1</descr>
        </opt1>
    </interfaces>
    <staticroutes/>
    <pppoe/>
    <pptp/>
    <bigpond/>
    <dyndns>
        <type>dyndns</type>
        <username/>
        <password/>
        <host/>
        <mx/>
        <server/>
        <port/>
    </dyndns>
    <dnsupdate/>
    <dhcpd>
        <lan>
            <range>
                <from>192.168.1.100</from>
                <to>192.168.1.199</to>
            </range>
        </lan>
        <opt1>
            <range>
                <from>10.10.10.100</from>
                <to>10.10.10.199</to>
            </range>
            <defaultleasetime/>
            <maxleasetime/>
            <enable/>
        </opt1>
    </dhcpd>
    <pptpd>
        <mode/>
        <redir/>
        <localip/>
        <remoteip/>
    </pptpd>
    <dnsmasq>
        <enable/>
    </dnsmasq>
    <snmpd>
        <syslocation>Office</syslocation>
        <syscontact>Russell Patterson</syscontact>
        <rocommunity>public</rocommunity>
        <bindlan/>
    </snmpd>
    <diag>
        <ipv6nat>
            <ipaddr/>
        </ipv6nat>
    </diag>
    <bridge/>
    <syslog>
        <reverse/>
        <nentries>50</nentries>
        <remoteserver>192.168.1.90</remoteserver>
        <filter/>
        <dhcp/>
        <portalauth/>
        <vpn/>
        <system/>
        <enable/>
    </syslog>
    <nat>
        <servernat>
            <ipaddr>xxx.xxx.xxx.145</ipaddr>
            <descr>Server NAT for x.145</descr>
        </servernat>
        <servernat>
            <ipaddr>xxx.xxx.xxx.147</ipaddr>
            <descr>Server NAT for x.147</descr>
        </servernat>
        <servernat>
            <ipaddr>xxx.xxx.xxx.148</ipaddr>
            <descr>Server NAT for x.148</descr>
        </servernat>
        <servernat>
            <ipaddr>xxx.xxx.xxx.146</ipaddr>
            <descr>Server NAT for x.146</descr>
        </servernat>
        <advancedoutbound/>
        <rule>
            <external-address>xxx.xxx.xxx.146</external-address>
            <protocol>tcp</protocol>
            <external-port>80</external-port>
            <target>onc01</target>
            <local-port>80</local-port>
            <interface>wan</interface>
            <descr>x.146 -&gt; onc01 http</descr>
        </rule>
        <rule>
            <external-address>xxx.xxx.xxx.145</external-address>
            <protocol>tcp</protocol>
            <external-port>80</external-port>
            <target>onc02</target>
            <local-port>80</local-port>
            <interface>wan</interface>
            <descr/>
        </rule>
    </nat>
    <filter>
        <rule>
            <type>pass</type>
            <interface>wan</interface>
            <protocol>tcp</protocol>
            <source>
                <any/>
            </source>
            <destination>
                <address>onc02</address>
                <port>80</port>
            </destination>
            <log/>
            <frags/>
            <descr>NAT </descr>
        </rule>
        <rule>
            <type>pass</type>
            <interface>wan</interface>
            <protocol>tcp</protocol>
            <source>
                <any/>
            </source>
            <destination>
                <address>onc01</address>
                <port>80</port>
            </destination>
            <log/>
            <frags/>
            <descr>NAT x.146 -&gt; onc01 http</descr>
        </rule>
        <rule>
            <type>pass</type>
            <descr>Default LAN -&gt; any</descr>
            <interface>lan</interface>
            <source>
                <network>lan</network>
            </source>
            <destination>
                <any/>
            </destination>
        </rule>
        <tcpidletimeout/>
    </filter>
    <shaper/>
    <ipsec/>
    <aliases>
        <alias>
            <name>inside</name>
            <address>192.168.1.1/24</address>
            <descr>Inside Network</descr>
        </alias>
        <alias>
            <name>onc01</name>
            <address>192.168.1.6</address>
            <descr>onc01</descr>
        </alias>
        <alias>
            <name>onc02</name>
            <address>192.168.1.7</address>
            <descr>onc02</descr>
        </alias>
        <alias>
            <name>onc04</name>
            <address>192.168.1.9</address>
            <descr>onc04</descr>
        </alias>
        <alias>
            <name>russell</name>
            <address>192.168.1.32</address>
            <descr>Russell's PC</descr>
        </alias>
        <alias>
            <name>russfly</name>
            <address>192.168.1.5</address>
            <descr>russfly</descr>
        </alias>
        <alias>
            <name>dmz</name>
            <address>10.10.10.1/24</address>
            <descr>DMZ</descr>
        </alias>
    </aliases>
    <proxyarp>
        <proxyarpnet>
            <interface>wan</interface>
            <network>xxx.xxx.xxx.145/32</network>
            <descr>Proxy ARP for x.145</descr>
        </proxyarpnet>
        <proxyarpnet>
            <interface>wan</interface>
            <network>xxx.xxx.xxx.146/32</network>
            <descr>Proxy ARP for x.146</descr>
        </proxyarpnet>
        <proxyarpnet>
            <interface>wan</interface>
            <network>xxx.xxx.xxx.147/32</network>
            <descr>Proxy ARP for x.147</descr>
        </proxyarpnet>
        <proxyarpnet>
            <interface>wan</interface>
            <network>xxx.xxx.xxx.148/32</network>
            <descr>Proxy ARP for x.148</descr>
        </proxyarpnet>
    </proxyarp>
    <wol/>
</m0n0wall>


One thing I noticed, and you can see it here is that when you look at the ARP table, it only list that for the DSL modem.  That might be normal, but I am not sure.  Other then that, I am at a complete loss...

(http://www2.oncinc.com/firewall/ProxyARP.JPG)

Any advice will be greatly appreciated!   
Russell

Since m0n0wall won't talk to itself, it won't show those proxy ARP entries in its ARP cache. What you see is normal.

Is the DSL modem actually the gateway of the WAN? Normally it would just be a "dumb bridge" and something at your ISP would be the gateway. In that case, you may need to wait hours before ARP caches at your ISP clear and you're able to use a new MAC address. MAC spoofing can work around this, but not since you're using an individual firewall for each IP, hence have a unique MAC for each. Or, it's possible your ISP locks those IP's to a MAC, and you have to contact them to get that changed.

At this point, I would plug a sniffer in on the WAN side and make sure ARP queries for those public IP's are answered as they should be for your proxy ARP entries. If so, and you see inbound traffic isn't being directed as it should be, it's an issue with your ISP or something upstream of m0n0wall. That's my guess at this point.

Hello cmb,

Thanks for your help.  At first I was thinking that it was a problem with the DSL modem.  I had another modem and I was able to get into it and verify that it was configured as a bridge.  So, with that in mind and that it is working with two Linksys routers already, I pretty much ruled that out.

I downloaded and installed IPCop.  I was able to get it working and functioning the way I wanted m0n0wall to.  So, it must be a problem with either my configuration and or my installation of m0n0wall (CD/floppy).

I really want to use m0n0wall, I like it better and it appears to be a better firewall.  It has more granular control over the port forwarding.  I also like the minimalistic OS that m0n0wall is built off of.  I will continue to see if I can make it work. 

Any other ideas will be appreciated.
Thanks!
Russell