[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

rdr not redirecting when target is localhost



This is my second adventure with pf.  The first was setting
up spamd on a system with an IP - not the preferred solution
but one that was forced on me at the time because I didn't
have physical access to the mailserver in a way that would
allow me to put a transparent bridge in front of it.  Now I do,
and so I'm upgrading our greylister to use transparent bridging
- but I've hit a problem which has driven me crazy for 2 days
now but which I'm hoping will be obvious to the folks here...
The short summary first: I'm pretty sure I've set up pf
properly - I've tried the out-of-the-box config, a simple
3-line very basic config, many example configs lifted from
web pages - they all behave the same way.  Which is that when
I redirect port 25 connections to port 8025 on localhost, the
rewritten packets go out via $int_if rather than lo0.  I've
confirmed this carefully with the log option, pflog0, and
tcpdumping of int_if and ext_if and lo0.
My guess at the moment is that there is something external to pf.conf
that I have missed.  I've turned on ip forwarding as per the docs,
but not ip6 forwarding because we don't use it here.  I've set
up the bridge interfaces, and I've enabled pf.
At the moment I'm not even looking at the use of tables - I'm
trying to redirect *all* smtp connections while testing, just to
keep it simple.  If I set up the rdr to redirect -> $server_ip 25
rather than -> 127.0.0.1 8025, that actually works.  So the problem
is not in the redirect rule matching the incoming packets, but in
the fact that the redirection is to localhost.
If that wasn't enough for someone to spot the problem, well,
here's the long version:
1) I've set up three machines in a test lab, not connected to
   the internet; one pretends to be our central mailer, another
   is a bogus dotcom; between them sits the OpenBSD configured
   as a transparent bridge.  The BSD has a third interface with
   an IP so I can work on it from home over the weekend, but I'm
   fairly sure that this is not involved in the issues I'm seeing.
   fxp1 and xl0 are nowhere to be found in netstat -nr (below)
   and fxp0 is never referenced at all in pf.conf.
                             129.113.28.220(real IP)
                                   |+----------------+          +------^------+          +---------------+
|                |          |     \_/     |          |               ||     (bogus IP) |          |             |          |   (bogus IP)  ||    129.113.1.6 >----------<.............>----------< 129.113.1.99  ||(mail.)utpa.edu |     no ip|   BRIDGE    |no ip     |(outside.)     ||                |          |             |          | somewhere.com |+----------------+          +-------------+          +---------------+
Dummy server                Greylister               Dummy client
I have set up the fake server and the fake client with fake DNS that
allows them to send mail backward and forward from utpa.edu and
somewhere.com.  I've tested this and monitored the data flowing through
the bridge interface on the OpenBSD system, with tcpdump.
Here are some sketchy notes of what I did on the OpenBSD which is
freshly installed from the current release over the net (3.7), just
to check that I did all the major configs:
========================================================================
network interfaces are xl0 (card) fxp0 (onboard, left as viewed from front
above), fxp1
- select fxp0 for internet (this lets us distinguish IN and OUT of the bridge
more easily)
edit /etc/sysctl.conf to add
net.inet.ip.forwarding=1
kern.emul.linux=1
set ntpd_flags="" in /etc/rc.conf (takes effect on reboot; meanwhile, issue
/usr/sbin/ntpd)
set up these files in /etc:
echo up>hostname.xl0
echo up>hostname.fxp1
(hostname.fxp0 should already be set up)
cat > bridgename.bridge0 <<EOD
add xl0
add fxp1
up
EOD
Test with "brconfig bridge0 add xl0 add fxp1 up". Rebooting makes that
permanent when pf is enabled:
edit /etc/rc.conf to set pf=YES
/etc/pf.conf:
uncomment all lines
remove "block in" and replace with "pass in keep state" - don't
want m/c configured as firewall, just as open transparent bridge
which intercepts only port 25.
edit /etc/rc.conf.local:
spamd_flags="-v -G 1:24:864"
spamd_grey=YES          # use spamd greylisting if YES
reboot
======================================================================
I've tried many pf.conf files including the stock on; I've tried them
with the default firewall config enabled and with firewalling completely
turned off.  The other ports work fine, it's just the port 25 redirection
that is problematic.  The simplest config was something like this:
ext_if="xl0"
rdr pass on $ext_if proto tcp from any to any port smtp -> 127.0.0.1 port spamd
and the current one is this:
ext_if="xl0"
int_if="fxp1"
scrub in
nat on $ext_if from !($ext_if) -> ($ext_if:0)
rdr pass on $ext_if proto tcp from any to any port smtp -> 127.0.0.1 port spamd
#block in
pass out log keep state
pass log quick on { lo $int_if }
antispoof log quick for { lo $int_if }
pass in log on $ext_if proto tcp to ($ext_if) port smtp keep state
pass out log on $ext_if proto tcp from ($ext_if) to port smtp keep state
======================================================================
this is what I'm seeing in the log for pflog0:
## 07:11:09.445062 129.113.1.99.46981 > 127.0.0.1.8025: S 961513652:961513652(0) win 5840 <mss 1460,sackOK,timestamp[|tcp]> (DF) [tos0x10]
## 07:11:33.438319 129.113.1.99.46981 > 127.0.0.1.8025: S 961513652:961513652(0) win 5840 <mss 1460,sackOK,timestamp[|tcp]>(DF) [tos 0x10]
and for xl0: (ext_if)
## 07:11:09.445019 129.113.1.99.46981 > 129.113.1.6.25: S 961513652:961513652(0) win 5840 <mss 1460,sackOK,timestamp 6943078 70,nop,wscale 2> (DF) [tos 0x10]
## 07:11:33.438286 129.113.1.99.46981 > 129.113.1.6.25: S 961513652:961513652(0) win 5840 <mss 1460,sackOK,timestamp 69454787 0,nop,wscale 2> (DF) [tos 0x10]
and for fxp1: (int_if)
## 07:15:30.076828 129.113.1.99.41109 > 127.0.0.1.8025: S 1258620181:1258620181(0) win 5840 <mss 1460,sackOK,timestamp 69691492 0,nop,wscale 2> (DF) [tos 0x10]
## 07:15:36.075129 129.113.1.99.41109 > 127.0.0.1.8025: S 1258620181:1258620181(0) win 5840 <mss 1460,sackOK,timestamp 69697492 0,nop,wscale 2> (DF) [tos 0x10]
nothing shows up on lo0.
======================================================================
greylist1# netstat -nr
Routing tables
Internet:
Destination Gateway Flags Refs Use Mtu
Interface
default            129.113.28.1       UGS        16    45806      -   fxp0
127/8              127.0.0.1          UGRS        0        0  33224   lo0
127.0.0.1          127.0.0.1          UH          2      167  33224   lo0
129.113.28/24      link#2             UC          3        0      -   fxp0
129.113.28.1       aa:0:4:0:ff:f      UHLc        1        0      -   fxp0
129.113.28.75      0:10:5a:29:62:b3   UHLc        2     1697      -   fxp0
129.113.28.220     127.0.0.1          UGHS        0       11  33224   lo0
129.113.28.221     0:4:76:cf:bb:cb    UHLc        1     1370      -   fxp0
224/4              127.0.0.1          URS         0        0  33224   lo0
Internet6:
Destination Gateway Flags Refs     Use    Mtu  Interface
::/104 ::1 UGRS 0        0      -   lo0 =>
::/96 ::1 UGRS 0        0      -   lo0
::1 ::1 UH 12       76  33224   lo0
::127.0.0.0/104 ::1 UGRS 0        0      -   lo0
::224.0.0.0/100 ::1 UGRS 0        0      -   lo0
::255.0.0.0/104 ::1 UGRS 0        0      -   lo0
::ffff:0.0.0.0/96 ::1 UGRS 0        0      -   lo0
2002::/24 ::1 UGRS 0        0      -   lo0
2002:7f00::/24 ::1 UGRS 0        0      -   lo0
2002:e000::/20 ::1 UGRS 0        0      -   lo0
2002:ff00::/24 ::1 UGRS 0        0      -   lo0
fe80::/10 ::1 UGRS 0        0      -   lo0
fe80::%xl0/64 link#1 UC 0        0      -   xl0
fe80::204:76ff:fecf:bd5d%xl0 0:4:76:cf:bd:5d UHL 0        0      -   lo0
fe80::%fxp0/64 link#2 UC 0        0      -   fxp0
fe80::2c0:9fff:fe04:7a19%fxp0 0:c0:9f:4:7a:19 UHL 0        0      -   lo0
fe80::%fxp1/64 link#3 UC 0        0      -   fxp1
fe80::2c0:9fff:fe04:7a18%fxp1 0:c0:9f:4:7a:18 UHL 0        0      -   lo0
fe80::%lo0/64 fe80::1%lo0 U 0        0      -   lo0
fe80::1%lo0 link#7 UHL 0        0      -   lo0
fec0::/10 ::1 UGRS 0        0      -   lo0
ff01::/32 ::1 UC 0        0      -   lo0
ff02::%xl0/32 link#1 UC 0        0      -   xl0
ff02::%fxp0/32 link#2 UC 0        0      -   fxp0
ff02::%fxp1/32 link#3 UC 0        0      -   fxp1
ff02::%lo0/32 ::1 UC 0        0      -   lo0
Encap:
Source Port Destination Port Proto SA(Address/Proto/Type/Direction)
======================================================================
I'd much appreciate any pointers anyone can give me.  I'ld like to
start with a guaranteed minimal pf file that does only one thing:
reroutes all connections in on 25 to localhost 8025.  If I can get
that working, I can take care of all of the rest myself.
If you can give me a known working pf file for the above configuration
and it *doesn't* work, I need pointers on where to look elsewhere
for the problem, because I'm all out of ideas.  Let's just say
Google is no longer my best friend :-)  I think I've read every
pf page on the net...
Thanks in advance,
Graham <[email protected]>