[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: problem load balancing incoming connections



On Fri, Nov 19, 2004 at 06:14:59PM -0300, Emilio Lucena wrote:
> This has done the trick. At least as far as incoming connections to the 
> internal webserver are concerned. However, there remains a problem. 
> Traffic from the Internet to the firewall itself is still not working as 
> desired. Somehow, the default route is still being used for outgoing 
> packets in this case. For example, if I ping the IP of the second 
> interface (not the one with the default route), the echo reply packet is 
> sent out through the default interface. Even if I use a reply-to rule 
> like:
> 
> pass in log-all on $ext_if2 reply-to ($ext_if2 $gw_if2) \
>      net proto icmp all icmp-type $icmp_types keep state
> 
> the result is not the expected. See the output of tcpdump:
> 
> Nov 19 17:12:01.557537 rule 11/0(match): pass in on rl0: 201.9.164.54 > 200.177.74.138: icmp: echo request
> Nov 19 17:12:01.557633 rule 11/0(match): pass out on tun0: 200.177.74.138 > 201.9.164.54: icmp: echo reply
> Nov 19 17:12:01.557679 rule 11/0(match): pass out on rl0: 200.177.74.138 > 201.9.164.54: icmp: echo reply
> 
> Yes ... the packet is duplicated on the other interface, and in fact it 
> gets lost or discarded, because the pinging host reports packet loss.
The tcpdump on pflog has a simple explanation. Remember how pf first
sees the reply trying to go out on tun0, where it matches the reply-to
state, gets routed out rl0, matches the same state again and gets
passed out on rl0?
The first filter pass, on tun0, sends the packet to pflog due to your
'log-all' option. This logging happens before the reply-to option is
processed, and isn't suppressed just because the packet later gets
routed by pf.
That doesn't mean the echo reply was sent out on tun0. pflog doesn't
really tell you what packets were sent out on an interface, it tells
you whether a packet passed the state table or ruleset on an interface.
Repeat the test and also run tcpdump on rl0 and tun0 themselves. This
tells you what packets are really sent out on these interfaces.
The rule looks fine, the pflog tcpdump confirms the reply-to state is
working. Maybe the packet loss has another explanation, things should
work like this.
> 1. Fault-tolerance. A very simple one. Something that avoided the packets 
> going to the wrong interface (when it goes down) would be sufficient. Do 
> you think "ifstated" could be used for that purpose (I am thinking about 
> using some anchors to have the ruleset updated when something bad 
> happens)?
Yes, that will work for new outgoing connections from you. When both
links are up, you round-robin connection out through both. When one goes
down, you remove the address from the table, and the round-robin rule
will use only the working address.
Incoming connections will simply cease to arrive on the non-working
interface, so there's nothing to handle there.
Ongoing connections on the interface that just died will stall and have
to be re-established, there is no way to change your address during a
connection, obviously.
> Is this still the recommended setup for 3.5 and 3.6? If so, how could one 
> add queue information for 2 interfaces with different bandwidth limits?
I guess you can use 'route-to' on the 'pass in on $int_if' rules, so
things better ressemble the reply-to rules. I don't remember why I
recommended re-routing from the default route interface, to be honest ;)
Packets will get refiltered on the external interface, you can use two
separate rules to pass outgoing connections there using separate queues.
You'll have two 'altq on $ext_if1/2' rules, and can setup separate
queues on each interface. Shouldn't be a problem.
> Also IP1 and IP2 will probably be from the same network. Do you see this 
> as potential problem (as far as PF is concerned)? 
I can't see any problems there.
Daniel