home 
wiki
wiki
PeopleCantConfigureTheirRoutersMagazine
People can't configure their routers, chapter 2, verse 16
-> "And thus spake the prophet Jerimiah, 'fucking a, another goddamned 192.168.x.x router, and this one is an external route with a fixed cost of 20. you know, like a 100mbit half duplex link (i.e. 50mbit). or 10 if we're being conservative. Hey, taht's hooked to an AP that's pinned to 1mbit, and won't be getting anywhere near that, even if external routes did work in OSPF, which they don't'. And God wept." <--
Be warmly welcomed to the second issue of "people can't configure their fucking routers", the mw network's favourite not-very-regular tabloid, once agian with your host Jaymz the arrogant pigfucker. Today we're going to talk about two seperate, and yet connected subjects when dealing with your very own routers.
'But Ja-aymz, you're oppressing me' or 'Advertise that route, and I rape your skull'
Allow me to paste a snippet, if you would, from the ifconfig output of the box on which I am presently typing:
eth0:1 Link encap:Ethernet HWaddr 00:A0:0C:C8:72:A0
inet addr:192.168.1.2 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING ALLMULTI MULTICAST MTU:1500 Metric:1
UP BROADCAST RUNNING ALLMULTI MULTICAST MTU:1500 Metric:1
vmnet1 Link encap:Ethernet HWaddr 00:50:56:C0:00:01
inet addr:192.168.20.1 Bcast:192.168.20.255 Mask:255.255.255.0
inet6 addr: fe80::250:56ff:fec0:1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1Now, allow me to paste a snippet of the route table for our GHO (and HVC) facing box (which is the trunk for the inner north regional group).
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.20.0 10.10.129.4 255.255.255.0 UG 1137 0 0 eth1
192.168.60.0 10.10.129.4 255.255.255.0 UG 20 0 0 eth1
192.168.1.0 10.10.129.1 255.255.255.0 UG 20 0 0 eth1I don't think it takes a rocket scientist to see what's wrong with this picture. When asked, the anonymous owner of 192.168.20.0/24 indicated that he thought it was likely no-one else was using that range for their lans, since everyone uses 192.168.1.0/24. Of course, he was wrong, but it does make one wonder what's going on with the person connected to the southern panel advertising the 192.168.1.0/24. Connecting to the webserver therin revealed an OpenWRT box... shame.
Today's fun fact: our link to GHO, at the time of that snapshot, had a cost of 1137 (more on why that particilar number later). "But Ja-aymz", you might ask, "why do those two other routes have a metric of 20, when in fact a hop on your crappy 10mbit lan is close to a metric if 10?". And you'd be right to ask. As it turns out, this is a combined configuration error on both my part, and the part of the advertiser - the table below demonstrates.
============ OSPF external routing table ===========
N E1 0.0.0.0/0 1247 tag: 0
via 10.10.129.1, eth1
N E2 10.10.128.96/28 [1146/20] tag: 0
via 10.10.129.1, eth1
N E2 10.10.128.176/28 [1146/20] tag: 0
via 10.10.129.1, eth1
N E2 192.168.1.0/24 [1146/20] tag: 0
via 10.10.129.1, eth1
N E2 192.168.60.0/24 [1137/20] tag: 0
via 10.10.129.4, eth1Firstly, kudos to the man with the balls to advertise a default route for everyone on the network. Well done. But thanks for the optusnet broadband connection that we used to post this while the one we normally use is down
. (if you are 10.10.128.180, btw, I am talking to you).In all seriousness, external routes in OSPF are not like normal routes - they are considered to come under a different administrative domain, and hence the cost for getting to them is not the default metric that is exported to the network - it is instead the metric defined by the advertiser of that route. D'oh. This is actually the correct behaviour, because these are considered to be things like routes to the internet - or other entities outside of your network - where the actual cost doesn't matter per se, as long as you pick the cheapest gateway to export to userspace (try running your internet connection with two default routes - without special loadbalancing software - and you'll see why
).Hence, ironically, the default route is the only acceptable use of the external route advertisment system.
A quick google didn't reveal much useful information as to how to configure quagga to ignore your bogus information, and make my router do the bogus, but correct-for-our-network thing. So I do take some responsibility for this. And yet, I feel I shouldn't be behaving badly in order to clean up from bogus routes...
How large is that link in the window
One true thing about wifi, is that it is about as reliable as an elephant guarding a peanut factory that is infested with mice. The actual useful cost of a link is not only probably not what you think it is, especially if you are one of these people costing your regional group -> GHO hop at _10_ (did you even consider multi-homing? Did you know that HVC is almost as accessable as GHO? Oh, you didn't? Ooops...).
As it happens, quagga is not entirely useless - it does have one, perhaps the only time this is true, useful feature - the concept of "auto cost", which calculates the cost of a link based on bandwidth and a magic number. The RFC correct way to cost links is actually 1000, but this is left to the discression of the network operator, since there is a hard limit of 32000 or so where certain routers start crashing. We actually ended up choosing 100, because, well, routers were crashing.
Brilliant.... routers crashing. Avoid quagga v0.99.1 if you can, btw, it crashes randomly under load. Yay. 0.98 doens't. The Quagga developers are aware of the problem, and claim it will be fixed in the next release.
Ironically, being that here at PCCTR headquaters are unix geeks, Microsoft's MSDN provided a really nice solution for working out how much capacity a link currently has. After all, we want to direct traffic across the links with the most capacity at the present time, right? It turns out, if you send a bunch of pings of 4kbytes, and then take the average return time, and divide 32000 by that number, you get an approximate measure of the available capacity in kilobits. Which, co-incidently, is exactly the format that quagga expects. The nice thing about this, is that you can do it against the AP, unlike iptraf or similar solutions which need both ends to co-operate.
Consider the following script, which we here run hourly on all of our routers:
#!/bin/sh
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/sbin
# change these two lines
PEERADDRESS=10.10.129.1
PEERINTERFACE=eth1
ZEBRAPASSWORD=not-telling
ENABLEPASSWORD=also-not-telling
ENABLEPASSWORD=also-not-telling
# you shouldn't need to change anything below here
NUMPINGS=20
SPEED=`ping -s 4096 -c $NUMPINGS -q $PEERADDRESS | tail -n 1 | awk -F '/' '![c = $4 ; printf]( "c = $4 ; printf")
'`
if [ "x$SPEED" != "x" ]; then
if [ $SPEED -lt 100000 -a $SPEED -gt 2 ]; then # sanity check to avoid killing zebrad - 2kbit/1Gbps
nc 127.0.0.1 2601 << EOF
$ZEBRAPASSWORD
enable SSWORD
$ENABLEPASSWORD
configure terminal
interface $PEERINTERFACE
bandwidth $SPEED
exit
exit
EOF
else
echo "Fux0red like rod stuart"
fi
fiNotice the sanity checks. Also notice that this runs completly unmodified on OpenWRT (where we are, in fact, running it). This script is not perfect yet, of course, and the person who spots the obvious piece of bogosity will get a cookie at the next meeting.
I don't really care if people agree on our costing system, or someone else's, but we do really need to agree on one. And one which considers all wifi links equal as completly fucking usless, because _the are not_. I realise that in wirelss fantasy land, it's just like ethernet, but back here in the real world, that's about as far from true as one can get.
Similarly, we're big fans of BGP over here - if you were at the last meeting, you might have noticed nevyn and myself agitating for it on GHO - but it is utterly important that we make these metrics survive there, if we are to have multiple backbones (which we have the ability to do, so wasting it would be a shame...).
But hey, what would I know about such things? Sane routing is just a thought...
Today's fun fact: if you have three clients on your AP, frottle really can improve your reliability and/or speed. Honestly. At least, it worked for us. Your milage my vary, but it's seriously worth trying, especially if you actually understand the iptables fu to make it useful (read: drop packets from clients who don't use it. Conntrack is your friend...
. Download a statically compiled binary of your very own from http://10.10.64.131/frottle.gz. Ask us today about your config fu!In a similar vein, if your link is nothing but a point to point trunk, consider TCP Westwood with a tcp-based vtun or openvpn running on top of that - we realise the internet might tell you that this is a horrible thing to do, and under TCP vegas it is, but you can seriously improve performance by layering dumb TCP that most people use (and dumb UDP algorythms) onto a tunnel with a Wifi aware TCP implementation such as westwood - if you're on linux, the magic is in /proc/sys/net/ipv4/tcp_westwood.
That's all for another issue - Who knows, since there's suddenly so many links around, maybe we'll make this a regular thing. We will certantly be back soon to talk about name servers, and maybe even direct connect hubs (in each case, just as soon as we either find or write the scripts required
). Until then, don't die of radiation poisoning!-- jaymz@artificial-stupidity.net
Version 1 (old) modified Mon, 26 Jul 2021 12:49:29 +0000 by jaymzj
[EditText] [Spelling] [Current] [Raw] [Code] [Diff] [Subscribe] [VersionHistory] [Revert] [Delete] [RecentChanges]