Well, I've been up since ~ 6.45 ish, and was in Telehouse from ~ 10.45am through till 4.40pm (ish). Yesterday, our server hosted by netkonect in Telehouse "magically" lost routing to the world, I repeatedly told them that they had a problem, that the machine was fine before they had the other outage, and that traffic was consistently being dropped at their gateway... They insisted on rebooting our server, claiming that that would probably fix it... when it rebooted, of course, the routes and interfaces where set up exactly as before. Still no network. At 9.30pm Friday night we gave up and I resigned my self to going to London Saturday morning, got to telehouse, got to the right room, discovered that I hadn't got a key to our rack, or the monitor and keyboard that I'd asked for. So, phone netkonect, get a "oh, they should have given you the key at reception", right, trundled back to reception, yes, got the key, rah, now have access to the rack, but still no sign of a monitor and keyboard, or the netkonect engineer that was supposed to visit me, or the mistral engineer that was supposed to visit me. OK, fine, out comes the laptop, crossover cable, and some fiddling at the back of the server. Rah, I have ssh to it. Right, set up the second ethernet card to have a private address just so that the laptop can sit on that port and talk to it. Plug back in the other network lead, hey presto, still nothing from our box to the outside world, all being dropped at this point by the switch (I couldn't ping between the box and it's RAC port, which are on the same switch). Right, phone netkonect, tell them this, wait for a response, get bored, it's now midday and I'm dying of thirst, of course, the telehouse coffee shop is shut sigh, so off I go in search of coffee, ended up in the Bar-Cafe of Travellodge, about 5 minutes wander from telehouse. Sat there for ~ 45 mins waiting for responses, go back, and yes, I can now ping between the machine and the RAC port, WOO! Next step, right, yes, packets are being dropped straight after that, fantastic, so, our default gateway is down... back on the phone, long conversation, hang up, wait 30 mins to hear anything back, finally get told that our default gateway should be a COMPLETELY different address to what it has always been set to, 12 IPs upwards from where it was previously, on an IP that (fortunately) we hadn't already used in our range. SIGH, not happy about this, confirm that if I set the gateway to that IP it does infact work, but switch it back straight after confirming because I want THEM to fix THEIR router mis-configuration, after 2 hours of backwards and forwards getting nowhere I eventually caved, but not before making damned sure that they knew exactly what I thought of this event, and that if it ever happens again I will be severely pissed off, and that I want it in writting that their network engineers will not randomly change our gateway without consulting us ever again. I have a signed fax in my bag from netkonect saying that in the event of needing to change our gateway for whatever reason they will have to contact us to get approval. (I would have thought that was a bloody standard requirement anyway).

Anyways - all dealt with, networking came back, left telehouse.... It wasn't till a little while later that John sent a message asking if I'd remembered to restart the chroot web server (oh bollocks), no, I hadn't, but given it was running nicely until netkonect rebooted our server, and given that I'd forgotten by the point I left telehouse, I gave rough destructions over the phone, and it was all back up shortly afterwards (rah! thanks john).

On a different note, it was good to see you piem, you mad french man you :)