I write a lot about products and companies that have potential, but aren’t quite perfect, like Amazon Unbox on TiVo and lots of Sun stuff. But this week’s outage at 365 Main, a datacenter in San Francisco (which we don’t use), reminded me that there are a few products and companies we love that I don’t say nearly enough about. So I’ll start with our datacenter, Equinix, and try to post about some of the others, too.
SmugMug got its start with 3 old used VA Linux boxes (dual 700mhz Pentium 3s with 2GB of RAM which are still in production today and have been our most reliable boxes) from a dead dotcom, which we threw into a friend’s cheap rack at Hurricane Electric. Once the money started flowing in, and we ran into HE’s power contraints and poor bandwidth, we hunted around for datacenter space. Equinix had the very best reputation among the Operations crowd here in Silicon Valley, so we gave them a shot and pulled out of Hurricane Electric.
I should warn you up front that there’s a little “sticker shock” when you first talk with Equinix (ok, and every time you need to buy more stuff from them, it returns), but in the end, it’s well worth it. It turns out that in life, some things are worth paying for. Datacenter space is certainly one of those things (and we feel like photo sharing is too!).
In the ~4 years we’ve been with Equinix, we’ve had only one major problem: They sold our power out from under us (to Yahoo) which forced us to move from one of their locations to another. Ugh. Datacenter moves, especially with hundreds of terabytes of disks, really suck. Luckily, thanks to decent system architecture and some magic from Amazon S3, we were able to do 99% of our move during normal business hours over the course of a month with no impact on our users.
In all fairness to Equinix (though this is no excuse), they weren’t the only datacenter that had poorly prepared for the ‘Power is King’ change in the datacenter landscape that happened a few years back. Plenty of other companies with other providers tell me the same story, so we’re not alone. Datacenters all over the place used to sell you mostly based on space (square footage) rather than power (watts). They all got burned when CPU and server vendors started getting really fast & dense gear. Nowadays, almost the entire negotiation is regarding power and everyone has empty dead space in their rented cages. Such is life.
On the bright side, everything else about Equinix rocks:
- Power. I’m surprised to hear all of the horror stories out of 365 Main because I assumed they were as good as Equinix has been for us. We haven’t had a single power-related outage in all of the years we’ve been there. It just works - and it’d better, that’s the biggest reason we use a datacenter.
- Metro cross-connects. If you’re hosted in multiple Equinix datacenters in a single metro area, like we are, you can get cheap (a few hundred bucks per month) GigE cross-connects wired between your various locations.
- Support. I’m still surprised every time we need to use Equinix’s support staff and they’re actually super-knowledgeable and helpful. I’m talking about hardcore networking and routing questions. BGP, whatever, you name it - they know it. Better than we do.
- Equinix Direct. I’m always surprised when I talk to other Equinix customers who don’t know about this gem. It’s a way to provision your IP transit providers on a month-by-month basis with no minimum commits or contracts. You pick your providers and pay-as-you-go. Pretty sweet. We’re already directly multi-homed on GigE with multiple providers, but we mix in Equinix Direct to have access to still more. Best thing? ED doesn’t add an extra BGP hop, so your routes still look fast (as opposed to someone like InterNAP who adds an extra BGP hop to do similar stuff).
- Security. 5 biometric scanners are between you and your cage when you enter the building, with live security on hand 24/7. Stuff like this is fairly common at high-end datacenters, but it’s important, so I’m mentioning it anyway.
- Bandwidth providers. Equinix is a carrier-neutral facility, and basically everyone has connectivity there, so you can easily pick whomever you’d like to carry your traffic.
Of course, they do all of the other myriad things a datacenter is supposed to do. One of the reasons I haven’t blogged about them in the past is because they just work - and they work so well, I just don’t spend much time thinking about them.
Which, of course, is the way it’s supposed to be. 
(Now, of course, I’ve jinxed the whole thing like Red Envelope and our datacenters are going to explode in a Martian Invasion. Sorry about that!)
