Things Fall Apart, Datacenter Edition

The relentless pursuit by Operations staff of 100% uptime has always struck me as something more than just a job, but a battle against the relentless forces of nature. Everything ultimately breaks down — systems, buildings, even people — and attempting to maintain 100% availability is the Ops equivalent of trying to cheat death. Sooner or later, despite our best efforts, our number will ultimately be up. Most recently in the news, self-proclaimed World’s Finest Data Center operator 365 Main suffered an approximately 45 minute power outage at their San Francisco facility. Much to their credit., and unlike most of their competitors, 365 Main has been extremely open about their investigation. I’ll examine this a bit today, as it’s a rare public glimpse into what goes on inside a large data center facility.
Continue reading

Geographic Distribution for Global Web Application Performance

I’m pleased to announce that on Tuesday, April 17th, I’ll be presenting a brief discussion of Geographic Distribution at Web 2.0 Expo in San Francisco. As the web matures, performance has become a tremendous issue, especially when deploying an application for a global audience. One important way to improve performance is the geographic distribution of application delivery. Join me at 8:30am tomorrow in 2018, or check out my slides, which will be posted shortly after the discussion.

Really Big Data Centers

While most of my time these days is spent contemplating software and application considerations, I’d like to take a moment to address a topic which only occasionally gets the attention it deserves: the role of a high quality data center. While a few folks may think that networking and data center infrastructure are dead arts, I’m quite confident that there is still significant work going on in this space. Case in point: Yahoo, Microsoft, Google, Google, and (shockingly) Google are building massive new data centers taking advantage of all of the latest features to increase density and automation and reduce cost. At the end of the day, scale wins, and these facilities (which have price tags in the half-a-billion dollar range) have scale. Not to be outdone, incidentally, AOL has built a few big data centers — and sold them too. Continue reading

Do Apple Users Just Have Stockholm Syndrome?

One of the more interesting questions in technology is how to know when it’s time for it to change. David Habib writes today about a concept he calls Technology Vendor Stockholm Syndrome, which occurs when technologists have worked so long with a vendor that they develop what he calls an unhealthy partnership with them. I’d argue it goes even deeper: there’s such a thing as a Technology Stockholm Syndrome that can develop around any sort of technology, even in the absence of vendor advocates.

Continue reading

You fix computers, right?

A surprisingly high number of the people I know who work in Technology Operations roles around the industry have parents who think they fix broken Outlook installations for a living. This is a not a reflection on their parents, per se, but rather it represents the challenge of distinguishing an Operations role from an IT role. Most people who work in any modern organization have interactions with IT staff, making the identification quite easy (Oh, that’s what little Johnny does). The problem with this identification is that it is probably wrong.

DISCLAIMER: This is also not a negative reflection on IT staff. Certainly, there a distinction between the roles, but I mean absolutely no disrespect of any kind to the helpful and hardworking people who keep computing systems running everywhere.
Continue reading

Mauled by Wolves

I was listening to the radio and heard that President Ford had passed away. Of course, the first thing I thought of was the 1996 Saturday Night Live episode featuring Dana Carvey as Tom Brokaw pre-taping obituaries for every possible way Gerald Ford could perish (e.g. “mauled by wolves”). Thanks to, let’s face is, the weird values of the internet at this point, I knew I could find it for free somewhere online.

Continue reading

Big Step 2: Going Multi-Site

Successful web products don’t just grow, they grow explosively. If people love something about them, they’ll tell everyone they know about them, and they tell their friends, and before you know it, a product in stealth mode is getting used everywhere from Akron, Ohio to Harare, Zimbabwe. It’s around this time that just being on a couple of servers in a rack somewhere isn’t enough. It’s time for the next Big Step in the evolution of a web site’s scale. Today, I’m covering the “why” of Big Step 2, going multi-site. Continue reading