Eric’s Email Misadventure

We love hosted software models. You can call it SaaS. You can call it cloud. You can even still call it ASP if you’re old-school like that. Scalable. One code base.  Subscription revenue. Users can’t screw it up and incur a bunch of support costs. All good.

But a few weeks back the potential downside of these services showed up in an unpleasant way, right in my inbox. All of a sudden I just wasn’t getting very much email. This isn’t ego – the volume of mail was suspiciously low, especially given how many emails I had sent out the previous week, having recently returned from vacation. There was nothing obviously wrong. Emails weren’t bouncing and if you sent an an email called “test” it arrived no problem. The IT guys sent me emails in from all kinds of crazy domains to see if I had an overactive spam filter. Mail from Russian ISPs arrived unmolested. “No problem” they declared.

Problem was I’d had several people tell me they’d sent me emails that I didn’t receive (including a company that showed up at our offices having confirmed a meeting via an email I never got). So they looked. And looked. They deleted my profile on the exchange server and reinstated it. They were about to blow up the whole exchange server and start over when they figured it out.

When I got back from vacation I had put in place a new signature for my emails, one which featured the URL for my tumblelog, wiesen.tumblr.com. It had not occurred to me (or to anyone) that this would be any kind of issue, but when the IT guys started pulling apart an email we knew failed through repeated testing, it was in fact my signature causing the problem. Specifically, the string “tumblr.com”.

Any email that contained that string failed to deliver to my inbox. Apparently (we later learned) a hosted security product used by our (hosted) disaster recovery product had somehow flagged poor tumblr as dangerous, and was simply killing any emails containing the domain. No quarantine, no spam filter. These were killed long before they ever got to our server.

All’s well that ends well, of course. I changed my sig (although I’m sure many people are deprived for the lack of my scintillating content) and sent out a big mea culpa to anyone I’d emailed in the affected week (since it was all replies with my quoted signature that got blasted) and we moved along.

But it certainly highlights a slight lack of control and transparency that comes with outsourcing (for lack of a better word) one’s software. Convenient and easy to administer? Absolutely. But imperfect in ways I hadn’t necessarily focused on prior to this misadventure.