Laughing Meme

Les’ “Delight Everyone” post is latest greatest addition to the 17th letter of the alphabet for savior conversation.

And believe me I’m a huge fan, and am busy carving out a night sometime this week to play with the RabbitMQ/XMPP bridge (/waves hi Alexis).

But …. there are a couple of caveats:

1) Some writes need to be real time.

Les notes this as well, but I just wanted to emphasize because really, they do.

If you can’t see your changes take effect in a system your understanding of cause and effect breaks down. It doesn’t matter that your understanding is wrong, you still need one to function. Ideally a physical analogy too. There are no real world effects that get queued for later application. Violate the principle of (falsely) seeming to respect real world cause and effect and your users will remain forever confused.

del.icio.us showing you the wrong state when you use the inline editing tool, and Flickr taking a handful of seconds to index a newly tagged photo are both good examples of subtly broken interfaces that can really throw people.

My data, now real time. Everyone else can wait (how long depends on how social your users are).

2) You’ve got to process that queue eventually.

Ideally you can add processing boxes in parallel forever but if your dequeuing rate falls below your queuing rate you are, in technical terms, screwed.

Think about it, if you’re falling behind 1 event per second, processing 1,000,000 events a second, but adding 1,000,001 for example, at the end of the day your 86,400 events in debt and counting. It’s likes losing money on individual sales, but trying to make it up in volume.

Good news: Traffic is spiky and most sites see daily cycles with quiet times.

Bad news: Many highly tuned systems exhibit slow down properties as their backlogs increase. Like a credit card, processing debt can get exponentially unmanageable.

In practice this means that most of the time your queue consumers should be sitting around bored. (see Allspaw’s Capacity Planning slides for more on that theme.)

If you can’t guarantee those real time writes for thems that cares, and mostly bored queue consumers the rest of the time then your queues might not delight you after all.

See also: Twitter, or Architecture Will Not Save You

Nice to see consistent hashing go from obscure to blindingly obvious in a few short whitepapers.

Dynamo is certainly the sexiest discussion of distributed hash tables (DHTs), while Programmer’s Toolbox Part 3: Consistent Hashing is the most straightforward. libketama is open and easy to use implementation of the 64-bit space mapped to a circle style consistent hash, discussed above and originally “popularized” by Chord. (and proposed over a decade ago)

And best quote:

“…and if anyone tells you that you shouldn’t use MD5 for this because it isn’t secure, just nod and back away slowly. You have identified someone not worth arguing with.”

Mike released HTTP_Request_OAuth today, so I spent a little while this evening coding up Service_Twitter as helper class for making OAuth authorized requests against the Twitter API.

Both are early enough in the dev cycle to be called proof of concepts.

Mostly I wrote it because I had always envisioned there being wrapper libraries around the low level OAuth implementations that wrapped the calls, and constants, and as Mike graciously went out and wrote a low level library I felt compelled to write a wrapper.

Also twittclient, an interactive client for getting an authed access token, essential to bootstrapping development.

And nota bene, HRO currently only supports the MD5 signing algorithm, which is undefined in the core spec, and subject to change. (Just in case you didn’t believe me about the early state of things.)

update 2008/4/18

This code no longer works because Twitter has taken down their (slightly non-compliant) OAuth endpoint. When they add OAuth support back in, I’ll link to it.

“I can’t tell you how many times I’ve heard people say they wouldn’t use Ruby because it lacks automated refactoring tools. Ruby doesn’t actually need them in the way Java does; it’s like refusing to switch to an electric car because there’s no place to put the gasoline.” - Steve Yegge on the NBL

Realized the other morning that I had half an article written just looking at my notes from some recent PHP hacking. Looked around a bit for somewhere to publish it, but most of the places I might have sent a PHP article a few years ago don’t seem to be around, or at least not accepting submission. Suggestions on a good venue?

Or is that why I’ve got a personal publishing platform?

Recently started using Simple Test for a couple of projects (even slipped it in at work, but sssh! don’t tell them!). So far I’m very happy with it. Using PHPUnit was always a bit of a non-starter, it felt heavy, and even which version (fork?) to use was ambiguous.

Simple Test’s documentation beyond the basics start to trail off, but the code is eminently readable (better then docs any day!), and I found writing a harness to work with the feedparser tests pleasantly straightforward.

Recommended.