Twitter, Ruby, and Scaling

April 12th, 2007

Alex gave a phenomenal interview on Twitter and Rails a couple of weeks ago. This morning its all over the Net — but folks I think are taking the wrong lessons from it.

  1. Ruby is dead slow. This is not news, though it can be surprising when you’re used to thinking about scripting languages as all being roughly equal.

  2. Rails trades developer performance for framework performance. Also not news, as this has been the mantra of Rails since day 1.

More importantly he gives a quick insight into the how of making social software scale. It’s hard, it has ugly network effects, it makes databases cry. Alex mentions cache like mad. (because frankly no one but the content creator needs to see fresh data)

Also denormalize like mad, federate like mad, and prune features that make your site slow. (and these are the same techniques that they’re working on behind the scenes at Twitter, and that we use to scale Flickr).

You’ll never build a successful site if you build to scale from day 1, scaling is always a catch up game, but it’s the best game there is.

(And yes, this is my all Twitter all the time blog week)

update: Blaine, lead Twitter engineer, is giving a talk on how they scale Rails/Twitter next weekend at the Rudy SD Forum. (which has done a terrible job of publicizing its existence, but has a pretty killer looking line up)

7 responses to “Twitter, Ruby, and Scaling”

  1. […] More thoughts from Rafe and Kellan. Kellan ends with: You’ll never build a successful site if you build to scale from day 1, scaling is always a catch up game, but it’s the best game there is. […]

  2. […] Twitter, Ruby, and Scaling […]

  3. […] If perchance, you’re part of the other 1%, you should be thrilled to have this problem, it means your software has been successful! Fact is, you don’t accidentally create a site/service which generates 11,000+ requests/sec. You explicitly set out to do so. And presumably, you chose Rails for a reason. Why shouldn’t we extend the concept of “no premature optimisation” to language and platform selection? If Twitter had chosen an “enterprise” class platform and tried to architect for 11k reqs/sec, chances are they would still be building… […]

  4. […] I think some of the best points on this whole thing is the following (paraphrased from somewhere 1, 2, 3): Rails is great because it trades developer performance for framework performance. Rails is good enough for almost every application. However, if you get big enough – rails will suffer. If you are writing an app that may get big, you may want to choose a different framework. […]

  5. […] I think Ruby’s edges (or core maybe) are showing here.  This has been discussed at length at Laughing Meme, but I haven’t seen much about this subject on Twitter’s blog.  Lots of folks are using Ruby daily, with many of them adopting Rails, and all of them have a stake in its future. […]

  6. […] Wanted to jump up and down when I read this. Building it “right” the first time is one of the best guarantees of failure I know. Scaling is always a catch up game. […]