Blog posts tagged "ruby"

Twitter, Ruby, and Scaling

April 12th, 2007

Alex gave a phenomenal interview on Twitter and Rails a couple of weeks ago. This morning its all over the Net — but folks I think are taking the wrong lessons from it.

  1. Ruby is dead slow. This is not news, though it can be surprising when you’re used to thinking about scripting languages as all being roughly equal.

  2. Rails trades developer performance for framework performance. Also not news, as this has been the mantra of Rails since day 1.

More importantly he gives a quick insight into the how of making social software scale. It’s hard, it has ugly network effects, it makes databases cry. Alex mentions cache like mad. (because frankly no one but the content creator needs to see fresh data)

Also denormalize like mad, federate like mad, and prune features that make your site slow. (and these are the same techniques that they’re working on behind the scenes at Twitter, and that we use to scale Flickr).

You’ll never build a successful site if you build to scale from day 1, scaling is always a catch up game, but it’s the best game there is.

(And yes, this is my all Twitter all the time blog week)

update: Blaine, lead Twitter engineer, is giving a talk on how they scale Rails/Twitter next weekend at the Rudy SD Forum. (which has done a terrible job of publicizing its existence, but has a pretty killer looking line up)

Coding a Twitter killbot

March 9th, 2007

j =, pass)

j.received_messages.each do |mesg|
    if mesg.body.match(/austin|sxsw/i)
        sender = mesg.elements['//screen_name'].text
        j.deliver('', "leave #{sender}")

Though truth be told, it isn’t running.

Weather over Twitter

January 12th, 2007

Too Close!!!

And while we’re talking about recent hacks, Blaine and I whipped up a Jabber bot using his Jabber::Simple and the Yahoo weather feeds, to provide twice daily weather updates via Twitter.

Jabber is an intriguing platform to build on top of, and the more I play with it the more potential I find. I keep checking in on it every few years (since MetaEvents days), but recently its gotten much more interesting. In part thats Google’s adoption of the standard (and the subsequent enhancement in tools, libraries, and clients), and partially standards bake slowly, but at the core of it I think we’re reaching a point in the evolution of the Web where Internet-scale deployed messaging standards have a lot to offer of us. A protocol for when HTTP fails you.

If you follow these bots, you’ll receive those updates wherever you normally get your Twitters; IM, Phone, RSS, or just on the web. So far, we have bots for the following cities: Boston, Brighton, Chicago, Helsinki, London, Los Angeles, New York, Paris, Portland, San Francisco, Seattle, Singapore, and Vancouver. If you’d like to see another city, just ask and we’ll provide.

Slightly out of date source available at twitter-weather – Google Code

And taking requests for new cities. Probably do a big batch of new ones sometime next week. (not really an automated process)

Photo by bonsaikiptb

Jabber::Simple for Ruby

November 10th, 2006

Blaine finally released Jabber::Simple! Jabber::Simple is a straightforward Ruby wrapper for talking XMPP, the perfect for scripted IM interactions, or any other async communication you’ve got in mind. Sweet.

Tagged: Uncategorized ,

OSCON: First Day of Sessions

July 27th, 2006

The day started out right, if a bit bleary-eyed, with Gnat giving OSCAL a lovely plug during the keynote. Also got to see Rael’s latest project, tres cool. (and in Rails, of course)


Embedded databases in browsers turned out to be a bit more theoretical then I was in the mood for, but the backchannel was buzzing about Justin Erenkrantz’s modproxytalk, so I skipped over there. Jackpot. mod_proxy has done gone and grown up. Elegant, serious, flexible, built in caching (with multiple backends), pluggable protocols. Squid’s days as a reverse proxy are numbered. Now if someone would just hurry up and package Apache 2.3 we could do without Pound. (I’ve misplace the slides link)


Tim’s talk on atompub was good, if basic. I’m still trying to figure out how to push it beyond basic publishing. An interesting challenge, but later Tim pointed out that there are is some intentional wiggle room left in the spec, and one or two holes that you could drive a truck through with enough determination. (don’t think there was a slides link?) Oddest new thought, we need a mime type for Markdown, or maybe a container type for the whole class of human readable markup.

Briefly met Chad and Jeremy, something I’ve failed to do at work.

On The Floor

Wandered the exhibit hall a bit. Blah. Just not feeling the vendor love. No surprise, but its slams home what a commercial conference OSCON is, with very little of the raw delite and innovation of the smaller events. All the good shirts cost $$ this year — wonder what that means for t-shirt driven development, and the t-shirt economic metric. Finally met Jason from Apress. They have a Flickr book out in August.

Too Many Codepoints

Andrei’s PHP6 and Unicode talk was impressive, and overwhelming. ICU is being baked deep into the core string object. An .ini setting to determine default behaviour, with Unicode (UTF-16) and binary string types. Automatic stream oriented encodings from input/output/file/cli etc. Look for a preview release this Fall. (Eclipse gets all the glory, but ICU is another amazing IBM open source contribution, worth checking out in its own right) I’m not sure anyone is thinking about the “How do I make charsets work across the PHP4, PHP5, PHP6 spectrum in an open source library?” Doesn’t seem like that out there a question. Looking forward to catching up with him back in Sunnyvale.

Ruby Rodeo!

FreeGeeK goes on being one of the coolest, most inspiring community projects anywhere. Packing it to gills with hyper excited [Ruby hackers] certainly didn’t detract. Lucas Carson’s talk on dRB/Rinda was cool and inspiring. Not as polished a delivery as some talks, but he coded up a server-client architecture for discovering primes and automatically deployed it to those of us in the audience running irb. In about 20 minutes. The hilight though was finally getting a chance to catch up with Scott after all these years. (Rinda may just be good old Linda retreads, but Ruby is so damn slow that distributed computing is with the effort)

Didn’t Make It

Most disappointed to have missed in retrospect, Kevin Henrikson Ajax Optimization Techniques: Working with Large Ajax Applications. Got rave reviews.

Tagged: Uncategorized , , , , , ,

Boston.rb, Next Tuesday (Feb 7th)

February 2nd, 2006

After a rather long hiatus, Boston.rb is reasonably likely to meet Feb. 7th. Website seems to experiencing the dreaded 502 Bad Gateway error (umm, yeah, secrets out, Rails hosting is non-trivial), but jump on the list for more info. (or at least join in the confusion)

Tagged: Uncategorized , ,

Pirate Testing (Because Only Ninjas Write Unit Tests)

January 30th, 2006

I’ve got a new favorite development technique, “pirate testing”. I’ve used it on 3 recent projects, and it rocks.

And while Sam might have meant it literally, I’ve found it perfectly describes the practice of shanghaiing another tool’s test suite to given your own TDD a jump start.

(n.b.: May be harder in languages which don’t allow reopening of classes. aka monkey patching)

Tagged: Uncategorized , , , , , ,

Lucene in Action (and Ferret)

January 23rd, 2006

I’ll admit I’m a freak, but I find a well written programming book as gripping as a well written novel, and have been known to sit down and read them cover to cover. (PofEAA was a page turner!) I’m about a third of the way through Lucene in Action, and it’s excellent: easy to read, compelling examples, deep insight, generally good stuff. A good tech book leaves your mind percolating with the all cool new things you can do with your new knowledge, and LiA is that kind of book.

I’m reading it in the contexts of Ferret and while there are some minor API differences (no Hits class in Ferret but it adds an Index::Index convenience class), for the most part the knowledge is directly applicable.

Tagged: Uncategorized , , , , , ,