Got me thinking about standards. I wrote back a rant to a mailing list of fellow senior hacker, and coders types. Below I’ve included that rant, largely verbatim. I’d been meaning to turn it into a more reasoned blog post, maybe something suitable for posting on a more official outlet, but life is short, and Rod’s post about Quora reminded me to get on it.

As software engineers, as social software engineers, it’s important to have standards. You can debate the how much of what we do can be called engineering, even charitably, but the code we write determines the rules that govern the spaces more and more people spend time in, and while “First, do no harm” might be reaching, a few standards that you should be embarrassed to not meet seem appropriate.

One of those is around data access, data ownership, and sharecropping. This is something Flickr takes very seriously.

The Minimum

With Flickr you can get out, via the API, every single piece of information you put into the system.

Every photo, in every size, plus the completely untouched original. (which we store for you indefinitely, whether or not you pay us) Every tag, every comment, every note, every people tag, every fave. Also your stats, view counts, and referers.

Not the most recent N, not a subset of the data. All of it.

It’s your data, and you’ve granted us a limited license to use it.

Additionally we provide a moderately competently built API that allows you to access your data at rates roughly 500x faster then the rate that will get you banned from Twitter.

Asking people to accept anything else is sharecropping. It’s a bad deal. Flickr helped pioneer “Web 2.0″, and personal data ownership is a key piece of that vision. Just because the wider public hasn’t caught on yet to all the nuances around data access, data privacy, data ownership, and data fidelity, doesn’t mean you shouldn’t be embarrassed to be failing to deliver a quality product.

The ability to get out the data you put in is the bare minimum. All of it, at high fidelity, in a reasonable amount of time.

The bare minimum that you should be building, bare minimum that you should be using, and absolutely the bare minimum you should be looking for in tools you allow and encourage people who aren’t builders to use.

A Reasonable Exchange of Value

Flickr actually goes a bit farther, not only can you get your data out, but it gets enriched as it passes through the system.

If you use the geotagging feature, you don’t just get the lat/long out you put in, but your photo comes back with a whole hierarchy of geographic descriptors, that are pointers into a publicly available gazetteer (Y! GeoPlanet). It would be good if there were pointers into other publicly available gazetteers (if for example Google ever released one) but there isn’t a good concordance service yet (but it’s being worked on)

You get structured access to all the metadata that people have added to your photos, with proper attribution available. (of course there is a working privacy model, so your “friends” aren’t getting data they aren’t supposed to, like your friend requests, and chat logs)

If you used our machine tags vocab, you get extra information pulled in from 3rd party APIs like Open Street Maps, Open Library, Last.fm, various transit administrations, and Foursquare.

Additionally you also have access to the data that was created in aggregate using the data you shared with us, like tag clusters, and the Creative Commons licensed neighborhood shape boundaries.

This isn’t the exhaustive list, just a few of the things Flickr does to respect, and collaborate with the people who share their time and data with us.

I’d certainly love to get a fraction of this data back from other services I use. Imagine getting access to all the data Google has about you, and everything they’ve learned partially based on observing you. I’ve gotten used to being disappointed by most of my fellow practitioners, but I still dream about using good tools that treat me with respect and want to collaborate.

Thanks go to Jesse Vincent, for the useful sharecropping metaphor.

(and I’ll state the obvious this is my personal blog, nothing I post here should be taken as official Flickr or Yahoo communication or policy, unless otherwise noted, that isn’t what they pay me to do.)

• REST, Atom, and Javascript APIs? CHECK
• Access to personal information using OAuth? CHECK
• Developer blog? CHECK
• API Explorer? CHECK (though why it’s a Firefox plugin, I can only guess)

Also versioned documentation, and a quite reasonable set of branding guidelines.

The Netflix Web APIs provide the ability for you to integrate Netflix user services into your application. The APIs provide the following capabilities:

• Performing searches of movies, TV series, cast members, and directors
• Retrieving catalog titles, including details about the title such as name, box art, director, cast, etc.
• Determining the subscriber’s relationship to a specific title, e.g, in queue, saved, available on DVD, etc.
• Managing and displaying queues for users
• Providing conveniences such as auto-completion of partial search terms typed by a user.
• Displaying a user’s ratings and reviews.
• Including functional Add and Play buttons in your web application.

Congratulations to Netflix, and Mashery.

You know the ones when all the MBAs left, and the people who loved the Web went on building it — building meaningful, crazy, artistic cool stuff, and the ethos of the social web was born, back before when that meant more then widget crazy/Facebook-tulip-bloom-madness. Yeah, that sure sucked.

Just thinking about it in the light of this week’s market silliness is enough to make me want to go back to SxSW again this year (where the torch was kept alight, like Ireland in the Dark Ages). And I’d sworn off it after this last year, but maybe budgets will be contracting again by then. And those projects that got started out in the darkness, say Flickr, and Upcoming and del.icio.us among others, wasn’t it all much better when the market got back involved and they got serious?

At least thats what reading Fred and Jason on “startup depression” reminded me of.

Jack Dorsey: Taking the subway to Union Sq. The NYC one. (July 23rd)

(actually I missed that one, but Twitters from Jack regarding the White Stripes were a dead give away!)

Bit late, but congrats to both Twitter and WeSabe on closing funding with Union Sq. Ventures.

Like Tony said, “All my friends go with Union Sq.”. I’ve been fan (a fan of a VC!?!?) since shortly before their del.icio.us investment, and they continue to fund my favorite start ups.

We walked by their office today, but too busy to stop in and say, “Hi”.

http://blog.newscloud.com/2007/06/newscloud_faceb.html

http://kevnull.com/2007/05/iminlikewithyou-and-game-design-in-the-web.html

http://www.kungfugrippe.com/post/1490606

In other Web2Expo news, my talk for Ignite Sunday night got accepted. Ack! Um, so I’ll be speaking (for at least 6 minutes) Sunday night on “casual privacy”.