Blog posts tagged "community" data and doing the right thing

January 5th, 2011 (“Caffeinated and Unstrung”) was a moderately successful community collaboration I built and ran to catalog good coffee shops to work out of, originally and most successfully in Seattle, then spread to Vancouver, Portland, Chicago, Boston, and New York.

It was also an experiment in extracting structured data from semi-structured, free form wiki like data entry.

I moved away, and the site kept running itself. Eventually the spammers overwhelmed the community and I had to shut it down. I feel bad about this. Bringing back (and rewriting) WifiMug is on the todo list, it’s on the given-the-ability-to-freeze-time todo list. (by far the longest of my todo lists)

So I spent 15 minutes attempting to do the right thing, and all the data (and all the spam) is available for download under a Creative Commons Attribution-ShareAlike license.

The dump contains the following directories per city:

  • database contains the raw text of the wiki pages that represented each of the cafes (and the various meta pages).
  • metadata contains some of the structured data, per page
  • rcs the history of each page. (yes rcs)

Ideally I’d do more. Ideally I’d scrub the data, port it to some sane format, tease out the implicit metadata encoded in the markup, attribute all of the various community members, etc. But I’m trying to not let the perfect be the enemy of the ok. This is a minimal competence thing.

And ideally we’d should be able to hope and expect that Yahoo! would do something like this with the infinitely more important and influential data set — dedicate a few weeks or months of time to preserving one of the greatest new libraries of our time, possibly donating it to or the LoC. But nothing makes you uncomfortable like holding Y! to a standard you aren’t personally living up to.

Get the data.

All done/written under the influence of 30k feet and 10 minutes of reflection, treat accordingly.

ps. I almost got all of this into 140 characters, but failed. I hate the way blog posts feel so flabby and fluffy after the compressed kinetic energy of a tweet. I mean Anil makes me feel all noble doing it, but I miss my creative restrictions.

Lessons of the Elders

August 27th, 2009


Walking into work this morning, thinking about the vibrancy of the Twitter API developer ecosystem. Twitter has embraced (not necessarily intentionally) what I see to be open source’s two key aphorisms on successful community engagement (lightly paraphrased)

“Get people laid” – JWZ
“Leave lots of low hanging fruit” – Karl Fogel

People don’t go back often enough to the well of hard fought wisdom on community and online collaboration which the open source community developer. Believe me you do not want to spend the blood, sweat and tears to re-learn those lesson the hard way, they were fighting for a noble cause they believed in and it still sucked. Your micro-poking app ain’t all that.

Photo by curlyjazz

Flickr, Twitter, OAuth: A Secret History

July 1st, 2009

I remember it as a dark and stormy night, that seems unlikely, but I’m sure it was late and chilly and damp.

I remember being tired from a long day in the salt mines; that was during a period when I was always tired after work.

I remember there being whiskey, and knowing @maureen, that seems likely.

I’d just won some internal battles regarding delegated auth, and implemented Google AuthSub for the new Blogger Beta, as well as Amazon auth for a side project. So when I wanted to share photos from Flickr to Twitter, I knew it wasn’t going to be over HTTP Basic Auth.

A few weeks earlier @blaine and @factoryjoe had pulled me a into a project called OpenAuth that they’d been talking about for a couple of months — an alternative to yet another auth standard, and a solution for authenticating sites using OpenID.

So one late, damp night along Laguna St. with whiskey, we did a pattern extraction, identifying the minimal possible set of features to offer compatibility against existing best practice API authorization protocols. And wrote down the half pager that became the very first draft of the OAuth spec.

That spec wasn’t the final draft. That came later, after an open community standardization process allowing experts from the security, web, and usability community to weigh in and iterate on the design. But many of those decisions (and some of the mistakes) from that night made it into the final version.

Yesterday, a little over two years later, we finally shipped Flickr2Twitter.

So it was nice yesterday when people commented on the integration:

“Uses OAuth!” “Doesn’t ask for your Twitter password” “Great use of OAuth”.

And I thought to myself, “It better be, this is what OAuth was invented for — literally”.

Notes from Social Graph Foo

February 4th, 2008

Here is my quick dump of the notebook, probably useful to no one but me. Names mostly removed to protect the guilty.

I think “Social Graph” is kind of a dumb phrase to apply to the back question of relationships. I promptly re-dubbed the event “Social Foo” and thereby found interesting things to talk about. Kevin Marks proposed “social cloud”, clouds hide details. (operations people get hives when you talk about clouds)

XMPP, OpenID, OAuth are all going to be huge in 2008; DiSo, DataPortability, and Social Graph API aren’t as clear winners to me.

Bowling Alone misses the point. There has been a transformative change from groups to networks. Groups are just a funny form of network.”

“Differentiated role networks”. Differentiated roles, and the failure of monolithic identity and friending were one of the things I went to Sebastopol to talk about this weekend, the people who got it got it, and everyone else wasn’t interested in the hard squishy details of real community. I think this might be the side effect of running social software for social softwares sake vs social software as bath for social media object sharing.

“Relationships can be broken down into 5 types: emotional aid, sociality, major help, minor help, and $$$”

Note to self: try block modeling interactions in high profile/high turn Flickr groups. (central, utata, etc)

No one really understands user expectations. Privacy expectation is currently, “unstable”.

Huge conceptual issues with the difference between public information hand aggregated, and public information computer aggregated. Cognitive dissonance ensues.

Rules, games, and rulesets. Modeling of social software as games. Tension of implicit vs. explicit rules. Mag.nol.ia’s altruism game derived from the cracks board (witnessing altruistic acts is a public good, way to update the Mag rules of game to support this?), Satisfaction’s status update game. Hoping Teresa can bring the quality gaming to BoingBoing’s anemic community. Social games + adversting.

Parody/pastiche as lit analysis. Investigate for web.

Social networks need NPCs. e.g. the Instructables Robot.

Standards works should be done in small groups, with a clear need, that selectively grow the list of participants. No hierarchy of early/late joiners (aka OAuth did it right)

“Everything public” bores me.

Beyond LAMP.

Find a feed for Nathan Eagle’s research.

“locations rights management”

“trusts are largely not transitive”

Language communities are “small world networks”, partitions communities by language. 2-5 hops vs 8 in analyzed network.

The Plaxo way: “We gets ze data Lebowski”

“Twitter is my early warning system. My blood pressure has gone down over the last 18 months”

Identity and sharing can make everyone warm and fuzzy, but also came face to face with sobering consequences that kept me up at night with a bottle of tequila. Re-thinking proposed Flickr features.


December 20th, 2007

“…it should take energy and thought to push issues upstream due to the associated costs of having to deal with them once they are propagated… when you optimise something you always do so at the expense of something else.” – Bill de hOra

“Social technologies that make things more efficient reduce the cost of action. Yet, that cost is often an important signal. We want communication to cost something because that cost signals that we value the other person, that we value them enough to spare our time and attention. Cost does not have to be about money. … Spending time with someone is a valuable signal that you care.” – Danah Boyd

Mind The Gap

Photo by bowbrick

Flickr: A Place of Our Own

December 10th, 2007

You might have seen the post on the Flickr blog announcing Places, or maybe the Good Reverend’s write up, but if you haven’t:

Places is a new Flickr feature that mines our corpus of geotagged photos, identifies characteristic features on a per location basis, and then goes back into the data looking for “iconic” beautiful photos. (btw try reloading that /places page, the feature places are random. As to a certain degree are the photos on the individual Places pages themselves)

It also is where a good chunk of my creative energy went for the last few months which is why the blog has been so quiet. And its a hell of a lot of fun, not to mention a privilege and pleasure to deep dive into our database and be reminded just how much fabulous photography there is on Flickr, and maybe just barely fumble around the edges of surfacing the diverse communities shared vision. Eyes of the world indeed.

A Place for GeoRSS feeds

Dan roped me in on Places months ago. We had geoFeeds working for semi-arbitrary places, and we needed a page to hang them off of. That page looked a lot like search result. You never saw it because the Flickr project management process (a blog post of its own) left that particular prototype a bloody, heaving wreck. Don’t worry, the current version is much much much better. (of course you also never saw Dan’s brilliant prototype of the current version, which was too cool to release on an unsuspecting public) And voila, many months later, the feeds are there. (though I’d still like to bring back that SRP view to allow rich searching within a location)

Increased Surface Area

We brought a bunch of different design goals to Places, but one of my obsessions that I think we nailed was the idea of “increasing the surface area” of Flickr. (also known as providing new ways to level up in the Game of Flickr[tm]). Only a few people, and a limited range of styles will ever be featured on the Flickr Explore pages. Which is fine, most people don’t care. But Places provides another way to recognize the contributions of Flickr members, by hilighting their geotagging and their photography skills. I’m looking forward to adding a couple more similar features to Places, recognizing other Flickr Games one can level up in, and other contributions back to the commons you can make.

Mo’ Betta

A bunch of stuff didn’t make our initial launch. Some of that has come in since then. More will be coming. I’m particularly excited about using adding some new data sources to improve the page. (e.g. the Groups right now a bit weak, and we don’t have reliable neighborhoods in cities, both of which are in process of being fixed)

Thats kH8dLOubBZRvX_YZ to You

Turns out there are a lot of San Franciscos in the world, and we personally struggle to keep track of which one is which. So we’ve been experimenting with giving them unique place_ids. If you look really close you’ll start to see these popping up around flickr, in photos.getInfo,, and as microformats on the Places pages. Its all very experimental, this unique identifiers thing, but we think it might work.

Arm Chair Travel

And because I love you, I’m going to let you in a on a secret. Have a great trip.

Just beyond the door

Personal Data Stores and the Network

October 31st, 2007

Thinking about what “personal data stores” are going to look like, how this interacts with decentralized models for community services, (I swear I’ve written something more recent then 2005 on that topic, but can’t find it), mulling models for updating clouds, wondering if projects like G’s OpenSocial, and Portable Social Networks are a step forward or back, speculating that digital curation is a viable near future business model, and that individual curations would work well as shareable social media objects.

Nothing necessarily novel. Just where my head is at.

Official Digitally Mediated Community Week

October 3rd, 2007

Online Community Summit pre-meetings kick off today, through the 5th.

Then CommunityNext runs the 5th and 6th.

While Graphic Social Patterns will be on the 7th, 8th, and 9th

Rick Skrenta: What do you do when your success … sucks?

April 3rd, 2007

Stone Cottage pointed to a great post by Rick Skrenta, CEO of Topix (and mad mind behind NewHoo/DMOZ for those who can remember back that far) on the Topix re-launch.

Lot of really interesting stuff about identifying a brand’s core value and putting it into practice. But also a phenomenal laundry for a problem that has stumped a lot of us, how to make a local news site succeed. Including:

  • Anthropomorphize our existing technology into the roboblogger. This was a brilliant idea from one of our lead engineers. It simultaneously solves three problems:

    1. Booting up a new city — you need posting activity to draw the first editors. The roboblogger would give us that. But he is shy and gets out of the way if humans show up and take over a page.
    2. If the community editors go on vacation, the roboblogger can step back in and take over while they’re gone.
    3. People know when a robot is editing the page vs. a human. His profile icon is a picture of a little tin-can robot. His handle is ‘roboblogger’.
    No more confusion.

A project has to already have value to draw a valuable volunteer base, this is the classic and yet fundamentally hard problem with boot strapping all local sites. But as soon as you have volunteers your contract with them is to rain attention and love down upon their contributions. roboblogger is a really neat hack to handle the delicate balance of a site’s lifecycle and mix community and data mining techniques in social software. Looking forward to watching it play out.

Early feedback on PMOG – Needs Community

March 14th, 2007

Okay PMOG is super early in its life, but it intrigues me on a couple of levels (not the least of which is the engaging archetype art).

However there are some things about it which are broken. Not surprising in and of itself, but in the process of trying to report said broken-ness I ran into a larger problem.

No community space.

There is a Google Group but it’s a moderated announce only kind of thing (HINT: thats what you’re blog is for!) not a public discussion space. No message boards, no wiki (though presumably we could start one, Twitter Fan style), no groups.

Someone needs to see Andy’s talk about group forming, social software, and out of band spaces.

Especially for a game, a social game, an experimental game.

Uninstalled for now, in an attempt to reduce unexplainable spinnies.

Twitter Curve

December 8th, 2006

First congrats to the Obvious kids (assuming any of you survived your wild bacchanalia of vegetarian and raw foods last night), you’ve made the big time!

You’ve got a Kathy Sierra graph named after you!

And she is talking about Continuous Partial Attention, which is also fascinating. But I think she is wrong about Brain 2.0 not being here yet.

Brain 2.0

I know my own brain is addicted to stimuli: flashing lights, amusing concepts, spinning tops, sugar, caffeine, loud noises, human connections, etc. Note the lack of “fascinating and important new information” in that list.

Unbolding and Broken Promises

That’s why email and feed readers can be so distracting, they slip past our intellectual safe guards by promising us “important and timely information”, but really we go to the trough hoping for stimuli, something to keep the howling 2 year hold cum crack fiend brains of ours from going into withdrawal.

And they don’t deliver. 99% of email is boring, 92% of RSS is boring.

Bad mornings are the ones where I sit at home compulsively unbolding things hoping that somewhere in there there will be the gem of connection and stimuli that gets me out the door.

So we have an activity which consumes a huge amount of time, and delivers low grade rewards intermittently.


And thats where Twitter comes in. I have Twitter going to IM (the excellent Adium). Messages appear at the top left of my screen almost in my peripheral vision, and fade away quickly with no intervention needed (thank you Growl), a quick squirt of connection without requiring agency.

IM is supposed to be asynchronous, but at that the same time you need someone to occasionally make “eye contact” with you to know you aren’t howling in the void, and Twit provides that, its stream of updates providing presence, and occasional directed response.

MySpace Hating

Social software as monolithic web destinations is going to go away over the next 5 years (with the exception of course of certain photo sharing sites). Right now tools like MySpace are structured around the very adolescent desire to make friendship and community a public and performative act, with everyone else piling on due to the network effect. Adults for the most part, don’t need, don’t want, and don’t have time to participate in the continuos and elaborate preening rituals. We’ll want tools that allow us to build tight knit groups, with low cost communication, asynchronous connection, and social discovery. They might look something like Twitter.

NYTimes on Friendster

October 15th, 2006

Reading the NYTimes Friendster post mortem , its 4 pages long, but the following paragraph jumped out at me as the most important.

Many people working at Friendster sneered at MySpace. The holy grail at Friendster — and the cause of most of its technical problems — was its closed system: users at Friendster could view only the profiles of those on a relatively short chain of acquaintances. By contrast, MySpace was open, and therefore much simpler from a technological standpoint; anybody could look at anyone else’s profile.

In less public spaces then the NYTimes business pages you hear a lot of gossip about Frienster, mostly the personality clashes and rock star egos, but I think it really was that simple.

Failure to do the simplest possible thing that will work, and a failure to be public by default. (though going to war with your users is never a winner) It hurts an engineer’s soul, but worse really is better. (thank goodness I’m not an engineer, just a failed lit major)

Tagged: Uncategorized , , ,