Laughing Meme

So a few months ago I started checking email again over POP3. (Go ahead, I’ll give you a minute to recover from your shock and dismay)

I’ve been using Apple Mail.app and found that I’m totally drowning under a flood of email that is barely a trickle of what I cope with through GMail. And I’m paid to deal with this stuff. Ugh. A better solution is clearly needed.

Digression

Is is too much to ask in 2006 that I might hope for a desktop application with the speed, flexibility, and responsiveness of a webapp? What I’d give for a GMail like client, with hackability, security, and offlineableness. (That was a hint!)

Working Notes on Thunderbird

I’ve only been using Thunderbird for a few hours now, but I’ve made some tweaks which make me think this might work.

Following these instructions, I’ve got the Buttons extension, which includes an “Archive” button. From there I assign a default Archive folder, and use keyconfig to bind archiving to ‘A’. While I’m in there bind delete to ‘D’. Nirvana!

Non-working Notes on Thunderbird

I describe my email coping strategy as “tag and release”. (it doesn’t work terrible well as my correspondents can tell you, but it keeps me sane). Respond (or not), tag a message, and get it the hell out of my inbox.

So now I’ve got release working, what about tag? Well I had high hopes for Tag the Bird. Unfortunately its mangling my emails beyond recognizability, and the UI needs a bit of work. I’ve got an email out to the developer so I’m keeping my fingers crossed. Expect updates, and watch the del.icio.us stream for XUL tutorials.

On Migrating

Mark’s right, emlx sucks, emlxconvert doesn’t work. JWZ’s exml.pl fared better for me. Haven’t spotted obvious problems yet, but I only ran it against a 2 month old account as well. Still this sucks.

I mentioned before how much I liked the Google Calendar daily agenda by email feature. I take it back. Its completely worthless. Why? Because they send it to you whether or not you have any events happening that day. How many times do you have to get a daily emails with zero content before your brain stops seeing it? For me it took about 3 days, and then it faded into the noise of uncaught spam.

I haven’t seen it, but someone has got to be working on it, so where is the open source webmail app, that can front-end my IMAP server, and works like Gmail? So calling out to the LazyWeb I haven’t seen it, but someone has got to be working on it, so where is the open source webmail app, that can front-end my IMAP server, and works like Gmail? So calling out to the LazyWeb

Its funny, Gmail just added the one feature that I was missing so much that I was ready to leave, the ability to customize the From: field, and yet I’m more ready to leave then ever. Why?

Spam false positives.

Bad Spam Filtering

False positives, are unforgivable in a spam filter, especially lots and lots of them. False positives mean you have to manually look through every spam message you get and manually check that each one isn’t spam. I don’t know what algorithm Google is using, but it sucks. I’ll admit my address has been out on the web for years, and so I’d understand if spam was getting through (and it does), but what I can’t understand is why:

• mail from the moderated mailing lists I’m on get flaggeds as spam
• mail from people already in my inbox gets flagged as spam
• mail from people who I’ve emailed gets flagged as spam
• mail from Google HR personnel

In particular Gmail seems to hate the microformats list, of which a significant percentage of the traffic gets flagged as spam. Editorial commentary I wonder?

Which is really a shame, as Gmail (or any of the centralized mail houses) should be in possession of plenty of information to do an excellent job on the filtering.

The Potential

And once we had our own Gmail-like client we could adding features without having to rely on Greasemonkey scripts!

My short list:

• mailing list aware
• roles ala Pine
• GPG integration
• archive this thread (aka conversation) and all future messages to it

That plus client independence with the IMAP backend. There have been a few good comments added to my original Gmail IMAP post if anyone is looking for inspiration.

Talk about getting it wrong. POP access to your GMail account is lame (who uses POP?), what I want is GMail to be an IMAP client to my own IMAP service, now that would be useful.

Does anyone know a good tool for indexing and searching IMAP folders? I’ve played with Zoe a bit, and I had trouble getting it (and keeping it) running, and it did more then I wanted, and was comfortable delegating.

Basic feature set would be a server resident process that indexed my IMAP folders and presented a simple search interface. Ideal features would include browsability, vfolders, and sorting by lists, etc. (yeah, yeah, sounds more and more like Zoe, I know)

I found a quick hack that implements the basic features, but I’m really worried about the foot print of Java on my server (though am willing to experiment if it turns out that all these products are built on Lucene), and am also hoping to find something that feels more mature. (and funes is another, which gets bumped up the interestingness scale by being written by Nelson Minar, and quoting Borges)

I suppose a tool for indexing and searching maildirs would also work.

Suggestions?

Next generation mailing lists are a subject close to my heart. Wedging our ongoing work and conversations into the confines of email has been convient over the years for leveraging an installed platform, but that doesn’t necessarily make it an appropiate fit.

A lot of work needs to be put into how to facilitate meaningful conversations that maintain state, how to make sure conversations are added to the institutional memory, how to do online descision making, etc, etc.

What is the upside?

One interesting thought expirement going on right now is, “what do we gain if we switch to publishing mailing lists via RSS”.(not to be confused with merely making lists available via RSS)

Chuq says, subscriptions are an artifact and non-essential, easily replaced as an authentication mechanism with challenge/response. (Something the W3C lists are already experimenting with, and some newsgroups like the comp.lang.perl.* have been doing for years.)

Brent riffs on password protected RSS as a replacement for email an idea that sounds very similar to the email over freenet ideas I heard a few years back.

Or maybe the whole idea of mailing lists is dead? (actually I disagree with that, lists create a space, a space has community, investment, and social norms, something a purely personal aggregated view can’t replicate)

Mailing lists are good for some things, and really really bad for most. In particular they scale poorly, both up and down. Many conversations suffer from crowded public space which a mailing list inhabits, one in which a rapid back and forth is either discouraged, or if allowed brings all other conversations to a screeching halt. One solution many people try is, “Start a new list”, which is why lists.indymedia.org had over 600 mailing lists last time I checked. Another is the private CC list which has the problems of being transient, difficult to join, difficult to remove yourself from, unstructured and unarchived.

I’ve proposed using quicktopic.com in the past, but people don’t seem to want to leave their email readers, and kicked around the idea of using Roundup, or at least extracting the ad-hoc list code from it.

Simplify the Problem

More recently, chromatic is rolling his own using Mail::Audit, which looks very interesting.(hooray for the power of procrastination!)

Dan Brickley just mentioned a patch to Mailman for producing RSS feeds for a list. While not the ideal feed described in my extended rant on the subject, its an incremental improvement, and much welcome.

• Dan’s email
• Mailman RFE
• The patch
• An example feed

A Case Study in Dysfunction

Mailing lists, as a discussion medium, particularily for online organizing, are dysfunctional and broken. Embedded in their nature is a discouragement of conversation, particularily the rapid back and forth which allows people to find common ground. As each email arrives, it raises the likelihood of earlier emails not being read. This culture has a tendency to produce, through social pressues, discussion that feels more like a duel, where each side is using mega-ton bombs, massive position statements, rather then negoiation and listening. These end when one side falls silent, leaving those left on the field unable to guess if agreement has been reached, or merely exhaustion.

There are many possible solutions ranging in scope and feasibility, technological, and social fixes. I think the ad-hoc mailing lists of Roundup are one possible solution. But what follows below are some ideas about a making the tool that most activists use for email reading, some form of webmail, better at facilitating mailing lists. (there ideas came out of talking with Riseup and Threespeed)

Threading

Threading is an obvious feature. Mutt has had it for forever, Usenet had it since before forever, its defacto for Web discussion boards, and even stately Pine recently added threading. So why not webmail? Threading is a great feature, not only doesn’t it make conversations more coherent to follow, giving each message an explicit context, and reduce the associated cost of each arriving message, threads, if done right, can mean that conversations can last much longer then the few days a thought normally sticks around a busy mailing list, before it is pushed inexorably out of the field of whats current. We have not only context, but memory, and history.

A wishlist for threading contains very little revolutionary. Messages should be able to be displayed in a threaded manner, threads some be collapsible, and one should be able to delete, or ignore (ie. delete any future message in the thread) an entire thread. Not sure what algorithm Mutt and Pine use for their threading, it might be worth checking out, because, unfortunately simply trusting the In-Reply-To header won’t be reliable out of the box, and threading on subjects, as some mail archivers do, is guarenteed to give you lousy resaults.

List Filtering

Simple filtering could go a long way towards dealing with the email overwhelm associated with mailing list. Either automatically or with a click of a button, my mail client should be able to detect a message as having come from a mailing list and send it to a mail folder, perhaps created on the fly, specific to that mailing list. It will be important to include visual and interface clues to the existance of these mailing list inboxes, perhaps borrowing the familiar visual language of bolding the name of inboxes containing unread messages, followed by the number of unread messages.
ListDetector, a plugin to Mail::Audit, contains recipes for detecting mailing lists, as does procmail-lib, so that shouldn’t be so hard.

Other features

Mutt contains some interesting mailing list features including a super-charged Reply feature when replying to a mailing list, allowing you to guide response to yourself, the list, or both.

A subtle visual cue that you are replying to a list rather then an individual (how about setting the web pages background to bright red?) would cut down on the number of private messages that find their way to lists cutting down on uneccessary traffic, and reducing the neccessity of list admins to go mucking in their archives deleting old messages containing sensitive info.

Interface widgets for handling the standard interactions with a list like unsubscribe, and switch to digest mode wouldn’t be too hard, especially with lists like Mailman (potentially others?) which include a slew of List-* headers for that express purpose.

And speaking of digests, how about an intelligent interface for interacting with mailing list digests? Something that links internally, maybe even allows to reply to chunks of the digest, but at the vary least refuses to send out emails containing subjects like “Re: digest”

Lattice asked me about resources for representing a mailing list as an RSS feed. Particularly, he was wanting to put together an alternative(post-email) interface to a Sympa mailing list, with an RSS aggregator for reading the list traffic, and the Sympa web interface for posting. I didn’t have any suggestions for Sympa, but noted that the script mmrss turns Mailman lists into RSS feeds.

A Thought

This inspired me to think about tackling the chronic lack of mail list archiving soltuions as multi-step problem. Perhaps if we could get the mailing lists into a suitable inbetween format, with all the assumptions exposed, and codified into XML, then perhaps the archivers simply focus on giving a decent presentation of the complex interactions of a mailing list. And having that format be compatible with the latest crop of desktop clients would allow people to build new and exciting ways of interacting with the lists.

So I compiled an overview of the problems, the challeges, and work that has gone before on building an RSS threading standard.

Read the rest of this entry »