The State of RSS, Magpie, and FoF

May 28th, 2003

I downloaded Feed On Feeds 0.1 (changes), and am very happy with it. Steve is right, the new features I asked for do clutter up his interface a bit, but they work great! I imported an OPML file of the 85 feeds I most often read and it gave me the chance to kick not only FoF‘s tires, but push Magpie a little more then I usually do, and check on the general state of RSS in the wild.

And the news is good!

FeedOnFeeds: The Good News

FoF smoothly imported my OPML subscription list of my 85 most read sites with only 5 failures, one of which was clearly marked as an HTTP timeout.(a protest.net feed, whoops!) I was thrilled. I’ve been living in Straw’s parse-at-all-costs bunker, while I assumed the horrors of invalid, unparseable RSS ravaged the outside world. Imagine my surprise to peak my head up expecting Armageddon and to be greeted with flowers, sunshine, and a 4% failure rate. It gets better.

RSS: The Good News

FeedOnFeeds generates a link to Sam and Mark’s RSS Validator(which, with its high visibility and plain language feedback has been hugely beneficial in cleaning up RSS), and I was able to quickly check why the feeds were failing. The validator threw “Sorry an error occurred” when parsing 1 of the 4, marked the 2 down for style points but passed it, and said 2 of my rejected 4 were perfectly fine and why was I wasting its time. A little puzzled, I went back to FoF and added each of these feeds one at a time, and now they all succeeded. I think it must have been timeout problems, as Magpie sets very low timeouts by default.

Of the 85 feeds I read, all of them were parseable with an XML parser, none of them were invalid. I know we’ve got to comprise our ideals of perfection, deal with the fact that content producers can’t be trusted to produce valid XML, I accept that, and I’m willing to accept a 0% failure rate if you are :)

Magpie: The Pretty Good News

As much as I’m loath to confess it (and I’ll probably have a pack of agile developers howling for my blood) I’ve never really tested Magpie extensively. It seemed to work, in theory it should work, and I assumed that users would let me know if it didn’t.(bless ESR’s little heart for such a well made rationalization) So while I wasn’t surprised to see that Magpie dealt handily with a diverse array of RSS files, I was very happy, and quietly relieved.. With one exception. Kevin Burton’s Peerfear feed.

I knew it was out there, lurking, waiting to pounce. I first heard dark rumors of this feed last Fall; whispers in the street, seen it in the haunted, hunted looks in the faces of the parsers and aggregators around me, and finally one day, XML::RSS turned up on my door step, confused, and clearly in pain. You see it isn’t just an 80k behemoth of a feed, Kevin, quite legally, uses the XHTML namespace as the default namespace for his feed, instead of the RSS namespace we all expect. Having patched XML::RSS I knew someday I would be faced with the problem in Magpie, but up until this point I had cleverly managed to avoid ever facing the issue. (Damn you Steve for seducing me into this with your nifty web aggregator.) I’ve known for a long time Magpie’s innocent approach to namespace could never last, but I think I’ll let it enjoy its youth for a little while longer.

Addendums

  • You could argue I stacked the deck a bit on the RSS failure rate, as yesterday I helped fix a broken that would have caused me problems today. But I like to think of that as a happy accident.
  • Does your spell checker ask “FoF: Did you mean FOAF?”, mine does (admittedly not be default)

Tagged: Uncategorized , , ,

Comments are closed.