Blog posts tagged "rss"

AOEMedia Sponsoring Magpie

June 7th, 2008

AOE media, a TYPO3 & open source provide from Germany, recently agreed to become a sponsor on Magpie. Which is very exciting!

Looking around I see being sponsored by AOE Media puts me in good company as they sponsor a number of interesting projects, including my favorite wiki software Doku

I’m hoping this will allow me to actually spend a little time with Magpie, maybe finally find the time to re-vamp the website into something a little more functional.

And last if you’re someone interested in sponsorship work as a whole on Magpie, or specific feature development get in touch.

Thanks again AOE Media.

Question: Worst possible sin in a feed aggregator?

April 20th, 2007

Answer: Showing old content as new

Ugh! Ack! Stop it. Life is too short to see old blog posts, god forbid blog posts I’ve already read.


In every application you have to figure out what your SLAs are, and this shapes every choice you make.

If you’re writing an aggregator (or maintaining an existing one) your first priority should be suppressing false positives for newness, everything else pales in comparison.

Despite the obvious geek-macho appeal optimizing timeliness is way way down the list.

LazyWeb: Suppressed Blog Posts

Wanted: a plugin for WordPress that holds back from the feeds all new content for 6 to 12 hours (depending on length, and categories) so you don’t have to see me correcting all my spelling mistakes in painful real time. (might also be useful in cutting down on real dumb blog posts)


Bloglines get your act together, you’ve been terrible for weeks, and I don’t have time for this. This morning I got a blog post from Caterina announcing Yahoo’s acquisition of I’ve been a user, fan and evangelist for 4 years, I hate every other online aggregator I’ve tried, but I’m leaving if this continues.

Magpie 2.0-alpha-alpha-alpha

October 16th, 2006

Probably surprises no one but me, but the work on Magpie has been moving slowly again lately. Rather then continue futzing with it in the rare spare moments, I’m going to push it out in a very raw state, where folks can give some feedback. (and because folks keep asking about it) This is a preview release, very alpha.


Alpha means its broken, there are missing features (many of which are actually in the current Magpie 0.7x releases), and stuff will change. Whether you find that exciting or off-putting is a personal thing.

Getting Started

So um, yeah, as of yet no documentation. That said I’ve included the classic magpie_simple.php script, unchanged except for the require statement. For really simple scripts that is all that is required.

Grab the current code, Magpie 2.0-alpha-PR1. This isn’t its final home, but I am taking this opportunity to finally break free of Sourceforge which has been a long standing goal.

(I know, we all prefer a good var_dump() plus source reading to docs, but their current non-existence won’t continue)

Goals for 2.0

There were three over-arching design goals in this rewrite, plus a slew of secondary goals.

1. Support new namespaces and elements, easily

Rather then go on trying to push the universal rules for mapping unknown elements to datastructures (the Magpie 1.0 approach) I’ve focused on making it simple to register custom parsing logic, and having intelligent defaults. (aka more like what does) I expect that we’ll handle most known namespaces in short order, and barring another total upheaval of the landscape ala the Pie/Atom project should sit us in good stead as feed use continues to become more sophisticated.

2. Pluggable components

You should be able to easily swap out the caching layer (database caches anyone?), the HTTP layer (multiplexed curl?), even the parser. Besides the added flexibility, the theory is this will make embedding simpler.

Who knows, maybe someone will even contribute a pluggable parser that can handle something other then well formed XML.

3. Mostly backwards compatible

For simple stuff, you’re scripts should go on working. There is still a 1 function interface (if you liked that), still bust everything down into a couple of nested arrays for easy looping and echoing. Even where different it should feel familiar.

Known Issues

  • Parser doesn’t support xml:base nor xml:lang nor Atom inheritance. It will, but these features still annoy me, and as long as no one is using the code I can’t seem to motivate to support them.

  • I’m also not doing all the normalization between feed types that I do in “Magpie classic”. Again, its coming.

  • Not sanitizing content yet.

  • Not just a new parser, but a new HTTP client as well. Basic HTTP auth support is there, digest isn’t. Haven’t added back in SSL support yet.

  • No documentation per se.

Incompatibilities and Gotcha

  • Just use $item['content'] instead of $item['content']['encoded'] or $item['atom_content'].

  • If caching is turned on, and Magpie can’t write to it’s cache, it will throw a fatal error, rather then quietly working in a degraded state. This might change, but its been a major support issue.

More features

  • fetch_rss() now supports taking per feeds options. This was the most requested feature, ever. (e.g. cache age, output encoding, user-agent, etc)

  • Parses enclosures, this was the second most requested feature.

  • Handles repeated elements properly. (dc:subject, link, whatnot)

    echo $item['dc']['subject'];
    foreach ($item['dc']['subjects'] as $subj) { ... }
  • Atom 1.0 support modulo known issues.

  • Tests. Most of them based on Mark’s FP tests. More added all the time. Currently not distributing with Magpie as I haven’t really figured out the license issues.

  • Confusing new licensing. Stated goal is to license under a dual GPL/BSD license. That means you get to choose if you’re using the software under the GPL, or the BSD. In addition you can upgrade your license to GPL from BSD (as you can with any BSD licensed software) merely by wishing it to be so.

  • Lots more, really.

Other then that, its just like Magpie :)

Tagged: Uncategorized , ,

State of the Art of Offline Aggregators?

May 20th, 2006

A lazy web request (or jet-lagged web as the case may be)

Is NetNewsWire still where its at for OSX desktop aggregators? I’ve been away from the desktop so long I don’t even know the players.

Do any of the aggregators do pre-fetching/offline-ing like NewsMonster did back in the old days? (Or did MarkP perhaps nip that line of development in the bud??) Something, you know, for long train rides?

Are there open source clients that sync with the NewsGator API, and run on Linux or do only NewsGator products sync with NewsGator?

Also want a host of other attention based/sensitive features, but to brain fried to put those into words. In the mean time I think the reading list will under go serious (if perhaps temporary) winnowing.


Tagged: Uncategorized , , , ,


January 11th, 2006

James Holderness (check the comments)

Do you ever think maybe the Apple guys are just winding you up? Nobody could possibly be that stupid.

Maybe, though I tend to share Phil’s skepticism. Lets start the with the name, “photocasting”. Worst name I’ve heard since “MacBook”. I’d speculate that Apple’s marketing department recently started outsourcing to Engineering, except I’d be slandering my own profession. That’s a minor thing, aesthetic really, but dear god, how could they screw up the RSS? Again? (especially as I know people at Apple who are not only smart and clueful, but get XML)

I mean the iTunes name space was a train wreck. (though truth be told podcasting for some reason produces the scariest, wackiest feeds on the planet, at one point roughly 1/3 of the feeds Odeo was crawling had serious errors)

User agent detection? Of RSS? In 2006?!? Come again?

Embedded CSS? Misformatted dates? Random, namespace-less new elements (PhotoDate?) A new standard for including comments within an item.

See Phil’s comment, Sam, Dave Winer

Hey Apple, consider hiring someone who knows something about syndication, it’s worth it.

Take a look (unless of course you’re using Firefox).

update: [2005/01/18] MarkP on “photocasting”. It’s not just bad, it’s spectaculary bad. (via)

Tagged: Uncategorized , , , , ,

Yelp, User Contribued Content and Feed Design

December 23rd, 2005

Yelp gets feeds just right. I’m not sure I’ve ever said that before about anybody.

They’ve got:

  • a feed of my reviews allows me to re-purpose the content I create (No RSS, No Content Creation],
  • a feed of my (as yet non-existent) network’s reviews
  • feeds contain the full content of my review (again my content, I created it, give it to me)
  • rich feeds; they use the geo namespace to embed lat/long
  • Both RSS and Atom 1.0 feeds

Very nice.

A minor nit. These are feeds of reviews, not feeds of places, so it makes sense that the rating is included in the title of the entry, but I’d still like to see the rating and location’s name presented in separate structured elements as well (say for example I want to syndicate my reviews locally, and display a graphic of the stars)

Also per category feeds might be useful.

Atom and Wiki Driven Testing

December 20th, 2005

Its been a long standing todo to port Mark’s FeedParser tests to work against Magpie, possibly with an intermediate representation to allow cross-language testing. (has any work been down on capturing unit tests/acceptance tests in XML?) Sam’s approach hilights Ruby-the-language’s awesome flexibility (I’d been playing with something similar for the parser we wrote for Odeo), but doesn’t map to PHP/Magpie very well.

Phil kicked off a new round of testing for Atom 1.0, the results of which are now captured in the Atom wiki. (not to mention a few gentle nudges on Magpie’s lack of 1.0 compliance.)

All of which got me thinking, it would be exceptionally cool if someone made the FeedParser’s tests available on the Atom wiki using Ward’s FIT concept in a documented, reportable fashion.

Any takers?

Tagged: Uncategorized , , , , , ,

Test Post

December 10th, 2005

This post isn’t here. Its a product of a gremlin-esque heisenbug. The rest of this post is considered classified.

Tagged: Uncategorized ,

Gmail: Web Clippings

December 9th, 2005

Gmail’s new web clipping feature is nice. Decent if sparse selection of feeds (doesn’t feel like a straight pay for placement deal), good interface for adding, selecting.

Minor detail: adding new feeds to the system seems to be broken. (Google is really cutting themselves slack these days about rolling out half finished features aren’t they?)

Best possible feature? No text to unbold! My days are already spent unbolding text, or feeling guilty about the ridiculously large bold (now turning red, now blinking) number which signifies my news folders’ backlog.

Much nicer. Obviously not a replacement, but as an addition to my existing aggregator, its great.

On a related note, what are people subscribed to for their news these days?

update: Well that was short lived. Should have seen it coming, but would it be too much to ad just one feature not targetted to selling more ads? I’m pretty sure “Catch More Bass”, wasn’t listed in any of my feeds. Siiiigh.

Tagged: Uncategorized , , , , ,