I’ve been playing with the idea of a devblog for a while, a place which collates the various feeds associated with a project into a simple location for easy viewing, and commenting it. Fisheye, and CIA are full blown attempts at solving this very problem for large open source projects. This morning, unable to get back to sleep, seemed simpler and faster to write something new, then finish reading the install instructions.

I present my quick and dirty RSS-to-Wordpress aggregator. There are a number of tools that allow you to display an RSS feed on your WordPress blog, and several that allow you to import an RSS into your blog, but I didn’t find any that worked as aggregators, periodically polling a feed, and creating new posts from the items found within. So that is what I built.

Some features:

  • dc:date becomes the postdate, dc:creator (or dc:contributor) becomes the postauthor, title maps to title, description to post_content, dc:subject to category.
  • a postmeta variable is used to track which RSS items have been seen before, and only insert them once.
  • you can assign 1 or more categories to be automatically attached to each post per feed. (e.g. a ‘CVS Commits’ category)
  • new authors, and new categories are auto-vivified if they don’t exist.

Limitations:

  • I wrote this to parse our internal cvs2rss feeds, and our Twiki feed. Those both produce RSS 1.0 feeds that conform to my particular RSS aesthetics. As such the script doesn’t try to hard to support other versions of RSS.
  • Links are treated as perma-links, and unique identifiers.
  • Ignores content:encoded
  • Only handles inserts, not updates
  • I wrote this with nightly build 2004-7-14, your milage may vary.

None of these would be hard to fix, but this was good enough for my needs this morning, in the short period of time I wanted to spend on it.

Todo

Some other features that would be nice to: - Store config in the database and add WP UI for managing aggregated feeds (should be doable with option groups?)

  • Support adding categories by name instead of id. (and auto-vivify categories)

Uses Magpie (surprise!), so you get to leverage support for fetching private RSS feeds. (which I’d recommend for serving up internal RSS feeds) Expanding on the devlog theme, you might want to include the RSS from your project management tool, and bug tracker. (we aren’t currently using RSS enable tools for this, but if, for example, you were using TasksProp or Basecamp, those would be good feed to include.) ### A Few Observations on WordPress

  • Doesn’t support PHP5 yet. I’m not sure how pervasive the problem is, but it uses a modified version of ezSQL for its DB abstraction layer, which isn’t PHP5 compatible. Too bad they aren’t building on top of PEAR DB.
  • Excellent for whipping up an attractive, feature rich blog.
  • Faster then expected, really zippy in fact, at least without load.
  • Code is kind of a mess (or at least old school PHP). Very little OO, SQL, and HTML is scattered around, core use of global variables.
  • I’m not a fan of the “PHP is already a template argument”, but I understand why some people are.
  • Doesn’t feel as polished as MT, but is certainly more hackable. Reminds me of the Kwiki “every installation is a snowflake” goal. Interesting to see if this creates a surge of creativity, or just balkanization.
  • Option groups, and the postmeta table make it incredibly simple to add new features.
  • PHP5+SQLite support, and a one-click install could make WP the Kwiki of blogging tools.

update [2004/10/28]: In Boston, but on Seattle time, fill in some extra details. This quick hack is growing faqs, and might need to sprout a page of its own pretty soon. In the mean time there is a new version which uses a simple config file, and has expanded del.icio.us support.

FAQ

  1. How do I print the link of the original RSS item? ```

    <?php echo getpostmeta($id, ‘wpaggrss_id’, true) ?>

    ```

  2. Dates aren’t working, all my posts are from 1969. Currently wp-rss-agg only supports dates in dc:date field, however there is a feature in magpie-cvs that should make it simple to provide Atom and RSS 2.0 date compatibility as well.

update: FeedWordPress is an actively developed and maintained version of this script. Charles has taken it beyond my simple proof of concept, and it is almost certainly what you’re looking for.