Self-Service Feed Aggregation with WPMu

This post will detail how to create an aggregator site wherein people can simply add their feeds to a site and have their content automatically re-published. This example is specifically for WordPress and/or WordPress Multi-User. It depends upon three plugins, so download them ahead of time from the links below:

1) Oz Politics’s BDP RSS Aggregator

2) Andre Malan’s Add RSS extension widget for BDP RSS

3) Charles Johnson’s Feed WordPress plugin

Here’s how (and note that all the images below link to larger versions for your viewing pleasure):

Setting up BDP RSS

First you need to install, activate and setup the aggregator plugin BDP RSS. I will leave the installation and activation of this plugin to you, because it is the same as installing any other. However, the setup may benefit from some detailing. Once you have installed and activated BDP RSS, go to the Manage tab and find and click on the RSS Feeds subtab. You will then be taken to the space for managing feeds with this plugin. Which will look like the following:

BDP RSs Managment screen

This is pretty straighforward, you add feeds here, and poll them to pull all the newst conenct (which happens automatically based on the time you set). Andre Malan’s Add RSS Extension for BDP RSS actually automates this process from the front page of the blog, but more on this shortly.

If you scroll down a bit, you will be taken to the “Output formats” section of this page, this is where you control the output of the feeds you are aggregating, and this is the portion of this plugin that needs some explaining.

Once a number of feeds have been added, click on the edit button of the output formats (of which you can have several, but for this functionality you will only use one output format with the id 1). Once you click on the edit button you will be taken to a configuration page with a lot of options that can be overwhelming, so let’s take a look at them in some detail:

Here is where you an name the output format and decide how you want the aggregated feeds to be listed, whether chronologically, alphabetically, etc. Additionally, You have the choice to select only certain feeds, or to list all feeds, for truly automating this function so you don;t have to keep coming back to this page, I would just leave the radio box checked with the default of “List all sites.” After this you will see a series of other options below for how many post, how characters to display, in addition to other settings. The XHTML formatting for list presentation shown below is for custom formatting, but I never mess with this.

The XHTML tags to retain in this re-posting is something I do use, and you can see the options I select below:

After this, you get to a series of custom options for archiving, caching, or creating a feed of your aggregated feeds (an OPML feed). I will ignore the archiving and caching options, and focus on the “RSS feed from list” option you will need to create a feed of your feeds. Also, they don’t make this clear, but once you create an overarching feed, it will have the following url:

http://yourdomain.org/?bdprssfeed=1 (with the number being the ID of the Output format)

For example, the feed of all the aggregated posts on Reading Capital would be as follows:

http://readingcapital.org/?bdprssfeed=1

After you check the box for allowing an RSS feed to be provided for the list, then you can save your changes and you should be done with setting up BDP RSS.

Setting Up FeedWordPress

Now that we have set up BDP RSS, we can now install and activate FeedWordPress, which will actually syndicate the feeds that are being added into BDP RSS. The setup for this will actually take the feed for the list of feeds we created in BDP RSS, and simply republish these feeds within, for this example, Reading Capital. So, for our example, take http://readingcapital.org/?bdprssfeed=1 and go to the Syndicate tab in your WordPress backend.

You add the url for the feed (http://readingcapital.org/?bdprssfeed=1) in the “Add new syndicated site” text field, and click on the syndicate button, which will then test and preview the feed to make sure it works, after that click on the “Use this feed” button.

Once you have done this, you need to set up the publishing options for the feed under the Syndication–>Options tab. Below are the settings I am using, you have numerous choces, and you can choose what works for you, but I prefer to turn off comments on the aggregation site, and make the permalink link back to the original post on the author’s blog.

Once you have set these options and saved them, you need to go back to the main syndication page, check the radio box of your feed, and click on “Update Checked Links” –which is you follow my settings in the Options above will happen automatically from now on.

3) Allowing Users to Add their RSS feeds from the Front page

Finally, install Andre Malan Add RSS extension plugin for BDP RSS (follow his instructions for installation) and go to the design–>Widgets section of your backend, and drag the widget into the sidebar. After that, as people add their feed on the front page of your blog, it will automatically be inserted into the BDP RSS list of feeds, which i turn will be run through FeedWordPress and re-published on the blog. Genius? Yeah, it is, isn’t it! And it’s all Andre Malan, so kudos to him!

This entry was posted in rss, widgets, WordPress, wordpress multi-user, wpmu and tagged , , , , , , , , , , , , . Bookmark the permalink.

13 Responses to Self-Service Feed Aggregation with WPMu

  1. Pingback: Reading Capital, Part 2: Blogs, Feeds, and Aggregation at bavatuesdays

  2. Luke says:

    Jim: this is great stuff. We’re trying to set up something similar on Blogs@Baruch but are running into the following problems.

    – Enabling mbstring in our PHP to get BDP RSS working (we have the sysadmin working on that one– should be easy. But then again…)

    – Feedwordpress creates a conflict with Flash interfaces in our Admin Panel on WPMU 1.51 (completely knocks out Flash media uploader in Write interface, can’t even “show” images uploaded with browser interface); forums for the plugin suggest that there are conflicts with the Flash sbm too, but I didn’t even test that once I saw the previous conflict. Seen anything like this in your backend with feedwp? What version are you running?

    Automattic suggested feedwp as an alternative to BDP, not as a complement… I’m a little confused as to how/why you’re using them together, since they share some functionality… far’s I can tell, it’s so that you can use Andre’s plugin to allow users to dump in their own feeds. That’s a good enough reason; but are there others?

    Thanks for blazing yet another trail on this one.

  3. Reverend says:

    Hey Luke,

    I believe that Charles Johnson is constantly developing FeedWordPress, so those issues should be fixd soon. Haven’t had the image issues, but I’ll double check. Also, it isn’t to much of a big deal for the reading capital site because the main blog will just be for aggregation not writing posts etc., and all the other blogs that don’t install FeedWordPress won’t have that issue.

    The FeedWordPress vs. BDP RSS is a good question, and you provide me the space here to explain the logic, which is something Andre and I talked about. Namely, the best thing about BDP RSS versus all the other auto-blog/spam plugins is that it is awesome at validating and accepting almost any feed. FeedWordPress is a lot more finnicky (sp?) and chokes on far more feeds. Having an extension for FeedWordPress like Andre has created for BDP RSS would make a lot of sense in the bear future, but I am still more comfortable with running all the feeds through BDP RSS to ensure that they get validated, don;t throw an error, and aggregate everything more cleanly than FeedWordPress may on its own.

    That said, different folks may have had different experiences with this process, and I don’ pretend that my experiences are true for everyone, but Andre and I separately had the same experience, which for the moment makes me comfortable with adding BDP RSS as a kind of go-between for organizing, customizing, and ensuring the feeds work.

    Make sense?

  4. Pingback: » Reading Capital, Part 2: Blogs, Feeds, and Aggregation Reading Capital

  5. Luke says:

    Totally. This really helps with thinking about our system. Once we get the thing configured, I’ll write something up.

    BTW, FeedWP is good, but it’s no Middle Passage.

  6. Reverend says:

    Cool Luke,

    I think a direct feed into FeedWordPress is ultimately how this narrative will end. And I may have to talk to Charles Johnson about this, for like you I am a fan, and read this in undergrad, a good book, but perhaps not quite up to par with Middle Passage or FeedWordPress.

  7. Reverend says:

    Also Luke,

    I tested FeedWordPress and the Flash issue you were talking about, and it is, indeed, an issue with image—and widgets too I discovered. It kills the AJAX as you said, and makes the general functionality of a site with this activated rather limited in that regard. I can withstand it for the time being, and I want to believe it will be remedied soon.

    Thanks for the heads up.

  8. Andre Malan says:

    Jim, this is a phenomenal explanaition of how everything works. As a side note, I just updated both the “add user” and “add feed” widgets to work on a password system instead of a list system. The rational is that if a professor wants students to add feeds to a course blog, all he has to do is give them the password in class and then they can add at will.

  9. Russ Cobb says:

    Hi Jim,

    I’m a little confused. I’m running WordPress MU 2.7.1 and I’ve set up BDP RSS on one blog to pull posts from another, but I’m still getting nothing under “Recent Posts” in my sidebar. I don’t have any errors and I’ve confirmed that the feed I entered is working.

    In the read-me that comes with the plugin, it says to hardcode the output code to the sidebar, but I’m assuming we don’t have to do this for Mu?

    So I guess my question is, do I need to install FeedWordPress for BDP RSS to actually work? What might I be doing wrong?

    Thanks,
    Russ

  10. Reverend says:

    Hey Russ,

    I would actually use FeedWordPress, not BDP RSS, it’s the next generation of RSS aggregators, and I think it will work far better for your purposes. It sill doesn’t aggregate comments, and that is what I often use BDP RSS for in the sidebar.

    Does this make sense?

  11. Russ Cobb says:

    Jim,

    Thanks a lot. I’ll give FeedWordPress a shot. I had wp-o-matic working like a charm until I noticed that it was creating duplicate posts. There’s been a lot of clamoring for the author to fix this because it’s a *great* plugin, but to be fair he might not have had the chance to test it past 2.6. WP seems to be cranking out new releases every couple of weeks, outpacing a lot of previously working plugins and authors’ ability to make appropriate fixes.

    Cheers,
    Russ

  12. Reverend says:

    Russ,

    Yeah, FeedWordPress has been developing in step with the recent WP releases, but I hear you on the firehosing of versions recently, and 2.8.2 is not for the faint of heart, there are many a bug yet to be squashed. Good luck.

  13. Pingback: BrSeek | Lista de plugin feed aggregator

Leave a Reply to Andre Malan Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.