Plugin: FeedGrab

Thanks for the quick reply Sue…. And thanks to Andrew for all his hard work on this plugin.

Let me just throw out two other questions for anyone…

First, it seems to me that if someone modifies content on the original RSS feed subsequent to the import which changes an item within the RSS, there is no way to import the changes. The imported copy will continue to reflect the older version of the RSS item. It would be nice if the plugin could identify such a discrepancy and change the content that it had previously imported. Is this an insurmountable problem?

Secondly, is there a way to include conditional logic to say something like “don’t import this item if the category is xxxxxx” or “only import if XML field xxx contains xxxxxx”

Hi Douglas,

Thanks for your interest…

You are right that existing items do not get updated. Furthermore, if an item’s title is changed (either in the feed or the weblog) that item will get imported again, as the title and the entry time are comnpared to test to see whether that item has been imported previously.

I’m not too happy with this and want to fix this in a future release of the plugin.

Once I can correctly identify a unique item, then it should also be possible to get items to update when they change.

I hadn’t any plans to include conditional logic, but I’m open to suggestions.

I’m currently planning improvements to the FeedGrab plugin and one of the issues I’m wrestling with is the trade off between extra features and ease of use (and efficiency to run for that matter).

I’m open to any suggestions or feature requests (from anybody) - you’re welcome to contact me ‘off-forum’ if you’d rather not post here.

• Thanks

Hi Andrew,

Sounds like this is a great plugin. I tried installing it but it broke the plugin manager – seemingly a bug that was already fixed. I get a blank screen.

I’m running the latest build, just installed it yesterday…

Any version above 0.5.3 should have fixed the majority of plugin manager problems. If you’re running a newer version than that, can you let me know what other plugins you have installed as it might be clashing with one of those - thanks.

I have the following installed:

WeatherIcon  (v.1.0)

Search Hilighter (v.1.2.1)

Character Limiter (v.1.0)

Technorati (v.1.0)

Word Limiter (v.1.0)

Magpie RSS Parser (v.1.3.3)

Randomizer (v.1.0)

Amazon ECS PHP5 (v.1.0 10/15/2004)

Comment Leaders (v.1.1 - 5/29/04)

XML Encode (v.1.1.1)

breadcrumb (v.0.1.2)

I’ll try installing it again right now and see what happens. BTW, it’s not in the Plugins list in the Plugin Manager, is that because it’s still considered beta?

Yep, broke again…

I’ve tried 6.2, 6.1 and 5.4 now. All seem to give me a blank page on the Plugin Manager.

Thanks in advance for any efforts on your part to help me here. Not sure why I’d be having problems when no one else is except that I upgraded to all the latest EE builds (forum too). Let me know if there’s anything else I can provide to troubleshoot.

Thanks - I’ll take a look at the plugins you’ve listed and see if I can see any problems.

I’ve added all those plugins (except the Amazon one that needs PHP5) to my install and don’t get any problems.

If you are happy dabbling with your EE installation you could try editing the file: system/cp/cp.utilities.php

On line 758 (on my EE install v.1.5.2) change:

if ( ! @include_once(PATH_PI.$file))

to

if ( ! include_once(PATH_PI.$file))

The @ sign suppresses errors, so removing it might gives us some clues to what’s breaking. Ta.

OK - I’ve a new version of the FeedGrab plugin ready to release.

A couple of exciting (and slightly scary) new features:

Attributes

You can now access attribute values from feed elements, eg, flickr uses fields like:

<media:content url="http://farm1.static.flickr.com/116/301933412_fad89245a8_o.jpg" 
                       type="image/jpeg"
                       height="1200"
                       width="1600">

You can now access the url, type, etc attributes using media:content@url, media:content@type at the field name

User defineable ‘uniqueness’

Previously, the plugin always used the title and entry date to determine whether a feed entry already existed in the weblog. Now you can use any (combination) of your weblog fields to test. (Most feeds provide a global unique identifier for each entry - I’d suggest adding a guid field to your weblog and sticking the guid in there.)

The parameter is called unique and takes a comma separated list of weblog fields to test for duplicates.

(Note for those interested: this is my first step towards allowing entries to be updated)

Here’s an example plugin call (to flickr) using the new features:

{exp:feedgrab url="http://api.flickr.com/services/feeds/photos_public.gne?id=YOURIDHERE@N00&format=rss_200" 
                          weblog="2" 
                          title="title"
                          date="dc:date.Taken"
                          use="media:content@url|media:thumbnail@url|guid" 
                          fields="feeds_url|feeds_thumbnail|feeds_guid" 
                          unique="feeds_guid" }

There’s also a debug=”off” parameter that stops the plugin reporting what it is doing, if you would like it to run silently.

Edit: Please see the FeedGrab page to download current version

This is unbelievable. I wish I had found it a year ago!! Thank you.

I’m gonna cry.

The new version does the same thing. I get nothing in my plugin manager, and this is on a completely different (client’s) EE site and install…

Hi wondermonkey,

I’m keen to find what’s wrong here (although I’m a bit snowed under at the moment, so excuse me if I’m slow to respond).

Could you email or PM me the results of: Control Panel > Admin > Utilities > PHP Info

That might give me a clue…

(The php change in post #38 above would be better, but I understand if you don’t want to go changing your codebase)

Andrew

Thanks for this - this is a really great. You should be proclaimed King for a week or at least a day!

A feature request [I’m sure you are fed up of these!] would be to have the categories from feeds dynamically inserted into the weblog category field. This is <dc:subject> in del.icio.us. Currently I think you need to create the categories before the feed comes in.

Of course you can get round this but it would be nice to have the categories assigned automatically.

I’m planning to map my delicious categories to my weblog categories and use them to serve targeted links according to posts. All doable with this great plugin and cron!

This plugin also breaks the plugin manager for me - on PHP 5.0.4 / Apache 2.0 / MySQL 5

Thanks again!

Sorry for the delayed reply…

I think the plugin can already do this (but it might not be documented).

There are 3 parameters:

cat_field - which would be, in your case, cat_field=”dc:subject” cat_delimiter - I think del.icio.us categories are space separated, so use cat_delimiter=”SPACE” cat_group - this is the id number of the category group you want to add new categories to (I cannot derive this anymore as weblogs can now have multiple category groups)

As I say, I think this should work 😊

I need to try and fix the plugin manager problem. As I cannot reproduce it, can I contact you for some help when I get a chance to look at it?

Thanks,