Plugin: FeedGrab

Awesome! This is just what I have been looking for. Will it work with Atom feeds?

Thanks for sharing this plugin.

This plugin is totally awesome, and just what I was looking for to duplicate some of my Flickr photos on my photolog!

The only problem I have spotted with it so far, is that it sometimes creates duplicate entries. Not for everything, but maybe every other entry or so. The first time it shows up in the RSS feed, it gets added fine. But if I then run the plugin again at a later date, when there are new items in the feed as well as old items, it will add some of the old items as new entries again, even though they already exist within EE. The title and date are still exactly the same for them - any ideas why it would be adding some as duplicate entries? It’s a pain having to weed through it every time to delete the duplicate entries!

In case it helps any, I am using this RSS feed (linked from this page):

http://api.flickr.com/services/feeds/photos_public.gne?id=95239135@N00&tags=stillburningcom&format=rss_200

And here is an example of the output I got when it created several duplicate entries, while adding a few new ones as well:

Fetching: http://api.flickr.com/services/feeds/photos_public.gne?id=95239135@N00&tags=stillburningcom&format=rss_200
Checking: St. Malachy's Church
Date: 2004-05-23T02:59:00 +0100 (2004-05-22T17:59:16-08:00)
Found: St. Malachy's Church
Checking: St. Anne's Cathedral Detail
Date: 2004-05-27T22:35:00 +0100 (2004-05-27T13:35:40-08:00)
Found: St. Anne's Cathedral Detail
Checking: Mount Stewart
Date: 2004-08-13T03:36:00 +0100 (2004-08-12T18:36:44-08:00)
Found: Mount Stewart
Checking: St. Anne Cathedral
Date: 2004-05-27T22:45:00 +0100 (2004-05-27T13:45:28-08:00)
Found: St. Anne Cathedral
Checking: St. Anne's Cathedral
Date: 2004-05-27T22:34:00 +0100 (2004-05-27T13:34:59-08:00)
Found: St. Anne's Cathedral
Checking: Stairs
Date: 2004-11-20T00:26:00 +0000 (2004-11-19T16:26:46-08:00)
Found: Stairs
Checking: My Street Sign
Date: 2004-05-17T00:21:00 +0100 (2004-05-16T15:21:00-08:00)
Found: My Street Sign
Checking: Greenhouse
Checking: Mural
Date: 2004-05-27T22:07:00 +0100 (2004-05-27T13:07:07-08:00)
Found: Mural
Checking: Falls Library
Date: 2004-05-27T21:52:00 +0100 (2004-05-27T12:52:07-08:00)
Found: Falls Library
Checking: Dirty White
Date: 2004-05-23T03:09:00 +0100 (2004-05-22T18:09:50-08:00)
Found: Dirty White
Checking: B&W Shed
Date: 2004-06-14T22:53:00 +0100 (2004-06-14T13:53:23-08:00)
Found: B&W Shed
Checking: Dry Boat
Date: 2004-07-27T23:07:00 +0100 (2004-07-27T14:07:51-08:00)
Found: Dry Boat
Checking: Barn
Date: 2004-07-29T21:27:00 +0100 (2004-07-29T12:27:23-08:00)
Found: Barn
Checking: Bobby Sands
Date: 2004-05-27T21:52:00 +0100 (2004-05-27T12:52:35-08:00)
Found: Bobby Sands
Checking: armedmural
Date: 2004-05-27T22:19:00 +0100 (2004-05-27T13:19:29-08:00)
Found: armedmural
Checking: Belfast Castle
Date: 2004-05-26T21:10:00 +0100 (2004-05-26T12:10:01-08:00)
Found: Belfast Castle
Checking: Sunbeam & Beach
Date: 2005-02-25T23:42:00 +0000 (2005-02-25T15:42:32-08:00)
Found: Sunbeam & Beach
Checking: St. Anne's Cathedral Inside
Date: 2004-05-27T22:39:00 +0100 (2004-05-27T13:39:58-08:00)
Found: St. Anne's Cathedral Inside
Checking: Snowy Car
Date: 2005-02-27T17:50:00 +0000 (2005-02-27T09:50:44-08:00)
Found: Snowy Car

Other than that, thanks a ton for creating this plugin! :D

Will it work with Atom feeds?

It should work with Atom feeds. It uses the Magpie RSS library (version 0.6a) which claims to support RSS 0.9x, RSS 2.0, RSS 1.0, and Atom 0.3.

The only problem I have spotted with it so far, is that it sometimes creates duplicate entries.

I’m sure there will be more problems than that!

Rather than checking for duplicate entries by title, the plugin attempts to look for entries created more recently than the previous time the plugin was called. This doesn’t seem to work very well, especially across different timezones and with daylight savings. It also requires the site that generates the feed gets its timestamp right, which I’ve noticed doesn’t always happen.

I’ll see if I can improve the method of checking for duplicates.

Many thanks for your feedback.

Thanks Andrew! For me personally, checking on entry title would be perfect - or possibly a combination of entry title and entry date…my photos would sometimes have the same title, but they’d all have different “taken on” dates, even if it’s just by minutes or seconds. 😊 If you need someone to do some testing, please don’t hesitate to let me know!

checking on entry title would be perfect - or possibly a combination of entry title and entry date

This version checks title and entry date. Its relatively untested, but doesn’t do anything dangerous - it just checks for matching titles and dates before adding a new entry.

Andrew

Edit: Please see the FeedGrab page to download current version

Andrew, does this new version still break the plugin manager?

Yes, the plugin manager still breaks.

I’m pretty sure it is because the plugin manager uses the same Magpie RSS code as the FeedGrab plugin. It would be nice to share the Magpie plugins code somehow, but I could code around this, I guess.

OK, another update…

I think this fixes the problem with the plugin manager not showing. The RSS code is still duplicated between this plugin and the Magpie one, but FeedGrab now checks to make sure it doesn’t redeclare the same functions (which was causing the error).

You really do need to fix that. 😊 I use the plugin manager all the time, so not being able to get to it is a major problem.

Edit: Works now! Thanks for fixing it.

Works now! Thanks for fixing it.

Thanks for the feedback, Sue.

Andrew - thanks for the fix earlier! I downloaded the plugin and ran it, but it had now completely stopped adding any entries. I made one small change in the source code that fixed that for me though. I changed line 183 to:

if ( $published < $date ) {

The < was the other way around first (>). I’m a bit confused regarding the logic around that…but it works for me now, even if I can’t figure out why. 😉 Thanks again!

Thanks Sasha.

The plugin is trying to check a timestamp so that previously added entries aren’t added again, but this test is now superceded by the test on entry name and date.

I’ll remove the timestamp check completely from the next version and just rely on checking the title and date. It hits the database a little more - which is why I didn’t do it initially - but it will work better.

I’ve removed any attempt to check timestamp - they were too problematic. A new version of the plugin is available below: