RSS Feeder

[steve51184]

RSS Feeder seems to automatically disable certain feeds after getting timeout errors. Is there any way to disable this?

i'd also like to know this

What error is in the error log?  is it fsocket timed out ?

damn i cleared my logs will need to wait till it does it again :\

[Spuds]

When it does post the error .... its easy to stop the mod from disabling a feed, but there are legit reasons to do so on certain occasions so we just want to tweak it so it does not do it on a recoverable (ie timeout) error.

[steve51184]

here's the error:

cURL error 6: name lookup timed out

[SlammedDime]

curl error 6

In a nutshell, means that something is wrong with your hosts DNS lookup system causing cURL to fail.

[steve51184]

curl error 6

In a nutshell, means that something is wrong with your hosts DNS lookup system causing cURL to fail.

so this is a problem with 'name-resolution-lookup' how do i fix this?

also i've just got a new error:

(cURL error 28: Operation timed out after 10000 milliseconds with 0 bytes received)

p.s. i think this might just be a problem with slow feeds as some of them can take 10+ seconds sometime... is there a way to change the time out?

[GreenMotion]

In my case, I am getting timeout errors. The site I am pulling feeds from isn't always the fastest.

These are the errors I occasionally get that in turn cause the feeds to disable themselves:

Code Select Expand


(cURL error 28: connect() timed out!)


Code Select Expand


(cURL error 28: Operation timed out after 10000 milliseconds with 0 bytes received)


Perhaps it'd be a good idea to implement some code that will only disable a feed if it encounters the same problem XX durations in a row. Where XX is configurable?

I.e. If the feed (that runs from a scheduled task say every 90 minutes) encounters the same problems on 4 consecutive iterations. Then have it disable itself. If it encounters the timeout on a couple consecutive iterations, but not yet hits the maximum configured number; and the next iteration is successful, then the counter would obviously reset itself.

  GM

[steve51184]

tbh i think it's best NOT to have it disable them at all...

[SlammedDime]

If you don't want it to disable them at all, feel free to modify the code in ScheduledTasks.php to do that... I won't be removing that code or part.

As for increasing the timeouts, I'll have to research a bit on that to see what SimplePie's capabilities are.

GreenMotion - that idea should be doable without much extra code... just an extra database column and some extra checking to see how many failed attempts.

[GazOutEast]

zx71 - I'd guess the problem with that, is that if you have a lot of feeds timing out, with a high frequency attempt to call them, then you could find yourself in a position of using excessive resources on a shared server, and your hosting account gets suspended by the hosting company - they would then insist that you disable or remove the script or mod causing the resource abuse.

I view the disablement as being a safety feature that keeps my site online.  More than once it has also made me go look at the source of the feed, only to discover they have given it a new address (e.g. changing domain name or feed supply script etc) - from those standpoints alone it is worth retaining the feed cancellation function.  I'm with SD on this, keep the disabling, it's useful if you "listen" to the fact it's trying to help you keep your feed list clean.

Gaz

[steve51184]

zx71 - I'd guess the problem with that, is that if you have a lot of feeds timing out, with a high frequency attempt to call them, then you could find yourself in a position of using excessive resources on a shared server, and your hosting account gets suspended by the hosting company - they would then insist that you disable or remove the script or mod causing the resource abuse.

i own the server so no problems there

I view the disablement as being a safety feature that keeps my site online.  More than once it has also made me go look at the source of the feed, only to discover they have given it a new address (e.g. changing domain name or feed supply script etc) - from those standpoints alone it is worth retaining the feed cancellation function.  I'm with SD on this, keep the disabling, it's useful if you "listen" to the fact it's trying to help you keep your feed list clean.

good point i guess but it means checking my feed list every few hours to constantly re-enable them :\

[GazOutEast]

If you own the server (i.e. the physical server, not a VPS) then why not configure an alerter to show a pop-up when the feed gets disabled - create a routine to capture a state change on the feed's database entry and either cause a popup or send an email to an admin address.  Then you don't need to physically check for the disabled feeds.

[steve51184]

If you own the server (i.e. the physical server, not a VPS) then why not configure an alerter to show a pop-up when the feed gets disabled - create a routine to capture a state change on the feed's database entry and either cause a popup or send an email to an admin address.  Then you don't need to physically check for the disabled feeds.

own as in a data centre in another country

[Spuds]

You can increase the timeout that simplepie uses with the set_timeout(seconds) construct, I think it defaults to something like 10 seconds so setting it to 30 seconds should be enough time to make a connection.  In ScheduledTasks.php find:

Code Select Expand

$rss_data = new RSS_Feeder();

add after

Code Select Expand

$rss_data->set_timeout(30); // 30 seconds

and if that helps with the curl 28 error .... 

[Chas Large]

I regret I cannot get this mod to work.

It installs OK. I can enter the data required but on the list of feeds, the Enabled column is blank, no matter whether the enabled checkbox is checked or not. I have it checked on two feeds (tested with simple portal RSS block so I know they can come into the forum) but using this mod, no posts appear in the boards specified.

Could it be that I have Simple portal installed?

[EDIT]
I discovered that you have to initially run the scheduled task to make the first batch of posts appear. Still nothing in the enabled column in the list of feeds. Should there be?

[EDIT 2]
All is well and feeds are being brought in manually via scheduled tasks or via the task itself. Thanks for a great mod.

[aw06]

Great mod, love it .. have a few errors thou, please advise how to fix

I'm using SMF 2.0 Rc3

Also, i want to pull full articles, can somebody explain the whole PCRE format thing, i don't quite get it 

[SlammedDime]

I regret I cannot get this mod to work.

It installs OK. I can enter the data required but on the list of feeds, the Enabled column is blank, no matter whether the enabled checkbox is checked or not. I have it checked on two feeds (tested with simple portal RSS block so I know they can come into the forum) but using this mod, no posts appear in the boards specified.

Could it be that I have Simple portal installed?

[EDIT]
I discovered that you have to initially run the scheduled task to make the first batch of posts appear. Still nothing in the enabled column in the list of feeds. Should there be?

[EDIT 2]
All is well and feeds are being brought in manually via scheduled tasks or via the task itself. Thanks for a great mod.

My guess is that you are using a custom theme other than default and didn't select the option when the mod installed to move the images installed with the mod to your custom theme directory... that's all you need to fix the issue with nothing in the enabled column.

[SlammedDime]

Great mod, love it .. have a few errors thou, please advise how to fix

I'm using SMF 2.0 Rc3

Also, i want to pull full articles, can somebody explain the whole PCRE format thing, i don't quite get it 

There was a post a page or two back that had the fix in it (can't check right now, limited time)... basically boils down to a missing global variable... as for the Regex thing... all I can really suggest is to follow my tutorial linked in the first post.

[Chas Large]

I regret I cannot get this mod to work.

It installs OK. I can enter the data required but on the list of feeds, the Enabled column is blank, no matter whether the enabled checkbox is checked or not. I have it checked on two feeds (tested with simple portal RSS block so I know they can come into the forum) but using this mod, no posts appear in the boards specified.

Could it be that I have Simple portal installed?

[EDIT]
I discovered that you have to initially run the scheduled task to make the first batch of posts appear. Still nothing in the enabled column in the list of feeds. Should there be?

[EDIT 2]
All is well and feeds are being brought in manually via scheduled tasks or via the task itself. Thanks for a great mod.

My guess is that you are using a custom theme other than default and didn't select the option when the mod installed to move the images installed with the mod to your custom theme directory... that's all you need to fix the issue with nothing in the enabled column.

Yup, that was the answer. Thanks.

[aw06]

OK thanks - have done it

The topper now looks like this -

Code Select Expand



2127: /* RSS Feeder Settings */
function ModifyRSSFeedSettings()
{
global $context, $txt, $sourcedir, $scripturl, $smcFunc, $board;

loadTemplate('RSSFeeds');
$context['page_title'] = $txt['rss_feeder'];



Is that the location you meant?

Gaz

OK, i made the change listed above, will watch and track if error is gone...

[aw06]

Still getting the errors, only thing it move from line 2135 to line 2138 ... any more fix ?