Topic: RSS Feeder

[SlammedDime]

Yea, I'm still gonna have to ponder on the Mark as Read option.

As for images, yes, I've edited the stripping function to convert image tags to [ img] bbc tags

I actually have version 1.1.0 ready to roll out and I've tested the upgrade script to upgrade anyone already running 1.0.0, but I want to further investigate the mark as read option first.

[SlammedDime]

After digging through the SMF code, adding Mark As Read support just isn't feasable, at least not at this point in SMF time.

Think about it this way...

Let's say you have 10000 members and 10 different feeds, and when the scheduled tasks run, you end up importing 50 items, creating 50 new topics... that is an extra 500,000 rows that would need to be added to the log_topics table, 1 row per item per member.

Likewise, if I enabled 'by board' mark as read, if you have 10000 members, that's at least 10000 rows that would need to be updated, or created, assuming all feeds post to the same board.  If each feed posts in a different board, with 10 feeds you'll be updating or creating 100,000 rows in the table.

As you can see, it's just not feasable to do, and I don't think you'll be able to find any mod that can do it.

A suggestion to try would be the ignore board feature.  Allow your members to Ignore Boards, then have your feeds post to boards that your members can ignore, then the new topics and posts won't show up in their Unread Topics list.

As a side note, I've decided to switch from MagPie RSS to SimplePie RSS for the API.  Much more solid, and much more recent (1 month since last version, vs 2+ years with MagPie).  There will be no difference to anyone using the mod.

[bloodfire]

shanksta - before I release the next version, I'll take a hard look at how I can make it work.  The only way I think I can do it is to mark the whole board that the news feed is posted to as read, but then if you have one feed you don't want marked as read, but one feed you do, and they post to the same board, then you're SOL.

ScheduledTasks.php

Code: [Select]

1731 $matches = array();
1732 preg_match($feed['regex'], $browser->results, $matches);
1733 $body = $matches[1];


14-Jan-2009 20:00:37] PHP Warning:  fsockopen() [<a href='function.fsockopen'>function.fsockopen</a>]: unable to connect to www.theborneopost.com:80 [nofollow] (Connection refused) in /home/bnmart/public_html/Sources/Subs-Rss.php on line 2040

as for this, it stops there everytime, only to this url... this is from bignewsnetwork.com not from my website... however, there are times it works...

Thanks for posting that code, I've corrected the code so an error will no longer be thrown in the first case above.

For the fsockopen error - it simply means fsockopen wasn't able to connect to the specified server, normally because the server denied the request.  Nothing I can really do to work around it except suppress the error, which I've done for the next version.

Great... can't wait for the update version.

Yeah... looks like the server denied the request... nothing to do with the script nor the command... most probably the connection to the server...

[bloodfire]

Another thing I found... since using this mod.... I found core dump files in my server....

core.12437
core.24225
core.9285
core.6443

Does any1 getting these files? At first, I just delete them, ignoring it. Only just now, when I test this mod by 'run now' on the schedule tasks and abt 1 minute or so, the page shows 500 Internal Service page. Then I try to retrieve the error.log using ftp from my server to find wat other errors... my server seems to be timeout as well... then i reconnect and found the core file... however, no error was shown.

[SlammedDime]

If you're trying to use the 'full article option', try disabling it and then try again.  It is possible that you're using an expression that is causing a core dump... if so, you'll need to revise the expression your using

If you get the 500 error and are able to retrieve the log to see what it shows, please post it.  I'd be interested to see where it came from.

[Nao 尚]

If you're trying to use the 'full article option', try disabling it and then try again.  It is possible that you're using an expression that is causing a core dump... if so, you'll need to revise the expression your using

Ohhhh...... Fun memories! lol

[Drivin and Cryin]

I'm not sure if this is a stupid mistake on my part or a genuine error.  I just setup 2.0 beta and have been trying to get this to work, but after i install it and it tries to redirect, i get this error:

Fatal error: Cannot redeclare modifyrssfeedsettings() (previously declared in /home/my2008/public_html/mytestforum/forum/Sources/ManageSettings.php:1982) in /home/my2008/public_html/mytestforum/forum/Sources/ManageSettings.php on line 2700

Any ideas would be greatly appreciated.

[bloodfire]

If you're trying to use the 'full article option', try disabling it and then try again.  It is possible that you're using an expression that is causing a core dump... if so, you'll need to revise the expression your using

If you get the 500 error and are able to retrieve the log to see what it shows, please post it.  I'd be interested to see where it came from.

I've tried disabling the full article option...still get the 500 error and the dump core... however... if the rss feed is around 30 like moreover.com... it gets all the news but shows the 500 error page but no dump core file... as for 100 rss list like bignewsnetwork.com... gets both but only until certain amount of rss being retrieved...

the expression is ok... at times... it works fine... retrieving all those match them...

Still no error logs or any indications of wat causing the error...

[Smoky "Rider" Blue]

http://en.wikipedia.org/wiki/Core_dump

[SlammedDime]

If you're trying to use the 'full article option', try disabling it and then try again.  It is possible that you're using an expression that is causing a core dump... if so, you'll need to revise the expression your using

Ohhhh...... Fun memories! lol

LOL

I'm not sure if this is a stupid mistake on my part or a genuine error.  I just setup 2.0 beta and have been trying to get this to work, but after i install it and it tries to redirect, i get this error:

Fatal error: Cannot redeclare modifyrssfeedsettings() (previously declared in /home/my2008/public_html/mytestforum/forum/Sources/ManageSettings.php:1982) in /home/my2008/public_html/mytestforum/forum/Sources/ManageSettings.php on line 2700

Any ideas would be greatly appreciated.

That means the mod was installed twice... you'll need to go through and remove the duplicate code by hand.

[Drivin and Cryin]

That means the mod was installed twice... you'll need to go through and remove the duplicate code by hand.

Thanks, i got that sorted out and it's working now.

Maybe this will help someone else out, but i couldn't get it working with the clean blue theme.  For some reason, the setup screen just wouldn't show up.  So when i kept going to the configuration screen|modifications - rss feeder wasn't listed, which is probably why i ended up with it installed twice in the first place.  I switched to the default theme and everything worked fine.

[SlammedDime]

Can you link me to which theme you are using?  This mod doesn't make any theme or template changes except to add information to the modifications.english.php, which only the default theme should have in it's language folder.

[Lamarck]

You are working hard in this mod,very good job.I'm looking forward the new release.

[SlammedDime]

Mainly because it's a crucial part of my new site I'm launching... lol.

[KahneFan]

Mainly because it's a crucial part of my new site I'm launching... lol.

Still, thank you for sharing it with us and for the quick responses.

What's your new site? (or did I miss a link?)

[SlammedDime]

No worries....

www.ccwlaws.com

[KahneFan]

Ah, that's right. I remember seeing that board. Going live next Tuesday (1-20). Looks good.

[/off topic]

[SlammedDime]

Okay, if anyone would be willing, can you give 1.1.0 a shot... I won't post it on the download site quite yet, so for now it'll have to be through my SVN...

[link removed]

Click on 'Tarball' next to the 1.1.0 under Tags and save the file to your desktop.

I would not recommend this for anyone already using 1.0.0 until I have fixed any bugs that may be found in this version, UNLESS you are comfortable in working in the database and able to drop and restore tables without help.  In that case, then you can install the above version and it will upgrade your existing install and update the database tables.  PLEASE take a backup of the rssfeeds and log_rssfeeds tables BEFORE upgrading, incase you do need to revert back to 1.0.0 for any reason.

If you do need to downgrade, uninstall version 1.1.0, then in the 'Browse Mods' section of your Package Manager, delete version 1.0.0, ignoring the error that it may still be installed, yada yada... then install 1.0.0 again, then go into the database drop, then recreate the tables from your backup.

(if you didn't understand all of the above, then you probably shouldn't upgrade just yet)

[Lamarck]

I have the new version installed in my test board and it seems to work fine.

I found a "bug" or two: 

-Spanish characters appear as "?" in footer and prefix.

-Also,I get this:

An Error Has Occurred!
The database value you're trying to insert does not exist: hash

In the scheduled tasks tab if I press "run now" with RSS Feeder option enabled.Despite this error,a new topic from the rss feed is created with no problem.But mod get only one topic from the feed,even If I have set that option to 2 or 3.


I will be testing this more tomorrow,thanks.


P.S: It's a "fresh" installation,not an updated version from 1.0.0 to 1.1.0,if that helps.

[SlammedDime]

The reason for the ? might be because of an incorrect collation on the database table... I'm not sure I can change it with the install of the mod using SMF's functions, but I'll try.  the ? problem will probably go away if you go into mysql to the tables and change their collation to utf-8

As for the hash... I know why that's not working... stupid last minute change, thanks.

Edit: The hash problem is fixed in the download now...