Topic: RSS Feeder

[GazOutEast]

Excellent mod SD and many thanks

Although I never had an issue with vb's for 1.1.x it did stop permitting direct edit of entries sometime around the forum upgrade to 1.1.12 so I just used to edit them in the database every three months, but the actual feeds behaved perfectly - on time and into the correct board etc.  So I was very happy to find yours that allows me control through the ACP again.

Unfortunately, yours has got me pulling my hair a bit

I'm using RSS Feeder 1.1.1 on SMF 2.0 RC1 and everything except one thing is working normally.  Feeds are scheduled for 3-hourly some to retrieve 1 item, some to retrieve up to 4 (they're announcement feeds normally updated 2 or 3 times a day with single posts).

The problem is they they are relentlessly marching backwards in time - after just 24 hours, one of the fed boards had over 500 posts going back to the Indian Ocean tsunami on Boxing Day 2004 and beyond.

I need them to only pull new posts from within the last 3 hours since their last check.

Any idea what could be causing this?

Gaz

[SlammedDime]

The feeder pulls from the bottom of the RSS feed to the top, so they will be posted in the order that the feed has them, oldest posted first.  If the feed has 100 items, and you're doing 1 at a time, it will start at the bottom and work it's way to the top.

[SlammedDime]

Hi,
can i have the SQL create table scrip for RSS Feeder? i'm having problem installing this mod.

Thanks.

Just put add_settings.php into your main SMF directory and run it.

i searched my bookmark for the answer i couldnt find it  now... ... where is the theme. go to forum and browse it. there is the answer or just ask a topic. is very simple to do it. I forgot how 


thank you SlammedDime. hope to do it someday (make clickable)

How to remove - Source: ... ?

You'll have to edit that in ScheduledTasks.php

[GazOutEast]

The feeder pulls from the bottom of the RSS feed to the top, so they will be posted in the order that the feed has them, oldest posted first.  If the feed has 100 items, and you're doing 1 at a time, it will start at the bottom and work it's way to the top.

OK - so would you recommend I do a single manual run of unlimited posts to clear the backlog and "bring it up to date"?  After which it should only pull new posts even if I've deleted all the old ones because they've already been captured and posted previously.

(appreciate the rapid help by the way - such a refreshing change)

Gaz

[SlammedDime]

If you are not retrieving the full article, then yes, I would recommend that.

[GazOutEast]

OK - have given it a go - now waiting for the feed creators to add something new and see if it performs the way I'm hoping it will.  I'll post back with results.

just a thought - if this does do what we're discussing, it could be a nice idea to add a little explanation of this particular bit of "how to" to the pop-up help for when selecting number of posts to pull - it might save less experienced users some heartache.  What do you think?

Gaz

[Hiver101]

Hello Everybody!
I'm having problem installing RSS Feed. The problem is, the tables were not created when i install it. I hope you guys can give me a create table sql script for RSS Feed.

Many thanks,

[SlammedDime]

Gaz - good idear

resti - As I said above, you need to put add_settings.php from the install package into your main SMF directory and run it.  There is more to it than just adding database tables, it also adds settings and values to the database.

[Hiver101]

Hi,
can i have the SQL create table scrip for RSS Feeder? i'm having problem installing this mod.

Thanks.

Just put add_settings.php into your main SMF directory and run it.

Hi i got this error when i executed the add_settings.php

Code: [Select]

/* RSS Feeder Settings */ function ModifyRSSFeedSettings() { global $context, $txt, $sourcedir, $scripturl, $smcFunc; loadTemplate('RSSFeeds'); $context['page_title'] = $txt['rss_feeder']; if (!empty($_REQUEST['feed']) && (isset($_REQUEST['enable']) || isset($_REQUEST['disable']))) { // Quick change on the status... $smcFunc['db_query']('', ' UPDATE {db_prefix}rssfeeds SET enabled = {int:option} WHERE id_feed = {int:feed}', array( 'option' => isset($_REQUEST['enable']) ? 1 : 0, 'feed' => (int) $_REQUEST['feed'], ) ); redirectExit('action=admin;area=modsettings;sa=rssfeeds'); } if (!empty($_REQUEST['feed']) || !empty($_REQUEST['add'])) { $context['sub_template'] = 'rss_feeder_add'; // Load the boards and categories for adding or editing a feed. $request = $smcFunc['db_query']('', ' SELECT b.id_board, b.name, b.child_level, c.name AS cat_name, c.id_cat FROM {db_prefix}boards AS b LEFT JOIN {db_prefix}categories AS c ON (c.id_cat = b.id_cat)', array() ); $context['categories'] = array(); while ($row = $smcFunc['db_fetch_assoc']($request)) { if (!isset($context['categories'][$row['id_cat']])) $context['categories'][$row['id_cat']] = array ( 'name' => strip_tags($row['cat_name']), 'boards' => array(), ); $context['categories'][$row['id_cat']]['boards'][] = array( 'id' => $row['id_board'], 'name' => strip_tags($row['name']), 'category' => strip_tags($row['cat_name']), 'child_level' => $row['child_level'], 'selected' => !empty($_SESSION['move_to_topic']) && $_SESSION['move_to_topic'] == $row['id_board'] && $row['id_board'] != $board, ); } $smcFunc['db_free_result']($request); if (empty($context['categories'])) fatal_lang_error('rss_feed_no_boards', false); // If we're just adding a feed, we can return, don't need to do anything further if (empty($_REQUEST['add'])) { // Lets get the feed from the database $request = $smcFunc['db_query']('', ' SELECT * FROM {db_prefix}rssfeeds WHERE id_feed = {int:feed} LIMIT 1', array( 'feed' => ((int) $_REQUEST['feed']), ) ); // No Feed?? ut oh... hacker!! if ($smcFunc['db_num_rows']($request) != 1) fatal_lang_error('rss_feed_not_found', false); $context['rss_feed'] = $smcFunc['db_fetch_assoc']($request); $context['rss_feed'] = htmlspecialchars__recursive($context['rss_feed']); $smcFunc['db_free_result']($request); } $context['icon'] = !empty($context['rss_feed']['icon']) ? $context['rss_feed']['icon'] : 'xx'; require_once($sourcedir . '/Subs-Editor.php'); // Message icons - customized icons are off? $context['icons'] = getMessageIcons(!empty($context['rss_feed']['id_board']) ? $context['rss_feed']['id_board'] : 0); if (!empty($context['icons'])) $context['icons'][count($context['icons']) - 1]['is_last'] = true; $context['icon_url'] = ''; for ($i = 0, $n = count($context['icons']); $i < $n; $i++) { $context['icons'][$i]['selected'] = $context['icon'] == $context['icons'][$i]['value']; if ($context['icons'][$i]['selected']) $context['icon_url'] = $context['icons'][$i]['url']; } if (empty($context['icon_url'])) { $context['icon_url'] = $settings[file_exists($settings['theme_dir'] . '/images/post/' . $context['icon'] . '.gif') ? 'images_url' : 'default_images_url'] . '/post/' . $context['icon'] . '.gif'; array_unshift($context['icons'], array( 'value' => $context['icon'], 'name' => $txt['current_icon'], 'url' => $context['icon_url'], 'is_last' => empty($context['icons']), 'selected' => true, )); } return; } // What about deleting one or more?? elseif (!empty($_REQUEST['delete'])) { //$_POST['checked_feeds'] // We already asked that they wanted to delete, so we'll just delete... no confirmation foreach ($_POST['checked_feeds'] as $key => $id) $_POST['checked_feeds'][$key] = (int) $id; $smcFunc['db_query']('', ' DELETE FROM {db_prefix}rssfeeds WHERE id_feed IN ({array_int:feed_list})', array( 'feed_list' => $_POST['checked_feeds'], ) ); } // Or are they saving one being modified?? elseif (!empty($_REQUEST['save']) && !empty($_POST['sc'])) { // First we check the session... checkSession(); // Put the insert array together... $insertOptions = array(); // Let's do the 'unrequireds' first... $insertOptions['id_board'] = (int)$_POST['feed_board']; $insertOptions['icon'] = isset($_POST['icon']) ? preg_replace('~[\./\\\\*\':"<>]~', '', $_POST['icon']) : 'xx'; $insertOptions['enabled'] = isset($_POST['feed_enabled']) ? 1 : 0; $insertOptions['keywords'] = trim($_POST['feed_keywords']) != '' ? trim($_POST['feed_keywords']) : ''; $insertOptions['locked'] = isset($_POST['feed_locked']) ? 1 : 0; $insertOptions['getfull'] = isset($_POST['feed_full']) ? 1 : 0; $insertOptions['approve'] = isset($_POST['feed_approve']) ? 1 : 0; $insertOptions['singletopic'] = isset($_POST['feed_singletopic']) ? 1 : 0; $insertOptions['topicprefix'] = trim($_POST['feed_prefix']) != '' ? trim($_POST['feed_prefix']) : ''; $insertOptions['footer'] = !empty($_POST['feed_footer']) ? trim($_POST['feed_footer']) : ''; $insertOptions['numbertoimport'] = trim($_POST['feed_import']) != '' ? (int) trim($_POST['feed_import']) : 0; $context['feed_errors'] = array(); // And now the requireds that we can throw errors on... if (empty($_POST['feed_title'])) $context['feed_errors']['feed_title'] = ($insertOptions['title'] = ''); else $insertOptions['title'] = preg_replace('~[&]([^;]{8}|[^;]{0,8}$)~', '&$1', $_POST['feed_title']); if (empty($_POST['feed_url'])) $context['feed_errors']['feed_url'] = ($insertOptions['feedurl'] = ''); else $insertOptions['feedurl'] = trim($_POST['feed_url']); if (empty($_POST['feed_poster'])) $context['feed_errors']['feed_poster'] = ($insertOptions['postername'] = ''); // Do a query to get the member's id else { $request = $smcFunc['db_query']('', ' SELECT id_member FROM {db_prefix}members WHERE real_name = {string:name} OR member_name = {string:name} LIMIT 1', array( 'name' => trim($_POST['feed_poster']), ) ); if ($smcFunc['db_num_rows']($request) != 1) $context['feed_errors']['feed_poster'] = ($insertOptions['postername'] = trim($_POST['feed_poster'])); else { $insertOptions['postername'] = trim($_POST['feed_poster']); list($insertOptions['id_member']) = $smcFunc['db_fetch_row']($request); } $smcFunc['db_free_result']($request); } $insertOptions['regex'] = trim($_POST['feed_regex']); if (!empty($insertOptions['getfull']) && empty($insertOptions['regex'])) $context['feed_errors']['feed_regex'] = ''; // if we had any errors, lets kick back a screen and highlight them... if (!empty($context['feed_errors'])) { $context['rss_feed'] = $insertOptions; if (!empty($_POST['feed_id'])) $_REQUEST['feed'] = $_POST['feed_id']; $_REQUEST['add'] = true; return ModifyRSSFeedSettings(); } // Looks like we're good. // Modifying an existing feed? if (!empty($_REQUEST['feed_id'])) { $smcFunc['db_query']('',' UPDATE {db_prefix}rssfeeds SET id_board = {int:id_board}, feedurl = {string:feedurl}, title = {string:title}, icon = {string:icon}, enabled = {int:enabled}, postername = {string:postername}, id_member = {int:id_member}, keywords = {string:keywords}, regex = {string:regex}, locked = {int:locked}, getfull = {int:getfull}, approve = {int:approve}, singletopic = {int:singletopic}, topicprefix = {string:topicprefix}, footer = {string:footer}, numbertoimport = {int:numbertoimport} WHERE id_feed = {int:id_feed}', array_merge(array('id_feed' => (int)$_REQUEST['feed_id']), $insertOptions) ); $context['feed_insert_success'] = $txt['rss_feed_update_success']; } // Or I guess we're inserting a new one else { // Fix up the stuff for insertion, make sure the arrays are aligned $insertRows = array( 'singletopic' => 'int', 'icon' => 'string', 'footer' => 'string', 'getfull' => 'int', 'id_board' => 'int', 'feedurl' => 'string', 'title' => 'string', 'enabled' => 'int', 'postername' => 'string', 'id_member' => 'int', 'keywords' => 'string', 'regex' => 'string', 'locked' => 'int', 'approve' => 'int', 'topicprefix' => 'string', 'numbertoimport' => 'int' ); ksort($insertRows); ksort($insertOptions); $smcFunc['db_insert']('', '{db_prefix}rssfeeds', $insertRows, $insertOptions, array('id_feed') ); $id_feed = $smcFunc['db_insert_id']('{db_prefix}rssfeeds', 'id_feed'); if (empty($id_feed)) $context['feed_insert_error'] = $txt['rss_feed_insert_error']; else $context['feed_insert_success'] = $txt['rss_feed_insert_success']; } } // If they deleted or saved, let's show the main list $context['sub_template'] = 'rss_feeder_list'; // Create the table that will display the feeds $listOptions = array( 'id' => 'rss_feeder_list', 'items_per_page' => 40, 'default_sort_col' => 'icon', 'base_href' => $scripturl . '?action=admin;area=modsettings;sa=rssfeeds', 'no_items_label' => $txt['rss_feed_none'], 'get_items' => array( 'function' => 'list_getFeeds', ), 'get_count' => array( 'function' => 'list_getNumFeeds', ), 'columns' => array( 'icon' => array( 'header' => array( 'value' => $txt['rss_feed_enabled'], ), 'data' => array( 'function' => create_function('$rowData', ' global $settings, $scripturl, $smcFunc, $txt; if (empty($rowData[\'name\']) && $rowData[\'enabled\']) { $smcFunc[\'db_query\'](\'\', \' UPDATE {db_prefix}rssfeeds SET enabled = 0 WHERE id_feed = {int:feed}\', array( \'feed\' => $rowData[\'id_feed\'], ) ); // Log an error about the issue, just so the user can see why their feed was disabled... log_error($txt[\'rss_feeder\'] . \': \' . $rowData[\'title\'] . \' (\' . $txt[\'rss_feed_board_error\'] . \')\'); $rowData[\'enabled\'] = 0; } return \'\';' ), 'style' => 'text-align: center; width: 130px;', ), 'sort' => array( 'default' => 'f.enabled', 'reverse' => 'f.enabled DESC', ), ), 'title' => array( 'header' => array( 'value' => $txt['rss_feed_title'], ), 'data' => array( 'db' => 'title', 'style' => 'text-align: center;', ), 'sort' => array( 'default' => 'f.title', 'reverse' => 'f.title DESC', ), ), 'url' => array( 'header' => array( 'value' => $txt['rss_feed_url'], ), 'data' => array( 'sprintf' => array( 'format' => '%1$s', 'params' => array( 'feedurl' => true, ), ), ), 'sort' => array( 'default' => 'f.feedurl', 'reverse' => 'f.feedurl DESC', ), ), 'board' => array( 'header' => array( 'value' => $txt['rss_feed_board'], ), 'data' => array( 'function' => create_function('$rowData', ' global $txt; return empty($rowData[\'name\']) ? \'<< \' . $txt[\'rss_feed_board_error\'] . \' >>\' : $rowData[\'name\'];' ), 'style' => 'text-align: center;', ), 'sort' => array( 'default' => 'b.name', 'reverse' => 'b.name DESC', ), ), 'count' => array( 'header' => array( 'value' => $txt['rss_feed_count'], ), 'data' => array( 'db' => 'importcount', 'style' => 'text-align: center;', ), 'sort' => array( 'default' => 'f.importcount', 'reverse' => 'f.importcount DESC', ), ), 'modify' => array( 'header' => array( 'value' => $txt['rss_feed_modify'], ), 'data' => array( 'sprintf' => array( 'format' => '' . $txt['rss_feed_modify'] . '', 'params' => array( 'id_feed' => false, ), ), 'style' => 'text-align: center; width: 50px;', ), ), 'check' => array( 'header' => array( 'value' => '', ), 'data' => array( 'sprintf' => array( 'format' => '', 'params' => array( 'id_feed' => false, ), ), 'style' => 'text-align: center; width: 32px;', ), ), ), 'form' => array( 'href' => $scripturl . '?action=admin;area=modsettings;sa=rssfeeds', 'name' => 'rssfeedForm', ), 'additional_rows' => array( array( 'position' => 'above_column_headers', 'value' => ' ', 'style' => 'text-align: right;', 'class' => 'titlebg', ), array( 'position' => 'below_table_data', 'value' => ' ', 'style' => 'text-align: right;', 'class' => 'titlebg', ), ), ); require_once($sourcedir . '/Subs-List.php'); createList($listOptions); } // Get information for the list above... grabs all of the feeds from the database function list_getFeeds($start, $items_per_page, $sort) { global $smcFunc; $request = $smcFunc['db_query']('', ' SELECT f.id_feed, b.name, f.title, f.feedurl, f.enabled, f.importcount FROM {db_prefix}rssfeeds AS f LEFT JOIN {db_prefix}boards AS b ON (b.id_board = f.id_board) ORDER BY {raw:sort} LIMIT ' . $start . ', ' . $items_per_page, array ( 'sort' => $sort, ) ); $feeds = array(); while ($row = $smcFunc['db_fetch_assoc']($request)) $feeds[] = $row; $smcFunc['db_free_result']($request); return $feeds; } // The number grabbed above could be limited at 40... this just gets a count of all of them. function list_getNumFeeds() { global $smcFunc; $request = $smcFunc['db_query']('', ' SELECT COUNT(*) FROM {db_prefix}rssfeeds', array( ) ); list($numFeeds) = $smcFunc['db_fetch_row']; $smcFunc['db_free_result']($request); return $numFeeds; }
Fatal error: Call to undefined function ModifyRSSFeedSettings() in D:\inetpub\vhosts\thegamershive.net\httpdocs\forum\Sources\ManageSettings.php on line 217
hope you can help me on this

[SlammedDime]

Looks like the mod did not install correctly to ManageSettings.php.

You'll need to open that file up and make sure that ?> is the VERY last line of the file.  It looks like it is probably above /* RSS Feeder Settings */

[mrtrc266]

Very nice MOD!

I had the same prob on the install, the >? was not at then end of the file. Easy fix though I'm sure :-)

[Killer-B]

If someone (anyone) has PCRE coding experience - please PM me... or leave you info as a post in this thread regarding how to get an entire RSS content to show in a feed -

Thank$

KB

[Hiver101]

Looks like the mod did not install correctly to ManageSettings.php.

You'll need to open that file up and make sure that ?> is the VERY last line of the file.  It looks like it is probably above /* RSS Feeder Settings */

Thanks and now i'm back to table problem and i got this error.

Table 's01win_mysqlb00fd8797d6746085396ce7e1314cece.******_rssfeeds' doesn't exist
File: D:\inetpub\vhosts\*********\httpdocs\forum\Sources\ManageSettings.php
Line: 2561

Please help me, it would be beter if you can give my the SQL string to create the table manualy... thanks

[Groundhog]

SlammedDime

Just had to say that this is such a good mod. Thanks so much for providing it and also for the very important info regarding how to use the "retrieve full article" feature.

What this mod does is turn a board that has members that don't post regularly into a board that now appears to be really busy. Hopefully this will attract more interest to our sites. Of coarse you know this already but had to drop by and say how much I appreciate your work. 

Regards

[Killer-B]

OK, new issue...

I have 43 RSS feeds (no worries, some are 24hour 1x weather updates) but, I need to "edit" 2 of them, but are not showing on my panel now?

I've just counted, and it displays 40 RSS Feeds, and lists "Page 1" at the bottom (making me assume, there should be a "Page 2" tab) but there is not? - Is there a reason for this I don't know about - or more importantly, how can I access #41, #42, #43 etc?

TIA -

KB

[Xavi-Nena]

anythign like this that is compatible with 1.1.8 ?

[SlammedDime]

Killer - I haven't had a chance to release a new version yet, but you can fix it by the following

rii - you'll have to modify the relevant code in ScheduledTasks.php, search for $subject =


MrSandman - What happens when you try to add another one?  Does it just not show up?  It should show multiple page selections at the top.
Edit, I found the problem, damn...  I'll release a new version in the next day or two to solve the issue.
If you want to fix it yourself for the time being (you'll have to reverse the change to upgrade or uninstall)
Open ManageSettings.php

Code: (Find) [Select]

list($numFeeds) = $smcFunc['db_fetch_row'];


Code: (Replace) [Select]

list($numFeeds) = $smcFunc['db_fetch_row']($request);


Nena - vbgamer has one for 1.1.x, but it is no longer supported, and I'm not sure how well it works.  Some people have reported problems with it.

[Killer-B]

Thanks SD - I couldn't find someone else who had the same problem... Will try the mod now...

Cheers

NB: That did it! Thanks!

[Xavi-Nena]

Nena - vbgamer has one for 1.1.x, but it is no longer supported, and I'm not sure how well it works.  Some people have reported problems with it.

Yeah the bugs w/out support is my worry...i would really like to be able to add this to my members ultimate profile so they can link their blogs to it to display their blog feed... I hope I get to use this mod soon...it sounds fantastic.

[mrtrc266]

This works great SD, thank you very much.

I'm having a problem with "Full Article" though. I would like to get the full articales from http://www.railbirds.com/rss/latestblogs.php

And I can't find the proper <div id... The div right before the content of the article is "<div class="blog-body">"

Here is  a link to a blog if anyone can look to see how I can get this working
http://www.railbirds.com/blog/255412/found-a-way-to-win-more-dollars-in-every-tourney.html

I tried http://www.railbirds.com/rss/latestblogs.php for the Feed URL ( This works good for just getting the feed)

and

~<div class="blog-body">(.*)<\/div>~siU for the Regular Expression. Does it have to be <div id... ?

Sorry for my ignorance in this stuff.

Thank you in advance.