Topic: RSS Feeder

[mrtrc266]

Just making sure this topic didn't get locked....Bumping after 3 days 

I know that the code I tried above is totally wrong but any ideas anyone?

Thanks again

[SlammedDime]

Sorry, didn't see your post before.  It can be *anything*, not just <div something... whatever works to get what you need.  I can't check the site right now to verify the regex (work restrictions), but I'll take a look when I get home from work.  Does it not work for you?

[Killer-B]

This works great SD, thank you very much.

I'm having a problem with "Full Article" though. I would like to get the full articales from http://www.railbirds.com/rss/latestblogs.php

And I can't find the proper <div id... The div right before the content of the article is "<div class="blog-body">"

Here is  a link to a blog if anyone can look to see how I can get this working
http://www.railbirds.com/blog/255412/found-a-way-to-win-more-dollars-in-every-tourney.html

I tried http://www.railbirds.com/rss/latestblogs.php for the Feed URL ( This works good for just getting the feed)

and

~<div class="blog-body">(.*)<\/div>~siU for the Regular Expression. Does it have to be <div id... ?

Sorry for my ignorance in this stuff.

Thank you in advance.

mrtrc... I've kinda asked about this before... with no luck - but where are you getting these "codes" to pull the full articles from? As I'm trying to do the same.. but have no idea about PRC coding... I've asked for links or manuals or anywhere where I can study up and learn... but to no avail - the RSS feeder works GREAT.. but am only getting about 300 characters... and am wanting more...

TIA for any guidance -

Cheers-

KB

[SlammedDime]

Killer - You never did answer me to if you checked out the small tutorial that I posted in this topic.  There really isnt' any more to it, and unfortunately, I don't think I can really explain it any simpler.  There is plenty of information on the net regarding regular expressions... http://www.regular-expressions.info/ is a popular site with lots of info.

[Killer-B]

SD - Sorry, I know you field tons of questions with this, and all your other mods - but honestly, I did go thru all the posts here - and never saw anything of value (sorry if I overlooked something) - I did see your response, where you doubted it would/could pull the photos (as it has been doing very well) - I'm even using it on flickr now and pulling in great feeds -

That said, thanks for the link here... I'll take a look and see if I can get my head around it... and maybe it'll work afterall

Cheers-

KB

[Xavi-Nena]

any plans on this working with 1.1.8?

[SlammedDime]

Sorry Nena, I won't be backporting this, it relys on too many of SMF 2.0's built in functions, and I would have to rewrite a lot of the mod to work.

[Xavi-Nena]

ehh thanks anyway maybe once 2.0 is more worked in I will upgrade. i do look forward to trying out your mod.

[mrtrc266]

Sorry, didn't see your post before.  It can be *anything*, not just <div something... whatever works to get what you need.  I can't check the site right now to verify the regex (work restrictions), but I'll take a look when I get home from work.  Does it not work for you?

It does work but I only get a few characters, I'm trying to get full articles to pull from this feed.

http://www.railbirds.com/rss/latestblogs.php

I did read the tutorial and tried everything I can think of but I can't get it going. I am truly clueless and any help is greatly appreciated.

[SlammedDime]

Well I did some preliminary testing with the regex you're using and that feed and it seems to work fine... I'll throw it into the mod itself and see how it does...

[mrtrc266]

Here is a link to my test site that I'm trying it out on.

http://monsters-mansion.com/testrc1/index.php

As you can see in the Latest Railbird Blogs it pulls the feed but only shows the title of the blog. Full articles would be awesome

[Judgey]

Hey could u look at this plz all

http://www.itsyoursite.info/index.php?action=admin;area=modsettings;sa=rssfeeds

It just gives me text, could anyone help me with this

Thank you

[Xavi-Nena]

judgey - this link is requiring us to log in?

[Judgey]

Sorry my bad -

Im just getting a screen with loads of text, this is just some of it

Code: [Select]

/* RSS Feeder Settings */ function ModifyRSSFeedSettings() { global $context, $txt, $sourcedir, $scripturl, $smcFunc; loadTemplate('RSSFeeds'); $context['page_title'] = $txt['rss_feeder']; if (!empty($_REQUEST['feed']) && (isset($_REQUEST['enable']) || isset($_REQUEST['disable']))) { // Quick change on the status... $smcFunc['db_query']('', ' UPDATE {db_prefix}rssfeeds SET enabled = {int:option} WHERE id_feed = {int:feed}', array( 'option' => isset($_REQUEST['enable']) ? 1 : 0, 'feed' => (int) $_REQUEST['feed'], ) ); redirectExit('action=admin;area=modsettings;sa=rssfeeds'); } if (!empty($_REQUEST['feed']) || !empty($_REQUEST['add'])) { $context['sub_template'] = 'rss_feeder_add'; // Load the boards and categories for adding or editing a feed. $request = $smcFunc['db_query']('', ' SELECT b.id_board, b.name, b.child_level, c.name AS cat_name, c.id_cat FROM {db_prefix}boards AS b LEFT JOIN {db_prefix}categories AS c ON (c.id_cat = b.id_cat)', array() ); $context['categories'] = array(); while ($row = $smcFunc['db_fetch_assoc']($request)) { if (!isset($context['categories'][$row['id_cat']])) $context['categories'][$row['id_cat']] = array ( 'name' => strip_tags($row['cat_name']), 'boards' => array(), ); $context['categories'][$row['id_cat']]['boards'][] = array( 'id' => $row['id_board'], 'name' => strip_tags($row['name']), 'category' => strip_tags($row['cat_name']), 'child_level' => $row['child_level'], 'selected' => !empty($_SESSION['move_to_topic']) && $_SESSION['move_to_topic'] == $row['id_board'] && $row['id_board'] != $board, ); } $smcFunc['db_free_result']($request); if (empty($context['categories'])) fatal_lang_error('rss_feed_no_boards', false); // If we're just adding a feed, we can return, don't need to do anything further if (empty($_REQUEST['add'])) { // Lets get the feed from the database $request = $smcFunc['db_query']('', ' SELECT * FROM {db_prefix}rssfeeds WHERE id_feed = {int:feed} LIMIT 1', array( 'feed' => ((int) $_REQUEST['feed']), ) ); // No Feed?? ut oh... hacker!! if ($smcFunc['db_num_rows']($request) != 1) fatal_lang_error('rss_feed_not_found', false); $context['rss_feed'] = $smcFunc['db_fetch_assoc']($request); $context['rss_feed'] = htmlspecialchars__recursive($context['rss_feed']); $smcFunc['db_free_result']($request); } $context['icon'] = !empty($context['rss_feed']['icon']) ? $context['rss_feed']['icon'] : 'xx'; require_once($sourcedir . '/Subs-Editor.php'); // Message icons - customized icons are off? $context['icons'] = getMessageIcons(!empty($context['rss_feed']['id_board']) ? $context['rss_feed']['id_board'] : 0); if (!empty($context['icons'])) $context['icons'][count($context['icons']) - 1]['is_last'] = true; $context['icon_url'] = ''; for ($i = 0, $n = count($context['icons']); $i < $n; $i++) { $context['icons'][$i]['selected'] = $context['icon'] == $context['icons'][$i]['value']; if ($context['icons'][$i]['selected']) $context['icon_url'] = $context['icons'][$i]['url']; } if (empty($context['icon_url'])) { $context['icon_url'] = $settings[file_exists($settings['theme_dir'] . '/images/post/' . $context['icon'] . '.gif') ? 'images_url' : 'default_images_url'] . '/post/' . $context['icon'] . '.gif'; array_unshift($context['icons'], array( 'value' => $context['icon'], 'name' => $txt['current_icon'], 'url' => $context['icon_url'], 'is_last' => empty($context['icons']), 'selected' => true, )); } return; } // What about deleting one or more?? elseif (!empty($_REQUEST['delete'])) { //$_POST['checked_feeds'] // We already asked that they wanted to delete, so we'll just delete... no confirmation foreach

[Xavi-Nena]

do you have a screen shot?

[Judgey]

Here you, and thanks for helping

[Xavi-Nena]

i recently had a similar problem and it ended up being my php tags were not closed...im sure if you attach the file in question someone with more experience would be able to help you out. But that looks almost exactly like an issue I was having and it was I was missing some php tags.

[Judgey]

Here is the whole page could not work out wot page to upload, again thank you

Code: [Select]

/* RSS Feeder Settings */
function ModifyRSSFeedSettings()
{
global $context, $txt, $sourcedir, $scripturl, $smcFunc;

loadTemplate('RSSFeeds');
$context['page_title'] = $txt['rss_feeder'];

if (!empty($_REQUEST['feed']) && (isset($_REQUEST['enable']) || isset($_REQUEST['disable'])))
{
// Quick change on the status...
$smcFunc['db_query']('', '
UPDATE {db_prefix}rssfeeds
SET enabled = {int:option}
WHERE id_feed = {int:feed}',
array(
'option' => isset($_REQUEST['enable']) ? 1 : 0,
'feed' => (int) $_REQUEST['feed'],
)
);
redirectExit('action=admin;area=modsettings;sa=rssfeeds');
}

if (!empty($_REQUEST['feed']) || !empty($_REQUEST['add']))
{

$context['sub_template'] = 'rss_feeder_add';

// Load the boards and categories for adding or editing a feed.
$request = $smcFunc['db_query']('', '
SELECT b.id_board, b.name, b.child_level, c.name AS cat_name, c.id_cat
FROM {db_prefix}boards AS b
LEFT JOIN {db_prefix}categories AS c ON (c.id_cat = b.id_cat)',
array()
);
$context['categories'] = array();
while ($row = $smcFunc['db_fetch_assoc']($request))
{
if (!isset($context['categories'][$row['id_cat']]))
$context['categories'][$row['id_cat']] = array (
'name' => strip_tags($row['cat_name']),
'boards' => array(),
);

$context['categories'][$row['id_cat']]['boards'][] = array(
'id' => $row['id_board'],
'name' => strip_tags($row['name']),
'category' => strip_tags($row['cat_name']),
'child_level' => $row['child_level'],
'selected' => !empty($_SESSION['move_to_topic']) && $_SESSION['move_to_topic'] == $row['id_board'] && $row['id_board'] != $board,
);
}
$smcFunc['db_free_result']($request);

if (empty($context['categories']))
fatal_lang_error('rss_feed_no_boards', false);

// If we're just adding a feed, we can return, don't need to do anything further
if (empty($_REQUEST['add']))
{
// Lets get the feed from the database
$request = $smcFunc['db_query']('', '
SELECT *
FROM {db_prefix}rssfeeds
WHERE id_feed = {int:feed}
LIMIT 1',
array(
'feed' => ((int) $_REQUEST['feed']),
)
);

// No Feed?? ut oh... hacker!!
if ($smcFunc['db_num_rows']($request) != 1)
fatal_lang_error('rss_feed_not_found', false);
$context['rss_feed'] = $smcFunc['db_fetch_assoc']($request);
$context['rss_feed'] = htmlspecialchars__recursive($context['rss_feed']);
$smcFunc['db_free_result']($request);
}

$context['icon'] = !empty($context['rss_feed']['icon']) ? $context['rss_feed']['icon'] : 'xx';

require_once($sourcedir . '/Subs-Editor.php');
// Message icons - customized icons are off?
$context['icons'] = getMessageIcons(!empty($context['rss_feed']['id_board']) ? $context['rss_feed']['id_board'] : 0);

if (!empty($context['icons']))
$context['icons'][count($context['icons']) - 1]['is_last'] = true;

$context['icon_url'] = '';
for ($i = 0, $n = count($context['icons']); $i < $n; $i++)
{
$context['icons'][$i]['selected'] = $context['icon'] == $context['icons'][$i]['value'];
if ($context['icons'][$i]['selected'])
$context['icon_url'] = $context['icons'][$i]['url'];
}
if (empty($context['icon_url']))
{
$context['icon_url'] = $settings[file_exists($settings['theme_dir'] . '/images/post/' . $context['icon'] . '.gif') ? 'images_url' : 'default_images_url'] . '/post/' . $context['icon'] . '.gif';
array_unshift($context['icons'], array(
'value' => $context['icon'],
'name' => $txt['current_icon'],
'url' => $context['icon_url'],
'is_last' => empty($context['icons']),
'selected' => true,
));
}


return;
}
// What about deleting one or more??
elseif (!empty($_REQUEST['delete']))
{
//$_POST['checked_feeds']
// We already asked that they wanted to delete, so we'll just delete... no confirmation
foreach ($_POST['checked_feeds'] as $key => $id)
$_POST['checked_feeds'][$key] = (int) $id;

$smcFunc['db_query']('', '
DELETE FROM {db_prefix}rssfeeds
WHERE id_feed IN ({array_int:feed_list})',
array(
'feed_list' => $_POST['checked_feeds'],
)
);
}
// Or are they saving one being modified??
elseif (!empty($_REQUEST['save']) && !empty($_POST['sc']))
{
// First we check the session...
checkSession();

// Put the insert array together...
$insertOptions = array();

// Let's do the 'unrequireds' first...
$insertOptions['id_board'] = (int)$_POST['feed_board'];
$insertOptions['icon'] = isset($_POST['icon']) ? preg_replace('~[\./\\\\*\':"<>]~', '', $_POST['icon']) : 'xx';
$insertOptions['enabled'] = isset($_POST['feed_enabled']) ? 1 : 0;
$insertOptions['keywords'] = trim($_POST['feed_keywords']) != '' ? trim($_POST['feed_keywords']) : '';
$insertOptions['locked'] = isset($_POST['feed_locked']) ? 1 : 0;
$insertOptions['getfull'] = isset($_POST['feed_full']) ? 1 : 0;
$insertOptions['approve'] = isset($_POST['feed_approve']) ? 1 : 0;
$insertOptions['singletopic'] = isset($_POST['feed_singletopic']) ? 1 : 0;
$insertOptions['topicprefix'] = trim($_POST['feed_prefix']) != '' ? trim($_POST['feed_prefix']) : '';
$insertOptions['footer'] = !empty($_POST['feed_footer']) ? trim($_POST['feed_footer']) : '';
$insertOptions['numbertoimport'] = trim($_POST['feed_import']) != '' ? (int) trim($_POST['feed_import']) : 0;

$context['feed_errors'] = array();
// And now the requireds that we can throw errors on...
if (empty($_POST['feed_title']))
$context['feed_errors']['feed_title'] = ($insertOptions['title'] = '');
else
$insertOptions['title'] = preg_replace('~[&]([^;]{8}|[^;]{0,8}$)~', '&amp;$1', $_POST['feed_title']);

if (empty($_POST['feed_url']))
$context['feed_errors']['feed_url'] = ($insertOptions['feedurl'] = '');
else
$insertOptions['feedurl'] = trim($_POST['feed_url']);

if (empty($_POST['feed_poster']))
$context['feed_errors']['feed_poster'] = ($insertOptions['postername'] = '');
// Do a query to get the member's id
else
{
$request = $smcFunc['db_query']('', '
SELECT id_member
FROM {db_prefix}members
WHERE real_name = {string:name}
OR member_name = {string:name}
LIMIT 1',
array(
'name' => trim($_POST['feed_poster']),
)
);
if ($smcFunc['db_num_rows']($request) != 1)
$context['feed_errors']['feed_poster'] = ($insertOptions['postername'] = trim($_POST['feed_poster']));
else
{
$insertOptions['postername'] = trim($_POST['feed_poster']);
list($insertOptions['id_member']) = $smcFunc['db_fetch_row']($request);
}
$smcFunc['db_free_result']($request);
}

$insertOptions['regex'] = trim($_POST['feed_regex']);
if (!empty($insertOptions['getfull']) && empty($insertOptions['regex']))
$context['feed_errors']['feed_regex'] = '';

// if we had any errors, lets kick back a screen and highlight them...
if (!empty($context['feed_errors']))
{
$context['rss_feed'] = $insertOptions;
if (!empty($_POST['feed_id']))
$_REQUEST['feed'] = $_POST['feed_id'];
$_REQUEST['add'] = true;
return ModifyRSSFeedSettings();
}

// Looks like we're good.
// Modifying an existing feed?
if (!empty($_REQUEST['feed_id']))
{
$smcFunc['db_query']('','
UPDATE {db_prefix}rssfeeds
SET
id_board = {int:id_board},
feedurl = {string:feedurl},
title = {string:title},
icon = {string:icon},
enabled = {int:enabled},
postername = {string:postername},
id_member = {int:id_member},
keywords = {string:keywords},
regex = {string:regex},
locked = {int:locked},
getfull = {int:getfull},
approve = {int:approve},
singletopic = {int:singletopic},
topicprefix = {string:topicprefix},
footer = {string:footer},
numbertoimport = {int:numbertoimport}
WHERE id_feed = {int:id_feed}',
array_merge(array('id_feed' => (int)$_REQUEST['feed_id']), $insertOptions)
);
$context['feed_insert_success'] = $txt['rss_feed_update_success'];
}
// Or I guess we're inserting a new one
else
{
// Fix up the stuff for insertion, make sure the arrays are aligned
$insertRows = array( 'singletopic' => 'int', 'icon' => 'string', 'footer' => 'string', 'getfull' => 'int', 'id_board' => 'int', 'feedurl' => 'string', 'title' => 'string', 'enabled' => 'int', 'postername' => 'string', 'id_member' => 'int', 'keywords' => 'string', 'regex' => 'string', 'locked' => 'int', 'approve' => 'int', 'topicprefix' => 'string', 'numbertoimport' => 'int' );
ksort($insertRows);
ksort($insertOptions);

$smcFunc['db_insert']('',
'{db_prefix}rssfeeds',
$insertRows,
$insertOptions,
array('id_feed')
);
$id_feed = $smcFunc['db_insert_id']('{db_prefix}rssfeeds', 'id_feed');
if (empty($id_feed))
$context['feed_insert_error'] = $txt['rss_feed_insert_error'];
else
$context['feed_insert_success'] = $txt['rss_feed_insert_success'];
}
}

// If they deleted or saved, let's show the main list
$context['sub_template'] = 'rss_feeder_list';
// Create the table that will display the feeds
$listOptions = array(
'id' => 'rss_feeder_list',
'items_per_page' => 40,
'default_sort_col' => 'icon',
'base_href' => $scripturl . '?action=admin;area=modsettings;sa=rssfeeds',
'no_items_label' => $txt['rss_feed_none'],
'get_items' => array(
'function' => 'list_getFeeds',
),
'get_count' => array(
'function' => 'list_getNumFeeds',
),
'columns' => array(
'icon' => array(
'header' => array(
'value' => $txt['rss_feed_enabled'],
),
'data' => array(
'function' => create_function('$rowData', '
global $settings, $scripturl, $smcFunc, $txt;

if (empty($rowData[\'name\']) && $rowData[\'enabled\'])
{
$smcFunc[\'db_query\'](\'\', \'
UPDATE {db_prefix}rssfeeds
SET enabled = 0
WHERE id_feed = {int:feed}\',
array(
\'feed\' => $rowData[\'id_feed\'],
)
);

// Log an error about the issue, just so the user can see why their feed was disabled...
log_error($txt[\'rss_feeder\'] . \': \' . $rowData[\'title\'] . \' (\' . $txt[\'rss_feed_board_error\'] . \')\');

$rowData[\'enabled\'] = 0;
}
return \'<a href="\' . $scripturl . \'?action=admin;area=modsettings;sa=rssfeeds;feed=\' . $rowData[\'id_feed\'] . ($rowData[\'enabled\'] ? \';disable\' : \';enable\') . \'"><img src="\' . $settings[\'images_url\'] . ($rowData[\'enabled\'] ? \'/rss_enabled.gif\' : \'/rss_disabled.gif\') . \'"></a>\';'
),
'style' => 'text-align: center; width: 130px;',
),
'sort' => array(
'default' => 'f.enabled',
'reverse' => 'f.enabled DESC',
),
),
'title' => array(
'header' => array(
'value' => $txt['rss_feed_title'],
),
'data' => array(
'db' => 'title',
'style' => 'text-align: center;',
),
'sort' => array(
'default' => 'f.title',
'reverse' => 'f.title DESC',
),
),
'url' => array(
'header' => array(
'value' => $txt['rss_feed_url'],
),
'data' => array(
'sprintf' => array(
'format' => '<a target="_blank" href="%1$s">%1$s</a>',
'params' => array(
'feedurl' => true,
),
),
),
'sort' => array(
'default' => 'f.feedurl',
'reverse' => 'f.feedurl DESC',
),
),
'board' => array(
'header' => array(
'value' => $txt['rss_feed_board'],
),
'data' => array(
'function' => create_function('$rowData', '
global $txt;
return empty($rowData[\'name\']) ? \'<em><< \' . $txt[\'rss_feed_board_error\'] . \' >></em>\' : $rowData[\'name\'];'
),
'style' => 'text-align: center;',
),
'sort' => array(
'default' => 'b.name',
'reverse' => 'b.name DESC',
),
),
'count' => array(
'header' => array(
'value' => $txt['rss_feed_count'],
),
'data' => array(
'db' => 'importcount',
'style' => 'text-align: center;',
),
'sort' => array(
'default' => 'f.importcount',
'reverse' => 'f.importcount DESC',
),
),
'modify' => array(
'header' => array(
'value' => $txt['rss_feed_modify'],
),
'data' => array(
'sprintf' => array(
'format' => '<a href="' . $scripturl . '?action=admin;area=modsettings;sa=rssfeeds;feed=%1$d">' . $txt['rss_feed_modify'] . '</a>',
'params' => array(
'id_feed' => false,
),
),
'style' => 'text-align: center; width: 50px;',
),
),
'check' => array(
'header' => array(
'value' => '<input type="checkbox" onclick="invertAll(this, this.form);" class="check" />',
),
'data' => array(
'sprintf' => array(
'format' => '<input type="checkbox" name="checked_feeds[]" value="%1$d" class="check" />',
'params' => array(
'id_feed' => false,
),
),
'style' => 'text-align: center; width: 32px;',
),
),
),
'form' => array(
'href' => $scripturl . '?action=admin;area=modsettings;sa=rssfeeds',
'name' => 'rssfeedForm',
),
'additional_rows' => array(
array(
'position' => 'above_column_headers',
'value' => '
<input type="submit" name="add" value="'. $txt['rss_feed_add'] . '" />',
'style' => 'text-align: right;',
'class' => 'titlebg',
),
array(
'position' => 'below_table_data',
'value' => '
<input type="submit" onclick="return confirm(\'' . $txt['rss_feed_confirm'] . '\')" name="delete" value="'. $txt['rss_feed_delete'] . '" />',
'style' => 'text-align: right;',
'class' => 'titlebg',
),
),
);


require_once($sourcedir . '/Subs-List.php');
createList($listOptions);
}

// Get information for the list above... grabs all of the feeds from the database
function list_getFeeds($start, $items_per_page, $sort)
{
global $smcFunc;
$request = $smcFunc['db_query']('', '
SELECT f.id_feed, b.name, f.title, f.feedurl, f.enabled, f.importcount
FROM {db_prefix}rssfeeds AS f
LEFT JOIN {db_prefix}boards AS b ON (b.id_board = f.id_board)
ORDER BY {raw:sort}
LIMIT ' . $start . ', ' . $items_per_page,
array (
'sort' => $sort,
)
);
$feeds = array();
while ($row = $smcFunc['db_fetch_assoc']($request))
$feeds[] = $row;
$smcFunc['db_free_result']($request);

return $feeds;
}

// The number grabbed above could be limited at 40... this just gets a count of all of them.
function list_getNumFeeds()
{
global $smcFunc;

$request = $smcFunc['db_query']('', '
SELECT COUNT(*)
FROM {db_prefix}rssfeeds',
array(
)
);
list($numFeeds) = $smcFunc['db_fetch_row'];
$smcFunc['db_free_result']($request);

return $numFeeds;
}

<br />
<b>Fatal error</b>:  Call to undefined function ModifyRSSFeedSettings() in <b>/var/www/vhosts/judgey/itsyoursite.info/httpdocs/Sources/ManageSettings.php</b> on line <b>226</b><br />


[SlammedDime]

Open up the ManageSettings.php and move ?> to the very end of the file.

[SlammedDime]

mrtrc - I ran your feed through with the Regex enabled with the regex you provided above, and it grabbed the full article without a problem.