Advertisement:

Author Topic: RSS Feeder  (Read 375291 times)

Offline mrtrc266

  • SMF Hero
  • ******
  • Posts: 2,164
  • Gender: Male
    • Monsters Mansion
Re: RSS Feeder
« Reply #480 on: May 26, 2009, 05:27:27 PM »
Ok I think I got it...is this correct? This will not make it disable the feed and it won't put any errors in the error log? One last question, is there any risk in doing this?
 
Code: [Select]

  /* If we don't get a valid chunk of data back, disable the feed
  if ($rss_data->error())
  {
   $smcFunc['db_query']('', '
    UPDATE {db_prefix}rssfeeds
    SET enabled = 0
    WHERE id_feed = {int:feed}',
    array(
     'feed' => $id,
    )
   );
   
   // Log an error about the issue, just so the user can see why their feed was disabled...
   log_error($txt['rss_feeder'] . ': ' . $feed['url'] . ' (' . $rss_data->error() . ')');
   continue;
  }*/

Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,276
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
Re: RSS Feeder
« Reply #481 on: May 26, 2009, 07:03:55 PM »
I'd only comment out the sql query and keep the error log entry in there, unless you really don't want any error logging of failed feeds, but at that point it will be quite hard to provide any support on the mod.

No risk in commenting that out.
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline mghq

  • Jr. Member
  • **
  • Posts: 194
    • Mega Gaming Hq
Re: RSS Feeder
« Reply #482 on: May 27, 2009, 09:48:16 PM »
Hey seeing as this mod is similar to the one for SMF 1.1.X could you tell me how you fixed the html problem because i cant figure it out

Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,276
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
Re: RSS Feeder
« Reply #483 on: May 27, 2009, 10:12:31 PM »
What HTML problem?
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline mrtrc266

  • SMF Hero
  • ******
  • Posts: 2,164
  • Gender: Male
    • Monsters Mansion
Re: RSS Feeder
« Reply #484 on: May 27, 2009, 10:15:29 PM »
Thank you very much Slammed, feeds have not been automatically disabled all day :D

Offline mghq

  • Jr. Member
  • **
  • Posts: 194
    • Mega Gaming Hq
Re: RSS Feeder
« Reply #485 on: May 27, 2009, 10:30:51 PM »
What HTML problem?

Well with the one from vbgamer, it will not add the  code tag even for an administrator. And if it does it always screws up these symbols <, >, "

i thought if i edit Subs-RSS.php it would fix it, but it messed the symbols up but put the html tags. So how did you go about fixing that

Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,276
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
Re: RSS Feeder
« Reply #486 on: May 27, 2009, 10:35:26 PM »
Here is the PHP function that I run everything through...

Code: [Select]
function striphtml($document)
{
// Just in case there aren't breaks after a </p>...
$document = str_replace(array('</p>', '<br />', '<br>',  '</div>'), array("</p>\r\n", "<br />\r\n", "<br>\r\n", "</div>\r\n"), $document);

$search = array('~<style[^>]*?>.*?</style>~siU',    // Strip style tags properly
"'<script[^>]*?>.*?</script>'si", // strip out javascript
"~<img.*?src=\"(.*?)\".*?>~si", // convert img tags
"'<[\/\!]*?[^<>]*?>'si", // strip out html tags
"'([\r\n])[\s]+'", // strip out white space
"'&(quot|#34|#034|#x22);'i", // replace html entities
"'&(amp|#38|#038|#x26);'i", // added hexadecimal values
"'&(lt|#60|#060|#x3c);'i",
"'&(gt|#62|#062|#x3e);'i",
"'&(nbsp|#160|#xa0);'i",
"'&(iexcl|#161);'i",
"'&(cent|#162);'i",
"'&(pound|#163);'i",
"'&(copy|#169);'i",
"'&(reg|#174);'i",
"'&(deg|#176);'i",
"'&(#39|#039|#x27);'",
"'&(euro|#8364);'i", // europe
"'&a(uml|UML);'", // german
"'&o(uml|UML);'",
"'&u(uml|UML);'",
"'&A(uml|UML);'",
"'&O(uml|UML);'",
"'&U(uml|UML);'",
"'&szlig;'i",
"~(\r\n|\n|\r)+~",
);
$replace = array( "",
"",
"",
"",
"\\1",
"\"",
"&",
"<",
">",
" ",
chr(161),
chr(162),
chr(163),
chr(169),
chr(174),
chr(176),
chr(39),
chr(128),
"ä",
"ö",
"ü",
"Ä",
"Ö",
"Ü",
"ß",
"\n\n",
);

$text = preg_replace($search,$replace,$document);

return $text;
}
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline mghq

  • Jr. Member
  • **
  • Posts: 194
    • Mega Gaming Hq
Re: RSS Feeder
« Reply #487 on: May 27, 2009, 10:39:44 PM »
Do you know how we could fix that for this one
Code: [Select]
<?php
/*
RSS Feed Poster
Version 1.0.4
by:vbgamer45
http://www.smfhacks.com
*/
if (!defined('SMF'))
die('Hacking attempt...');

// Globals
$feedcount 0;
$maxitemcount 0;
$tag '';
$insideitem false;

function 
verify_rss_url($url)
{
global $txt$modSettings;

// Rss Data storage
$finalrss '';
$failed true;

if ($modSettings['rss_feedmethod'] == 'All' || $modSettings['rss_feedmethod'] == 'fopen')
{
$fp2 = @fopen($url"r");
if ($fp2)
{
$failed false;
$contents '';
while (!feof($fp2))
{
  $contents .= fread($fp28192);
}

fclose($fp2);

$finalrss $contents;
}
}


// Use Fsockopen
if ($modSettings['rss_feedmethod'] == 'All' || $modSettings['rss_feedmethod'] == 'fsockopen')
{


if($failed == true)
{
$url_array parse_url($url);

$fp = @fsockopen($url_array['host'], 80$errno$errstr30);
if (!$fp)
{

}
else
{
$failed false;

   $out "GET " $url_array['path'] . @$url_array['query'] . "  HTTP/1.1\r\n";
   $out .= "Host: " $url_array['host'] . "\r\n";
   $out .= "Connection: Close\r\n\r\n";

   fwrite($fp$out);

   $rssdata '';

   while (!feof($fp))
   {
       $rssdata .= fgets($fp128);
   }
   fclose($fp);

   // Get rid of the stupid header information! Wish the function did it for me.
   $rss2 explode("\\r\\"$rssdata);
   @$finalrss = @$rss2[1];

}
}
}

// Use cURL
if ($modSettings['rss_feedmethod'] == 'All' || $modSettings['rss_feedmethod'] == 'curl')
{
if($failed == true)
{
if(function_exists("curl_init"))
{
$failed false;
// Last but not least try cUrl
$ch curl_init();

// set URL and other appropriate options
curl_setopt($chCURLOPT_URL$url);
curl_setopt($chCURLOPT_RETURNTRANSFERtrue);

// grab URL, and return output
$output curl_exec($ch);

// close curl resource, and free up system resources
curl_close($ch);
return $output;
}
}
}



// XML Parser functions to verify the XML Feed
if($failed == false)
{
$depth = array();

$xml_parser xml_parser_create();
xml_set_element_handler($xml_parser"startElement2""endElement2");


   if (!xml_parse($xml_parser$finalrss)) {
      fatal_error(sprintf($txt['feedposter_err_xmlerror'],
                   xml_error_string(xml_get_error_code($xml_parser)),
                   xml_get_current_line_number($xml_parser)), false);
   }

xml_parser_free($xml_parser);
}
else
{
// We were not able to download the feed :(
fatal_error($txt['feedposter_err_nodownload'], false);
}


}


function 
startElement2($parser$name$attrs)
{
   global 
$depth;
   
$depth[$parser]++;
}

function 
endElement2($parser$name)
{
   global 
$depth;
   
$depth[$parser]--;
}

function 
UpdateRSSFeedBots()
{
global $db_prefix$context$sourcedir$feedcount$func$maxitemcount$insideitem$tag$modSettings;

// First get all the enabled bots
$context['feeds'] = array();
$request db_query("
SELECT
ID_FEED, ID_BOARD, feedurl, title, postername, updatetime, enabled, html,
ID_MEMBER, locked, articlelink, topicprefix, numbertoimport, importevery
FROM 
{$db_prefix}feedbot
WHERE enabled = 1"
__FILE____LINE__);

while ($row mysql_fetch_assoc($request))
{
$context['feeds'][] = array(
'ID_FEED' => $row['ID_FEED'],
'ID_BOARD' => $row['ID_BOARD'],
'feedurl' => $row['feedurl'],
'title' => $row['title'],
'postername' => $row['postername'],
'enabled' => $row['enabled'],
'html' => $row['html'],
'ID_MEMBER' => $row['ID_MEMBER'],
'locked' => $row['locked'],
'articlelink' => $row['articlelink'],
'topicprefix' => $row['topicprefix'],
'numbertoimport' => $row['numbertoimport'],
'importevery' => $row['importevery'],
'updatetime' => $row['updatetime']
);
}

mysql_free_result($request);

// For the createPost function
require_once($sourcedir '/Subs-Post.php');

// Check if a field expired
foreach ($context['feeds'] as $key => $feed)
{

$current_time time();


// If the feedbot time to next import has expired
if (($current_time mktime(0$feed['importevery'])) > $feed['updatetime'])
//if( 1 == 1)
{

$feeddata GetRSSData($feed['feedurl']);



if ($feeddata != false)
{

// Process the XML
$xml_parser xml_parser_create();
$context['feeditems'] = array();
$feedcount 0;
$maxitemcount $feed['numbertoimport'];
$tag '';
$insideitem false;
$context['feeditems'][0] = array();
$context['feeditems'][0][] = array();
$context['feeditems'][0]['title'] = '';
$context['feeditems'][0]['description'] = '';
$context['feeditems'][0]['link'] = '';



xml_set_element_handler($xml_parser"startElement1""endElement1");
xml_set_character_data_handler($xml_parser"characterData1");

if (!xml_parse($xml_parser$feeddata))
 {
// Error reading xml data

     xml_parser_free($xml_parser);
 }
else
{
    // Data must be valid lets extra some information from it
    // RSS Feeds are a list of items that might contain title, description, and link


    // Free the xml parser memory
xml_parser_free($xml_parser);

// Loop though all the items

for ($i 0$i < ($maxitemcount); $i++)
{
// Check feed Log
// Generate the hash for the log
if(!isset($context['feeditems'][$i]['title']) || !isset($context['feeditems'][$i]['description']) || !isset($context['feeditems'][$i]['link']))
continue;


$itemhash md5($context['feeditems'][$i]['title'] . $context['feeditems'][$i]['description']);
$request db_query("
SELECT
feedtime
FROM 
{$db_prefix}feedbot_log
WHERE feedhash = '
$itemhash'"__FILE____LINE__);

mysql_freeresult($request);


// If no has has found that means no duplicate entry
if (db_affected_rows() == 0)
{


// Create the Post
$msg_title $func['htmlspecialchars'](($feed['html'] ? $context['feeditems'][$i]['title'] : strip_tags($context['feeditems'][$i]['title'])), ENT_QUOTES);

$msg_body =  $func['htmlspecialchars'](($feed['html'] ? $context['feeditems'][$i]['description'] . "\n\n" $context['feeditems'][$i]['link']  : strip_tags($context['feeditems'][$i]['description'] .  "\n\n" $context['feeditems'][$i]['link'])), ENT_QUOTES);

if ($feed['html'])
{
$msg_body '[html]' $msg_body '[/html]';
preparsecode($msg_body);
}
$msgOptions = array(
'id' => 0,
'subject' => $feed['topicprefix'] . $msg_title,
'body' => '[b]' $msg_title "[/b]\n\n" $msg_body,
'icon' => 'xx',
'smileys_enabled' => 1,
'attachments' => array(),
);
$topicOptions = array(
'id' => 0,
'board' => $feed['ID_BOARD'],
'poll' => null,
'lock_mode' => $feed['locked'],
'sticky_mode' => null,
'mark_as_read' => true,
);
$posterOptions = array(
'id' => $feed['ID_MEMBER'],
'name' => $feed['postername'],
'email' => '',
'update_post_count' => (($feed['ID_MEMBER'] == 0) ? 1),
);

createPost($msgOptions$topicOptions$posterOptions);

// Add Feed Log
$fid $feed['ID_FEED'];
$ftime time();

db_query("
INSERT INTO 
{$db_prefix}feedbot_log
(ID_FEED, feedhash, feedtime)
VALUES
(
$fid,'$itemhash',$ftime)"__FILE____LINE__);

}
}

 } // End valid XML check



}  // End get feed data

// End expire check


// End for each feed

}

function 
GetRSSData($url)
{
global $modSettings;
$url_array parse_url($url);


if ($modSettings['rss_feedmethod'] == 'All' || $modSettings['rss_feedmethod'] == 'fopen')
{
$fp2 = @fopen($url"r");
if ($fp2)
{
$contents '';
while (!feof($fp2))
{
  $contents .= fread($fp28192);
}

fclose($fp2);

return $contents;
}
}

if ($modSettings['rss_feedmethod'] == 'All' || $modSettings['rss_feedmethod'] == 'fsockopen')
{
$fp fsockopen($url_array['host'], 80$errno$errstr30);
if (!$fp)
{

}
else
{


   $out "GET " $url_array['path'] . @$url_array['query'] . "  HTTP/1.1\r\n";
   $out .= "Host: " $url_array['host'] . "\r\n";
   $out .= "Connection: Close\r\n\r\n";

   fwrite($fp$out);

   $rssdata '';

   while (!feof($fp))
   {
       $rssdata .= fgets($fp128);
   }
   fclose($fp);

   // Get rid of the stupid header information! Wish the function did it for me.
   $rss2 explode("\\r\\"$rssdata);
   $finalrss $rss2[1];

   return  $finalrss;
}
}

if ($modSettings['rss_feedmethod'] == 'All' || $modSettings['rss_feedmethod'] == 'curl')
{
if(function_exists("curl_init"))
{
// Last but not least try cUrl
$ch curl_init();

// set URL and other appropriate options
curl_setopt($chCURLOPT_URL$url);
curl_setopt($chCURLOPT_RETURNTRANSFERtrue);

// grab URL, and return output
$output curl_exec($ch);

// close curl resource, and free up system resources
curl_close($ch);

return $output;
}
}


// Failure return false
return false;

}

function 
startElement1($parser$name$attrs)
 {
global $insideitem$tag;
if ($insideitem)
{
$tag $name;
}
elseif ($name == "ITEM")
{
$insideitem true;
}
}

function 
endElement1($parser$name)
{
global $insideitem$tag$feedcount$context;

if ($name == "ITEM")
{
$feedcount++;
$context['feeditems'][$feedcount] = array();
$context['feeditems'][$feedcount][] = array();
$context['feeditems'][$feedcount]['title'] = '';
$context['feeditems'][$feedcount]['description'] = '';
$context['feeditems'][$feedcount]['link'] = '';
$insideitem false;
}
}

function 
characterData1($parser$data)
 {
global $insideitem$tag,  $feedcount$context$maxitemcount;

if ($insideitem && $feedcount $maxitemcount)
 
{
switch ($tag)
{
case "TITLE":
$context['feeditems'][$feedcount]['title'] .= $data;
break;

case "DESCRIPTION":
$context['feeditems'][$feedcount]['description'] .= $data;

break;

case "LINK":
$context['feeditems'][$feedcount]['link'] .= $data;
break;
}
}
}
?>

Offline Galaxy Computers

  • Full Member
  • ***
  • Posts: 612
  • Gender: Male
    • mtechama on Facebook
    • @mtechama on Twitter
    • Morris Technologies
Re: RSS Feeder
« Reply #488 on: June 03, 2009, 11:12:20 AM »
I have enter the address of an feeder from a website and that website has updates of news but my mod isn't pulling the feed from that site. Why?

Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,276
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
Re: RSS Feeder
« Reply #489 on: June 03, 2009, 03:34:20 PM »
Sorry mghq - I'm not sure how his mod works, so I wouldn't be able to give any good advised on how to do that there.

mtechama - I have no clue why... what is the time you have set for the scheduled task to run?  Any errors in your forum error log related to the mod?
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline Galaxy Computers

  • Full Member
  • ***
  • Posts: 612
  • Gender: Male
    • mtechama on Facebook
    • @mtechama on Twitter
    • Morris Technologies
Re: RSS Feeder
« Reply #490 on: June 03, 2009, 06:09:43 PM »
These are the Scheduled Tasks Settings are as follows:

Next Due: Today at 04:00:00 PM
Regularity: Starting at 04:00, repeating every 6 hour(s)


Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,276
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
Re: RSS Feeder
« Reply #491 on: June 03, 2009, 07:40:38 PM »
mmk.... does the Scheduled task log show any runs for the RSS Feeder?  If not, did you try manually running it (Click check box next to task and click 'Run Now')?  Any errors in the error log?
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline Galaxy Computers

  • Full Member
  • ***
  • Posts: 612
  • Gender: Male
    • mtechama on Facebook
    • @mtechama on Twitter
    • Morris Technologies
Re: RSS Feeder
« Reply #492 on: June 03, 2009, 07:59:23 PM »
It shows alot of logs on Task Logs but it is still not pulling News feed from a webiste. No errors.
« Last Edit: June 03, 2009, 08:03:00 PM by mtechama »

Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,276
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
Re: RSS Feeder
« Reply #493 on: June 03, 2009, 10:13:47 PM »
mmk.... does the Scheduled task log show any runs for the RSS Feeder?  If not, did you try manually running it (Click check box next to task and click 'Run Now')?  Any errors in the error log?


Also, what is the feed you are trying to pull?
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline Galaxy Computers

  • Full Member
  • ***
  • Posts: 612
  • Gender: Male
    • mtechama on Facebook
    • @mtechama on Twitter
    • Morris Technologies
Re: RSS Feeder
« Reply #494 on: June 03, 2009, 10:16:01 PM »

Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,276
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
Re: RSS Feeder
« Reply #495 on: June 03, 2009, 10:41:03 PM »
Ok, I'm only going to ask one more time... have you tried running the scheduled task manually to see what happens?
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline Galaxy Computers

  • Full Member
  • ***
  • Posts: 612
  • Gender: Male
    • mtechama on Facebook
    • @mtechama on Twitter
    • Morris Technologies
Re: RSS Feeder
« Reply #496 on: June 03, 2009, 10:42:58 PM »
What do mean manually? How do you do that?

Offline Galaxy Computers

  • Full Member
  • ***
  • Posts: 612
  • Gender: Male
    • mtechama on Facebook
    • @mtechama on Twitter
    • Morris Technologies
Re: RSS Feeder
« Reply #497 on: June 03, 2009, 11:21:58 PM »
Would you like to know what I have when add a new feed?

Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,276
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
Re: RSS Feeder
« Reply #498 on: June 03, 2009, 11:58:16 PM »
Screen shot of run now...

SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline Galaxy Computers

  • Full Member
  • ***
  • Posts: 612
  • Gender: Male
    • mtechama on Facebook
    • @mtechama on Twitter
    • Morris Technologies
Re: RSS Feeder
« Reply #499 on: June 04, 2009, 12:03:07 AM »
I have done that but still no changes.