News:

Join the Facebook Fan Page.

Main Menu

RSS Feeder

Started by SlammedDime, January 11, 2009, 06:06:42 AM

Previous topic - Next topic

mrtrc266

Ok I think I got it...is this correct? This will not make it disable the feed and it won't put any errors in the error log? One last question, is there any risk in doing this?


  /* If we don't get a valid chunk of data back, disable the feed
  if ($rss_data->error())
  {
   $smcFunc['db_query']('', '
    UPDATE {db_prefix}rssfeeds
    SET enabled = 0
    WHERE id_feed = {int:feed}',
    array(
     'feed' => $id,
    )
   );
   
   // Log an error about the issue, just so the user can see why their feed was disabled...
   log_error($txt['rss_feeder'] . ': ' . $feed['url'] . ' (' . $rss_data->error() . ')');
   continue;
  }*/

SlammedDime

I'd only comment out the sql query and keep the error log entry in there, unless you really don't want any error logging of failed feeds, but at that point it will be quite hard to provide any support on the mod.

No risk in commenting that out.
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

mghq

Hey seeing as this mod is similar to the one for SMF 1.1.X could you tell me how you fixed the html problem because i cant figure it out

SlammedDime

What HTML problem?
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

mrtrc266

Thank you very much Slammed, feeds have not been automatically disabled all day :D

mghq

Quote from: SlammedDime on May 27, 2009, 10:12:31 PM
What HTML problem?

Well with the one from vbgamer, it will not add the  code tag even for an administrator. And if it does it always screws up these symbols <, >, "

i thought if i edit Subs-RSS.php it would fix it, but it messed the symbols up but put the html tags. So how did you go about fixing that

SlammedDime

Here is the PHP function that I run everything through...

function striphtml($document)
{
// Just in case there aren't breaks after a </p>...
$document = str_replace(array('</p>', '<br />', '<br>',  '</div>'), array("</p>\r\n", "<br />\r\n", "<br>\r\n", "</div>\r\n"), $document);

$search = array('~<style[^>]*?>.*?</style>~siU',    // Strip style tags properly
"'<script[^>]*?>.*?</script>'si", // strip out javascript
"~<img.*?src=\"(.*?)\".*?>~si", // convert img tags
"'<[\/\!]*?[^<>]*?>'si", // strip out html tags
"'([\r\n])[\s]+'", // strip out white space
"'&(quot|#34|#034|#x22);'i", // replace html entities
"'&(amp|#38|#038|#x26);'i", // added hexadecimal values
"'&(lt|#60|#060|#x3c);'i",
"'&(gt|#62|#062|#x3e);'i",
"'&(nbsp|#160|#xa0);'i",
"'&(iexcl|#161);'i",
"'&(cent|#162);'i",
"'&(pound|#163);'i",
"'&(copy|#169);'i",
"'&(reg|#174);'i",
"'&(deg|#176);'i",
"'&(#39|#039|#x27);'",
"'&(euro|#8364);'i", // europe
"'&a(uml|UML);'", // german
"'&o(uml|UML);'",
"'&u(uml|UML);'",
"'&A(uml|UML);'",
"'&O(uml|UML);'",
"'&U(uml|UML);'",
"'&szlig;'i",
"~(\r\n|\n|\r)+~",
);
$replace = array( "",
"",
"",
"",
"\\1",
"\"",
"&",
"<",
">",
" ",
chr(161),
chr(162),
chr(163),
chr(169),
chr(174),
chr(176),
chr(39),
chr(128),
"ä",
"ö",
"ü",
"Ä",
"Ö",
"Ü",
"ß",
"\n\n",
);

$text = preg_replace($search,$replace,$document);

return $text;
}
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

mghq

Do you know how we could fix that for this one
<?php
/*
RSS Feed Poster
Version 1.0.4
by:vbgamer45
http://www.smfhacks.com
*/
if (!defined('SMF'))
die('Hacking attempt...');

// Globals
$feedcount 0;
$maxitemcount 0;
$tag '';
$insideitem false;

function 
verify_rss_url($url)
{
global $txt$modSettings;

// Rss Data storage
$finalrss '';
$failed true;

if ($modSettings['rss_feedmethod'] == 'All' || $modSettings['rss_feedmethod'] == 'fopen')
{
$fp2 = @fopen($url"r");
if ($fp2)
{
$failed false;
$contents '';
while (!feof($fp2))
{
  $contents .= fread($fp28192);
}

fclose($fp2);

$finalrss $contents;
}
}


// Use Fsockopen
if ($modSettings['rss_feedmethod'] == 'All' || $modSettings['rss_feedmethod'] == 'fsockopen')
{


if($failed == true)
{
$url_array parse_url($url);

$fp = @fsockopen($url_array['host'], 80$errno$errstr30);
if (!$fp)
{

}
else
{
$failed false;

   $out "GET " $url_array['path'] . @$url_array['query'] . "  HTTP/1.1\r\n";
   $out .= "Host: " $url_array['host'] . "\r\n";
   $out .= "Connection: Close\r\n\r\n";

   fwrite($fp$out);

   $rssdata '';

   while (!feof($fp))
   {
       $rssdata .= fgets($fp128);
   }
   fclose($fp);

   // Get rid of the stupid header information! Wish the function did it for me.
   $rss2 explode("\\r\\"$rssdata);
   @$finalrss = @$rss2[1];

}
}
}

// Use cURL
if ($modSettings['rss_feedmethod'] == 'All' || $modSettings['rss_feedmethod'] == 'curl')
{
if($failed == true)
{
if(function_exists("curl_init"))
{
$failed false;
// Last but not least try cUrl
$ch curl_init();

// set URL and other appropriate options
curl_setopt($chCURLOPT_URL$url);
curl_setopt($chCURLOPT_RETURNTRANSFERtrue);

// grab URL, and return output
$output curl_exec($ch);

// close curl resource, and free up system resources
curl_close($ch);
return $output;
}
}
}



// XML Parser functions to verify the XML Feed
if($failed == false)
{
$depth = array();

$xml_parser xml_parser_create();
xml_set_element_handler($xml_parser"startElement2""endElement2");


   if (!xml_parse($xml_parser$finalrss)) {
      fatal_error(sprintf($txt['feedposter_err_xmlerror'],
                   xml_error_string(xml_get_error_code($xml_parser)),
                   xml_get_current_line_number($xml_parser)), false);
   }

xml_parser_free($xml_parser);
}
else
{
// We were not able to download the feed :(
fatal_error($txt['feedposter_err_nodownload'], false);
}


}


function 
startElement2($parser$name$attrs)
{
   global 
$depth;
   
$depth[$parser]++;
}

function 
endElement2($parser$name)
{
   global 
$depth;
   
$depth[$parser]--;
}

function 
UpdateRSSFeedBots()
{
global $db_prefix$context$sourcedir$feedcount$func$maxitemcount$insideitem$tag$modSettings;

// First get all the enabled bots
$context['feeds'] = array();
$request db_query("
SELECT
ID_FEED, ID_BOARD, feedurl, title, postername, updatetime, enabled, html,
ID_MEMBER, locked, articlelink, topicprefix, numbertoimport, importevery
FROM 
{$db_prefix}feedbot
WHERE enabled = 1"
__FILE____LINE__);

while ($row mysql_fetch_assoc($request))
{
$context['feeds'][] = array(
'ID_FEED' => $row['ID_FEED'],
'ID_BOARD' => $row['ID_BOARD'],
'feedurl' => $row['feedurl'],
'title' => $row['title'],
'postername' => $row['postername'],
'enabled' => $row['enabled'],
'html' => $row['html'],
'ID_MEMBER' => $row['ID_MEMBER'],
'locked' => $row['locked'],
'articlelink' => $row['articlelink'],
'topicprefix' => $row['topicprefix'],
'numbertoimport' => $row['numbertoimport'],
'importevery' => $row['importevery'],
'updatetime' => $row['updatetime']
);
}

mysql_free_result($request);

// For the createPost function
require_once($sourcedir '/Subs-Post.php');

// Check if a field expired
foreach ($context['feeds'] as $key => $feed)
{

$current_time time();


// If the feedbot time to next import has expired
if (($current_time mktime(0$feed['importevery'])) > $feed['updatetime'])
//if( 1 == 1)
{

$feeddata GetRSSData($feed['feedurl']);



if ($feeddata != false)
{

// Process the XML
$xml_parser xml_parser_create();
$context['feeditems'] = array();
$feedcount 0;
$maxitemcount $feed['numbertoimport'];
$tag '';
$insideitem false;
$context['feeditems'][0] = array();
$context['feeditems'][0][] = array();
$context['feeditems'][0]['title'] = '';
$context['feeditems'][0]['description'] = '';
$context['feeditems'][0]['link'] = '';



xml_set_element_handler($xml_parser"startElement1""endElement1");
xml_set_character_data_handler($xml_parser"characterData1");

if (!xml_parse($xml_parser$feeddata))
 {
// Error reading xml data

     xml_parser_free($xml_parser);
 }
else
{
    // Data must be valid lets extra some information from it
    // RSS Feeds are a list of items that might contain title, description, and link


    // Free the xml parser memory
xml_parser_free($xml_parser);

// Loop though all the items

for ($i 0$i < ($maxitemcount); $i++)
{
// Check feed Log
// Generate the hash for the log
if(!isset($context['feeditems'][$i]['title']) || !isset($context['feeditems'][$i]['description']) || !isset($context['feeditems'][$i]['link']))
continue;


$itemhash md5($context['feeditems'][$i]['title'] . $context['feeditems'][$i]['description']);
$request db_query("
SELECT
feedtime
FROM 
{$db_prefix}feedbot_log
WHERE feedhash = '
$itemhash'"__FILE____LINE__);

mysql_freeresult($request);


// If no has has found that means no duplicate entry
if (db_affected_rows() == 0)
{


// Create the Post
$msg_title $func['htmlspecialchars'](($feed['html'] ? $context['feeditems'][$i]['title'] : strip_tags($context['feeditems'][$i]['title'])), ENT_QUOTES);

$msg_body =  $func['htmlspecialchars'](($feed['html'] ? $context['feeditems'][$i]['description'] . "\n\n" $context['feeditems'][$i]['link']  : strip_tags($context['feeditems'][$i]['description'] .  "\n\n" $context['feeditems'][$i]['link'])), ENT_QUOTES);

if ($feed['html'])
{
$msg_body '[html]' $msg_body '[/html]';
preparsecode($msg_body);
}
$msgOptions = array(
'id' => 0,
'subject' => $feed['topicprefix'] . $msg_title,
'body' => '[b]' $msg_title "[/b]\n\n" $msg_body,
'icon' => 'xx',
'smileys_enabled' => 1,
'attachments' => array(),
);
$topicOptions = array(
'id' => 0,
'board' => $feed['ID_BOARD'],
'poll' => null,
'lock_mode' => $feed['locked'],
'sticky_mode' => null,
'mark_as_read' => true,
);
$posterOptions = array(
'id' => $feed['ID_MEMBER'],
'name' => $feed['postername'],
'email' => '',
'update_post_count' => (($feed['ID_MEMBER'] == 0) ? 1),
);

createPost($msgOptions$topicOptions$posterOptions);

// Add Feed Log
$fid $feed['ID_FEED'];
$ftime time();

db_query("
INSERT INTO 
{$db_prefix}feedbot_log
(ID_FEED, feedhash, feedtime)
VALUES
(
$fid,'$itemhash',$ftime)"__FILE____LINE__);

}
}

 } // End valid XML check



}  // End get feed data

// End expire check


// End for each feed

}

function 
GetRSSData($url)
{
global $modSettings;
$url_array parse_url($url);


if ($modSettings['rss_feedmethod'] == 'All' || $modSettings['rss_feedmethod'] == 'fopen')
{
$fp2 = @fopen($url"r");
if ($fp2)
{
$contents '';
while (!feof($fp2))
{
  $contents .= fread($fp28192);
}

fclose($fp2);

return $contents;
}
}

if ($modSettings['rss_feedmethod'] == 'All' || $modSettings['rss_feedmethod'] == 'fsockopen')
{
$fp fsockopen($url_array['host'], 80$errno$errstr30);
if (!$fp)
{

}
else
{


   $out "GET " $url_array['path'] . @$url_array['query'] . "  HTTP/1.1\r\n";
   $out .= "Host: " $url_array['host'] . "\r\n";
   $out .= "Connection: Close\r\n\r\n";

   fwrite($fp$out);

   $rssdata '';

   while (!feof($fp))
   {
       $rssdata .= fgets($fp128);
   }
   fclose($fp);

   // Get rid of the stupid header information! Wish the function did it for me.
   $rss2 explode("\\r\\"$rssdata);
   $finalrss $rss2[1];

   return  $finalrss;
}
}

if ($modSettings['rss_feedmethod'] == 'All' || $modSettings['rss_feedmethod'] == 'curl')
{
if(function_exists("curl_init"))
{
// Last but not least try cUrl
$ch curl_init();

// set URL and other appropriate options
curl_setopt($chCURLOPT_URL$url);
curl_setopt($chCURLOPT_RETURNTRANSFERtrue);

// grab URL, and return output
$output curl_exec($ch);

// close curl resource, and free up system resources
curl_close($ch);

return $output;
}
}


// Failure return false
return false;

}

function 
startElement1($parser$name$attrs)
 {
global $insideitem$tag;
if ($insideitem)
{
$tag $name;
}
elseif ($name == "ITEM")
{
$insideitem true;
}
}

function 
endElement1($parser$name)
{
global $insideitem$tag$feedcount$context;

if ($name == "ITEM")
{
$feedcount++;
$context['feeditems'][$feedcount] = array();
$context['feeditems'][$feedcount][] = array();
$context['feeditems'][$feedcount]['title'] = '';
$context['feeditems'][$feedcount]['description'] = '';
$context['feeditems'][$feedcount]['link'] = '';
$insideitem false;
}
}

function 
characterData1($parser$data)
 {
global $insideitem$tag,  $feedcount$context$maxitemcount;

if ($insideitem && $feedcount $maxitemcount)
 
{
switch ($tag)
{
case "TITLE":
$context['feeditems'][$feedcount]['title'] .= $data;
break;

case "DESCRIPTION":
$context['feeditems'][$feedcount]['description'] .= $data;

break;

case "LINK":
$context['feeditems'][$feedcount]['link'] .= $data;
break;
}
}
}
?>

Galaxy Computers

I have enter the address of an feeder from a website and that website has updates of news but my mod isn't pulling the feed from that site. Why?
Wade Morris
Amarillo, Texas

Morris Technologies Computer Support Forum

SlammedDime

Sorry mghq - I'm not sure how his mod works, so I wouldn't be able to give any good advised on how to do that there.

mtechama - I have no clue why... what is the time you have set for the scheduled task to run?  Any errors in your forum error log related to the mod?
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Galaxy Computers

These are the Scheduled Tasks Settings are as follows:

Next Due: Today at 04:00:00 PM
Regularity: Starting at 04:00, repeating every 6 hour(s)

Wade Morris
Amarillo, Texas

Morris Technologies Computer Support Forum

SlammedDime

mmk.... does the Scheduled task log show any runs for the RSS Feeder?  If not, did you try manually running it (Click check box next to task and click 'Run Now')?  Any errors in the error log?
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Galaxy Computers

#492
It shows alot of logs on Task Logs but it is still not pulling News feed from a webiste. No errors.
Wade Morris
Amarillo, Texas

Morris Technologies Computer Support Forum

SlammedDime

Quote from: SlammedDime on June 03, 2009, 07:40:38 PM
mmk.... does the Scheduled task log show any runs for the RSS Feeder?  If not, did you try manually running it (Click check box next to task and click 'Run Now')?  Any errors in the error log?


Also, what is the feed you are trying to pull?
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Galaxy Computers

Wade Morris
Amarillo, Texas

Morris Technologies Computer Support Forum

SlammedDime

Ok, I'm only going to ask one more time... have you tried running the scheduled task manually to see what happens?
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Galaxy Computers

What do mean manually? How do you do that?
Wade Morris
Amarillo, Texas

Morris Technologies Computer Support Forum

Galaxy Computers

Would you like to know what I have when add a new feed?
Wade Morris
Amarillo, Texas

Morris Technologies Computer Support Forum

SlammedDime

Screen shot of run now...

SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Galaxy Computers

I have done that but still no changes.
Wade Morris
Amarillo, Texas

Morris Technologies Computer Support Forum

Advertisement: