RSS Feeder

[SlammedDime]

What feed are you trying to use to create posts?  Do you understand the purpose of this modification?

[supercarp]

Wow. Seriously? Nobody is going to offer any input on this?

The mod is not doing what it's supposed to do on an error-free feed, and apparently I'm not the only one having this issue.

It's been a while since I looked at my own code, but I'm willing to bet it's because your 'error-free' feed doesn't have any content in the 'description' tag, so the mod ignores it.  If there is no description or content, why post it?  In comparison, the other link you provided does have content in the description tag.

There's enough info in the feed that I'd like to be able to have it posted, regardless of what's inside the description tag.

I will, however, see if I can't get the person who publishes the feed to populate the description tag so I don't have to try and succeed at breaking something.

Thanks again.

[GazOutEast]

Wow. Seriously? Nobody is going to offer any input on this?

The mod is not doing what it's supposed to do on an error-free feed, and apparently I'm not the only one having this issue.

It's been a while since I looked at my own code, but I'm willing to bet it's because your 'error-free' feed doesn't have any content in the 'description' tag, so the mod ignores it.  If there is no description or content, why post it?  In comparison, the other link you provided does have content in the description tag.

There's enough info in the feed that I'd like to be able to have it posted, regardless of what's inside the description tag.

I will, however, see if I can't get the person who publishes the feed to populate the description tag so I don't have to try and succeed at breaking something.

Thanks again.

SC

As SlammedDime said - but also, browse to the feed page and look at the source on that page.
All the description fields contain -

Code Select Expand


<description><![CDATA[]]></description>


That's it - just the dynamic data script starter / call - problem with that is that the script call has no source info for when it is run remotely (such as in a feed reader) therefore it doesn't know where to look to get the content.

Additionally - if you look at the source URL, it is a live feed for and from a MMORPG - my betting is that you actually have to view it from within the RPG itself to view the data from the script - this is likely to stop server leeching and programmed deliberately by the game's authors.

Looking at the titles of the feed contents, if that's all you want to post - be prepared for a heavy demotion in page rank by Search Engines - those titles are duplicate content slap magnets.

Your choice as webmaster of course, but I wouldn't even have tried to get the URL for a feed with that type of content.

Gaz

[supercarp]

yeah, the stuff from that feed is going into a very private, locked-down part of the forum, so it'd never be seen by a search engine.  It's purely informational.

[atdev]

I can't seem to install this mod. I placed the newest release  .ZIP in the "packages" folder through FTP, and it shows up, but without an "install" link.

   Mod Name   Version   
1.   RSS Feeder    1.1.4          [ List Files ] [ Delete ]

is all I see.


Any help would be greatly appreciated.

[SlammedDime]

Are you using 2.0 RC3?

[atdev]

Are you using 2.0 RC3?

No. SMF 1.1.10

[Galaxy Computers]

Can someone tell me if this is an actual feed from Symantec's Website?
If it is how can put it in the RSS feeder on the forum?


http://www.symantec.com/xml/rss/listings.jsp?lid=latestthreats30days

[SlammedDime]

Are you using 2.0 RC3?

No. SMF 1.1.10

There-in lies the issue... this mod is for 2.0RC3 only.

Can someone tell me if this is an actual feed from Symantec's Website?
If it is how can put it in the RSS feeder on the forum?


http://www.symantec.com/xml/rss/listings.jsp?lid=latestthreats30days

Yes it is... you add it like you add any other.

[atdev]

Are you using 2.0 RC3?

No. SMF 1.1.10

There-in lies the issue... this mod is for 2.0RC3 only.

Now I feel like a n00b... Thanks! 

I installed 2.0 rc3 (very easy +1 to the devs)
and wonder of wonders I now have an install link! However, I get this error:
ERROR
The requested URL could not be retrieved
While trying to retrieve the URL:

The following error was encountered:

Zero Sized Reply

Squid did not receive any data for this request.

EDIT:
Turns out I can't install other MODs either.

[akbora]

How can I take full page? Retrieve Full Article

I don't understand PCRE? Is there anyoıne knows this as practical? Thanks

[MoreBloodWine]

The mod works flawlessly on my test site, however, I found something interesting I hope you can fix.

Click This: http://feeds.feedburner.com/NaughtyDog?format=xml
Find This (2nd down as of posting): What we're up to at PAX Prime

Then Click This: http://www.eojmarket.com/sigtest/index.php?topic=25.0

Now compare the feed with the post of the feed on my site, what do you see, rather don't you see ?

The picture, it didn't copy / migrate the picture from any of the NaughtyDog feeds. I think it may be because that the NaughtyDogg feed is Atom based or so I was told by someone else.

That said, this mod is copying feeds just fine as far as I can tell, images included from the following (below) feed so I'm guessing that it has to do with what someone said about the other feed (NaughtyDog) being Atom based as to why it's not doing pictures or something. Anyway, is this something you can look into / fix please ?

http://feeds.feedburner.com/PSBlog?format=xml

Edit: Also, would it be possible to add in a Last Fed field on the admin page where you see the following...

Enabled  Feed Title Feed URL Board Posts Modify

I think it would be nice to see when the last feed was posted for those who don't normally check feed boards but want to make sure it's still feeding.

Oh, and it doesnt seem to be posting videos or their links, I don't much care about the video actually embedding because I use AEVA but in order for that to work this mod would atleast have to post the video link(s).

Edit 2: Here's something else all be it minor, it doesnt seem to be copying the articles over as is in that its forgetting simple line breaks and stuff that should be bold.

Case in point: http://feeds.feedburner.com/PSBlog?format=xml
And on my site: http://www.eojmarket.com/sigtest/index.php?topic=12.0

What should be...

PlayStation 3
Spider-Man: Shattered Dimensions – Blending together the many alternate universes of the world-famous webslinger with four unique take on Spider-Man's history in comics. Spider-Man attempts to retrieve a piece of a tablet that's been scattered across multiple parallel dimensions, and in doing so he crosses over to storylines famous to comic book aficionados as variant tellings of the Spider-Man mythology.

In coming out on my site...

PlayStation 3 Spider-Man: Shattered Dimensions – Blending together the many alternate universes of the world-famous webslinger with four unique take on Spider-Man's history in comics. Spider-Man attempts to retrieve a piece of a tablet that's been scattered across multiple parallel dimensions, and in doing so he crosses over to storylines famous to comic book aficionados as variant tellings of the Spider-Man mythology.

Like I said it's minor things that most people wouldnt catch or care about but I really would love it if the transferred data could keep all original formatting from line breaks to new paragraphs, Thx. For the most part the mod seems to work flawlessy all be it missing some line breaks and stuff as well as not bolding what should be bolded sometimes.

Edit 3: It's also missing simple line breaks for other stuff to...

Like if something looks like below...


1. text here

2. text here

3. text here

More text here

It comes out like this...

1. text here
2. text here
3. text here
More text here

Edit 4: Aside from copying the source HTML from within the <content:encoded></content:encoded> tags for non Atom based feeds and <content type="html"></content> for Atom based feeds I cant think of any other way to copy the article to display as it does on the blog pages which is what I really want as does alot of other people I'd imagine. Even so there would still be minor display errors with stuff like below...

What&#8217;s that you say?

It prints the &#8217; instead of making it look like this...

What's that you say?

This is a great mod but I would really love it if the article would display on my site as it does on the blog, I'm assuming whatever the reason it's not now has to do with all of the HTML to BBC conversions.

If anything, would you consider atleast adding a do no strip HTML option to this mod so the mod would post the unedited version it grabs ?

So basically when this option is ticked it would put the entire post inside of BBC HTML tags. The only thing to note would be to make sure the mod is posting as an admin or that in addition to this mod the site owner uses the following mod and provides the appropriate permissions for the use of the tag.

http://custom.simplemachines.org/mods/index.php?mod=411

As for stripping the videos, just make a note that all someone needs to do for thise to show is to use the AEVA mod.

Either way I would really love it if articles werent being butchured to have HTML stripped and replaced with BBC which is throwing off alot of text alignment / placement.

Also lets for a minute assume that Retrieve Full Article does just that, grabs the article and prints it to near perfect quality as seen on the blogs, it would be really cool if you dove a little deeper into that instead of attaching the following note which most people have no clue to even come close to understanding.

Required if retrieving full article. This must be PCRE format. The match is expected to be in second index of the array ([1]).

Edit 5: I also just noticed that it grabs more data for some feeds than it does for others. This really is a great mod but parts of it just seem really flaky at times.

I'd have to say my biggest thing about all this is the stipping all of the HTML and converting it to BBC, if the reason for this is because of people having issues with the HTML tag thats easy enough to get around with the permissions mod I linked.

Lastly, as I understand them schedualed tasks only run when someones visiting the site as in theyre not automated. With that said, would it be possible to by pass schedualed tasks for this mod and run them as a cron job so regardless of wether or not someones visiting the site the feeds are always harvested and then posted ?

[Akyhne]

Suddently got this error message:

http://www.xxx.com/index.php?action=admin;area=scheduledtasks
RSS Feeder: http://uk.playstation.com/rss/news/en_GB/ (fsocket timed out)

Unfortunately, it disables the feed to http://uk.playstation.com. Pretty annoying.
Whenever I enable the feed again, it will be disabled on next attempt.

[MoreBloodWine]

Almsot all hosts I've ever seen allow cURL, try using that and see if it keeps happening.

[Akyhne]

How?

[MoreBloodWine]

How?

Sorry, was thinking of another mod which allows you to switch gather methods...

and note to mod author... getting alot of stuff like this in an error_log file. 333 lines per hit which is every 30 mins since thats what my scheduals set for.

Also, don't forget to reaply my 3rd reply counting this as the first.

Code Select Expand

[07-Sep-2010 16:31:26] PHP Strict Standards:  Non-static method SimplePie_Misc::absolutize_url() should not be called statically, assuming $this from incompatible context in /home/.../public_html/sigtest/Sources/Subs-Rss.php on line 15074
[07-Sep-2010 16:31:26] PHP Strict Standards:  Non-static method SimplePie_IRI::absolutize() should not be called statically, assuming $this from incompatible context in /home/.../public_html/sigtest/Sources/Subs-Rss.php on line 9209
[07-Sep-2010 16:31:26] PHP Strict Standards:  Non-static method SimplePie_Misc::element_implode() should not be called statically, assuming $this from incompatible context in /home/.../public_html/sigtest/Sources/Subs-Rss.php on line 15075

[MoreBloodWine]

Bumping for the errors and this.

[MoreBloodWine]

Liam said he won't be supporting this mod any more because of time constraints or something, he also said he was gonna be contacting the original author about picking this back up. With that said I hope someone does something soon... but ya, just letting anyone who cares about this mod know whats up.

[SlammedDime]

You can disable the errors from showing up by altering the error_reporting line near the top of SMF's index.php to exclude E_STRICT messages (~E_STRICT).

As for about how the feed is parsed into a topic... the purpose of the mod was never to grab a fully detailed feed with images, links, videos or other types of rich content.  I originally designed the mod for use on simplistic RSS feeds that normally only contained simple descriptions and maybe a few images here and there with links to the full article, as that is what my needs were at the time.  Retrieving a full article will behave in the same manner, chopping out all of the gory mess.  I would never feed this as raw html into the database due to the numerous security risks in doing so (I know I could take the time to sanitize the data, but honestly, I don't have the time nor need to do it).  Converting to BBC is the safest and easiest thing to do at this point in time.

The 'Full Article' option is meant for people that know what they're doing... I have an example linked in the first post that covers the majority of feeds and situations...  I would love to implement a DOM based tool that perhaps used XPath to grab the data, but again, I don't have the time nor need.

For CRON and Scheduled Tasks... sure it's possible, a simple script could be written that includes SSI.php and then called using wget via a CRON job to run the script.  Would really only need to be a 4 line script...

Code (rssfeeder_cron.php) Select Expand

<?php
require_once('SSI.php');
require_once($sourcedir . '/ScheduledTasks.php');
scheduled_rss_feeder();


Then setup your cron to call wget http://mydomain.com/forum/rssfeeder_cron.php

[MoreBloodWine]

Well, least the cron thing seems to be out of the way with your solution because now the feeds should post regardless of wether or not anyone is visiting the site but does the task need to stay enabled for that 4 lie file to work or can it be disbled ? I'm guessing it needs to stay enabled since it "tags" the task file.

As for the errors, what would even be causing those anyway and woudlnt disabling that line affect other things when they decide to error ?

Oh, and as for using that code stuff to grab full article, I looked at it and maybe understoood 2% of it and even modifying your "example" to suit my needs didnt seem to work. I really like this mod but I would like it if certain things were looked at / address for people having issues / want proper text formatting and alignment.

Come to think of it, would you for arguments sake consider a paid gig to modify this mod to suit my personal needs, likes / dislikes since you don't seem to want to do anything to the free public version for lack of a better name for it ?