Advertisement:

Author Topic: RSS Feeder  (Read 361036 times)

Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,280
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
    • ZubaFitness
RSS Feeder
« on: January 11, 2009, 06:06:42 AM »
Link to Mod

RSS Feeder
This allows you to add RSS Feed 'Bots' to your forum to display information from RSS Feeds.

Features of this mod:
  • Unlimited Feeds
  • Post to any board on your site
  • Specify which user to post as
  • Single click enable/disable of any feed from the admin panel
  • Lock feed topics once posted
  • Require approval of feeds before publicly viewable (only if Post Moderation is enabled)
  • Prefix feed with any title (Ie: [FEED], [RSS], [NEWS], etc)
  • Retrieve full article (Regular Expression REQUIRED)
  • Find articles by keyword in the Title or Description
  • Specify number of items to import from each feed
  • Caching of feeds for faster performance in case the feed hasn't changed
  • Logging of posted feeds to avoid duplicates
  • Uses Scheduled Tasks to determine when to fetch feeds (you can set the frequency)
  • Log pruning - removes log entries that are 30 days old (this can be changed too)
  • Option to post all feed items into a single topic, instead of seperate topics
  • See how many items each feed as posted
  • Selectable image icons for feeds
  • Add a custom footer to each post for feeds

Change Log:
Version 1.1.0
-------------
* Added a 'Source' bit at the bottom of each post (ManageSettings.php, RSSFeeds.template.php, ScheduledTasks.php)
* Add a footer to the bottom of the post after the feed content (ManageSettings.php, RSSFeeds.template.php, ScheduledTasks.php)
* Selectable message icons for each feed (ManageSettings.php, RSSFeeds.template.php)
+ Allow images in the posts by converting <img> tags to bbcode [img] tags (Subs-Rss.php)
* Feeds keep track of items posted from them, viewable in the feed setup page (ManageSettings.php, ScheduledTasks.php)
? Mark as read option causing errors on scheduled task run, removed feature (ManageSettings.php, RSSFeeds.template.php, ScheduledTasks.php)
! Forgot to check if description was empty (ScheduledTasks.php)
* Option to post all feed items into a single topic, instead of seperate topics (ManageSettings.php, RSSFeeds.template.php)
! Error might be thrown if regular expression didn't find anything (ScheduledTasks.php)
+ Switched from MagPie to SimplePie for the RSS api (Subs-Rss.php)
+ Added logging for feeds that become disabled when the scheduled task runs (ScheduledTasks.php)

Version 1.0.0
-------------
* Release

Legend:
* = New feature
+ = Improved feature
! = Fixed bug
? = Other stuff


Regex Help
Attached to this post is a file named regex.php... you can upload this script to your web server to test out your regular expressions with a given input.  It's pretty self explanatory... the goal is end up with what you want in the [1] => part of the array returned.  For debugging purposes, the entire matches array is displayed.  See this post of the topic as well.



*Known Issues*
This was meant to work with Auto Suggest for choosing who to post as, but I didn't realize that there was a change in the auto suggest script from Beta 4 to RC1, so auto suggest will only work on RC1+.  This does not hinder the mod in any way, except that you must properly enter the username / displayname otherwise you will get thrown back to an error.

Doesn't play nice with utf-8 characters, cuts off strings or converts to ?s in the Prefix or Footer fields, possible database collation issue.

Suggestions
  • Create an additional section in a user's profile to allow them to create and maintain their own feeds that post to boards of their choice or admin selected boards.
« Last Edit: January 16, 2009, 10:48:41 PM by SlammedDime »
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline qubbah

  • Full Member
  • ***
  • Posts: 559
Re: RSS Feeder
« Reply #1 on: January 11, 2009, 06:56:30 AM »
hope next time it can be used with 1.1.7


Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,280
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
    • ZubaFitness
Re: RSS Feeder
« Reply #2 on: January 11, 2009, 07:47:57 AM »
Sorry, I won't be making a version of this for 1.1.x.  vbgamer has an archived version that still works with 1.1.7 as far as I know, you may want to give that one a shot.
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline Nao 尚

  • SMF Friend
  • SMF Hero
  • *
  • Posts: 5,928
  • Gender: Male
    • wedgebook on Facebook
    • Wedge
Re: RSS Feeder
« Reply #3 on: January 11, 2009, 08:05:10 AM »
Oh... By any chance, is this mod related to the one Smoky Blue was looking into making at some point last week?

I'll have a look into it later if I don't forget. :)
I will not make any deals with you. I've resigned. I will not be pushed, filed, stamped, indexed, briefed, debriefed or numbered.

Aeva Media rocks your life.

Offline Robi052

  • Full Member
  • ***
  • Posts: 647
  • Gender: Male
    • @robi052 on Twitter
Re: RSS Feeder
« Reply #4 on: January 11, 2009, 09:10:57 AM »
my IPP disallow url_fopen. Working this mod with curl?

Offline Nao 尚

  • SMF Friend
  • SMF Hero
  • *
  • Posts: 5,928
  • Gender: Male
    • wedgebook on Facebook
    • Wedge
Re: RSS Feeder
« Reply #5 on: January 11, 2009, 09:25:02 AM »
The mod seems to use fsockopen, just like SMF does for its fetch_web_data function (which retrieves external packages or SMF news etc). If your SMF works fine, then this should work fine too.
I will not make any deals with you. I've resigned. I will not be pushed, filed, stamped, indexed, briefed, debriefed or numbered.

Aeva Media rocks your life.

Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,280
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
    • ZubaFitness
Re: RSS Feeder
« Reply #6 on: January 11, 2009, 10:07:12 AM »
It uses Snoopy php Web browser to retrieve feeds... I really didn't look into it too much, but I do believe it supports curl as well.

Nao - I'm not sure... I didn't know SmokyBlue was working on something like this.  Looking at topics in mod requests, I see that KahneFan was wanting something similar to this and Smoky was working on it.
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline TheDisturbedOne

  • SMF Hero
  • ******
  • Posts: 1,700
    • Disturbed Message Board
Re: RSS Feeder
« Reply #7 on: January 11, 2009, 01:50:04 PM »
Sorry, I won't be making a version of this for 1.1.x.  vbgamer has an archived version that still works with 1.1.7 as far as I know, you may want to give that one a shot.
It doesn't work well I'm afraid.  It posts once and then it dies :(

Offline Yağız...

  • SMF Super Hero
  • *******
  • Posts: 10,811
  • Gender: Male
    • MMO Browser
Re: RSS Feeder
« Reply #8 on: January 11, 2009, 01:58:58 PM »
There is an error with Turkish characters. It shows '?' instead of 'ı', 'İ', 'ğ', 'ş', 'Ş'.

Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,280
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
    • ZubaFitness
Re: RSS Feeder
« Reply #9 on: January 11, 2009, 02:54:10 PM »
I wonder if it's because of the __striptext function of Snoopy... Can you provide a feed as an example.
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline Yağız...

  • SMF Super Hero
  • *******
  • Posts: 10,811
  • Gender: Male
    • MMO Browser
Re: RSS Feeder
« Reply #10 on: January 11, 2009, 03:07:41 PM »

Offline Smoky "Rider" Blue

  • SMF Hero
  • ******
  • Posts: 1,682
  • Gender: Female
  • It's all in your head! :p
    • smoky.blue.01 on Facebook
Re: RSS Feeder
« Reply #11 on: January 11, 2009, 03:11:11 PM »
yes i had something in the works like this

i have a bunch of stuff for rss posters from vb, ipb, drupal, wp, ee, and more..

good to know you tried SlammedDime, and thank you..

on the issues of it only posting once, will check things out..

i did get vbgamers mod working perfect, so..  ;)
Currently in secondlife, contact me for any image creation via "pm" thanks!

**Take the time to remember friendships and family.. Sometimes it's all we have, and missed very much**

Offline KahneFan

  • SMF Hero
  • ******
  • Posts: 1,850
  • Gender: Male
    • I Post You Post
Re: RSS Feeder
« Reply #12 on: January 11, 2009, 03:21:51 PM »
Currently trying this on a test site.

1) Is there a cron page I need to have my server hit, or does it run a cron job automatically or....?

2) Will it import old articles from feeds or only new articles?

Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,280
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
    • ZubaFitness
Re: RSS Feeder
« Reply #13 on: January 11, 2009, 03:28:19 PM »
It runs off of SMF's Scheduled tasks - Fake Cron if you will.

You *could* create your own cron if you wanted by writing a simple PHP script that included SSI.php, Sources/ScheduledTasks.php, then ran the scheduled_rss_feeds() function.

It will import anything in the feed based on what you specify in the settings for that feed.  Want to import the entire feed, set the number to import to 0.  Want to import only feeds with certain keywords, put in the keywords... or any combination of both.
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline KahneFan

  • SMF Hero
  • ******
  • Posts: 1,850
  • Gender: Male
    • I Post You Post
Re: RSS Feeder
« Reply #14 on: January 11, 2009, 03:34:08 PM »
Do you know how often SMF's tasks run? I ask for 2 reasons. 1) 24 hours would be minimum (I would think). 2) On another site I had with feeds, I had so many that 15 minutes was actually too far apart.

Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,280
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
    • ZubaFitness
Re: RSS Feeder
« Reply #15 on: January 11, 2009, 03:36:14 PM »
You can go to the scheduled tasks part of SMF's admin panel and specify how often you want any of your scheduled tasks to run, including this one.  I set the default to every 6 hours.
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline Bigguy

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 12,594
  • Gender: Male
  • Be nice, or else....
    • smfbigguy on GitHub
    • Whats Ur Beef
Re: RSS Feeder
« Reply #16 on: January 11, 2009, 03:37:41 PM »
I have it running here and it works great, thanks SlammedDime.

http://forum.pctweakr.com/index.php/board,3.0.html

Offline SlammedDime

  • SMF Friend
  • SMF Super Hero
  • *
  • Posts: 10,280
  • School of Air Ride
    • matt.zuba on Facebook
    • @mattzuba on Twitter
    • ZubaFitness
Re: RSS Feeder
« Reply #17 on: January 11, 2009, 03:39:31 PM »
Thanks for posting that Bigguy.
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Offline KahneFan

  • SMF Hero
  • ******
  • Posts: 1,850
  • Gender: Male
    • I Post You Post
Re: RSS Feeder
« Reply #18 on: January 11, 2009, 03:41:13 PM »
You can go to the scheduled tasks part of SMF's admin panel and specify how often you want any of your scheduled tasks to run, including this one.  I set the default to every 6 hours.

Learn something new every day. Thanks!

Offline KahneFan

  • SMF Hero
  • ******
  • Posts: 1,850
  • Gender: Male
    • I Post You Post
Re: RSS Feeder
« Reply #19 on: January 11, 2009, 03:43:55 PM »
"Ran Now" and works like a charm. This is great, thanks!