RSS Feed Poster

[chuck23322]

I have enabled this MOD and enabled a feed but it does not seem to be importing all the data in the feed.   

Refer to the attached images.

The first image shows the feed that I have enabled.  It is setup to update every 5 minutes and to import 10 items each update. 
The second image shows that board that the feed is posting to.  From this image, you can see that it is importing 2 items every 6 hours. 

Can anyone suggest what could be causing this?

1) How are you running the RSS feeder -- eg, have you set up "true" Cron jobs,  or are you using the "fake" cron job configuration?

a) If using true cron jobs,  see if they really are running on whatever frequency you specified.   Eg, if you cron every hour, but you set the feed to every 15 minutes,  it's really every hour...
b) If using 'fake' cron jobs, remember, it only runs when somebody visits your site - so even if you said 15 minutes,  if nobody visited for 5 hours, the next run would really be 5 hours later...

{The idea for both is that the time you specify between runs is really more of -- "if xx time has expired since the last time cronrss.php has been called, then go and update this RSS}

2) When you enter the RSS (the http://) im a regular web browser -- do you see the updates there?   Until the actual HTTP// changes, there's nothing new to 'fetch'

3) Is it possible that your feed only had two updates each time?   Eg, are you just seeing the 'reality' of what is actually changing?


For testing, you might want to set up a dummy board/forum and pull something really high-volume into it.   Earlier in this thread,  I suggested using the USGS Earthquake RSS and/or the Wikipedia topic updates RSS to accomplish it.   That would give you an indication that you've got cron running right, and the # of posts that can be pulled is working, etc...   you can always "disable" the test feeds at any time....    Read earlier in this thread for the URLs to those feeds...

[mageguild]

There are more updates available in the RSS feed.  The RSS feed itself is out of date and has only imported a few of the objects in the RSS feed. 

I am using the Fake Cron method.  How would I implement a 'true' cron to import my RSS feeds? 

Not sure if this helps, but when I try to access the CronRSS URL (http://www.tennis-chat.com/cronrss.php), I get this error:

Code Select Expand

Fatal error: Cannot redeclare ssi_shutdown() (previously declared in /home/tbassett/webapps/tennischat/SSI.php:197) in /home/tbassett/webapps/tennischat/SSI.php on line 199

I have not made any changes to the SSI.php file and I have restored a new version of the SSI.php file.  This has not helped. 

I have enabled this MOD and enabled a feed but it does not seem to be importing all the data in the feed.   

Refer to the attached images.

The first image shows the feed that I have enabled.  It is setup to update every 5 minutes and to import 10 items each update. 
The second image shows that board that the feed is posting to.  From this image, you can see that it is importing 2 items every 6 hours. 

Can anyone suggest what could be causing this?

1) How are you running the RSS feeder -- eg, have you set up "true" Cron jobs,  or are you using the "fake" cron job configuration?

a) If using true cron jobs,  see if they really are running on whatever frequency you specified.   Eg, if you cron every hour, but you set the feed to every 15 minutes,  it's really every hour...
b) If using 'fake' cron jobs, remember, it only runs when somebody visits your site - so even if you said 15 minutes,  if nobody visited for 5 hours, the next run would really be 5 hours later...

{The idea for both is that the time you specify between runs is really more of -- "if xx time has expired since the last time cronrss.php has been called, then go and update this RSS}

2) When you enter the RSS (the http://) im a regular web browser -- do you see the updates there?   Until the actual HTTP// changes, there's nothing new to 'fetch'

3) Is it possible that your feed only had two updates each time?   Eg, are you just seeing the 'reality' of what is actually changing?


For testing, you might want to set up a dummy board/forum and pull something really high-volume into it.   Earlier in this thread,  I suggested using the USGS Earthquake RSS and/or the Wikipedia topic updates RSS to accomplish it.   That would give you an indication that you've got cron running right, and the # of posts that can be pulled is working, etc...   you can always "disable" the test feeds at any time....    Read earlier in this thread for the URLs to those feeds...

[graaywolf]

The mod is working fine except it don't post the latest feed it only posts the second to the last post from the sites... How would i configure it to post the latest post from the other sites?

{Make sure, first, that you are saying in the feed config, that you want more than 1 post to be pulled each time...  The system will not double-post -- it keeps a MD5 hash in the MySQL table of every post to make sure it doesn't double post -- so even if you say post 10 each time, if 9 of them have been posted before, it will post only the new 1, because there is no MD5 match to previous posts -- all my feeds say pull 15 posts -- I think that's the max}

Did that it's posting 3 at a time... but once it catches up to the second to the last posts it stops, and doesn't post till another post is made then it posts the second to the last on only.

[chuck23322]

The mod is working fine except it don't post the latest feed it only posts the second to the last post from the sites... How would i configure it to post the latest post from the other sites?

{Make sure, first, that you are saying in the feed config, that you want more than 1 post to be pulled each time...  The system will not double-post -- it keeps a MD5 hash in the MySQL table of every post to make sure it doesn't double post -- so even if you say post 10 each time, if 9 of them have been posted before, it will post only the new 1, because there is no MD5 match to previous posts -- all my feeds say pull 15 posts -- I think that's the max}

Did that it's posting 3 at a time... but once it catches up to the second to the last posts it stops, and doesn't post till another post is made then it posts the second to the last on only.

Yup.  Seen that as well on some of my feeds.  I don't have an answer or solution.  My fix, isn't a fix -- it is a really bad hack.   I use Yahoo pipes and their "reverse" operator to reverse the order of the feed.   Then the item being missed isn't the most recent one.   

[Biology Forums]

Could someone please tell me the code I need to input to get this load automatically twice a day at 12:00 AM/PM using a cron job?

I'm using puTTY to configure my server.

[IchBin™]

Explanation of cron:

# Example of job definition:
# .---------------- minute (0 - 59)
# |  .------------- hour (0 - 23)
# |  |  .---------- day of month (1 - 31)
# |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
# |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# |  |  |  |  |
# *  *  *  *  * command to be executed

I don't know what the file is for this mod but let's say it's call cron_rss.php

You'd then setup the cronjob command like this:
15 11,23 * * * /path/to/php_executable -f /path/to/your/cron_rss.php

This would run at 11:15 and 23:15 every day, month, and day of week.

[chuck23322]

Explanation of cron:

# Example of job definition:
# .---------------- minute (0 - 59)
# |  .------------- hour (0 - 23)
# |  |  .---------- day of month (1 - 31)
# |  |  |  .------- month (1 - 12) OR jan,feb,mar,apr ...
# |  |  |  |  .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# |  |  |  |  |
# *  *  *  *  * command to be executed

I don't know what the file is for this mod but let's say it's call cron_rss.php

You'd then setup the cronjob command like this:
15 11,23 * * * /path/to/your/cron_rss.php

This would run at 11:15 and 23:15 every day, month, and day of week.

I know on my server, I had to do a->>>>    wget -q /path/to/cron_rss.php

There's no on correct answer on the command itself -- this thread has several permutations -- depends on the hosting provider, etc.

[IchBin™]

bah... that reminded me I left out part of the command. php needs to run it, so php -f /path/to/cron_rss.php needs to be hit that way instead. wget doesn't make sense to me though chuck. lol I guess it depends on the host.

[chuck23322]

bah... that reminded me I left out part of the command. php needs to run it, so php -f /path/to/cron_rss.php needs to be hit that way instead. wget doesn't make sense to me though chuck. lol I guess it depends on the host.

Yea, I know.  wget is how I had to get it working -- wget acts like a "browser" to the page, and somehow PHP kicks in.   That's why I was alluding to there's "more than one way to skin the cat" -- the wget example, I got earlier in this thread.  And it worked for me.

[lifeguard81]

I install this mod, but after 12 o clock, it doenst do anything....
i set update every 60 minutes, but there are no new posts.

also, i create a new user, and group for this mod, but the postcount of this user doesnt work. it is alway 0 after post 44 messages


What can i do?

Excuse me for my english, i am from holland.

[chuck23322]

I install this mod, but after 12 o clock, it doenst do anything....
i set update every 60 minutes, but there are no new posts.

also, i create a new user, and group for this mod, but the postcount of this user doesnt work. it is alway 0 after post 44 messages


What can i do?

Excuse me for my english, i am from holland.

My "user" never shows anything but 0.  I think that is normal.

You haven't told us if you are using "cron", "fake cron", or manually running the http://forumname/forum/cron_rss.php

Earlier here I give some examples of RSS feeds (like the USGS or even WIKIPEDIA changes) -- they are helpful in getting things up and running since they are feeds with frequent changes...   Once you get your testing done, then you can move on to lower-volume feeds.


If fake cron, it will only run when users come into your forum _AND_ the time has passed for the next run.

If using cron, (preferred),  have you set up the cron to "email" you so you can see any errors, etc?   Are you sure your cron times are set correct?

If you are using a "scheduled task" -- then look at the task log.

Sadly,  the HOW to set up cron is different all over the place -- so you will need to read much in this thread and try many different solutions if necessary.

[lifeguard81]

Ok, chuck. Thanks for your answer.

But i cannot understand, why its working for a hole day, and after couple of hours it stops working

[Biology Forums]

For some reason, apostrophes are coming up as question marks, how do I change this?

[chuck23322]

For some reason, apostrophes are coming up as question marks, how do I change this?

Possibly the character set being used by the add-on vs. the database character set.  Look through the threads about changing that.

I'm betting you may be seeing the "back-quote" and "forward-quote" vs. a regular (simple vertical) quote mark -- the special ones convert poorly when there are differences in character sets involved.

Eg there are quotes that look like:   \   others that look like /   -- and then there's the regular quote something like    |

[OldShaterhan]

I have another problem - a modification does not receive the newest entry only to the entry before it. For example - I have posts: 5, 4, 3, 2, 1, but on the forum there are only 4, 3, 2, 1. What to do?

[chuck23322]

I have another problem - a modification does not receive the newest entry only to the entry before it. For example - I have posts: 5, 4, 3, 2, 1, but on the forum there are only 4, 3, 2, 1. What to do?

I don't know why the mod does that on some feeds and not others.   My hack was to feed the "feed" into Yahoo pipes and use their "reverse" operator to change the order of the feed -- and that fixed it.   At least the one dropped was the "oldest" item, not the newest.

{And, FYI, #5 will appear when #6 comes in, etc...}

http://www.simplemachines.org/community/index.php?topic=141776.msg3350405#msg3350405

[chovy]

I tried installing this. Feed was recognized but when I hit the cronjob url in my browser, nothing happens. I can't seem to get it to post anything.

[chuck23322]

I tried installing this. Feed was recognized but when I hit the cronjob url in my browser, nothing happens. I can't seem to get it to post anything.

Four questions:

1) When you "hit the cronjob url in your browser" -- do you see "Feed Poster Configuration" ?  {Log out of your forum, and do the test again}
2) Have you tried manually running it in scheduled tasks?
3) Do you allow "guests" to use your forum?  Or must every member be registered and have a password?
4) Have you tried more than one feed?  --  Say, try the USGS feed {into a dummy sub-board} -- http://earthquake.usgs.gov/earthquakes/catalogs/eqs1hour-M0.xml

[chovy]

Four questions:

1) When you "hit the cronjob url in your browser" -- do you see "Feed Poster Configuration" ?  {Log out of your forum, and do the test again}
2) Have you tried manually running it in scheduled tasks?
3) Do you allow "guests" to use your forum?  Or must every member be registered and have a password?
4) Have you tried more than one feed?  --  Say, try the USGS feed {into a dummy sub-board} -- http://earthquake.usgs.gov/earthquakes/catalogs/eqs1hour-M0.xml

1) Yes, I see "Feed Poster Configuration" when I hit the cron url.
2) I did not try this, I only tried fake cron. That didn't work either.
3) No, I do not allow guests. However I did enter a username to post as in the feed configuration
4) No I have only tried one feed.

[chuck23322]

3) No, I do not allow guests. However I did enter a username to post as in the feed configuration

If you don't support guests (which I didn't, and it drove me crazy) -- see this info on the modification you must do.

http://www.simplemachines.org/community/index.php?topic=141776.msg3333090#msg3333090