RSS Feeder

Started by SlammedDime, January 11, 2009, 06:06:42 AM

Previous topic - Next topic

SlammedDime

I'm waiting to hear from the dev's on the utf8 table creation issue, but until that point, i've released 1.1.0, it's available for download on the mods download page.

Anyone currently running 1.0.0, just upload this and click the upgrade option and you're existing installation will be upgraded.
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

KahneFan

I have this on two sites I'm setting up. On 1 site I tried the upgrade and it didn't work, so I tried uninstalling old, installig new... got a few errors on install, but it appeared to go through. After Install I ran it and it uploaded a couple of articles again that it had loaded previously. I will monitor this and my error log. As long as I don't see anymore duplicates I'm OK with it.

On the other board, I fully uninstalled 1.0 and inistalled 1.1 and now I get this message on the RSS Admin page

Quote
Unknown column 'f.icon' in 'field list'
File: /home/content/b/u/d/budaadmin/html/sites/pickaline/Sources/ScheduledTasks.php
Line: 1611

On a side note: It pulls images through PERFECTLY! and I like the date/source additions.

Anymore thought on videos? :P  (what a pain in the @$$ I am :))
#eric_on_twit| (video) Creating Themes for SMF | Reset Your Forum
NOTE: No PM's for support please.

SlammedDime

if you uninstalled 1.0, it didn't remove the tables from the database, so your old tables are still there.  Try running upgrade.php from the download package and it should upgrade your tables for 1.1.

To verify this before hand, look at the rssfeeds table.  If you're missing an 'icon', or 'id_topic' row, the new structure is not there and upgrade.php should be run.
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

bloodfire

So far the new update works fine for me by uninstalling the old one and manually delete the database... the only thing is the images are not gathered... is it implemented? I have not fully tested it yet.. but so far getting rss from moreover.com [nofollow] with full article features... works fine and no more dump cores or time out... will try them in full capacity and c how far can it goes...

Good work with the updates...

SlammedDime

The only thing that I worked on images with was if they are in the description, or summary, or if they are contained in the full article.  If the feed uses an <img> xml tag, I didn't implement that.  To be honest, I wasn't even thinking about that.  I'll keep it in mind for future versions.
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

bloodfire

I actually meant when the full article that is to be retrieve has <img>(s) in it not the feed with img tag in xml. :)

digger

Hi.
"Post Items In Single Topic" work perfect! Thanks. And I think no need to show "Source: ... " in every post in this case.
There is one problem with mod. In all posts tag <br> removed, and posts looks not so good.

digger

Another problem.
Posts looks good when i start update task manually.
When it starts automatically Cyrillic characters looks like ?????????

Sudhakar Arjunan

Quote from: bloodfire on January 16, 2009, 12:34:56 AM
Another thing I found... since using this mod.... I found core dump files in my server....

core.12437
core.24225
core.9285
core.6443

Does any1 getting these files? At first, I just delete them, ignoring it. Only just now, when I test this mod by 'run now' on the schedule tasks and abt 1 minute or so, the page shows 500 Internal Service page. Then I try to retrieve the error.log using ftp from my server to find wat other errors... my server seems to be timeout as well... then i reconnect and found the core file... however, no error was shown.

I do have many core dumps, and i have enable full article mode and not upgraded to latest version.
Working on New Mods & Themes for SMF... Will update soon... My Blog page
My Smf forum : Discuss ITAcumens :: My SMF Forum

bloodfire

#129
Quote from: A.SK on January 17, 2009, 07:40:54 AM
Quote from: bloodfire on January 16, 2009, 12:34:56 AM
Another thing I found... since using this mod.... I found core dump files in my server....

core.12437
core.24225
core.9285
core.6443

Does any1 getting these files? At first, I just delete them, ignoring it. Only just now, when I test this mod by 'run now' on the schedule tasks and abt 1 minute or so, the page shows 500 Internal Service page. Then I try to retrieve the error.log using ftp from my server to find wat other errors... my server seems to be timeout as well... then i reconnect and found the core file... however, no error was shown.

I do have many core dumps, and i have enable full article mode and not upgraded to latest version.

My advice... Download the updated mod and you won't get the dump core... however, I still get the 500 internal server error... but i guessed the limit this mod can gather the rss feed but then it gathered all the news.. hmmmm... no error shown in the error log...

SlammedDime

Quote from: bloodfire on January 17, 2009, 05:13:49 AM
I actually meant when the full article that is to be retrieve has <img>(s) in it not the feed with img tag in xml. :)
Can you give me a specific article that has images that I can use to test?  In my tests it worked okay.

Quote from: digger on January 17, 2009, 06:06:00 AM
Hi.
"Post Items In Single Topic" work perfect! Thanks. And I think no need to show "Source: ... " in every post in this case.
There is one problem with mod. In all posts tag <br> removed, and posts looks not so good.
I thought I had fixed the <br> problem, UNLESS, the site is not using xhtml <br />s... and just using <br>.  I didn't think of that.  I'll try and account for that next version.  As for source in every post, one thing that the API I use is capable of is 'MultiFeeds', which I may implement in the mod, where you can specify multiple feeds that can all post to the same topic.  I'm still learning more about the API and seeing what it can do.

Quote from: digger on January 17, 2009, 07:21:37 AM
Another problem.
Posts looks good when i start update task manually.
When it starts automatically Cyrillic characters looks like ?????????
Does the actual post have ? in it?  Can you link me to a specific article this happens with?

Quote from: A.SK on January 17, 2009, 07:40:54 AM
Quote from: bloodfire on January 16, 2009, 12:34:56 AM
Another thing I found... since using this mod.... I found core dump files in my server....

core.12437
core.24225
core.9285
core.6443

Does any1 getting these files? At first, I just delete them, ignoring it. Only just now, when I test this mod by 'run now' on the schedule tasks and abt 1 minute or so, the page shows 500 Internal Service page. Then I try to retrieve the error.log using ftp from my server to find wat other errors... my server seems to be timeout as well... then i reconnect and found the core file... however, no error was shown.

I do have many core dumps, and i have enable full article mode and not upgraded to latest version.
As said earlier, if you're getting core dumps with this mod, it is normally because of an invalid, or ambiguous regular expression that causes PHP to crash.  I think the latest builds of PCRE are having some issues, I've seen a lot of core dumps lately from PCRE.
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

bloodfire

Feed URL:
http://www.bt.com.bn/en/front/feed

Regular Expression:
~<h1 class="title">(.*)printer friendly version~siU

Example of the news from the rss website:
http://www.bt.com.bn/en/home_news/2009/01/14/imported_items_may_pose_threat_to_brunei_eco_system [nofollow]

This is what it gathered in my website
http://www.bruneimarket.co.cc/index.php/topic,1617.0.html [nofollow]


bloodfire

#132
Quote from: bloodfire on January 17, 2009, 08:24:18 AM
Feed URL:
http://www.bt.com.bn/en/front/feed

Regular Expression:
~<h1 class="title">(.*)printer friendly version~siU

Example of the news from the rss website:
http://www.bt.com.bn/en/home_news/2009/01/14/imported_items_may_pose_threat_to_brunei_eco_system [nofollow]

This is what it gathered in my website
http://www.bruneimarket.co.cc/index.php/topic,1617.0.html [nofollow]



There's nothing wrong with the mod..it's getting the img... it's the url of the image in that the original website. The image is working fine.

My bad... I just try to modify the post to try to add the img manually... however... the img is in the post, it's the url of the img... the original url of the website have src of the image without it's full domain... the post is trying to find the img in my website...

In the post
[img]/en/files/images/photos/2009-01-14/13_envir.jpg[/img]

original from the website

<img src="/en/files/images/photos/2009-01-14/13_envir.jpg" alt="" title="" border="0" style="width:200px; height:auto;" />

now the problem is how to get the full url of the image heheehhe...

digger

Quote from: SlammedDime on January 17, 2009, 08:10:37 AM
Quote from: digger on January 17, 2009, 07:21:37 AM
Another problem.
Posts looks good when i start update task manually.
When it starts automatically Cyrillic characters looks like ?????????
Does the actual post have ? in it?  Can you link me to a specific article this happens with?

http://bash.org.ru/rss

SlammedDime

Quite odd... So if you run the task manually, the post turns out just fine, but if you let the scheduled task run as it should, then it goes crazy?
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Lamarck

#135
I don't know why,but in my forum (not the test site),It still cuts the text with spanish characters.But now,this only happens with post,not in the topic subject.

I uninstalled version 1.0.0 and deleted the data base tables (rss_feeds,log_rss and that colum in scheduled tasks) and then installed version 1.1.0.

It must be something related to the "upgrade",because I have no problems with spanish characters in my test site.I don't know what to do.

digger

Quote from: SlammedDime on January 17, 2009, 12:33:13 PM
Quite odd... So if you run the task manually, the post turns out just fine, but if you let the scheduled task run as it should, then it goes crazy?
yes

Lamarck

#137
I have tested more,and if i let the scheduled task run by itself,I get 1 post of 3 correctly (I have set to 3 posts every time),and the other 2 with the problem in spanish characters.Any ideas?

EDIT:If I run it manually,every post has the problem with the characters  >:(

SlammedDime

I'll have to do more testing on that... it's quite odd that sometimes it works, and others it doesn't.  Seems to be a similar problem as digger has....
SlammedDime
Former Lead Customizer
BitBucket Projects
GeekStorage.com Hosting
                      My Mods
SimpleSEF
Ajax Quick Reply
Sitemap
more...
                     

Smoky "Rider" Blue

hmm ok boys...

here is something to think on:

QuoteAuthor : Bryan (IP: 67.177.216.184 , c-67-177-216-184.hsd1.co.comcast.net)
E-mail : [email protected]
URL    : http://www.grandmasterb.com
Whois  : http://ws.arin.net/cgi-bin/whois.pl?queryinput=67.177.216.184
Comment:
This is a Notice of Infringement as authorized in § 512(c) of the U.S. Copyright Law under the Digital Millennium Copyright Act (DMCA).
This article is an unauthorized reproduction of copyrighted material originally found at: GrandmasterB dot com (http://www.grandmasterb.com)
Please remove this article and others originally published by GrandmasterB dot com immediately or we will file an official complaint with the U.S. Copyright Office, FeedBurner and Google, Inc.  Google's response may include removing or disabling access to material claimed to be the subject of infringing activity and/or terminating subscribers.
Thank you,
Bryan Davis

i got this nice notice from this swell guy that called me a "scrapper" wth??  :o

is this going to be a serious issue or?? just wondering..  was using rss feed poster.. so.. i dont know..
**Take the time to remember friendships and family.. Sometimes it's all we have, and missed very much**

Advertisement: