Auto Link Checker

[Dzonny]

mmmh... import the next code to remove the "junk". (actually, it's an uninstaller)

hm, got this error when run that code:

Error

SQL query:

ALTER TABLE pn_1462104_settings DROP COLUMN auto_link_checker_enabled;

MySQL said: Documentation
#1091 - Can't DROP 'auto_link_checker_enabled'; check that column/key exists

esttecb, yes i've change value from 100 to 1000 too...
Is that a problem? I will now change back to 100 to see if error still exist.

EDIT: It looks that problem is in value as esttecb noticed. When i set it to 100 mod works fine...
Can we increase this value somehow?

[esttecb]

mmmh... import the next code to remove the "junk". (actually, it's an uninstaller)

hm, got this error when run that code:

Error

SQL query:

ALTER TABLE pn_1462104_settings DROP COLUMN auto_link_checker_enabled;

MySQL said: Documentation
#1091 - Can't DROP 'auto_link_checker_enabled'; check that column/key exists

Well, there is not such a code like "DROP COLUMN IF EXISTS" (not that I know of), so if it doesn't exist u'll get that error (it's normal, nothing to worry about).
So if u try to import the tables now, u should not have any problem.

[Dzonny]

esttecb, thing about run this code again?

Code Select Expand

ALTER TABLE PREFIX_messages ADD checker_time INT(10) DEFAULT '0' NOT NULL;

CREATE TABLE PREFIX_log_link_checker_status (
ID_MSG int(10) UNSIGNED NOT NULL,
url varchar(255) NOT NULL,
life tinyint(1) DEFAULT '1' NOT NULL,
INDEX (ID_MSG)
);

INSERT INTO PREFIX_settings (variable, value) VALUE ('auto_link_checker_enabled', '1');
INSERT INTO PREFIX_settings (variable, value) VALUE ('auto_link_checker_time', '2');
INSERT INTO PREFIX_settings (variable, value) VALUE ('auto_link_checker_rapi', '1');
INSERT INTO PREFIX_settings (variable, value) VALUE ('auto_link_checker_mega', '1');
INSERT INTO PREFIX_settings (variable, value) VALUE ('auto_link_checker_file', '1');
INSERT INTO PREFIX_settings (variable, value) VALUE ('auto_link_checker_sendpm', '2');

[esttecb]

After run the ''remover code'', of course u have to import that code (one time only), other way the mod won't work properly (logs won't be saved).

As u already know, the issue was coz the change of value, so run the "remover code" was a mistake in first place.

[Dzonny]

Okay, i've done that and still same:

Error

SQL query:

ALTER TABLE pn_1462104_messages ADD checker_time INT( 10 ) DEFAULT '0' NOT NULL ;

MySQL said: Documentation
#1060 - Duplicate column name 'checker_time'

esttecb , do i need to do this thing couse we know what issue was (change of value) I've set value to 150 and it work fine now, but i want that value to be increased if it is possible...
Othervise, i can give you admin account data if you can / want / need / to do this for me.
Regards.

[Exelance]

Wrong;

Code Select Expand

"In this message [url=http://%1]%1[/url] haven been found dead links.\n\n

True;

Code Select Expand

"In this message [url=%1]%1[/url] haven been found dead links.\n\n



Example;

Wrong;

http://http:www.example.com

True;

http://www.example.com

[esttecb]

let see, if u  import this code:

Code Select Expand

ALTER TABLE pn_1462104_messages DROP COLUMN checker_time;

The 'checker_time' column will be removed (if it doesn't exist, u would get an error).

Then if u import this code:

Code Select Expand

ALTER TABLE pn_1462104_messages ADD checker_time INT(10) DEFAULT '0' NOT NULL;

The 'checker time' column will be created/added.

Now...

Error

SQL query:

ALTER TABLE pn_1462104_messages ADD checker_time INT( 10 ) DEFAULT '0' NOT NULL ;

MySQL said: Documentation
#1060 - Duplicate column name 'checker_time'

It means that u don't delete the 'checker_time' column before to import the add column code, this means u tried to import the add column code two times (the second time u get that error).

Whatever, it doesn't matter coz If u get "duplicate error" it means u already have the table/column in ur database.
So try to import the next part of the code (the next table)...
it would be:

Code Select Expand

CREATE TABLE pn_1462104_log_link_checker_status (
ID_MSG int(10) UNSIGNED NOT NULL,
url varchar(255) NOT NULL,
life tinyint(1) DEFAULT '1' NOT NULL,
INDEX (ID_MSG)
);

In case u get duplicate error, it's the same thing (doesn't matter)...
Then continue with the next table...

it would be:

Code Select Expand

INSERT INTO pn_1462104_settings (variable, value) VALUE ('auto_link_checker_enabled', '1');

and with the next one and so on...



About the link check increase... I'm not sure but I think that value means how many links u want to check at the time. If it's true, as M-DVD said the max is 100 by host (except megaupload).

is this mod checking more than 100 links by topic?

[ceewee]

...........
About the link check increase... I'm not sure but I think that value means how many links u want to check at the time. If it's true, as M-DVD said the max is 100 by host (except megaupload).

is this mod checking more than 100 links by topic?

I have only rapidshare at the moment, if I understand max. is 100 for rapidshare per msg?
So I have to make at least 30 msg from the present 8?

Because it didn't work I uninstalled the mod so after I changed the msgs to max. 100 per msg I can just install the mod again?

[esttecb]

I was talking about 100 links at a time...
With megaupload it's only checking 10 links at a time, but obviously if u have more than 10 links all will be checked. So I assume the link checker can check any amount of links by topic.

Someone asked if the mod can check more than 100 links at a time, and the answer is no, but that doesn't means it can check only 100 links (I guess), it'll check 100 by request. So if it have to check 200 links, it will make two requests (or remote query).

[Dzonny]

Someone asked if the mod can check more than 100 links at a time, and the answer is no, but that doesn't means it can check only 100 links, it'll check 100 by request. So if it have to check 200 links, it will make two requests (or remote query).

Okay, but why are we getting errors when set it to be more than 100?

[esttecb]

Because rapidshare and filefactory only allow to check 100 links (megaupload 10) at a time. So the mod will check 100 links... then other 100 links (in case u've more than 100 links)... and so on...

btw, I'm just guessing here.

[Dzonny]

No, i guess that this can't be the case.
I think that if there is more than 100 links we get eror like this:

There are over 100 links to a FileHost in this message page.
Please, delete some links, use other filehost or open other topic.

EDIT:
Hm..
In this topic I have more than 100 links, and they're all alilve, but it shows like it's dead!
(That is just in this topic as i noticed)
See attached image.

[marciano]

muy buen mod cheee .....

no hay posibilidad de que en vez de aparecer el error de mas de 100 links, al menos muestre el post normalmente sin chequear nada?

[ɔɔɔɔɔɔuɥoɾ]

Im having the same issue as Dzonny with the links been marked as dead when there not

any ideas anyone?

[esttecb]

No, i guess that this can't be the case.
I think that if there is more than 100 links we get eror like this:

There are over 100 links to a FileHost in this message page.
Please, delete some links, use other filehost or open other topic.

Actually, if u open the AutoLinkChecker-Subs.php and replace:

Code Select Expand

fatal_lang_error('auto_link_checker_error', false);

with:

Code Select Expand

fatal_lang_error('auto_link_checker_error', true);

u should get that error message.

Or maybe already is giving that error... I don't know... I don't have any topic with more than 100 links 

[M-DVD]

Reply 1

Hi M-DVD

Looks to be a useful mod I think 

Is it possible for us to be able to substitute one or more of the host sites ***.com that you have listed with one or more of our own within the code for our specific needs?

I have linked to many files on my website over the years but may have moved the files at some point. It would be helpful if I could use this mod to check these links in the various posts.

If this is possible can you suggest where in the code I should change or would I just have to experiment?

Many Thanks

To add a site, you must study this site and make some adjustments in the code, ie, there is no magic formula for adding other filehosts.

Moreover, if this filehost doesn't have a tool or an API, I don't recommend an automatic checks. (click)

--

@M-dvd.
Sorry for asking too much but.
Could you release a version of your script that deosnt have something to do with any forum?
If> could you make so the script will email you instead of pm?

1.- I am currently very very busy.

You can remember this in a couple of months and could do so. I have no problem in doing so, in fact, I encouraged the idea.

2.- Currently with the PMs can get a email notification.

Thanks.

--

Asking once again: How can I test this out?

It seemed to have installed properly but when I tried to post a bad link, the bad link showed up as is (as if there is no mod installed.)...ie - the mod does not appear to be working.

Have you edited the style.css of your custom theme?

Have you make hard refresh?

--

M-DVD, La parte de importar los datos a la base de datos esta mal en la guia de pasos a seguir del mod... al menos para las versiones 1.1.x... checa al final donde dice

Code
install_1.php

el codigo dentro del Query: esta erroneo... puede ser que por eso hayan varios con problemas... no checque si en los pasos de la version 2.0 tmb paso lo mismo.

¿Cuál guía de pasos? ¿aquí? http://custom.simplemachines.org/mods/index.php?action=parse

Si te refieres a eso, el contenido de los archivos php no se muestran completo, sea cual sea el MOD.

PD: El codigo para solucionar lo de megaupload funciona solo en algunos temas, algunos links si los checkea pero otros no, no se porq...

Tambien da falsos negativos (aparecen como negaticos esos links pero quizas si funcionan) con los temas ya indexados (se podria decir?  ).. si se cambia lo de recheckear a 0 funciona bien (en los temas en los que si checkea)... pregunta: si se pone en 0, vacia los logs de temas ya visitados?? si es asi se podria poner en cero y luego volver a 2... o simplemente vaciar esa tabla...
Pero no entiendo porque en algunos temas si los analisay otros no. Solo pasa con los links de megaupload con /es/... los de megaupload normales, rapidshare y filefactory si lo analisa siempre...

EDIT: En aquellos temas en los que noe sta chequeando el link porq tiene /es/ entre medio... si edito y guardo asi como esta el tema, si lo chequea.. 

Lo que ocurre es que cuando el mensaje fue chequeado esos links eran "invisibles" para el MOD. En el log solo fueron guardados como vivos los links encontrados y revisados como vivos.

Luego del cambio, los links que antes no se veían ahora se ven, pero como "no están vivos", entonces "están muertos"

Siempre que modificas un mensaje se re-revisa (ignorando el log; porque ahora puede haber algo nuevo). Por eso luego de modificar se corrige y ahora sí se ven bien los links de megaupload con /es/, etc.

I was wondering... what would happen if some server is down?.. I guess there is some kind of timeout?

IMPORTANT QUESTION:

If the server is down, in the moment that is checked, then the MOD aborts and goes off and makes no changes.

So don't get data false and the next time someone enters the topic will be checked (as if nobody had entered the first).

If at any time (eg, when posting a reply) you see the links are displayed normal (as if there is no MOD) and before are displayed green and red, is that for this reason. If you refresh (and the server is on again) it should work fine.

--

tengo una pequeña pregunta que tanto aumenta esto la carga de servidor?

mi foro es de enlaces y no se si sea  lo ideal instalarlo

Todo MOD implica una carga nueva. En este caso, la mayor carga está en la consulta remota.

Por esta razón he incluido un log para evitar hacer esta carga a cada momento que alguien visite el tema (o cada cierto tiempo como con cronjob), sino cada vez que sea necesario y que el tiempo del log haya expirado (*).

Por esta razón es que no pienso añadir otros filehost que no tengan una API/tools, porque revisar link por link es una carga enorme, que puede enlentecer la carga de la página hasta más de un minuto.

También he diseñado el MOD para que detecte y marque los mensajes que no son de links; y si la página del tema que es visitado es 100% de estos mensajes, el MOD evitará cargar.

(*) el log debe tener un tiempo de vida medio, como 1, 2 ó 3 días, no recomiendo cero porque entonces se revisará siempre, y no recomiendo 90 días porque los links mueren de un día a otro.

--

IMPORTANT QUESTION: How much it increases the weight on server?

Any MOD = new weight for the server. In this case, the greatest weight is in the remote check.

For this reason, I have included a log to avoid this burden that every time someone visits the topic (or with time as cronjob), but, whenever necessary and if the time log's have expired (*).

For this reason I don't plan on adding another filehost if not having an API/tools, because checks link by link is an enormous burden, that slow loading the page until more than a minute.

I have also designed the MOD to detect and mark the messages that not have links, and if the topic page have 100% messages marked as without links, the MOD will be not load.

(*) The log should have a time life relative, such as 1, 2 or 3 days. I don't recommend 0 because then always will be checks, and I don't recommend 90 days because the links are dying from day to day.

--

i keep getting
The package you tried to upload either is not a valid package or has become corrupted.

im running 1.1.9

Try deleted the packeage in your host.

Download again and upload and try Install.

If the problem persists, Then:
1.- Try manual installation.
2.- And calls for help in this board (isn't problem of this MOD).

--

Edit:
I've install this successfully on my forum, but on test forum (free host) it wont work....

In the free host: don't Installs? or don't works?

In the fourm host: yes installs? and yes works?

M-DVD: If you plan to develop this pod in fututre, can we give you some future ideas and requests here?

Yes not problem. But currently I cannot. In the future.

--

I have another useful idea, how about adding an option where we can choose which board(s) the link checker will scan for dead links, options could include, ALL boards, which categories, which boards.

I think that would make this already great mod that much greater, of course we can wait for future updates

If you want to avoid the weight caused by this MOD. You don't have much to worry about. You can read this post with more explanation.

But if you still want this feature, I can give you some simple manual steps to add it.

--

Do you think, is smf themed sites gives downloads .

I'm sorry, I don't understand. Can you repeat?

--

PS. is possible send pm for dead links  to moderator global too ?

Is possible, Also the Admin, etc.

But is necessary other query (for example: it should work).

For this reason I don't included this, and I left only the "local moderator" and the "post author".

I didn't want make heavy the MOD with an query "not indispensable", that is simple but can be heavy.

--

Wrong;

Code Select Expand

"In this message [url=http://%1]%1[/url] haven been found dead links.\n\n

True;

Code Select Expand

"In this message [url=%1]%1[/url] haven been found dead links.\n\n

Where? 

Currently is:

Code Select Expand

$txt['auto_link_checker_message'] = "In this message [url]%1[/url] haven been found dead links.\n\nYou can restore these links, close the topic, or delete the links.\n\nThanks.\n\nAuto Link Checker";

--

muy buen mod cheee .....

no hay posibilidad de que en vez de aparecer el error de mas de 100 links, al menos muestre el post normalmente sin chequear nada?

No porque entonces la gente pensará que el MOD no está funcionando con algunos links 

--

Im having the same issue as Dzonny with the links been marked as dead when there not

any ideas anyone?

If you have done the change in the 100 max links, read the next reply. These change cause errors.

[M-DVD]

Reply 2

is it possible to increase the link limit from 100 to 1000?

I'm also interested in this...

is it possible to increase the link limit from 100 to 1000?

change in AutoLinkChecker-Subs.php this :

Code Select Expand

if (count($linkCheker['recipiensA'][$f_host]) > 100)

to:

Code Select Expand

if (count($linkCheker['recipiensA'][$f_host]) > 1000)

PS. is possible send pm for dead links  to moderator global too ?

edi67, thanks for tip...

Sorry for bumping again but i have this error when i enable this mod and when go to topic that have some links in it:

You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 2
File: /home/samoo/public_html/forum/Sources/AutoLinkChecker-x1.php
Line: 127

Note: It appears that your database may require an upgrade. Your forum's files are currently at version SMF 1.1.9, while your database is at version 1.1.4. The above error might possibly go away if you execute the latest version of upgrade.php.

My Database version is at version 1.1.9 for shore. (I've runned connect.php and check it)
So, anyone has idea what can be problem?

So both of u guys have the same error... Dzonny, is this a coincidence or u tried to increase the link limit from 100 to 1000 too?

EDIT: It looks that problem is in value as esttecb noticed. When i set it to 100 mod works fine...
Can we increase this value somehow?

IMPORTANT:

All persons who have made the increase in the number of links, should do:

1.- Uninstalls the MOD.

2.- Via phpmyadmin. Delete the changes made by the MOD in the tables
     (delete the table {db_prefix}log_link_checker_status
      and the column checker_time in {db_prefix}messages)

3.- Reinstalls the MOD.

It is the only way to clean the false data by changing the maximum number of links

------------------------

About the link check increase... I'm not sure but I think that value means how many links u want to check at the time. If it's true, as M-DVD said the max is 100 by host (except megaupload).

is this mod checking more than 100 links by topic?

100 by filehost by message page.

Example 1:
- 100 links rapidshare's + 100 links filefactory's + 100 links megablah's = GOOD
- 101 links anyfilehost = BAD

Example 2:
In X topic:
- In the first page: 100 links rapidshare's = GOOD
- In the second page: other 100 links rapidshare's = GOOD


---

I was talking about 100 links at a time...
With megaupload it's only checking 10 links at a time, but obviously if u have more than 10 links all will be checked. So I assume the link checker can check any amount of links by topic.

Someone asked if the mod can check more than 100 links at a time, and the answer is no, but that doesn't means it can check only 100 links (I guess), it'll check 100 by request. So if it have to check 200 links, it will make two requests (or remote query).

Because rapidshare and filefactory only allow to check 100 links (megaupload 10) at a time. So the mod will check 100 links... then other 100 links (in case u've more than 100 links)... and so on...

btw, I'm just guessing here.

Yes, rapidshare and filefactory only allow to check 100 links.

Unfortunately, megaupload only allow 10 links by check.

I must offer the same functionality for all filehost, then, for to place megaupload at the same level as the others, I should do several checks.

But it has a price. Remote checks consume resources and consume much time.

Yes, is possible check 200, 300, 1000 links rapidshare's/filefactory's; and I can do two, three or ten remote checks, BUT with megablahblah will be x10. Then if have posted 1000 links megaupload's, will be ¡¡100!! remote checks, and the page will load many minutes or fails for timeout.

Then I have set a reasonable standard of 100 links, for all filehost currently supported. Lite and enough for most uses.

--

un saludo, y gracias otra vez por el mod 

Many Thanks

vaya tus mods siempre son inovadores

saludos y excelente mod

Maaan this is a cool mod!!
Good work M-DVD
I'll give it a few days before I install though, give you time to update
Cheers

Anyway, thanks for great mod! 

Thanks for filling my mod request M-DVD, I will try this, I have SMF 2 RC1, will I need to upgrade to 2 RC1-1?

EDIT

works perfect for me (no manual coding required) on my custom theme with SMF 2 RC1.

Thanks again for such a great mod.

Good Mod.

Another great mod, really nice buddy

marvelous mod, good job

Thanks for your comments 

[Groundhog]

To add a site, you must study this site and make some adjustments in the code, ie, there is no magic formula for adding other filehosts.

Thanks for feedback M-DVD 

[Sabre™]

Very good support mate
+k

[ceewee]

[quot

IMPORTANT:

All persons who have made the increase in the number of links, should do:

1.- Uninstalls the MOD.
2.- Via phpmyadmin. Delete the changes made by the MOD in the tables. (delete the table and the column)
3.- Reinstalls the MOD.

------------------------

I have 3000 rapidshares in 8 msgs, below 154 it worked fine, I have changed 100>1000>999 then 600 then 100, ten uninstalled, so applies to me.

Will make 30 msgs from those 8 and then reinstall after deleteling table/colum.

But which table/colum I have to delete (after backup Mysql) ?