Hello guys, I've written a simple code which can check the referrer of the users coming to your site. It will find out the referrers of the users. But I have made it so that the file won't be huge and it will still track perfect referrers. For that I've done following things.
1. Removed bots, slurp, crawlers, spiders from the referrer's list.
2. Removed the referrers which are coming from your own site
3. Removed the users who don't have referrers (They know your site :P)
And The referrer file will display the following information,
Referrer : http://search.live.com/results.aspx?q=project-bb.org&sourceid=Mozilla-search&form=CHROME
User Agent : mozilla/5.0 (windows; u; windows nt 6.1; en-us; rv:1.9.0.5) gecko/2008120122 firefox/3.0.5
IP Address : http://whois.domaintools.com/127.0.0.1
Date and Time : Sunday 01st February 2009 04:59:45 AM
So you have to follow the just simple steps. If you have installed custom actions mod, it can be improvised. But for the time being, I'm giving instructions for normal users.
Steps :
1. In the first step, you have to find out the index.template.php file. This file is in the \Themes\Default folder. If you are using custom theme other than default, then go browse to \Themes\ThemeName and then edit the index.template.php file.
Make sure to make backup of file first.
2. You have to put the following code anywhere in the body tag. I've written it inside function template_body_above().
Just search for template_main_above() and inside that put the following code.
/* Getting the referrer information and saving it to text file */
$referrer=strtolower(getenv('HTTP_REFERER'));
$agent=strtolower($_SERVER['HTTP_USER_AGENT']);
if (empty($referrer) || ereg("slurp", $agent) || ereg("bot", $agent) || ereg("spider", $agent) || ereg("crawler", $agent) || ereg("http://www.project-bb.org/", $referrer)) {
}
else{
$message = "\n\nReferrer : " . getenv('HTTP_REFERER') . "\nUser Agent : ". strtolower($_SERVER['HTTP_USER_AGENT']) . "\nIP Address : http://whois.domaintools.com/" . getenv('REMOTE_ADDR') . "\nDate and Time : " . date("l dS F Y h:i:s A"). "\n";
$fp=fopen("referrer.txt", 'a' );
fwrite($fp, $message);
fclose($fp);
}
/* End of referrer information retrieval code */
3. You need to edit the code to match the information of your site. Replace http://www.project-bb.org/ with your site name and also replace referrer.txt with the filename you want to save the referrer information.
4. Now save the file and check if site works. If the site works, then everything is working fine and function is writing the file. If not the check for the syntax errors or file permissions.
5. Now the basic functionality is achieved. The log will be saved in the file referrer.txt or the filename you've specified. Just browse that file from the browser to see the referrer information.
go to this URL : http://www.yoursitename.com/referrer.txt or http://www.yoursitename.com/referrerfilename.txt
If there is no referrer information found in the file. Let's make a new entry in it. Just open google and put your site name. open the result of your site from the google. You'll get entry added to that file.
6. That's the end of it. :D
If you have custom actions mod installed, you can create custom action to check the referrer file instead of typing URL each time.
Just go to the custom action, create new action with type php.
Set proper permissions.
And paste the following code in the Template Code box:
To check the referrer file
echo '<a href="URL OF THE ACTION TO CLEAR THE REFERRER FILE"><b>Delete Administrative Referrer File</b></a>';
$fp = fopen ('referrer.txt', 'r');
$contents = fread ($fp, filesize ('referrer.txt'));
fclose ($fp);
$contents = str_replace ("\n", "<br />", $contents);
echo $contents;
To clear the referrer file
unlink('referrer.txt');
echo '<a href="URL OF THE CHECK REFERRER FILE CUSTOM ACTION"><b>Go Back to Administrative Referrer File</b></a>';
echo '<br /><a href="URL OF THE SITE"><b>Go Back to Project-BB.org</b></a>';
Also remember to replace the name of your site, custom referrer file name and URL's of the custom action accordingly.
I recommend to give something other name for privacy reasons. ;D
Replies, comments are welcome.
Thanks to Tanchannel, Erry, Triston and Eddie in writing this code. :)
SMF has given me a lot for free, its just a simple try to give something in return. Hope you understand. :)
PS: I will be happy if this moved to Tips and Tricks section.
NOw.. if you had this packaged as a mod with a quick delete option and what not Id be delighted.
I don't know much about creating mods. Also If someone wants to make a mod of this code. They are more than welcome. :)
Thanks for the code! This should help out a great deal with my advertising efforts, thank you for sharing it.
Zitat von: weisus in Februar 10, 2009, 12:29:50 NACHMITTAGS
Thanks for the code! This should help out a great deal with my advertising efforts, thank you for sharing it.
Atleast someone is there, who is getting help from my code. :)
You're welcome bud. :D
Works great, allthough you should change the explanation. There is no template_body_above(), it's called template_main_above()
Zitat von: NDFF in Februar 22, 2009, 06:33:30 VORMITTAG
Works great, allthough you should change the explanation. There is no template_body_above(), it's called template_main_above()
Ohh sorry then. Because I'm using custom theme that's why. :P
thanks, works great!
;)
Tweaking... 2.0b4
./Themes/default/languages/Modifications.english.php
Find:
?>
Add Before that:
// Additional Logging
$txt['enableLogReferral'] = 'Enable referrals logging?';
$txt['enableLogReferralDesc'] = 'Additional logging to see where users are coming from';
$txt['LogReferralPath'] = 'Path to store referral logs:<br><a href="' . $modSettings['LogReferralPath'] . '">°Download</a>';
$txt['LogReferralPathDesc'] = 'Using forum root as point of reference <i>ex. Logs/filename</i>';
./Sources/ManageSettings.php
Find:
array('check', 'enableErrorLogging'),
array('check', 'enableErrorQueryLogging'),
array('check', 'securityDisable'),
'',
Add After that:
array('check', 'enableLogReferral', 'subtext' => $txt['enableLogReferralDesc']),
array('text', 'LogReferralPath', 'subtext' => $txt['LogReferralPathDesc']),
'',
This gives us a checkbox to enable/disable and a link to download in Security & Moderation.
the mod additions to index.template can then use
if (!empty($modSettings['enableLogReferral']) && !empty($modSettings['LogReferralPath']))
So it can be turned on/off, of course for it to work it will require a path as well as activation.. the above template addition will need altered to use $modSettings['LogReferralPath'] rather than a filename..
But Im just talking off the top of my head.
great thing, work excelent... :)
May i have a chance to disturb you guys please ? lol
I am sorry for my ignorance. I am new for SMF. So, still i don't know about HTML page in ACP. Where i have to go to look on it, and set codes ?
And this post is great at me. I want it. But, don't know where (in ACP) to put them lol
Please, if you guys don't mind, can you tell me, where to go and put codes ?
Oh, and my forum's present version is 1.1.7 :(
Trying to upgrade it too.
John, you have to add this code to index.tempalte.php of theme that u're using, somewhere before or after template_main_above().
/* Getting the referrer information and saving it to text file */
$referrer=strtolower(getenv('HTTP_REFERER'));
$agent=strtolower($_SERVER['HTTP_USER_AGENT']);
if (empty($referrer) || ereg("slurp", $agent) || ereg("bot", $agent) || ereg("spider", $agent) || ereg("crawler", $agent) || ereg("http://www.project-bb.org/", $referrer)) {
}
else{
$message = "\n\nReferrer : " . getenv('HTTP_REFERER') . "\nUser Agent : ". strtolower($_SERVER['HTTP_USER_AGENT']) . "\nIP Address : http://whois.domaintools.com/" . getenv('REMOTE_ADDR') . "\nDate and Time : " . date("l dS F Y h:i:s A"). "\n";
$fp=fopen("referrer.txt", 'a' );
fwrite($fp, $message);
fclose($fp);
}
/* End of referrer information retrieval code */
Just rename project bb with adress of your forum.
And then browse www.blah.com/forumdir/referrer.txt
HR, any chance of making one more setting to read the file from admin panel itself. I mean there will be a tab in modification section where you'll have a setting to turn it on or off. Also you can set 2 values. First one which you want to exclude from referrer and second one you want to exclude from agent. And those values will be set in comma seperated form. And then there will be a link to check the file. :)
Just a though, I know you know a lot in SMF. I just looked at your coding, you rock buddy. thanks. :)
@HR
You going to package this up?
Dude, this is like unspeakably kick@$$. (https://www.simplemachines.org/community/proxy.php?request=http%3A%2F%2Fgregnmary.gotdns.com%3A8080%2Fpix%2Ftwo_up.gif&hash=ae691750b60495090408fd0752e480158ba99dfd) Works like a charm and took 2 minutes to set up. For some reason adding the code after template_main_above() broke my site in a hideous way, but adding it before works jist peachy.
Zitat von: Tyrsson in April 25, 2009, 03:10:10 VORMITTAG
@HR
You going to package this up?
I dot actually do packages however I might be able to follow the suggestion above so that it could be accessed from the admin area.. ATM IM still using Beta 4 so let me get up to RC1 and see what I can do with it.. might be able to go so far as to expand the internal logging features to include the reffed. Been awhile sine I had this one in my head so I need to freshen up.
Sweet!!!!!!!!!! cant wait to implement this
I want to add these informations in this mod (http://www.simplemachines.org/community/index.php?topic=19243.0) for each user.
can this be modified and placed in a block, visible to all users?
anyone? :|
Alex: yes, it probably could but it's unfortunately not as simple as that. At a minimum which portal are you using?
Fancy running into Arantor in this thread too after he helped me out with what turned out to be this tweak in my thread.
He can explain this better than I can, it appears that the deprecated regular expression "ereg", which is used in this tweak can quickly tally up pages and pages of errors in your log.
See HERE (http://www.simplemachines.org/community/index.php?topic=342736.new;topicseen#new) for more info.
That's not actually the problem here. Adaptation to put it in a block is not trivial.
Zitat von: Arantor in Oktober 16, 2009, 07:50:38 NACHMITTAGS
That's not actually the problem here. Adaptation to put it in a block is not trivial.
Were you addressing this to me?
Indirectly, sorry. The last problem left in this thread was putting it into a portal block, which ereg probably won't be an issue with.
ereg being deprecated is only actually a problem on hosts that bother to update. Most users that run SMF are still running earlier 5.2 or 5.1, with a not inconsiderable percentage running 4.4 and 4.3, meaning that while this will become a problem, it probably won't be a huge problem for a while.
Zitat von: Arantor in Oktober 17, 2009, 06:56:55 NACHMITTAGS
Indirectly, sorry. The last problem left in this thread was putting it into a portal block, which ereg probably won't be an issue with.
ereg being deprecated is only actually a problem on hosts that bother to update. Most users that run SMF are still running earlier 5.2 or 5.1, with a not inconsiderable percentage running 4.4 and 4.3, meaning that while this will become a problem, it probably won't be a huge problem for a while.
Oh I see. I just was doing what Mr. Phil asked by putting the info from the other thread in here where the referrer thing got started. Sorry, I should have been clearer too. Nothing to do with the block question.
Zitat von: Arantor in Oktober 11, 2009, 06:00:35 VORMITTAG
Alex: yes, it probably could but it's unfortunately not as simple as that. At a minimum which portal are you using?
TinyPortal v0.9.8
I had to put in the code above the template_main_above for it to work, however, it also logs referrers which are coming from our own site. Also I replaced the ereg with preg_match. Could that be the cause? If not, any idea what could be?
(ereg gives me errors in the log: function ereg() is deprecated)
edit: Forgot to mention: I did escape the slashes with \/ but with or without, does not make a difference and every time its accompanied by the following error:
2: preg_match() [<a href='function.preg-match'>function.preg-match</a>]: Delimiter must not be alphanumeric or backslash
forumurl/Sources/Load.php(1999) : eval()'d code
64
Zitat von: JulesW in Oktober 30, 2009, 03:15:32 NACHMITTAGS
<<< I had to put in the code above the template_main_above for it to work, however, it also logs referrers which are coming from our own site. >>>
I also am this way. On both counts. It seems like it didn't used to log hits from my own site though at first. I think, if I remember correctly.
I said on the previous page:
ZitatFor some reason adding the code after template_main_above() broke my site in a hideous way, but adding it before works jist peachy.
Zitat von: MrPhil in Oktober 16, 2009, 05:36:48 NACHMITTAGS
Trivial.
if (empty($referrer) || preg_match("/slurp/", $agent) || preg_match("/bot/", $agent) || preg_match("/spider/", $agent) || preg_match("/crawler/", $agent) || preg_match("/http:\/\/gregnmary\.gotdns\.com:8080\//", $referrer)) {
I haven't tried it myself, but you may be able to simplify it to
if (empty($referrer) || preg_match("/slurp|bot|spider|crawler/", $agent) || preg_match("/http:\/\/gregnmary\.gotdns\.com:8080\//", $referrer)) {
Thanks Tiribulus, that solved my problem! I have been reading that other thread you quote, but I probably missed the fact you were talking about the refferer tip!
Also in case ppl are wondering. The if (empty($referrer) || preg_match("/slurp|bot|spider|crawler/", $agent) || preg_match("/http:\/\/gregnmary\.gotdns\.com:8080\//", $referrer)) {
works!
Zitat von: JulesW in November 03, 2009, 01:35:17 NACHMITTAGS
Thanks Tiribulus, that solved my problem! I have been reading that other thread you quote, but I probably missed the fact you were talking about the refferer tip!
Also in case ppl are wondering. The if (empty($referrer) || preg_match("/slurp|bot|spider|crawler/", $agent) || preg_match("/http:\/\/gregnmary\.gotdns\.com:8080\//", $referrer)) {
works!
No trouble.
Except people should take note to change the URL to their own. I should have said that too.
Does anybody know why I'm getting this error
http://gregnmary.gotdns.com:8080/forum1/index.php?action=admin;area=logs;sa=errorlog;desc;start=0
2: preg_match(): Unknown modifier '/'
File: /srv/www/htdocs/forum1/Themes/brightforest_22/index.template.php
Line: 72 Repeatedly?
Here is the Code: Lines 72 and 73:
if (empty($referrer) || preg_match("/slurp/", $agent) || preg_match("/bot/", $agent) || preg_match("/spider/", $agent) || preg_match("/crawler/", $agent) || preg_match("/http:\/\/gregnmary\.gotdns\.com:8080\forum1//", $referrer)) {
}Thank you very much
There's a / too many in the gregnmary.gotdns.com expression.
Thank you Arantor my old friend. Would it be that double // after forum1 at the end?
Also while were at it, did anybody ever figure out how to have this tweak not report internal hits from ip addresses on your own lan?
Thanks again
Yup; the last / marked the end of the regular expression, but the extra one before it wasn't escaped properly. Either escape the second to last one (as \//) or remove it, it wouldn't hurt to leave it as a single one there.
Thanks, this will be very useful in order to really understand your analytics and how to improve them!