[Tip and Trick] Finding out referrer of the users coming to your site

Started by SwapsRulez, February 01, 2009, 05:46:35 AM

Previous topic - Next topic

SwapsRulez

Hello guys, I've written a simple code which can check the referrer of the users coming to your site. It will find out the referrers of the users. But I have made it so that the file won't be huge and it will still track perfect referrers. For that I've done following things.

1. Removed bots, slurp, crawlers, spiders from the referrer's list.
2. Removed the referrers which are coming from your own site
3. Removed the users who don't have referrers (They know your site  :P)

And The referrer file will display the following information,

Referrer : http://search.live.com/results.aspx?q=project-bb.org&sourceid=Mozilla-search&form=CHROME
User Agent : mozilla/5.0 (windows; u; windows nt 6.1; en-us; rv:1.9.0.5) gecko/2008120122 firefox/3.0.5
IP Address : http://whois.domaintools.com/127.0.0.1
Date and Time : Sunday 01st February 2009 04:59:45 AM

So you have to follow the just simple steps. If you have installed custom actions mod, it can be improvised. But for the time being, I'm giving instructions for normal users.

Steps :

1. In the first step, you have to find out the index.template.php file. This file is in the \Themes\Default folder. If you are using custom theme other than default, then go browse to \Themes\ThemeName and then edit the index.template.php file.

Make sure to make backup of file first.

2. You have to put the following code anywhere in the body tag. I've written it inside function template_body_above().
Just search for template_main_above() and inside that put the following code.

/* Getting the referrer information and saving it to text file */
$referrer=strtolower(getenv('HTTP_REFERER'));
$agent=strtolower($_SERVER['HTTP_USER_AGENT']);
if (empty($referrer) || ereg("slurp", $agent) || ereg("bot", $agent) || ereg("spider", $agent) || ereg("crawler", $agent) || ereg("http://www.project-bb.org/", $referrer)) {
}
else{
$message = "\n\nReferrer : " . getenv('HTTP_REFERER') . "\nUser Agent : ". strtolower($_SERVER['HTTP_USER_AGENT']) . "\nIP Address : http://whois.domaintools.com/" . getenv('REMOTE_ADDR') . "\nDate and Time : " . date("l dS F Y h:i:s A"). "\n";
$fp=fopen("referrer.txt", 'a' );
fwrite($fp, $message);
fclose($fp);
}
/* End of referrer information retrieval code */


3. You need to edit the code to match the information of your site. Replace http://www.project-bb.org/ with your site name and also replace referrer.txt with the filename you want to save the referrer information.

4. Now save the file and check if site works. If the site works, then everything is working fine and function is writing the file. If not the check for the syntax errors or file permissions.

5. Now the basic functionality is achieved. The log will be saved in the file referrer.txt or the filename you've specified. Just browse that file from the browser to see the referrer information.

go to this URL : http://www.yoursitename.com/referrer.txt  or http://www.yoursitename.com/referrerfilename.txt

If there is no referrer information found in the file. Let's make a new entry in it. Just open google and put your site name. open the result of your site from the google. You'll get entry added to that file.

6. That's the end of it.  :D

If you have custom actions mod installed, you can create custom action to check the referrer file instead of typing URL each time.

Just go to the custom action, create new action with type php.
Set proper permissions.

And paste the following code in the Template Code box:

To check the referrer file

echo '<a href="URL OF THE ACTION TO CLEAR THE REFERRER FILE"><b>Delete Administrative Referrer File</b></a>';

$fp = fopen ('referrer.txt', 'r');
$contents = fread ($fp, filesize ('referrer.txt'));
fclose ($fp);
$contents = str_replace ("\n", "<br />", $contents);
echo $contents;


To clear the referrer file

unlink('referrer.txt');
echo '<a href="URL OF THE CHECK REFERRER FILE CUSTOM ACTION"><b>Go Back to Administrative Referrer File</b></a>';
echo '<br /><a href="URL OF THE SITE"><b>Go Back to Project-BB.org</b></a>';


Also remember to replace the name of your site, custom referrer file name and URL's of the custom action accordingly.

I recommend to give something other name for privacy reasons.  ;D

Replies, comments are welcome.

Thanks to Tanchannel, Erry, Triston and Eddie in writing this code. :)

SMF has given me a lot for free, its just a simple try to give something in return. Hope you understand.  :)

PS: I will be happy if this moved to Tips and Tricks section.
Project-BB.org : Educational Forum For Engineering, Diploma & Technical Students

The Engineering, Diploma & All technical students lounge for Free Projects, Seminars, Syllabus, Question Papers, College Assignments, Placement Papers, E-Books, Company Information & other technical stuffs.

HR

NOw.. if you had this packaged as a mod with a quick delete option and what not Id be delighted.

I can explain this as simply as possible.. If I do it & implement I guarantee it.
If I do it and you implement it its a crap shoot.

SwapsRulez

I don't know much about creating mods. Also If someone wants to make a mod of this code. They are more than welcome. :)
Project-BB.org : Educational Forum For Engineering, Diploma & Technical Students

The Engineering, Diploma & All technical students lounge for Free Projects, Seminars, Syllabus, Question Papers, College Assignments, Placement Papers, E-Books, Company Information & other technical stuffs.

weisus

Thanks for the code! This should help out a great deal with my advertising efforts, thank you for sharing it.

SwapsRulez

Quote from: weisus on February 10, 2009, 12:29:50 PM
Thanks for the code! This should help out a great deal with my advertising efforts, thank you for sharing it.

Atleast someone is there, who is getting help from my code. :)
You're welcome bud. :D
Project-BB.org : Educational Forum For Engineering, Diploma & Technical Students

The Engineering, Diploma & All technical students lounge for Free Projects, Seminars, Syllabus, Question Papers, College Assignments, Placement Papers, E-Books, Company Information & other technical stuffs.

NDFF

Works great, allthough you should change the explanation. There is no template_body_above(), it's called template_main_above()

SwapsRulez

Quote from: NDFF on February 22, 2009, 06:33:30 AM
Works great, allthough you should change the explanation. There is no template_body_above(), it's called template_main_above()

Ohh sorry then. Because I'm using custom theme that's why. :P
Project-BB.org : Educational Forum For Engineering, Diploma & Technical Students

The Engineering, Diploma & All technical students lounge for Free Projects, Seminars, Syllabus, Question Papers, College Assignments, Placement Papers, E-Books, Company Information & other technical stuffs.


HR

Tweaking... 2.0b4

./Themes/default/languages/Modifications.english.php
Find:

?>

Add Before that:

// Additional Logging
$txt['enableLogReferral'] = 'Enable referrals logging?';
$txt['enableLogReferralDesc'] = 'Additional logging to see where users are coming from';
$txt['LogReferralPath'] = 'Path to store referral logs:<br><a href="' . $modSettings['LogReferralPath'] . '">°Download</a>';
$txt['LogReferralPathDesc'] = 'Using forum root as point of reference <i>ex. Logs/filename</i>';


./Sources/ManageSettings.php
Find:

            array('check', 'enableErrorLogging'),
            array('check', 'enableErrorQueryLogging'),
            array('check', 'securityDisable'),
        '',

Add After that:

            array('check', 'enableLogReferral', 'subtext' => $txt['enableLogReferralDesc']),
            array('text', 'LogReferralPath', 'subtext' => $txt['LogReferralPathDesc']),
        '',


This gives us a checkbox to enable/disable and a link to download in Security & Moderation.
the mod additions to index.template can then use

if (!empty($modSettings['enableLogReferral']) && !empty($modSettings['LogReferralPath']))

So it can be turned on/off, of course for it to work it will require a path as well as activation.. the above template addition will need altered to use $modSettings['LogReferralPath'] rather than a filename..

But Im just talking off the top of my head.

I can explain this as simply as possible.. If I do it & implement I guarantee it.
If I do it and you implement it its a crap shoot.

Dzonny


John Sivags

May i have a chance to disturb you guys please ? lol

I am sorry for my ignorance. I am new for SMF. So, still i don't know about HTML page in ACP. Where i have to go to look on it, and set codes ?

And this post is great at me. I want it. But, don't know where (in ACP) to put them lol

Please, if you guys don't mind, can you tell me, where to go and put codes ?

Oh, and my forum's present version is 1.1.7 :(

Trying to upgrade it too.

Dzonny

John, you have to add this code to index.tempalte.php of theme that u're using, somewhere before or after template_main_above().
/* Getting the referrer information and saving it to text file */
$referrer=strtolower(getenv('HTTP_REFERER'));
$agent=strtolower($_SERVER['HTTP_USER_AGENT']);
if (empty($referrer) || ereg("slurp", $agent) || ereg("bot", $agent) || ereg("spider", $agent) || ereg("crawler", $agent) || ereg("http://www.project-bb.org/", $referrer)) {
}
else{
$message = "\n\nReferrer : " . getenv('HTTP_REFERER') . "\nUser Agent : ". strtolower($_SERVER['HTTP_USER_AGENT']) . "\nIP Address : http://whois.domaintools.com/" . getenv('REMOTE_ADDR') . "\nDate and Time : " . date("l dS F Y h:i:s A"). "\n";
$fp=fopen("referrer.txt", 'a' );
fwrite($fp, $message);
fclose($fp);
}
/* End of referrer information retrieval code */


Just rename project bb with adress of your forum.
And then browse www.blah.com/forumdir/referrer.txt

SwapsRulez

HR, any chance of making one more setting to read the file from admin panel itself. I mean there will be a tab in modification section where you'll have a setting to turn it on or off. Also you can set 2 values. First one which you want to exclude from referrer and second one you want to exclude from agent. And those values will be set in comma seperated form. And then there will be a link to check the file. :)

Just a though, I know you know a lot in SMF. I just looked at your coding, you rock buddy. thanks. :)
Project-BB.org : Educational Forum For Engineering, Diploma & Technical Students

The Engineering, Diploma & All technical students lounge for Free Projects, Seminars, Syllabus, Question Papers, College Assignments, Placement Papers, E-Books, Company Information & other technical stuffs.

Tyrsson

PM at your own risk, some I answer, if they are interesting, some I ignore.

Tiribulus

Dude, this is like unspeakably kick@$$. Works like a charm and took 2 minutes to set up. For some reason adding the code after template_main_above() broke my site in a hideous way, but adding it before works jist peachy.

HR

Quote from: Tyrsson on April 25, 2009, 03:10:10 AM
@HR

You going to package this up?

I dot actually do packages however I might be able to follow the suggestion above so that it could be accessed from the admin area.. ATM IM still using Beta 4 so let me get up to RC1 and see what I can do with it.. might be able to go so far as to expand the internal logging features to include the reffed. Been awhile sine I had this one in my head so I need to freshen up.

I can explain this as simply as possible.. If I do it & implement I guarantee it.
If I do it and you implement it its a crap shoot.

MikeJones


skydiamond


- danny

can this be modified and placed in a block, visible to all users?


Advertisement: