News:

SMF 2.1.4 has been released! Take it for a spin! Read more.

Main Menu

Forum Firewall

Started by butchs, January 15, 2011, 11:00:37 AM

Previous topic - Next topic

An0nymousHelper

Oh ok, thanks for the quick response!

Glasso

Hi butchs,

Would you mind telling me if this is correct blocking? To my untrained eye, it appears to be genuine googlebot requests being blocked, as corroborated by my google webmaster logs. Thank you.


butchs

#262
So many bots pretend to be google and etc because the google UA is generally allowed a free pass most sites.  It is the most spoofed ip and UA in the net.

2899 is from a mobile phone and as far as I know Google is not using a mobile phone.

Why would google need to find your Sources directory?  They should not be snooping there.  I think you should block access in your robots.txt file.  Making changes to your robots.txt file is a important security measure.  It keeps the good bots away from sensitive areas.  I recommend a robots.txt file like the following:

User-agent: *
Disallow: /cgi-bin/
Disallow: /smf/Sources/
Disallow: /smf/Themes/
Disallow: index.php?action=admin*
Disallow: index.php?action=calendar*
Disallow: index.php?action=login*
Disallow: index.php?action=printpage*
Disallow: index.php?action=profile*
Disallow: index.php?action=register*
Disallow: index.php?action=search*
Disallow: index.php?action=stats*
Disallow: index.php?PHPSESSID=*
Disallow: index.php?*rss*
Disallow: index.php?*wap*
Disallow: index.php?*wap2*
Disallow: index.php?wwwRedirect*
Crawl-delay: 5


The latest BB mod has a nice reverse DNS check that catches several fake bot attempts.
I have been truly inspired by the SUGGESTIONS as I sit on my throne and contemplate the wisdom imposed upon me.

butchs

Quote from: An0nymousHelper on March 04, 2011, 09:08:32 PM
Thats just two of them and there are quite a few more. If anyone knows what this is it would be greatly appreciated if you could let me no! Thanks!

I would verify that the "Longterm Ban" ban is set to 1 (during the week) or 24 (weekend) hours and let SMF block it for a while.
I have been truly inspired by the SUGGESTIONS as I sit on my throne and contemplate the wisdom imposed upon me.

Glasso

Quote from: butchs on March 05, 2011, 08:19:45 AM
2899 is from a mobile phone and as far as I know Google is not using a mobile phone.
I think this is the google mobile bot. In fact, I raised a similar query on Bad Behavior and got information that google mobile bot is covered in the latest code too!

Quote from: butchs on March 05, 2011, 08:19:45 AM
Why would google need to find your Sources directory?  They should not be snooping there.  I think you should block access in your robots.txt file.  Making changes to your robots.txt file is a important security measure.  It keeps the good bots away from sensitive areas.  I recommend a robots.txt file like the following:...
I now understand 'Hack: Sources!' means an attempt to snoop into the Sources directory  - I have applied the additional blocks in robot.txt now as you suggest, thank you.

Quote from: butchs on March 05, 2011, 08:19:45 AM
The latest BB mod has a nice reverse DNS check that catches several fake bot attempts.
Butchs, this is causing some genuine googlebot requests to be blocked as I see in my webmaster logs. I can send you some trail that I had with Michael where he thought roundtripdns is not fully reliable, if you believe it is worthwhile to go deeper.

Again, I can't thank you guys enough for this great work.


butchs

 The latest version of  BB mod includes the Google Wireless Transcoder changes and as such you should not get blocked.

Google has no business snooping in the sources directory.  Anyone snooping there is trying to read and/or modify the heart of SMF.  No one outside of the admin should be able to see that directory.  Any outside attempts by anyone but yourself should be blocked.  Once you make the changes I requested in robots.txt (if there is something you can do at the google webmaster site please advise) google should stop or be blocked.  Honestly as a security specialist my motto is:  Trust no one even google!   :laugh:   O:)

Quote from: Glasso on March 05, 2011, 10:43:35 AM
Butchs, this is causing some genuine googlebot requests to be blocked as I see in my webmaster logs. I can send you some trail that I had with Michael where he thought roundtripdns is not fully reliable, if you believe it is worthwhile to go deeper.

That could have been avoided.  Not to be rude but please do not make a negative statement unless you have read and understand all the help "?" icons.  This question will be answered to it its correct place:  BB forum.
8)
I have been truly inspired by the SUGGESTIONS as I sit on my throne and contemplate the wisdom imposed upon me.

Glasso

Quote from: Glasso on March 05, 2011, 10:43:35 AM
Butchs, this is causing some genuine googlebot requests to be blocked as I see in my webmaster logs. I can send you some trail that I had with Michael where he thought roundtripdns is not fully reliable, if you believe it is worthwhile to go deeper.
Quote
That could have been avoided.  Not to be rude but please do not make a negative statement unless you have read and understand all the help "?" icons.  This question will be answered to it its correct place:  BB forum.
8)
Sorry, I honestly do not realize what you saw as a negative comment, but that was certainly not my intention. I will respond to you further on the BB topic. Thanks.

butchs

Understood.  I am just making sure people do not get the wrong impression.
:)
I have been truly inspired by the SUGGESTIONS as I sit on my throne and contemplate the wisdom imposed upon me.

wickedgood

I installed this mod and it works really good................too good ;D

Got many emails from members who were getting blocked. I whitelisted members in Permissions but still many getting blocked.

I honestly don't know what all the values in the settings mean. Before I start messing with those...........Is there a way to tone it down?

Don't mind changing the settings but I could use some recommendations on something like a low, medium and high setting?

Appreciate all the hard work you put into this project.

butchs

#269
You should run it for a few days with blocking disabled to prevent that issue.  White list only protects against false dos attacks.

uncheck "Block Violations"

check the following:
"Enable Testing"
"Logging"
"User-Agent Inspection"
"DOS Attack"
"Enable IP Validation"

everything else should be unchecked.  Watch the log and make sure members are not getting blocked before you enable blocking.
I have been truly inspired by the SUGGESTIONS as I sit on my throne and contemplate the wisdom imposed upon me.

wickedgood

I did that? Its still running in analysis mode. 

Didn't notice any members getting blocked before I enabled blocking? Also seemed to block Google, and some other "good" bots. At least that's what it showed?

If I do notice a member getting blocked what can I do to unblock them?

Jesna

Im thinking of using the country test but im not sure about those country codes i have to fill in. Can I use the country codes from here http://countrycode.org/

example: South africa is ZA / ZAF and then i put ZA / ZAF in the field

/Jakob

TheMortician4

I am running SMF 1.1.13, simple Protal 2.3.3, and Emulation drop down to use the Email Validator.


Is this mod compatible with my system?

Bigguy

As far as I know it is. :)

TheMortician4

no coding necessary? I am using Godaddy

Bigguy


busterone

It should install on 1.1.13 with no edits required. It is listed as 1.1.13 compatible on the mod's download page.

Kindred

that, of course, depends on whether you have any mod which conflicts with the code changes that this one makes....
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

butchs

wickedgood,
I will need more information to help you.  Maybe you will like to try reviewing some of the past questions in this thread? ???
I have been truly inspired by the SUGGESTIONS as I sit on my throne and contemplate the wisdom imposed upon me.

butchs

Quote from: Jesna on March 09, 2011, 12:26:45 PM
Im thinking of using the country test but im not sure about those country codes i have to fill in. Can I use the country codes from here http://countrycode.org/

example: South africa is ZA / ZAF and then i put ZA / ZAF in the field

The country code feature for this mod is limited.  It may not work with every system.  Please test before enabling it live.

But if it does work simply use the two letters like "ZA" for "South africa" in the field.  Do not forget "|" between the two letters. ie "ZA|ZM|ZW"
:)
I have been truly inspired by the SUGGESTIONS as I sit on my throne and contemplate the wisdom imposed upon me.

Advertisement: