banning entire countries

[Vince S]

So, now that I went and got excited about that discovery, if there is a mod that would be beautiful and simple it is to take the simple implementation script in the previous post, combine it with a monthly wget database update as explained in http://www.maxmind.com/app/installation and add a simple table to inject countries into the script. How about it legends  , any takers? 

Edit, and include a variable "die" message. as an eg this is the one I use: http://hunterdog.org.au/DiscussionBoard/index.php?topic=630 - which also happens to include a very fine demonstration of the problem that is real, "in your face" stuff for real webmasters out there that, despite their doofus-fingered knowledge of such matters they DO choose to shut down forums when confronted by the perfectly fine logic presented here .vs. what that means in the real world. which is why I kept harping on about how real the problem is despite all the very finely expressed logical reasons that this is basically tough luck! Its what i call "focus" and I'm glad now that anyone arriving here will actually go away with a real answer. Hopefully a later post will have a link to a mod to make it even easier but let me just say to those people, you CAN do this yourself, just read it a couple of times and use your host's support line if having any trouble with the uploading bits etc.

Edit again since no-one else has posted: I note that IPv6 is also being covered, see about 2/3 the way down http://www.maxmind.com/app/geoip_country. Obviously on closer scrutiny the SiteGround tutorials are a bit dated (they refer to version 1.1.5), but still quite valid. Muddling through adding the IPv6 info to the mix is certainly looking like great fodder for what would still be a fairly light weight bit of mod work for someone with the relevant skills.

I am also noticing that the availability of free geo database with what looks like an easy way to keep it current via wget IS a reasonable basis to suggest that geographical features be incorporated to the base SMF version, if nothing else than for the "cool factor" of the example given in the SiteGround tutorial for that "other" forum software (I am referring to http://www.siteground.com/knox/tutorials/uploaded_images/images/geoip/1.jpg which is from http://www.siteground.com/tutorials/geoip/forums.htm). Or more clearly stated it would be a great feature for many forums that would like to display different content to the user based on country - or, as in this thread, basically restrict what they can do. With the kind of backing that maxmind provide I don't doubt they would be interested in even helping out as I'm sure they know there will be a conversion rate for the % that are happy to pay for the next level up in accuracy and updates, bit of a win-win really and would be likely to be a robust solution to what spammers might do in the future - I am suggesting it would be to "go bug non-SMF forums!!!"

[青山 素子]

Note that using the Maxmind GeoIP database inside PHP requires that your host have installed the GeoIP PECL extension, or possibly the Net_GeoIP PEAR module. While the PEAR module means you don't need server access, it will also be much slower as it would need to be fully loaded on each page execution.

Any SMF modification would therefore require an external dependency of one sort or another. Going the PECL route would limit the number of hosts on which it could be used while the PEAR route might be very demanding for busy sites or on oversellers such as SiteGround.

[galahad]

If I understand correctly the proposal that Vince  S has put forward is not difficult to implement but requires hardware resources.  Is that correct?  And if so, is there a reasonable way to quantify the resources required so hosts can determine if the tradeoff between less spam and customer satisfaction is balanced with hardware requirements? 

I suspect that MANY hosts would find the overall lower traffic resulting from fewer spammers invading customers sites just might be of some advantage as well.

[butchs]

No.  It requires software resources.  青山 素子 basically said that it will be faster if the host installed the PECL extension.

[butchs]

Edit again since no-one else has posted: I note that IPv6 is also being covered,

Good grief.  I posted that a few days ago... 

[Vince S]

I'm paddling around in stuff that is a bit over my head so, just for the dummies, what is the "GeoIP PECL" and is running it really a problem? I see that a "PECL" is a "PHP Extension Community Library" and obviously the relevant library required is the GeoIP one. Isn't that what the first referenced item is in the files that are said to be what needs to be installed? ie geoip.inc (http://www.maxmind.com/download/geoip/api/php/geoip.inc)

If something else is meant could a clear explanation of what the real issue is here be given please? ie "Any SMF modification would therefore require an external dependency of one sort or another. Going the PECL route would limit the number of hosts on which it could be used while the PEAR route might be very demanding for busy sites" sounds very important, but the "how to" stuff doesn't suggest anything of the kind, so what am I missing about what requiring software resources really means? From my perspective to suck up a bit of software resources to shut a spammer out hardly seems a big deal. And I see at MaxMind that they can do a million lookups a second, probably that is their commercial solution. However their explanation of how to make this efficient in the free system might be over my head, but it doesn't sound hard. See http://www.maxmind.com/app/php. Thank you.

[青山 素子]

No.  It requires software resources.  青山 素子 basically said that it will be faster if the host installed the PECL extension.

Both, actually. There is additional software needed and additional hardware will be needed (memory, CPU) to support the lookups. Especially on every request, since the noted way in this topic doesn't do any caching.

I'm paddling around in stuff that is a bit over my head so, just for the dummies, what is the "GeoIP PECL" and is running it really a problem? I see that a "PECL" is a "PHP Extension Community Library" and obviously the relevant library required is the GeoIP one. Isn't that what the first referenced item is in the files that are said to be what needs to be installed? ie geoip.inc (http://www.maxmind.com/download/geoip/api/php/geoip.inc)

PECL is a repository for shared objects (DLLs in Windows-speak) for PHP. These have to be loaded at the server level (well, if you have access for a custom php.ini, you could load it yourself if you have a version compiled for the PHP on the server).

PEAR is a repository for pure-PHP functions and classes.

The PECL version will use less memory and less CPU since it will be written in C, use the C API MaxMind coded, etc. PEAR is a bit easier to use if you don't have access to the server configuration, but will require more configuration to use, will use more memory, and will likely be slower.

but the "how to" stuff doesn't suggest anything of the kind

That's because SiteGround already has the PECL extension installed and enabled. That's what those function calls use. As to if the system IP database is up-to-date, that's a whole different issue - it's probably not.

From my perspective to suck up a bit of software resources to shut a spammer out hardly seems a big deal.

See my above remark about spammers just moving to using botnet resources in "desirable" countries. Of course, once implemented, it will hardly be removed (grey-listing still remains popular despite e-mail spammers easily bypassing that now - it just prevents two copies of the spam from being handled).

Plus don't discount resource usage of these kinds of solutions. It may work if you only have ten simultaneous requests, but it won't scale well.

And I see at MaxMind that they can do a million lookups a second, probably that is their commercial solution.

With enough hardware and caching, you can easily handle that rate. However, it's not cheap.

However their explanation of how to make this efficient in the free system might be over my head, but it doesn't sound hard. See http://www.maxmind.com/app/php. Thank you.

That section involves using the Apache HTTPd mod_geoip module from within PHP. It also only works when PHP is installed as a module. It will not work when PHP is used via CGI or FastCGI.

[galahad]

Are we looking for ways to solve the problem or reasons not to address it?   

[青山 素子]

Are we looking for ways to solve the problem or reasons not to address it?   

The problem is discouraging spammers. The solution is not "banning countries".

[Vince S]

Rather than just sticking to the somewhat philosophical debate I at least went and asked for practical "what does it really mean" answers from SiteGround. I would first say that I am aware it is not always OK to regurgitate helpdesk style responses but I understand stripping identities out and making sure there is nothing which could be seen as offensive or a problem and obviously that there is no possibility such support could be viewed as "policy" unless separately confirmed, I think the following is at least a useful contribution to the topic. It doesn't seem like too much of a problem would arise from pursuing this path, indeed this host recommends it (country IP banning) as an appropriate control tool for forum registration - in pretty much the same way that Butchs initially did in his (joke) oh-so-cryptic way! I have assembled my q's with the answers for readability.

Before posting I must admit I think the wider benefits of integrating GeoIP as a feature, or at least as a mod, could bear some discussion without necessarily having the blinkered approach that this is only about the merits or otherwise of one particular possible use (country banning) where at best, "the case against" is founded on a lot of presumptions and assumptions about future / uncertain impacts without actually dealing with the real world in-your-face impacts of the status quo. Sorry that sounds a bit soap-box-ish but it is how I feel, here is the promised info:

1) Do I need to get SiteGround to install the GeoIP PECL extension, or possibly the Net_GeoIP PEAR module to use this feature for an SMF forum?

1. I don't think you have to install additional modules, in order to use the GeoIP feature of your SMF forum. If you are following the steps provided in the tutorial you should have no problems using it: https://www.siteground.com/tutorials/geoip/forums.htm

2) Does using either of the modules above or whatever is needed to make GeoIP work cause any performance degradation issues that matter for an SMF forum or SiteGround?

2. Please note that any additional component of your application will need more resources. If the component code is well written and it is compatible with the application you should have no problems using it and there should be no overloading of the system resources.

3) Maybe you can't answer this but do you think hosts generally would have problems with using GeoIP in SMF forums?

3. I can not guarantee that there will be no problems when you are using the GeoIP feature with your application, because it is a third party component and there could be always some requirements that are not met.

4) And one for the "to do" list: How about getting SiteGrounds tutorials for SMF on the GeoIP topic updated to mention IPv6? Which is now available through a separate Maxmind database.

4. I will address your request to my supervisors and we will try to update the necessary information in short term.

5) I have looked into it some more, Maxmind talk about doing it like this: http://www.maxmind.com/app/c which is different to the SiteGround tutorials, is that a better way?

5. Please note that you won't be allowed to install the GeoIP API on your shared server. Our system administrators are constantly updating the base OS of the servers and they are installing components which are providing optimal stability of the shared server.

I do of course "get" that these answers won't be applicable to many hosts, which of course could be an issue in itself as the fact that this and some other hosts go out of their way to be SMF friendly (check this out http://www.siteground.com/smf-hosting.htm) could be something we here should have more than a passing interest in, don't you think?

[galahad]

Are we looking for ways to solve the problem or reasons not to address it?   

The problem is discouraging spammers. The solution is not "banning countries".

As I have explained, in the case of my boards that IS the solution as there will be no legitimate members from China.   

[IchBin™]

I could see the option to have a registration check to not allow certain countries being useful.

[NanoSector]

Are we looking for ways to solve the problem or reasons not to address it?   

The problem is discouraging spammers. The solution is not "banning countries".

As I have explained, in the case of my boards that IS the solution as there will be no legitimate members from China.

I can see your point there, but some IPs are completely random so if you ban people from a "country", some other people from other country's also bet blocked.

There is no real way to address which guest is in which country.

[Vince S]

I can see your point there, but some IPs are completely random so if you ban people from a "country", some other people from other country's also bet blocked.

There is no real way to address which guest is in which country.

eh? Does that mean the whole GeoIP thing is bs? The claim is 99.5% accuracy (see http://www.maxmind.com/app/geolitecountry) and, with an appropriate pathway to deal with any unintended rejections, either as per my earlier examples or some other way, it would seem that there COULDN'T be an objection on the grounds of unintended consequences. It also seems there are other benefits as outlined earlier to picking up this sort of technology as a base feature or mod at least, what about them?

The other argument "against" we are hearing here is that the technique would soon be rendered ineffective. Well this is real guesswork "fear of the bogeyman" stuff and it seems like something else is going on here, anyone care to elaborate? Or tell us what the gaping hole in my and others logic that one could drive a double decker bus through really is?

People are driven, in this order, by Fear, Greed, Love, All the other stuff. So, what's the REAL fear that is being brought to the table here? Buggered if I can see it since the job seems easy and the benefits seem real!

[NanoSector]

They claim, but do they really offer? no. Because they cannot.

[Arantor]

Buggered if I can see it since the job seems easy and the benefits seem real!

OK, I'll answer this one and say what everyone else has thought and hopefully use terms you're going to follow because you appear to have misinterpreted what everyone else has already said.

Your solution works for you. It works for others. It doesn't work for everyone - it wouldn't work on this very site, for example. It's unreliable now, it's going to get more and more unreliable as IPv4 fragments.

Spam is NOT a technological problem. Don't try using technological means to solve it.

[galahad]

Spam is NOT a technological problem. Don't try using technological means to solve it.

Spam would not exist without technology and controlling that technology is the only way to get rid of it.  Neither you nor I are qualified to create the technology that will allow us to catch 90%+ of the spammers before they hit our boards but outfits like g-mail have been extremely successful in reducing it to a manageable level.   We need similar, perhaps even identical technologies that we can apply to our boards.

[NanoSector]

Spam DOES exist without technology... And even then, you should not strike back with the same cause..

Oh and spam is defeatable for the most part WITHOUT banning country's..

[Vince S]

They claim, but do they really offer? no. Because they cannot.

This is a ridiculous discussion, where's the maturity of a bit of horse sense here? I really don't get this and I didn't think I was a dummy. The above comment is totally baseless in the first instance, and absolutely ignores that there would be a solid bypass system, PLUS there are other benefits to adopting this technology that are blindly ignored.

Who CARES if the accuracy rate is 99.5% and 0.5% get "Sorry, you'll need to send an email to us" or pass a test, or whatever? So what if its 97% to 3%? Or do you have some fabulous knowledge that says GeoIP is a scam or something?

I have no problem with people disagreeing, and even bringing a bit of "colour via idiosyncracies or whatever. But please Keep it real people!

It doesn't work for everyone - it wouldn't work on this very site, for example. It's unreliable now, it's going to get more and more unreliable as IPv4 fragments.

Spam is NOT a technological problem. Don't try using technological means to solve it.

I must be REALLY thick. But please let me first say I TOTALLY respect that some of the people saying it is a stupid idea put in a lot of time to supporting the SMF community and I soooo don't mean to be disrespectful in any way. Nor like a petulant child, which I was beginning to think was happening in reverse, people get an idea in their heads and ego won't let it change. Could that be happening for me and others? I don't think so as all that SEEMS to be happening is a refusal to engage on the logic, instead something else is going on.

So, to put it plainly, why wouldn't we look at using easy to apply technology to get a BIG benefit for many easily and quickly with the only POSSIBLE downside being that those that chose to use it might slightly increase the demand on their servers?

I also don't get why it wouldn't work on this site? If there are countries that provide say 95% spam registrants and 5% real ones what's the drama of redirecting the real ones to a test, or sending an email if the registration rates aren't too high? Remember we are just talking about an attempt to filter out human registrants not block forums with total bans. The only risk seems to me to be that the database we are relying on may not exist in perpetuity into the future, but switching sources is likely to be easy and the "other benefits" one can derive from this technology would suggest it is only going to get better not worse into the future.

[NanoSector]

Eeh, my pardon?
Really, take it with a bit of salt...

Of course they cannot offer that, they can offer 50% but more or less not higher.