Is There a Real Anti-Spam-Bot Strategy Being Worked On?

[ForumGuy789]

This is a question for the devs or others working on the SMF project.

I'll start by saying SMF 2.0 is a great piece of work. Adding an extra security question for registration has all but eliminated spam-bots creating 40+ accounts every day. True post/topic moderation has also been great.

But I think it's becoming more clear that any forum software that is going to survive anymore is going to have to deal with these bots head on. We can't just rely on mods from the community, etc. We can not just ignore the problem.

Telling users to ban their IP is telling the user to go off and waste their time. I doesn't matter if they spend all day, every day, doing it because it's not a solution. Anyone who has actually looked at the problem knows that these spam-bots have an almost infinite number of IP addresses that they use at different times.

I realize that it's an extremely difficult thing to deal with - almost impossible. But a few small things would be extremely nice - and EASY to implement. For instance, I'd love to actually be able to USE my Error Log. But if you dare ban any spammer then your error log is constantly filled with garbage when they keep coming back trying to access the website.

Easy and Quick Suggestions:

• Let admins choose whether or not certain errors are shown on the error log. Right now it's all errors are shown or no errors are shown
• Do I really need to see pages of errors saying "Sorry XXXXX, you are banned from using this forum!"
• Do I need to see "Password incorrect - XXXXXX" every time hoards of spambots fail to log into someone elses account?
• When moderating a post/topic a "Delete Post/Topic and Ban Member" option would be much quicker for removing spam and banning someone at the same time. Otherwise once you remove the post/topic you have to search for the member (if you wrote the name down even). Or, you can click on the members name, ban them, then work your way back to the topic/post again and remove it.

More Aggressive Suggestions:

• Look at the security/spam-bot protections that other forum software has used. IP-Board is a great example. They have some type of huge database of spammers and check every user accessing the website against this database. It works wonders. I've tested their software. If something slips through, then admins can add them to the database. But usually it's completely automated and turn-key. An admin rarely even needs to deal with it.
• IP-Board also has many other features for security and spam-bot prevention.
• When banning a member/IP actually ban them, instead of giving them a message. Ban them like a firewall does - so they use very little of your resources.

So is development of features still continuing for SMF and are any of my suggestions or other solutions being planned or worked on? If so I will probably become a charter member.

Btw, I can tell you the spam-software doing most of the damage out there is called XRumer. If you need more info about it and what it's doing let me know. I've never used it but have heard much about it - SEO companies use it.

[青山 素子]

Let admins choose whether or not certain errors are shown on the error log. Right now it's all errors are shown or no errors are shown

Would be useful. Depending on how errors are logged, it shouldn't be too difficult to do. Right now, it looks like they are only logged by severity. This means the logging system might need an overhaul to make more granular options.

Look at the security/spam-bot protections that other forum software has used. IP-Board is a great example. They have some type of huge database of spammers and check every user accessing the website against this database. It works wonders. I've tested their software. If something slips through, then admins can add them to the database. But usually it's completely automated and turn-key. An admin rarely even needs to deal with it.

That sounds like an online service. The SMF software has traditionally avoided depending on external services in the core product. If those services change, go offline, or otherwise have issues, it will be SMF that is blamed. Also, many SMF users have installs where the general Internet isn't accessible (via closed internal networks). Having such services required would make it more difficult for those users.

IP-Board also has many other features for security and spam-bot prevention.

Like what?

When banning a member/IP actually ban them, instead of giving them a message. Ban them like a firewall does - so they use very little of your resources.

Handling an actual IP ban at the server level will vary considerably depending on how the install is done. Most of the time, web server applications aren't allowed to modify a server software firewall. Likewise, you can't depend even on htaccess restrictions as some hosts don't allow htaccess directives, or use different server software (IIS, nginx, ...) that doesn't use that type of file. Then you have the added work of removing the ban that was put in place. Note that this all would only work for an IP-based ban.

Even if SMF doesn't display an error, it still has to process the request and check the ban list.

So is development of features still continuing for SMF and are any of my suggestions or other solutions being planned or worked on? If so I will probably become a charter member.

Your post probably really should have gone in the feature requests section. I'm sure a staff member will move this eventually.

Btw, I can tell you the spam-software doing most of the damage out there is called NAME REMOVED. If you need more info about it and what it's doing let me know. I've never used it but have heard much about it - SEO companies use it.

That's the thing. It's a commercial product. There is a lot of money involved to make sure it can bypass any kind of protection out there. We're talking millions of dollars in income annually, more than likely. Heck, the newer versions even integrate a captcha-bypassing service that uses real humans (not dropping names, they don't deserve the publicity).

It's like e-mail spam. As long as there is a huge economic benefit to the tactics, they will exist and all you'll get is an arms race between the spammers and those trying to stop them. The only real way to stop it is to provide very large disincentives (huge monetary payments for damages, prison time, etc) that outweigh the benefits and to then enforce these at a level that convinces 90% of the spammers that it's a bad idea to pursue that avenue of income.



By the way, some of the better services right now are CloudFlare and Project Honeypot. CloudFlare actually uses the Project Honeypot database, so the two services keep growing in sophistication and detection of threats.

[Kindred]

As noted: The best anti-spam measures depend on a third party (stop forum spam and project honeypot) As such, they will probably never be included in the base install - for the reasons motoko already mentioned.  That being said, they are one click installs as mods.

[xpubstargamingx]

I have 1.1.14 with Stop Forum Spam and Project Honeypot. No issues anymore for my community.

[Norv]

I'd have just a quick question for the moment, ForumGuy (I will come back on this),

Can you please tell, why do you say that you can't rely on mods from the community? There are (as you can see here in this topic as well as elsewhere) mods that have proven very useful, actively maintained and supported by their developers.

[ForumGuy789]

Norv, I mostly say this because I've had experiences with mods where they ended up not being supported, or compatible with upgrades like SMF 2.0 for instance. Then, there's always the possibility that the mods are not compatible with other mods.

I am definitely up for trying some mods though now. Can someone please point me to the right direction for which mods I should be using? Ideally the best, least buggy mods. Do I need to pay for some service?

[Kindred]

personally, for 2.0, I like using the built in questions, bad-behavior (which also has HoneyPot) and Stop Forum Spam.

The three of those combined have my spam registrations down to 3 a month (which are caught and flagged at registration) and my spam posters who have gotten past registration at ZERO

[ForumGuy789]

Thanks, I'll give bad-behavior and Stop Forum Spam a shot.. and hopefully will be able to turn on my Error Log again.

[ForumGuy789]

I have a Windows IIS server btw. Hopefully there will be no issues with these mods.

[ForumGuy789]

Ok I do like the 2 mods but it still does not let me use my Error Log because it gets filled with
"Sorry, XXX, you are banned from the forum!"

But that might be IP addresses that have not been caught by Honeypot.

It would still be extremely useful to turn off certain error types in the error log (feature request).

But after installing the 2 mods Error log messages are reduced I think. So that's good.

[busterone]

There never will be a completely dependable anti-spam method. As said before, for every new prevention method created, money is poured into a way to bypass it.  Banning by IP is pointless, considering that most spambot masters continually change and rotate IPs and use proxies, etc. When IPv6 is completely up and active, there will be an endless supply of more IPs for them to use. Banning will be a complete waste of time.
The simplest method is anti-spam verification questions that are unique and possibly pertain to your forum's subject or theme. Those using the botware can program in answers to questions, but when they are multiple and unique, they will not bother to even try.  There will always be the human spammers out there that can and will answer the questions, but only if they are too easy to solve. A layered approach is essential.  I use the questions, Project Honeypot, and Stop Spammer. In most cases, if they are already in the Project Honeypot db, they never even reach the registration process. If they are not, then the bulk of those never make it past the questions. The few that make it past the questions are humans, and most of them do not pass Stop Spammer. I get an actual registrant to make it through all the defenses about once every 3 or 4 months. 

It also helps to regularly change your questions. In the event that a human spammer makes it through, they often communicate to others about which sites to target. The answers can easily be shared on their network to be programmed into the bots for a particular site. Changing them or rotating them regularly will hinder their efforts. The dumb ones will continue to come over and over with no success, but the smarter ones will move on to easier pickings.   

[ForumGuy789]

Ya, woke up with 7 pages of errors this morning.
"Sorry XXXXXXX, you are banned from using this forum!"

So I guess I'll be turning off the Error log again.

[Illori]

that is not really an error it is an alert, and it is related to you having people on your ban list trying to access your forum. that would happen on any smf forum that has this happen.

[ForumGuy789]

that is not really an error it is an alert, and it is related to you having people on your ban list trying to access your forum. that would happen on any smf forum that has this happen.

Wow, I almost knew someone would say this - because I remember someone saying this to another user here.

I realize it's not a true error.. that it is an alert. But it doesn't change the fact that it makes finding any REAL errors completely impossible, because you have to find them in pages and pages of worthless alerts. Therefore error log file = worthless.

[青山 素子]

I realize it's not a true error.. that it is an alert. But it doesn't change the fact that it makes finding any REAL errors completely impossible, because you have to find them in pages and pages of worthless alerts. Therefore error log file = worthless.

If you're running 2.0, just choose the "Critical" filter. That will only show severe problems that have been logged or potential security issues like failed admin access attempts. It's not the best solution, but it does work for now.

[kathiemt]

I note no comment or answer about why when you delete an account and ALL posts that the posts are STILL there. I'm having exactly the same problem - almost 400 entries by the one person while I was in bed at night. Now I have to delete them all - one by one. NOT fun at all.

[青山 素子]

I note no comment or answer about why when you delete an account and ALL posts that the posts are STILL there. I'm having exactly the same problem - almost 400 entries by the one person while I was in bed at night. Now I have to delete them all - one by one. NOT fun at all.

I've done the account deletion + post&topic deletion before several times and haven't had this happen to me. Sure, I'm one datapoint, but at the very least I can't replicate the bug.

If you have steps that will consistently cause this issue and can detail the webserver configuration, please provide more information so maybe someone can duplicate the issue.

[Arantor]

Firstly, you can now turn off ban entries going into the error log as of 2.1.

Secondly, IP.Board's principle methodologies are to use Stop Forum Spam and reCAPTCHA, both of which are supported with mods here, and both of which would violate our general policy of not having big ticket features that rely on third party services (OpenID being one of the only exceptions for this, if only because it's not a single third party service)

Thirdly, Q&A is one method we always advocate, and 2.1 has better Q&A than 2.0 does (multiple languages, multiple answers per question are supported), and there are some subtle methods involved as well in 2.1.

Given these, I'm going to be moving this to the 'dealt with' pile of requests but if anyone has any better ideas for anti spam, I'm more than interested to listen.