Advertisement:

Author Topic: Spammer-defence by entry-page?  (Read 2246 times)

Offline walrus2019

  • Semi-Newbie
  • *
  • Posts: 21
Spammer-defence by entry-page?
« on: September 04, 2019, 03:51:43 PM »
Dear all,

Forbes has a very annoying page: when you get the url of an article of Forbes, you do not read the article web-page, but land in a page, where you have to click on a link, AND THEN are directed automatically to the actual article page.

We never thought that we would consider installing such a mechanism. But in the last weeks we were flooded by Chinese and other spammers, who spider the forum and abuse its contents on their web-farms. 2 Chinese internet access providers are responsible for over 500000 forum reads in just 2 weeks. This is a severe abuse, and we are facing problems with our web-hoster.

The abuse is much worse, because the attackers show up with up to 1000 diffferent IPs simultaneously. So we can not block their IPs. The Chinese were easy to block, but a cloud of IPs we can not handle.

So we need some mechanism to separate spiders from real humans. But we were not able to find such a mechanism in the forum. Did we miss something?

If someone knows a way to handle the problem, we would be very happy.

Regards,

Walrus
« Last Edit: September 05, 2019, 01:03:13 PM by Aleksi "Lex" Kilpinen »

Offline Arantor

  • Resident Overthinker
  • SMF Friend
  • SMF Legend
  • *
  • Posts: 71,608
    • StoryBB/StoryBB on GitHub
Re: Spammer-defence by entry-page?
« Reply #1 on: September 04, 2019, 03:57:08 PM »
Having the landing page doesn't really protect you significantly in terms of bandwidth etc.

Something like Cloudflare is probably a better bet, and you can just block all the traffic from China and be done with it.
Don’t try to tell me that some power can corrupt a person. You haven’t had enough to know what it’s like.

No good deed goes unpunished / No act of charity goes unresented.

Offline walrus2019

  • Semi-Newbie
  • *
  • Posts: 21
Re: Spammer-defence by entry-page?
« Reply #2 on: September 04, 2019, 04:05:54 PM »
As far as I understand, Cloudflare is a commercial company. But we are guests on a webspace given to us by friends.

As I said: We can not block all the various IPs, because they  do not come from distinguishable internet access providers. There are hundreds of simultaneous read requests. We think they are from hijacked PCs. There is no other explanation because of the sequence of requests.

If the spiders were hindered to access the forum pages, after some time they would give up, because they can no get what the want to harvest.

We do know that the additional layer is annoying, but we think that human guests of the forum will understand the matter.

Offline Arantor

  • Resident Overthinker
  • SMF Friend
  • SMF Legend
  • *
  • Posts: 71,608
    • StoryBB/StoryBB on GitHub
Re: Spammer-defence by entry-page?
« Reply #3 on: September 04, 2019, 04:08:17 PM »
That’s why you use something like CloudFlare, by way of them having a list of IP addresses linked to a given country.

And no, I guarantee you that for the bots to give up in the fashion you’re talking about, it will take months for them to actually notice.
Don’t try to tell me that some power can corrupt a person. You haven’t had enough to know what it’s like.

No good deed goes unpunished / No act of charity goes unresented.

Online vbgamer45

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 21,555
    • smfhacks on Facebook
    • VBGAMER45 on GitHub
    • @createaforum on Twitter
    • SMF For Free
Re: Spammer-defence by entry-page?
« Reply #4 on: September 04, 2019, 04:23:34 PM »
Cloudflare  does have a free plan which I believe will give some of the blocking features. Give it a try.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Offline walrus2019

  • Semi-Newbie
  • *
  • Posts: 21
Re: Spammer-defence by entry-page?
« Reply #5 on: September 04, 2019, 04:24:01 PM »
To me Cloudflare just looks line one more parasite. Just like the anti-virus companies.

There must be better things than those gangs which just drain money.

We have a small and tough forum, and we are guests. We can not put the burden of attacks onto our generous hosts. we have to do it our way.

Offline Arantor

  • Resident Overthinker
  • SMF Friend
  • SMF Legend
  • *
  • Posts: 71,608
    • StoryBB/StoryBB on GitHub
Re: Spammer-defence by entry-page?
« Reply #6 on: September 04, 2019, 04:25:53 PM »
Then I wish you the very best possible luck.
Don’t try to tell me that some power can corrupt a person. You haven’t had enough to know what it’s like.

No good deed goes unpunished / No act of charity goes unresented.

Offline walrus2019

  • Semi-Newbie
  • *
  • Posts: 21
Re: Spammer-defence by entry-page?
« Reply #7 on: September 04, 2019, 04:27:17 PM »
Cloudflare  does have a free plan which I believe will give some of the blocking features. Give it a try.


That looks so at the first sight. But we have 30 to 50 GB traffic per month. This is the region where Cloudflare DOES want money. We encountered other "free" offers which are much too small, like for 10 web pages, or something like that. But we already outgrew that size.

Online vbgamer45

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 21,555
    • smfhacks on Facebook
    • VBGAMER45 on GitHub
    • @createaforum on Twitter
    • SMF For Free
Re: Spammer-defence by entry-page?
« Reply #8 on: September 04, 2019, 04:35:29 PM »
You can do geoip solution with apache  or other firewall.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Offline walrus2019

  • Semi-Newbie
  • *
  • Posts: 21
Re: Spammer-defence by entry-page?
« Reply #9 on: September 04, 2019, 04:45:28 PM »
You can do geoip solution with apache  or other firewall.


This will not work, because they come from all over the world via hijacked PCs. That is the problem. They behave like ordinary guests, which - in fact - they are not.

So we need a means to make guests undergo a manual procedure, just like Forbes. As I said: We do not like it that way, but we see no other chance.

The entry page, even if hit by hundreds of thousands of requests, is so small in size that it will save more than 95 percent of the traffic. The forum pages easily exceed 300 to 500 kB.

Online Mick.

  • SMF Friend
  • SMF Hero
  • *
  • Posts: 5,026
  • Gender: Male
    • idesignSMF on Facebook
    • @idesign360 on Twitter
    • idesignSMF.com
Re: Spammer-defence by entry-page?
« Reply #10 on: September 04, 2019, 04:48:43 PM »
I use recaptcha mod only. No smf security questions no nothing. Yet I have no spammers. Tho, i see them sign up but they dont post.

Offline walrus2019

  • Semi-Newbie
  • *
  • Posts: 21
Re: Spammer-defence by entry-page?
« Reply #11 on: September 04, 2019, 04:56:16 PM »
I use recaptcha mod only. No smf security questions no nothing. Yet I have no spammers. Tho, i see them sign up but they dont post.

Those spammers who want to register in the forum, do not succeed.

Our problem is GUESTS. They are not registered. They simply drain traffic. This is why we need something to force guests to undergo a manual procedure. But we do not see any way to do that.

Offline walrus2019

  • Semi-Newbie
  • *
  • Posts: 21
Re: Spammer-defence by entry-page?
« Reply #12 on: September 04, 2019, 05:08:18 PM »
I use recaptcha mod only. No smf security questions no nothing. Yet I have no spammers. Tho, i see them sign up but they dont post.

I have to add something about the term "spammers":

The spammers that we encounter are NOT those which place crappy forum posts. The spammers we encounter read forum pages, then shred them to particles of some sentences, and then mount those particles with particles taken from other web-sites in new web-pages the fill their web-sites with: hundreds of thousands of pages.

This way they abuse the original writers of the original web-pages, attract search engines, and pull the major traffic of real people to their sites.

This has a dramatic effect:

1. The huge number of targets for search engines leads to only a fraction of the surfers go and read the original pages. The original pages simply drown in an ocean of crap. This is a deadly means to suppress unwanted web-sites.

2. The search engines might consider the original sites as paying partners of spammers with the INTENT of flooding the search engines - and so the search engines will drop the original sites in ranking. This, too, is a deadly means to suppress unwanted web-sites.


The waves of attack show up suddenly, exceed HUNDREDS of read requests per 15 minutes, and disappear as fast as they began. Which definitely is not the usual traffic caused by humans living in other time zones. Such traffic goes up slowly, and fades away, on a daily basis.

Also, very peculiar: the requests are not for the latest forum entries, but for stuff several years old. Humans would rather read new topics.

Online vbgamer45

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 21,555
    • smfhacks on Facebook
    • VBGAMER45 on GitHub
    • @createaforum on Twitter
    • SMF For Free
Re: Spammer-defence by entry-page?
« Reply #13 on: September 04, 2019, 05:27:35 PM »
You can try mod_evasive on apache might help just adjust the settings.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Offline walrus2019

  • Semi-Newbie
  • *
  • Posts: 21
Re: Spammer-defence by entry-page?
« Reply #14 on: September 04, 2019, 05:37:40 PM »
If "mod-evasive" is something to be installed on the server: we have no means to do that. We can only modify the forum software. Access to the server itself we do not have.

Online vbgamer45

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 21,555
    • smfhacks on Facebook
    • VBGAMER45 on GitHub
    • @createaforum on Twitter
    • SMF For Free
Re: Spammer-defence by entry-page?
« Reply #15 on: September 04, 2019, 05:48:18 PM »
If "mod-evasive" is something to be installed on the server: we have no means to do that. We can only modify the forum software. Access to the server itself we do not have.
it is. could be installed though you might be able to control though .htaccess or contact your host.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Offline walrus2019

  • Semi-Newbie
  • *
  • Posts: 21
Re: Spammer-defence by entry-page?
« Reply #16 on: September 04, 2019, 05:54:22 PM »
it is. could be installed though you might be able to control though .htaccess or contact your host.

I have no idea what you mean. Where is that modification? How can it be controlled by .htaccess?

Online vbgamer45

  • Customizer
  • SMF Super Hero
  • *
  • Posts: 21,555
    • smfhacks on Facebook
    • VBGAMER45 on GitHub
    • @createaforum on Twitter
    • SMF For Free
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Offline walrus2019

  • Semi-Newbie
  • *
  • Posts: 21
Re: Spammer-defence by entry-page?
« Reply #18 on: September 04, 2019, 06:24:54 PM »
https://www.linode.com/docs/web-servers/apache-tips-and-tricks/modevasive-on-apache/ [nofollow]

Thank you, but we have no access to that. Our hosts only can work on a cpanel. so they have no access to the server itself.

Offline Herman's Mixen

  • SMF Hero
  • ******
  • Posts: 7,252
    • HermanMixen on Facebook
    • @Hermans_Mixen on Twitter
    • Herman's Mixen
Re: Spammer-defence by entry-page?
« Reply #19 on: September 04, 2019, 06:36:17 PM »
You can use the SMF GeoIP modification written by a former SMF Member...
It needs a bit of updating, but there are here people around who can help you with that...
Met vriendelijke groet, The Burglar!

 House Mixes | Mixcloud | Any Intelligent fool can make things bigger, more complex, and more violent.
It takes a touch of genius - and a lot of courage - to move in the opposite direction. - Albert Einstein

Former Godfather of our dutch community ;)