Customizing SMF > Building Your Community and other Forum Advice

How do I reduce bandwidth usage?

(1/6) > >>

ajac63:
Firstly, apologies if this is in the wrong section of SMF and also for not having posted for a while, I've had problems...  ::)  Basically, a fair amount of bandwidth has been used up for the current month for my forum, over 86%, and so I'm looking for ways to reduce further excessive bandwidth usage.  What things should I be doing?  I've discovered that spider and crawlers can use up bandwidth a lot and that a robots.txt file in the route directory can help with this, but I'm not that familiar with the syntax.  I've also discovered that turning on http compression can help, but don't know if SMF powered forums have this by default.

To stop more bandwidth being used up, I put my forum offline for maintenance about a day ago, but the usage has gone up to 89.45%. How can more bandwidth be used up when it's offline?

Thanks if anyone can help :)

shadav:
if your users post a lot of images
this mod can help a bit by making the filesizes smaller
https://custom.simplemachines.org/mods/index.php?mod=4082

using a program to optimize all the images on your site itself can help
we're not talking much but in the end every little bit does help

you can sign up for cloudflare and use their services also to help save a bit

Antechinus:
Sounds like it's the spiders that are causing a lot of the problem.* Banning the bad ones via .htacess is the way to go. That will nail the ones that ignore robots.txt. They'll still be able to ping your domain, but they won't get any bytes back (just a 403 error message).

The ,htaccess for the root directory of my domain looks like this:


--- Code: ---#BlockBotsByUserAgent
SetEnvIfNoCase User-Agent (Ahrefs|Baidu|BLEXBot|Brandwatch|DotBot|Garlik|Knowledge|libwww-perl|Linkdex|MJ12bot|omgili|PetalBot|Proximic|Semrush|Seznam|Sogou|Tweetmeme|Trendiction|Wordpress) bad_bot
<RequireAll>
Require all Granted
Require not env bad_bot
</RequireAll>

<Files .htaccess>
order allow,deny
deny from all
</Files>

<Files 403.shtml>
order allow,deny
allow from all
</Files>
--- End code ---

Adding more bots is easy. I only threw in the ones that were giving me trouble.

*Spiders will often index everything on your site, multiple times in succession, when they are on the rampage. This can chew masses of bandwidth, particularly if you have a lot of images and/or downloadable zips.

ajac63:

--- Quote from: shadav on February 18, 2021, 11:14:48 PM ---if your users post a lot of images
this mod can help a bit by making the filesizes smaller
https://custom.simplemachines.org/mods/index.php?mod=4082

using a program to optimize all the images on your site itself can help
we're not talking much but in the end every little bit does help

you can sign up for cloudflare and use their services also to help save a bit

--- End quote ---
Really glad you mention images as this is one of the main things that my hosting provider suggested, so thanks for the mod link.  There's a lot of posts on my forum and so checking them manually would take too long...

Thank you. :)

ajac63:

--- Quote from: Antechinus on February 18, 2021, 11:33:39 PM ---Sounds like it's the spiders that are causing a lot of the problem.* Banning the bad ones via .htacess is the way to go. That will nail the ones that ignore robots.txt. They'll still be able to ping your domain, but they won't get any bytes back (just a 403 error message).

The ,htaccess for the root directory of my domain looks like this:


--- Code: ---#BlockBotsByUserAgent
SetEnvIfNoCase User-Agent (Ahrefs|Baidu|BLEXBot|Brandwatch|DotBot|Garlik|Knowledge|libwww-perl|Linkdex|MJ12bot|omgili|PetalBot|Proximic|Semrush|Seznam|Sogou|Tweetmeme|Trendiction|Wordpress) bad_bot
<RequireAll>
Require all Granted
Require not env bad_bot
</RequireAll>

<Files .htaccess>
order allow,deny
deny from all
</Files>

<Files 403.shtml>
order allow,deny
allow from all
</Files>
--- End code ---

Adding more bots is easy. I only threw in the ones that were giving me trouble.

*Spiders will often index everything on your site, multiple times in succession, when they are on the rampage. This can chew masses of bandwidth, particularly if you have a lot of images and/or downloadable zips.

--- End quote ---
Thanks. :) Crawlers and spiders was the other thing my host mentioned, so this rings a bell.  Where exactly in my htaccess file do I paste this code and should I also use a robots.txt file in the route directory, would I need both or is it one or the other?

Navigation

[0] Message Index

[#] Next page

Go to full version