Database abnormally increases from 130 to 1400 MB

[Sono]

My forum has been running with a database size between 100-200 MB for many years. Recently something happened because since August the database size increased to 1.4GB

I checked the database, and everything seems to be fine, except for 1 row: smf_log_errors. That has a size: 1,3 GB, so I guess the problem happens there. Such things are inside it:

https://i.ibb.co/RDvXKtk/smf-error-2.jpg

You may notice that Pretty URLs mod is mentioned in some fields, but before you started thinking about that is malfunctioning, let me mention something that may be relevant.

My forum has been using abnormally high bandwidth for a while. The forum is 1,2GB with 118210 Posts in 20186 Topics. Yet, the monthly bandwidth is now totalling at 300GB. When I check usage logs, I see approaches to redirect my pages, but I think theser are just attempts, because noone has ever experienced a redirection appearing. Anyhow I see such lines:

Code Select Expand

35.91.39.172 - - [15/Jan/2024:18:49:21 -0800] "GET /index.php?thememode=full;redirect=https://sftmn.lichtundwort.de/tank-trouble-unblocked-games-77.html HTTP/1.1" 302 664 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/605.1.15 (KHTML, like Gecko; compatible; FriendlyCrawler/1.0) Chrome/120.0.6099.216 Safari/605.1.15"

If I check the link intended to be redirected (thememode=full) nothing happens, I stay on my site. So I assume it is just an attempt.

My friend suggested to disable certain crawlers from htaccess:

Code Select Expand

#disable bad crawlers
RewriteCond %{HTTP_USER_AGENT} inetdex [NC,OR]
RewriteCond %{HTTP_USER_AGENT} MJ12bot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SemrushBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SentiBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SeznamBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} YaK/ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} YandexBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} coccocbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} bitlybot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Mail.Ru [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Baidu [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Jakarta [NC,OR]
RewriteCond %{HTTP_USER_AGENT} seostar.co [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Neticle [NC,OR]
RewriteCond %{HTTP_USER_AGENT} AhrefsBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} DotBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Bytespider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Sogou [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Exabot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} MauiBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ia_archiver [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Kinza [NC,OR]
RewriteCond %{HTTP_USER_AGENT} PetalBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} AspiegelBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Aspeigel [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Neevabot
RewriteRule ^.* - [F,L]

And indeed, after adding this, the daily 10GB bandwidth decreased to 1GB per day. But, only for a while. After about a month it started to crawl back. I have checked the ip from a redirection attempt, it is a Chinese ip. For my forum the whole of Asia is irrelevant as member source, so I could just ban China as a whole, or certain ranges there. How could I do that? To ban an ip range in htacces? I only found how to ban a single ip address, not a range. It seems the hackers are finding their way through other servers now.

Back to the database problem, what can I do with it? I guess I could just wipe smf_log_errors but how to do that properly? I know nothing about database management, sorry.

[vbgamer45]

Yes you can truncate the table.

But make sure you are on latest version of PrettyUrls
if you are what is the full error message?

[Sono]

I don't know because since then I tried clicking on "delete unneccessary logs" in the admin menu of the forum, and the big table got wiped. I use a 10 year old version of Prettyurls, you are kidding there is newest! I thought it has never been updated. But I fear this log multitude was because of the hackers' activity.

[Steve]

Newest PrettyUrls: https://custom.simplemachines.org/index.php?mod=636

Let's get that updated first and then see what your symptoms are.

[Sono]

Newest PrettyUrls: https://custom.simplemachines.org/index.php?mod=636

Let's get that updated first and then see what your symptoms are.

Thanks! I have checked the link on the mod page to follow for Installing Instructions, but on the SMFHACKS.com page opening, I don't see any links to instructions. I don't want to screw my forum up, how do I update correctly? Reinstall the old one and install the new one simply? No additional "tweaks" required?

[vbgamer45]

Just uninstall old then install new and should be set.

[Sono]

Just uninstall old then install new and should be set.

I have removed Pretty URLs, but the problem still persists. 40MB daily increase of the SMF_LOG_ERRORS table. In spite of that Pretty URLs has been removed, it still appears in the error list. The table is full of these things, and besides the daily Bandwidth of my 1GB forum is now 150GB. I wonder when the host will notice and ban my forum...

Code Select Expand

   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
51
1705824508
0
52.167.144.221
?action=dlattach;attach=309;type=avatar
8: Undefined index: html_headers
ba7ad476f6b7f23cfcebec35e26f08b6
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
3557
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
52
1705824508
0
52.167.144.221
?action=dlattach;attach=309;type=avatar
8: Undefined index: template_layers
ba7ad476f6b7f23cfcebec35e26f08b6
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
3595
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
53
1705824508
0
52.167.144.221
?action=dlattach;attach=309;type=avatar
2: Invalid argument supplied for foreach()
ba7ad476f6b7f23cfcebec35e26f08b6
general
/home/congovib/domains/congovibes.com/public_html/...
3595
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
54
1705824508
0
52.167.144.221
?action=dlattach;attach=309;type=avatar
8: Undefined index: character_set
ba7ad476f6b7f23cfcebec35e26f08b6
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
161
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
55
1705824508
0
52.167.144.221
?action=dlattach;attach=309;type=avatar
8: Undefined index: session_id
ba7ad476f6b7f23cfcebec35e26f08b6
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
161
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
56
1705824508
0
52.167.144.221
?action=dlattach;attach=309;type=avatar
8: Undefined index: session_var
ba7ad476f6b7f23cfcebec35e26f08b6
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
188
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
57
1705824508
0
52.167.144.221
?action=dlattach;attach=309;type=avatar
8: Undefined index: session_id
ba7ad476f6b7f23cfcebec35e26f08b6
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
188
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
58
1705824508
0
52.167.144.221
?action=dlattach;attach=309;type=avatar
8: Undefined index: template_layers
ba7ad476f6b7f23cfcebec35e26f08b6
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
3702
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
59
1705824508
0
52.167.144.221
?action=dlattach;attach=309;type=avatar
2: array_reverse() expects parameter 1 to be array...
ba7ad476f6b7f23cfcebec35e26f08b6
general
/home/congovib/domains/congovibes.com/public_html/...
3702
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
60
1705824508
0
52.167.144.221
?action=dlattach;attach=309;type=avatar
2: Invalid argument supplied for foreach()
ba7ad476f6b7f23cfcebec35e26f08b6
general
/home/congovib/domains/congovibes.com/public_html/...
3702
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
61
1705825650
0
47.128.119.183
?pretty;action=profile&u=1009%3BPHPSESSID=9o6g...
8: Undefined index: maintain_mode
2602fc564f519f5ab71909f11e8c4518
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
257
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
62
1705825650
0
47.128.119.183
?pretty;action=profile&u=1009%3BPHPSESSID=9o6g...
8: Undefined index: in_maintain_mode
2602fc564f519f5ab71909f11e8c4518
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
167
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
63
1705825651
0
51.75.149.142
?thememode=full;redirect=https://indiapills.pro
8: Undefined index: notify_announcements
dbbc83492a5d95c0f641249639aaea72
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
102
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
64
1705825651
0
51.75.149.142
?thememode=full;redirect=https://indiapills.pro
8: Undefined index: code
dbbc83492a5d95c0f641249639aaea72
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
1090
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
65
1705825651
0
51.75.149.142
?thememode=full;redirect=https://indiapills.pro
8: Undefined index: code_select
dbbc83492a5d95c0f641249639aaea72
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
1090
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
66
1705825651
0
51.75.149.142
?thememode=full;redirect=https://indiapills.pro
8: Undefined index: code
dbbc83492a5d95c0f641249639aaea72
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
1132
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
67
1705825651
0
51.75.149.142
?thememode=full;redirect=https://indiapills.pro
8: Undefined index: code_select
dbbc83492a5d95c0f641249639aaea72
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
1132
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
68
1705825651
0
51.75.149.142
?thememode=full;redirect=https://indiapills.pro
8: Undefined index: quote
dbbc83492a5d95c0f641249639aaea72
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
1414
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
69
1705825651
0
51.75.149.142
?thememode=full;redirect=https://indiapills.pro
8: Undefined index: quote_from
dbbc83492a5d95c0f641249639aaea72
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
1423
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
70
1705825651
0
51.75.149.142
?thememode=full;redirect=https://indiapills.pro
8: Undefined index: quote_from
dbbc83492a5d95c0f641249639aaea72
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
1430
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
71
1705825651
0
51.75.149.142
?thememode=full;redirect=https://indiapills.pro
8: Undefined index: quote_from
dbbc83492a5d95c0f641249639aaea72
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
1444
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
72
1705825651
0
51.75.149.142
?thememode=full;redirect=https://indiapills.pro
8: Undefined index: quote_from
dbbc83492a5d95c0f641249639aaea72
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
1453
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
73
1705825651
0
51.75.149.142
?thememode=full;redirect=https://indiapills.pro
8: Undefined index: modSettings_title
dbbc83492a5d95c0f641249639aaea72
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
4174
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
74
1705825651
0
51.75.149.142
?thememode=full;redirect=https://indiapills.pro
8: Undefined index: package
dbbc83492a5d95c0f641249639aaea72
undefined_vars
/home/congovib/domains/congovibes.com/public_html/...
4179
   
Módosítás Módosítás
Másolás Másolás
Törlés Törlés
75
1705825651
0
51.75.149.142
?thememode=full;redirect=https://indiapills.pro

[vbgamer45]

That's a lot of different errors.
You can turn off error logging in SMF there is a setting for that which will stop the table from growing. That is a band aid though.

[Sono]

How can you stop those idiots trying to redirect my pages? What can I do with that? There is a ton of attacks every day, how can I block them from accessing the site? I guess their system stored my prettyurl type links, that's why they still trying to reach those.

[vbgamer45]

If a certain ip address you can block in cpanel or your hosting control panel.
Or lastly .htaccess file if using apache.

[Sono]

I have checked the smf_log_errors table again, and I found that about 50% of that multitude of errors come from some ativity in connection with avatar attachments, see this example:

Code Select Expand

<!-- Tábla smf_log_errors -->
        <table name="smf_log_errors">
            <column name="id_error">1</column>
            <column name="log_time">1705824026</column>
            <column name="id_member">0</column>
            <column name="ip">157.55.39.10</column>
            <column name="url">?action=dlattach;attach=246;type=avatar</column>
            <column name="message">8: Undefined index: forum_name_html_safe</column>
            <column name="session">697a64ac14fc2de57ecd691766dc15a6</column>
            <column name="error_type">undefined_vars</column>
            <column name="file">/home/congovib/domains/congovibes.com/public_html/Themes/default/languages/Login.english.php</column>
What causes this? I did have problems with avatars recently. When moving my forum they did not appear after reinstalling, and I reverted to a 2 year old forum backup to copy back the Attachments folder from that one, to make avatars appear again. The avatar of some members did went missing and appear as an unfound img in the forum. They did not replace it yet. Can this error be related to that? If yes, is it possible to reset avatars? To make it deleted for all members requiring them to upload again?

[Sir Osis of Liver]

Are you in 2.0 or 2.1?

[Kindred]

You have a problem in your theme and possibly source files because standard variables are missing.

What Mods installed?
You may have to load a clean set of files

[Sono]

Are you in 2.0 or 2.1?

2.0.19

[Sono]

You have a problem in your theme and possibly source files because standard variables are missing.

What Mods installed?
You may have to load a clean set of files

This is my recent mod list:

Code Select Expand

1. Simple Audio Video Embedder 7.0.3a [ Uninstall ]
2. SMF 2.0.19 Update 1.0 [ Uninstall ]
3. SMF 2.0.18 Update 0.1.0 [ Uninstall ]
4. SMF 2.0.17 Update 1.0 [ Uninstall ]
5. SMF 2.0.16 Update 1.0 [ Uninstall ]
6. SMF 2.0.15 Update 1.0 [ Uninstall ]
7. SMF 2.0.14 Update 1.0 [ Uninstall ]
8. SMF 2.0.13 Update 1.0 [ Uninstall ]
9. SMF 2.0.12 Update 1.0 [ Uninstall ]
10. SMF 2.0.11 Update 1.0 [ Uninstall ]
11. SMF 1.1.21 / 2.0.10 Update 1.0 [ Uninstall ]
12. SMF 1.1.20 / 2.0.9 Update 1.0 [ Uninstall ]
13. SMF 2.0.8 Update 1.0 [ Uninstall ]
14. SMF 2.0.7 Update 1.0 [ Uninstall ]
15. Recent Post Settings 1.3 [ Uninstall ]
16. Bot Buster 1.1 [ Uninstall ]
17. Simple Image Upload 1.4.0 [ Uninstall ]
18. Add Facebook Like to Posts 1.1 [ Uninstall ]
19. Peoplesign 1.6 [ Uninstall ]
20. Optimus Brave 1.8.7 [ Uninstall ]
21. Subject length on Board Index 1.3 [ Uninstall ]
22. SimplePortal 2.3.5 [ Uninstall ]

What kind of process is this that is resulting in the error message? What is happening originally? Some request wants to display the avatar?

As I mentioned apart from that some avatar files are missing, the rest displays fine when browsing the forum.

[Kindred]

It has nothing to do with the avatar.   That's just the trigger -- the problem is that the forum_name_html_safe is not defined

[Arantor]

On the contrary it has everything to do with the avatar. Specifically that the dlattach action doesn't load a bunch of things because it expects to always serve a file, but if you've done certain things it can fall foul that way.

I seem to recall if you take the step of defining a custom avatar folder in 2.0 and have done the migration step but then still use the *old* URL you get a 404 (because the file is not where it expects to find it) but also a bunch of errors because it ends up loading the theme system without half the setup.

But it's been years since I had to look into this, I could be misremembering.

[Sono]

It has nothing to do with the avatar.   That's just the trigger -- the problem is that the forum_name_html_safe is not defined

Becuase it is some silly bot trying to bother with the forum, isn't it? 300MB of error log in 2 days, I don't have that many members that would explain this multitude of requests. 

[pingtoip]

I had encountered a similar situation before. I was able to come up with a solution by painstakingly examining the error_log file.

[Sono]

I had encountered a similar situation before. I was able to come up with a solution by painstakingly examining the error_log file.

And what was the solution?