Upgrade fail on large forum RC1 to RC3

[chadon]

No but there is another big issue. When I browse files and try to sort them by date our click on other pages, I get a white page with this content:

Forbidden
You don't have permission to access this resource.

Additionally, a 403 Forbidden error was encountered while trying to use an ErrorDocument to handle the request.

When I read my private messages, I see the last page but I get the same Forbiden error when I changes pages or when I want to send a private message.

[shawnb61]

Are there other changes underway?  Did your host make changes?  What happened in addition to an RC3 upgrade?

If you do a quick search, that is most often associated with a host turning on an overaggressive mod_security rule, e.g.:
https://www.simplemachines.org/community/index.php?topic=572656.0

To fix, you normally have to ask your host to disable that...

[shawnb61]

Back to the attachments issue, I wrote up a little doc to help you audit your attachment folders. 

This should help you understand which id_folder should be associated with which directory on disk, and how that relates to the list of folders you see in the ACP. 

This should help you ID the 36K attachments in the "wrong folder". 

I would look at each physical folder on disk, find an attachment ID for a doc in that folder, and use that to tell me which id_folder that should be associated with in the ACP.

[chadon]

Are there other changes underway?  Did your host make changes?  What happened in addition to an RC3 upgrade?

If you do a quick search, that is most often associated with a host turning on an overaggressive mod_security rule, e.g.:
https://www.simplemachines.org/community/index.php?topic=572656.0

To fix, you normally have to ask your host to disable that...

lol Do you have a solution for everything? I disabled mod_security and the problem is solved. But is it safe to run without it?

Back to the attachments issue, I wrote up a little doc to help you audit your attachment folders. 

This should help you understand which id_folder should be associated with which directory on disk, and how that relates to the list of folders you see in the ACP. 

This should help you ID the 36K attachments in the "wrong folder". 

I would look at each physical folder on disk, find an attachment ID for a doc in that folder, and use that to tell me which id_folder that should be associated with in the ACP.

That looks more complicated, I'm going to read that. Thank you.

[chadon]

OK if I understood right, here are my discoveries.

I have the folder "attachments2/2020/05" with files missing in the ACP.
It's the 19th folder in the ACP.
In the smf_attachments table, the folder ID 19 has minatt: 491405 and maxatt: 496843
In the physical folder "attachments2/2020/05" via ftp, the file names start with: 491405_.... but ends with the file name: 500981_...

If I understand right, there shouldn't be any file name with numbers higher than 496843. How is that possible? It's not like files were moved or renamed during the upgrade?

[shawnb61]

I have no idea how or when that would have happened...

What I would do is see if moving one or two of those to the right folder fixes it.  Find what message 500981 is used in.  View the message.  Move that one file to the proper folder # (you can confirm the folder # on the attachment record).  View the message again...  If the attachment then appears properly, you're good.

If it works, move more, spot checking & confirming along the way.

I don't know of any process that moves the files or changes the folder ids.  Pretty sure the upgrader would not do that.   (Though I'm going to spend some time trying to duplicate this....)

You can navigate directly to a message easily by adding:
/index.php?msg=56765

...once you know the message #.

[chadon]

I don't know of any process that moves the files or changes the folder ids.  Pretty sure the upgrader would not do that.

Wouldn't the upgrader mess up the smf_attachments table and change the value of maxatt?

If I replace the value of maxatt 496843 with 500981 in the table, wouldn't it work? It would at least be easier done than moving files and checking where they have to go.

[chadon]

I have these reports that don't look good in the ACP Error log:

Type d'erreur: Base de données (database)
Database Error: Unknown column 'filename' in 'field list'


SELECT code, filename, description
FROM smf_smileys https://www.mustangv8.com/fr/index.php?/trackback/
/home/mustvco/public_html/forum/Sources/Subs.php (Ligne 2489)

And

8192: Function create_function() is deprecated https://www.mustangv8.com/fr/index.php?/trackback/
/home/mustvco/public_html/forum/Sources/Subs.php (Ligne 1305)   
Invité
Aujourd'hui à 11:58:00
209.90.225.218
5e71eecdd0efb53c28f8f482178c5dc7
Informations de traçage
Type d'erreur: Générale
8192: Function create_function() is deprecated https://www.mustangv8.com/fr/index.php?/trackback/
/home/mustvco/public_html/forum/Sources/Subs.php (Ligne 1225)   
Invité
Aujourd'hui à 11:58:00
209.90.225.218
5e71eecdd0efb53c28f8f482178c5dc7
Informations de traçage
Type d'erreur: Générale
8192: Function create_function() is deprecated https://www.mustangv8.com/fr/index.php?/trackback/
/home/mustvco/public_html/forum/Sources/Subs.php (Ligne 1214)

[chadon]

Here is the structure of the smf_attachments table in the attached image.

The id folders with missing images in the ACP panel are:

2
19
20
21
25
26

The id folder 1 is the base directory but what is the id folder 27? I only have 26 in the ACP.

The id folder 2 and 3 have 0 files when I view them with ftp.

It seems to me that the database should be cleaned instead of moving files, doesn't it?

[shawnb61]

Maxatt doesnt exist in the db, its calculated based on values found.

I do have a theory why/how the upgrader might do this.  Hint: it uses json_done to determine whether attachments have been renamed yet.  This might be a byproduct of that missing flag.

I see two broad paths for what to do next....    Restore & rerun the upgrader (fixing known issues first, eg, setting json_done & increasing the length of password_salt).  We would have fewer issues to clean up.

Or, try to fix these errors.  Regarding attachments, to align id_folder with the folder on disk, we either need to move the file or change id_folder.  Either would need to be done by hand.  If you have a gui like File Manager, moving the files would be trivial.

[chadon]

If I restore and upgrade without the json step, are you sure it won't mess up the attachments again?

I don't know what is a gui or how to use one so I would have to do it by hand. I think I'd rather restore and upgrade but not if there is still a chance to mess up the attachments.

[Illori]

GUI is a graphical user interface, rather then using a command line to do the task.

[chadon]

I have a co-administrator who did an upgrade with the forum's database in local and he also had issues with MySQL but it went smoothly with MariaDB.
I have the option to do a MySQL/MariaDB Upgrade on my server, should I try that before to do a new upgrade? I never used MariaDB and don't know anything about it.

[shawnb61]

If I restore and upgrade without the json step, are you sure it won't mess up the attachments again?

No.  There is a chance we still have the attachments issue.  We don't know root cause yet.  That will take time.

I suspect the create_function warnings are due to a mod, pretty sure there are none in 2.1.

If you wanted to try to update id_folder in the attachments table, these queries might help:

To update one:

Code Select Expand

update smf_attachments set id_folder = 11 where id_attach = 12

To update a range:

Code Select Expand

update smf_attachments set id_folder = 8 where id_attach >= 10 and id_attach <= 12

And you can confirm you did it correctly by rerunning:

Code Select Expand

SELECT id_folder, min(id_attach) as minatt, max(id_attach) as maxatt FROM smf_attachments GROUP BY id_folder ORDER BY id_folder


BACKUPS... BACKUPS... BACKUPS...  It's easy to type a typo...  Not all mistakes will be reversible.

[shawnb61]

I have a co-administrator who did an upgrade with the forum's database in local and he also had issues with MySQL but it went smoothly with MariaDB.
I have the option to do a MySQL/MariaDB Upgrade on my server, should I try that before to do a new upgrade? I never used MariaDB and don't know anything about it.

No.  There are some known issues with MariaDB in SMF that do not exist with MySQL.

These are not MySQL issues.  We shouldn't add another layer of changes & potential problems. 

[shawnb61]

I would attempt to fix the id_folders.  Give that a few hours of effort.  If it fails, fall back to the restore/upgrade. 

[chadon]

I suspect the create_function warnings are due to a mod, pretty sure there are none in 2.1.

I had mods before the upgrade but none have been installed since.

I would attempt to fix the id_folders.  Give that a few hours of effort.  If it fails, fall back to the restore/upgrade. 

OK, I will try that, thanks.

[chadon]

I just discovered that one of my attempts to upgrade created 8 backup tables in the database and one is the attachments one and the structure is how it should be. How should I do to restore it?
I tried te rename the    smf_attachments by smf_attachments2 and rename backup_smf_attachments by    smf_attachments but there is no change in the ACP.
I checked the content of smf_attachments and it's how it's supposed to be, I cleared the cache from the ACP
What am I missing?

[albertlast]

8192: Function create_function() is deprecated https://www.mustangv8.com/fr/index.php?/trackback/
/home/mustvco/public_html/forum/Sources/Subs.php (Ligne 1305)   
Invité
Aujourd'hui à 11:58:00
209.90.225.218
5e71eecdd0efb53c28f8f482178c5dc7
Informations de traçage
Type d'erreur: Générale
8192: Function create_function() is deprecated https://www.mustangv8.com/fr/index.php?/trackback/
/home/mustvco/public_html/forum/Sources/Subs.php (Ligne 1225)   
Invité
Aujourd'hui à 11:58:00
209.90.225.218
5e71eecdd0efb53c28f8f482178c5dc7
Informations de traçage
Type d'erreur: Générale
8192: Function create_function() is deprecated https://www.mustangv8.com/fr/index.php?/trackback/
/home/mustvco/public_html/forum/Sources/Subs.php (Ligne 1214)

could you upload your Subs.php?

[chadon]

Oh! The problem is coming from the path but how is it possible?

The path in the error is showing:
/home/mustvco/public_html/forum/Sources/Subs.php
It should be :
/home/mustvco/public_html/fr/Sources/Subs.php

In the "forum" directory it's an old test forum running under SMF 2.0.15

I just tried to run repair_settings.php but everything is set with the "fr" directory".