News:

Want to get involved in developing SMF, then why not lend a hand on our github!

Main Menu

Upgrade fail on large forum RC1 to RC3

Started by chadon, December 18, 2020, 09:56:54 AM

Previous topic - Next topic

chadon

I tried to upgrade the Mustang forum in my signature for the last 36h with 3 attempts but never managed to get it done till the end.

It mostly stops every 30 seconds with a message saying the server is not responding and asking me to try again and it always starts again until it doesn't.

The best I did was to reach about 60 %, till the begining of step 5. That was yesterday and I waited about 12h for it to start again but it never did.

Last night, I started to upgrade again from the begining and right now I am at 55% general progress,  76% step progress with Exécuter : "Cleaning up after old UTF-8 languages" (57 de 69)  in step 4 with the server not responding error and keep clicking on try again but nothing happens since the last 30 mn.

It's a large forum with over a million posts, I already posted here asking help after I had big problems when I upgraded from SMF 2.0 to 2.1 RC1. I was afraid to upgrade again because of that until yesterday because I had enough of all the bugs and the nightmare starts again.

All files from the RC3 version have been uploaded in the server including the French language ones.

What should I do? Should I wait hours or days till the upgrade process starts again by itself and reaches 100%, modify something in SMF's files or could there be something wrong with my VPS server's configuration?

chadon

OK here are the news.

After 896 minutes, I reached the same point as yesterday. My forum is in French so I'm not sure if I will use the correct English terms.

General progess: 64%
Progress of this step: 7%

JSON conversion:

Done 1 out of 13 tables
Current table: "log actions"

It was stuck for about 10 hours yesterday on that table before I started all over again.

The log_actions table has a size of 35.6 MiB.

I don't have any error message or the link to try again so what can I do to get it done? Just wait?

Thanks.

Sir Osis of Liver

You should have a choice of resuming or restarting upgrade.  2.0 upgrader can resume an incomplete upgrade and finish it successfully, I would think 2.1 can do it as well. 
Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

shawnb61

#3
The 2.1 & 2.0 upgrades are almost exactly the same.  2.1 has a utf8 conversion & a json conversion (additional steps) - but the rest is basically the same.

Those "30 seconds" error messages can mean one of two things:
- It's still executing, give it more time
- It died and is never coming back

Unfortunately, it is impossible to know which.  I generally wait at least 10 minutes.  Beyond 10 minutes, I'm fairly sure it died.  (If you know how to look at the processes on your computer, you can tell for certain if it is still working or dead...)

The last thing I'd try is to run the upgrader via the CLI.  The command line is much better at giving you helpful error info when there is a failure.

To do so, go to a command line prompt, navigate to the folder where your settings.php & upgrade.php files are, and issue the following command:
Code:
php upgrade.php

There are some options, which you can query using --help.  You'll see something like this:
Code:
$ php upgrade.php --help
SMF Command-line Upgrader
Usage: /path/to/php -f upgrade.php -- [OPTION]...

        --language=LANG         Reset the forum's language to LANG.
        --no-maintenance        Don't put the forum into maintenance mode.
        --debug                 Output debugging information.
        --backup                Create backups of tables with "backup_" prefix.


Note sometimes your host offers multiple versions of php, so you need to specify the right version when running the command.  You might need to ask your host what the proper path to use is.  If this is the case, your command line would end up looking something like this:
Code:
/opt/php56/bin/php upgrade.php

CLI should give you a much better error to work from.

At first guess, you may have some corrupted JSON strings in log_actions that is causing the upgrade to fail.  But we need to know for sure.  With a proper error message, we will know how to fix the problem.

Hope this helps,
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

shawnb61

Waitaminute...

If you were on RC1, why is it even attempting the JSON conversion again? 

Check out your forum...  You may be done...
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

chadon

Oh you are right. I disabled the maintenance mode with ftp and the forum is showing RC3 on the footer. The home page, section and topic pages all seem to work.

The only issue is that I get a database error message when I try to login as the administrator.

QuoteDatabase Error
Please try again. If you come back to this error screen, report the error to an administrator.

chadon

The error log file is full of this error message:

[18-Dec-2020 20:01:29 Europe/Paris] PHP Warning:  md5_file(/335650_c81155677793117f7543bda495afb40733dae9c9): failed to open stream: No such file or directory in /home/mustvco/public_html/fr/Sources/ShowAttachments.php on line 156

chadon

#7
Here is the result when we click on an attachment link:
https://www.mustangv8.com/fr/index.php?action=dlattach;topic=58146.0;attach=519789

Quote<br />
<b>Notice</b>:  Undefined offset: 27 in <b>/home/mustvco/public_html/fr/Sources/Subs.php</b> on line <b>4451</b><br />
<br />
<b>Warning</b>:  md5_file(/519789_6c35a3a0a0e4d47dc66f24fbe936e7b6692a02f5): failed to open stream: No such file or directory in <b>/home/mustvco/public_html/fr/Sources/ShowAttachments.php</b> on line <b>156</b><br />
<br />
<b>Notice</b>:  Undefined offset: 27 in <b>/home/mustvco/public_html/fr/Sources/Subs.php</b> on line <b>4451</b><br />
<br />
<b>Warning</b>:  md5_file(/519790_2dcb3c9da67ce19d5bfbd728d91d72d68001ef6e.dat): failed to open stream: No such file or directory in <b>/home/mustvco/public_html/fr/Sources/ShowAttachments.php</b> on line <b>182</b><br />
File not found.

shawnb61

Two issues here...

First issue, attachments.  Are you using the RC3 version downloaded from this site?  Or...  Did you download files fresh from GitHub?

Second issue, SMF doesn't realize your JSON conversions had already taken place.  This appears to have been a bug in RC1.  To fix, using phpmyadmin or similar, add the following to your settings table:
   variable: json_done
   value: 1

Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

shawnb61

Regarding the attachment error, I think that somehow your attachment directory is incorrect.  Those are the errors you get if you are pointing to the wrong attachment directory.  Not sure how that would have happened... 

Two ways to fix...  Either take a peek at the directory under Admin | Forum | Attachments and Avatars | Attachment Settings and correct. 

OR... 

Download the latest repair_settings.php and use that to double-check all your directories.  Download from here:
     https://download.simplemachines.org/index.php?tools
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

chadon

"First issue"
I downloaded it from this site just before I did the upgrade.

"Second issue"
Thanks, I just did that and I will try to upgrade again.

About the attachment error , I can't login or access the admin area at the moment but I will see what happens if the upgrade goes all the way threw.

Thank you for your help Shawn.


shawnb61

I don't think you need to run the upgrader again.

Just fix the attachment directories.  If you cannot use the admin panel, then use repair_settings.php.

Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

chadon

I did run the upgrader again last night and it went all the way without a problem.

I also ran repair_settings.php, there was no value in the attachment directory field so I added it but it didn't make any difference.

I'm also still unable to login without getting a database error. I tried to repair the database but with no success.

If you have no suggestions, I think I will be good for a backup restoration and another upgrade.

shawnb61

If you do restore back to 2.1 RC1, make sure you set the value of json_done in the settings table before running the upgrader.  That is definitely part of the problem.

The attachment issue is caused by your attachment directory entry being wrong.  The current version of repair_settings.php (from the download link above) should help you fix it.  I would try that again.  (Don't forget to hit Save!)  Or you could edit it directly using a tool like phpmyadmin in the settings table.  The table is JSON encoded, but it's pretty easy to do if you have a single attachment folder.
  variable: attachmentUploadDir
  value: {"1":"foldername"}

We don't know which error is preventing you from logging on.  I would look in the error log for a database-related error. 
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

chadon

I used the repair_settings.php file from the download link and entered the right path of the attachment directory but it didn't fix the issue.

I checked the settings table and here is the content

{"1":"\/home\/mustvco\/public_html\/fr\/attachments","3":"\/home\/mustvco\/public_html\/fr\/attachments2"}

I'm not sure if this is right but I replaced the default attachments directory by an attachments2 one from the admin pannel of the forum early this year.

QuoteWe don't know which error is preventing you from logging on.  I would look in the error log for a database-related error.
If you are talking about the error log file located in the SMF directory, nothing is generated when I try to log in.

shawnb61

Quote from: ForumMustang.com on December 19, 2020, 12:34:24 PM
{"1":"\/home\/mustvco\/public_html\/fr\/attachments","3":"\/home\/mustvco\/public_html\/fr\/attachments2"}
Are you using one or two directories?  Which one is correct?

If you execute the following in the SQL tab of phpmyadmin, it will tell you which folder is being actually used:
select distinct id_folder FROM smf_attachments
The id_folder will correspond to the # you see in the attachmentUploadDir you see above.

This also helps you tie everything together...  The current folder in use is ID'd here:
SELECT *  FROM smf_settings WHERE variable LIKE '%attachmentupload%'

Quote from: ForumMustang.com on December 19, 2020, 12:34:24 PM
If you are talking about the error log file located in the SMF directory, nothing is generated when I try to log in.
No, I mean in the SMF log_errors table.  Since you cannot access the ACP, you may need to look at the entries via phpmyadmin.
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

chadon

QuoteAre you using one or two directories?  Which one is correct?
The "attachments" directory is the one I used for years. It is still used by the forum to access the old attachments but new uploads since a year are sent to the "attachments2" directory.

QuoteIf you execute the following in the SQL tab of phpmyadmin, it will tell you which folder is being actually used:
It shows me a list of 26 rows.

Yes, I should have said that the attachments2 directory was set to have a new subdirectory each month. I just checked with ftp and this was set up like that since february 2019 so I must in fact be using 26 directories. I did that because the forum has XXX thousands of attachments and it was unmanageable to have them in one directory.
That might explain why repair_settings didn't fix the issue but if so, what should I enter?

QuoteThis also helps you tie everything together...  The current folder in use is ID'd here:
This one:
{"1":"\/home\/mustvco\/public_html\/fr\/attachments2"}

That's what I entered with the repair_setting file but the files are in subdirectories like for example:
attachments2/2019/01/file.jpg

QuoteNo, I mean in the SMF log_errors table.  Since you cannot access the ACP, you may need to look at the entries via phpmyadmin.
The table is empty:
QuoteMySQL returned an empty result set (i.e. zero rows). (Query took 0.0003 seconds.)


shawnb61

Do you have a way to get the contents of that field from BEFORE the upgrade?

Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

chadon

I have a backup on my computer but I don't know how to read it from there.

shawnb61

If the backup ends in .sql, you can just open it in Notepad++, and search for "attachments2".  I want to determine if it just has the 2 entries like we see above, or, if it has 26 entries.  I suspect the latter.  We may be able to just copy & paste the text from the old backup. 


Regarding the logon issue...  One thing to check...  If you look at the structure of the smf_members table, what is the length of the password_salt field?   You can use the Structure tab in phpmyadmin.

(It should look like the attached...)
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Advertisement: