Advertisement:

Author Topic: Upgrade fail on large forum RC1 to RC3  (Read 2194 times)

Offline ForumMustang.com

  • Full Member
  • ***
  • Posts: 521
  • Gender: Male
  • SMF French Translator
    • forummustang on Facebook
    • Ford Mustang Mach E
Upgrade fail on large forum RC1 to RC3
« on: December 18, 2020, 09:56:54 AM »
I tried to upgrade the Mustang forum in my signature for the last 36h with 3 attempts but never managed to get it done till the end.

It mostly stops every 30 seconds with a message saying the server is not responding and asking me to try again and it always starts again until it doesn't.

The best I did was to reach about 60 %, till the begining of step 5. That was yesterday and I waited about 12h for it to start again but it never did.

Last night, I started to upgrade again from the begining and right now I am at 55% general progress,  76% step progress with Exécuter : "Cleaning up after old UTF-8 languages" (57 de 69)  in step 4 with the server not responding error and keep clicking on try again but nothing happens since the last 30 mn.

It's a large forum with over a million posts, I already posted here asking help after I had big problems when I upgraded from SMF 2.0 to 2.1 RC1. I was afraid to upgrade again because of that until yesterday because I had enough of all the bugs and the nightmare starts again.

All files from the RC3 version have been uploaded in the server including the French language ones.

What should I do? Should I wait hours or days till the upgrade process starts again by itself and reaches 100%, modify something in SMF's files or could there be something wrong with my VPS server's configuration?

Offline ForumMustang.com

  • Full Member
  • ***
  • Posts: 521
  • Gender: Male
  • SMF French Translator
    • forummustang on Facebook
    • Ford Mustang Mach E
Re: Upgrade fail on large forum RC1 to RC3
« Reply #1 on: December 18, 2020, 12:17:52 PM »
OK here are the news.

After 896 minutes, I reached the same point as yesterday. My forum is in French so I'm not sure if I will use the correct English terms.

General progess: 64%
Progress of this step: 7%

JSON conversion:

Done 1 out of 13 tables
Current table: "log actions"

It was stuck for about 10 hours yesterday on that table before I started all over again.

The log_actions table has a size of 35.6 MiB.

I don't have any error message or the link to try again so what can I do to get it done? Just wait?

Thanks.

Offline Sir Osis of Liver

  • SMF Super Hero
  • *******
  • Posts: 11,123
  • We were all equal in the end.
Re: Upgrade fail on large forum RC1 to RC3
« Reply #2 on: December 18, 2020, 12:26:11 PM »
You should have a choice of resuming or restarting upgrade.  2.0 upgrader can resume an incomplete upgrade and finish it successfully, I would think 2.1 can do it as well. 
“The best laid schemes o' mice an' men / Gang aft a-gley.” - Robert Burns

Offline shawnb61

  • Developer
  • SMF Hero
  • *
  • Posts: 3,310
    • sbulen on GitHub
Re: Upgrade fail on large forum RC1 to RC3
« Reply #3 on: December 18, 2020, 12:48:41 PM »
The 2.1 & 2.0 upgrades are almost exactly the same.  2.1 has a utf8 conversion & a json conversion (additional steps) - but the rest is basically the same.

Those "30 seconds" error messages can mean one of two things:
 - It's still executing, give it more time
 - It died and is never coming back

Unfortunately, it is impossible to know which.  I generally wait at least 10 minutes.  Beyond 10 minutes, I'm fairly sure it died.  (If you know how to look at the processes on your computer, you can tell for certain if it is still working or dead...)

The last thing I'd try is to run the upgrader via the CLI.  The command line is much better at giving you helpful error info when there is a failure.

To do so, go to a command line prompt, navigate to the folder where your settings.php & upgrade.php files are, and issue the following command:
Code:
Code: [Select]
php upgrade.php
There are some options, which you can query using --help.  You'll see something like this:
Code:
Code: [Select]
$ php upgrade.php --help
SMF Command-line Upgrader
Usage: /path/to/php -f upgrade.php -- [OPTION]...

        --language=LANG         Reset the forum's language to LANG.
        --no-maintenance        Don't put the forum into maintenance mode.
        --debug                 Output debugging information.
        --backup                Create backups of tables with "backup_" prefix.

Note sometimes your host offers multiple versions of php, so you need to specify the right version when running the command.  You might need to ask your host what the proper path to use is.  If this is the case, your command line would end up looking something like this:
Code:
Code: [Select]
/opt/php56/bin/php upgrade.php
CLI should give you a much better error to work from.

At first guess, you may have some corrupted JSON strings in log_actions that is causing the upgrade to fail.  But we need to know for sure.  With a proper error message, we will know how to fix the problem.

Hope this helps,
« Last Edit: December 18, 2020, 01:12:17 PM by shawnb61 »
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline shawnb61

  • Developer
  • SMF Hero
  • *
  • Posts: 3,310
    • sbulen on GitHub
Re: Upgrade fail on large forum RC1 to RC3
« Reply #4 on: December 18, 2020, 01:19:34 PM »
Waitaminute...

If you were on RC1, why is it even attempting the JSON conversion again? 

Check out your forum...  You may be done...
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline ForumMustang.com

  • Full Member
  • ***
  • Posts: 521
  • Gender: Male
  • SMF French Translator
    • forummustang on Facebook
    • Ford Mustang Mach E
Re: Upgrade fail on large forum RC1 to RC3
« Reply #5 on: December 18, 2020, 01:52:21 PM »
Oh you are right. I disabled the maintenance mode with ftp and the forum is showing RC3 on the footer. The home page, section and topic pages all seem to work.

The only issue is that I get a database error message when I try to login as the administrator.

Quote
Database Error
Please try again. If you come back to this error screen, report the error to an administrator.

Offline ForumMustang.com

  • Full Member
  • ***
  • Posts: 521
  • Gender: Male
  • SMF French Translator
    • forummustang on Facebook
    • Ford Mustang Mach E
Re: Upgrade fail on large forum RC1 to RC3
« Reply #6 on: December 18, 2020, 02:05:19 PM »
The error log file is full of this error message:

Code: [Select]
[18-Dec-2020 20:01:29 Europe/Paris] PHP Warning:  md5_file(/335650_c81155677793117f7543bda495afb40733dae9c9): failed to open stream: No such file or directory in /home/mustvco/public_html/fr/Sources/ShowAttachments.php on line 156

Offline ForumMustang.com

  • Full Member
  • ***
  • Posts: 521
  • Gender: Male
  • SMF French Translator
    • forummustang on Facebook
    • Ford Mustang Mach E
Re: Upgrade fail on large forum RC1 to RC3
« Reply #7 on: December 18, 2020, 02:11:40 PM »
Here is the result when we click on an attachment link:
https://www.mustangv8.com/fr/index.php?action=dlattach;topic=58146.0;attach=519789

Quote
<br />
<b>Notice</b>:  Undefined offset: 27 in <b>/home/mustvco/public_html/fr/Sources/Subs.php</b> on line <b>4451</b><br />
<br />
<b>Warning</b>:  md5_file(/519789_6c35a3a0a0e4d47dc66f24fbe936e7b6692a02f5): failed to open stream: No such file or directory in <b>/home/mustvco/public_html/fr/Sources/ShowAttachments.php</b> on line <b>156</b><br />
<br />
<b>Notice</b>:  Undefined offset: 27 in <b>/home/mustvco/public_html/fr/Sources/Subs.php</b> on line <b>4451</b><br />
<br />
<b>Warning</b>:  md5_file(/519790_2dcb3c9da67ce19d5bfbd728d91d72d68001ef6e.dat): failed to open stream: No such file or directory in <b>/home/mustvco/public_html/fr/Sources/ShowAttachments.php</b> on line <b>182</b><br />
File not found.
« Last Edit: December 18, 2020, 03:04:12 PM by ForumMustang.com »

Offline shawnb61

  • Developer
  • SMF Hero
  • *
  • Posts: 3,310
    • sbulen on GitHub
Re: Upgrade fail on large forum RC1 to RC3
« Reply #8 on: December 18, 2020, 08:16:31 PM »
Two issues here...

First issue, attachments.  Are you using the RC3 version downloaded from this site?  Or...  Did you download files fresh from GitHub?

Second issue, SMF doesn't realize your JSON conversions had already taken place.  This appears to have been a bug in RC1.  To fix, using phpmyadmin or similar, add the following to your settings table:
   variable: json_done
   value: 1

Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline shawnb61

  • Developer
  • SMF Hero
  • *
  • Posts: 3,310
    • sbulen on GitHub
Re: Upgrade fail on large forum RC1 to RC3
« Reply #9 on: December 18, 2020, 08:36:58 PM »
Regarding the attachment error, I think that somehow your attachment directory is incorrect.  Those are the errors you get if you are pointing to the wrong attachment directory.  Not sure how that would have happened... 

Two ways to fix...  Either take a peek at the directory under Admin | Forum | Attachments and Avatars | Attachment Settings and correct. 

OR... 

Download the latest repair_settings.php and use that to double-check all your directories.  Download from here:
     https://download.simplemachines.org/index.php?tools
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline ForumMustang.com

  • Full Member
  • ***
  • Posts: 521
  • Gender: Male
  • SMF French Translator
    • forummustang on Facebook
    • Ford Mustang Mach E
Re: Upgrade fail on large forum RC1 to RC3
« Reply #10 on: December 18, 2020, 08:54:20 PM »
"First issue"
I downloaded it from this site just before I did the upgrade.

"Second issue"
Thanks, I just did that and I will try to upgrade again.

About the attachment error , I can't login or access the admin area at the moment but I will see what happens if the upgrade goes all the way threw.

Thank you for your help Shawn.


Offline shawnb61

  • Developer
  • SMF Hero
  • *
  • Posts: 3,310
    • sbulen on GitHub
Re: Upgrade fail on large forum RC1 to RC3
« Reply #11 on: December 19, 2020, 03:37:35 AM »
I don’t think you need to run the upgrader again.

Just fix the attachment directories.  If you cannot use the admin panel, then use repair_settings.php.

Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline ForumMustang.com

  • Full Member
  • ***
  • Posts: 521
  • Gender: Male
  • SMF French Translator
    • forummustang on Facebook
    • Ford Mustang Mach E
Re: Upgrade fail on large forum RC1 to RC3
« Reply #12 on: December 19, 2020, 10:34:07 AM »
I did run the upgrader again last night and it went all the way without a problem.

I also ran repair_settings.php, there was no value in the attachment directory field so I added it but it didn't make any difference.

I'm also still unable to login without getting a database error. I tried to repair the database but with no success.

If you have no suggestions, I think I will be good for a backup restoration and another upgrade.

Offline shawnb61

  • Developer
  • SMF Hero
  • *
  • Posts: 3,310
    • sbulen on GitHub
Re: Upgrade fail on large forum RC1 to RC3
« Reply #13 on: December 19, 2020, 11:33:46 AM »
If you do restore back to 2.1 RC1, make sure you set the value of json_done in the settings table before running the upgrader.  That is definitely part of the problem.

The attachment issue is caused by your attachment directory entry being wrong.  The current version of repair_settings.php (from the download link above) should help you fix it.  I would try that again.  (Don't forget to hit Save!)  Or you could edit it directly using a tool like phpmyadmin in the settings table.  The table is JSON encoded, but it's pretty easy to do if you have a single attachment folder.
  variable: attachmentUploadDir
  value: {"1":"foldername"}

We don't know which error is preventing you from logging on.  I would look in the error log for a database-related error. 
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline ForumMustang.com

  • Full Member
  • ***
  • Posts: 521
  • Gender: Male
  • SMF French Translator
    • forummustang on Facebook
    • Ford Mustang Mach E
Re: Upgrade fail on large forum RC1 to RC3
« Reply #14 on: December 19, 2020, 12:34:24 PM »
I used the repair_settings.php file from the download link and entered the right path of the attachment directory but it didn't fix the issue.

I checked the settings table and here is the content

Code: [Select]
{"1":"\/home\/mustvco\/public_html\/fr\/attachments","3":"\/home\/mustvco\/public_html\/fr\/attachments2"}
I'm not sure if this is right but I replaced the default attachments directory by an attachments2 one from the admin pannel of the forum early this year.

Quote
We don't know which error is preventing you from logging on.  I would look in the error log for a database-related error.
If you are talking about the error log file located in the SMF directory, nothing is generated when I try to log in.

Offline shawnb61

  • Developer
  • SMF Hero
  • *
  • Posts: 3,310
    • sbulen on GitHub
Re: Upgrade fail on large forum RC1 to RC3
« Reply #15 on: December 19, 2020, 02:26:10 PM »
Code: [Select]
{"1":"\/home\/mustvco\/public_html\/fr\/attachments","3":"\/home\/mustvco\/public_html\/fr\/attachments2"}
Are you using one or two directories?  Which one is correct?

If you execute the following in the SQL tab of phpmyadmin, it will tell you which folder is being actually used:
Code: [Select]
select distinct id_folder FROM smf_attachmentsThe id_folder will correspond to the # you see in the attachmentUploadDir you see above.

This also helps you tie everything together...  The current folder in use is ID'd here:
Code: [Select]
SELECT *  FROM smf_settings WHERE variable LIKE '%attachmentupload%'
If you are talking about the error log file located in the SMF directory, nothing is generated when I try to log in.
No, I mean in the SMF log_errors table.  Since you cannot access the ACP, you may need to look at the entries via phpmyadmin.
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline ForumMustang.com

  • Full Member
  • ***
  • Posts: 521
  • Gender: Male
  • SMF French Translator
    • forummustang on Facebook
    • Ford Mustang Mach E
Re: Upgrade fail on large forum RC1 to RC3
« Reply #16 on: December 19, 2020, 04:24:10 PM »
Quote
Are you using one or two directories?  Which one is correct?
The "attachments" directory is the one I used for years. It is still used by the forum to access the old attachments but new uploads since a year are sent to the "attachments2" directory.

Quote
If you execute the following in the SQL tab of phpmyadmin, it will tell you which folder is being actually used:
It shows me a list of 26 rows.

Yes, I should have said that the attachments2 directory was set to have a new subdirectory each month. I just checked with ftp and this was set up like that since february 2019 so I must in fact be using 26 directories. I did that because the forum has XXX thousands of attachments and it was unmanageable to have them in one directory.
That might explain why repair_settings didn't fix the issue but if so, what should I enter?

Quote
This also helps you tie everything together...  The current folder in use is ID'd here:
This one:
Code: [Select]
{"1":"\/home\/mustvco\/public_html\/fr\/attachments2"}
That's what I entered with the repair_setting file but the files are in subdirectories like for example:
attachments2/2019/01/file.jpg

Quote
No, I mean in the SMF log_errors table.  Since you cannot access the ACP, you may need to look at the entries via phpmyadmin.
The table is empty:
Quote
MySQL returned an empty result set (i.e. zero rows). (Query took 0.0003 seconds.)


Offline shawnb61

  • Developer
  • SMF Hero
  • *
  • Posts: 3,310
    • sbulen on GitHub
Re: Upgrade fail on large forum RC1 to RC3
« Reply #17 on: December 19, 2020, 05:31:46 PM »
Do you have a way to get the contents of that field from BEFORE the upgrade?

Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Offline ForumMustang.com

  • Full Member
  • ***
  • Posts: 521
  • Gender: Male
  • SMF French Translator
    • forummustang on Facebook
    • Ford Mustang Mach E
Re: Upgrade fail on large forum RC1 to RC3
« Reply #18 on: December 19, 2020, 05:37:19 PM »
I have a backup on my computer but I don't know how to read it from there.

Offline shawnb61

  • Developer
  • SMF Hero
  • *
  • Posts: 3,310
    • sbulen on GitHub
Re: Upgrade fail on large forum RC1 to RC3
« Reply #19 on: December 19, 2020, 05:42:58 PM »
If the backup ends in .sql, you can just open it in Notepad++, and search for "attachments2".  I want to determine if it just has the 2 entries like we see above, or, if it has 26 entries.  I suspect the latter.  We may be able to just copy & paste the text from the old backup. 


Regarding the logon issue...  One thing to check...  If you look at the structure of the smf_members table, what is the length of the password_salt field?   You can use the Structure tab in phpmyadmin.

(It should look like the attached...)
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp