News:

Want to get involved in developing SMF, then why not lend a hand on our github!

Main Menu

SMF 2.0 RC2 Upgrading waiting at log_search_subjects - Step 6: Converting to UTF

Started by Sudhakar Arjunan, April 29, 2020, 09:56:16 AM

Previous topic - Next topic

Sudhakar Arjunan

SMF 2.0 RC2 Upgrading waiting  - Step 6: Converting to UTF - for several house at log_search_subjects

After a long time, i have decided to upgrade 2.1 RC2 from github as it was 98% completion except few policies and gdpr information's.
Even i have waited for 2 years for an upgrade to 2.1 Final Stable which may take some time now.

Everything went good on upgrade from 2.1 Beta 1 to 2.1 RC2.
Struck at the same issue as this topic, where in Step 6: Converting to UTF-8

Shows completed 128 out of 224 tables.
Current Table: "log_search_subjects"

Where smf_log_search_words has -5,477,880 rows - type - InnoDB - Size - 150 mb

Time Elapsed : 331 Mins, 25 seconds. - while posting in other reply
Time Elapsed : 708 Mins, 14 seconds. - now after 12 hours of upgrade on a dedicated server which has ssd , 64 gb ram , 40 core processor with cloud linux uses myisam ( mysql ) with php myadmin.
Working on New Mods & Themes for SMF... Will update soon... My Blog page
My Smf forum : Discuss ITAcumens :: My SMF Forum


vbgamer45

I saw this in another post backup up your tables first
https://www.simplemachines.org/community/index.php?topic=572888.msg4053543;topicseen#msg4053543

But you can truncate log_search_* tables then rebuild the index from what it looks like.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Sudhakar Arjunan

Working on New Mods & Themes for SMF... Will update soon... My Blog page
My Smf forum : Discuss ITAcumens :: My SMF Forum

Sudhakar Arjunan

Quote from: vbgamer45 on April 29, 2020, 10:11:05 AM
I saw this in another post backup up your tables first
https://www.simplemachines.org/community/index.php?topic=572888.msg4053543;topicseen#msg4053543

But you can truncate log_search_* tables then rebuild the index from what it looks like.

Thank you for the knowledge, can you tell me how do i code that on upgrade.php to just skip only log_search_tables instead of truncating them.
Or do you think cut the table until the upgrade is completed.
Working on New Mods & Themes for SMF... Will update soon... My Blog page
My Smf forum : Discuss ITAcumens :: My SMF Forum

vbgamer45

You can truncate the tables via phpmyadmin Always backup first! Easy way.

I don't suggest modifying upgrade system to skip that upgrade.
Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

Sudhakar Arjunan

Quote from: vbgamer45 on April 29, 2020, 10:38:52 AM
You can truncate the tables via phpmyadmin Always backup first! Easy way.

I don't suggest modifying upgrade system to skip that upgrade.

Sure i will try it then, but if i truncate. Will the upgrader does not wait for that table. Are we sure.
Working on New Mods & Themes for SMF... Will update soon... My Blog page
My Smf forum : Discuss ITAcumens :: My SMF Forum

vbgamer45

Community Suite for SMF - Take your forum to the next level built for SMF, Gallery,Store,Classifieds,Downloads,more!

SMFHacks.com -  Paid Modifications for SMF

Mods:
EzPortal - Portal System for SMF
SMF Gallery Pro
SMF Store SMF Classifieds Ad Seller Pro

shawnb61

I would just rerun the upgrader.  It is safe to rerun it. 

5M rows in log_search_words is not very big at all.  The utf8 conversion should just take a few minutes.  It appears the server quit for some reason.  I would search all the error/apache logs very carefully for clues. 

In my experience, rebuilding the search index takes ***much*** longer than the UTF8 conversion...   (On my forum, with ~12M rows & 325Mb in log_search_words, the utf8 conversion step for the entire forum takes ~3 minutes.  I have no idea how long rebuilding the search index takes - when I have tried that, I have had to let it run overnight...)

Note that the utf8 conversion is the very last step.  If a rerun does not work, you could complete the utf8 conversion manually.  The SMF upgrader uses a very safe method to convert to UTF8, though, that takes extra measures to avoid double-encoding...  I would only do it by hand after a couple more upgrader attempts.  If you do it by hand, you must also set the $db_character_set and global_character_set settings manually. 
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Sudhakar Arjunan

HI Shawnb61, thank you for the detailed explanation. Waited for so much time and took a backup of smf_log_search_words in to other database and renamed the table log_search_words so the upgrader skipped the table as per vbgamer45's advice

Time Elapsed: 865 mins, 52 seconds. ( more than 14 hours spent with 4-5 refresh every 2-3 hours ).

Converting to UTF8

Please wait while your database is converted to UTF-8. For large forums this may take some time!
Completed 128 out of 223 tables.
Current Table: "log_search_subjects"

I hope smf_log_search_subjects has only 9 lakh rows and 30 mb sql file table. Let me wait and see if now, i will check for error / apache logs for any clues and try to increase db run time size to monitor its happening and report here shawnb61.

For community reference, added before and after screenshot attached.

thank you , lets hope the for the best.
Working on New Mods & Themes for SMF... Will update soon... My Blog page
My Smf forum : Discuss ITAcumens :: My SMF Forum

Kindred

Quote from: Sudhakar Arjunan on April 29, 2020, 10:27:17 AM
Quote from: Arantor on April 29, 2020, 10:09:40 AM
Making a new topic isn't going to get an answer quicker...

https://www.simplemachines.org/community/index.php?topic=572888.msg4054181#msg4054181

Waiting for 6 hours before creating a new one , am sorry.

well, as it turns out, you posted in someone else's thread, which was already marked as solved.

However, "waiting 6 hours"??    We're not a real-time support site.   We're all volunteers. We ask that people wait AT LEAST 24 hours before even BUMPING a post, let alone posting a new one.
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

shawnb61

Those are tiny tables.  5M rows is nothing for log_search_words (it loosely corresponds to words used everywhere; log_search_subjects loosely translates to a count of distinct words used). 

There is something else wrong.  This should be done in minutes, seriously. 

Some random thoughts, if you are still experiencing issues:

1.  DB is not configured for InnoDB?  What are the InnoDB buffer pool sizes?

2.  Permissions?  Does your DB account have all permissions granted to do everything to the DB?  There is a lot of table manipulation going on. 

3.  Something in the content is really confusing?  What are the collations for the log_search_* tables before starting?  What are the primary languages used in the forum?  If you browse log_search_words and log_search_subjects, are they readable?
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

shawnb61

Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Antes


Arantor


Sudhakar Arjunan

Quote from: Kindred on April 29, 2020, 02:38:04 PM

well, as it turns out, you posted in someone else's thread, which was already marked as solved.

However, "waiting 6 hours"??    We're not a real-time support site.   We're all volunteers. We ask that people wait AT LEAST 24 hours before even BUMPING a post, let alone posting a new one.

Am Sorry ! My sincerely apologies Kindred & all volunteers !!

I have noticed that recent communication after posting on resolved topic.

So i have opened up a new topic to bring moderators attention as my server resources, time and fourm is down for last 1 week with null issue on new server where i cannot make post and reply topic.
Working on New Mods & Themes for SMF... Will update soon... My Blog page
My Smf forum : Discuss ITAcumens :: My SMF Forum

Sudhakar Arjunan

Dear Team,

Step 6: Convert to UTF-8
Step 7: Delete Upgrade.php

After completing the step 6 by deleting the files log search words and log search subjects.

========Converting to UTF8Please wait while your database is converted to UTF-8. For large forums this may take some time!Completed 222 out of 222 tables.Conversion Complete! Click Continue to Proceed.========

Step 7: Delete Upgrade.php

click continue - error -

This page isn't working www.forumname.com is currently unable to handle this request.
HTTP ERROR 500

Error Log shows this,

[04-May-2020 04:34:07 Asia/Kolkata] PHP Notice:  Undefined index: htmlspecialchars in /home/website/public_html/community/Sources/ScheduledTasks.php on line 1022
[04-May-2020 04:34:07 Asia/Kolkata] PHP Fatal error:  Function name must be a string in /home/website/public_html/community/Sources/ScheduledTasks.php on line 1022

but forum page is loading with guest access to topics and boards but cannot login to admin account or any account -

Database Error
Please try again. If you come back to this error screen, report the error to an administrator.
Working on New Mods & Themes for SMF... Will update soon... My Blog page
My Smf forum : Discuss ITAcumens :: My SMF Forum

shawnb61

Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Sudhakar Arjunan

Quote from: shawnb61 on May 03, 2020, 07:15:46 PM
php version?
mysql version?
Does settings.php look intact?

Thank you , Settings.php file is excellent. All details are perfect.

Apache version - 2.4.43 ( but using litespeed )
php version - 7.2 ( 5.6 and 7,2 both tried )
mysql version - 5.7.29-cll-lve
Architecture   x86_64
Operating System   linux

forum.com is live
can see all the post

cannot login to admin accounts

Also upgrade.php went till step 6 and i tried with this option also

Note: If necessary the above security check can be bypassed for users who may administrate a server but not have admin rights on the forum. In order to bypass the above check simply open "upgrade.php" in a text editor and replace "$disable_security = false;" with "$disable_security = true;" and refresh this page.

changed false to true and no answer
Also php 7.2 - cannot complete the page, php 5.6 - cannot complete the page
Working on New Mods & Themes for SMF... Will update soon... My Blog page
My Smf forum : Discuss ITAcumens :: My SMF Forum

Sudhakar Arjunan

My error log for your reference, cannot delete upgrade.php after step 6 is over.

Stack trace:
#0 /home/website/public_html/community/Sources/ScheduledTasks.php(1071): loadEssentialThemeData()
#1 /home/website/public_html/community/upgrade.php(1739): scheduled_fetchSMfiles()
#2 /home/website/public_html/community/upgrade.php(332): DeleteUpgrade()
#3 {main}
  thrown in /home/website/public_html/community/Sources/ScheduledTasks.php on line 1022
[04-May-2020 06:03:05 Asia/Kolkata] PHP Notice:  Undefined index: htmlspecialchars in /home/website/public_html/community/Sources/ScheduledTasks.php on line 1022
[04-May-2020 06:03:05 Asia/Kolkata] PHP Fatal error:  Function name must be a string in /home/website/public_html/community/Sources/ScheduledTasks.php on line 1022
[04-May-2020 06:04:11 Asia/Kolkata] PHP Notice:  Undefined index: htmlspecialchars in /home/website/public_html/community/Sources/ScheduledTasks.php on line 1022
[04-May-2020 06:04:11 Asia/Kolkata] PHP Fatal error:  Function name must be a string in /home/website/public_html/community/Sources/ScheduledTasks.php on line 1022
[04-May-2020 06:04:49 Asia/Kolkata] PHP Notice:  Undefined index: htmlspecialchars in /home/website/public_html/community/Sources/ScheduledTasks.php on line 1022
[04-May-2020 06:04:49 Asia/Kolkata] PHP Fatal error:  Function name must be a string in /home/website/public_html/community/Sources/ScheduledTasks.php on line 1022

Working on New Mods & Themes for SMF... Will update soon... My Blog page
My Smf forum : Discuss ITAcumens :: My SMF Forum

Advertisement: