News:

Want to get involved in developing SMF, then why not lend a hand on our github!

Main Menu

scheduled_fetchSMfiles() truncates files, breaking Latest Packages interface

Started by Orbfighter, June 30, 2015, 01:31:58 PM

Previous topic - Next topic

Orbfighter

Versions of SMF: 2.0.8 - 2.0.10 (as best as I can tell)

Error Caused From: Sources/ScheduledTasks.php, In function scheduled_fetchSMfiles(), line 1292

        $smcFunc['db_query']('substring', '
            UPDATE {db_prefix}admin_info_files
            SET data = SUBSTRING({string:file_data}, 1, 65534)
            WHERE id_file = {int:id_file}',
            array(
                'id_file' => $ID_FILE,
                'file_data' => $file_data,
            )
        );

For reasons I don't really understand, the SQL query apparently truncates files it downloads from the simplemachines.org server as it uploads them to the database, to 65,534 characters. This is problematic because the download http://simplemachines.org/smf/latest-packages.js at the time of this writing, is 65,706 bytes, cutting off the javascript function function smf_packagesBack() and causing a general JS syntax error when trying to load the latest packages info in the Packages Manager.

To fix this issue on my end, I simply changed the SUBSTRING function in the query to allow for a larger amount of characters, and ran the Fetch Simple Machines Files scheduled task again. After doing this the Latest Packages interface displayed as normal and the full file was loaded in the database.

Illori

we had seen this around the release of 2.0.10 or so, since it resolved itself it was not looked into much further that i can see. maybe the file should be fixed on the site unless they wish to fix it in SMF itself.

for team members reference topic id 535850 has some further info on this.

dcmouser

I got a database error today due to the sql update line above (smf 2.0.10):
"Database Error: Data too long for column 'data' at row 1"
Not sure how to fix or the ramifications..
proud member of donationcoder.com (forum)

Kindred

 just don't use the inline mod downloader...   come to the smf customization site, download the mod to your computer and the upload it directly to the package manager.

unfortunately, the issue is related to the content that mod authors put on our server here... so there is not much else you can do.
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

Orbfighter

That's well and good but this issue is more serious than regarding the installation of customization/mods. I noticed and reported this bug because the forum was failing to notify me that there was a new patch of 2.0 to install (as it previously had.) I was still running 2.0.8 when I realized there was a new version, and yet the system had no idea, and I tracked it down to this line-of-code. If there are security patches such as 2.0.9 that are not getting applied because of this issue, that goes beyond a minor inconvenience. It should at the very least not break the update notification based on the content that mod authors put on your server.

Joshua Dickerson

Quote from: Kindred on July 17, 2015, 03:23:45 PM
just don't use the inline mod downloader...   come to the smf customization site, download the mod to your computer and the upload it directly to the package manager.

unfortunately, the issue is related to the content that mod authors put on our server here... so there is not much else you can do.
Truncate it here and check the file size before you make the file.
Come work with me at Promenade Group



Need help? See the wiki. Want to help SMF? See the wiki!

Did you know you can help develop SMF? See us on Github.

How have you bettered the world today?

Kindred

yeah... are you volunteering to do the work on the customization site?
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

Illori

since the file contains html, you cant really truncate it easily. if you do then you will break the html. the size of the description box on the mod site needs to be modified to be smaller then this would not happen.

Joshua Dickerson

Quote from: Kindred on August 05, 2015, 02:56:16 PM
yeah... are you volunteering to do the work on the customization site?
No.

Quote from: Illori on August 05, 2015, 02:59:10 PM
since the file contains html, you cant really truncate it easily. if you do then you will break the html. the size of the description box on the mod site needs to be modified to be smaller then this would not happen.
True, but not impossible.

Or have a smaller, shorter description for those files.
Come work with me at Promenade Group



Need help? See the wiki. Want to help SMF? See the wiki!

Did you know you can help develop SMF? See us on Github.

How have you bettered the world today?

live627

There must be a better way to handle this. Why are we saving it to the database? It is a FILE. Why not download it to the file cache?

Illori

and what would happen if the cache is cleared and it included this file? it is only populated when the fetch files scheduled task is run. if it could be stored as a file and not cause issues if the file is deleted that is fine.

i am not even sure this file is really called in 2.1... i know it does not show the latest packages etc in 2.1 like it does in 2.0. so maybe we dont need to fix it in 2.1 because of that.

Advertisement: