[2.0.14 - newly patched] Problem with attachmentUploadDir causing 404 errors

Started by perkele, July 11, 2017, 07:13:47 PM

Previous topic - Next topic

perkele

Hello!

We recently upgraded our forum to SMF version 2.0.14 (all the way through patches from 2.0.10).

After that, it stopped working, because we ran only PHP 5.3, and SMF 2.0.14 requires PHP 5.4 at the least (as opposed to what the installer says - but this has been mentioned often enough, hopefully it is fixed now).

So this was a good reason to finally let the server migration happen that our hosting provider had freely offered some time ago. So now we are running PHP 5.6.30.

However, before the migration happened, I first tried to revert back to a backed up version before the SMF update. Things were very strange after the backup "downgrade", with mangled characters due to wrong character encodings, and 404 errors for any attachments and avatars. None of these problems existed right before the SMF update, when the backup was done. So after some fiddling I decided to revert back to the state after the update to SMF 2.0.14 (of which I had also made a backup), hoping that all the data is good and the new server will figure it out, and then initiated the server migration to the new server with PHP version 5.6.30.

After the migration, things look mostly good (including the character encodings, I had been very worried about this. :) ). However, there are some issues still.

Most notably, there are still 404 errors for all attachments and avatars.

So I thought, probably I have to change some paths somewhere. And I tried the following:


  • I changed the $boarddir, $sourcedir and $cachedir variables in Settings.php to the correct new values. And, yes, they are definitely the correct values now!
    But it seems it does not even matter if I change them to "hurz", "schlurz" and "wurz", because this code block in Settings.php:

# Make sure the paths are correct... at least try to fix them.
if (!file_exists($boarddir) && file_exists(dirname(__FILE__) . '/agreement.txt'))
$boarddir = dirname(__FILE__);
if (!file_exists($sourcedir) && file_exists($boarddir . '/Sources'))
$sourcedir = $boarddir . '/Sources';
if (!file_exists($cachedir) && file_exists($boarddir . '/cache'))
$cachedir = $boarddir . '/cache';


  • I changed the following variables in the database table settings to the correct new values:

    • attachmentUploadDir
    • smileys_dir
    • avatar_directory
Another curious thing I found while doing this was that the variables table contains the variable smfVersion with value 2.0.2.
Is that a sign that something got broken with the backups (but such an old version? I certainly did not back up to something as old as that; and otherwise the important forum data (posts etc.) seems to be complete and up-to-date), or has this smfVersion variable in that table simply never been changed through any update since then?

Still, after these reasonable changes, the 404 errors for avatars and attachments persist.

So I tried to trace back where the error happens in the code.

I found that the attachment retrieval happens in the function Download() within the php file Sources/Display.php, which in turn calls the function getAttachmentFilename(), defined in Sources/Subs.php. It turns out that getAttachmentFilename() returns the correct file name, but not prepended by the attachment directory path.

This is what the function getAttachmentFilename() looks like (the bottom part of it, which seems to be where things are going wrong):
// Get an attachment's encrypted filename.  If $new is true, won't check for file existence.
function getAttachmentFilename($filename, $attachment_id, $dir = null, $new = false, $file_hash = '')
{
global $modSettings, $smcFunc;

// ... other code ...

// Are we using multiple directories?
if (!empty($modSettings['currentAttachmentUploadDir']))
{
if (!is_array($modSettings['attachmentUploadDir']))
{
$modSettings['attachmentUploadDir'] = safe_unserialize($modSettings['attachmentUploadDir']);
}
$path = $modSettings['attachmentUploadDir'][$dir];
}
else
$path = $modSettings['attachmentUploadDir'];

return $path . '/' . $attachment_id . '_' . $file_hash;

}


I did some debugging and found out that

  • !empty($modSettings['currentAttachmentUploadDir']) evaluates to true (or something truthy).
  • !is_array($modSettings['attachmentUploadDir']) also evaluates to true (or something truthy).
  • Namely, $modSettings['attachmentUploadDir'] simply evaluates to the value of attachmentUploadDir in the settings database table - which is exactly what should be assigned to $path here!
  • But safe_unserialize($modSettings['attachmentUploadDir']) evaluates to false. (And the safe_unserialize function is a bit too much for me at the moment to find my way through.)
  • As a result of all this, the line $path = $modSettings['attachmentUploadDir'][$dir]; will simply assign an empty string.

Now I do not exactly understand what all this safe_unserialize and attachmentUploadDir as an array stuff is supposed to be about. Why should attachmentUploadDir be an array? And how should it be an array when it simply comes from that string column variable in the settings database table? And if it is not an array, as in my case, it simply breaks. Is this a bug? It sure seems a bit strange to me, codewise. But I am not exactly familiar with PHP, nor with the history of how the codebase here has grown.

How can I work around it without messing too much with the code in a way that will break after an upgrade?

Thank you for your time!

shawnb61

You can safely ignore the odd old smfVersion in the settings table...  Mine says 2.0.1...

You can also safely ignore all the comments in settings.php.  SMF only updates the setting values as you patch, not the comments.   Mine says its version 1.1...

You should NOT update those paths in the settings table.  Only use repair_settings.php to do so.   

If SMF can't find attachments, then the values you see in repair_settings.php do not match what is in the filesystem.    Carefully review everything in File Manager, then doublecheck everything in repair_settings.php.

Finally make sure you're using the right version of repair_settings.php.    if you're on 2.0, it should say so across the top.
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

shawnb61

If your attachment value is an empty string, that's your problem.    Click on the link below the empty field to populate it.
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

perkele

Quote from: shawnb61 on July 11, 2017, 07:28:54 PM
You should NOT update those paths in the settings table.  Only use repair_settings.php to do so.   

If SMF can't find attachments, then the values you see in repair_settings.php do not match what is in the filesystem.    Carefully review everything in File Manager, then doublecheck everything in repair_settings.php.
Thanks for the hint. What is repair_settings.php? And where do I get it? And why should I need it?
Honestly, I cannot really simply accept that should or shouldn't this or that without a reasonable explanation what these things mean or do.

I have done what seemed to be reasonable so far and traced the problem back to what looks like a bug in the PHP code. I think this piece of code which I analyzed in the last part of my post is somewhat dubious. I would like to understand it.

Quote from: shawnb61 on July 11, 2017, 07:34:09 PM
If your attachment value is an empty string, that's your problem.    Click on the link below the empty field to populate it.
I don't know what attachment value and where that should be an empty string. I found that attachment file names are generated correctly, but the directory path is not prepended, i.e. for an attachment that exists on the server with filename /path/to/attachments/123_xyABCdefG, I simply get /123_xyABCdefG as path to the attachment, but it should be /path/to/attachment/123_xyABCdefG. The /path/to/attachment/ which should be stored in some variable named $attachmentUploadDir is swallowed, because of some black magic happening in the safe_unserialize() PHP function. Or something like that. ::)

I am wondering why it should be necessary to use that tool named "repairSettings.php" in the first place. Why is it broken? Why must something be repaired by a tool?  ??? I have read about some bad stuff happening with this tool. So I want to make sure that I don't blow anything up.

Where do I get the correct version of repairSettings.php? But I found this topic. Maybe I should read it first before asking anything further.  :-X

Thanks for your help in any case!   :)

shawnb61

Quote from: perkele on July 11, 2017, 08:03:38 PM
Thanks for the hint. What is repair_settings.php? And where do I get it? And why should I need it?
Honestly, I cannot really simply accept that should or shouldn't this or that without a reasonable explanation what these things mean or do.

I have done what seemed to be reasonable so far and traced the problem back to what looks like a bug in the PHP code. I think this piece of code which I analyzed in the last part of my post is somewhat dubious. I would like to understand it.

Read here:
https://wiki.simplemachines.org/smf/Repair_settings.php

The idea is to make it maintaining your forum easy.  You shouldn't have to putz with code or the database directly to do something basic like tell the system where the attachment directory is. 

Quote from: perkele on July 11, 2017, 08:03:38 PM

I don't know what attachment value and where that should be an empty string. I found that attachment file names are generated correctly, but the directory path is not prepended, i.e. for an attachment that exists on the server with filename /path/to/attachments/123_xyABCdefG, I simply get /123_xyABCdefG as path to the attachment, but it should be /path/to/attachment/123_xyABCdefG. The /path/to/attachment/ which should be stored in some variable named $attachmentUploadDir is swallowed, because of some black magic happening in the safe_unserialize() PHP function. Or something like that. ::)

FYI - the system allows for multiple directories.  If there are multiple directories, it stores that list of attachment directories in a special string called a serialized string.  You shouldn't have to fret how it's stored.  Just use the proper admin screen or repair_settings.php to set the directory. 

The path to the attachment directory **IS** stored in a special variable, which you can update using the admin function under Admin | Forum | Attachments and Avatars | Attachment Settings. 

repair_settings.php is a valuable tool to help you get straightened out easily when doing tasks like moving your forum to another host.  All your paths & URLs change.  repair_settings.php allows you to audit and update all paths in one simple place.  Including the Attachment directory.

It also helps when the forum somehow gets confused...   
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

perkele

Thank you for the detailed explanation!

Yes, I see the purpose that it was supposed to make life easier. Maybe I should have trusted the tool in the first place. But I like to learn the hard way. ???

Turns out, I must have replaced the previously already existing serialized attachmentUploadDir array with a single path string, not even knowing that we already had two attachmentUploadDirs configured previously and stored in this serialized form.

So there I was confused, why suddenly the forum is changed to multiple paths, while I only know of one path, and I thought I had only replaced a single path string in the database. :P

But in the end it was still time worth spent to figure these things out, and see that the code actually does reasonable things. Now I still have to fix some little things. But I still do not dare to use the tool to do something automatically which I don't know about. I guess I have trust issues. :-X It looks so big and convoluted.
My other problems have to do with the TinyPortal mod. I guess the tool would not be able to help me with that, or would it?

But it is a pleasure to wade through some arcane PHP code and learn how things work. :)

I'll mark this as solved. Thanks for the assistance!

Advertisement: