Images not renaming

Started by barrie, August 26, 2022, 04:13:48 AM

Previous topic - Next topic

barrie

Hi Guys, our forum at https://www.aircooledrdclub.com/smf has had an issue for a while that now needs more attention. I believe that when a member uploads an image the software automatically renames it to prevent overwriting images with the same name but ours does not do this and we are getting close to 50,000 images ! We have asked members to rename images with something random like drghh6u8jbbn.jpg but many of them are Luddites and its not in their capability. Can someone point me in the direction as to how we overcome this ? All help is much appreciated.

Steve

Your forum is 2.0.15 which is 4 versions behind in the 2.0 series. You're missing some very important security updates. I highly recommend upgrading to 2.0.19 and consider going to 2.1.2.
DO NOT pm me for support!

barrie

Thanks for the reply. I am not very technical. I look after the Forum after the passing of the guy who set it up. Is it just a case of downloading the next version from within the Forum and installing it and then the next version etc ? Excuse my ignorance.

Steve

Go to Admin -> Package Manager and it will show what the next version is and give you a link to install it. Keep doing this until you reach 2.0.19.
DO NOT pm me for support!

barrie

Thanks again. First upgrade did not go well as I got errors in line 38 : ./Sources/Profile/-Modify.php
Test failed

I will make a back up first but its 700mb so may take a while.

Think I may be out of my depth here !!


Doug Heffernan

Quote from: barrie on August 26, 2022, 08:44:07 AMThanks again. First upgrade did not go well as I got errors in line 38 : ./Sources/Profile/-Modify.php
Test failed

I will make a back up first but its 700mb so may take a while.

Think I may be out of my depth here !!



Click the icon next to the rror message and it will tell you what needs to be changed and where. You can apply the changes manually to the file(s) mentioned in the error.

My advice, if I may, is to upgrade to 2.1.2, as @Steve mentioned above. This upgrade is done manually. For more info please see this link:

Upgrading SMF

Something to note, all mods and themes for 2.0.x are not compatible with the 2.1.x serie.

Sir Osis of Liver

Attachment file names are hashed when they're uploaded for security reasons, you can see that in /attachments directory.  Having users rename their files before upload accomplishes nothing.  Upgrading to 2.1 before resolving this problem may not be the best idea, as the upgrade renames the attachments by adding a .dat extension.  If you're already having difficulties with file names, upgrading could make it worse.  I would clone the forum and run upgrade on the clone before attempting it on production install.
Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

GL700Wing

#7
Quote from: barrie on August 26, 2022, 04:13:48 AMHi Guys, our forum at https://www.aircooledrdclub.com/smf has had an issue for a while that now needs more attention.
Hi @barrie
Are you able to provide details of what the problem actually is (eg, whenever a member tries to do A, B happens)?

I'm guessing it relates to attachments but are members able to successfully attach files to messages?, do attached files display correctly in messages?, and/or are members able to download attached files?  Is the issue affecting all messages, members, attachment types or just some?

Also, I've had a look at your forum (as a guest) and I can see that it is for a 50+ year old series of motorcycles which means many of your members are probably 70+ years old (and some are probably in their 80s and 90s).  I own/manage a forum for a 40+ year old series of motorcycles (I set the forum up 15 years ago) and I know that many of my members (most of whom are 60+ years old) would struggle with the very different look and feel of SMF 2.1.x - especially the attachments interface.  Based on this my recommendation is that, and for security reasons, you consider upgrading to al least SMF 2.0.19.
Life doesn't have to be perfect to be wonderful ...

Kindred

See the faq entry regarding "errors during mid installation" regarding your patch error...
Слaва
Украинi

Please do not PM, IM or Email me with support questions.  You will get better and faster responses in the support boards.  Thank you.

"Loki is not evil, although he is certainly not a force for good. Loki is... complicated."

Illori

take a look at https://www.simplemachines.org/community/index.php?topic=573100.0

you will need to check for attachmentEncryptFilenames in the settings table in your database.

barrie

Quote from: GL700Wing on August 26, 2022, 05:10:54 PM
Quote from: barrie on August 26, 2022, 04:13:48 AMHi Guys, our forum at https://www.aircooledrdclub.com/smf has had an issue for a while that now needs more attention.
Hi @barrie
Are you able to provide details of what the problem actually is (eg, whenever a member tries to do A, B happens)?

I'm guessing it relates to attachments but are members able to successfully attach files to messages?, do attached files display correctly in messages?, and/or are members able to download attached files?  Is the issue affecting all messages, members, attachment types or just some?

Also, I've had a look at your forum (as a guest) and I can see that it is for a 50+ year old series of motorcycles which means many of your members are probably 70+ years old (and some are probably in their 80s and 90s).  I own/manage a forum for a 40+ year old series of motorcycles (I set the forum up 15 years ago) and I know that many of my members (most of whom are 60+ years old) would struggle with the very different look and feel of SMF 2.1.x - especially the attachments interface.  Based on this my recommendation is that, and for security reasons, you consider upgrading to al least SMF 2.0.19.

When a member uploads an attachment for example called yamaha.jpg if there is already an attachment with the same name it overwrites it causing great confusion for posts going back many years !!

Illori

did you look at the link i provided above?

barrie

Quote from: Illori on August 27, 2022, 05:32:39 AMdid you look at the link i provided above?

Yes and many thanks. As mentioned, this is all new to me so I need to get my head around all these strange files with the codes in. I am really nervous about changing anything  :P

Illori

then do a backup first.

https://wiki.simplemachines.org/smf/Backup

in this case you need to modify the database, so you really need to do a database backup first.

Sir Osis of Liver

Quote from: barrie on August 27, 2022, 05:01:02 AMWhen a member uploads an attachment for example called yamaha.jpg if there is already an attachment with the same name it overwrites it causing great confusion for posts going back many years !!

That doesn't happen in 2.0.19 -

You cannot view this attachment.

Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

GL700Wing

Quote from: Sir Osis of Liver on August 27, 2022, 11:17:30 PM
Quote from: barrie on August 27, 2022, 05:01:02 AMWhen a member uploads an attachment for example called yamaha.jpg if there is already an attachment with the same name it overwrites it causing great confusion for posts going back many years !!
That doesn't happen in 2.0.19 -
Apparently this can happen in SMF 2.0.19 if the setting attachmentEncryptFilenames is missing from the smf_settings table (or it has a value of 0) - more information in the What can cause error message "Sorry! There is already an attachment..."? topic @Illori has referred to a couple of times ...

Life doesn't have to be perfect to be wonderful ...

Sir Osis of Liver

Hmm, didn't think that would apply to this situation, but setting attachmentEncryptFilenames = 0 replicates the problem.  If that's no longer an admin option, and filenames are always hashed in 2.0, why is the setting still used?  And it would have to carry over from a 1.1 install to be set to 0, as there's no way to set it in 2.0.

Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

GL700Wing

Quote from: Sir Osis of Liver on August 28, 2022, 12:05:16 AMHmm, didn't think that would apply to this situation, but setting attachmentEncryptFilenames = 0 replicates the problem.  If that's no longer an admin option, and filenames are always hashed in 2.0, why is that setting still used?  And it would have to carry over from a 1.1 install to be set to 0, as there's no way to set it in 2.0.
The oldest topics on the affected forum date back to SMF 1.x when hashing attachment filenames was optional.
Life doesn't have to be perfect to be wonderful ...

Sir Osis of Liver

Do the unhashed files still display correctly after upgrading to 2.0?  If OP changes the setting to 1, will that affect unhashed files?
Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

GL700Wing

#19
Quote from: Sir Osis of Liver on August 28, 2022, 12:21:48 AMDo the unhashed files still display correctly after upgrading to 2.0?  If OP changes the setting to 1, will that affect unhashed files?
I'm just testing that now with SMF 2.0.19 but the weird thing is that even if the setting attachmentEncryptFilenames is missing or has a value of '0' the attachment filename is actually hashed/encrypted and the hashed/encrypted file name is stored as 'file_hash' in the 'attachments' table and the attachment name in the 'attachments' folder is the hashed/encrypted file name.

Maybe the empty($modSettings['attachmentEncryptFilenames']) checks in Subs-Post.php and Subs.php need to be reviewed ...
Life doesn't have to be perfect to be wonderful ...

Sir Osis of Liver

It doesn't seem to affect new attachments uploaded in 2.0 either way.  Simplest thing would be to have upgrader set attachmentEncryptFilenames = 1, as long as unhashed attachments still work.  Or just remove the setting check.
Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

GL700Wing

#21
Quote from: Sir Osis of Liver on August 28, 2022, 12:48:44 AMIt doesn't seem to affect new attachments uploaded in 2.0 either way.  Simplest thing would be to have upgrader set attachmentEncryptFilenames = 1, as long as unhashed attachments still work.  Or just remove the setting check.
I've just confirmed that unhashed attachments and their thumbnails still display/function correctly (I manually renamed an attachment and its thumbnail in the 'attachments' folder to match the values stored in the database and removed the corresponding 'file_hash' values from the 'smf_attachments' table) and that a message with a mix of hashed/unhashed attachment file names does not throw any errors.

As such, I think manually adding the 'attachmentEncryptFilenames' setting with a value of '1' to @barrie's database should resolve the issue he currently has.

Also, and given that the default setting for 'attachmentEncryptFilenames'  has been '1' since 2009/1010, and that unhashed attachments and their thumbnails still display/function correctly, I agree that (and subject to further testing/confirmation) upgrade scripts should set 'attachmentEncryptFilenames' to '1'.
Life doesn't have to be perfect to be wonderful ...

GL700Wing

Quote from: shawnb61 on August 28, 2022, 02:07:55 AMIn 2.x, there is no danger of overwriting attachments, no matter what the name, because that's not how it's stored in the db or in the file system.

You can have 100s of files named cat.jpg.  That's not a problem, because that's not a key for how it's stored anywhere. 

The filename of the uploaded file is separate & distinct from the hashed filename in the file system.  The name of the uploaded file is just an attribute of the attachment.

In 2.x...
The affected forum is SMF 2.0.15 and even though, as you say "You can have 100s of files named cat.jpg.  That's not a problem, because that's not a key for how it's stored anywhere.", the issue is that if the setting 'attachmentEncryptFilenames' is missing or has a value of '0' SMF 2.0 does check for duplicate attachment file names (it checks to see if an identical attachment file name is already stored in the 'filename' column of the 'attachments' table) even though this isn't the name the file is actually stored with - you can check/confirm this easily by changing the value of the setting 'attachmentEncryptFilenames' to '0' and then trying to attach the same file twice.
Life doesn't have to be perfect to be wonderful ...

GL700Wing

@barrie - I've sent you a PM offering my assistance to help you resolve this issue.
Life doesn't have to be perfect to be wonderful ...

Sir Osis of Liver

Wonder what happens if you upgrade to 2.1.  Does upgrader add .dat extension to plain text filenames (i.e., cat.jpg.dat)?  What effect would that have?
Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

GL700Wing

#25
Quote from: Sir Osis of Liver on August 28, 2022, 05:12:23 PMWonder what happens if you upgrade to 2.1.  Does upgrader add .dat extension to plain text filenames (i.e., cat.jpg.dat)?  What effect would that have?
I've just finished upgrading a test forum from SMF 2.0.14 to SMF 2.1.2 and these are my findings:

1. The intermediate SMF 2.0 upgrades (ie, 2.0.15 - 2.0.19) - which I did out of interest - did not change the value of the 'attachmentEncryptFilenames' setting from '0' to '1' even though these versions of SMF 2.0 encrypt/hash attachment filenames (and store them using these encrypted/hashed filenames) by default;

2. The 'attachmentEncryptFilenames' setting no longer exists in SMF 2.1.2;

3. The filenames of attachments (and image thumbnails) that were stored on the file system without an encrypted/hashed filename (eg, abc.jpg, abc.jpg_thumb) prior to the upgrade to SMF 2.1.2 were encrypted/hashed during the upgrade process (and the 'file_hash' column in the 'attachments' table is updated accordingly) and the files were renamed to the format 'NNN_######.dat' (where 'NNN' is the attachment number and '######' is the encrypted/hashed filename); and

4. Despite the 'attachmentEncryptFilenames' setting not existing in SMF 2.1.2 an error message is no longer generated if multiple attachments with the same filename are uploaded - this indicates that the duplicate filename check that is performed in SMF 2.0 has been removed.
Life doesn't have to be perfect to be wonderful ...

barrie

#26
Quote from: GL700Wing on August 28, 2022, 10:01:27 AM@barrie - I've sent you a PM offering my assistance to help you resolve this issue.
Reply sent GL700Wing. Many thanks.


Edit: try to avoid using Team Member's real names on open boards please. ~ Steve

Sir Osis of Liver

Quote from: GL700Wing on August 28, 2022, 05:55:31 PMThe filenames of attachments (and image thumbnails) that were stored on the file system without an encrypted/hashed filename (eg, abc.jpg, abc.jpg_thumb) prior to the upgrade to SMF 2.1.2 were encrypted/hashed during the upgrade process (and the 'file_hash' column in the 'attachments' table is updated accordingly) and the files were renamed to the format 'NNN_######.dat' (where 'NNN' is the attachment number and '######' is the encrypted/hashed filename)

That being the case, OP can set attachmentEncryptFilenames = 1 in 2.0 database, or upgrade to 2.1.  Either should solve the duplicate filename problem.
Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

GL700Wing

#28
Quote from: Sir Osis of Liver on August 29, 2022, 12:40:04 PMThat being the case, OP can set attachmentEncryptFilenames = 1 in 2.0 database, or upgrade to 2.1.  Either should solve the duplicate filename problem.
Man - what an issue to troubleshoot and work out the cause of!

@barrie sent me a PM and in it he stated that if a member uploaded a file named ABC.jpg and there was already a file with the same name the existing file would be overwritten by the new file.

@barrie provided me with cPanel access to his forum which enabled me to work out the following:
1. The setting 'attachmentEncryptFilenames' is defined and has a value of '1';
2. More than 62,000 attachments/thumbnails are defined in the 'attachments' table;
3. All the attachments in the 'attachments' table have a 'file_hash' value; and
4. Only about 600 attachments, all of which were uploaded via Tapatalk, have an encrypted/hashed file name in the 'attachments' directory/folder.

The next thing I did was work out which mods were installed on @barrie's forum and I created an SMF 2.0.15 forum on one of my websites with exactly the same versions of these mods installed (and I installed them in the same order).  No joy, I could not reproduce the problem ...

So then I took a copy of the Sources and Themes folders from @barrie's website and after copying them to the test forum I'd created I was finally able to reproduce the problem!!

I then did a comparison of each of the files from @barrie's website and my test forum (I started with the Sources folder) until I found the code that was causing this issue - it was in Subs.php!

The code at the end of the 'getAttachmentFilename' function had been changed:
Original code:
return $path . '/' . $attachment_id . '_' . $file_hash;
Changed code:
if (!file_exists($path . '/' . $attachment_id . '_' . $file_hash))
return getLegacyAttachmentFilename($filename, $attachment_id, $dir, $new);
else
return $path . '/' . $attachment_id . '_' . $file_hash;

There was no information as to who had changed the code or when/why the change was made (the file was last modified in December 2017 when the SMF 2.0.15 patch was installed) so I then searched this forum to find out if the change had ever been documented here - and I found it!

Almost 10 years ago, in October 2012, @barrie started a topic about Attachments not showing and it seems he made a code change to the 'getAttachmentFilename' function in ./Sources/Subs.php that was suggested by Kays.  Unfortunately, and even though this code change didn't resolve the issue he reported, @barrie didn't delete the code he'd added to the file. 

As a result, and for almost 10 years now, attachments on his forum have been routinely overwritten whenever a new file with the same name as an existing file is uploaded!

Done!!
Life doesn't have to be perfect to be wonderful ...

Steve

Marking solved then and well done GL!
DO NOT pm me for support!

barrie

#30
A huge thanks to GL700Wing for solving this for us Code has been changed and tested and all working well  ;D  ;D


Edit: Again I ask you, try to avoid using Team Member's real names on open boards please. ~ Steve

Sir Osis of Liver

Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

Advertisement: