News:

Want to get involved in developing SMF, then why not lend a hand on our github!

Main Menu

Bad links for SOME attachments

Started by SethStudent, November 03, 2019, 06:26:17 PM

Previous topic - Next topic

SethStudent

Just found out from a new member that some of my forum's attachments are "missing." He pointed out two image attachments in a post from last year. They had been viewed several times, and suddenly would not open. The URLs looked ok. I downloaded the Attachments folder and the images were in there, I was able to open them in Photoshop. I renamed them and replaced the broken ones and they are fine now.

The member just now said there are a LOT of attachments missing, but didn't give me a list. The attachments look ok in the Browse Attachments list, so I can't tell which links are bad.

I ran repair_settings and everything was set to the recommendations.

Has anyone heard of this before? Short of me looking at every one of the 12,000+ posts and replacing some of the images, is there a maintenance thing I can run to have the links repaired?

I have not added any new mods for years, everything seems to be up to date. https://speakingofseth.com

SethStudent

Sorry to bring this up again, but it's been a while with no response. I came across another post today on my forum where while the attachments had been viewed in the past, suddenly the links are broken: 404 - Attachment Not Found

Image browser URLs from clicking the links:

https://speakingofseth.com/index.php?action=dlattach;topic=1375.0;attach=2046

https://speakingofseth.com/index.php?action=dlattach;topic=1375.0;attach=2048

The images were in the Attachments folder. I was able to open them in Photoshop so they are not corrupted. I'm attaching a composite of screen captures showing the attachment files in various places (folder, database, Browser).

Any idea how I can fix this? Not all attachments on my forum have broken links, just an unknown percentage at this point. There are 2000+ attachments and 12,000 posts and I can't look at every single one of them.

Sir Osis of Liver

There was a change in 2.0 database around 2.0.6 involving mime_type in smf_attachments table.  If mime_type is empty, attachment will not load.  Attachments posted after the change will load correctly.  Look in your database, see if bad attachments have empty mime_type.  If that's the problem, you can run database query to fix it.
Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

SethStudent

Thanks for the idea. I did take a look at the db and can't see anything different about the attachments that are broken compared to the working ones (mime_type is there), but I'm no db whiz. I did a screen capture of several rows, maybe someone else with more experience can spot a discrepancy. Attachments up through 2054 have bad links (images, pdfs, html, txt, doc or avatars). After 2054, they work.

The date of 2054 is 6/9/18. The next attachments were added on 07/16/18. There were four of them added at approximately the same time/same post: 2059, 2061, 2063 are ok, 2057 is not. After that the rest of the attachments appear to be fine.

I looked at dates in the Package directory thinking maybe I did a patch or something around 07/06/18 that would affect the db, but no. 2.0.12 and .13 were installed Feb 2017. 2.0.14 and .15 were installed 08/20/18.

Any suggestions would be greatly appreciated. Needless to say, this is upsetting.

SethStudent

I did a test update to 2.1 on a copy of my forum, and that fixed the attachment links for images and docs. The personal avatars are still gone, but I'm not as concerned about that.

I did also do a second test on another copy of my site, turning off mods one by one and checking the attachments to see if it could have just been a mod conflict, but that didn't fix the problem. It was the upgrade that fixed it. It's so tempting to just update my production site at this point.

Sir Osis of Liver

There's an 'Attachment Integrity Check' in Admin -> Attachments and Avatars -> File Maintenance, but I've always been hesitant to run it.  You can try it, but back up your database and /attachments directory before doing so.

Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

SethStudent

Thanks for telling me about the attachments File Maintenance, it was worth a shot. I'm attaching a capture of the Attachment Stats showing 985 attachments, while the Maintenance shows 1540 attachments "no longer exist on disk." They are on the disk, it's just not seeing them for some reason.

When I ran Maintenance it removed a lot of attachment links from the posts, while a few older attachments reappeared. So I restored my forum back to the way it was before I ran Maintenance. I guess I'll just have to live with it until I upgrade to 2.1. I didn't see RC2 on the Milestones list, RC3 is probably on its way.

Sir Osis of Liver

The different attachment counts is something I've seen before, and the primary reason I don't trust the maintenance task.  In your db screenshot some of the mime types are incorrect.  Any correlation with missing attachments?
Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

SethStudent

Do you mean like where it will say the image thumb is a jpg, but mime type shows image/png? Or are you seeing something else? Attachment Settings has "Save thumbnails as PNG" checked. I don't think I changed that, it must be a default. But I looked at a recent image attachment that works, and in the db it shows the the thumb the same way as above.

I went to an old topic where there were 3 posts with bad attachment links. Attached is a comparison of how the links appear in the posts and how they appear in the database.

Sir Osis of Liver

I'm not seeing anything wrong with any of the db fields.  Screenshot does not show last column 'approved', but they should all be '1' as in first ss.  Would be interesting to see screenshot of same attachments in 2.1.
Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

shawnb61

What exactly is the error message? 

Does your browser console show something, e.g., like a 404?

Are the permissions consistent across all the files (in the file system)?

Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

SethStudent

Thank you both!

I do get a 404 when clicking on the "bad" links. The thumbnail does not show in the attachment post and there's a link to the attachment, which results in a 404. It's not consistent—I went to one post from 02/14/17 and while there were no thumbnails showing, the first three of the consecutively numbered links resulted in the 404, the next three links downloaded the image to my computer, the last one brought up a 404.

I can put together a graphic of how those links appear in the current db, compared to how they appear in the one that's been updated to 2.1, but they look the same to me. I'm attaching a simpler example of only one attachment & thumbnail in how they appear in the production site db, the 2.1 update test db, and a db from an old test site that was originally a clone of my current production site. The only place the attachment is messed up is in my production site.

I did also look at the permissions for the attachments and they are all set to 0644.

I don't know anything about databases, is there some sort of master index it uses in order to know where to locate things such as attachments? Could it have "forgotten" where some of the attachments are? By updating to 2.1 on the test site, would the db update process have refreshed such an index?

Sir Osis of Liver

smf_attachments tables are pretty much identical in 2.0 and 2.1, no idea why upgrade fixes the links.  Can you copy the image links for a good and bad attachment?  Mouseover the thumb and view image info.  If there's no thumb, use inspect element.
Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

Sir Osis of Liver

Unable to register on your forum, first verification question won't work.
Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

SethStudent

Image links attached. Good idea, even I see a difference! The top one is broken, the second one is from the upgrade test and works.

Sorry you couldn't join the forum, I've had new members join very recently. I took out the first question for now (btw Jane and her husband had different last names). Hopefully you can get in this time.

If you want to check out some bad links, go to https://speakingofseth.com/index.php?topic=530.msg8610#msg8610 but you have to be logged in to see the post.

Sir Osis of Liver

Ok, I'm registered as Krash.  Gotta go, will take a look tonight (NY time).

Ashes and diamonds, foe and friend,
 we were all equal in the end.

                                     - R. Waters

shawnb61

The ";image" is added to the url in both 2.0 and 2.1 for thumbnails - I think you're comparing a thumbnail link to an image link above.

The 2.1 upgrader doesn't really do any magic for the attachments.  It renames them (adding a .dat at the end to solve some issues) but there's no real magic via the upgrade.  If your 2.1 environment works & your 2.0 environment doesn't, it may be environment related. 

More questions:
- How does your CPU chart look?
- Any error messages in your apache log? 
- Any error messages in your SMF error log?
- Any errors in your browser console (usually F12 & click on the console link or anything red...)?

Does the behavior change when you press refresh the screen?   (usually F5)
Does the behavior change when you do a hard refresh?   (usually shift-F5)

Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

shawnb61

If you want to audit your attachments in depth, to diagnose an issue, here is a list of things to look at...

Start with a message id, in these examples I used 179083:
select id_attach, id_msg, filename, id_folder, file_hash, fileext, mime_type
from smf_attachments
where id_msg = 179083;


If you look in your attachment folder on disc, you should see the file, named <id_attach>_<file_hash>

It might help to look at the message, to ensure that the topic in the image link matches the topic on the message:
select id_msg, id_topic
from smf_messages
where id_msg = 179083;


Finally, look at your folder structure, two ways:

In the app, look under Admin | Forum | Attachments and Avatars | Attachment Settings and click on the "Configure multiple attachment directories" link. 

What you see should correspond to the result of this query:
select variable, value
from smf_settings
where variable = 'attachmentUploadDir';


You may either see a single, simple folder, or a serialized list of folders. 

If you see a single, simple folder, than the id_folder on ALL your attachments should be 1.  And only ever 1.  This query should return nothing:
select * from smf_attachments
where id_folder != 1;


If you see a list of folders like:
a:2:{i:1;s:28:"/my/path/attachments";i:2;s:29:"/my/path/attachments2";}

That tells you that you have 2 folders. The i:# corresponds to the id_folder you should see on the attachment record.

If this all lines up properly, then there isn't an issue with storage of your attachments.

Hope this helps... 
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

SethStudent

Quote from: shawnb61 on November 25, 2019, 12:34:59 PM
The ";image" is added to the url in both 2.0 and 2.1 for thumbnails - I think you're comparing a thumbnail link to an image link above.

Yep, you're right, I just took a second look. In Chrome there's no visible graphic for the missing thumbnail so I couldn't tell. I'm attaching a view of the Abby attachment as seen in Safari instead of Chrome.

Quote from: shawnb61 on November 25, 2019, 12:34:59 PM
More questions:
- How does your CPU chart look?
- Any error messages in your apache log? 
- Any error messages in your SMF error log?
- Any errors in your browser console (usually F12 & click on the console link or anything red...)?

Does the behavior change when you press refresh the screen?   (usually F5)
Does the behavior change when you do a hard refresh?   (usually shift-F5)

• What is a CPU chart? Assuming it's my computer's, I'm on Mac. But I'm not the only one having the problem, it was pointed out to me by a new member in Korea.
• Apache log: Sounds like this would be with my hosting... I'm not that technical. I did find an Errors area in my cPanel, am attaching a txt document with a small sampling.
• SMF error log: nothing other than occasional forgotten passwords.
• Browser console errors: again not sure what that is, but back in Chrome I was able to look in the JavaScript Console and just got this after clicking the attachment link: Failed to load resource: the server responded with a status of 404 ()
•Refeshing the screen or changing browsers changes nothing, but then again I'm not the only one having the issue. I cloned my forum the other day to test the attachments Integrity Check and the bad links carried over to the test site.

"Sir Osis" joined my forum this morning, said he would take a look around from the inside tonight. We'll find out if he sees the same issues.

Quote from: shawnb61 on November 25, 2019, 01:09:07 PM
If you want to audit your attachments in depth, to diagnose an issue, here is a list of things to look at...

Thank you, very detailed information. I'll work on this today and get back with any new discoveries.

shawnb61

The CPU chart I'm looking for is for your hosted computer.  Sometimes they max out and strange things happen.

Look around in cpanel.  It's important to watch the CPU.  If you can't find it, your host should be able to show you where it is.

Let's see if the audit turns up anything...



Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Advertisement: