Attachments suddenly not showing up

Started by Oldiesmann, March 24, 2022, 11:51:29 AM

Previous topic - Next topic

Oldiesmann

Having a weird problem on my server. Within the past week, we are suddenly unable to view any attachments. They are uploaded to the server just fine, and the access logs don't show anything unusual.

If I try to view a topic with attachments, I just see the alt text (filename) instead of the actual image (with a broken image indicator thing in Chrome/Chromium-based browsers).

If I try to download the attachment:
Chrome on Android 12 - attachment downloads properly
Other browsers (Firefox 98, Vivaldi [Chromium-based browser], Opera all on Arch Linux) - "Network error" message
wget - "404 not found"

I have done just about all I can in an attempt to resolve the issue - switched PHP handlers from FPM via nginx to FPM via Apache, restarting Apache/Nginx/the entire VPS...

It also isn't limited to a single site or PHP version, and the file size doesn't make a difference either (happens with both a 1kb icon and a 500MB photo).

What's really weird is this doesn't affect the gallery (Levertine or SMF Gallery Pro) - just attachments.

Anyone have any idea of what could be causing the issue? I've got unmanaged VPS so my host isn't really able to help me with this.
Michael Eshom
Christian Metal Fans

Aleksi "Lex" Kilpinen

I don't have many good ideas off the top of my head, but to narrow things down -
Any idea what could have changed around the time this started to happen?
Is this happening with every attachment on the site, or just new ones?
Could you link to an example of this issue?
Slava
Ukraini!


"Before you allow people access to your forum, especially in an administrative position, you must be aware that that person can seriously damage your forum. Therefore, you should only allow people that you trust, implicitly, to have such access." -Douglas

How you can help SMF

Oldiesmann

Happening with all attachments on all sites on the server. Here's a link to an example. https://www.seniorsandfriends.org/index.php?topic=767.3210

No idea what could have changed. I didn't find out about this until a couple of days after it first started.
Michael Eshom
Christian Metal Fans

Aleksi "Lex" Kilpinen

Seems like guests aren't allowed to see attachments on that board.
But here I can see there is supposed to be an attachment available https://www.seniorsandfriends.org/index.php?topic=243.0

Can't see anything obviously wrong, apart from the missing image,
and attempting to download the attachment with Chrome indeed fails with "Network error".

That's an odd one - Have you checked your server's error log?
Slava
Ukraini!


"Before you allow people access to your forum, especially in an administrative position, you must be aware that that person can seriously damage your forum. Therefore, you should only allow people that you trust, implicitly, to have such access." -Douglas

How you can help SMF

Sir Osis of Liver

Can guests see attachments?

Registered as Krash, activate me.
"The best laid schemes o' mice an' men / Gang aft a-gley." - Robert Burns

Oldiesmann

@Sir Osis of Liver You've been activated. I also fixed permissions so guests can see attachments. Most of the boards are set so members can't start new topics and guests viewing attachments somehow got left out of that profile.

Server log doesn't show anything unusual - just a standard 200 response code.

I should also mention that ModSecurity is running on the server but it was installed before this started and disabling it has no effect on this problem
Michael Eshom
Christian Metal Fans

Illori

i believe there have been some fixes on github for attachments. can you try upgrading to what is on github and see if that has any effect on this issue?

Arantor

Try flipping the option for enableCompressedOutput in the admin panel (Maintenance > Server > General > Enable compressed output) and see whether that makes a difference.

I'm wondering if the gzhandler is no longer valid (or, conversely, it's just been installed) and something isn't right somewhere.

Sir Osis of Liver

Inserting attachment just displays image name in post, but it's in attach tags in editor.

Attaching without inserting displays empty frame where thumb should be, image link gives me this -

C:\Users\Krash\AppData\Local\Temp\7vmALeNS.jpg.part could not be saved, because the source file could not be read.
"The best laid schemes o' mice an' men / Gang aft a-gley." - Robert Burns

Oldiesmann

Quote from: Illori on March 24, 2022, 07:38:56 PMi believe there have been some fixes on github for attachments. can you try upgrading to what is on github and see if that has any effect on this issue?

I have a test forum that's automatically synced with GitHub and the same thing happens there.

Quote from: Arantor on March 24, 2022, 07:47:44 PMTry flipping the option for enableCompressedOutput in the admin panel (Maintenance > Server > General > Enable compressed output) and see whether that makes a difference.

I'm wondering if the gzhandler is no longer valid (or, conversely, it's just been installed) and something isn't right somewhere.

You may be on to something... I disabled compressed output and everything is working again.
Michael Eshom
Christian Metal Fans

Sir Osis of Liver

"The best laid schemes o' mice an' men / Gang aft a-gley." - Robert Burns

Oldiesmann

Yes I do. Here's an excerpt from phpinfo.
You cannot view this attachment.
Michael Eshom
Christian Metal Fans

Sir Osis of Liver

Compressed output is supposed to work with zlib installed.  IIRC, there's a warning in 2.0 & 2.1 installers that it may not work on all servers.  Must be something else that affects it.
"The best laid schemes o' mice an' men / Gang aft a-gley." - Robert Burns

Oldiesmann

I'm not sure what else could be affecting it. That setting has been enabled for ages without issue - the problem just started within the past week.
Michael Eshom
Christian Metal Fans

Sir Osis of Liver

Do you run the vps yourself, or is host support involved?  Something must have changed.
"The best laid schemes o' mice an' men / Gang aft a-gley." - Robert Burns

Oldiesmann

Looks like it might be related to recent changes in PHP - PHP on my server was recently upgraded to 8.0.17 and 8.1.4. The changelog for both versions includes this:

Zlib:

    Fixed bug #7953 (ob_clean() only does not set Content-Encoding).

More info: https://github.com/php/php-src/issues/7953
Michael Eshom
Christian Metal Fans

Sir Osis of Liver

You can enable compressed output, confirm that problem returns, then drop php version and see if it clears.  Devs should probably have a look at this.
"The best laid schemes o' mice an' men / Gang aft a-gley." - Robert Burns

Oldiesmann

Did some more digging and it appears that zlib-related bug fix breaks things. I tracked it on GitHub at https://github.com/SimpleMachines/SMF2.1/issues/7395 and referenced the relevant PHP issues.
Michael Eshom
Christian Metal Fans

Sir Osis of Liver

Am unable to replicate the problem. Attachments work fine in php 8.1.3 with compressed output enabled.

ZLib Version    1.2.3
"The best laid schemes o' mice an' men / Gang aft a-gley." - Robert Burns

Oldiesmann

It won't happen on 8.1.3. You need to be on the latest version of 8.0 (8.0.17) or 8.1 (8.1.4). The issue is specifically related to a bug fix in 8.0.17/8.1.4 (see the two php-src issues I referenced on GitHub).
Michael Eshom
Christian Metal Fans

Sir Osis of Liver

Confirmed, I'm getting it in php 8.0.17.
"The best laid schemes o' mice an' men / Gang aft a-gley." - Robert Burns

Advertisement: