Your attachment has failed security checks and cannot be uploaded

Started by badon, March 02, 2016, 07:52:35 AM

Previous topic - Next topic

badon

This is still a problem in the year 2016:

https://forum.coincompendium.com/index.php?topic=4513.msg20221#msg20221

Technology marches on, but we still can't rely on SMF for images. Maybe someone can explain how to gut the code to remove these security checks manually?

Suki

Regardless of the year, security measures still need to be applied.

There is a setting to disable extensive checks, do you have that setting on or off?

If you have that setting off and your images are still being blocked then we will have to see that image, perhaps upload it to an external image hosting site for us to take a look at it.

Disclaimer: unless otherwise stated, all my posts are personal and does not represent any views or opinions held by Simple Machines.

badon


Suki

I've checked your images and both of your files contains the opening php tag:  <?  which is why the extensive security check returns false.   The extensive security check explicitly looks for php and/or asp tags.

Could be a false positive but I find it weird that both formats has it exactly two times each.


Anyway, the "normal check" does return true since it doesn't check for php tags, make sure you do have the extensive check turned off and clean your forums cache. Also,make sure that the call to checkImageContents() respects the $modSetting entry for extensive checks.

Disclaimer: unless otherwise stated, all my posts are personal and does not represent any views or opinions held by Simple Machines.

badon

As you can see from my screenshot, extensive security checks are indeed turned off, and they always have been. I don't know what you're talking about with "make sure that the call to checkImageContents() respects the $modSetting entry for extensive checks" - if extensive checks are disabled, this seems irrelevant. Maybe you can tell me how to gut the "security checks" code? I'm pretty annoyed, and I don't care if it prevents some hypothetical attack. Making the forum unusable is unacceptable, and it has to go, as quickly as possible.

Suki

Disclaimer: unless otherwise stated, all my posts are personal and does not represent any views or opinions held by Simple Machines.


badon

Another file that fails security checks when security checks are disabled.


Suki

Need more info, whats your php version?  what libraries do you have installed? mods? do you have the "Re-encode potentially dangerous image attachments"  setting on?
Disclaimer: unless otherwise stated, all my posts are personal and does not represent any views or opinions held by Simple Machines.

badon

Quote from: Suki on October 03, 2016, 12:05:32 PM
Need more info, whats your php version?  what libraries do you have installed? mods? do you have the "Re-encode potentially dangerous image attachments"  setting on?

Re-encode is not enabled because we are preserving information and automatically modifying everything with degraded quality would be bad. I have attached screenshots of my forum mods installed, and my attachment settings. PHP version and Apache modules are below.

PHP Version 5.3.28

Installed modules:

core
http_core
mod_actions
mod_alias
mod_asis
mod_auth_basic
mod_auth_digest
mod_authn_alias
mod_authn_anon
mod_authn_dbm
mod_authn_default
mod_authn_file
mod_authz_dbm
mod_authz_default
mod_authz_groupfile
mod_authz_host
mod_authz_owner
mod_authz_user
mod_autoindex
mod_cache
mod_cern_meta
mod_cgi
mod_charset_lite
mod_dav
mod_dav_fs
mod_deflate
mod_dir
mod_disk_cache
mod_dumpio
mod_env
mod_expires
mod_file_cache
mod_filter
mod_headers
mod_imagemap
mod_include
mod_info
mod_log_config
mod_logio
mod_mime
mod_mime_magic
mod_negotiation
mod_php5
mod_reqtimeout
mod_rewrite
mod_setenvif
mod_so
mod_speling
mod_ssl
mod_status
mod_unique_id
mod_userdir
mod_usertrack
mod_version
mod_vhost_alias
prefork


badon

More files that allegedly fail security checks even though security checks are disabled.



badon

I just discovered a workaround for this bug that also isolates it. If post text is removed, the files will attach to the draft message successfully. Then add text, and post. I have been testing this workaround for about a week, and I have not encountered the bug. I just tested the workaround on a known problem-file, and it posted successfully.

badon

I also just discovered that when attaching multiple files, sometimes adding the text for the "final save" will trigger the bug. Then, when going back, the files will be a mess with duplicates and some of them unchecked. I check all of them, save WITHOUT TEXT, and only a few are still unchecked. Check those, save WITHOUT TEXT, and now all of the attachments are checked. Add text, save, and it works.

There is obvious something wrong in both the handling of the file attachments, and some bizarre interaction between the attachments and the post text.

badon

I just noticed SMF is somehow duplicating files. Some of them are missing, and some of them are duplicated. Weird.

badon

A series of screenshots that shows the addition of post text triggers the bug, even though the files are already attached successfully when the post is made without post text (with an error message).

badon

I just went back and unchecked the last file and saved the post, then all the files disappeared. Did they time-out and get deleted?

Also, could this bug be related to some kind of bizarre permissions problem? I'm running this on FreeBSD, and we don't nortmally have any problems with uploading attachments. We have an SMF 1.x forum on the same server that doesn't have any problems.

I'm going to try reattaching the files, and then unchecking the last one to see what happens. I'm going to uncheck them one at a time and save the post with text to identify which file is the problem file in combination with that particular post text.

Illori

are you using a drafts mod? if so uninstall it and see if you can still duplicate the issue.

badon

I didn't see any drafts mods in my screenshot here:

Re: Your attachment has failed security checks and cannot be uploaded

Maybe you can see something there that might be the culprit?

Illori

Quote from: badon on February 16, 2017, 07:27:28 PM
the files will attach to the draft message successfully.

SMF has no feature to attach to a draft message, can you explain what you mean there?

badon

I mean that attempting to post without any text will upload and attached the files to the message, but will also give an error and ask for post text before it will accept a finished post. You can see that in my screenshot, here:

http://www.simplemachines.org/community/index.php?action=dlattach;topic=544243.0;attach=251864

In this post:

Re: Your attachment has failed security checks and cannot be uploaded

nend

The system will always do some sort of security checking no matter if the "Perform extensive security checks" box is checked or not.

On the false positive you got with the successful file upload. What you're seeing there is place markers after submitting your post, the script immediately returns and tells you that you have no message body. Even if files were uploaded they were never accepted or checked and this is why when you put your message in and then post you get the security error.

I did spend some time on this, tracing the functions and routine and couldn't find anything wrong with the code. As Suki pointed out on a previous reply, there were some malicious looking content at the beginning of the files that were being uploaded.

I would recommend reexamining your desktop and server security as this may be the cause. If any other forum users are having issues then I would also recommend reviewing linked and downloadable content from the server to make sure that none of these files aren't infected with anything nasty.

In the end if you want to bypass the security then you would have to make checkImageContents() in Subs-Graphics.php always return true. This is a pretty straight forward thing to do so I'll not go into details on how to achieve this as it is breaking security and shouldn't be encouraged.

badon

Thank you for your research, nend. Suki and Illori too.

Everyone who uploads files to our forum has this problem occasionally. If I get the file directly from them and try to upload it myself, which is exactly what happened in this case, I will get the same problem, so that rules out the files being tampered-with on the server side. I also experience this problem myself, and the odds of it happening increases if more and larger files are attached. As far as I can tell, this issue is completely random, with SMF tripping over random data combinations it doesn't like.

Based on your study of this problem, nend, it appears I was mistaken that it is possible to workaround the issue by not inputting text at the time of upload. In reality, the security checks are being done only after text is entered. That fits with my experimentation.

I altered checkImageContents() in Subs-Graphics.php to always return true, so there will never be any security checks on image files. I tested the result, and a known problem file was successfully uploaded:

Test of elimination of image file security check incorrect rejections

This problem killed our forum during the year it took to find a way to gut the erroneous security checks. Security doesn't matter very much on a dead forum. I'm not sure why some file were reject and some not rejected. It was so random I couldn't narrow it down to a single camera, application, editor, browser, etc, but with more effort I might be able to. Is there any reason to think these files are definitely malicious? If not, then is there any good reason to keep the code arbitrarily handing out random rejections?

We have probably had a thousand rejections, and if our forum didn't die because of them, we would have had far more. I am extremely skeptical that rejecting all those files serves any legitimate purpose, and I'm completely convinced not even 1 of them is actually malicious. The only thing worse than no security at all is constant alarms that everyone eventually ignores. Maybe a warning or notification for a manual admin check would be better than an unbeatable, blind rejection that cannot be turned off, cannot be overridden, and does nothing other than discourage people from using the forum.

badon

I just compared hashes from the server-hosted file and the offline file copy, and the hashes match, so that rules out server-side tampering. I'm surprised more people aren't having this problem. Maybe something else server-dependent could be increasing the likelihood of this happening?

shawnb61

I have definitely seen this as well.  The same photos fail an upload in the 2.1 beta as well.   

It all boils down to the edits in checkImageContents.  It would be interesting to know exactly which edits are failing in that lengthy preg_match.  I suspect the checks in there need to be rethought. 
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

nend

Looked at a few images you linked to and compared the rules for the non-extensive security checks.

One was due to following in code.
57 53 07

The other was due to following in exif information.
html
Which looks to be the page to where the image was downloaded from.

Two different rules triggering two different files.

badon

Both false positives, unilaterally rejected with no recourse, no intelligent examination, nothing = frustrated users = dead forum.

landyvlad

I am continuing to have this same problem with false positives and yes, it is extremely frustrating to forum members (and me of course) and drives some away in frustration which is very bad....

Like badon experienced, the instances it happens appear relatively random and can't be isolated to any one factor in particular.

I share badon's frustrations and opinions entirely.

The rules for non-extensive security need to be re-examined, as far too many are failing these checks which can't (aside from disabling checking entirely) be circumvented.

There is evidently a major issue which needs to be addressed PLEASE.
"Put as much effort into your question as you'd expect someone to give in an answer"

Please do not PM, IM or Email me with questions on astrophysics or theology.  You will get better and faster responses by asking homeless people in the street. Thank you.

Be the person your dog thinks you are.

badon

I have been sending out instructions for manually disabling this problem in the code, to people who PM me to request it. It ought to be posted publicly. Is that OK?

nend

Quote from: badon on February 19, 2017, 11:37:36 PM
I have been sending out instructions for manually disabling this problem in the code, to people who PM me to request it. It ought to be posted publicly. Is that OK?
I would rather the operator enable re-encoding of potentially dangerous files as it wouldn't pose a possible security risk.

In short, no, it shouldn't be encouraged that forum operators break important security measures put in place to protect themselves and their members.

landyvlad

I'm trying the re-encoding thing - will see how that goes. Cheers to both of you :)
"Put as much effort into your question as you'd expect someone to give in an answer"

Please do not PM, IM or Email me with questions on astrophysics or theology.  You will get better and faster responses by asking homeless people in the street. Thank you.

Be the person your dog thinks you are.

badon

We can't re-encode on our forum because we're trying to preserve quality images, and re-encoding degrades quality. Maybe re-encoding could be an option for use on manually confirmed malicious files.

As it is, I'm skeptical that a malicious JPEG could be exploited. It would probably require a permissions change or other higher-level access to the server, and at that point, the server has already been compromised. I doubt there's any way to exploit the users with an image file. Those kinds of exploits seem to have all been resolved after the 1990's - at least I don't recall any since then, largely because of the greater diversity of and higher quality of the software that reads them (like web browsers).

Illori

there have been security issues with SMF related to images containing scripts in the past that is why we are hashed them when saving [the name] and check for security in general on the images.

shawnb61

Quote from: badon on February 20, 2017, 02:37:51 AM
We can't re-encode on our forum because we're trying to preserve quality images, and re-encoding degrades quality. Maybe re-encoding could be an option for use on manually confirmed malicious files.

As it is, I'm skeptical that a malicious JPEG could be exploited. It would probably require a permissions change or other higher-level access to the server, and at that point, the server has already been compromised. I doubt there's any way to exploit the users with an image file. Those kinds of exploits seem to have all been resolved after the 1990's - at least I don't recall any since then, largely because of the greater diversity of and higher quality of the software that reads them (like web browsers).

I'm curious - what are the sources of your files?   I'm a bit of a photo nut and some of my .jpgs are being denied by SMF.  In my case they're taken on my Nikon D800 as NEF but exported as .jpgs after processing in Lightroom.  I am seeing more rejections, & have been wondering if the latest version of Lightroom is putting more content in the EXIF info that confuses the SMF security checker.

The reason you haven't seen so many attacks is because websites have gotten better at protecting themselves.  Exploits thru images are a very real - and current - threat:
https://www.trustwave.com/Resources/SpiderLabs-Blog/Hiding-Webshell-Backdoor-Code-in-Image-Files/ 

For the most strict security, re-encoding is probably the way to go because it is the ONLY way to remove the non-photo content.  I'm sure there are ways to re-encode without losing image quantity - copy the parts you want (all the pixels) and don't copy the other parts (exif, etc.). 

I suspect all the 'smart' photo features these days are causing false-positives in SMF's security check, which may need to adapt. 
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Illori

if you open the image that is rejected in a text editor can you post the results in the next reply to this topic?

shawnb61

My files are all too big.   Each file that fails seems to fail on a different edit, even on the 'normal' edits.   I can copy excerpts of the files that fail below. 

Nothing for: iframe, eval, script\W

html:
s$«5c#ªÌV%¾—¸–ƒb5ÒRE¬f¿bÖ´   ®ý`‹éÓ·üË]r˜áqy›q/¢ÖN¡wI´ž±Z'Û?>ªÚ¬§N›DoÀájEÇLú-¬)ºyê#†'x[r·ô3sF†³%¡" ;ïý³öqx\ŒæÓL—1ÄO@abÏÖõÑeôðY3KÌ,kfMµm}æZœâ]V%©‰ Âaê$ú]È
OļìhTmL;]V¥C¯g>ôÇÓWÝSoÒL˜†Ž*...MyõÑ{[Iÿ êM]ß󪏩,u ãÿ
À#sês£ñiSìjÇžçöU¦2Ò@ ègÔå.bÈâ³Z­~2¯–eÐP>÷ù—‡ºõð‰–ÞË"¬GWp¬J±L,ŠY"
@°,,Vz^Všx+
èˆq

body:
HGîŸ"ÞA(1W¨£a‹\:£Ë''ƒ¾¤A±ýԁ{x!DFáÕ='¨©o"ÙD½ç•HûNÈç×;¨EÈ:#{û bOdyƒ²[ßÔ#Ì="FÇšÞ¡Øz$dö@"9   £bÙÙÎÉ—ŽÈÞ;+¡ÿÓù×–"›Ñ]T•.ZtU°#`Víî&©ÙҐYjݽµ4mHgp™ŒvÛKodЧÊöH°uZ*ÒÚSB,#Êj¾»#jhڏ(vG"ûrŠ²šyB¸KÉ´õFÔÒíŸÉƒ V¨¤Ñ¶o%§¢<–ô FÛL7ºhÛ/<†­{'·©M ß«´ éDzӶÐ[X)¡›õfž‰~¬ÎËVÔmMdýY½RýU½–ͨÛÙ4méÙ/ÔÛÙlÛhÛI£l_ª7²_©·²ÜÑJhÛêmì'Ñ7²ÞZ•&†Ô›Ù¨·²ß¶ÐZšÿ Ô€èÔZº;p'bqз²_¨it¶ ±7õ¤ttöå¨9¨³ý—P5-QÊÿ g,,¿ÙôºÛi0-AÈÿ g,—û=uöR6Ò9ðô¿Ùë³±"ÀƒþÏGû=v¶¤iäðâþ z%þÏ=kfSòÂyC )³Êîyv,Äò8_¨;,,~ WpÆ/,,y}SÈáúƒ—wb[òxpÿ P(ýA˹å,,l   äðáþ¢RýEË»±XO'‡õ¿RweßòÀAŒ<?ú",GFîËÐyMè%½"Èóߪ;²_ª8/Cä·²F›£ÏþªîÊ'Jå輆"y


[CF]WS[\x01-\x0C] (the binary character is an ACK, displayed here as an -):
¸   T–v*   ÁC¿Ü¡`¢ÓIÛnQQvmEßr‡·Ø)V"@B–þ—ö¶šµ'^à7Jvy`La/`4¦ §TSlÀÚÍÏàáèß.:"O‰84X]È   †Ø#-I1FwsAMS`Àû€ç¥À|"7›à»ÀCØ¢ªiD&ìP5`)>@n0Q.öǺ InPÒì é¾ )I4M¤ªx i¼ ÿ ˆ¨'Ü


Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

Advertisement: