Topic: SMF 2.0 RC4 unable to download and install new packages [SOLVED]

[holodoc]

First of all I am sorry if this issue has been brought to attention before but since I am on a very tight schedule with one of my own projects I am going to simply outline the issue I came across today while trying to install some packages on an upgraded SMF 2.0 RC4.

Environment:
- Apache 2.2.3 (CentOS)
- PHP 5.2.10 (no significant modifications / no Suhosin installed)
- MySQL 5.2
- SMF 2.0 RC4 - plain vanilla RC4 upgraded from SMF 2.0 RC3 with no modifications installed, upgrade performed with the small update package
- The Packages folder has been chmod-ed with 777 permission recursively.

How to reproduce:
1) Upload the required mod package archive through SMF admin panel (Admin -> Packages -> Download Packages -> Upload a Package) [upload is successful]
2) Use the modification installation link presented after the successful package archive upload.

What happens?
The installation process reports the following error:

You cannot download or install new packages because the Packages directory or one of the files in it are not writable!

What is actually supposed to happen?
The modification is supposed to install successfully of course

After doing some research in the SMF bug tracker I came up with a bug report for SMF 1.1 that pretty well describes the situation in my case (http://dev.simplemachines.org/mantis/bug_view_advanced_page.php?bug_id=166) 

The report also provides a solution for this problem.

Solution to this problem:
Create a folder called temp inside the Packages folder (Packages/temp) and chmod it to 755. If that doesn't help try chmode-ing it to 775 and lastly 777.

I checked my old SMF 2.0 RC3 dailly file backups but I could not find any trace of a temp folder inside Packages which leads me to believe that perhaps the SMF mod installation routine fails to create such folder or at least give a more informative description about whats actually going on - perhaps even check if temp folder actually exists.

Once again I am sorry if a similar SMF 2.0 related bug report exists and I hope that at least the provided solution solves the same problem for other SMF users.

[N. N.]

Thank you for the report. Indeed every once in the while this comes up on Support forums, and SMF could probably provide a more informative message at least - if it cannot do something else.

I believe that normally, SMF should create the folder, but delete it after installing, I'm not sure it can (should) be found any other time or place than /Packages during installation of the mod.

[holodoc]

I've done some further check-ups and it seems to me that SMF actually doesn't even try to remove the temp folder after the whole installation process is over (no log complaints although all folders are highly permissive). In other words whatever modification is installed its files remain inside the temp folder after the whole installation process is over.

Either way I have a question and perhaps a proposal you could consider for one of your next SMF updates (perhaps even SMF 2.0 Final). My question is if it wouldn't be much better to use the standard system /temp folder to store the mod installation files instead of storing them in a rather unpredictable home folder environment.

You see, on Unix/Linux based systems the /temp folder is always present and allows full access to all users while on Windows based machines privileges are something not to worry about since if a requested folder doesn't exist there is no fuss about creating one. Now there is an additional reason why I usually use the system /temp folder in my applications to store non-critical information. Its the fact that in shared hosting environments, which are very common, its not unusual for Apache to run under a separate user which doesn't have required privileges to create folders inside the users home folder (folders are usually set to 0755). That also usually means chmod-ing the whole Packages folder to 777 or at least temporarily changing ownership of the folders to Apache in order for it to be able to write inside these folders. I do realize that changing permissions to 777 for some folders like cache, attachments etc. is inevitable but since you already have reports about this particular problem you might want to reconsider eliminating it by taking a different approach with the system wide /temp folder

I do hope that I explained my proposal well enough and that you will at least take it into consideration although I have a feeling that you might have already talked about this

Qapla'!

[Oya]

but surely the systemwide temp folder is similarly locked down on most shared hosts, and where it's not, there's an implicit security risk from other users on the same host being able to get files in?

[Kindred]

if you are on a shared host, you can not get to the system temp folder, as Oya points out.

and, BTW: If a windows server has permissions correctly set, there could indeed be a problem with creating files or folders...

[holodoc]

but surely the systemwide temp folder is similarly locked down on most shared hosts, and where it's not, there's an implicit security risk from other users on the same host being able to get files in?

if you are on a shared host, you can not get to the system temp folder, as Oya points out.

and, BTW: If a windows server has permissions correctly set, there could indeed be a problem with creating files or folders...

It seems to me that I was not clear enough in my previous post so I'll try to further clarify what I meant to say

When I first mentioned the /temp folder in my previous post I was not referring to any specific web server setup and layout which means that the term /temp folder was supposed to represent that what the web application and the end user see as the /temp folder

In other words both dedicated and shared hosting web servers will have access to the /temp folder however the important difference is that in case of a dedicated system web applications will access the real top-level /temp folder but in shared hosting environments web applications will actually access the chrooted (jailed) /temp folder Of course I don't have to emphasize that any shared hosting provider which doesn't enforce user chrooted accounts is not worth mentioning, right? Practically that means that the web application doesn't have to worry about where the actual /temp folder is just that its always accessed with the same path (/temp) and that it allows full access for the web application.

and, BTW: If a windows server has permissions correctly set, there could indeed be a problem with creating files or folders...

Well whatever restrictions and policies you enforce on a Windows based web server a /temp (or /tmp) folder is usually always present and available to the end web applications to prevent compatibility issues with software that might require it (PHP etc). However the good thing about PHP and Windows based web servers is that these two things combined are usually not the weapon of choice for any web site or application

[Oya]

um... i think you're missing one little detail

when not using db sessions, php writes those as files to the system level temp folder, which means it has access to that as the apache user, or it's chrooted anyway. there is no difference at the php level when it's accessing temp since it's still accessing the same filespace the same way - there aren't two levels of /temp folder normally and most shared hosts don't support it, which is exactly why smf has db sessions in the first place: to bring sessions outside the temp area where they can't be poisoned by other users

[holodoc]

when not using db sessions, php writes those as files to the system level temp folder, which means it has access to that as the apache user, or it's chrooted anyway.

Well I am not quite sure how we got to sessions in this debate but shared hosting web servers are actually never configured to store session data in the system-wide /temp folder and session.save_path is always configured to point at the chrooted user /temp folder I also think that there is really no reason to involve sessions in this debate since configuring them can be performed completely independent from the settings we are talking here about. The same applies for the temporary upload folders etc.

there is no difference at the php level when it's accessing temp since it's still accessing the same filespace the same way - there aren't two levels of /temp folder normally and most shared hosts don't support it, which is exactly why smf has db sessions in the first place: to bring sessions outside the temp area where they can't be poisoned by other users

As a chrooted user you will never be able to reach the "real" world (i.e. real /temp) and everything you see is stuff inside your user folder so when you make a request for /temp in PHP you will actually access something like /home/chrooted/holodoc/temp because your script will regard /home/chrooted/holodoc as its base (/) folder. However the issue at hand is not how temporary folders are structured on different web server layouts but the fact that /temp (or /tmp) is almost allays available and accessible on all those layouts

[Oya]

except that experience disagrees with you - most shared hosts are not so effectively locked down as you think, again this is the reason for having db sessions because for a very long time, /temp was NOT jailed on shared hosts, meaning that every user had access to the real /temp for session storage, or whatever folder - either way, everyone had access to it.

i still know two or three shared hosts that run like that,

[holodoc]

@Oya
While I certainly don't disagree with the idea of storing session data inside a database (not to mention the fact that its a very common way to store sessions in the last couple of years) I still fail to understand why do you keep bringing sessions up in this discussion Yes they can be stored inside the system-wide temp folder (but they are usually not) and yes they tend to be susceptible to session hijacking if they are stored in that way but here we are not talking about session or any other kind of sensitive data which would present a threat even on a shared hosting where the system-wide temp folder is shared among users.

Why would it be almost impossible for the attacker in this case to do mischief by using any kind of temp folder, even the one that is provided by the system? Well it would be enough to say that whatever mod files are extracted during the installation process they would not live long enough to even present the attacker with the opportunity to alter them because it would take the forum software at most a couple of seconds to extract the files into the temp folder, use them and delete them after the installation process Even if your SMF installation doesn't delete the files the next user which installs the same modification will still overwrite the residual files in the temp folder with the fresh, completely untainted and trusted, version of the files.

Regarding shared hosts which don't enforce jailing of their users I usually don't participate in any kind of discussion which involves discussing the quality of web hosting providers. Why not? Because today if you decide to throw a stone in any direction you will most certainly going to hit a person or a company that provides web hosting services To cut the long story short, its up to you to choose what level of quality you want for your web site, forum or custom application and you are most certainly not going to use a free web hosting provider (which most certainly doesn't provide user jailing) to host a web site or application you consider to be important. If that would be the case then all of the major companies would host they corporate web sites on free web hosting providers too

[Oya]

i keep bringing them up as the principle example of where php is accessing the system temp folder under non chrooted credentials, which argues with your suggestion about having a physical and virtual temp folder, ultimately they could well be one and the same

and you sit there and say that users don't use such hosts - they do. regularly. repeatedly. even when they've been advised not to. keeping stuff out of the system temp folder is a very good idea generally, be it session data or mod data

[holodoc]

I believe this would be a perfect place to explain my signature 

Code: [Select]

<?php
abstract class Ignorance extends Stupidity implements Unavoidable { 
     public static $humiliation; 

     private function __construct(){
parent::__destruct();
 }; 
} 
?>In other words ignorance is something that people usually implement alone in their own way but in the end it inevitably destroys (or at least damages) themselves

Bottom line is that you can't always blame the software for your site being hacked or important information being destroyed. How many people do you know which have never created a backup of their web site or forum? I personally know a lot of them but it doesn't prevent me from making daily backups of every web site, application or forum I manage or have made You alone are the person which determines how safe you want your web sites to be and if you fail on such significant task as choosing a reliable web hosting provider you can't actually blame anyone else but yourself after you get hacked right?

[Oya]

yeah i saw your signature, thought it was hilarious, just waiting for someone to actually define the interface Unavoidable now really

yes its all too true how many people dont back up regularly, dont bother to read things before trying them

but id argue it is the responsibility of the app developer to avoid developing in a way that encourages problems - going back to the original point, the system temp folder is volatile and generally should not be trusted for serious applications where executable code is potentially at stake in a shared environment and smf already avoids using the shared temp area for sessions, extending stuff back into the shared area instead of avoiding it is a bad idea - i guess thats what ive been trying to get at

[kurosaki00]

Im having this problem too.
By adding the temp folder, I wont crash my site?
I think this happeend to me once.

[N. N.]

It won't do any harm. Please try adding it, and make sure it's chmodded as 755, or if that doesn't work, 775, or 777.

[kurosaki00]

Ok, thanks to all of you. Its working now.

[N. N.]

kurosaki00,
Good to hear that one worked!

holodoc, oya,
Thank you very much for the feedback and the very interesting debate.

Unfortunately ... I will have to agree with Oya on this one. Making a software package for a large community of people, which simply want to have a little way of building a community of their own online, on all kinds of hosts and with all kinds of backgrounds... forces us, IMHO, to take into account the target of the software, and the real conditions out there in which it is used. Specially since here, we're talking about security.
I wish I could see a way to do "the right thing" for the software itself (which is this case, would rather amount to let the host do their job and provide proper separation, let the operating system do its job, and do the forum software package job, which is to provide forum software nicely written, taking advantage of operating system resources and use them to run nicely and with less quirks - as holodoc advised)... but I cannot agree with a compromise with security.


holodoc, I don't doubt you're right... it is not exactly a forum software job to provide for hosts' shortcomings, and yet, if security is concerned, we can try to do our best. Maybe it's not "the right thing" for the forum software, but it is the right thing to do for our users, their data, their security. IMHO.
SMF is not for professionals or for Academia, it's for every user who wishes to try building a community quick and see what is like, and which may learn in time what is a good host and what is a bad host, but doesn't have to know from the start. SMF is not meant to run in vitro... and we're talking about security. Anything else, that bad/outdated hosts might not provide, I'd be happy to compromise about - and do the right thing for the software (ditch them outdated stuffs if necessary! ). (and you know there are , and will be considered for next versions). But with security, no, can't do.

[holodoc]

@Norv
No sweat about it Every suggestion, no matter how unlikely it is, followed by a constructive debate is for me personally always a win.

P.S. I've just modified my initial post to make the solution part more distinctive in order for people to actually see it.

[Joshua Dickerson]

Norv, are you saying this won't be committed and should be moved to fixed/bogus board?

[N. N.]

Yes. Thanks for the heads-up.