Simple Machines Community Forum

SMF Support => SMF 1.1.x Support => Aiheen aloitti: archiebald - helmikuu 04, 2005, 02:47:00 AP

Otsikko: Big Attachments Working But Not Efficiently [solved]
Kirjoitti: archiebald - helmikuu 04, 2005, 02:47:00 AP
Hi there,
My first post here and a general nooby to forum creation.

I wanted to specifically create a forum to include large file attachments for distribution of data files with comments in the text.  I know I should use FTP but this project is to be targetted at the computer illiterate so needs to be unbelievably simple and I want users  to be able to upload as well, not only the admins.

Before I start, all of this is being done on a localhost computer on my company network for experimentation before trying to deploy it.  2.0GHz P4 running Windows XP Pro, IIS 5.1, PHP 4.3.1, mySQL 4.1 and SMF 1.0.1

OK, here is where I am at.  I have managed to succeed in getting any size attachment to be handled by SMF and PHP.  For those that might be interested, see the link below.

Anyhow, now to my problem.  I have set up everything to handle 100MB sized attachments and uploading them to the localhost server is no problem at all.  The problem is with downloading.

When a user tries to download an attachment, the following things occur.

> The localhost CPU becomes 100% loaded - The process using the CPU is dllhost.exe and the user is IWAM_PAUL (PAUL is the name of the computer)
> The Page File Usage rises by about the same size of the file to be downloaded.
> For a 17MB file. the whole process takes about 2 minutes.  For a 90MB file, I gave up waiting after about 5 minutes.
> There is no hard disk activity during this delay time, only CPU usage.

Once the "preparation for download" has been completed, the download confirmation box appears on the user computer.  After clicking the confirm button, actual downloading speed is no problem.

With IE6, this occurs on every subsequent attempt but in Firefox 1.0 it only occurs on the first try, subsequent response time is instant.  If I clear the Firefox cache, it again shows the long delay so I guess it has something to do with the user's cache in Firefox, .

If I post a link to the file and use that to "Save Target As...", the response time is immediate.  As far as I can see the problem only lies within attachments.  I really need this function to enable my users to upload onto the server easily.

Am I therefore correct in assuming this is some limitation in PHP and hence the general limit of 2MB set in the standard php.ini file?  If so, any workarounds would be most welcome.

For anyone frustrated by the 2MB limit, go here - http://www.squirrelmail.org/wiki/en_US/AttachmentSize.
Otsikko: Re: Big Attachments Working But Not Efficiently
Kirjoitti: [Unknown] - helmikuu 04, 2005, 04:08:44 IP
Well, the major problem here is going to be compressed output.  It can hardly stream the file if it needs to gzip it on the way.

In Display.php, find and remove:
if (!empty($modSettings['enableCompressedOutput']) && @version_compare(PHP_VERSION, '4.2.0') >= 0)
@ob_start('ob_gzhandler');
else


Then find:
if (@readfile($filename) === null)
echo implode('', file($filename));


Replace:
if (filesize($filename) > 1024 * 1024)
{
@ob_end_clean();
@ob_end_clean();
@ob_end_clean();

$fp = fopen($filename, 'rb');
while (!feof($fp))
{
echo fread($fp, 8192);
flush();
}
fclose($fp);
}
elseif (@readfile($filename) === null)
echo implode('', file($filename));


Any better?

-[Unknown]
Otsikko: Re: Big Attachments Working But Not Efficiently
Kirjoitti: archiebald - helmikuu 06, 2005, 08:12:39 IP
Thanks for the input, unfortunately the code causes some other problem as described.

In IE, the users browser switches to a blank new window with the address http://192.168.0.85/index.php?action=dlattach;topic=6.0;id=13.

The window remains blank and the file download dialog box never appears.

In Firefox, even more strange, the file download dialog box appears but seems to have been corrupted.  Please see the attached files for what I mean.   AAARGH!! I just noticed there is no option for attachments here!!!  Anyhow, the combo box for opening the file is displaced to the bottom of the dialog box and becomes the full width of the dialog box (just above the OK / Cancel buttons).  Oh, and BTW, nothing happens - no download when you click OK.

Same thing occurs for attachments both large and small

Just to be sure I made no mistakes, I copied and pasted your code into Display.php

I also reverted back to the original Display.php to test that nothing else had somehow been changed.  When I did this, all went back to normal (i.e. the original overloading on CPU usage)

If I can just overcome this one obstacle, I will be over the moon.

waiting in anticipation,

Archie
Otsikko: Re: Big Attachments Working But Not Efficiently
Kirjoitti: [Unknown] - helmikuu 06, 2005, 08:23:55 IP
Hmm, okay, for the first step, instead of:

In Display.php, find and remove:
if (!empty($modSettings['enableCompressedOutput']) && @version_compare(PHP_VERSION, '4.2.0') >= 0)
@ob_start('ob_gzhandler');
else


Try replacing that code with:

header('Content-Encoding: none');

-[Unknown]
Otsikko: Re: Big Attachments Working But Not Efficiently
Kirjoitti: archiebald - helmikuu 06, 2005, 09:09:09 IP
 :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :)

Sheer Bloody Genius!!!!!!


:) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :)

BRILLIANT!!!  Many Thanks!!!

Now takes just 3 or 4 seconds to prepare for download, even for 100MB files

BTW, another question.

In the Admin setup --> Edit Features & Options  --> Basic Forum Features there is a check box titled "Enable Compressed Output"

Does this have any relation with the problem I was experiencing?

Just curious

Archie
Otsikko: Re: Big Attachments Working But Not Efficiently [solved]
Kirjoitti: [Unknown] - helmikuu 06, 2005, 09:13:05 IP
Yes, had it been unchecked you wouldn't have noticed my mistake.

-[Unknown]
Otsikko: Re: Big Attachments Working But Not Efficiently [solved]
Kirjoitti: vodkarev - tammikuu 08, 2006, 05:27:01 IP
Does this work with all versions? I'm using SMF 1.1 RC2