Package is empty - Workaround?

[NanoSector]

When I get my program to ZIP a package together it gets zipped fine and every file manager I know can read it - But SMF thinks it's empty.

Is there any workaround for this?

[Joker™]

Which OS and zip tool are you using?

[NanoSector]

Which OS and zip tool are you using?

OS - Windows 7 64-bit
My program uses Ionic.Zip.dll, found here: http://dotnetzip.codeplex.com/

If I zip things up with PeaZip again, it works.

[NanoSector]

Okay... When I look at the package that SMF has extracted:
http://prntscr.com/nj91g

And the package that PeaZip extracts:
http://prntscr.com/nj93k

SMF adds weird stuff...

EDIT: Got SMF to read the package by disabling all compression.. Trying to get it up a bit so the packages aren't that big.
EDIT2: Apparently not possible. Meh, people'll have to do with big packages then I suppose...

My code:

Code Select Expand

            // Start the ZIP process.
            using (ZipFile zip = new ZipFile())
            {
                zip.CompressionMethod = CompressionMethod.Deflate;
                zip.CompressionLevel = Ionic.Zlib.CompressionLevel.Level0;
                zip.UseZip64WhenSaving = Zip64Option.Always;

                // Add the Package directory to the root of the ZIP file.
                zip.AddDirectory(workingDirectory + "/Package");

                // Then add the Source directory to the files directory of the ZIP file.
                zip.AddDirectory(workingDirectory + "/Source", "files");

                // Now we can save the ZIP.
                zip.Save(sf.FileName);
            }

[emanuele]

Which OS and zip tool are you using?

OS - Windows 7 64-bit
My program uses Ionic.Zip.dll, found here: http://dotnetzip.codeplex.com/

I was going to ask you about that because it's the first error I got running it on monodevelop...

[NanoSector]

Which OS and zip tool are you using?

OS - Windows 7 64-bit
My program uses Ionic.Zip.dll, found here: http://dotnetzip.codeplex.com/

I was going to ask you about that because it's the first error I got running it on monodevelop...

I didn't even get it to run on MonoDevelop...

[emanuele]

Commented out a couple of things and it works...except that on the file picker it crashes... lol

[Arantor]

I'd be interested to see the zip files you were getting, though I suspect I already know what the problem is, it's producing a zip file that doesn't fit with the way SMF unpacks them.

[NanoSector]

I'd be interested to see the zip files you were getting, though I suspect I already know what the problem is, it's producing a zip file that doesn't fit with the way SMF unpacks them.

If you want me to I can generate one while disabling the "correct" code
Give me a few moments.

EDIT: Attached. brokenPackage is broken and workingPackage is working, quite straightforward I believe

I viewed both packages in Notepad++ and they did have differences.

[Arantor]

Yeah, it's because SMF is buggy, not your project. The broken package is set up in a different order, and with some slightly different parameters, especially with handling the 'extra' field differently.

[emanuele]

In 2.1 works! (Spuds's job) ...at least here.

[Arantor]

Let's just say I've gotten to know zip files *very* well lately, having to write my own handling class from scratch because nothing else out there does what I need as efficiently as I can do it.

[NanoSector]

Yeah, it's because SMF is buggy, not your project. The broken package is set up in a different order, and with some slightly different parameters, especially with handling the 'extra' field differently.

I'm wondering, does the PHP ZIP reader read the broken package correctly?
In the past it read all the packages SMF did not read.

Glad to know it's not my project though

[Arantor]

Yes, it does, but it has several PHP 5.3 issues, while PclZip can read them as well - except it's about 3000 lines longer than it needs to be.

[NanoSector]

Yes, it does, but it has several PHP 5.3 issues, while PclZip can read them as well - except it's about 3000 lines longer than it needs to be.

Sorry for asking so many questions, but why exactly does SMF use a custom reader?

[Arantor]

Because back in 2004 when this stuff was first written, there wasn't anything else. The PEAR class didn't exist, PclZip didn't exist, so it was written from scratch. It's never been ideal.

[NanoSector]

Because back in 2004 when this stuff was first written, there wasn't anything else. The PEAR class didn't exist, PclZip didn't exist, so it was written from scratch. It's never been ideal.

Oh okay, thanks

[Arantor]

FWIW when I wrote my own package server, I just invoked 7-zip via command line to generate packages, it seemed much easier than trying to programmatically generate packages.

[NanoSector]

FWIW when I wrote my own package server, I just invoked 7-zip via command line to generate packages, it seemed much easier than trying to programmatically generate packages.

I tried doing that with RAR.exe, but gave up and searched for a native C# library when it did work from the command line yet didn't work when running from my program. Don't regret doing that now

[Arantor]

There is a reason I *specifically* said 7zip. Not only is it pretty easy to drive from the command line, it is absolutely known to work at generating SMF compliant zip files.

Lemme explain. When I was developing SimpleDesk, I created a package server. Really simple - a static packages.xml file that referenced simpledesk.zip, a simple .htaccess rule to redirect requests for 'simpledesk.zip' to my download.php file.

Then the download file itself:

Code Select Expand

delTree('C:/Dev/public_html/intranet/simpledesk_server/package');
smartCopy('C:/Dev/public_html/intranet/mods/simpledesk/', 'C:/Dev/public_html/intranet/simpledesk_server/package');
chdir('package/');
exec('"C:\\Program Files\\7-Zip\\7z" a -rtzip simpledesk.zip *.*');

header('Content-type: application/zip');
readfile('simpledesk.zip');

function smartCopy($source, $dest, $options=array('folderPermission'=>0755,'filePermission'=>0755))
{
$result=false;

if (strpos($source, '.svn') !== false || strpos($source, 'simpledesk.zip') !== false)
return;

if (is_file($source)) {
if ($dest[strlen($dest)-1]=='/') {
if (!file_exists($dest)) {
cmfcDirectory::makeAll($dest,$options['folderPermission'],true);
}
$__dest=$dest."/".basename($source);
} else {
$__dest=$dest;
}
$result=copy($source, $__dest);
chmod($__dest,$options['filePermission']);

} elseif(is_dir($source)) {
if ($dest[strlen($dest)-1]=='/') {
if ($source[strlen($source)-1]=='/') {
//Copy only contents
} else {
//Change parent itself and its contents
$dest=$dest.basename($source);
@mkdir($dest);
chmod($dest,$options['filePermission']);
}
} else {
if ($source[strlen($source)-1]=='/') {
//Copy parent directory with new name and all its content
@mkdir($dest,$options['folderPermission']);
chmod($dest,$options['filePermission']);
} else {
//Copy parent directory with new name and all its content
@mkdir($dest,$options['folderPermission']);
chmod($dest,$options['filePermission']);
}
}

$dirHandle=opendir($source);
while($file=readdir($dirHandle))
{
if($file!="." && $file!="..")
{
if(!is_dir($source."/".$file)) {
$__dest=$dest."/".$file;
} else {
$__dest=$dest."/".$file;
}
//echo "$source/$file ||| $__dest<br />";
$result=smartCopy($source."/".$file, $__dest, $options);
}
}
closedir($dirHandle);

} else {
$result=false;
}
return $result;
}

// Courtesy of http://www.php.net/manual/en/function.rmdir.php#91797
function delTree($dir) {
if (!file_exists($dir)) return true;
if (!is_dir($dir)) return unlink($dir);
foreach (scandir($dir) as $item) {
if ($item == '.' || $item == '..') continue;
if (!delTree($dir.DIRECTORY_SEPARATOR.$item)) return false;
}
return rmdir($dir);
}

The first line erased the 'local working copy', the second copied from the local SVN repo, recursively, while ignoring .svn entries (this was before SVN 1.7 removed the whole 'every folder has to have a .svn folder' thing), and because it was less effort than getting svn command line binaries on Windows.

The third got to the right folder, the fourth calls the generation of zip files, a (add), -rtzip (recursive, type zip), into simpledesk.zip and applying all files.

Then the last two lines of the core script would set the correct header and just pass through the file. Not elegant, not pretty at all but very good at making sure the package was clean and instantly gave me a perfect export of my local repo in the format I wanted.

[NanoSector]

I thought about using 7zip, yes, before realizing that tinkering with the settings for DotNetZip may of have worked, which it did in the end. I'm still new to C# so I might of have made a mistake in my command line call in my program when using RAR.exe. Thanks for showing me your code

[Arantor]

I just figured my PHP might be useful to someone

[NanoSector]

I just figured my PHP might be useful to someone

Yeah. If only I were writing it in PHP...

[Arantor]

*shrug* I shared it because it covers more than just the code.

[NanoSector]

*shrug* I shared it because it covers more than just the code.

True, it covers how to use 7zip too. I might even port my program to use 7zip later on when users are gonna complain the packages are too big, but I think most users will not care. Thanks though

[Arantor]

If you're building a .NET app, it's already going to be a multi-MB download in the first place. Bundling the 7zip CLI executable (licence permitting) will add a few hundred KB at most.

[NanoSector]

If you're building a .NET app, it's already going to be a multi-MB download in the first place. Bundling the 7zip CLI executable (licence permitting) will add a few hundred KB at most.

I'll try to build a better zipping process using 7zip now, just to see how this works. DotNetZip is an external library too.

[Arantor]

Just remember that 7zip isn't a library, you can't use it as such.

[NanoSector]

Can't get it to run, sorry :S

Code Select Expand

            // Start the ZIP process.
            Directory.CreateDirectory(workingDirectory + "/temp");
            bool stat = CopyFolderContents(workingDirectory + "/Package", workingDirectory + "/temp");
            if (stat == false)
                MessageBox.Show("Unable to compile package; an error occured while copying files.", "Compiling Project", MessageBoxButtons.OK, MessageBoxIcon.Error);

            Directory.CreateDirectory(workingDirectory + "/temp/files");
            stat = CopyFolderContents(workingDirectory + "/Source", workingDirectory + "/temp/files");
            if (stat == false)
                MessageBox.Show("Unable to compile package; an error occured while copying files.", "Compiling Project", MessageBoxButtons.OK, MessageBoxIcon.Error);

            string cmdText = "7za a -rtzip \"" + sf.FileName + "\" \"" + workingDirectory + "/temp\"";
            System.Diagnostics.Process.Start("CMD.exe", cmdText);

            Directory.Delete(workingDirectory + "/temp");

I know it errors when entering the last line but it's not that what I'm worried about.

It opens a cmd.exe process, jumps to the directory but does nothing more.
This is the cmd.exe window:
http://prntscr.com/no0ee

Dunno what I did wrong

[Arantor]

Probably because you're calling the wrong thing? 7za != 7z

[NanoSector]

Probably because you're calling the wrong thing? 7za != 7z

The executable is called 7za.exe in my project. The 7z command only works if someone has the actual program installed
Even when I had the .exe added in my code it would do the same thing.

[Arantor]

*shrug* It worked for me, can't say any more than that.

[NanoSector]

*shrug* It worked for me, can't say any more than that.

Well, that leaves me with my old working code, thanks for trying to help me anyway