[Free or Paid] Mod to alternatively Upload & Restore attachments to Forum v2.1.4)

[Ufopolitics]

Hello, below is the form to request a Mod,

Subject: [FREE or PAID] Alternate Attachment Upload and Restore System

Version: 2.1.4

Description:  Mod should be able to offer an Alternate way for uploading attachment files from a specific folder on the root, to be uploaded via FTP and handled through Admin Panel.

1-Mod Panel should be able to show a list of all uploaded attachment files on that folder and have its own search engine within those uploaded (browsed) files, also displaying the attachment ID#
File would be uploaded FTP as (example):'20538_image.jpg' where first numbers before underscore hyphen is the attachment ID#

2- Mod would be able to -based on 'attachment ID#'- search and locate the placement of attachment(s) on Forum Topic#, Post# and Member# and ADD it to the MySQL DB Table 'attachments'.

3- Have the ability to compile large number of files processing on a 'batch type' operation.

Permissions: Only Administrators and Global Moderators would be able to use this Mod.
However, if possible, to open it up to specified member groups.

Feature Set: I need this Mod to have an Alternate way to Upload and Restore attachment files from an FTP Root Folder and add them on the Existing MySQL database 'attachments'.
It is a large Forum that I upgraded from v 2.0.19 to 2.1.4 and have like 135,000 attachments.
Unfortunately, the upgrade did some attachments (close to 20,000) but not the majority.

There is an existing great Mod written by ShawnB61==> AttachmentBrowser-v202 that I uploaded, and it works perfectly well even I have like close to 20,000 attachments now.
Mod can open attachments from the MySQL DB and browse them on Admin Panel, where it has two options: List or Grid, where it shows the actual image of attachment (except if it is a pdf or txt file) and gives exact location (Topic) of attachment, Member who uploaded, date it was uploaded, filename, size and number of times it was downloaded.
Unfortunately, this nice Mod cannot upload attachments from an FTP Folder and add them to MySQL Database Attachments Table.
It has only public permissions and cannot search within browsed files, nor gives the attachment ID# Field.

Thanks very much in advance

Regards

Ufopolitics

[Ufopolitics]

Hello again,

If it may be of any help,

I located a code written (inactive) within the smf 2.1.4 Source Folder: 'attachment.php' dedicated to Mod Authors for an Alternate Attachment Upload System

}
                //Below script is from Sources/attachment.php file:

      // Mod authors, finally a hook to hang an alternate attachment upload system upon
      // Upload to the current attachment folder with the file name $attachID or 'post_tmp_' . $user_info['id'] . '_' . md5(mt_rand())
      // Populate $_SESSION['temp_attachments'][$attachID] with the following:
      //   name => The file name
      //   tmp_name => Path to the temp file ($this->_attchDir . '/' . $attachID).
      //   size => File size (required).
      //   type => MIME type (optional if not available on upload).
      //   id_folder => $modSettings['currentAttachmentUploadDir']
      //   errors => An array of errors (use the index of the $txt variable for that error).
      // Template changes can be done using "integrate_upload_template".
      //call_integration_hook('integrate_attachment_upload', array());
   }

Thank you!!

[dodos26]

I understand that you want to send a package with attachments (example backup folder) and then add them to the database to make them visible, this is possible, but you need to know that attachments do not contain metadata needed to reproduce certain things.

Attachment Browser is good for u, and can be add feature but...
Only the attachment ID (unnecessary) and file hash can be recreated in DB. This essentially makes it impossible to create a search engine by name.

Unless you want to create the ability to send attachments manually where the file is not changed, has a name, etc., this can be done very easily.

[Ufopolitics]

Hello Dodos,

And thanks for your answers!

Ok, this is the issue I am having,

I have on MySQL DB the attachment folder which have like 135,000 attachments, and they have all the fields.
However, whenever I run the maintenance on attachments database it responds as image below:

You cannot view this attachment.

Now, as you can see, it reads:

136,127 attachments/avatars have an entry but no longer exist on disk?

And honestly, I do not understand this error description, as I am seeing all the database table is completely full with all attachments?

An "entry" from where?...and "disk" is referring to MySQL DB Table?

I am lost as I do not see the connections here...

[shawnb61]

The files themselves are stored in the file system.  As files.  The db stores info about the files, but not the actual files.

You need a file system backup for that.

But it sounds like you do?   Because that's what you want to upload, right?   With the hash names?

[Ufopolitics]

The files themselves are stored in the file system.  As files.  The db stores info about the files, but not the actual files.

You need a file system backup for that.

Hello Shawn,

Thanks, and where is that 'file system'?...is it handled through FTP?...

[shawnb61]

Ftp will let you see the files in a folder.  That's one way, yes.

It is usually easier to use the file manager in cpanel.

Do you have cpanel access?  Can you see the attachments folders?

[Sir Osis of Liver]

As per your other topic, attachment file maintenance is reporting the files do not exist because they don't have the .dat extension.  You don't need a mod to upload/restore attachments, you need to rerun the upgrade because it failed to add the extension to the attachments.  I'm guessing you have 135,000 attachments in one /attachment directory, so it's not surprising the server choked on it when running the upgrade.  You need to do what was suggested in other topic, restore the 2.0.19 forum, rerun the upgrade, if .dat extensions are still missing run the script I posted.

[Ufopolitics]

Ftp will let you see the files in a folder.  That's one way, yes.

It is usually easier to use the file manager in cpanel.

Do you have cpanel access?  Can you see the attachments folders?

Hello Shawn,

Do you mean on the Hosting Panel?...all I have there is the FTP, same as FileZilla but slower.

I see the files on the attachment folder, and yes, I added the .dat extn...

Only thing I have not done is to run the script that Sir Osis gave me...I will try to run that

[Ufopolitics]

As per your other topic, attachment file maintenance is reporting the files do not exist because they don't have the .dat extension.  You don't need a mod to upload/restore attachments, you need to rerun the upgrade because it failed to add the extension to the attachments.  I'm guessing you have 135,000 attachments in one /attachment directory, so it's not surprising the server choked on it when running the upgrade.  You need to do what was suggested in other topic, restore the 2.0.19 forum, rerun the upgrade, if .dat extensions are still missing run the script I posted.

Hello Sir Osis,

It is not that I am not paying attention to what you are writing, it is that doing an upgrade again is going to ruin all the hundreds of hours I already have done as adding mods, plus theme changes, new topics, posts etc...
I want to run your script I already downloaded, so it goes on the main root of system like:

myforum.com/yourscript.php?

So, what happens to the files I already added the .dat extn?

Thanks so much!!

[Sir Osis of Liver]

Assuming the 2.1 upgrade completed successfully, and you haven't damaged your database by tinkering with it, and the attachment files are in the correct forum directory, and settings are pointing to that directory, and you did not run the fix in attachments integrity check, adding the .dat extension to all files that don't have it should correct the problem.  If you did run the fix in attachments integrity check, you'll have to restore the database from backup, because all of the attachment links will have been removed from the db.

The script adds the .dat extension only to files that don't already have it.

[Ufopolitics]

Hello to All and many thanks for your time,

I just realized what happened here...

The owner of this old Forum sent me the My SQL DB in a compressed (zip) package and also on MySQL format, which I uploaded to the SQL Server...

But He sent me separated the attachment folder with all files...but the transfer never completed!

I only have like 35,000 attachment files...I am missing around 100,000 files!!

He is very ill, and had zero patience when handling this transfer, and more likely he stopped it before it completed.

There is absolutely nothing we could do about this...

So, what I am doing is adding a Gallery with all attachments that are loose, which are like 20,000...and members should sort them out...
Only thing this gallery by smf tricks is great, but it upload images "one by one" it does not allows the selection of multiple files...
So, I am guessing the Pro Version-(Paid for), would do that?...batch upload?

Anyways, I will ask them before I purchase it.

Thanks again to all!!

[Sir Osis of Liver]

You'd have to ask in the mod support topic, but I don't think any gallery mod will do batch uploads, it's not how they work.  The other problem is the filenames are all hashed, so how would you or your users identify them if you uploaded them to a gallery?  The attachment files you have should work correctly in the forum once you've added the .dat extension, the missing ones will just not load, you'll get a link below the post with no thumb and it will 404. 

It may be possible to restore the original filenames because they are saved in database attachments table, but not easily, and don't know if that would really help.

If the forum has been running recently, host may have backups of the forum files and database.  Any decent host will have 30 days of backups for everything in the account.

[Sir Osis of Liver]

You can look through some of this for ideas -

https://www.startpage.com/do/dsearch?q=gallery+batch+upload+images&cat=web&language=english

[Ufopolitics]

The other problem is the filenames are all hashed, so how would you or your users identify them if you uploaded them to a gallery? 

Hello Sir Osis,

Because these images are about their work...just by looking at them they (each member) would recognize it as his displayed work,,,
It is about electrical tests, measurements, building prototypes, the processes, electrical an electronic circuits, diagrams, 3d images cgi, animated gif's...short movies... etc, etc...

Thks

[Sir Osis of Liver]

The remaining attachments should display correctly on the forum if the database hasn't been damaged, so you may as well go with that.

[dodos26]

You'd have to ask in the mod support topic, but I don't think any gallery mod will do batch uploads, it's not how they work.  The other problem is the filenames are all hashed, so how would you or your users identify them if you uploaded them to a gallery?  The attachment files you have should work correctly in the forum once you've added the .dat extension, the missing ones will just not load, you'll get a link below the post with no thumb and it will 404. 

It may be possible to restore the original filenames because they are saved in database attachments table, but not easily, and don't know if that would really help.

If the forum has been running recently, host may have backups of the forum files and database.  Any decent host will have 30 days of backups for everything in the account.

It has a database of these attachments. And about 40% of the files in the folder are attachments. I can write a simple script that changes the names and other metadata from the database to recreate the files.

[Sir Osis of Liver]

The files are missing, OP has a partial download of /attachments, but it's not complete.  You can't recreate the files that don't exist.

[dodos26]

From what you said, you have some attachments (these can be recovered if you have a database), unless you don't have them at all, in which case you'll have to try to recover the data from the old hosting.

[Kindred]

The attachments that he has are just fine.  No need to be recovered.

He's missing some 100,000 additional files which, in addition to bad assumptions, is what caused his problems in the first place.

In other words,  as we have said several times now --- there is literally nothing that can be done without the files, snd if he gets the files,  then the system can be used in one of the three ways, as he has been instructed to restore them.