Missing avatars and attachments

[mantu2]

I tried to find help to my problem, but I couldn't find exact mach anywhere. It might be that I have two seperate issues making things go wrong. Main problem is, that some attachemnts and avatars just gets lost with no reason. I'm using SMF 2.0.7.

- Some attachements stopped working after I moved my forum to new server 1-2 years ago. It might be because of using Filezilla, the attachment files got corrupted (bad back upps). Is there way to make these files to work again? During the move, I used SMF 1.

- I updated to 2.0.7. and after that there has been some avatars and attachments losts. No direct reason found. Once I did the forum repair and therefore some of non-working files were lost, not all though.

- Some files works, some doesn't. Forexample one attachement which is working, doesn't have a thumb-file in the server (attachement-folder). In other hand the file which doesn't work, has a thumb-file. The non-working-file is not showing the thumb-file on the forum site. I checked differences in PHPMyAdmin and I found out that those files not working has no lost of bytes, but mime_type were not found from the file which didn't work. I tried to put correct mime_type to one attachement line (and the thumb-file line) but no changes happened.

So... Some help needed.

[Kindred]

if you have files which were transferred incorrectly by filezilla the nly way to recover them would be to re-transfer the ORIGINAL, WORKING file...   there is no way to fix a broken file.


aside from that, I have no real idea what could be causing the other issues you describe...

(you should upgrade to 2.0.8 )
url?
mods installed?

url which shows a broken attachment?

[mantu2]

if you have files which were transferred incorrectly by filezilla the nly way to recover them would be to re-transfer the ORIGINAL, WORKING file...   there is no way to fix a broken file.

Damn. I think this is impossilbe now.

Few mods have been installed:
- Ad Managment (not in use)
- Ohara YouTube Embed (Gives error about language file missing (I'm not using English)),
- side bar (own code, seperated from main code, shouldn't cause any problems)

www.vapaalehdykka.net/forum

I updated to 2.0.8. Need to have a bit time to find out did that help or not. So far I think not. The test attachement files can be found from this topic: http://www.vapaalehdykka.net/forum/index.php?topic=1102.0

[kat]

If you still have the files on your hard disk, you could try uploading them, again, overwriting what's there. Before you do, change Filezilla's "Transfer type" to "binary".

If you deleted them, you could try a file recovery program. But, I doubt that'll work, coz there'll be no file extensions.

You might be lucky, if you try to get them from your previous host. My old host's files till existed five years after I left them!

[Sir Osis of Liver]

if you have files which were transferred incorrectly by filezilla the nly way to recover them would be to re-transfer the ORIGINAL, WORKING file...   there is no way to fix a broken file.

IIRC, underdog found a way to fix attachment files damaged by FZ ascii transfer.  Believe it involved using a different ftp client to dl/ul the borked files.  Don't have time now, but will see if I can find it tonight. 

[mantu2]

if you have files which were transferred incorrectly by filezilla the nly way to recover them would be to re-transfer the ORIGINAL, WORKING file...   there is no way to fix a broken file.

IIRC, underdog found a way to fix attachment files damaged by FZ ascii transfer.  Believe it involved using a different ftp client to dl/ul the borked files.  Don't have time now, but will see if I can find it tonight.

Yey - that would be very good info to get. Looking forward to hear more!

[Sir Osis of Liver]

Didn't find underdog's post, but here's one with same info -

http://www.simplemachines.org/community/index.php?topic=519142.msg3675133#msg3675133

Haven't tried this myself, but looks like you just have to copy the damaged files from one folder to another with WinSCP, and it fixes them.

Let us know if it works.

[mantu2]

Ok. Didn't work. Notes:

- I'm using Ubuntu -> programs works with Wine -> wery slow to handle anything with that program.
- I copied files to new folder in the server and tried also to download them from my computer and then transfer to server. I made a new folder each time and after transfers I renamed it to "attachments" as it should be.
- Didn't work. Don't know did I do something wrong. The work took much time.

[kat]

I know that this might sound totally barking... But, quite by chance, I found a video editing program that fixes quite a few doinked picture files.

If you load a file into Virtualdub and the file's corrupt, it attempts to repair it and seems to do a pretty decent job of it, too.

To save them, you need to "Export as image sequence".

[Sir Osis of Liver]

Will try WinSCP when I get a chance.  I believe underdog used it to repair attachments on a trashed forum we worked on earlier this year.  When I d/l attachments from my forums as ascii, they are damaged but still work, so I'm not duplicating the problem most people have.

[Arantor]

I thought WinSCP wouldn't actually touch the contents of a file, so that when it was downloading, it was downloading purely in binary the whole way through anyway (meaning the problem is a non-issue)?

Then again I use WinSCP for the fact it actually encrypts the password in transit unlike FTP (yes when you use FTP you are sending your password through the internet unprotected)

[mantu2]

Well this gets interesting!

I got back ups from my old server (yey, they did have them after 18 months I closed the service \o/ Needed to pay a bit, though) and I downloaded the files with FileZilla (Binary mode) to my computer and from there to my test forum site. And guess what. It didn't work. I'm getting a bit frustrated. Just don't get it where the problem is. Let's make this as simple as it can get so there is no possibilities for errors in my acts, please.

Notes:
- After I did the back up transfer, nothing changed. Older files than February 2013 is not working (In this time and earlier I used SMF 1.X.X)
- When I look the attachments from database, I found out that not working files are having a thumb file which has 0  b as file size. (Let's say attachement 1037 is the real pic and 1038 is the thumb file). Thumb files can also be found from attachment folder. Thumb files are not shown in the forum area, only link to the file itself can be found. If willing to see the attachment, forum asks me to download the picture and after that the picture opens in a picture viewing program, however, no picture can be seen. There comes an error note: "Error while reading JPEG picture (Quantization table 0x00 was not defined)". On these files there is no data in mime_type column.
- On a working file the structure is same: A file and thumb file (with zero bytes). In mime_type field there is "image/jpeg" for the attachment file and "image/png" for the thumb file.
- These notes has been taken from few files, not all of them. There is totally a bit under 1000 attachment files in the forum.

EDIT 1: In the server, you can find files 1037, 1037 (as a thumb-file) and 1038 (the real thumb file according to the database)

EDIT 2: Stupid me. I was looking at database for my real forum and dealing with the test forum site. Well. I transfered the new files (as in the first one) to my real forum and I got the attachment 1037 to work. Well - what happened. I have no idea. Somehow it just started to work. Now there is no line for 1038, but a new for thumb file. So apparently the system deleted itself line 1038 and made new one for thumb file. But, but, but... There is still lots of files not working. Many of them doesn't have file_hash, which is a bit weird. I tried to copy the line from the file name, but it didn't work.

[kat]

Is your test site a copy of your live site (With paths changed)?

[mantu2]

Yap. It is.

[kat]

Something's telling me that there's something crucial, here, that I'm missing... I have a bit of brain-fog, though, coz my mind's on a couple of friends who have recently died...

So, this is me, kinda mumbling, out-loud, with a bit of reasoning. Some of which MIGHT be sound...

Let me go through this...

You had the images. They were corrupt. That's just a possibility, though. Downloading them should've been OK, even in "auto" mode. But, uploading them to your new server, using "auto", might well have screwed them. What about the actual PHP files, themselves? I know that "auto" SHOULDN'T affect PHP files. But, from personal experience, I know that it damned-well CAN. Have you tried replacing the PHP files, too? Would that even affect what we're doing? I'm thinking "Possibly".

Paths... If the database, on your test site, is an exact copy of the one on your live site, would there not be paths, there, that point to your live site, rather than your test one? Would that affect attachments? Something tells me that it might well. I can't seem to pinpoint why I think that, though, at this, precise, moment...

Heck... I'm not being terribly helpful, here, am I?

Am I making any sense, thus far?

[mantu2]

Hmm. I need to think a bit. It might be good thing to find the pictures which doesn't work and see if there is similarities between those files. This takes 1-2 evenings me to complete. I would say at least some files were corrupted because some (not all) attachments started to work after I moved back ups (although I don't understand why = some database changes as I wrote earlier). Some time ago, I did attachment fixing and after that I found out that some files were deleted during the fixing process. It might be that the files I'm looking after at the moment are those which are deleted. Don't know for sure, though.

But. I'll go a bit deeper in the filing system and try to understand what is the logic for the non-working files.

[mantu2]

I did some research for the database. I cannot find any reason (like one main reason) why some files are working and why some are not working. The uploading date for files doesn't play big role in the old files because some files from 2009 are working, some are not. I went through ~100 files. Notes:

- Mid attachments files works, but some pictures and pdf files are partly working (color mistakes mostly) and some picture and pdf files are don't working.
- In some cases there is one post with many attachments. Some times some of attachments are not working, some are. There is no pattern which are working which are not. Feels very random.
- Some of non-working files are having thumb files two times. For example:

100_C-kaavio_JPG652d518fde96dc3b920362e90c47b8a0 (actual file, database links this to file number 101)
100_C-kaavio_JPG652d518fde96dc3b920362e90c47b8a0_thumb (thumb file)
101_C-kaavio_JPG_thumb3566469755081b9372eb289034d7c660 (thumb file)

- Many files have no file_hash, specially when the attachment file numbers are small. I think the forum coding has changed during the time and therefore there is coding differences between old and new files.
- Some attachments and thumb files are working so that there is new line for the thumb. It seems that the forum makes now line for thumb if the original thumb is not working... ...but not always.

What kind of information do you guys need to solve this problem? For me is ok if some body can help me with Skype. Might be faster.

[kat]

Have you tried going to Forum Maintenance and repairing/recounting?

And, perhaps, repairing the database from CPanel?

[mantu2]

I have done once the forum maintenance repair action. If I remember correctly it found quite much to repair and after I pushed the repair-button, it lost some files. At least I think so.

repairing the database from CPanel - how do I do that?

[kat]

CPanel>Databases>MySQLDatabases (Oddly enough).

You'll find options to check/repair, there.