Simple Machines Community Forum

SMF Support => Server Performance and Configuration => Topic started by: AncientDragonfly on April 02, 2010, 11:52:18 AM

Title: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on April 02, 2010, 11:52:18 AM
Filezilla has changed one of their default settings which will destroy your forum's avatars and attachments if you depend on it to transfer your files. 

There is a setting under the Edit menu --> Settings --> Transfers --> File Types: "Treat files without extension as ASCII file."

(https://www.simplemachines.org/community/proxy.php?request=http%3A%2F%2Fwww.simplemachines.org%2Fcommunity%2Findex.php%3Faction%3Ddlattach%3Btopic%3D370199.0%3Battach%3D137471%3Bimage&hash=1c3ca21e7420582031e29ac2122d9b48)

This should be unchecked when transferring directories that contain avatars or attachments because these are binary files.

kerbob has written instructions for backing up your site using Filezilla.

Note to all FTP users: If you are backing up your site via FTP and you're not changing the mode of transfer for files without extensions.. your doing it wrong and risk having bad backups.  The process below is specific to FileZilla but applies to any FTP client that auto determines the transfer file type (ASCII or BIN) for certains files.

SMF BACKUP PROCEDURE USING FILEZILLA FTP
1.) Make certain that FTP client is configured to use transfer type AUTO for all files and that "Treat files without extension as ASCII file" is checked and "Treat dotfiles as ASCII files" is checked.
2.) Re-check Step 1.  Highlight all remote folders on your ftp site EXCEPT the Attachments folder and download them. Why?  These folders contain ASCII files without extensions and must be transferred as ASCII.
3.) Change the FileZilla configuration so that "Treat files without extension as ASCII file" is unchecked.
4.) Re-check Step 3.  Highlight the remote Attachments folder and download it.  Why? This folder must be downloaded as Binary because it contains Binary files (jpg, gif, png, etc).

Does anyone know if there is any other folder within SMF that may have BINary files that have no extensions?

It is also a good idea to make changes (updates, modifications, etc.) on a test forum before changing your live forum, because you have an opportunity to check that everything is working correctly before you make irreversible changes to your live forum.  Using a test forum prevented me from losing my avatars and attachments because I noticed the problem on my test forum first, and left my live forum alone until I found the problem. 

To make a test forum which is a mirror of your existing forum:

1.  Backup your current forum's files and database.  If you are using Filezilla, be sure to follow kerbob's instructions above.  I like to use phpMyAdmin to back up my database, but the SMF Admin section also provides a method to do this.

2. Create a new space within your website for the test forum, and upload the files you backed up.  Again, if you are using Filezilla, be sure to follow kerbob's instructions above.

3. Create a new database through your web host's control panel with phpMyAdmin.  Make note of the database name, username, and password which you created when you made the new database.

4. In the new database, use phpMyAdmin to restore your backed up database from the live forum. 

5. Upload the file "repair_settings.php (http://download.simplemachines.org/index.php?thanks;filename=repair_settings.php)" to the root directory for your test forum.  (For example, if you put your test forum in the directory, www.example.com/testforum/, put repair_settings.php in that directory. I would recommend using a unique name for your test forum directory, as "testforum" is too easily guessed.  If you're of the tinfoil hat variety, I would also recommend password protecting the directory with .htaccess.)

6. Run repair_settings.php by going to http://www.example.com/testforum/repair_settings.php and change all the entries to match your test forum.  What is repair_settings.php (http://wiki.simplemachines.org/smf/What_is_repair_settings.php)? 

If you do not do this, any changes to the database, theme, posts, etc. that are done through the forum software will be made to your live forum instead and defeat the purpose of having a test forum.

7. Important: DELETE repair_settings.php from your site; it can be used to get your database password, and to change crucial settings that make the forum work.

8. Test your test forum.



Thanks to kerbob for the Backup instructions and for bringing this issue to light.  There is more discussion in the thread: http://www.simplemachines.org/community/index.php?topic=370199.0

Thanks to Jason of CharlottezWeb.com (http://www.charlottezweb.com/) for the instructions and video tutorials (I hope he doesn't mind me linking them) and for being a super great host for SMF forums.

There is more information about restoring a MySQL database here: Restoring a MySQL Database (http://wiki.simplemachines.org/smf/Restoring_a_MySQL_Database)

And yet more about backing up and restoring here: [Tutorial] ''How to backup & restore your forum in several different ways'' (http://www.simplemachines.org/community/index.php?topic=184356.0)
Title: Re: Important Notice: Do you use Filezilla?
Post by: Adish - (F.L.A.M.E.R) on April 02, 2010, 11:57:15 AM
I had this issue previously. Since then I had been really careful about it. Thanks for the heads up once again. I am sure this will help alot of users around. :)
Title: Re: Important Notice: Do you use Filezilla?
Post by: Arantor on April 02, 2010, 11:58:20 AM
Thanks for the tip - this has been an issue with FileZilla since SMF 1.1.9 / 2.0 RC1-1.
Title: Re: Important Notice: Do you use Filezilla?
Post by: AncientDragonfly on April 11, 2010, 09:48:26 AM
I had this issue previously. Since then I had been really careful about it. Thanks for the heads up once again. I am sure this will help alot of users around. :)

It will help, if it can stay up where people who take the trouble to look can see it.  I see a new thread from yesterday about more lost avatars and attachments. 

What is the proper way to ask for the thread to be stickied?
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Adish - (F.L.A.M.E.R) on April 11, 2010, 09:53:10 AM
You should ask one of the support team members to do it for you. They are the one who have the permissions to do it.

You may also hit the "Report to moderator" and write there what you want done. Maybe that will send a message to everyone.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on April 11, 2010, 10:04:20 AM
Thanks, (F.L.A.M.E.R).  I'll use "Report to moderator" since so many staff members have "do not PM me" in their sigs. Just didn't want to report my own post and maybe get myself banned.   :)

Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on April 11, 2010, 10:09:34 AM
The reason for 'no PMs' in our sigs (both when I was staff and not) is simple: so many people send support PMs expecting an answer when they should use the forum.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on April 11, 2010, 10:11:52 AM
The reason for 'no PMs' in our sigs (both when I was staff and not) is simple: so many people send support PMs expecting an answer when they should use the forum.

I know.   :)  Just makes me hesitant to PM anyone, whether it's for support or not.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on April 11, 2010, 10:12:38 AM
I can't speak for anyone else, but I know that if you PM me and it isn't support, good chance of getting a reply. But I have to just say 'no PMs' otherwise most people don't listen.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Chas Large on April 11, 2010, 10:17:29 AM
Topic Stickied as requested. Good tip. ;D
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on April 11, 2010, 10:46:23 AM
Thanks for letting me know, Arantor.  I sympathize.  There may be a PM from me sometime in your future.

Topic Stickied as requested. Good tip. ;D

Thanks, Chas; that was fast service.  Let's hope it helps some people.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Chas Large on April 11, 2010, 02:31:33 PM
I hope so too. You beat me to recommending this topic to another member who may well have suffered the same problem. If only my memory wasn't limited to 64K, I might remember a few more things :D
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on April 11, 2010, 09:59:47 PM
I hope so too. You beat me to recommending this topic to another member who may well have suffered the same problem. If only my memory wasn't limited to 64K, I might remember a few more things :D

Are you a Commodore 64?   ;D
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on April 12, 2010, 05:05:13 AM
Could be a ZX Spectrum; or indeed any machine based on a 16-bit address bus (i.e. M6502, M6800, Z80, 8080 type processor amongst many others)
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Chas Large on April 12, 2010, 06:32:28 AM
Now you're all taking the proverbial P*** :D

I'll have you know I started out with a ZX80. 4K memory (2KROM + 2KRAM). I wrote a word processing program for it. It displayed Word Proce.... and ran out of memory (LOL)

Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on April 12, 2010, 10:24:08 AM
Ah, the good old ZX80, with integer handling, a Z80 processor clocked at 3.57MHz IIRC, much improved when ZX81 came out :)
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on April 16, 2010, 10:46:09 AM
My first was a VIC-20.  Display was on a TV, and my black and white just wasn't cutting it, so my first color TV was bought just for the VIC.  I wrote some simple graphics programs in BASIC - simple (by today's standards) was all the machine was capable of - and typed in a lot of programs from magazines.  http://en.wikipedia.org/wiki/Type-in_program
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: ~DS~ on May 02, 2010, 06:26:11 PM
Ok, I haven't backup in a week so I am gonna backup today but I want to make sure the image shown:
http://www.simplemachines.org/community/index.php?topic=377117.msg2594267#msg2594267
is correct? SO that it doesn't not mess up the avatars and attachments.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Antechinus on May 02, 2010, 06:58:05 PM
Of course by far the fastest way to back up folders, if you have cPanel access, is to use the cPanel file manager. This is a lot faster than FTP and has no problems with any folders or their content.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: ~DS~ on May 03, 2010, 06:04:26 AM
Ok, the setting should be uncheck ONLY on attachments and avatar folder, is there other folders I should be worried about that need to be download as Binary?
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on May 03, 2010, 06:18:13 AM
It doesn't actually hurt to transfer everything as binary, really.

The whole concept of text transfer as text is an outdated concept that's really not an issue with modern editors; it's only for the olden days when editors couldn't handle Linux and Windows file endings (like Windows Notepad still can't, heh), if you're using a modern editor like Notepad++, it's a non issue.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: ~DS~ on May 03, 2010, 06:23:47 AM
It doesn't actually hurt to transfer everything as binary, really.

The whole concept of text transfer as text is an outdated concept that's really not an issue with modern editors; it's only for the olden days when editors couldn't handle Linux and Windows file endings (like Windows Notepad still can't, heh), if you're using a modern editor like Notepad++, it's a non issue.
I will try everything as binary the next time I back up. It's better than sorry not to try both methods. For now I am leaving it AUTO and check "Treat files without extension as ASCII file" and uncheck it for attachment and avatars only.

When you say binary...you mean not auto and and what about "Treat files without extension as ASCII file"?
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on May 03, 2010, 06:30:01 AM
Text mode == ASCII mode, they're one and the same.

When I'm using either FileZilla or WinSCP, I have it set to treat *everything* as binary.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on May 03, 2010, 06:22:16 PM
Ok, I haven't backup in a week so I am gonna backup today but I want to make sure the image shown:
http://www.simplemachines.org/community/index.php?topic=377117.msg2594267#msg2594267
is correct? SO that it doesn't not mess up the avatars and attachments.

No, that image in your other post is incorrect

This image from the first post in this thread is correct:

(https://www.simplemachines.org/community/proxy.php?request=http%3A%2F%2Fwww.simplemachines.org%2Fcommunity%2Findex.php%3Faction%3Ddlattach%3Btopic%3D370199.0%3Battach%3D137471%3Bimage&hash=1c3ca21e7420582031e29ac2122d9b48)

As Arantor says, however, you should be ok to download ASCII (plain text) files using binary mode.  Auto mode is just a list of extensions that specifies which files should be treated as ASCII but since the files we are talking about don't have extensions, it can't be used in this situation.

Basically, if you open a file in a text editor and it looks like non-alphabetic, non-numeric, non-standard-punctuation characters, it is a binary file (see attachment).

ASCII can be transferred as binary without corruption.
Binary files will be corrupted if they are not transferred as binary.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on May 03, 2010, 06:27:38 PM
Quote
ASCII can be transferred as binary without corruption.

Not entirely true, in fact, it depends what you define as 'corruption'.

If you mean 'without corruption' as the actual, byte for byte file it will be corrupted.

The file will be silently converted for line endings. For example, go grab the SMF install package and open any SMF file in Windows Notepad. Not Notepad++, not anything fancy, just normal Notepad. You get a mess where everything is on one line.

Now download the same file from FTP in ASCII mode and open it. Now it's not a mess. Why? Because FileZilla has silently converted the line endings on your behalf.

If you have ANY doubt, use binary mode. I expect the files I send and receive to be the actual files, not a modified-in-ANY-way version.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: ~DS~ on May 03, 2010, 06:39:07 PM
Ok uncheck the one from the image and set it to binary mode instead of auto. I open with textwrangler, seem fine. No lines messed up. 
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on May 03, 2010, 06:40:09 PM
That could be because TextWrangler can handle Windows, Linux and Mac line endings, which would hide if the file format has been changed on you without you being aware of it.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on May 03, 2010, 06:43:07 PM
Quote
ASCII can be transferred as binary without corruption.

Not entirely true, in fact, it depends what you define as 'corruption'.

If you mean 'without corruption' as the actual, byte for byte file it will be corrupted.

The file will be silently converted for line endings. For example, go grab the SMF install package and open any SMF file in Windows Notepad. Not Notepad++, not anything fancy, just normal Notepad. You get a mess where everything is on one line.

Well, yes, that is true in the sense it wouldn't pass a CRC (or some other kind of integrity) check, but it will be recoverable with one of the many line-ending converter programs, unlike a binary file like an image, which, once it's lost every 8th bit, it's gone for good.

Quote
Now download the same file from FTP in ASCII mode and open it. Now it's not a mess. Why? Because FileZilla has silently converted the line endings on your behalf.

If you have ANY doubt, use binary mode. I expect the files I send and receive to be the actual files, not a modified-in-ANY-way version.

But you said:
Text mode == ASCII mode, they're one and the same.

When I'm using either FileZilla or WinSCP, I have it set to treat *everything* as binary.

 :P  (I'm on FreeBSD - binary doesn't mess up my line endings.)

Ok uncheck the one from the image and set it to binary mode instead of auto. I open with textwrangler, seem fine. No lines messed up. 

Good.  Glad to hear it.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on May 03, 2010, 06:49:14 PM
/me thinks people don't understand what text mode is.

It DOESN'T eat every 8th bit.

What it does is convert line endings depending on what the server and client are using.

Windows uses \r\n (ASCII codes 13, 10, aka carriage return and newline) to mean end of line
Linux, *nix generally, including niche OSes derived from that heritage (including AmigaOS, randomly) uses \n
Mac historically used \r, don't know if it still does

Some images are fine, in fact, because they contain neither \r or \n, but it's incredibly rare for that to be the case.

Thus, text == ASCII mode, because it's not stripping bit 7 unless it's REALLY stupid, because it doesn't foul up extended bit characters, or UTF-8 characters for that matter (except line endings which are the same across UTF-8 due to UTF-8 reusing the same 0-127 sequence)


Binary wouldn't mess up your line endings. It would preserve it, that's kind of the point!! Text mode WILL screw your line endings unless you happen to be on the same OS as the server in which case I'd hope your client was smart enough to realise.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on May 05, 2010, 05:59:38 PM
/me doesn't want to argue with Arantor, so will just include a few links with accompanying text.

http://www.faqs.org/rfcs/rfc959.html
Quote
RFC 959
File Transfer Protocol
[snip]
3.4.3.  COMPRESSED MODE

         There are three kinds of information to be sent:  regular data,
         sent in a byte string; compressed data, consisting of
         replications or filler; and control information, sent in a
         two-byte escape sequence.  If n>0 bytes (up to 127) of regular
         data are sent, these n bytes are preceded by a byte with the
         left-most bit set to 0 and the right-most 7 bits containing the
         number n.

http://www.faqs.org/rfcs/rfc1635.html
Quote
RFC1635 - How to Use Anonymous FTP

- You may set BINARY mode to transfer executable programs or files of data. Type "binary" to do so. Usually FTP programs assume files use only 7 bits per byte, the norm for standard ASCII-encoded files. The BINARY command allows you to transfer files that use the full 8 bits per byte without error, but this may have implications on how the file is transferred to your local system.

http://courses.wccnet.edu/computer/mod/na36c.htm
Quote
FTP was developed at a time when typical modem speeds were 110 to 300 bits per second (as compared with 28,000 to 56,000 today). Since ASCII only used 7 bits, long files could be transmitted more quickly by not sending all the unused bits.

The big drawback is that if a file that uses all 8 bits in each byte is accidentally sent using ASCII transfer, it will lose 1/8 of its information content. In most files, even the loss of one bit is enough to make it invalid, and losing 1/8 makes them totally unreadable. So ASCII transfer can be fatal to a file's health!

With today's higher speeds, the time lost by sending all 8 bits of an ASCII file is practically unnoticeable. But FPT has incorporated features into ASCII transfer that make it useful for other reasons, so the two modes remain.

http://mc-computing.com/winexplorer/ftp.html
Quote
Binary vs ASCII
The internet was developed to transfer information in 7-bit packets.

Computers store data in 8-bit bytes.

Normal ASCII text data is stored using only 7 bits per byte (the 8th bit is always zero). (This kind of document is created using programs like notepad.exe) As a result, it is fairly easy to transfer this type of data over the internet. This is one reason the html pages are always written in ASCII.

Other types of data (images, programs, MS Word documents, and the like) use all 8 bits to store data. As a result, special algorithms are required to transfer 8-bit data over a 7-bit interface.

When you are using FTP, you need to be aware of the type of data being transferred - ASCII or binary (7-bit or 8-bit).
If you try to transfer 8-bit data in ASCII mode, data WILL be lost (the 8th bit will be set to zero).

http://www.ietf.org/mail-archive/web/ietf/current/msg40628.html
Quote
To: Sandeep Srivastava , ietf at ietf . org
Subject: Re: [RFC 959] FTP in ASCII mode
From: John C Klensin (http://en.wikipedia.org/wiki/John_Klensin)
Date: Tue, 21 Feb 2006 03:34:27 -0500
[snip]
> I knew that a FTP transfer in ASCII mode does EOL and EOF
> conversions based on the OS of the receiving system.

No, it doesn't.  That was part of the point.  It does no EOF
conversions at all.   The command and data channels were
separated for several reasons, but the desire to stay out of the
EOF business was an important one.  And the server is required
to convert whatever line-end convention it uses to CRLF, and any
characters it uses to ASCII, and transmit that over the wire.
If the client then converts from CRLF and ASCII to some local
convention, that is its business, not that of the protocol.  In
other words, there are, at most, conversions to and from CRLF
and ASCII. There are no FTP-specified conversions based on the
properties of the receiving system. 

/me thinks this esoteric discussion may confuse people who just want to make sure they don't lose their files.

Bottom line, or tl;dr: Transfer your avatars and attachments directory as binary files. 
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on May 05, 2010, 06:07:58 PM
Originally, yes that was the case. But 959 has been superceded multiple times since 1985, meaning that in the current specifications, it's not a strict 7-bit safe only environment.

If it were, every instance of SMF would be broken by FTP clients that treat PHP files as 'text' because there is a function in Subs.php that has 8-bit characters in it. As would any language file that isn't English.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on May 05, 2010, 08:30:54 PM
Originally, yes that was the case. But 959 has been superceded multiple times since 1985, meaning that in the current specifications, it's not a strict 7-bit safe only environment.

Do you mean to say that the only difference between an ASCII transfer and a binary transfer is what happens with line endings?  John Klensin says that any translation besides that to CR/LF for transfer (specified by network ASCII) is performed by the client for the local system, not by the server (if the server follows protocol).  So wouldn't that imply that between two identical systems, there would be no difference in line endings once the transfer is complete?  So if I transfer a binary file between 2 FreeBSD systems in ASCII, the LFs will be converted to CR/LFs for transfer, then the client will convert them back to LFs, so the file would essentially be the same, wouldn't it?  So a binary file (say, like an avatar) transferred as ASCII between identical systems will not be corrupted?  (No, I haven't tested this yet, I'll have to do that later sometime.)

Arantor,  I am the kind of person who likes to understand things, not just take someone else's word for them (not at all implying, though, that your word isn't good), and there is no question that you are far smarter than I am or ever will be, so how about sharing the source of these new (to me) revelations? 

I have found RFC 2640 (http://tools.ietf.org/html/rfc2640), which is titled "Internationalization of the File Transfer Protocol," and says this:
Quote
Abstract

   The File Transfer Protocol, as defined in RFC 959 [RFC959] and RFC
   1123 Section 4 [RFC1123], is one of the oldest and widely used
   protocols on the Internet. The protocol's primary character set, 7
   bit ASCII, has served the protocol well through the early growth
   years of the Internet. However, as the Internet becomes more global,
   there is a need to support character sets beyond 7 bit ASCII.

   This document addresses the internationalization (I18n) of FTP, which
   includes supporting the multiple character sets and languages found
   throughout the Internet community.  This is achieved by extending the
   FTP specification and giving recommendations for proper
   internationalization support.

but it also says this:
Quote
1 Introduction


   As the Internet grows throughout the world the requirement to support
   character sets outside of the ASCII [ASCII] / Latin-1 [ISO-8859]
   character set becomes ever more urgent.  For FTP, because of the
   large installed base, it is paramount that this is done without
   breaking existing clients and servers. This document addresses this
   need. In doing so it defines a solution which will still allow the
   installed base to interoperate with new clients and servers.

   This document enhances the capabilities of the File Transfer Protocol
   by removing the 7-bit restrictions on pathnames used in client
   commands and server responses, RECOMMENDs the use of a Universal
   Character Set (UCS) ISO/IEC 10646 [ISO-10646], RECOMMENDs a UCS
   transformation format (UTF) UTF-8 [UTF-8], and defines a new command
   for language negotiation.

   The recommendations made in this document are consistent with the
   recommendations expressed by the IETF policy related to character
   sets and languages as defined in RFC 2277 [RFC2277].

and this:

Quote
2 Internationalization


   The File Transfer Protocol was developed when the predominate
   character sets were 7 bit ASCII and 8 bit EBCDIC. Today these
   character sets cannot support the wide range of characters needed by
   multinational systems. Given that there are a number of character
   sets in current use that provide more characters than 7-bit ASCII, it
   makes sense to decide on a convenient way to represent the union of
   those possibilities. To work globally either requires support of a
   number of character sets and to be able to convert between them, or
   the use of a single preferred character set. To assure global
   interoperability this document RECOMMENDS the latter approach and
   defines a single character set, in addition to NVT ASCII and EBCDIC,
   which is understandable by all systems. For FTP this character set
   SHALL be ISO/IEC 10646:1993.  For support of global compatibility it
   is STRONGLY RECOMMENDED that clients and servers use UTF-8 encoding
   when exchanging pathnames.  Clients and servers are, however, under
   no obligation to perform any conversion on the contents of a file for
   operations such as STOR or RETR.


   The character set used to store files SHALL remain a local decision
   and MAY depend on the capability of local operating systems.
Prior to
   the exchange of pathnames they SHOULD be converted into a ISO/IEC
   10646 format and UTF-8 encoded. This approach, while allowing
   international exchange of pathnames, will still allow backward
   compatibility with older systems because the code set positions for
   ASCII characters are identical to the one byte sequence in UTF-8.

This seems to mean it is only talking about pathnames changing, not the ASCII spec.  But are the local systems changing the way things are done depending on their localization?  They know that some characters in that localization require all 8 bits?  Or the local system determines that a file has non-ASCII characters, and adapts accordingly?

I have found RFC 3659 (http://tools.ietf.org/html/rfc3659), which updates (but does not obsolete) 959, and it says this:

Quote
   This document also uses notation defined in STD 9, RFC 959 [3].  In
   particular, the terms "reply", "user", "NVFS" (Network Virtual File
   System), "file", "pathname", "FTP commands", "DTP" (data transfer
   process), "user-FTP process", "user-PI" (user protocol interpreter),
   "user-DTP", "server-FTP process", "server-PI", "server-DTP", "mode",
   "type", "NVT" (Network Virtual Terminal), "control connection", "data
   connection", and "ASCII", are all used here as defined there.

which is defined there as:
Quote
ASCII

         The ASCII character set is as defined in the ARPA-Internet
         Protocol Handbook.  In FTP, ASCII characters are defined to be
         the lower half of an eight-bit code set (i.e., the most
         significant bit is zero).

This  is knowledge-seeking on my part, not a "my * is bigger than your *" challenge to you.

If it were, every instance of SMF would be broken by FTP clients that treat PHP files as 'text' because there is a function in Subs.php that has 8-bit characters in it. As would any language file that isn't English.

Then if the 8th bit in ASCII transfers can now be 0 or 1, how come the images in the Avatars and Attachments directory get broken?

Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on May 05, 2010, 08:37:13 PM
OK, hereafter I'm going to throw out the rulebook and the specs and go purely on what I have seen and observed because when it comes down to it, quoting spec after spec doesn't solve the issue - because it's how things implement the spec that matters.

(Heck, if everyone observed the spec right, how come IE6 is such a pile of ****?)

Anyway.

There are two VERY DIFFERENT issues here.

One is 8-bit sanctity, one is line ending sanctity.

The former is essentially a non issue for modern FTP servers and clients because funnily enough the spec is ignored, and they send 8 bit files as is. In today's UTF-8 world, it's pretty much a necessity to send 8-bit files, because UTF-8 is a full 8-bit text format.

The latter is still very much an issue because line ending conversions get done. THIS is what breaks attachments, not loss of the 8th bit.

How do I know this? I've seen attachments broken by this, specifically I've observed the differences in the files, and viewed them in more tolerant viewers. The files get broken part way through, not totally. If 8th bit loss were the case, on average 50% of the file would be damaged since the odds of any one bit being set are non exclusive 50%. But that's not the case.

From experience, FTP between two Linux servers hasn't damaged such files. That isn't to say it wouldn't but in my experience that was the case.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on May 07, 2010, 02:32:45 PM
OK, hereafter I'm going to throw out the rulebook and the specs and go purely on what I have seen and observed because when it comes down to it, quoting spec after spec doesn't solve the issue - because it's how things implement the spec that matters.

Ok, fair enough.

Quote
(Heck, if everyone observed the spec right, how come IE6 is such a pile of ****?)

Microsoft wanted to write their own spec for the internet - they thought they were an exception to the existing specs, that they were powerful enough to write their own, and that everyone else would adjust to it because they were Microsoft. 

This was obvious in their early implementations of FrontPage - I was working tech support at a web hosting company at the time, and we had many, many customers call to find out why their pages written with FrontPage which displayed perfectly in IE (3? 4?) would not display in Netscape.  Try telling a bunch of customers who thought Microsoft invented the internet that FrontPage and IE didn't follow the already established standards!   ::)

Microsoft has since realized that they need to work with the internet technical community, instead of trying to take it over.   :)

You might find this thread on the ASCII issue interesting (that is, unless you're bored with the whole thing by now): http://www.securityfocus.com/archive/1/437948/100/0/threaded

Quote
Anyway.

There are two VERY DIFFERENT issues here.

One is 8-bit sanctity, one is line ending sanctity.

The former is essentially a non issue for modern FTP servers and clients because funnily enough the spec is ignored, and they send 8 bit files as is. In today's UTF-8 world, it's pretty much a necessity to send 8-bit files, because UTF-8 is a full 8-bit text format.

The latter is still very much an issue because line ending conversions get done. THIS is what breaks attachments, not loss of the 8th bit.

How do I know this? I've seen attachments broken by this, specifically I've observed the differences in the files, and viewed them in more tolerant viewers. The files get broken part way through, not totally. If 8th bit loss were the case, on average 50% of the file would be damaged since the odds of any one bit being set are non exclusive 50%. But that's not the case.

From experience, FTP between two Linux servers hasn't damaged such files. That isn't to say it wouldn't but in my experience that was the case.

Ok, fwiw, I tested 2 files on my BSD machines.  I did this with both the command line FTP client and with Filezilla, with the same result.  One was a plain text file, one was a png file.  I downloaded both using both ASCII and binary.  The ASCII downloaded png file wouldn't open in GIMP; GIMP suggested that the file might be corrupt.  I then opened both the ASCII-downloaded and the binary-downloaded png files with Diffuse, a file comparison editor (similar to Diff, but graphical), and the only differences in the two files that I could see were some blank lines in one but not the other.  There were about 8-10 instances of this in the whole file.  With the plain text file, there were no differences apparent in Diffuse.

This suggests you're right about the line ending translation being what messes up the binary files instead of the 8th bit, since every other character was the same, and none of the 8-bit characters had been converted to standard ASCII characters.  If the 8th bit had been changed to 0 on all the bytes, I would have expected to see only alpha-numeric and punctuation characters in the ASCII-downloaded binary file.

Anyway, thanks for the stimulating discussion, and for bringing my FTP/ASCII knowledge up to date.  I never mind having what I think I know challenged if it might be wrong. 
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on May 07, 2010, 06:47:36 PM
*nods* Experience definitely wins - and it's great to get more experience on the matter, and good to have references to things to read up on.

That's the thing - you appreciate the journey as well as the destination itself, and it's been a pleasure discussing this :)
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: jimbouk1977 on May 20, 2010, 03:51:03 PM
LOL so on or off, auto or binary that's all we needed to know ! :P
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on May 20, 2010, 03:53:39 PM
If in doubt, binary.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: bahgheera on May 31, 2010, 03:17:34 AM
Here's a question: in the original post, it says the option should be UNCHECKED in the image. Then directly after, there are some instructions that state the option should be CHECKED. So which is it, checked or unchecked??

EDIT: Doh... I'm an idiot. Guess I couldn't make my brain go past what I thought was a confusing error.  ::)
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Forum Guy on May 31, 2010, 03:52:35 AM
nice glitch, really!

that check box could be completely removed as it should transfer everything in BINARY until known extensions transfer in ASCII.

Consequentially this is a complete mess and got it all wrong  :o


Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: knightofdoom on June 10, 2010, 05:44:33 AM
thanks for sharing
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: lastminute on September 03, 2010, 06:28:03 AM
I use Filezilla
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: autoreg on October 12, 2010, 07:30:06 AM
Thanks for this interesting information. Really helpful
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Kimmel on October 16, 2010, 08:15:33 AM
nice glitch, really!

that check box could be completely removed as it should transfer everything in BINARY until known extensions transfer in ASCII.

Consequentially this is a complete mess and got it all wrong  :o

So, it is enough to transfer all files in Binary and it will be alright?
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Red G. Brown on November 09, 2010, 01:09:08 AM
This information should be on the forum backup page in the forum admin. This problem could be completely resolved if the forum backup function really did backup the forum, including attachments and avatars. Give it all to me, in one nice tar.gz. Simple, no caveats. The tricks only come when you have to extract the file and decide where to put the contents. That's easier to do than discovering you didn't actually have a correct and complete backup in the first place!
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Aleksi "Lex" Kilpinen on November 09, 2010, 01:11:14 AM
This information should be on the forum backup page in the forum admin.
Well, perhaps - but then again, this post is to support users of one specific FTP client, and it really isn't SMFs place to support 3rd party software IMO. Something general to remind about FTP backups, and how to transfer files could be done of course.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Red G. Brown on November 09, 2010, 01:23:40 AM
Right, I've complained about this before when I was surprised to find out the backup function didn't really do a backup. It just gave me bits and pieces, and I had to go hunt down the other bits and pieces separately before I had a full backup. That would be easy to fix, and would probably save a lot of forums.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Aleksi "Lex" Kilpinen on November 09, 2010, 01:26:05 AM
Well, the point of the admin panel backup is, that it is a Database backup. And all you need to do to save an installation, with all of it's actual contents is the database backup. Agreed, that won't help you save attachments, avatars, themes etc, but for real emergency situations the DB is the biggest worry for most admins.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: crotalusco on November 23, 2010, 01:36:30 AM
I made this mistake a year ago and am just finding this thread. Does anyone know if i re-upload all the attachment files under the new filezilla settings. replacing existing files if this will correct the images?
Title: Re:I have also lost my attachments and Profile Pictures..HELP Please
Post by: kamkar on November 23, 2010, 07:38:03 AM
I have shifted my forum from one host to another. [www.merapahadforum.com]

While I got all the post back but lost all my attacments. Yes I did use Filezilla.

After reading this thread I have unchecked the Filezilla setting as mentioned and downloaded all the attachements in my windows machine. I have uploaded all the attachments on my new host (deleting the old one) but I could not get them back.

Then I went to old host and create the tar backup using

tar -cvf att.tar /home/username/public_html/attachments

downloaded it using sftp and uploaded to new host. The result remained same. I have also done chmod 777 for attachment directory.

Am i missing something, somewhere.

I am using RC4
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: kamkar on November 23, 2010, 07:55:58 AM
I could sort it out. It was a file path related problem
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Alison Black on December 22, 2010, 06:18:01 PM
Is filezilla truly a good ftp program to use, as I read somewhere that it can sometimes leave out some type of file, like pictures or something. I cannot remember the actual topic completely but it was something about images?
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on January 07, 2011, 05:53:49 PM
I made this mistake a year ago and am just finding this thread. Does anyone know if i re-upload all the attachment files under the new filezilla settings. replacing existing files if this will correct the images?

If the files have been corrupted by an ASCII download, that is, if data has been stripped from them, it appears that there is no getting them back.  Uploading them with the correct settings will not restore the missing data.

If there is a piece of software which can restore the files that someone knows of, it would probably do a lot of people a big favor to post the name of it.

Is filezilla truly a good ftp program to use, as I read somewhere that it can sometimes leave out some type of file, like pictures or something. I cannot remember the actual topic completely but it was something about images?

Alison, go back and read this thread for the details.

I like Filezilla quite a bit as an ftp program, but it has to be configured correctly.  But then, all software must be configured correctly to work as one intends it to, unless it's a "one trick pony" kind of software.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on January 07, 2011, 07:18:30 PM
Quote
If there is a piece of software which can restore the files that someone knows of, it would probably do a lot of people a big favor to post the name of it.

There isn't, because it has no way of knowing the content of the original file to be restored. If there were some signature of the original file (md5 of the file contents), there's a chance but if you realise that for every instance of \r or \n in the file - and that's likely to be a lot in images, actually, there are 2 combinations, and those 2 combinations are exponential - so if there was a file with 10 original entries of either \r or \n, each of those is now an entry of \r\n, and you need to go through every combination, which is 2^10 = just 10 original values is 1024 combinations. 1024 combinations to try, calculate new signature, it's computationally expensive.

And there are usually a LOT more than 10 instances in a large file... often thousands. 2^1000 is a very large number.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on January 08, 2011, 11:48:39 AM
Theoretically, it could be done, like the NSA might do with Osama Bin Laden's encrypted files if they were to find them, but practically speaking, for the little guys (us), it's foolish to think about.  (Just an example, not meaning to start a political discussion.)  Your web host's backup, if there is one, is the best possibility.

/me laughs at the whimisical idea of someone starting a distributed computing project, like Folding or SETI, to get their forum attachments and avatars back.

Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Red G. Brown on January 09, 2011, 10:25:07 PM
I still like the idea of SMF just giving people a real backup when it says it's giving people a backup in the administration panel. That ought to be plan A, if plan B is using the web host's backup, and plan C is becoming a terrorist leader so the NSA will repair your files for you.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on January 10, 2011, 01:59:32 AM
Which would lead to even more support issues as enough people have trouble with backups timing out as it is, and most server configurations would not support sending every attachment through the admin panel without hitting either the PHP or Apache timeouts. (cPanel runs its own web server and so has no such direct limitation)
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Red G. Brown on January 10, 2011, 02:09:12 AM
Which is what volume splitting is for, so you have smaller files that are more manageable to download without timeouts.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on January 10, 2011, 02:15:17 AM
Sure, but either you prepare the volumes up front (which means doubling the data stored on disk while you download it), or you prepare it on the fly, which can lead to memory timeouts as people already experience when gzipping their content.

Ideally, SMF needs to have the entire backup code ripped out and rewritten from scratch to support this kind of thing though that's clearly not going to happen prior to 2.1 at the very earliest.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Red G. Brown on January 10, 2011, 02:23:19 AM
I think it would be really cool if SMF could figure out a way to "chunk" it's database, so older bits of it get stored separately from newer bits. That way, you wouldn't have to backup the entire thing every time, just the parts that have changed. Of course, that leads into the idea of diffs and deltas, which may not be used directly, but could provide some models for simplified techniques to achieve something similar.

On low to moderate traffic forums, it becomes feasible to email a backup every day automatically. Wouldn't that be cool?
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on January 10, 2011, 02:32:55 AM
I'd hate to get that by email every day when there are much more efficient ways of doing it, like automating a scheduled task on my PC... like I already do now, but that's just me.

The problem with deltas is that to restore the database you have to rebuild it incrementally rather than a single restore motion, and I'd personally hesitate to build such an option for users that will inevitably get it wrong, which is why I abandoned development of so many mods over the time I was modding, knowing the support issues, so ultimately I'd suggest SMF not take that course as a standard feature personally.

It's not impossible to chunk, of course, just the restoration is sufficiently complicated it would confuse most users here, I think, knowing the number of times I got fed up with people who didn't even read what was put in front of them multiple times in large red letters. That said, it gives me an idea for the other thing I'm working on...
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: johny28 on January 11, 2011, 10:26:01 PM
i using filezilla,,,its work.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Omebolle on January 13, 2011, 07:08:12 AM
Good tip about Filezilla, the settings were indeed wrong. Could that be why some of my members can't access the forum anymore? I just upgraded to RC4 and some members say they get these messages
- unable to verify url, please go back and try again
- Maintanance Mode and we're attempting to restore an older back up.

 The problem doesn't appear on a Mac. One member has a PC and can't access with that, but the Mac doesn't give problems..

Any tips?
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on January 13, 2011, 07:15:39 PM
Could that be why some of my members can't access the forum anymore?  I just upgraded to RC4 and some members say they get these messages
- unable to verify url, please go back and try again
- Maintanance Mode and we're attempting to restore an older back up.

 The problem doesn't appear on a Mac. One member has a PC and can't access with that, but the Mac doesn't give problems..

Any tips?

I don't think so (I could be wrong), but you might want to search this forum for the error messages, and if you don't find anything that fixes it, start a new topic.   ;)

Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: monolower on January 16, 2011, 09:39:34 PM
thanks for info 8)
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: headstone on March 12, 2011, 09:43:31 AM
Just a thought... Wouldn't it be far simpler to give these files an extension that can then be removed by the forum code upon detection?

Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Arantor on March 12, 2011, 12:09:54 PM
Just a thought... Wouldn't it be far simpler to give these files an extension that can then be removed by the forum code upon detection?



Funny you should say that. The reason the files are stored without extension is to prevent the files being abused; there was a vulnerability a bit back that allowed files with extensions to be triggered by the server itself, which lead to a lot of forums being hacked.

Honestly, though, it's a bit of a poor showing that FileZilla assumes anything without an extension is 'likely a text file' by default... which is where the problem comes in.

Yes, in theory, it can be modified to have a generic extension that won't normally be executable - but it's a change that you won't see in SMF, certainly not before 2.1 at the earliest (if at all)
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on March 16, 2011, 06:34:17 AM
Honestly, though, it's a bit of a poor showing that FileZilla assumes anything without an extension is 'likely a text file' by default... which is where the problem comes in.

Yes, I agree.  And if I remember correctly, this was an unannounced change in an upgrade from its previous default behavior where it assumed it was not a text file, which further compounded the problem for people who had the previous version installed.  Upgrades shouldn't change user settings without some sort of notice to the user.  (imo)
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Ljungberg on March 17, 2011, 12:58:10 AM
Text mode == ASCII mode, they're one and the same.

When I'm using either FileZilla or WinSCP, I have it set to treat *everything* as binary.
including .htaccess?
Can I uncheck both "Treat files without extension as ASCII file" and "Treat dotfiles as ASCII file"
and then upload everything as binary?

thanks
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: ryry46d9 on May 09, 2011, 04:02:20 AM
OMFG ...
This should be it the admin panel.
I have moved my site 3 times and wondered why 50% of the attachments never made it.

now my question that I didn't see a answer on is, why don't all said images have extensions? this would make life so much easier  :P


but now I know what I been doing wrong, ThankYou for the heads up  ;D
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Aleksi "Lex" Kilpinen on May 09, 2011, 06:56:33 AM
The attachments don't have file extensions to make it harder to upload an executable file like a shell script and use it to take over the server ;)
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Red G. Brown on May 09, 2011, 01:53:42 PM
OMFG ...
This should be it the admin panel.
I have moved my site 3 times and wondered why 50% of the attachments never made it.

Yes.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: susanexpress01 on May 23, 2011, 03:01:31 AM
Thanks.My brother is searching for this board.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: ~DS~ on June 12, 2011, 11:32:16 AM
Text mode == ASCII mode, they're one and the same.

When I'm using either FileZilla or WinSCP, I have it set to treat *everything* as binary.
including .htaccess?
Can I uncheck both "Treat files without extension as ASCII file" and "Treat dotfiles as ASCII file"
and then upload everything as binary?

thanks
Yes.
Both for upload and download...everything binary.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Aleksi "Lex" Kilpinen on July 10, 2011, 01:42:51 AM
Most of the time binary is quite safe to use on everything, although I do not know if htaccess and other text only files like robots.txt would ever have any real benefit from being treated as binary... The most important thing is to make sure that Filezilla uses binary as default for files without extension, and for files with extensions it doesn't recognise, as this would make sure your image attachments are treated as binary like they should be.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: StuartMorris on July 11, 2011, 04:02:22 AM
This is Stuart from Minneapolis (America's best city). I came across this thread and found it interesting. Hopefully, we can get new ideas from each other as we all are here for sharing our knowledge and experience.  :)
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: TheChief60 on July 28, 2011, 12:08:24 AM
I just transferred my forum and used FileZilla....

Couldn't see attachments and avatars....

Thanks to everyone as yes the ascii transfer in auto mode was the culprit. Re-transferred everything without the treat files w/o extensions as ascii...and set it to transfer everything as binary and it worked.

This should be a required part of any move SMF to new host documentation. Again, thanks...
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Red G. Brown on August 01, 2011, 12:53:39 AM
I'm glad we made a big enough deal about this that you were able to find it and make use of the information here.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: outlawsys on October 15, 2011, 01:40:41 AM
Why would they do that? I mean do you know any extensionless files that AREN'T binary files? Other way around yeah maybe, .htaccess no name, just file with extension its ASCII... but FILE.   ect.... chances are its binary, default to that unless told otherwise....geez... you would think OS guys would know better.... guess not, they must work for Apple   lol     :(
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Bet99ty00 on November 03, 2011, 10:24:51 PM
I had this issue previously. Since then I had been really careful about it. Thanks for the heads up once again. I am sure this will help alot of users around.


EDIT by (F.L.A.M.E.R):
Links/Images for advertisement removed.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: live627 on November 04, 2011, 02:36:55 AM
Why would they do that? I mean do you know any extensionless files that AREN'T binary files? Other way around yeah maybe, .htaccess no name, just file with extension its ASCII... but FILE.   ect.... chances are its binary, default to that unless told otherwise....geez... you would think OS guys would know better.... guess not, they must work for Apple   lol     :(
Complain to FileZilla, not Apple. Who- who am I kidding? That was funny!
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: KeeKira on January 08, 2012, 09:25:22 PM
you all are bringing back memories.  I still have a working Commodore 64  :)

Thanks for posting this.. It is helpful for those of us who would otherwise not know such things and be banging head on table.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Chalky on April 22, 2012, 09:53:31 AM
I'm confused!  I get the bit about checking the ASCII box to transfer all files and folders except Attachments and Avatars, and then unchecking the box to transfer those folders separately.... but then you all start going on about binary!  I've left the Auto option checked throughout the whole backup as per the picture in the first post of the thread - should I have changed this to Binary as well?

Does anyone know if this is also an issue when backing up Wordpress files?
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: wardrobechoice on May 16, 2012, 12:56:36 AM
Thank you for your post!This can help a lot of people who doesn't know what to do on such situation.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: live627 on May 16, 2012, 03:35:49 AM
I'm confused!  I get the bit about checking the ASCII box to transfer all files and folders except Attachments and Avatars, and then unchecking the box to transfer those folders separately.... but then you all start going on about binary!  I've left the Auto option checked throughout the whole backup as per the picture in the first post of the thread - should I have changed this to Binary as well?

Does anyone know if this is also an issue when backing up Wordpress files?
set everything to binary and you'll be fine. ASCII changes the line endings, and binary does not.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: jonyparks on May 16, 2012, 12:33:47 PM
Ya i use filezilla.For transferring a site in Live i think it no 1
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: gerzok on July 27, 2012, 07:38:39 PM
Wooooow thanks for share this info. Now i know why happens this :/
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: lindaf on July 23, 2013, 07:30:07 AM
I am very grateful this post was stickied!  I would have had no clue about any of this!  Thanks many times over!! :)

My question is same as above.   Do I also uncheck it for avatars folder?  You mention to uncheck it for attachments directory but didn't mention avatars folder.  I'm just assuming to also uncheck it for avatars folder since avatars are the topic subject.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: AncientDragonfly on August 04, 2013, 08:42:05 PM
You can just transfer everything as binary and it will be ok, as Arantor says here:
http://www.simplemachines.org/community/index.php?topic=374178.msg2610981#msg2610981

Or use your host's cpanel filemanager to make a zip file of everything, then download the zip.  If you do this, make sure to read the popup window that shows what has been zipped because any error messages will show up in here.  (For instance, if you don't have enough space for the zip file in your web space, it will fail.)
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: SatTrkr on July 12, 2014, 07:17:24 PM
Thank you all for this thread as I had the very same problem. Godaddy was able to restore my forum directory from a backup and I have now moved it to my new domain name correctly and all now works okay. I thought, for a moment, I had lost all of my attachments and avatars but this thread alerted me to the problem and I was able to recover. That was a close call!
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Colin on July 12, 2014, 07:43:11 PM
Glad to hear all is working now, SatTrkr. Let us know if you have any other issues.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: GRoberts on February 13, 2015, 09:52:32 AM
Thanks for the fix. I was worried that I'd lost all my photos and avatars after I moved hosts.
I just reuploaded my 'attachments' folder contents in 'binary' rather that 'ACSII'

FYI - my filezilla options was under 'Transfer > Transfer Type > select Binary'.
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: suhaneankit on March 21, 2015, 12:08:50 PM
I am using FileZilla 3.10.2 version the option 'Treat Files Without Extension' is checked by default. Do i need to uncheck it ?
I don't thinks so because i have been using same settings since long without any problem in my SMF forum like Avatar, Images etc you mentioned.
Correct me if i am wrong.
Hoping for a reply. Thanks :)
Title: Re: Important Notice: Do you use Filezilla? Avatars and Attachments lost?
Post by: Kindred on March 21, 2015, 01:17:52 PM
you are wrong.