News:

Wondering if this will always be free?  See why free is better.

Main Menu

index.english-utf8.php is missing

Started by frakme, March 23, 2012, 06:27:24 PM

Previous topic - Next topic

frakme

We recently converted our forum from 1.14 to 2.02. We followed the usual steps and did so without any noticeable issue, except for a bug in the subs.php menu referenced HERE. Mods installed are SMF Gallery Lite and Simple Portal.

A few days ago, members started reporting problems with characters in new posts. When members have used other languages (likely copied directly from a translator)  Å shows instead of some of the typed characters. In older posts, everything from the first "foreign" character on will have disappeared. I checked the database and that information is still "there" it just is not showing up on the smf side of things.

The admin panel on the smf side of the forum shows that ISO-8859-1/English is the language installed. I did convert HTML-entities to UTF-8 characters through the SMF control panel.  The file index.english-utf8.php is missing from the files on the host. I thought it was there when we ran 1.14. The other account manager on this forum/host says he did not remove the file when he upgraded to 2.02. but said when he upgraded,  index.english.php replaced the index.english-utf8.php in the themes/default/languages file.

So my question is

1) is this just a language problem  that can be fixed by finding and adding the english-utf8.php file?
2) if not, what do I need to do from here?

kat

Finding the language files, here, is a bit of a nightmare, I'm afraid. They're in a really illogical place, for some reason.

English UTF-8 is here:

http://download.simplemachines.org/?smflanguages;lang=english

PROPER English UTF-8 ;) is here:

http://download.simplemachines.org/?smflanguages;lang=english_british

frakme

#2
Thank you so much for the quick reply. I have been looking for that languages page for days! :)

I added the utf-8 files in the themes/default/languages file, the forum is set to english/utf8 but foreign characters are still not showing. This is where I lack knowledge because I thought the purpose of utf8 was so that one didn't have to add each individual language fie. It is impossible for me to know which language someone might use as the forum itself is in english but many people use various languages as the traditional language of characters in certain parts of the stories they are telling.

So, what should I try next?

frakme

Just an update because I failed to mention this in the earlier post.
1)Database through phpmyadmin shows that all tables are collated with utf8 and character sets are utf8 as well.
2) HTML convert to utf8 was done and shows completed
3) Settings php file on the server also lists $language = 'english-utf8'; in Forum Information and $db_character_set = 'utf8'; in Database Information
4)All foreign characters are still displaying incorrectly once posted.  For example bohové becomes bůh sakra BUT in the modify post screen they characters appear as they should. For example bůh sakra becomes bohové.

Any help on where to go from here is appreciated.

Kermit

Are you sure,that all english-UTF-8 language files have been replaced properly ? I don't have much idea about this UTF-8 thing,but the problem is mostly,that languages files are not completely in UTF-8 format
My Mods
Please don't PM/mail me for support,unless i invite you
Formerly known as Duncan85
Quote
"Two things are infinite: the universe and human stupidity; and I'm not sure about the the universe."

A. Einstein

kat

#5
Quote from: frakme on March 23, 2012, 07:19:49 PMforeign characters are still not showing.

Thinking logically, the foreign characters won't show, because, by definition, they're not... er... English! ;)

So, one would surmise that you need the language packs for those other languages, no?

Those well-hidden language packs are here:

http://download.simplemachines.org/?smflanguages

Edit: Thinking further on it, and after a nudge from Kermit (Ta, Kermit) ;) what actual characters do you mean, exactly?

I was thinking of umlauts and that kinda thing, which we English just don't have. But, of course, we have words like "Café" that have accents, although most of us are too lazy to use 'em. ;)

frakme

QuoteThinking logically, the foreign characters won't show, because, by definition, they're not... er... English!

To respond logically, they showed without issue until the upgrade to smf 2.02. We only had english UTF8 installed, both on the SMF control panel and on the host in our php files, when it was using the 1.14 version. This is the reason for my initial post. Even though the forum itself is in English,  we have members that often insert character language into their story posts, so we have always used UTF8 instead of basic english.

QuoteAre you sure,that all english-UTF-8 language files have been replaced properly ? I don't have much idea about this UTF-8 thing,but the problem is mostly,that languages files are not completely in UTF-8 format

If by properly, you mean the suddenly missing utf8 files are now placed in the appropriate folders on the host? Then yes. They are located in the themes/default/languages file. If you mean something else, then I won't know unless you explain step by step.

QuoteEdit: Thinking further on it, and after a nudge from Kermit (Ta, Kermit)  what actual characters do you mean, exactly?

Yes, that is correct. Any foreign character with any kind of non typical character is appearing as (I think) something that looks like latin.  Někdy jsou lidé kreténi. Život někdy je sračky. becomes Někdy jsou lidé kreténi. Život někdy je sračky. Again, please note that the correct form is shown in the  post or modify post box but  if you hit the post button/preview button the character issue appears.

The above is Czech and upon the above advice, I installed the czech utf8 files on the server, in the appropriate file just to check. Nothing has changed.

kat

Just a hunch, this.

Try going to Admin>Languages>Add Language and search.

frakme

QuoteTry going to Admin>Languages>Add Language and search.

When I uploaded the language files to the server, host-side, the SMF admin language panel automatically added the czech language options, both standard and UTF8. However, it did not fix the character issue within the posts.


The only thing it does (which is what it is meant to do) is change the entire forum to Czech when I select that radio button. The characters on the rest of the forum do appear correct. For example on the admin menu, any forum menu or button a č remains a č.  However, inside the posts the errors still occur. Latin-esque characters replace the Czech characters (or any other language except standard English).


Edit: I am working off the default smf theme to ensure this isn't just a language issue specific to our custom themes. I contacted the custom theme creator who verified his theme is supports multiple languages.

frakme

#9
Scratch what I said about the czech utf8 files doing anything.

The individual language utf8 files being installed do not actually have anything to do with the other parts of the forum to work correctly. I just tested by adding menu buttons via the subs.php file as well as using foreign language in other parts of the global template and those ares show the characters correctly. I tested in german, hebrew, arabic, greek, romanian, latvian, urdu and turkish and I do not have those individual utf8 language files installed anywhere in my server side files or on the SMF language panel.  The only issue that remains is in the post and preview areas. Those two areas display the characters incorrectly. How is that possible that the characters work in some areas of the site but not others? Does that help clarify the database rather than any smf php files is the culprit?

Again, any help is appreciated. This is a creative arts/hobby forum and without the ability to post their work and writings, it really doesn't serve its purpose.

kat

This is getting a bit over my head, now.

What I'll do, is give the localisation team a nudge, to see if they have any ideas. :)

kat

I've had a suggestion, from someone who knows WAY more than I do, that you need to convert your database to UTF-8.

This, apparently, is a bit out of date.

http://wiki.simplemachines.org/smf/UTF-8_Readme

If it is, maybe you can ask your host to do it?

kat

By the way, in case you're not a GOOD admin, read my sig, before you attempt this, woncha? ;)

frakme

QuoteI've had a suggestion, from someone who knows WAY more than I do, that you need to convert your database to UTF-8.

The database is already in UTF-8. I've attached the screen shots of the phpadmin screen. Did I miss something?

Thanks so much for looking into this. I'm happy to PM anyone you think I should talk to or happy to have anyone PM me.

Dzonny

Hello there.

I would firstly suggest to reupload english and other custom language packs manually, as you were missing some files there may be some more problems with it packs you have there. That's 2.0.2 language packs i believe?

You have something in your error log?

When you open *new* posts with those special characters in phpMyAdmin, can you see them properly?

frakme

#15
QuoteI would firstly suggest to reupload english and other custom language packs manually, as you were missing some files there may be some more problems with it packs you have there. That's 2.0.2 language packs i believe?

Okay, I re-uploaded the utf8 language packs today. I had already done that when advised to do so in the above posts, but I did it again tonight. There is no change on the forums. The type appears correctly when you type or paste it in the edit post screen. It appears correctly in the modify post screen. It appears with (what looks like to me) latin type characters once the preview and post buttons have been pressed. If you hit modify after posting, the type appears correctly in the edit box but is once again in strange characters in the thread after the the post button is selected.

QuoteYou have something in your error log?

There are two errors recorded over the course of the last 4 days. 

1."The attachments upload directory is not writable. Your attachment or avatar cannot be saved." (1)


2.  "Sorry Guest, you are banned from using this forum! (15)
Approved players only
This ban is not set to expire."




QuoteWhen you open *new* posts with those special characters in phpMyAdmin, can you see them properly?
Yes, all foreign characters appear correctly in the phpmyadmin browse tabs.

Also wanted to add that, in case it wasn't clear above, special characters, foreign or otherwise, all appear normal in other sections of the live forum. Menu bars, topic bars, topic titles within the posts -all show normal characters. It is only the body of posts, simple portal pages and the shoutbox that are affected. Articles and news blocks of simple portal post fine with the characters displayed properly.


Dzonny

So, i think it's clear now that problem is not in language packs. It's in database probably.

Quote from: frakme on March 28, 2012, 12:18:38 AM
QuoteWhen you open *new* posts with those special characters in phpMyAdmin, can you see them properly?
Yes, all foreign characters appear correctly in the phpmyadmin browse tabs.

And when you open old posts, is there any broken characters, or messed up signs?

Your error log is not related to this problem, so it's okay.

frakme

Alright, so to "fix" the database, what is my next step?


QuoteAnd when you open old posts, is there any broken characters, or messed up signs?

Yes, the problem occurs in older posts too. The account executive who handled the forum before I took over to try and fix the issue, said he noticed the problem right after the original upgrade to 2.02. They do not appear correct in the database when I looked at them.

Dzonny

Can you please copy/paste some of the lines from the database (with "broken" characters) just to see them firstly?

frakme

Palom’s 
war…

Those are the common characters that appear in the old posts.


Advertisement: