SMF moves to a new translation platform: switching to Crowdin

Started by Dzonny, March 16, 2021, 04:01:20 AM

Previous topic - Next topic

m4z

Quote from: spiros on April 11, 2021, 03:19:31 PM
Not an issue of browser, tried different. And at first I did not have this issue. Maybe point me to a specific string URL that should be editable/translatable so that we are on the same page.

Since you probably want to translate Greek (and have the appropriate permissions in Crowdin), try this: https://crowdin.com/translate/simple-machines-21-beta/all/en-el?filter=basic&value=0 (or click https://dev.simplemachines.org/crowdin.php → select a language → click "Translate All")
(PS: What exactly did you try? I don't know where the "View strings" link you mentioned is located.)

Quote from: spiros on April 11, 2021, 03:19:31 PM
Well, I am a translator by profession and I also teach CAT tools. One can always leverage the translations from an earlier version :) Not sure if it has been done and to what extend.

I had to google what CAT could be, never heard of that acronym or term (I assume you mean "Computer-assisted translation").
What I meant is: I don't know if the 2.0 strings were ever imported into the TM (and I don't think so). Dzonny might know.
"Faith is what you have in things that don't exist."
--Homer Simpson

Es gibt hier im Forum ein deutsches Support-Board!

spiros

Tried logging out and logging in again. It works if I start from home page.

Yes, all localization is carried out nowadays using said tools (whether desktop or online).
https://en.wikipedia.org/wiki/Computer-assisted_translation#Translation_memory_software

Then it would be a good idea to leverage those legacy translations for all languages (so as not to enter again existing translations).

Also, maybe enable MT suggestions in the system?

@rjen

Quote from: m4z on April 11, 2021, 02:56:44 PM
Quote from: spiros on April 11, 2021, 07:35:01 AM
Also, I see quite a lot of strings untranslated that have been translated in SFM 2. Is there a language-specific TM/glossary used?

2.0 isn't in Crowdin at all; so if the string wasn't in 2.1 in LE, it won't be in Crowdin, AFAIK.

I am also finding many strings in Crowdin supposedly untranslated that were already translated in LE brach 2.1 before. As a matter of fact the dutch translation was at 100% in LE and now is far from that: there is definately progress lost.

I have started updating some strings from the 2.1 language files I previously downloaded from LE, but I do not understand when the are actually adopted in Crowding: the process is not yet clear to me...
Running SMF 2.1 with latest TinyPortal at www.fjr-club.nl

Dzonny

Thanks for your comments guys ! :)

2.0 branch is not planned to be moved to Crowdin, but we did sync between 2.0 -> 2.1 branch in LE before moving 2.1 to Crowdin.

The whole moving process took some time, as you may know we announced moving for translators quite some time ago, and in the mean time there were additional changes in 2.1 branch, new strings etc. Crowdin is now synced with latest github files, so changes from GH will be added to Crowdin automatically.

During the whole moving process, we worked with Crowdin support team in order to adjust our current syntax and logic to their system, so we had to make some additional customization in our code and strings in order for the whole system to work.
We're sorry if there's some translations lost during this, as we had to manually transfer some files, and even manually c/p specific strings.
We already stated on Crowdin that, for example registration agreement must be re-translated for some languages, as we couldn't bulk upload .txt translations etc.

Anyway, if you spot major part of specific file in language that is not translated on Crowdin, and was translated in LE - let us know so we can check, and possibly transfer translations to Crowdin.

We're sure that this kind of things won't happen in the future, but radical changes like moving from outdated LE to new software like Crowdin, sadly caused some glitches, but we're also happy that all is fixed now :)

About Machine translation -  MT, we have that disabled for now. You may remember that we have no automated translation scripts policy which is still in effect. We'll surely consider your suggestion though, as translation scripts improved significantly over the last years.

However, there's a Translation Memory enabled, and you can see more about it here:
https://support.crowdin.com/translation-memory/

Let us know if you need any help with the error you got earlier, or that is sorted now? :)



spiros

Machine-aided human translation is different than using MT in an automated way ("automated translation scripts"). In modern translation environments the translator has the MT suggestion available (as well as the TM and glossary suggestions, if they exist) and decides what should be done with it, i.e. edited or accepted as is.

Also MT has evolved massively and now there are systems that can even take into account the translator's choices in order to finetune their output in the process.

As for 2 > 2.1 syncing, was that done on a segment-match level or on a token level. I.e. was the actual source text taken into account or something like $txt[65]? As the two versions could use different tokens for the same actual text.

I have the feeling that quite a lot of linguistic info is missing there from previous version.

Dzonny

I actually believe that the sync is performed for the exact string match, so if the string name and the value of source string is the same. Note that LE is made more than a decade ago, and there may be bugs even in the sync function, though it worked well the last time we used it.

About the MT, what service you found most reliable?
I'd like to test it thoroughly, and would like you provide you access to our test project on Crowdin if you'd like to see it as well, considering that you have experience with it.

spiros

About MT: the simplest thing is just to activate it on Crowdin. I see they support various engines. Not clear to me whether your package already supports some of them for free or extra payment has to be made. Reading the above link I see "Watson (IBM) Translator offers 1,000,000 characters per month free of charge". You can start with the free one. I never tested it personally and quality of MT engines also depends on language pair and subject matter (for example some of them might not support specific languages or provide poor results).

Personalized engines are better. That means that you have to feed them with the existing human translation memory (bilingual parallel strings) for each language so that their suggestions will be based off that (Google Cloud AutoML Translation, Microsoft and DeepL do that). But simply using the standard Google Translate is quite good anyway and has a small charge ($10 for up to 500,000 characters per month).

m4z

Quote from: @rjen on April 11, 2021, 04:23:17 PM
Quote from: m4z on April 11, 2021, 02:56:44 PM
Quote from: spiros on April 11, 2021, 07:35:01 AM
Also, I see quite a lot of strings untranslated that have been translated in SFM 2. Is there a language-specific TM/glossary used?

2.0 isn't in Crowdin at all; so if the string wasn't in 2.1 in LE, it won't be in Crowdin, AFAIK.

I am also finding many strings in Crowdin supposedly untranslated that were already translated in LE brach 2.1 before. As a matter of fact the dutch translation was at 100% in LE and now is far from that: there is definately progress lost.

I'm not sure about the timeline, but I think this is caused by two (and a half) things:
* Part of that might be caused by this issue, are these the affected strings? During the move to Crowdin, many strings had to be changed because Crowdin didn't support the way we put strings together, so we had to adopt a more "standard" way (called "sprintf"). Since (IIRC) LE was freezed already and Crowdin didn't 100% recognize the strings as identical after the change, some of those had to be translated again (at least for German).
* In Crowdin we have two "versions" of SMF 2.1:
  * the current "tag" (RC3), called SMF_2-1 (with a high translated percentage for Dutch) and
  * the changes since then, let's call it the "development" version (RC4), called SMF_2-1_NEXT (0% translated for Dutch).

Crowdin recognizes identical strings, but not all strings are identical, so this brings down the total percentage a bit. The last half of this second point is that NEXT contains *a lot* of new timezone strings, so that plays an especially big role.


Quote from: @rjen on April 11, 2021, 04:23:17 PM
I have started updating some strings from the 2.1 language files I previously downloaded from LE, but I do not understand when the are actually adopted in Crowding: the process is not yet clear to me...

They should end up in the 2.1 downloads (updated daily?). Or what exactly do you mean?
"Faith is what you have in things that don't exist."
--Homer Simpson

Es gibt hier im Forum ein deutsches Support-Board!

@rjen

Just had another go at Crowdin: we are not friends yet! Why is it not easy to look at all strings in a file?
Can someone point me to a 'get started' page?

Just for the heck of it I downloaded the Dutch files and put them in my test forum: many, many errors due to language files containing way more / characters in the Dutch files then in the English files. What's up with that. Then trying to compare the file I can only see 'untranslated' strings?

Had to download both English and Dutch to compare?

Seriously???
Running SMF 2.1 with latest TinyPortal at www.fjr-club.nl

@rjen

Ok, after a good meal and a few youtube tutorials it now starts to come together. Starting to finally understand its workings.
I found how to download the file form Crowdin and now I am comparing it to the translated version I have from RC2/3

Now for a more procedural question: going over the missing translations that I still have from RC2 I am finding some text strings in Crowdin that are pretty obviously wrong. The correct translation is there, but has been superceeded by an incorrect translation.

I can now 'upvote' the correct translation, but when I again download the file it still shows the incorrect translations.
How and when are these 'upvoted' corrections reviewed and /or adopted?
Running SMF 2.1 with latest TinyPortal at www.fjr-club.nl

Dzonny

Glad that you find a way to what you needed, sorry for not answering earlier though!
We figured that everyone who may need additional help would find Crowdin documentation as a good place to start, though we're gonna update our current docs regards translating to answer on all the questions you may have while translating. This is the big change for all of us, but hopefully we'll end up with quality translated packs after all.

You're added as a proofreader to Dutch pack, so you should be able to have additional options on the platform now.

About having all strings in one place - not really sure how you mean, but in each file you have an option to show all strings by using CTRL + SHIFT + A -> or use those 3 little lines in upper left corner for more display options though.

@rjen

Quote from: Dzonny on April 18, 2021, 06:51:28 PM
You're added as a proofreader to Dutch pack, so you should be able to have additional options on the platform now.

About having all strings in one place - not really sure how you mean, but in each file you have an option to show all strings by using CTRL + SHIFT + A -> or use those 3 little lines in upper left corner for more display options though.

Thanks, that helps me progressing...

The 'all strings in one place' was indeed related to the filters: that helps already.

As proofreader I find that I also get the 'list option' with the strings listed showing both original string and translated string..  that was pretty much what I was looking for. Does not seem to be available for translators though?

Running SMF 2.1 with latest TinyPortal at www.fjr-club.nl

@rjen

Quote from: m4z on April 12, 2021, 06:06:58 PM

* In Crowdin we have two "versions" of SMF 2.1:
  * the current "tag" (RC3), called SMF_2-1 (with a high translated percentage for Dutch) and
  * the changes since then, let's call it the "development" version (RC4), called SMF_2-1_NEXT (0% translated for Dutch).


Ok, so RC3 is done (again).

Now I am trying to understand what the idea is with SMF_2-1_NEXT?
How are the RC3 entries that are unchanged moved to the NEXT release?

Do we have to move them ourselves? Via upload?
Or is there some automated process that will do that?

This is something that is not in the Crowdin documentation is as far as I can see.
Some explanation / instructions would be nice...
Running SMF 2.1 with latest TinyPortal at www.fjr-club.nl

Dzonny

Basically, the NEXT branch is automatically updated from github changes, so we have all new strings or changes in existing strings in that branch. Once the new version is released e.g. RC4 the "Next" branch will became main one, and new "next" branch will be created for upcoming changes after the release. You shouldn't worry about the process itself though, as it's completely handled by Crowdin and our Team, so it should work seamless.

shawnb61

Quote from: @rjen on April 21, 2021, 04:04:42 PM
Now I am trying to understand what the idea is with SMF_2-1_NEXT?
How are the RC3 entries that are unchanged moved to the NEXT release?

Do we have to move them ourselves? Via upload?
Or is there some automated process that will do that?

This is something that is not in the Crowdin documentation is as far as I can see.
Some explanation / instructions would be nice...

The documentation is here:
https://support.crowdin.com/versions-management/

The different branches share the same translation memory.  If a string is unchanged across branches, its translations will be shared across branches.  No copying is needed.

The NEXT branch gives us is the ability to work on the next release in advance, based on the current GitHub code, so translations exist upon release.  No more time lag post-release for language support.
Address the process rather than the outcome.  Then, the outcome becomes more likely.   - Fripp

@rjen

Ok, it makes sense now. Although not that obvious at first the language strings that are unchanged are showing grayed-out in the next branch.

I assume that any changes approved bin rc3 will immediately show up in rc4?

EDIT; not always it seems: there are text strings that have been translated and approved in rc3 that are nottranslated in rc4. Even though the text string name is identical.

What about the other way around? If a string has been approved in rc3 and is textually updated in rc4, will that update the rc3 translation too?
Running SMF 2.1 with latest TinyPortal at www.fjr-club.nl

Dzonny

Quote from: @rjen on April 22, 2021, 02:12:20 AM
EDIT; not always it seems: there are text strings that have been translated and approved in rc3 that are nottranslated in rc4. Even though the text string name is identical.
Can you tell us exactly which strings?
Please make sure it's 100% identical. That most likely happen because there's quite a number of fixes that we pushed through the GH, typos, removing empty spaces in strings, quotes etc. Those kind of stuff will indeed be shown as "not translated", as it needs to be reviewed, similar like we had in LE.

@rjen

Example is in index.dutch.php

text string: $txt['agreement_missing']

was translated and approved in rc3 this week, still showing as untranslated in rc4
Running SMF 2.1 with latest TinyPortal at www.fjr-club.nl

Dzonny

Thanks!
That was changed with simple comma removing from the string, see attached.
System detected that original/source string was changed, and that's why it's showing it as untranslated until you review it.

@rjen

Ok, that's clear then: ANY change to the actual Source string (English) will make the text relevant for translation in the new version...

That's good to know... so I think all my questions have been adressed...
Running SMF 2.1 with latest TinyPortal at www.fjr-club.nl

Advertisement: