SMF moves to a new translation platform: switching to Crowdin

[Dzonny]

SMF moves to a new translation platform: switching to Crowdin

As SMF continues to grow, it has adapted to the increasing needs of the community and our translators. Up until this point, we have been using our own translation management tool called 'Language Editor (LE)'.  This has served the community very well for years, however, over time, it has become more difficult for the Team to maintain.  As such, we have decided to change to Crowdin.



Crowdin works smoothly and allows for a more flexible translation process. Furthermore, it provides many useful features, such as global translation memory, machine translations, and online editors. It offers convenient ways to comment on existing texts and to report mistakes or problems. As a result, using Crowdin will make it much easier to discuss, for example, the terminology, meaning, and optimal translation of each and every software string.  We believe that Crowdin will help us to optimize the translation process. It will make the process of translating faster and facilitate problem solving, resulting in higher quality translations


Want to improve SMF in your language? Here is how you can help

If you like SMF and want to contribute by translating it into your language, sign up at Crowdin, go to our page and start translating. We are looking forward to seeing you there.  If you have already registered on our SMF community forum, just use this link to proceed and join our project on Crowdin without the need to additionally sign up on their page.  All you need to do is to log in with your SMF account.

https://dev.simplemachines.org/crowdin.php

We understand that using the new platform may seem quite complicated for users who have not used it before, so we have written this article for current and potential translators, which includes detailed guidelines on how to translate SMF on Crowdin to make it easier for you to work with this software.

Please note that Crowdin will be officially used as the Translation Management Software from branch 2.1 onwards. Older branches of the software will continue to be updated using the LE (Language Editor) until EOL (end of life).


How translation is changing

Nothing has significantly changed in our translation management system on the new platform, but there are some important differences in terminology that Senior Translators, in particular, need to be aware of.  For instance, SMF Senior Translators will become Proofreaders on Crowdin, but their role remains the same.
Proofreaders (Senior Translators) will continue to:
•   Approve/unapprove translations;
•   Be responsible for overseeing the accuracy of translations;
•   Answer questions and requests in comments;
•   Resolve issues.

Please note that on the SMF community, Senior Translators will still be referred to as such and the change in Crowdin is purely one of terminology, rather than a change of role.  If anything, the new tools available in Crowdin enhance the role of a Senior Translator and make it easier for them to carry out their duties.

[peterbehlendorf]

Signing up now  hope all goes well!

[chadon]

Thanks for your hard work. I will try it later but it looks vers promising.

[-Rock Lee-]

Ohh it looks very good, as soon as I can I will give it atoms thanks for so much work!

[Oldiesmann]

Hopefully this will make translations much easier for everyone. Unfortunately I'm not fluent in any foreign languages so I probably can't be much help (I did know quite a bit of German at one time but have forgotten most of it in the 20 years since I graduated high school )

[saosangmo]

Thank Dzonny,

I have started translating Vietnamese.

[GravuTrad]

Cool news.


I know well the tool so it's cool.

[Deaks]

this is great to see well done to everyone who has been working on it in the background.  It has definitely been needed for a while now!

[Suki]

Congratulations!!!! 🎉🎉🎉

[shadav]

  great work on everyone involved and I'm sure this will help streamline things
even though it's all gibberish to me 

[Robert.]

Nice work team.

[Olivia232425]

Thanks for your hard work. I will try it later but it looks vers promising.




Signature line removed. Please do not add "fake" signatures to your posts....once you are an active member you will be allowed to use the signature feature of the forums.

[Apllicmz]

Yes
nice Work thank you

[Looking]

Interesting move.

[gisfreak]

wowow, congratulations,
gotta try this new beast, nice

All you need to do is to log in with your SMF account.

seems i cant login with my account, 

[Dzonny]

Did you try to use the link we posted in the first post though?
https://dev.simplemachines.org/crowdin.php

Through that link you should be able to easily create new account using all the info from SMF account, unless you want to change something during the process. Let us know if you need a hand though.

[gisfreak]

Did you try to use the link we posted in the first post though?
https://dev.simplemachines.org/crowdin.php

just misunderstanding, now all sorted out, thank you 

[spiros]

When I click on view strings I get "error loading resources".

Also, I see quite a lot of strings untranslated that have been translated in SFM 2. Is there a language-specific TM/glossary used?

[m4z]

When I click on view strings I get "error loading resources".

Never seen that; which browser (version, addons) are you using?

Also, I see quite a lot of strings untranslated that have been translated in SFM 2. Is there a language-specific TM/glossary used?

2.0 isn't in Crowdin at all; so if the string wasn't in 2.1 in LE, it won't be in Crowdin, AFAIK.

[spiros]

Not an issue of browser, tried different. And at first I did not have this issue. Maybe point me to a specific string URL that should be editable/translatable so that we are on the same page.

Well, I am a translator by profession and I also teach CAT tools. One can always leverage the translations from an earlier version Not sure if it has been done and to what extend.

[m4z]

Not an issue of browser, tried different. And at first I did not have this issue. Maybe point me to a specific string URL that should be editable/translatable so that we are on the same page.

Since you probably want to translate Greek (and have the appropriate permissions in Crowdin), try this: https://crowdin.com/translate/simple-machines-21-beta/all/en-el?filter=basic&value=0 (or click https://dev.simplemachines.org/crowdin.php → select a language → click "Translate All")
(PS: What exactly did you try? I don't know where the "View strings" link you mentioned is located.)

Well, I am a translator by profession and I also teach CAT tools. One can always leverage the translations from an earlier version Not sure if it has been done and to what extend.

I had to google what CAT could be, never heard of that acronym or term (I assume you mean "Computer-assisted translation").
What I meant is: I don't know if the 2.0 strings were ever imported into the TM (and I don't think so). Dzonny might know.

[spiros]

Tried logging out and logging in again. It works if I start from home page.

Yes, all localization is carried out nowadays using said tools (whether desktop or online).
https://en.wikipedia.org/wiki/Computer-assisted_translation#Translation_memory_software

Then it would be a good idea to leverage those legacy translations for all languages (so as not to enter again existing translations).

Also, maybe enable MT suggestions in the system?

[@rjen]

Also, I see quite a lot of strings untranslated that have been translated in SFM 2. Is there a language-specific TM/glossary used?

2.0 isn't in Crowdin at all; so if the string wasn't in 2.1 in LE, it won't be in Crowdin, AFAIK.

I am also finding many strings in Crowdin supposedly untranslated that were already translated in LE brach 2.1 before. As a matter of fact the dutch translation was at 100% in LE and now is far from that: there is definately progress lost.

I have started updating some strings from the 2.1 language files I previously downloaded from LE, but I do not understand when the are actually adopted in Crowding: the process is not yet clear to me...

[Dzonny]

Thanks for your comments guys !

2.0 branch is not planned to be moved to Crowdin, but we did sync between 2.0 -> 2.1 branch in LE before moving 2.1 to Crowdin.

The whole moving process took some time, as you may know we announced moving for translators quite some time ago, and in the mean time there were additional changes in 2.1 branch, new strings etc. Crowdin is now synced with latest github files, so changes from GH will be added to Crowdin automatically.

During the whole moving process, we worked with Crowdin support team in order to adjust our current syntax and logic to their system, so we had to make some additional customization in our code and strings in order for the whole system to work.
We're sorry if there's some translations lost during this, as we had to manually transfer some files, and even manually c/p specific strings.
We already stated on Crowdin that, for example registration agreement must be re-translated for some languages, as we couldn't bulk upload .txt translations etc.

Anyway, if you spot major part of specific file in language that is not translated on Crowdin, and was translated in LE - let us know so we can check, and possibly transfer translations to Crowdin.

We're sure that this kind of things won't happen in the future, but radical changes like moving from outdated LE to new software like Crowdin, sadly caused some glitches, but we're also happy that all is fixed now

About Machine translation -  MT, we have that disabled for now. You may remember that we have no automated translation scripts policy which is still in effect. We'll surely consider your suggestion though, as translation scripts improved significantly over the last years.

However, there's a Translation Memory enabled, and you can see more about it here:
https://support.crowdin.com/translation-memory/

Let us know if you need any help with the error you got earlier, or that is sorted now?

[spiros]

Machine-aided human translation is different than using MT in an automated way ("automated translation scripts"). In modern translation environments the translator has the MT suggestion available (as well as the TM and glossary suggestions, if they exist) and decides what should be done with it, i.e. edited or accepted as is.

Also MT has evolved massively and now there are systems that can even take into account the translator's choices in order to finetune their output in the process.

As for 2 > 2.1 syncing, was that done on a segment-match level or on a token level. I.e. was the actual source text taken into account or something like $txt[65]? As the two versions could use different tokens for the same actual text.

I have the feeling that quite a lot of linguistic info is missing there from previous version.

[Dzonny]

I actually believe that the sync is performed for the exact string match, so if the string name and the value of source string is the same. Note that LE is made more than a decade ago, and there may be bugs even in the sync function, though it worked well the last time we used it.

About the MT, what service you found most reliable?
I'd like to test it thoroughly, and would like you provide you access to our test project on Crowdin if you'd like to see it as well, considering that you have experience with it.

[spiros]

About MT: the simplest thing is just to activate it on Crowdin. I see they support various engines. Not clear to me whether your package already supports some of them for free or extra payment has to be made. Reading the above link I see "Watson (IBM) Translator offers 1,000,000 characters per month free of charge". You can start with the free one. I never tested it personally and quality of MT engines also depends on language pair and subject matter (for example some of them might not support specific languages or provide poor results).

Personalized engines are better. That means that you have to feed them with the existing human translation memory (bilingual parallel strings) for each language so that their suggestions will be based off that (Google Cloud AutoML Translation, Microsoft and DeepL do that). But simply using the standard Google Translate is quite good anyway and has a small charge ($10 for up to 500,000 characters per month).

[m4z]

Also, I see quite a lot of strings untranslated that have been translated in SFM 2. Is there a language-specific TM/glossary used?

2.0 isn't in Crowdin at all; so if the string wasn't in 2.1 in LE, it won't be in Crowdin, AFAIK.

I am also finding many strings in Crowdin supposedly untranslated that were already translated in LE brach 2.1 before. As a matter of fact the dutch translation was at 100% in LE and now is far from that: there is definately progress lost.

I'm not sure about the timeline, but I think this is caused by two (and a half) things:
* Part of that might be caused by this issue, are these the affected strings? During the move to Crowdin, many strings had to be changed because Crowdin didn't support the way we put strings together, so we had to adopt a more "standard" way (called "sprintf"). Since (IIRC) LE was freezed already and Crowdin didn't 100% recognize the strings as identical after the change, some of those had to be translated again (at least for German).
* In Crowdin we have two "versions" of SMF 2.1:
  * the current "tag" (RC3), called SMF_2-1 (with a high translated percentage for Dutch) and
  * the changes since then, let's call it the "development" version (RC4), called SMF_2-1_NEXT (0% translated for Dutch).

Crowdin recognizes identical strings, but not all strings are identical, so this brings down the total percentage a bit. The last half of this second point is that NEXT contains *a lot* of new timezone strings, so that plays an especially big role.

I have started updating some strings from the 2.1 language files I previously downloaded from LE, but I do not understand when the are actually adopted in Crowding: the process is not yet clear to me...

They should end up in the 2.1 downloads (updated daily?). Or what exactly do you mean?

[@rjen]

Just had another go at Crowdin: we are not friends yet! Why is it not easy to look at all strings in a file?
Can someone point me to a 'get started' page?

Just for the heck of it I downloaded the Dutch files and put them in my test forum: many, many errors due to language files containing way more / characters in the Dutch files then in the English files. What's up with that. Then trying to compare the file I can only see 'untranslated' strings?

Had to download both English and Dutch to compare?

Seriously???

[@rjen]

Ok, after a good meal and a few youtube tutorials it now starts to come together. Starting to finally understand its workings.
I found how to download the file form Crowdin and now I am comparing it to the translated version I have from RC2/3

Now for a more procedural question: going over the missing translations that I still have from RC2 I am finding some text strings in Crowdin that are pretty obviously wrong. The correct translation is there, but has been superceeded by an incorrect translation.

I can now 'upvote' the correct translation, but when I again download the file it still shows the incorrect translations.
How and when are these 'upvoted' corrections reviewed and /or adopted?

[Dzonny]

Glad that you find a way to what you needed, sorry for not answering earlier though!
We figured that everyone who may need additional help would find Crowdin documentation as a good place to start, though we're gonna update our current docs regards translating to answer on all the questions you may have while translating. This is the big change for all of us, but hopefully we'll end up with quality translated packs after all.

You're added as a proofreader to Dutch pack, so you should be able to have additional options on the platform now.

About having all strings in one place - not really sure how you mean, but in each file you have an option to show all strings by using CTRL + SHIFT + A -> or use those 3 little lines in upper left corner for more display options though.

[@rjen]

You're added as a proofreader to Dutch pack, so you should be able to have additional options on the platform now.

About having all strings in one place - not really sure how you mean, but in each file you have an option to show all strings by using CTRL + SHIFT + A -> or use those 3 little lines in upper left corner for more display options though.

Thanks, that helps me progressing...

The 'all strings in one place' was indeed related to the filters: that helps already.

As proofreader I find that I also get the 'list option' with the strings listed showing both original string and translated string..  that was pretty much what I was looking for. Does not seem to be available for translators though?

[@rjen]

* In Crowdin we have two "versions" of SMF 2.1:
  * the current "tag" (RC3), called SMF_2-1 (with a high translated percentage for Dutch) and
  * the changes since then, let's call it the "development" version (RC4), called SMF_2-1_NEXT (0% translated for Dutch).

Ok, so RC3 is done (again).

Now I am trying to understand what the idea is with SMF_2-1_NEXT?
How are the RC3 entries that are unchanged moved to the NEXT release?

Do we have to move them ourselves? Via upload?
Or is there some automated process that will do that?

This is something that is not in the Crowdin documentation is as far as I can see.
Some explanation / instructions would be nice...

[Dzonny]

Basically, the NEXT branch is automatically updated from github changes, so we have all new strings or changes in existing strings in that branch. Once the new version is released e.g. RC4 the "Next" branch will became main one, and new "next" branch will be created for upcoming changes after the release. You shouldn't worry about the process itself though, as it's completely handled by Crowdin and our Team, so it should work seamless.

[shawnb61]

Now I am trying to understand what the idea is with SMF_2-1_NEXT?
How are the RC3 entries that are unchanged moved to the NEXT release?

Do we have to move them ourselves? Via upload?
Or is there some automated process that will do that?

This is something that is not in the Crowdin documentation is as far as I can see.
Some explanation / instructions would be nice...

The documentation is here:
https://support.crowdin.com/versions-management/

The different branches share the same translation memory.  If a string is unchanged across branches, its translations will be shared across branches.  No copying is needed.

The NEXT branch gives us is the ability to work on the next release in advance, based on the current GitHub code, so translations exist upon release.  No more time lag post-release for language support.

[@rjen]

Ok, it makes sense now. Although not that obvious at first the language strings that are unchanged are showing grayed-out in the next branch.

I assume that any changes approved bin rc3 will immediately show up in rc4?

EDIT; not always it seems: there are text strings that have been translated and approved in rc3 that are nottranslated in rc4. Even though the text string name is identical.

What about the other way around? If a string has been approved in rc3 and is textually updated in rc4, will that update the rc3 translation too?

[Dzonny]

EDIT; not always it seems: there are text strings that have been translated and approved in rc3 that are nottranslated in rc4. Even though the text string name is identical.

Can you tell us exactly which strings?
Please make sure it's 100% identical. That most likely happen because there's quite a number of fixes that we pushed through the GH, typos, removing empty spaces in strings, quotes etc. Those kind of stuff will indeed be shown as "not translated", as it needs to be reviewed, similar like we had in LE.

[@rjen]

Example is in index.dutch.php

text string: $txt['agreement_missing']

was translated and approved in rc3 this week, still showing as untranslated in rc4

[Dzonny]

Thanks!
That was changed with simple comma removing from the string, see attached.
System detected that original/source string was changed, and that's why it's showing it as untranslated until you review it.

[@rjen]

Ok, that's clear then: ANY change to the actual Source string (English) will make the text relevant for translation in the new version...

That's good to know... so I think all my questions have been adressed...

[Dzonny]

Yes, exactly, as change in source string requires action/update for each language

That's good to know, if you find anything else we're here to help.

Thanks for doing the great work @rjen!

[PoML]

Currently, the situation is more or less like so:
[...]
- the first translation from accepted translators (that's most of you) will be auto-approved
[...]
- once one translation for a string has been approved, future suggestions for that string have to be manually approved

Looking into the crowdin overview, almost no languages has a significant portion of approved words.
For Norwegian it is stuck on 0% as for many other languages.  I have no obvious way of approving anything, and download as ZIP from within Crowdin only lists strings that were translated in LE and uploaded when we started using it...

Is there a simple fix?

[@rjen]

For Dutch I am now granted Proofreader rights. Proofreaders can approve the translations

There has not been any auto approval for as far as I know...

[Dzonny]

Even though translation is not approved - it will be included in exported file. Approval is just one additional step to ensure translation quality where we have bigger translation teams with proofreaders.

[Masterd]

Took you long enough. I remember you silently mentioning this back in May. Congratulations on the move. Using the LE was a truly painful experience sometimes. I hope this is a significant improvement. Well, as much as it can be.

[PoML]

Even though translation is not approved - it will be included in exported file. Approval is just one additional step to ensure translation quality where we have bigger translation teams with proofreaders.

Good, but then why are all the files in the downloaded archive given a date of Feb 26th instead of the date it was translated or changed?
Is it a bug?

[PoML]

I just tried downloading and took a random sample:

1) Download from https://download.simplemachines.org/index.php?thanks;filename=smf_2-1-rc3_norwegian.zip
Dates within file :  March 30 2021.
Example untranslated:  $txt['mc_warning_template_body_default']  in moderationcenter.norwegian.php  (it has english text in file)

2) Manual download from within Crowdin today using the "download" function -- shortcut <ctrl>S:
Dates within file :  Feb 26 2021.
Example untranslated:  $txt['mc_warning_template_body_default']  in moderationcenter.norwegian.php (it has english text in file)

3) Actual translation in Crowdin:
$txt['mc_warning_template_body_default']  in moderationcenter.norwegian.php   26 days ago today. All OK, translated to Norwegian but not marked as approved.

Code Select Expand

{MEMBER},\n\nDu har mottatt en advarsel for upassende aktivitet. Vennligst slutt med det og følg forumreglene ellers vil vi måtte ta ytterligere tiltak. \n\n{REGARDS}

I seem not to find any way of actually getting to the translated files neither within Crowdin nor in the download area.
Seeing them on screen in the crowdin editor is of course slightly satisfying, but not what I was hoping for.

[d3vcho]

I just downloaded the language pack of the URL you put in 1). The file you say, ModerationCenter.norwegian.php, has today's date (01/05/2021) and the string you mention appears translated, as you state in 3).

[PoML]

Yes!
Now, it seems OK. Maybe some sort of delay

[Irisado]

Debate about Crowdin and 2.1 development has been split and moved to Chit Chat: https://www.simplemachines.org/community/index.php?topic=578047.0.

[vkot]

Crowdin seems nice! I tested it by translating many strings already.

1) Me too, I see strings that are already translated in previous versions, at least I think so... Maybe the variable names are different, I didn't check.
2) Can you add Google Translate together with the TM and MT suggestions? Sometimes there are no suggestions, especially in big strings, and I am lazy to type the whole translation, I prefer to copy/paste/edit.
3) Sometimes while I am translating a file Crowdin asks me to login again (I was not inactive)

[wintstar]

What Version is mean with SMF_2-1_NEXT?

[shawnb61]

What Version is mean with SMF_2-1_NEXT?

SMF_2-1_NEXT refers to current GitHub code.

The main branch SMF_2-1 is tied to the latest release - RC4.

I.e., we can now prepare translations for the upcoming release, instead of waiting until after each release, as in the past.

[wintstar]

What Version is mean with SMF_2-1_NEXT?

SMF_2-1_NEXT refers to current GitHub code.

The main branch SMF_2-1 is tied to the latest release - RC4.

I.e., we can now prepare translations for the upcoming release, instead of waiting until after each release, as in the past.

Many thanks for Info. Then I can get started there. I have translated the current commit for the German "Sie".

[jaideejung007]

Great project!