Multilingual Gecko Status Update 2018.2

Welcome to the third edition of Multilingual Gecko Status Update!

In the previous update we covered the work which landed in Firefox 59 and Firefox 60.

At the time, we’ve been finalizing the platform work to support Fluent localization system, and we were in the middle of migration of the first Firefox UI component – Preferences – to it.

Today, we’ll pick up right where we left off!

Firefox 61 (June)

Firefox 61 fits into the trend of things calming down in Intl – and it’s a great news! It means that we are reaching platform maturity after all the groundbreaking refactors in 2017, and we all can focus on the work on top of the modules, rather than playing whack-a-mole fixing bugs and adding missing features.

The biggest platform change is really just an update to ICU 61 which Andre landed in March and my work on adding mozIntl.RelativeTimeFormat and mozIntl.getLocaleDisplayNames.

The former gave us a stable unified API for presenting relative time (such as “In 5 minutes” or “10 days ago”) while the latter unified how we present language, region and combinations of those in our user interface based on the Unicode CLDR representation (example: “English (United States)”).

As I explained in my earliest posts, one of the things I’m particularly proud of is that we go the extra mile to use every such opportunity to not only fix the immediate Firefox UI need, but also push such proposals for standardization and in result make the Web Platform more complete.

In this case, Intl.RelativeTimeFormat has been proposed and thanks to amazing work by Daniel Ehrenberg is now in Stage 3 and soon will be exposed to all web developers in all browsers! Intl.getLocaleDisplayNames is less mature but the work on it just picked up.

Firefox migration to Fluent reached its next milestone moving from just a couple messages to over 100 Fluent messages in Firefox!

Notable changes [my work] [intl module]:

Firefox 62 (September)

Another calm cycle! The biggest feature was the introduction of the pseudolocalization in Firefox which I blogged about, and landing of the developer documentation.

The documentation has been insanely useful in distributing knowledge and helping engineers feel more comfortable working with the new stack, and I’m very happy we reached a stage where landing a documentation is the big news ­čÖé

In the Fluent land we spent the March-May timeframe moving from a 100 messages to 500 Fluent messages in Firefox!

Notable changes [my work] [intl module]:

Firefox 63 (October)

This cycle was quite similar to the previous one, with a bulk of work going into regular maintenance and cleanups.

For my work, the trend is to start integrating Fluent deeper into Gecko with more work around L10nRegistry v1 limitations and getting deeper DOM integration to improve XUL performance.

In this cycle I landed a XPCOM mozIDOMLocalization API which allows us to call Fluent from C++ and was required for a bigger change that we landed in Firefox 64.

One new theme is Kris Maglione who started working on reducing the performance and memory overhead coming from the old StringBundle API (used for .properties). With all the new work centralized around Fluent, but with a large portion of our strings still using StringBundle, it becomes a great target for optimizations by cutting out everything we don’t use and now we know – we never will.

Notable changes [my work] [intl module]:

Firefox 64 (December)

This release, is still getting stabilized and will get released in December, but the work cycle on it happened between September and October, so I can already provide you an account of that work!

Besides of a regular stack of cleanups coming from Henri and me, we’ve seen Ehsan Akhgari taking over from Kris to remove more lines of unused code.

The really big change was the introduction of DocumentL10n API which is a C++ API with its own WebIDL tightly integrated into the very core of DOM module in Gecko – nsIDocument.

Before that Fluent lived in Gecko in some form of a glorified javascript library. While it is Fluent’s goal to target the web platform, Firefox UI is inherently different from the web content and benefits from better integration between the DOM and its localization component.

This change allowed us to better integrate localization into the document’s life cycle, but what’s even more important, it allowed us to expose Fluent to documents that usually do not have special privileges and could not access Fluent before.

As for migration, we moved along nicely bumping from 500 to around 800 messages thanks to hard work of a number of students mentored by Jared Wein and Gijs Kruitbosch. The students picked up work on the migration as their Capstone project.

Notable changes [my work] [intl module]:

Summary

2018 has been much “easier” for the intl module than 2017 was. It’s great to see the how all pieces fit together and for me personally, it enabled me to focus on getting Fluent better integrated into Gecko.

There’s still a lot of work but it now is fully focused on Fluent and localization, while our intl module as a whole goes through a more well earned peaceful period.

Between now and the next status update, I hope to publish a summary post about the last two years of work. Stay tuned!

Pseudolocalization in Firefox

One of the core projects we did over 2017 was a major overhaul of the Localization and Internationalization layers in Gecko, and all throughout the first half of 2018 we were introducing Fluent into Firefox.

All of that work was “behind the scenes” and laid the foundation to enable us to bring higher level improvements in the future.

Today, I’m happy to announce that the first of those high-level features has just landed in Firefox Nightly!

Pseudolocalization?

Pseudolocalization is a technology allowing for testing the localizability of software UI. It allows developers to check how the UI they are working on will look like when translated, without having to wait for translations to become available.

It shortens the Test-Driven Development cycle and lowers the burden of creating localizable UI.

Here’s a demo of how it works:

How to turn it on?

At the moment, we don’t have any UI for this feature. You need to create a new preference called intl.l10n.pseudo and set its value to accented for a left-to-right, ~30% longer strategy, or bidi for a right-to-left strategy. (more documentation).

If you test the bidi strategy you also will likely want to switch another preference – intl.uidirection – to 1. This is because right now the directionality of text and layout are not connected. We will improve that in the future.

We’ll be looking into ways to expose this functionality in the UI, and if you have any ideas or suggestions for what you’d like to see, let’s talk!

Nitty-gritty details

Although the feature may seem simple to add, and the actual patch that adds it was less than 100 lines long, it took many years of prototyping and years of development to build the foundation layers to allow for it.

Many of the design principles of Project Fluent combined with the vision shaped by the L10n Drivers Team at Mozilla allowed for dynamic runtime locale switching and declarative UI localization bindings.

Thanks to all of that work, we don’t have to require special builds or increase the bundle size for this feature to work. It comes practically for free and we can extend and fine tune pseudolocalization strategies on fly.

Kudos

If that feature looks cool, in the esoteric way localization and internationalization can, please, make sure to high-five the people who put a lot of work to get this done: Sta┼Ť Ma┼éolepszy, Axel Hecht, Francesco Lodolo, Jeff Beatty and Dave Townsend.

More features are coming! Stay tuned.

Multilingual Gecko Status Update 2018.1

As promised in my previous post, I’d like to do a better job at delivering status updates on Internationalization and Localization technologies at Gecko at shorter intervals than once per year.

In the previous post we covered recent history up to Firefox 58 which got released in January 2018. Since then we finished and shipped Firefox 59 and also finished all major work on Firefox 60, so this post will cover the two.

Firefox 59 (March)

Firefox 58 shipped with a single string localized using Fluent. In 59 we made the next step and migrated 5 strings from an old localization system to use Fluent. This allowed us to test all of our migration code to ensure that as we port Firefox to the new API we preserve all the hard work of hundreds of localizers.

In 59 we also made several improvements to performance of how Fluent operates, all of that while waiting for the stylo-chrome to land in Firefox 60.

In LocaleService, we made another important switch. We replaced old general.useragent.locale and intl.locale.matchOS prefs with a single new pref intl.locale.requested.

This change accomplished several goals:

  • The name represents the function better. Previously it was pretty confusing for people as to why Gecko doesn’t react immediately when they set the pref. Now it is more clear that this is just a requested locale, and there’s some language negotiation that, depending on available locales, will switch to it or not.
  • The new pref is optional. Since by default it matches the defaultLocale, we can now skip it and just treat its non-presence as the default mode in which we follow the default locale. That allowed us to remove some code.
  • The new pref allows us to store locale lists. The new pref is meant to store a comma separated list of requested locales like "fr-CA, fr-FR, en-US", in line with our model of handling locale lists, rather than single locales.
  • If the pref is defined and the value is empty, we’ll look into OS for the locale to use, making it a replacement for the matchOS pref.

This is important particularly because it took us very long time to unify all uses of the pref, remove it from all around the code and finally be able to switch to the new one which should serve us much better.

Next come a usual set of updates, including update to ICU 60 by Andre, and cleanups by Makoto Kato – we’re riding the wave of removing old APIs and unifying code around ICU and encoding_rs.

Lastly, as we start looking more at aligning our language resources with CLDR, Francesco started sorting out our plural rules differences and language and region names. This is the first step on the path to upstream our work to CLDR and reuse it in place of our custom data.

Notable changes [my work] [intl]:

Firefox 60 (May)

Although Firefox 60 has not yet been released as of today, the major work cycle on it has finished, and it is currently in the beta channel for stabilization.

In it, we’ve completed another milestone for Fluent migrating not just a couple, but over 100 strings in Firefox Preferences from the old API. This marks the first release where Fluent is actually used to localize a visible portion of Firefox UI!

As part of that work, we pushed our first significant update of Fluent in Gecko, and landed a special chrome-only API to get Fluent’s performance on par with the old system.

With an increase in the use of Fluent, we also covered it with Mozilla eslint rules, improved missing strings reporting, and wrote an Introduction to Fluent for Firefox Developers.

On the Locale Management side, we separated out mozilla::intl::Locale class and further aligned it with BCP47.

But the big change here is the switch of the source of available locales from the old ChromeRegistry to L10nRegistry.

This is the last symbolic hand-over from the old model to the new, meaning that from that moment the locales registered in L10nRegistry will be used to negotiate language selection for Firefox, and ChromeRegistry becomes a regular customer rather than a provider of the language selection.

We’re very close to finalize the LocaleService model after over a year of refactoring Gecko!

Regular healthy number of cleanups happened as well. Henri switched more code to use encoding_rs, and updated encoding_rs to 0.7.2, Jeff Walden performed a major cleanup of our SpiderMonkey Intl source code, Andre added caches for many Intl APIs to speed them up and Axel updated compare-locales to 2.7,

We also encountered two interesting bugs – Andre dove deep into ICU to fix `Intl.NumberFormat` breaking on roundings in Persian, and I had to disable some of our bidirectionality features in Fluent due to a bug in Windows API.

Notable changes [my work] [intl]:

Summary

With all that work in, we’re slowly settling down the new APIs and finalizing the cleanups and the bulk of work now is going directly into switching away from DTD and .properties to Fluent.

As Firefox 60 is getting ready for its stable release, we’re accelerating the migration of Preferences to Fluent hoping to accomplish it in 61 or 62 release. Once that is completed, we’ll evaluate the experience and make recommendations for the rest of Firefox.

Stay tuned!

Multilingual Gecko in 2017

The outline

In January 2017, we set the course to get a new localization framework named Fluent into Firefox.

Below is a story of the work performed on the Firefox engine – Gecko – over the last year to make Fluent in Firefox possible. This has been a collaborative effort involving a lot of people from different teams. It’s impossible to document all the work, so keep in mind that the following is just the story of the Gecko refactor, while many other critical pieces were being tackled outside of that range.

Also, the nature of the project does make the following blog post long, text heavy and light on pictures. I apologize for that and hope that the value of the content will offset this inconvenience and make it worth reading.

Continue reading “Multilingual Gecko in 2017”

All Hands On Deck – How you can use your skills to contribute to Firefox 57 success

Firefox 57 in making

If you’ve been following Firefox development over the last year, you probably know that we’re hard at work on a major refactor of the browser, codenamed Quantum.

It’s been a very exciting and challenging time with hundreds of engineers bringing to life new concepts and incorporating them into our engineGecko. Those refactors, which will culminate in the release of Firefox 57, touch the very foundation of our engine and require massive changes to it.

We’ve moved the whole engine from a single to multi-process model which is perhaps the hardest possible refactor a major piece of software can go through. We’ve added a whole new systems level programming language, and we’re replacing major components of the engine such as styling and rendering systems. There are substantial changes to our networking layer and security layers as well as completely new components like a new extension ecosystem, virtual reality stack and APZ. We’ve added another new programming language as part of our JS engine and significantly remodeled our build process and infrastructure allowing for quick feature testing, telemetry analysis to understand how our changes impact users and many, many more things.

The changes are massive. This is the largest refactor to Gecko and Firefox in my 17 years of contributing to the project. But we’re not done yet…

Continue reading “All Hands On Deck – How you can use your skills to contribute to Firefox 57 success”

One year with the Firefox OS L10n framework

For several years now, the Localization team at Mozilla has been working on a modern localization framework based on the following set of principles and architectural choices that we consider fundamental for the next generation multilingual UI’s.

  • Principle 1: Localizers should be in control of translations. The localization framework should be grammar-agnostic, whether it’s about grammatical cases, genders, or tenses.┬á Localizers should be able to use the entire expressive power of their language to author translations which create the best experience for the users.
  • Principle 2: Language fallback should be robust and graceful. When a translation is missing or broken the user should be presented with a translation into the next best language, given their preferences. There might be more than one fallback language.
  • Principle 3: Translations should be isolated and asymmetric if needed. The source language of the application should not define structure of the translations (e.g. the lack of pluralization in English should not make it impossible for other languages to use plurals in a given message).
  • Principle 4: The framework should embrace the Web. Localization should react to changes of the runtime environment (e.g. resizing of the app’s window, change of orientation, incrementation of a number of unread messages) and should add as little overhead for developers as possible.

Exactly one year ago, on April 8th 2014, Stas landed the initial rewrite of l10n.js – the localization framework used in Firefox OS. This set us on the path to enable the vision of a modern localization framework driven by the design principles outlined above.

Since then, we have a dedicated, two person team, working full time on advancing this vision and learning how to improve upon it in the process.

The full year of work has resulted in many important features being developed for the platform, including:

  • Language packs: Small packages that decouple language resources from the application allowing us to extend language coverage dynamically when users request it, even after the device has been already released on the market.
  • Pseudo-locales: Programmatically built language resources that emulate different languages allowing developers to test their applications for any multilingual problems before localizers have time to provide translations
  • Asynchronous l10n: Major shift away from using synchronous API’s for retrieving localizations from JavaScript. This results in clean, condition race free code that is easier to write and maintain. It also sets us on the path to enable runtime language fallback.
  • Security: While not traditionally big thing in localization, having an open runtime ecosystem of localizations requires us to make sure that translations cannot accidentally or maliciously impact our code and break it.
  • Error reporting: We’ve made major advancements to help developers and localizers find potential errors early. We reject malformed strings, report missing strings and duplicates, recover from exceptions in our code.

A couple weeks ago we have finalized the work scheduled for Firefox OS 2.2 and begun development for the next major release. The clean and reliable API has given us a good base to start implementing the remaining components of the vision behind L20n in this cycle.

For the current cycle we have scheduled:

  • DOM Overlays: Ability for a localizer to use HTML syntax in their translations and also provide whole localized DOM Fragments to be merged with developer provided skeletons via a secure algorithm. This increases the system’s security and empowers localizers to provide better translations.
  • L20n format: One of the last remaining pieces of the puzzle is the new file format that is designed to store localization data like multivariant strings, entities with values and attributes and variant selectors. This will allow us to start introducing new features to the system that are impossible using the current data storing formats.
  • Lightweight l10n contexts: Together with the whole platform, we want to make a heavier use of the concept of multiple small localization contexts to replace the single-context-per-app approach. It will improve performance and isolation resulting in easier maintenance.
  • API 3.0: Our current API still contains remaining pieces from the old, synchronous API that we’d like to remove. Together with lightweight contexts and on the path to WebAPI we will want to make sure to organize our events, methods and objects to fit the design of other W3C APIs.

With ever-growing understanding of the environment and how the web stack matures, we are also getting close to start extracting the core of our framework to offer for standardization, and that’s an exciting opportunity to fulfill the vision of both Firefox OS and L20n and bring the modern localization framework to the whole web, making it more multilingual and global.

Stas & Zibi

Reducing MozL10n+Gaia technical debt in Firefox OS 2.1 cycle

Firefox OS is becoming a more mature platform, and one of the components that is maturing with it is the mozL10n library.

In 2.0 cycle, which is already feature complete, we introduced the codebase based on the L20n project.

The 2.1 cycle we’re currently in is a major API cleanup effort. We’re reviewing how Gaia apps use mozL10n, migrating them to the new API and minimizing the code complexity.

Simplifying the code responsible for localizability of Firefox OS is crucial for our ability to maintain and bring new features to the platform.

There are four major areas we’re working on:

  • mozL10n.translate – with introduction of Mutation Observer, we want to phase out manual DOM translation calls.
  • mozL10n.localize – this function is extremely hard to maintain and does enough “magic” to confuse devs.
  • mozL10n.get – manual l10n gets are the biggest cause of bugs and regressions in our code. They are synchronous, not retranslatable and badly misused
  • mock_l10n – many apps still use custom MockL10n class in tests, some even use real MozL10n code in tests. This makes testing harder to maintain and develop new l10n features.

We’re working on all four areas and would love to get your help.

No matter if you are a Gaia app owner, or if you’ve never wrote a patch for Gaia. If you know JavaScript, you can help us!

All of those bugs have instructions on how to start fixing, and I will be happy to mentor you.

We have time until October 13th. Let’s get Gaia ready for the next generation features we want to introduce soon! ­čÖé

Localization framework changes in Firefox OS 2.0 and plans for 2.1

On Monday we branched Firefox OS 2.0 which is the first branch to contain a new localization library that has been developed by my team.

What landed for 2.0

Library landing and reactions

The library itself landed exactly two months ago. In order to avoid any potential regressions, we’ve put a lot of work to ensure that it matches the behavior of the old code that it replaced. I believe we can now claim a success, because with two months of baking on master we didn’t get any serious regressions that would require us to change anything in our code.

The new library comes with a lot of unit tests and is stricter than the old code, so we had to fix a couple of small bugs where code has been passing an object instead of a string to our API and one related to one test failing on old machine with too little memory. That’s been simple to catch and fix.

We also got a few requests to improve the console log output and error output that the library produces in order to simplify developers work.

New features

Pseudo-locales

The major new thing that Stas completed in this cycle is the support for pseudo-locales. While this could be done with the old code, it was significantly easier with the new code thanks to some architectural decisions like separation of buildtime and runtime code.

Pseudo-locales allow developers to evaluate their UI’s localizability against an artificially generated english-like locale to catch any hardcoded strings. It also generates right-to-left locale for testing purposes. Before that, we’ve been relying on often outdated localizations that we kept with our source code. Now we can always test against fully generated pseudo localization.

mozL10n.once wrapper

Another new feature is the introduction of mozL10n.once wrapper. We identified that a lot of Gaia apps are waiting for localization to be ready before they initialize themselves. That makes sense since a lot of those apps want to work with UI and localized strings, but the challenge in asynchronous world is that you never know if your code has been fired after or before mozL10n is ready.

Because of that, simply setting an event listener and waiting for window.onlocalized event is not enough (what if it already has fired before your code was launched?). Developers were using mozL10n.ready wrapper, but the problem with that is that is has been designed to re-fire on reach retranslation which meant that your init code has been fired every time user changed language. That’s not an intended behavior, but admittedly a rare one. What’s worse is that we retained all the init code in memory.

Now, with mozL10n.once we can safely initialize code when l10n resources are available and free the memory right after that.

Uses of l10n API

On top of adding new features, we’ve been mostly busy investigating and improving how the default Firefox OS apps interact with localization API. That led to multiple design decisions including introduction of the described above mozL10n.once.

Once we got the new wrapper, we started analyzing bootstrapping code of each and every Gaia app and updating it to use the proper L10n API. Twenty two fixed bugs later, we’re done!

It’s incredible how much we were able to accomplish in just two short months. We feel much better right now about the bootstrap process and we have a clear picture on what we want to do next.

Use new navigator.languages API

Thank’s to Mounir’s work on navigator.languages API and implementation we were able to remove the only Mozilla specific API in mozL10n. That means that mozL10n should work in any modern browser now!

What we’re working on for 2.1

2.1 will be as ambitious for us as 2.0 was. The hashtag of the work is still #cleanup, but now it’s much more about modifying our API so that it’s more transparent to developers and requires less manual code in their apps.

entity.attributes become node attributes

First thing we were able to land is the transition away from assigning l10n entity attributes as node properties. We cleaned up the hacks that have been used and switched to entity attributes as node attributes.

DOM overlays

Next, we have one leftover from the previous change and that is an infamous innerHTML. We currently don’t have any clear way to inject localizable DOMFragment in Gaia. Fortunately, we have one that fits perfectly in L20n. It’s called DOM Overlays, and we’re working on getting them into mozL10n. That will allow us to further secure L10n API and remove any innerHTML calls.

Mutation Observers

Majority of localizability code in Gaia apps is related to localization of DOM nodes. With Mutation Observers we will be able to significantly reduce the amount of manual calls to mozL10n API and make majority of calls be just about settings data-l10n-id attribute with Mutation Observers doing the rest.

Not only will it reduce the use cases of mozL10n.translate and mozL10n.localize, but I expect to be able to cut by over 50% the number of manual mozL10n.get calls and manual operations that are currently used to set the result of that call into the node.

Mutation Observers will simplify Gaia code, reduce the amount of bugs related to language switching and get us closer to runtime l10n API that we want to offer for Gecko.

Bootstrap

There are still some interesting edge cases around how code boostrapping relies on particular pieces of environment. Does your code need DOM to be interactive? Does it need l10n resources to be loaded? Or maybe you need DOM to be localized? All those events happen asynchronously and we currently do not have a clean way to guard against any combination of those that your init code may require.

We’re working on a bootstrapping wrapper that will allow app developers to simply define which pieces of environment should their initialization be blocked by.

That will further secure the app boostrapping process and limit the risk of condition races.

mozDOMLocalized

One part of the bootstrap puzzle is how we fire event when certain bootstrap things happen. Right now we fire window.onlocalized event that means that mozL10n resources have been loaded, but doesn’t tell us anything about if document’s DOM has been localized (and is ready to be displayed).

With the work on the new event, we’ll be able to remove the global one, and settle on triggering on event on document and one on mozL10n object. Did I tell you that we’re still cleaning up? ­čÖé

Move mozL10n to document

We originally placed mozL10n object as a property on navigator object. Because our API is transitioning to be per-document, it becomes an inconsistency and an obstacle to keep mozL10n API on navigator object. It hardly fits the world with iframes, ShadowDOM and HTML Templates. We’re going to move it to document.mozL10n.

Remove inline l10n

One of the built time optimizations that we have in Gaia is called inline l10n. We store some portion of the l10n resources within each HTML file in order to localize the UI before it is displayed. It’s not very scalable, costs us memory and performance, but historically helped us prevent flashes of untranslated content. We hope to be able to remove this optimization in this cycle which will significantly simplify our internal code and give us some small memory and performance wins.

Is this L20n yet?

While we introduce L20n concepts into mozL10n, we’re still pretty far from being able to say that we support L20n API in Gaia. There’s a lot of work to do and it’s going to be a challenging work as we port L20n concepts to Gaia, and merge lessons we learned while working on Gaia into L20n spec and implementation. What we hope to end up with is a single codebase used in Gaia and offered to web developers.

It’s an exciting journey and I’m so happy to make Firefox OS’s localizability the most modern among all OSes!

Wymy┼Ťlanie Sieci na nowo

poni┼╝szy post jest t┼éumaczeniem postu Mitchell Baker “Reinventing the Web

W zaledwie 20 lat Sie─ç sta┼éa si─Ö jedn─ů z najwa┼╝niejszych infrastruktur wsp├│┼éczesnego ┼╝ycia. Dzi┼Ť dostrzegamy szans─Ö, by Sie─ç, przez kolejny skok, sta┼éa si─Ö jeszcze bardziej u┼╝yteczna, przyjemniejsza, by otworzy┼éa nowe szanse przed biznesem i mog┼éa lepiej s┼éu┼╝y─ç celom spo┼éecznym.

Wyobra┼║ sobie bogactwo i wolno┼Ťci Sieci p┼éynnie zintegrowane w urz─ůdzeniach mobilnych. Wyobra┼║ sobie, ┼╝e to wszystko dzia┼éa i to dzia┼éa niezale┼╝nie od tego czy u┼╝ywasz aplikacji do specjalnego celu czy przegl─ůdarki by odkry─ç swoj─ů w┼éasn─ů drog─Ö.

Wyobra┼║ sobie, ┼╝e pobierasz aplikacj─Ö i ona dzia┼éa na wielu urz─ůdzeniach, niezale┼╝nie od jakich producent├│w pochodz─ů. Wyobra┼║ sobie, ┼╝e mo┼╝esz wybra─ç czy chcesz kontaktowa─ç si─Ö z dostawc─ů aplikacji bezpo┼Ťrednio czy przez Apple, Google albo Microsoft.

Wyobra┼║ sobie, ┼╝e masz wyb├│r tego kto kontroluje Twoj─ů to┼╝samo┼Ť─ç. Logujesz si─Ö na stron─Ö i wszystko co zrobi┼ée┼Ť w sieci staje si─Ö dost─Öpne dla ┼Ťwiata? A mo┼╝e logujesz si─Ö i tylko to co Ty uznasz za publiczne staje si─Ö dost─Öpne dla ┼Ťwiata?

Wyobra┼║ sobie poczucie bezpiecze┼ästwa kiedy jeste┼Ť online – kto ci─Ö ┼Ťledzi, kto sprzedaje informacje o tobie, kto ci─Ö chroni i jak mo┼╝esz ochroni─ç siebie sam.

Wyobra┼║ sobie, ┼╝e rozumiesz Sie─ç, poczucie kompetencji by robi─ç rzeczy, by zmienia─ç te kt├│re istniej─ů i tworzy─ç kolejne, kt├│re spe┼éniaj─ů twoje potrzeby. Wyobra┼║ sobie, ┼╝e uczenie si─Ö Sieci jest fajne i wykracza poza ┼Ťwiat programist├│w obejmuj─ůc zwyk┼éych ludzi.

Mozilla tworzy ten ┼Ťwiat. Mamy jego wizj─Ö, architektur─Ö, technologie i plany produkt├│w. Te produkty tworzymy w┼éa┼Ťnie teraz. Mamy zasoby finansowe, by zrealizowa─ç te cele. To ekscytuj─ůcy i bardzo produktywny okres. Sp├│jrz w nasz raport roczny, aby zobaczy─ç co zrobili┼Ťmy i co jest na horyzoncie. Do┼é─ůcz do nas, by razem budowa─ç ten ┼Ťwiat.

L20n command-line toolchain

While we do hope that most of the work around l20n installations will have visual, GUI web tools, we also recognize that it’s important to provide simple, robust command-line tools that will cover the entire workflow.

For the l20n workflow to happen, we want to have a complete command-line toolchain that will automate common tasks required to operate and maintain localizations.

Behind this toolchain we provide a powerful API that localization and release tools will use, but the first consumer of those API will be the command-line based environment for l20n.

For this toolchain, we have three personas in mind:

  • a localizer
  • release driver
  • developer

Localizer

Localizer should be able to start a new localization, stay informed about the status/completeness of his localization and have easy way to learn about updates. On top of that, he should be aided in navigating through the changes like new, modified or removed entities, get help in maintaining his localization between releases and work with the build team on marking which localization version is ready to be shipped.

Initially we want to provide four elements that will enable this:

  • on demand activated shell prompt customization that will provide contextual information while the localizer navigates around the l10n project
  • bootstrap command that will help initialize a new localization sandbox for a project
  • set of diagnosis tools like validator, completness check and localization status, all of which will analyze given localization and provide detailed information or what remains for the localization to be completed
  • an automated update tool that will help localizer keep the localization structure in shape, helping adding new strings, removing obsolete and provide information on the updated ones

The latter two concepts are powerful, and we have to be very careful when we design them. Each operation may harm the system, since we can easily loose the ability to review changes made by localizers in VCS history view if we apply changes that the localizer did not work on, or we can hide from the localizer the contextual information by removing entities that became obsolete, but their value has been reused in a new entity added just next to the old one.

All this kind of stuff has to be taken into account and we want to aim for a balance between simplifying day-to-day localization work, and maintaining VCS friendliness and sense of control that a localizer must have over his localization project.

Release driver

As with a localizer, we want to provide methods to automate common tasks, like selecting locales for a release, learning about status and progress of multiple locales at once, accepting new locales from localizers and batch updating locales with changes.

In order to achieve that, we want to provide:

  • a custom shell prompt that will aid release driver while he navigates between localization projects
  • scripts that will make it easy to analyze the status of localizations and validate them
  • tools that will allow release driver to maintain the list of locales and their version, to be released with a given version of the product
  • a script that will take a small “l10n patch” like a number of new entities, or value changes, and apply it across multiple locales

Developer

For the developer, we want to make sure that it’s easy to make his code localizable and maintain it that way.

This means that we want to make it easy to add new entities manually and verify that they work with tests, that we can extract entities from code where possible (like – XML), and it’s easy to spot hardcoded strings.

For this, we plan to provide the following tools:

  • a validator that outputs unit-test readable outcome statuses, so that unit-tests can be written against localizable code
  • an extractor for simple extracting and maintenance. The extractor should be able to help developer clean l10n files by removing strings that are no longer used in the code.
  • a tool that generates dummy test locale which enables the developer to visually find hardcoded strings

We want to make the whole environment project independent by allowing each project to define how process like bootstrapping, extracting should exactly work, where the l10n resources are stored and others.

The first set of tools will be aimed to help localizers, then we’ll work on tools for release managers and developers will come after that.

We’re in the process of writing API’s and designing the syntax for those commands. You can find the document with a proposal and the current source code here.