Multilingual Gecko Status Update 2018.2

Welcome to the third edition of Multilingual Gecko Status Update!

In the previous update we covered the work which landed in Firefox 59 and Firefox 60.

At the time, we’ve been finalizing the platform work to support Fluent localization system, and we were in the middle of migration of the first Firefox UI component – Preferences – to it.

Today, we’ll pick up right where we left off!

Firefox 61 (June)

Firefox 61 fits into the trend of things calming down in Intl – and it’s a great news! It means that we are reaching platform maturity after all the groundbreaking refactors in 2017, and we all can focus on the work on top of the modules, rather than playing whack-a-mole fixing bugs and adding missing features.

The biggest platform change is really just an update to ICU 61 which Andre landed in March聽and my work on adding mozIntl.RelativeTimeFormat and mozIntl.getLocaleDisplayNames.

The former gave us a stable unified API for presenting relative time (such as “In 5 minutes” or “10 days ago”) while the latter unified how we present language, region and combinations of those in our user interface based on the Unicode CLDR representation (example: “English (United States)”).

As I explained in my earliest posts, one of the things I’m particularly proud of is that we go the extra mile to use every such opportunity to not only fix the immediate Firefox UI need, but also push such proposals for standardization and in result make the Web Platform more complete.

In this case, Intl.RelativeTimeFormat has been proposed and thanks to amazing work by Daniel Ehrenberg is now in Stage 3 and soon will be exposed to all web developers in all browsers! Intl.getLocaleDisplayNames is less mature but the work on it just picked up.

Firefox migration to Fluent reached its next milestone moving from just a couple messages to over 100 Fluent messages in Firefox!

Notable changes [my work] [intl module]:

Firefox 62 (September)

Another calm cycle! The biggest feature was the introduction of the pseudolocalization in Firefox which I blogged about, and landing of the developer documentation.

The documentation has been insanely useful in distributing knowledge and helping engineers feel more comfortable working with the new stack, and I’m very happy we reached a stage where landing a documentation is the big news 馃檪

In the Fluent land we spent the March-May timeframe moving from a 100 messages to 500 Fluent messages in Firefox!

Notable changes [my work] [intl module]:

Firefox 63 (October)

This cycle was quite similar to the previous one, with a bulk of work going into regular maintenance and cleanups.

For my work, the trend is to start integrating Fluent deeper into Gecko with more work around L10nRegistry v1 limitations and getting deeper DOM integration to improve XUL performance.

In this cycle I landed a XPCOM mozIDOMLocalization API which allows us to call Fluent from C++ and was required for a bigger change that we landed in Firefox 64.

One new theme is Kris Maglione who started working on reducing the performance and memory overhead coming from the old StringBundle API (used for .properties). With all the new work centralized around Fluent, but with a large portion of our strings still using StringBundle, it becomes a great target for optimizations by cutting out everything we don’t use and now we know – we never will.

Notable changes [my work] [intl module]:

Firefox 64 (December)

This release, is still getting stabilized and will get released in December, but the work cycle on it happened between September and October, so I can already provide you an account of that work!

Besides of a regular stack of cleanups coming from Henri and me, we’ve seen Ehsan Akhgari taking over from Kris to remove more lines of unused code.

The really big change was the introduction of DocumentL10n API which is a C++ API with its own WebIDL tightly integrated into the very core of DOM module in Gecko – nsIDocument.

Before that Fluent lived in Gecko in some form of a glorified javascript library. While it is Fluent’s goal to target the web platform, Firefox UI is inherently different from the web content and benefits from better integration between the DOM and its localization component.

This change allowed us to better integrate localization into the document’s life cycle, but what’s even more important, it allowed us to expose Fluent to documents that usually do not have special privileges and could not access Fluent before.

As for migration, we moved along nicely bumping from 500 to around 800 messages thanks to hard work of a number of students mentored by Jared Wein and Gijs Kruitbosch. The students picked up work on the migration as their Capstone project.

Notable changes [my work] [intl module]:

Summary

2018 has been much “easier” for the intl module than 2017 was. It’s great to see the how all pieces fit together and for me personally, it enabled me to focus on getting Fluent better integrated into Gecko.

There’s still a lot of work but it now is fully focused on Fluent and localization, while our intl module as a whole goes through a more well earned peaceful period.

Between now and the next status update, I hope to publish a summary post about the last two years of work. Stay tuned!

Multilingual Gecko Status Update 2018.1

As promised in my previous post, I’d like to do a better job at delivering status updates on Internationalization and Localization technologies at Gecko at shorter intervals than once per year.

In the previous post we covered recent history up to Firefox 58 which got released in January 2018. Since then we finished and shipped Firefox 59 and also finished all major work on Firefox 60, so this post will cover the two.

Firefox 59 (March)

Firefox 58 shipped with a single string localized using Fluent. In 59 we made the next step and migrated 5 strings from an old localization system to use Fluent. This allowed us to test all of our migration code to ensure that as we port Firefox to the new API we preserve all the hard work of hundreds of localizers.

In 59 we also made several improvements to performance of how Fluent operates, all of that while waiting for the stylo-chrome to land in Firefox 60.

In LocaleService, we made another important switch. We replaced old general.useragent.locale and intl.locale.matchOS prefs with a single new pref intl.locale.requested.

This change accomplished several goals:

  • The name represents the function better. Previously it was pretty confusing for people as to why Gecko doesn’t react immediately when they set the pref. Now it is more clear that this is just a requested locale, and there’s some language negotiation that, depending on available locales, will switch to it or not.
  • The new pref is optional. Since by default it matches the defaultLocale, we can now skip it and just treat its non-presence as the default mode in which we follow the default locale. That allowed us to remove some code.
  • The new pref allows us to store locale lists. The new pref is meant to store a comma separated list of requested locales like "fr-CA, fr-FR, en-US", in line with our model of handling locale lists, rather than single locales.
  • If the pref is defined and the value is empty, we’ll look into OS for the locale to use, making it a replacement for the matchOS pref.

This is important particularly because it took us very long time to unify all uses of the pref, remove it from all around the code and finally be able to switch to the new one which should serve us much better.

Next come a usual set of updates, including update to ICU 60 by Andre, and cleanups by Makoto Kato – we’re riding the wave of removing old APIs and unifying code around ICU and encoding_rs.

Lastly, as we start looking more at aligning our language resources with CLDR, Francesco started sorting out our plural rules differences and language and region names. This is the first step on the path to upstream our work to CLDR and reuse it in place of our custom data.

Notable changes [my work] [intl]:

Firefox 60 (May)

Although Firefox 60 has not yet been released as of today, the major work cycle on it has finished, and it is currently in the beta channel for stabilization.

In it, we’ve completed another milestone for Fluent migrating not just a couple, but over 100 strings in Firefox Preferences from the old API. This marks the first release where Fluent is actually used to localize a visible portion of Firefox UI!

As part of that work, we pushed our first significant update of Fluent in Gecko, and landed a special chrome-only API to get Fluent’s performance on par with the old system.

With an increase in the use of Fluent, we also covered it with Mozilla eslint rules, improved missing strings reporting, and wrote an Introduction to Fluent for Firefox Developers.

On the Locale Management side, we separated out mozilla::intl::Locale class and further aligned it with BCP47.

But the big change here is the switch of the source of available locales from the old ChromeRegistry to L10nRegistry.

This is the last symbolic hand-over from the old model to the new, meaning that from that moment the locales registered in L10nRegistry will be used to negotiate language selection for Firefox, and ChromeRegistry becomes a regular customer rather than a provider of the language selection.

We’re very close to finalize the LocaleService model after over a year of refactoring Gecko!

Regular healthy number of cleanups happened as well. Henri switched more code to use encoding_rs, and updated encoding_rs to 0.7.2, Jeff Walden performed a major cleanup of our SpiderMonkey Intl source code, Andre added caches for many Intl APIs to speed them up and Axel updated compare-locales to 2.7,

We also encountered two interesting bugs – Andre dove deep into ICU to fix `Intl.NumberFormat` breaking on roundings in Persian, and I had to disable some of our bidirectionality features in Fluent due to a bug in Windows API.

Notable changes [my work] [intl]:

Summary

With all that work in, we’re slowly settling down the new APIs and finalizing the cleanups and the bulk of work now is going directly into switching away from DTD and .properties to Fluent.

As Firefox 60 is getting ready for its stable release, we’re accelerating the migration of Preferences to Fluent hoping to accomplish it in 61 or 62 release. Once that is completed, we’ll evaluate the experience and make recommendations for the rest of Firefox.

Stay tuned!

Multilingual Gecko in 2017

The outline

In January 2017, we set the course to get a new localization framework named Fluent into Firefox.

Below is a story of the work performed on the Firefox engine – Gecko – over the last year to make Fluent in Firefox possible. This has been a collaborative effort involving a lot of people from different teams. It’s impossible to document all the work, so keep in mind that the following is just the story of the Gecko refactor, while many other critical pieces were being tackled outside of that range.

Also, the nature of the project does make the following blog post long, text heavy and light on pictures. I apologize for that and hope that the value of the content will offset this inconvenience and make it worth reading.

Continue reading “Multilingual Gecko in 2017”

Du偶o si臋 dzieje

Zbli偶aj膮ce si臋 wydanie Firefoksa 3.1 jest najciekawszym w mojej historii… Troszeczk臋 przypomina wydanie Fx 1.5. Stoimy na stabilnej, szybkiej nowoczesnej platformie jak膮 jest Gecko 1.9 i mo偶emy si臋 skupi膰 na tych drobnych detalach i mi艂ych dodatkach, kt贸re sprawiaj膮, 偶e korzystanie z sieci jest przyjemne.

Zatem w por贸wnaniu z ogromem zmian jakie wprowadzi艂 Fx 3.0, wersja 3.1 b臋dzie zmian膮 znacznie bardziej ewolucyjn膮 i spokojn膮, co nie znaczy, 偶e niepotrzebn膮 馃檪

Prawid艂owa obs艂uga profili kolor贸w ICC, media queries, d艂ugo oczekiwany text-shadow,聽 niezwykle wa偶na dla przysz艂o艣ci otwartego Internetu obs艂uga znacznik贸w <audio/> i <video/>, querySelector,聽 ustandaryzowany Drag&Drop, API do geolokacji, przeci膮ganie kart pomi臋dzy oknami, autotagowanie zak艂adek, znacz膮ce przyspieszenie aplikacji webowych i samego Firefoksa dzi臋ki Tracemonkey… a to tylko czubek g贸ry lodowej, zapraszam do poczytania Fx3.1 dla programist贸w, i Fx 3.1 draft plan.

Cz臋艣膰 to usprawienienia widoczne go艂ym okiem, inne dadz膮 nowe mo偶liwo艣ci autorom stron i autorom rozszerze艅, wszystkie s膮 “spokojne”. Nie wiem jak to inaczej opisa膰, ale nie ma tu wielkich, ryzykownych zmian (mo偶e poza API do pobierania zdalnych czcionek), du偶ych przeobra偶e艅 interfejsu czy ogromnych przetasowa艅 w g艂贸wnych elementach silnika.

Dwa ciekawe elementy, kt贸re spokojnie ewoluuj膮 w Fx3.1 to obs艂uga standard贸w i dodatkowe interfejsy do zarz膮dzania sesjami i prywatno艣ci膮 sesji. I to z tym zwi膮zane s膮 dwa poni偶sze screeny:

Session restore updated
Session restore updated
Acid 3 w fx3.1 nightly
Acid 3 w fx3.1 nightly

Pierwszy pokazuje nowe mo偶liwo艣ci podczas przywracania sesji, drugi to aktualizacja stanu ACID3 w Firefoksie. Te 93% to to co ju偶 wyl膮dowa艂o w g艂贸wnym repozytorium, bo istniej膮 te偶 patche poprawiaj膮ce wszystkie pozosta艂e b艂臋dy, ale nie zosta艂y jeszcze w艂膮czone do g艂贸wnej ga艂臋zi. Z tego co pisa艂 Dbaron, prawdopodobnie na wydanie Fx3.1 uda si臋 podnie艣膰 to do 98%.

Czemu nie 100%? Nie ma powodu do po艣piechu. Acid3, jak ju偶 wielokrotnie pisa艂o wiele os贸b, jest testem do艣膰 “abstrakcyjnym” w tym sensie, 偶e nie testuje najpopularniejszych element贸w standard贸w, tylko w艂a艣nie te, kt贸re nie dzia艂aj膮 np. w Fx. Spe艂nienie wymaga艅 testu ma sens tylko w贸wczas gdy jest elementem prac nad og贸lnym poprawieniem obs艂ugi standard贸w, a nie gdy jest sztucznym pompowaniem zmian, aby tylko zapewni膰 sobie “setk臋” w jakim艣 testowym buildzie, kt贸ry nigdy nie zostanie wydany i wywala si臋 na wszystkim poza testem Acid 3.

To oczywi艣cie nie wszystko, uwa偶ni obsewatorzy planety Mozilli z pewno艣ci膮 zauwa偶yli prac臋 ekipy od Ux nad poprawkami u偶yteczno艣ci, nie wspominaj膮c ju偶 o setkach poprawionych b艂臋d贸w.

W tym samym czasie grupa z labs pracuje nad nowymi wersjami Ubiquity, Weave czy Prism, Mozilla Messaging eksperymentuje z UI i pracuje nad wydaniem Thunderbirda 3, a zesp贸艂 z Mozilla Mobile szykuje si臋 na wydanie Fenneca 1.0.

Ach, zapomnia艂bym, 偶e tak偶e teraz, inna grupa, pod wodz膮 Brendana Eicha, pracuje nad Mozill膮 2 – now膮 wersj膮 platformy, kt贸ra wprowadzi naprawde ogromne zmiany w funkcjonowanie Gecko, a w艣r贸d zakresu prac Gecko 2.0 pojawiaj膮 si臋 takie rzeczy jak wsparcie dla animacji 3D – SVG, Canvas 3D, nowy standard lokalizacji – L20n, instalacja rozszerze艅 bez restartu, Compositor – nowy system prezentacji uk艂adu strony i wiele innych…

Je艣li jednak tego jest Ci ma艂o, masz poczucie niedosytu, albo uznajesz, 偶e Mozilla to co艣 wi臋cej ni偶 grupa programist贸w i ich kod, to… masz absolutn膮 racj臋 Mozilla Foundation oraz Mozilla Europe pracuj膮 nad projektem o nazwie “2010 Goals“, kt贸ry ma za zadanie zdefiniowa膰 rol臋 Mozilli jako aktywatora i uczestnika dyskusji na temat rozwoju Internetu w takich dziedzinach jak edukacja, dost臋p w najgorzej zinformatyzowanych regionach 艣wiata, czy otwieranie rynku mobilnego tak jak zrobili艣my to z WWW dzi臋ki Firefoksowi.

Je艣li masz ochot臋 co艣 porobi膰, rozwija膰 si臋 pomagaj膮c w realizacji naszej misji, to otwieramy w艂a艣nie dla Ciebie nowy portal – contribute.mozilla.org, kt贸ry ma na celu pom贸c Ci odnale藕膰 si臋 na pocz膮tku swojej przygody z Mozill膮. Du偶o si臋 dzieje i jest mn贸stwo miejsca dla Ciebie 馃檪

not-invented-here syndrome in Mozilla

(follow up to lilmatt’s blog post)

I want to write a bit on what Ian McKellar, my old fellow from Flock, and now a proud member of the Songbird team, called Not Invented Here attitude.

Actually, NIH syndrom is well known and described in the memoirs of Wikipedia. It’s a persistent sociological culture that prevents the organization from using existing knowledge, code or research because it has different origins.

While this topic has been widely discussed all around the globe, and even in open source world, it has never been a topic in a Mozilla ecosystem debate.

Pity, since there is an elephant in the room, I can swear.

Continue reading “not-invented-here syndrome in Mozilla”

ZipWriter hits trunk

聽Woho! Bug 379633 is fixed!

ZipWriter just has been included into trunk.

You may say it’s a small thing. Who cares? Opera, IE, Safari lives without it and you can rarely see users complaining that their browser lacks scriptable zipwriter, right?

Hah! Mozilla is a platform, not “a browser”. It *does* influence any effort towards Xul editors, Mozilla IDE’s and… L10n tool 馃槈