Day: January 12, 2011

Categories
main mozilla tech

Using Common Pool – Tutorial

In this post I’m going to show you how to make your extension localizable via Common Pool and how it can be localized.

early adopter warning: It’s the first attempt, it’s a new concept, things may brake, things may be a little rough, stay calm. It’s not a replacement for a well known solutions yet, but it may be one day. Play with it, hack the service and the library, suggest improvements, report the issues, but stay calm whatever happens – there may be dragons.

Using the library

Since Common Pool is currently implemented in Addon SDK, I assume you’re using it for writing your extension. First thing is to add my localization module to your extension. It’s available at http://builder.addons.mozilla.org.

Once you have it in your dependencies, you can just include it into your code.

Let’s take an example Hello World code:

var notifications = require("notifications");

var menuItem = contextMenu.Item({
  label: "Hello World",
  contentScript: 'on("click", function (node, data) {' +
    '  postMessage(node.src);' +
  '});',
  onMessage: function (imgSrc) {
    notifications.notify({
      title: "Hello World",
      text: "Welcome to the new, wonderful world!",
    });
  }
}

Now, in order to make it localizable, all we need to do is to add one require, and change the string initialization:

var notifications = require("notifications");
var _ = require("localization").get;

var menuItem = contextMenu.Item({
  label: _("Hello World"),
  contentScript: 'on("click", function (node, data) {' +
    '  postMessage(node.src);' +
  '});',
  onMessage: function (imgSrc) {
    notifications.notify({
      title: _("Hello World"),
      text: _("Welcome to the new, wonderful world!"),
    });
  }
}

Done! That’s what I meant when I wrote that Common Pool is asking as little attention from the developer as possible. If it reminds you of common patterns used by GetText, you’re right on spot. We like this notation, we reuse it.

With such code, you’re ready to go 🙂 As a developer, you can move along, do your work, deploy your extension on AMO and never ever bother about this again, while we will make sure that the localizers can localize, and users will receive the localized version dynamically. 🙂

Early adopter warning: For the time of the field testing of this library, we do not actually have any integration with AMO, so when you want your addon to be localized, you need to “cfx xpi” it, or if you’re using the builder just click on “Download” and upload the result xpi here: https://l10n.mozillalabs.com/projects/p/mozilla-jetpacks/upload/. This is a temporary step

Using the web service

Once a developer uploaded his addon and his job is done, the camera shifts onto another hero – a localizer. This brave soul, can just go to https://l10n.mozillalabs.com, log in and start localizing. As a localizer, you can just select your locale and start localizing. You can filter by an extension and you can view how other locales localized it.

You can also select overrides and add one for a given jetpack or occurrence. When you finish localizing your jetpack, you’re done. We will spread your translation to all users of the jetpack who use your locale. If you later update your translation, all users will receive an update on the next day. Isn’t that awesome? 🙂

Early adopter warning: Until we fix bug 10, we refer to jetpacks by their unique ID, instead of the human readable name. It should be fixed soon 🙂

The beauty of early adopting

Well, that being said, you know you would not be happy to receive a complete, ready to use, bug free solution. That’s not what hackers life is about. In fact, we decided to blow quite a big hole through the middle of the road that’s called “First Run Experience“.

As we were working on the library, and the webservice, and how nicely they’re tied, and we enjoyed the experience of a user staying up to date with localization of his extensions, and being able to download an extension in en-US and then see it localized on the day when a localizer finishes localizing it… we actually did not solve a pretty important case. Here’s the issue:

When a user installs an extension it contains no translation resources bundled with it. On the first run, we asynchronously ask the localization web service for the entities used by the extension. So far so good.

The server asynchronously sends the user all the entities and they’re cached. From that moment every request for a string will work by providing a translation of the string. But what happens before we cache?

Well, for now, nothing. Bummer 🙁 Basically, until we cache the strings, all we have is an empty cache and the localization library will just return the english strings which is the default fall back.

The bug for that is reported, bug 619807, and we’re working on a solution for that. We may decide to bundle entities that we have available for the extension together with it at the time of download, but that will be just a workaround. An ideal solution would be to pull the localizations together with the addon (as a part of addon installation experience) and then check if there are any localization updates on a normal daily routine.

If you have any ideas, suggestions or code snippets that solves it – feel free to jump in and help! 🙂

Real Life Example

Take a look at this example extension, Translate Selection with l10n. Now, go to l10n.mozillalabs.com to see its localization and localize it live.

Next time you run this extension you should be able to see it working with localization (assuming your browser is using your locale and minus the First Run Experience bug).

That’s all. 🙂

Summary

Common Pool is a simple, yet very powerful solution. It solves a lot of issues that we’ve been struggling with since the beginning of times. It makes internationalization easier than ever, localization easier than ever and synchronization easier than ever. Bare in mind, it’s intended as a solution for simple strings, for developers who don’t have much time to think through their UI’s and plan it for localization, and for massive localization of tons of thousands of extensions.

We believe it nails it perfectly (with the few early adopters hiccups that we’re working on right now, and an unlimited number of issues that are awaiting to be discovered by you!), and we want to improve it until we feel it to be the right one.

At the same time, please, remember that for a more complex and sophisticated UI’s, the ones where a developer has to think about entities, and the localizer wants to fine tune each and every sentence – we’re going to work on a complementary, L20n based solution.

Now, it’s your turn to play with it, hack on it, give us feedback and help us evaluate the solution. We’re available at irc.mozilla.org #l10n and #jetpack channels, and we use mozilla-labs-jetpack group that you can use. For bugs in the l10n web service, please, use its bitbucket issue reporting system. Thanks! 🙂

Categories
main mozilla tech

Jetpack Localization – Common Pool approach

Jetpack project is taking an aim at rewriting the extension system, learning from the past, responding to the feedback and thinking outside of the box. So when the Jetpack team approached L10n-Drivers and asked for support with building localization infrastructure for new addons, we decided to go the same way. 🙂

Common Pool

The first approach we’re presenting you is called Common Pool (yup, that’s not a very creative name) – it’s basic assumption is that most jetpacks are small applications with a minimal UI. And there are going to be tons of them. We already have tens of thousands of extensions, and Jetpack with its nice API and builder are going to make it much, much easier to write a good extension.

We also made some observations – There are common phrases, like “Cancel“, “Ignore“, “Log in” that are replicated over and over in almost every extension. It’s becoming hard to localize it, hard to keep it consistent and maintain the localization. On the other hand, extension authors are struggling to work with localizers as they need to iterate fast and  string freeze phases are adding a lot of cost to the release process.

For that reasons we decided to build on a basic concept of entities shared across extensions, stored on a server, and pulled by the jetpack itself.

Shared entities are powerful. For a localizer it means that we can show you the list of mostly used entities and if you localize 20 of them, you localized 50, 100 or maybe 500 extensions. You can update a translation and have it translated the same way across all extensions. You can localize only one extension and your work will probably benefit other extensions as well. You can localize today, and tomorrow all users will get updated localizations. How cool is that? 🙂

For a add-on developer it also means more good. You can pick entities that are already localized into many languages and have you extension Just Work. You don’t have to worry about releases and localization because if you only make thing localizable, users will be able to pull localizations as they appear. It means less to think about.

Now, there are of course some trade offs. One of them is that not all translations of a word should be equal. For that reason we propose a concept of an exception. Actually, two of them.

First one is a per-addon exception which allows localizers to localize a given string differently for this particular addon. It’s important when the addon uses some specific vocabulary.

Second one is a per-case exception which allows localizers to divert from the common translation for a specific use case. Say, the use case in jetpack X, file main.js line 43 should be translated differently.

Library and the Web Service

So, that’s the concept, and it has been implemented as a library. Yahoo! Now, for such a solution to work, we knew we need a partner in crime. We need a web service that will support Common Pool and allow for addon localization. We knew we will want to merge it with AMO and the Builder, so django was a preferred technology, and one other requirement was to be able to support a localization storage model that is substantially different from the industry standard.

Fortunately, at the time when we conceptualized Common Pool, our friends from Transifex were reworking their storage model away from gettext-specific to much more generic database driven one. Well then – good time to see how generic it can be, right?

Together with Transifex team, mainly Dimitris Glezos and Seraphim Mellos we’ve been working on the Common Pool web service that can back up the library and become a localization web tool. Today, the first iteration of the tool is alive at – https://l10n.mozillalabs.com! The source code is GPL and available at transifex-commonpool bitbucket repository and that’s where we’re collecting bugs as well.

What’s next?

Now we have to prove that the concept in the field. We’re challenging the status quo, so it’s on us to validate it. We’re going to reach out to extension authors and localizers alike to get support and testing coverage and if we that it works, we’ll include common pool library into Addon SDK 1.0. It’s up to you to help us evaluate it!

Then, it’ll be time to complement Common Pool with another architecture, that does not solve the simple cases that easily, and require much more active work from the developer, but will provide better level of details for both parties. The solution we want to offer is L20n.

Bundle pack ahead

Together, Common Pool and L20n should provide everything that localizers and developers may need pushing the limits and stretching what’s possible with the localization. We think of a Common Pool as a great generic, easy solution for most common strings, while L20n complements it with more sophisticated, more powerful approach for the developers whose extension reached the level when they want it to be localized with more attention and where the UI requirements are higher.

Let me know what you think – I know it’s early, I gave many talks about those technologies to our Mozilla communities, but I realize that it’s hard to evaluate without actual code in hand, so now is the time for feedback, feedback, and even more feedback 🙂

In the next blog post I’ll explain the steps to create your first common-pool driven extension.