I’m currently working on Silme 0.9 which is shaping up to be a release where we packed up all the lessons we learned while working on the project for the last year and a half. It’s going to be the last release with non-frozen API before 1.0 so we have to be sure we’re happy with 0.9 before we release it.

Then, we’ll have Silme 1.0 which is a goo moment to revisit the name of the library. I know native english speakers may find it a good idea

Sam mam niestety mało czasu (tak, tak, sesja…), ale spróbuję w telegraficznym skrócie, w losowej kolejności, opisać kilka tematów dla zainteresowanych:

• Wydanie Firefoksa 3.5 zbliża się wielkimi krokami. Tym razem, w odróżnieniu od naszej tradycji, chcielibyśmy, aby wydanie RC faktycznie było kandydatem na wersje finalną. Koniec z wydawaniem RC1 i planowaniem trzech następnych równocześnie… Jak coś jest kandydatem na wydanie, to ma być gotowe. Dlatego Mike (Beltzner) i Mike (Shaver) bardzo poważnie uparli się, że nie będzie RC2. No, chyba, że niebo spadnie nam na głowy.
• Nowy addons.mozilla.org. A w nim – łatwiejsza nawigacja, wygodniejsze zarządzanie i nowa funkcja – Kolekcje – która umożliwia tworzenie własnych zestawów rozszerzeń i motywów. W przyszłości mamy nadzieję zaoferować Ci możliwość spakowania takiej kolekcji i wydania jako własnego Firefoksa.
• Projekt hack.mozilla.org – przez 35 dni zostanie zaprezentowanych 35 nowych funkcji dla webmasterów, które pojawią się w przeglądarkach nowej generacji – Firefoksie 3.5, następnej Operze, Safari i Chrome.
• Thunderbird 3.0beta3 zbliża się. Została jedna duża rzecz (globalne wyszukiwanie), która powinna wylądować w ciągu tygodnia i potem już przygotowania do wydania.
• Trwają prace nad Firefoksem 3.0.12. Tradycyjnie, kilka poprawek wydajności, kilka bezpieczeństwa i kilka błędów zwykłych.
• Zespół Aviary.pl przygotował polskie tłumaczenie Poradnika Recenzenta do Firefoksa 3.5. Będzie to mała książeczka z opisem nowych technologii i funkcji jakie będą dostępne dla autorów stron WWW i użytkowników wraz z nowym Firefoksem.
• Zaczęły się przygotowania do planowanego na wrzesień Tygodnia Mozilli (po angielsku – Mozilla Service Week, ale niestety brzydko brzmi tłumaczenie dosłowne). Będzie to kampania trochę podobna do banków czasu – chcemy zachęcić ludzi posiadających zdolności techniczne do pomagania osobom mniej uzdolnionym i potrzebującym pomocy. W ramach tej kampanii będziemy szukać chętnych do prowadzenia wykładów na temat obsługi Internetu, osoby chcące pomóc z konfiguracją Internetu, lub w inny sposób pomóc bliźniemu.
• Pierwsze prace nad Firefox.Next – tutaj zidentyfikowaliśmy 35 błędów, których rozwiązanie sprawi, że Firefox będzie reagował szybciej przez co poprawi się wrażenie z interakcji z przeglądarką. Błędy takie oznaczamy TSnap. W przyszłym tygodniu w trunku wyląduje Canvas3D, a zespół odpowiedzialny za layout zaczął prace nad XBL2. O innych planach na Firefox.Next można przeczytać na Wiki.
• Seamonkey 2.0b1 powinno wyjść w ciągu tygodnia.
• Microsoft wstępnie ogłosił, że zamierza wydać Windows 7 bez IE w Europie. Bardzo trudno okreslić, czy to jest właściwy ruch. Mitchell świetnie podsumowała (jeszcze przed tą decyzją) jak ten dylemat wygląda z naszej strony. Ja mam bardzo mieszane odczucia. Nie do końca rozumiem co Microsoft zdecydował tak naprawde (co dokładne oznacza “we will offer it separately”?) i mam poważne wątpliwości, czy wydawanie w 2009 roku systemu operacyjnego bez przeglądarki jest dobre dla użytkowników.
• JestPack to nowy projekt z Mozilla Labs będący kontynuacją idei projektu FUEL – budowania łatwych klocków z których można tworzyć rozszerzenia. JetPack pozwala tworzyć rozszerzenia korzystając wyłącznie ze znajomości technologii webowych – HTML, JS, CSS.
• Projekt Personas – rozszerzenia pozwalającego na łatwe zmienianie motywów Firefoksa bije rekordy popularności. Nigdy nie sądziłem, że coś może przebić popularność AdBlocka
• Projekt Weave rozwija Mozillowe podejście do “chmury” pozwalając synchronizować swoje dane z przeglądarki między komputerami, ale w sposób zaszyfrowany, tak, aby ten kto hostuje serwer Weave nie miał dostepu do Twoich danych. W ostatniej wersji pierwsze eksperymenty z OpenID.
• Bespin to z kolei edytor w “chmurze” – dostepny przez stronę WWW edytor kodu, który pozwala pracować nad projektem w grupie ludzi, a swoje projekty mieć dostępne z każdej maszyny bez potrzeby przygotowywania środowiska.
• Kampania FastestFirefox zbiera filmiki ludzi robiących to co umieją najszybciej. Chcemy zebrać je razem i stworzyć kompilację ludzkich pasji związanych z szybkością, ciągle pracujących by być jeszcze szybszym – tak jak społeczność pracująca nad Firefoksem pracuje, by każdy kolejny Firefox był szybszy.
• Wydaliśmy wersję 0.5 i 0.5.1 biblioteki do lokalizacji Silme. Adrian wraz z zespołem oraz Romi pracują teraz nad dwoma projektami korzystającymi z Silme. Projekt Adriana nazywa się Koala i będzie rozszerzeniem do lokalizacji do Komodo, a projekt Romiego to próba połączenia Silme z Narro.

Tak więc, dzieje się!

Staramy się jak najwięcej z tego przenieść do Polski, ale brakuje nam rąk do pracy. Jeśli jesteś zainteresowany/a pomocą, to szukamy na pewno chętnych do pomocy z:

• SUMO. support.mozilla.com to strona z artykułami do pomocy osobom używającym Firefoksa. GmbH (mwawoczny at aviary dot pl) chętnie przyjmie pomoc
• Eventy. Chcemy zorganizować kilka spotkań na temat technologii Web w Polsce. Szukamy osób chętnych do organizaci MozCampów, lub innej formy spotkań. Dzieje się dużo, jest o czym rozmawiać
• I jak zawsze, mamy wakaty dla chętnych do tłumaczenia. Aviary.pl szuka tłumaczy, bo projektów nam się zrobiło co nie miara. Pamiętaj jednak, zanim wyślesz zgłoszenie, tłumaczenia to odpowiedzialna praca. A my chcemy być w tym najlepsi.
• Z tego co wiem, nikt nie tłumaczy hacks.mozilla.org – jeśli ktoś jest zainteresowany, to z chęcią pomogę z organizacją tego

Silme 0.5.1 carries 5 changes, each of them making it a little better. In particular it fixes bugs that were blocking Adrian Kalla‘s version of compare-locales to get its first release.

So, what’s new in 0.5.1?

• New method: silme.diff.EntityList.has_entity() should make testing entities in diff faster
• Two bugs in silme.io.jar that made it impossible to use
• Performance improvements in silme.io.jar.get_package() (can we upload silme script to Fastest Firefox? ) by just ignoring to load zip entries for directories
• We don’t close ZIP file in get_package if we didn’t open it there. It was a bug that caused silme.io.jar to fail with subpackages
• Major rework of URI schema accepted in silme.io.jar. It is now possible to do:

  ioc = silme.io.Manager.get('jar')
  l10npack = ioc.get_package('jar:chrome/package.jar!locale/ab-CD/package')

This is the very first minor release in the history of Silme projects, and we’re happy to make it!

You can download it from HG repository or use pypi for silme0.5.1.

I’d like to thank Adrian who helped by identifying bugs and writing patches for this release. You may also want to check his newest project which uses Silme to create a sleek localization extension for Komodo! It’s called Koala

Next in line is Silme 0.7 which will bring new features and optimizations and we’re planning an alpha release to happen early next week. If you’re interested in hacking localization related code and never tried silme, it may be a good moment to give it a try and help us on the way to 0.7

It can’t get any simpler than:

easy_install silme

Also, if you missed it (I know you did), here comes the very first documentation. It comes out of doxygen (which is probably the best of the worst choices for python docs) but it works and it’s there for your convenience.

If you happen to be a master of doxygen and you know how to make it work better, contact me pls!

Release

So, here we are, Silme was asking for a release for long enough and I postponed it over and over so it’s time to make the cut. Today, I’m proud to announce the very first official release of Silme – python l10n library. Silme has been announced to mozilla.dev.l10n long time ago, and since then it has been continuously developed in a small, but quite interesting project structure with support from Adrian Kalla, Stefan Plewako, Ricardo Palomares, Staś Małolepszy and management guidance from Seth Bindernagel.

It’s very, very hard to explain Silme concept to those who never tried to work on localization development.

Let me try: It’s like a DOM API for localization.

Works? Probably not… Well. Let me try the descriptive way. Silme is a toolset for a developer who wants to work on localization tools. It can read localization files, it can write them, it can modify them, it can search through them, it can process them, merge, split, localize and help you get some statistics out of the localization files. It probably can juggle them, although support for this is rather experimental.

Standing on the shoulders…

Silme is definitely not the first library of that kind and has been created basing on lessons from TranslationToolkit masters, who for years lead the development of most widely used l10n dev library in the world. TT is not the only one. In fact, anyone who created a tool related to localization had to create a library for it but it was probably limited to on-demand needs, and was treated without enough love and respect from its masters.

The reason is that number of people working on localization tools is… let’s put it that way. The number could not qualify for the big number competition. Even for a descend number in fact. And those who do work on the tools, are really extremely focused on getting the output here and now, and everything that has to happen on the way from PO or DTD file to the moment when you can see a window with a textbox asking you to translate the string is less important than the actual UI and workflow. Thanks to this approach we at least have some tools that we can offer to localizers and make their life happier.

The one particular example where the situation is different is the world of gettext, with its great tools like KBabel (now Lokalize), Pootle, PoEdit and swarm of little helpful scripts around Translation Toolkit project.

Format equality

And now, Silme tries to replicate this success on the cross-format level, giving some sense of self-respect to the non-gettext formats like DTD and Properties. In fact, Silme is intended to be format-neutral with support for as many formats as possible. Gettext, DTD, XLIFF, TS/QM (Nokia QT) and others. It also can read entities directly from SQLite or MySQL and we’re experimenting with grabbing entities directly from an HTML file. We hope that this will open an easier way for developers around the world to craft their own tools, or new projects to focus on the actual tool, not having to reinvent the way to parse entities or compare them.

More that that. We hope that new localization formats and languages will have an easier start being just able to reuse everything that Silme has to offer and tie it with their vision and potential. Yes, L20n may be that language.

Some numbers

Silme 0.5 is a result of countless hours of testing, coding, debugging, making mistakes and fixing them, analyzing performance, and introducing bottle necks only to spend another couple days trying to remove them. Adrian, in particular, spent significant amount of time sending me testcases and letting me know that I broke everything with my shiny, new API, but all in all what we have here is a pretty neat piece of code, if you ask me.

What you see on the right is a cute result of hg activity script  run on all 385 commits to our HG repo. Not sure if its self-explaining so left me give you a hint. 35 commits per day is a big number. (and our module owners are not in the race, they have some godlike powers for sure).

In terms of LOC, the stats are as follow:

-------------------------------------------------------------------------------
Language          files     blank   comment      code    scale   3rd gen. equiv
-------------------------------------------------------------------------------
Silme library        49       580       413      3784 x   4.20 =       15892.80
With scripts etc.    78       982       529      5574 x   4.20 =       23410.80
-------------------------------------------------------------------------------
(data from CLOC)

Done?

Ha! Of course not! Silme is just at the beginning of its journey and although we already can see first projects investigating potential use of it, we have a long way ahead till we can mark stable 1.0 release, and claim success.

In particular, on the Wiki page you can read about our plans for the next release – Silme 0.7. You can also find basic tutorial (work in progress) and a few example code pieces that you can use to figure out some bolts and nits used in here.

As this post gets to its closing, and your excitement about the potential uncovered by the tales of python libraries, localization developers and mysterious land of happy i18n, l10n, l12y and other abbreviations where all strings fit the space, all words have perfect translations, declensions and countless plural forms are well handled and…

Umm… where was I… ah! yea. So… it’s time for me to close this gathering and let you surf ahead on your Internet waves. And if you happen to find this project interesting, don’t hesitate to step by our newborn IRC channel – #silme on irc.mozilla.org and cheer the brave men and women who work there.

From now on, you can expect short incremental updates on the way to Silme 0.7.

Expedia screenshot

wondering if that gives me the “longest trip home” award from EuMozCamp crew. William, beer will do this.

The event was great, I’ve been at so many mozilla events during last 8 years yet beyond “normal” awesomeness of those events, once againt some elements of what happened were a big surprise (Mix group – you know who you are)

For now I’m attaching Silme talk slides, for those who couldn’t make it to the talk. (it’s also the very best documentation we have so far):

• PDF
• ODP
• Keynote

It requires tengwar-gandalf font to display some elements, but is pretty readable without it.

If you have any other questions regarding silme, jump in on #l10n channel, silme folks are there.

Right now I’m working on logging system and then PEP08, API freeze, documentation and then release!

Beside of that I spent some time during Firefox Summit on talking with people of Pootle/TranslationToolkit fame to identify potential problems that they faced. It was extremely supportive for me and gave my major take away is that if I want to reach my goal of having one, common abstraction layer for l10n objects I have to merge two very different concepts – single locale files (like DTD, properties, ini) and multi-locale files (XLIFF, GetText, tc).

Multilocale branch has been ignited to address this. I already did several tests and it seems that I will be able to support both models in one API without making both feel like hacks.

Pike raised another concern, that I tried to keep for later which is a concept of entity/l10nObject processing. Initially I assumed that it’s a minor topic, and on this level of abstraction I assumed that leaving the entity values unprocessed is OK for now. Unfortunately, especially with L20n being the next Big Thing for Silme, entity processing becomes very important and has to become a legimate element of the library skeleton.

I started early hacks of l20n.py format parser leaving my brain in free conceptual thinking mode and waiting for Pike’s time to talk about grammar inconsistencies.

Last big thing to come is a soon to happen switch from svn repo on my server, to shiny new hg.mozilla.org. This requires me to spend a few hours on svn-to-hg migration tools, but should help with later branching and support easier collaboration between many developers.

Current roadmap is pretty dense for stage3, and may be latter splitted, but does not currently involve work on end-user oriented apps like a webtool. Once I have this two major restructurizations ready (multilocale/pre-processing), I’ll get back to providing proof-of-concept tools. We’re of course looking for more developers so let me know if you’re interested

Mitchell Baker speaking

Last half a year was amazing for me. I joined Mozilla Corp., reignited my participation in Mozilla project, worked on several exciting projects and it’s an amazing coincident that this half a year is almost over exactly now, when I’m sitting at Whistler, with 400 creative, innovative people and we’re all listening to Mitchell Baker speaking about our open source roots, and the trunk of the project being about “Human interaction with the Internet”… It’s thrilling.

There are several projects that I’m trying to launch related to Mozilla, I will be speaking about them during the Firefox Summit, and I will blog about it once the summit people share their feedback on it.

One of the elements of Mozilla ecosystem that has been super exciting for me is localization and internationalization. And the great news is that there’s a lot happening around L10n in Mozilla these days. We’re improving the l10n build system, l10n processes, but there’s one very visibly missing area in L10n land – tools. We finally have a project that is going to target this – Verbatim. Verbatim is a project aiming for a webtool that will allow localizers focus on the translation instead of having to spend time trying to figure out how to do the actual translation. If you’re in Whistler, we have a presentation on Verbatim today

But that’s not all! There is one project that was something I’ve been coding over weekends and while at the airports and on flights and in many other places. It’s inspiring and challenging for me enough to keep working on it over evenings, nights, sundays, and holidays which I have to apologize for to my girlfriend

Yesterday, I tagged stage2 of the project which means it’s more or less ready to go public and be reviewed by you all, and has a chance to explain itself to the level that may attract others to join me and participate in it.

The project name is Silme, and it is a python localization library that has been structured in a multi-abstract level model. Thanks to it, the library works natively with DTD, Properties, GetText, can work with XLIFF, L20n, ini files and any other l10n data format. Beside, it can work with files stored on drive, in SVN, in CVS, in MySQL, SQLite and virtually any other data source.

On the other hand it’s extensible and flexible enough to work in web app, command line tools, or GUI tools. It’s not only for Mozilla, as any other project can build apps on top of this library. Be it Songbird, Miro, Seamonkey, Firefox, Thunderbird, Addons.mozilla.org, Flock, GNOME, KDE, etc., etc.

If you find above description interesting, please read the original announcement and join the project. It’s totally open and very alpha. If I try to express where I am with it, I think I just passed the mark when the code self explains the idea. Nothing more.

Beside, it’s a lib. Unless people like to localize directly in python env, we need apps on top of it. And it’s at least as challenging as a library itself, but if you want to write a localization app, I think it’ll be easier if you will be able to focus on UI and features of the app, and minimize the work on sole entity operations that Silme does cover.

So… here we are. I’m confident I will keep working on it, and some small simple apps will be created for my own use, but it’s very, very far from being useful, and need a lot of work to get there. Ultimate goal will be to work smoothly with an exciting project that is being mindcrafted in Mozilla these days – L20n.

Please, remember that it’s a hobby library for now, a product of several years of work as localizer, countless nights spent with my friends from Aviary.pl – Polish Localization Team, and the creative environment of Mozilla project. It’s up to you all what will happen next with it So mail me, or reach me on IRC if you want to discuss things about possibilities that Silme opens!