Nightly builds for translation packages
One of the issues our translation team has been facing for a while, was the long delay between the moment at which translations of messages are recorded on Transifex and the moment at which they are available in a stable release of DoudouLinux. This issue was highlighted more than one year ago as a potential cause for motivation loss of our translators (now more than 200 people working on 50 languages!). One of the proposed solution was to automatically fetch translation updates from Transifex every night in order to build nightly updates of all the translations, using the standard Debian package system (“apt-get upgrade”). This solution is now active: new translations should be available to every one the day after they are recorded on Transifex :). Instructions can be found on our website.
To achieve this, we set up a dedicated repository for our nightly Debian packages, that we called “DoudouLinux volatile”. It provides 2 types of packages:
doudoulinux-l10n for applications,
doudoulinux-icons for icons on the desktop of the most advanced activities. Around 00:30 GMT every night, a script is launched on our build server. It will offer newer updates packages two hours later whenever a message has been detected as changed or new on Transifex. Of course this assumes we can get all the modified files from Transifex without being blocked by their anti-DOS attack system (which already happened during our tests). May this happen, we would have one day or more of additional delay before publishing translations updates.
How does it work? The server owns a local copy of several branches from our SVN code repository and is able to query Transifex servers using their Python client application (CLI client). The process of building package updates is the following:
- SVN files are re-synchronized with the HEAD version.
- If SVN updates cause translation files to be updated (PO/TS files), any previous local change in these files is first reverted.
- All the PO/TS files get updated from Transifex.
- A Python script determines if there are really changes in message translations. Insignificant changes are reverted.
- If there were real changes in message translations,
doudoulinux-l10npackages are rebuilt with a package name containing nightly and the date of the day.
- If translations of icons have been changed,
doudoulinux-iconspackages are rebuilt too.
- New packages are registered into a Debian repository structure using reprepro, with key signing.
- The repository is made available through an Apache web server.
The main build script is on our SVN server: server/build/l10n-updates. It relies on several other scripts that have been adapted or rewritten:
- lang/trunk/l10n-updates.sh fetches new messages from Transifex.
- lang/trunk/podiff.py checks that fetched files really have new or modified translation messages.
- packages/trunk/l10n/make-l10n.sh prepares the files required for
- packages/trunk/build-debs.sh builds Debian packages.
- packages/trunk/register-debs.sh registers packages into the repository.
- packages/trunk/upload.sh signs and uploads the package repository. In this case we just sign files.