Internationalization and Localization of Google App Engine Python Webapp Using webapp2 and Jinja2

This post shows how to use webapp2 web framework and Jinja2 templating system to achieve i18n of Google App Engine Python webapp with Babel and pytz (or pytz-appengine, gae-pytz).

Install Babel

# install babel
$ sudo pip install babel

# download babel 1.3
$ wget https://github.com/mitsuhiko/babel/archive/1.3.zip

# unzip and enter babel source directory
$ unzip 1.3.zip
$ cd babel-1.3/

# create babel.zip for zipimport
$ zip -r babel.zip babel/*

# move babel.zip to your project root directory

Install pytz (use gae-pytz in this example)

# download gaepytz 2011h
$ wget https://pypi.python.org/packages/source/g/gaepytz/gaepytz-2011h.tar.gz#md5=b7abe173cd98b417fab3e91c1498cdd2

# unzip and enter gaepytz source directory
$ tar xvzf gaepytz-2011h.tar.gz
$ cd gaepytz-2011h/

# create pytz.zip for zipimport
$ zip -r pytz.zip pytz/*

# move pytz.zip to your project root directory

Sample GAE project

Download the following files to your project root directory:

i18n.py (GAE Python webapp): The webapp determine the locale by the query string in URL.

i18n.py | repository | view raw

#!/usr/bin/env python
# -*- coding:utf-8 -*-

# for webapp2 to import babel and pytz
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'babel.zip'))
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'pytz.zip'))

import webapp2
import jinja2
from webapp2_extras import i18n


JINJA_ENVIRONMENT = jinja2.Environment(
    loader=jinja2.FileSystemLoader(os.path.dirname(__file__)),
    extensions=['jinja2.ext.i18n', 'jinja2.ext.autoescape'],
    autoescape=True)

JINJA_ENVIRONMENT.install_gettext_translations(i18n)

class MainPage(webapp2.RequestHandler):
    def get(self):
        locale = self.request.GET.get('locale', 'en_US')
        i18n.get_i18n().set_locale(locale)

        template_values = {}
        template = JINJA_ENVIRONMENT.get_template('index.html')
        self.response.write(template.render(template_values))

app = webapp2.WSGIApplication([
    ('/', MainPage),
], debug=True)

index.html (HTML template): Wrap the string you want to tranlsate in {{ _("string") }}. See [18] for more syntax you can use.

index.html | repository | view raw

<!doctype html>
<html>
<head>
  <meta charset="utf-8">
  <title>i18n of GAE Webapp</title>
</head>
<body>
  {{ _("home") }}
  <br />
  {{ _("about") }}
  <br />
  {{ _("'link'")|safe }}
  <br />
</body>
</html>

app.yaml (GAE Python config)

app.yaml | repository | view raw

application: i18n-gae-python
version: 1
runtime: python27
api_version: 1
threadsafe: true

handlers:
- url: /.*
  script: i18n.app

libraries:
- name: webapp2
  version: latest
- name: jinja2
  version: latest

babel.cfg (config for Babel extraction): This babel.cfg tells babel to extract all translations from all HTML files in your webapp and the encoding of HTML files are utf-8. For more information of the config file, refer to [16] and [17].

babel.cfg | repository | view raw

[javascript: **.html]
encoding = utf-8

Extract and compile translations

By default, webapp2 looks for pot and po files in locale directory under your project root directory, so first create a directory named locale:

# in your project root directory:
$ mkdir locale

Then extract all translations (create pot file).

# in your project root directory:
$ pybabel extract -F ./babel.cfg -o ./locale/messages.pot ./

The pot file looks like:

# Translations template for PROJECT.
# Copyright (C) 2015 ORGANIZATION
# This file is distributed under the same license as the PROJECT project.
# FIRST AUTHOR <EMAIL@ADDRESS>, 2015.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PROJECT VERSION\n"
"Report-Msgid-Bugs-To: EMAIL@ADDRESS\n"
"POT-Creation-Date: 2015-04-12 03:32+0800\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 1.3\n"

#: index.html:8
msgid "home"
msgstr ""

#: index.html:10
msgid "about"
msgstr ""

#: index.html:12
msgid "'link'"
msgstr ""

Then initialize the directory for each locale that your webapp will support. en_US and zh_TW are supported in our example. See [19] for table of locales.

# in your project root directory:
$ pybabel init -l en_US -d ./locale -i ./locale/messages.pot
$ pybabel init -l zh_TW -d ./locale -i ./locale/messages.pot

Two po files (locale/en_US/LC_MESSAGES/messages.po and locale/zh_TW/LC_MESSAGES/messages.po) are created. You do not need to do anything with the en_US po file because English is default language. Translate only non-default-language po files. In our exmaple, the zh_TW po file after translation looks like:

# Chinese (Traditional, Taiwan) translations for PROJECT.
# Copyright (C) 2015 ORGANIZATION
# This file is distributed under the same license as the PROJECT project.
# FIRST AUTHOR <EMAIL@ADDRESS>, 2015.
#
#, fuzzy
msgid ""
msgstr ""
"Project-Id-Version: PROJECT VERSION\n"
"Report-Msgid-Bugs-To: EMAIL@ADDRESS\n"
"POT-Creation-Date: 2015-04-12 03:32+0800\n"
"PO-Revision-Date: 2015-04-12 03:35+0800\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: zh_Hant_TW <LL@li.org>\n"
"Plural-Forms: nplurals=2; plural=(n != 1)\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Generated-By: Babel 1.3\n"

#: index.html:8
msgid "home"
msgstr "首頁"

#: index.html:10
msgid "about"
msgstr "關於"

#: index.html:12
msgid "'link'"
msgstr "'連結'"

After all translations done, compile po file with the following command:

# in your project root directory:
$ pybabel compile -f -d ./locale

Now we can run this GAE Python webapp, and then open the browser with URL:

http://localhost:8080/

You will see the webpage in default language. Then open the browser with URL:

http://localhost:8080/?locale=zh_TW

You will see the webpage in Traditional Chinese.

Update translations

When the strings to be translated change, re-create pot file:

# in your project root directory:
$ pybabel extract -F ./babel.cfg -o ./locale/messages.pot ./

Then update each locale:

# in your project root directory:
$ pybabel update -l en_US -d ./locale/ -i ./locale/messages.pot
$ pybabel update -l zh_TW -d ./locale/ -i ./locale/messages.pot

Again, translate the strings in each po file, and then compile again:

# in your project root directory:
$ pybabel compile -f -d ./locale

References:

[1]	Internationalization and localization with webapp2

[2]	python - How to enable {% trans %} tag for jinja templates? - Stack Overflow

[3]	I18N support · Issue #92 · getpelican/pelican · GitHub

[4]	python - i18n with jinja2 + GAE - Stack Overflow

[5]	Enable jinja2 and i18n translations on Google AppEngine \| Mikhail Shilkov

[6]	How to use i18n from webapp2_extras? - Google Groups

[7]	google app engine - Internationalization with python gae, babel and i18n. Can't output the correct string - Stack Overflow

[8]	Internationalization and localization - Wikipedia, the free encyclopedia

[9]	python - How to import modules in Google App Engine? - Stack Overflow

[10]	Max number of files and blobs is 1000 - Google Code

[11]	Moon blue diary: Using zipped pytz on GAE

[12]	Moon blue diary: Using the newest zipped pytz on GAE

[13]	brianmhunt/pytz-appengine · GitHub

[14]	Babel (old site)

[15]	gaepytz on Python Package Index

[16]	Babel Integration - Jinja2 Documentation

[17]	Extraction Method Mapping and Configuration - Working with Message Catalogs - Babel 1.0 documentation

[18]	i18n - Template Designer Documentation - Jinja2 Documentation

[19]	Table of locales - MoodleDocs