Best Practices: Localization of Erlang Applications. Also, Error Messages!

[Alex Arnon]

Hi All,

We're building a nontrivial Erlang application, which will provide a potentially high-volume, public RESTful Web Service. As part of the API, we will be providing both informative text (status, labels, layout templates) and error messages. A twofold question, then, to the experienced Grandmasters in the audience:
1. How have you implemented localization of text? This includes parameterized format strings.
2. Has anyone used composite error values? For example, instead of {error, badarg}, use {error, {badarg, [{arg, ArgumentName}, {message, LocalizedMessageId}, {message_args, [...]}... Should we steer clear of these, and if so, what alternative would you suggest?

Thanks in advance!

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Steve Davis]

Given that "strings" are either ambiguous or implicitly constrained
inventions of data representation, then you may consider applying
something along the lines of:

-record(text, {value, charset = utf8, locale = 'us-en'}).

...to all user-viewable messages. The transformation would use
whatever hideously ambiguous or implicitly constrained "string" that
bubbles up from infrastructure as the "key" to that tuple value.

BTW This problem isn't really an erlang/otp but one that every
platform that I have ever investigated. What makes erlang different is
that you can easily forsee a potential, and implementable, solution to
this legacy issue in CS.

/s

On Jul 13, 2:19 pm, Alex Arnon <[hidden email]> wrote:

> Hi All,
>
> We're building a nontrivial Erlang application, which will provide a
> potentially high-volume, public RESTful Web Service. As part of the API, we
> will be providing both informative text (status, labels, layout templates)
> and error messages. A twofold question, then, to the experienced
> Grandmasters in the audience:
> 1. How have you implemented localization of text? This includes
> parameterized format strings.
> 2. Has anyone used composite error values? For example, instead of {error,
> badarg}, use {error, {badarg, [{arg, ArgumentName}, {message,
> LocalizedMessageId}, {message_args, [...]}... Should we steer clear of
> these, and if so, what alternative would you suggest?
>
> Thanks in advance!
>
> _______________________________________________
> erlang-questions mailing list
> [hidden email]://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Loïc Hoguin]

On 07/13/2011 09:19 PM, Alex Arnon wrote:
> Hi All,
>
> We're building a nontrivial Erlang application, which will provide a
> potentially high-volume, public RESTful Web Service. As part of the API,
> we will be providing both informative text (status, labels, layout
> templates) and error messages. A twofold question, then, to the
> experienced Grandmasters in the audience:
> 1. How have you implemented localization of text? This includes
> parameterized format strings.

I don't know how well it works but there's always:
  http://www.trapexit.org/Gettext_-_An_internationalization_package.

I'll be interested hearing what you find works best.

> 2. Has anyone used composite error values? For example, instead of
> {error, badarg}, use {error, {badarg, [{arg, ArgumentName}, {message,
> LocalizedMessageId}, {message_args, [...]}... Should we steer clear of
> these, and if so, what alternative would you suggest?

OTP uses format_error functions to format the errors. You could use the
same pattern in your case but internationalized. For example passing
{error, badarg, What} to the function would give the formatted version
in the currently selected language.

--
Loïc Hoguin
Dev:Extend
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Alex Arnon]

On Thu, Jul 14, 2011 at 2:53 AM, Steve Davis <[hidden email]> wrote:

Given that "strings" are either ambiguous or implicitly constrained
inventions of data representation, then you may consider applying
something along the lines of:

-record(text, {value, charset = utf8, locale = 'us-en'}).

This is similar to what we had in mind, for localized parts of a more generic text structure (similar to iolists, but with records such as the above supported as well).
Arguments might also be added as part of #text{}.
This covers the "bit up to rendering" - everything would be rendered to UTF-8 binaries eventually for public consumption.

... snip ...

BTW This problem isn't really an erlang/otp but one that every
platform that I have ever investigated. What makes erlang different is
that you can easily forsee a potential, and implementable, solution to
this legacy issue in CS.

Yes, this is indeed a universal "problem". I have seen several solutions in other environments, usually using either a global string definitions file (macros/constants/hashes, or a simple flat data file) or a database (usually in larger systems).
For our application we think a file containing all the definitions could be used, since the size of the message set will not be in the many-hundreds. Would you suggest that this be a properties/XML/whatever data file, or a module containing a translation function (or more)?

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Alex Arnon]

On Thu, Jul 14, 2011 at 1:56 PM, Loïc Hoguin <[hidden email]> wrote:

On 07/13/2011 09:19 PM, Alex Arnon wrote:
> Hi All,
>
> We're building a nontrivial Erlang application, which will provide a
> potentially high-volume, public RESTful Web Service. As part of the API,
> we will be providing both informative text (status, labels, layout
> templates) and error messages. A twofold question, then, to the
> experienced Grandmasters in the audience:
> 1. How have you implemented localization of text? This includes
> parameterized format strings.

I don't know how well it works but there's always:
 http://www.trapexit.org/Gettext_-_An_internationalization_package.

I'll be interested hearing what you find works best.

Thank you, I shall look into it.
And report :)
 

> 2. Has anyone used composite error values? For example, instead of
> {error, badarg}, use {error, {badarg, [{arg, ArgumentName}, {message,
> LocalizedMessageId}, {message_args, [...]}... Should we steer clear of
> these, and if so, what alternative would you suggest?

OTP uses format_error functions to format the errors. You could use the
same pattern in your case but internationalized. For example passing
{error, badarg, What} to the function would give the formatted version
in the currently selected language.

--
Loďc Hoguin
Dev:Extend

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Richard Carlsson-3]

On 07/13/2011 09:19 PM, Alex Arnon wrote:
> We're building a nontrivial Erlang application, which will provide a
> potentially high-volume, public RESTful Web Service. As part of the API,
> we will be providing both informative text (status, labels, layout
> templates) and error messages. A twofold question, then, to the
> experienced Grandmasters in the audience:
> 1. How have you implemented localization of text? This includes
> parameterized format strings.

At Klarna we are using (and maintaining) the gettext library, and it
works very well. You'll find the latest version at
https://github.com/etnt/gettext - the version at Jungerl is very much
out of date.

In particular, you'll probably want to use the new (not yet documented)
macro ?STXT(...) instead of the ?TXT(...) macro. This is less error
prone, since the fields and their meaning are more easily identified and
do not have to be listed in order of occurrence. For example:

   ?STXT("Hello, $name$! Today's date is $date$ and the time is $time$.",
         [{date, current_date_as_string()},
          {time, current_time_as_string(),
          {name, customer_name(CustomerID)}])

For the documentation (not very up to date, sorry), run 'make docs' and
open doc/index.html in a browser.

Tobbe also made some support for gettext in rebar:
http://www.redhoterlang.com/entry/138dc1b1f8720f5c4f38864e4f0c338f

To make it easier to actually get the translations into the system, or
update translations as the original text changes, you may also want to
use the Polish tool:

http://www.redhoterlang.com/entry/8af2641026c0dfa27e54cb0da57eb392

http://www.youtube.com/watch?v=UdhE2YOkBCU

http://www.redhoterlang.com/entry/98951c7196e0d91999bb238de5defe47


> 2. Has anyone used composite error values? For example, instead of
> {error, badarg}, use {error, {badarg, [{arg, ArgumentName}, {message,
> LocalizedMessageId}, {message_args, [...]}... Should we steer clear of
> these, and if so, what alternative would you suggest?

Who is expected to read these messages? End users, or developers? For
developers, it's better if the message is in plain English. First, you
may have people from different countries working on your code. Second,
it's harder to google for the error message if the one you got was in
Spanish, but the only people who have previously reported the problem
got their messages in German. At the very least, always provide a
summary or short identifier in English.

     /Richard
_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Alex Arnon]

On Thu, Jul 14, 2011 at 3:33 PM, Richard Carlsson <[hidden email]> wrote:

On 07/13/2011 09:19 PM, Alex Arnon wrote:

We're building a nontrivial Erlang application, which will provide a
potentially high-volume, public RESTful Web Service. As part of the API,
we will be providing both informative text (status, labels, layout
templates) and error messages. A twofold question, then, to the
experienced Grandmasters in the audience:
1. How have you implemented localization of text? This includes
parameterized format strings.

At Klarna we are using (and maintaining) the gettext library, and it works very well. You'll find the latest version at https://github.com/etnt/gettext - the version at Jungerl is very much out of date.

In particular, you'll probably want to use the new (not yet documented) macro ?STXT(...) instead of the ?TXT(...) macro. This is less error prone, since the fields and their meaning are more easily identified and do not have to be listed in order of occurrence. For example:

 ?STXT("Hello, $name$! Today's date is $date$ and the time is $time$.",
       [{date, current_date_as_string()},
        {time, current_time_as_string(),
        {name, customer_name(CustomerID)}])

For the documentation (not very up to date, sorry), run 'make docs' and open doc/index.html in a browser.

Tobbe also made some support for gettext in rebar:
http://www.redhoterlang.com/entry/138dc1b1f8720f5c4f38864e4f0c338f

To make it easier to actually get the translations into the system, or update translations as the original text changes, you may also want to use the Polish tool:

http://www.redhoterlang.com/entry/8af2641026c0dfa27e54cb0da57eb392

http://www.youtube.com/watch?v=UdhE2YOkBCU

http://www.redhoterlang.com/entry/98951c7196e0d91999bb238de5defe47

Thank you, this looks promising!

 

2. Has anyone used composite error values? For example, instead of
{error, badarg}, use {error, {badarg, [{arg, ArgumentName}, {message,
LocalizedMessageId}, {message_args, [...]}... Should we steer clear of
these, and if so, what alternative would you suggest?

Who is expected to read these messages? End users, or developers? For developers, it's better if the message is in plain English. First, you may have people from different countries working on your code. Second, it's harder to google for the error message if the one you got was in Spanish, but the only people who have previously reported the problem got their messages in German. At the very least, always provide a summary or short identifier in English.

   /Richard

I was thinking of two purposes for the structured Reason:
1. The error handling code - which may receive extra information regarding the error itself.
2. Logging code - the handling code can forward the error to a logger, with attached message format + args etc.
Localization of the message is probably pointless, I agree - I kind of overloaded [2] with [1]. :)

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

[Håkan Stenholm]

The thread "[erlang-questions] utf-8 PB" in the erlang mailing list (archive) during 2010-06-16/17/18 may be of interest to you.
I (Håkan Stenholm) wrote a bunch of replies about using gettext and .po files in that thread.

I used to work with internationalization/localization issues at Klarna back in 2005-2010, but not having worked there (or with erlang) since 2010-04 makes me fairly useless as a source for current information.



On 2011-07-14 16.46, Alex Arnon wrote:

On Thu, Jul 14, 2011 at 3:33 PM, Richard Carlsson <[hidden email]> wrote:

On 07/13/2011 09:19 PM, Alex Arnon wrote:

We're building a nontrivial Erlang application, which will provide a
potentially high-volume, public RESTful Web Service. As part of the API,
we will be providing both informative text (status, labels, layout
templates) and error messages. A twofold question, then, to the
experienced Grandmasters in the audience:
1. How have you implemented localization of text? This includes
parameterized format strings.

At Klarna we are using (and maintaining) the gettext library, and it works very well. You'll find the latest version at https://github.com/etnt/gettext - the version at Jungerl is very much out of date.

In particular, you'll probably want to use the new (not yet documented) macro ?STXT(...) instead of the ?TXT(...) macro. This is less error prone, since the fields and their meaning are more easily identified and do not have to be listed in order of occurrence. For example:

 ?STXT("Hello, $name$! Today's date is $date$ and the time is $time$.",
       [{date, current_date_as_string()},
        {time, current_time_as_string(),
        {name, customer_name(CustomerID)}])

For the documentation (not very up to date, sorry), run 'make docs' and open doc/index.html in a browser.

Tobbe also made some support for gettext in rebar:
http://www.redhoterlang.com/entry/138dc1b1f8720f5c4f38864e4f0c338f

To make it easier to actually get the translations into the system, or update translations as the original text changes, you may also want to use the Polish tool:

http://www.redhoterlang.com/entry/8af2641026c0dfa27e54cb0da57eb392

http://www.youtube.com/watch?v=UdhE2YOkBCU

http://www.redhoterlang.com/entry/98951c7196e0d91999bb238de5defe47

Thank you, this looks promising!

 

2. Has anyone used composite error values? For example, instead of
{error, badarg}, use {error, {badarg, [{arg, ArgumentName}, {message,
LocalizedMessageId}, {message_args, [...]}... Should we steer clear of
these, and if so, what alternative would you suggest?

Who is expected to read these messages? End users, or developers? For developers, it's better if the message is in plain English. First, you may have people from different countries working on your code. Second, it's harder to google for the error message if the one you got was in Spanish, but the only people who have previously reported the problem got their messages in German. At the very least, always provide a summary or short identifier in English.

   /Richard

I was thinking of two purposes for the structured Reason:
1. The error handling code - which may receive extra information regarding the error itself.
2. Logging code - the handling code can forward the error to a logger, with attached message format + args etc.
Localization of the message is probably pointless, I agree - I kind of overloaded [2] with [1]. :)

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions

_______________________________________________
erlang-questions mailing list
[hidden email]
http://erlang.org/mailman/listinfo/erlang-questions