in phoenix elixir i18n ~ read. profile image for Erik Reedstrom by Erik Reedstrom
Practical i18n with Phoenix and Elixir

Practical i18n with Phoenix and Elixir

With the recent addition of Gettext to the Phoenix framework, there is now a production worthy way to ensure one's application can reach the broadest possible market.

Internationalization is an important part of application design, yet implementing i18n in a practical manner can be daunting, especially in a new technology. While there are examples showing the basics of utilizing gettext in a Phoenix project, we find none that describe a full implementation effectively segmenting content based on locale. Additionally, we want to ensure that the site's content can be effectively crawled in different languages, following guidelines provided by Google and Facebook.

The following describes our experience and implementation.

Subdirectories with gTLDs

Google provides 4 examples of locale specific architecture. Given limited resources, our choice is to follow the Subdirectories with gTLDs methodology as it provides a good balance of low maintenance, without the downsides of using query string based segmentation. This architecture requires urls in the form of /:locale/*, yet this adds a wrinkle to our routing strategy: How do we route without the locale present?

We want people to be able to hit an endpoint like https://helloheadlamp.com/get-started without thinking, or even having to know, what locales are supported. That means we need to support both explicit locale codes entered into the url, and implicit codes defined by the user agent in the 'Accept-Language' header. Essentially, if a user is on a Spanish language browser they should automatically route to /es/get-started, yet retain the ability to navigate to /en/get-started.

Supported Locales

Our first requirement is to ensure that locales are limited to ones we support. This has to be environment specific, however, as production should only show languages we have translated and QA'd, but development and test need to display all of them. This is accomplished with the config:

config/config.exs

config :headlamp, Headlamp.Gettext,  
  locales: ~w(en es nb-no)

web/gettext.ex

defmodule Headlamp.Gettext do  
  use Gettext, otp_app: :headlamp

  def supported_locales do
    known = Gettext.known_locales(Headlamp.Gettext)
    allowed = config[:locales]

    Set.intersection(Enum.into(known, HashSet.new), Enum.into(allowed, HashSet.new))
    |> Set.to_list
  end

  defp config, do: Application.get_env(:headlamp, __MODULE__)
end  

Now we can override the supported locales with an environment specific configuration. That is, if Bokmål isn't fully translated yet, we can excluded it from production.

Building the Plug

The Phoenix guides describe the creation of a locale switching plug under the Module Plugs section, however this does not meet our needs. Because we have the potential for the first route segment to not be a locale tag, we need to validate and reroute in that situation.

lib/headlamp/plug/locale.ex

defmodule Headlamp.Plug.Locale do  
  import Plug.Conn

  def init(default), do: default

  def call(conn, default) do
    locale = conn.params["locale"]
    is_ietf_tag = BCP47.valid?(locale, ietf_only: true)

    if is_ietf_tag && locale in Headlamp.Gettext.supported_locales do
      # Check if path contains a valid Locale
      conn |> assign_locale! locale
    else
      # Get locale based on user agent and redirect
      locale = List.first(extract_locale(conn)) || default
      # Invalid or missing locales redirect with user agent locale
      path = if is_ietf_tag do
        localized_path(conn.request_path, locale, conn.params["locale"])
      else
        localized_path(conn.request_path, locale)
      end
      conn |> redirect_to(path)
    end
  end

  defp assign_locale!(conn, value) do
    # Apply the locale as a process var and continue
    Gettext.put_locale(Headlamp.Gettext, value)
    conn
    |> assign(:locale, value)
  end
  ...
end  

The first conditional of the plug is very straight forward: if locale is a valid tag and we support it, assign the locale and move on. Validation is carried out in this case by the IETF tag check of our BCP47 parser. There's no magic to it though, as the IETF subset is just a list in the parser; the same check can be done against any list of tags. It's only used here since it was on hand.

Why IETF tags only? We just didn't need the full breadth of BCP47. Additionally, full BCP47 support could give false positives due to the wide spectrum of formats allowed.

The above section also includes the conditional switch for the less straight forward: an invalid tag or no locale provided. In either case, we're going to redirect, yet the manner in which we do is slightly different. First however, we need to extract the user's locale.

lib/headlamp/plug/locale.ex

defmodule Headlamp.Plug.Locale do  
  import Plug.Conn
  ...
  defp extract_locale(conn) do
    if Blank.present? conn.params["locale"] do
      [conn.params["locale"] | extract_accept_language(conn)]
    else
      extract_accept_language(conn)
    end
    # Filter for only known locales
    |> Enum.filter(fn locale -> Enum.member?(Headlamp.Gettext.supported_locales, locale) end)
  end

  defp extract_accept_language(conn) do
    case conn |> get_req_header("accept-language") do
      [value|_] ->
        value
        |> String.split(",")
        |> Enum.map(&parse_language_option/1)
        |> Enum.sort(&(&1.quality > &2.quality))
        |> Enum.map(&(&1.tag))
      _ ->
        []
    end
  end

  defp parse_language_option(string) do
    captures = ~r/^(?<tag>[\w\-]+)(?:;q=(?<quality>[\d\.]+))?$/i
    |> Regex.named_captures(string)

    quality = case Float.parse(captures["quality"] || "1.0") do
      {val, _} -> val
      _ -> 1.0
    end

    %{tag: captures["tag"], quality: quality}
  end
  ...
end  

These three functions are used to extract and parse the Accept-Language header provided by user agents. It sorts based on quality, allowing an ordered list of fallbacks. That is, if we are provided Accept-Language: nn-no, nb-no;q=0.8, en;q=0.7 but don't support Nynorsk, we could still fall back to Bokmål.

What's Blank.present?? This is just a protocol implementation convenience to match some of Ruby's sugar. The gist is here, but it is almost entirely pulled from the Elixir lang chapter on Protocols.

And this leaves the last part of the plug: generating a localized path and redirection.

lib/headlamp/plug/locale.ex

defmodule Headlamp.Plug.Locale do  
  import Plug.Conn
  ...
  defp localized_path(request_path, locale, original) do
    # If locale is an ietf tag, we don't support it. In this case,
    # replace the tag with the new locale.
    ~r/(\/)#{original}(\/(?:.+)?|\?(?:.+)?|$)/
    |> Regex.replace(request_path, "\\1#{locale}\\2")
  end

  defp localized_path(request_path, locale) do
    # If locale is not an ietf tag, it is a page request.
    "/#{locale}#{request_path}"
  end

  defp redirect_to(conn, path) do
    # Apply query if present
    unless Blank.blank? conn.query_string do
      path = path <> "?#{conn.query_string}"
    end
    # Redirect
    conn |> Phoenix.Controller.redirect(to: path)
  end
  ...
end  

We have two different signatures for the localized_path. The first localized_path/3, is used in situations where a tag is provided, but is unsupported. In this case we want to replace the original locale with a supported one. The second form, localized_path/2, is the case we expect more often: an unlocalized path is provided and must be redirected with a supported locale. Going back to our call function, we see that the plug has completed its transformation.

Now that we have a plug, we need to adjust our router.

Localizing the Router

As mentioned before, we need to have the ability to redirect unlocalized paths to their localized equivalents. The first step is to ensure the plug sits in our browser pipeline.

web/router.ex

defmodule Headlamp.Router do  
  use Headlamp.Web, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :put_secure_browser_headers
    plug :protect_from_forgery
    plug Headlamp.Plug.Locale, "en"
  end
  ...
end  

Then we have to move all localized paths under a /:locale scope.

web/router.ex

defmodule Headlamp.Router do  
  use Headlamp.Web, :router
  ...
  scope "/", Headlamp do
    pipe_through [:browser]

    # The dummy route is never hit, but exists so that the router has a "/" path.
    get "/", HomeController, :dummy
  end

  scope "/:locale", Headlamp do
    pipe_through [:browser]

    get "/", HomeController, :index

    resources "/get-started", GetStartedController, only: [:index, :create]
    ...
  end
end  

At this point all localized routes are available to us, and unlocalized paths will redirect. Note the root scope "/" has a single route that goes nowhere. This is required so that it appears that a / path exists, even though it will never be hit.

Note: All routes require :locale! Since we have localized the routes, path helpers will require the locale passed as the first parameter. For instance, get_started_path(@conn, :create, @locale). In views, this is available as @locale, or as conn.assigns.locale elsewhere.

We can still have unlocalized paths if required. This may be if we deploy exq_ui in conjunction with the app, or use Überauth and don't want to localize our vendor redirects. To accomplish this we only need to place those routes before the root scope is declared.

View Helpers and Templates

The last step in our internationalization journey is the templates. Google and Facebook guidelines both discuss tags used to help in crawling multilingual sites. We will add these to our layout, but require some view helpers to assist in the generation.

web/views/layout_view.ex

defmodule Headlamp.LayoutView do  
  use Headlamp.Web, :view

  import Phoenix.Naming

  @doc """
  Renders current locale.
  """
  def locale do
    Gettext.get_locale(Headlamp.Gettext)
  end

  @doc """
  Provides tuples for all alternative languages supported.
  """
  def fb_locales do
    Headlamp.Gettext.supported_locales
    |> Enum.map(fn l ->
      # Cannot call `locale/0` inside guard clause
      current = locale
      case l do
        l when l == current -> {"og:locale", l}
        l -> {"og:locale:alternate", l}
      end
    end)
  end

  @doc """
  Provides tuples for all alternative languages supported.
  """
  def language_annotations(conn) do
    Headlamp.Gettext.supported_locales
    |> Enum.reject(fn l -> l == locale end)
    |> Enum.concat(["x-default"])
    |> Enum.map(fn l ->
      case l do
        "x-default" -> {"x-default", localized_url(conn, "")}
        l -> {l, localized_url(conn, "/#{l}")}
      end
    end)
  end

  defp localized_url(conn, alt) do
    # Replace current locale with alternative
    path = ~r/\/#{locale}(\/(?:[^?]+)?|$)/
    |> Regex.replace(conn.request_path, "#{alt}\\1")

    Phoenix.Router.Helpers.url(Headlamp.Router, conn) <> path
  end
  ...
end  

At this point we can update our template to allow for multilingual content. We just need to add a dynamic lang value to the <html> tag, and generate the alternative links and OG tags.

web/templates/layout/app.html.eex

<!DOCTYPE html>  
<html lang="<%= locale %>">  
  <head>
    <meta property="og:site_name" content="Headlamp">
    <meta property="og:url" content="https://www.helloheadlamp.com">
    <meta property="og:title" content="<%= gettext "Let your kids explore Instagram. See only what matters." %>">
    <meta property="og:description" content="<%= gettext "Kids are meant to explore. Get weekly insights about how your kids use Instagram™, sent directly to your phone. Be informed without being invasive." %>">
    <meta property="og:image" content="<%= static_url(@conn, "/images/OG_Facebook.png") %>" />
    <meta property="og:type" content="website">
    <!-- Generate og:locale tags -->
    <%= for {property, content} <- fb_locales do %>
      <%= Phoenix.HTML.Tag.tag(:meta, property: property, content: content) %>
    <% end %>

    <!-- Generate link alternate tags -->
    <%= for {lang, path} <- language_annotations(@conn) do %>
      <%= Phoenix.HTML.Tag.tag(:link, rel: "alternate", hreflang: lang, href: path) %>
    <% end %>
  </head>
  ...
</html>  

Wrapping up

The addition of Gettext to Elixir and Phoenix' ever growing arsenal of tools is very welcome. Although it has taken us some effort to implement the i18n in an effective way, it now allows us to produce applications with the broadest reach.

This implementation is by no means authoritative, but we hope the community can build upon it and continue to innovate!