Multi language support in AngularJS app using angular-translate that always sends and fetches data in English.

Recently we at ICAN came across an interesting issue that needed solving. We’re developing an app with localisation. Users can choose to use the app in either English or Chinese currently. As we’re using AngularJS we decided to go with Angular-translate to achieve this. The issue we were facing was simple, all data sent to our backend needed to be sent in English for future analytical processing.

I’ll elaborate. With angular-translate you can convert strings of text into any language but this means that if you’re using the app in Chinese and you update your country to China the app will save that data as 中国. We needed this information to be sent to our server as ‘China’ rather than ‘中国’. Similarly, if the data stored in the server is the literal string ‘China’ and a Chinese user fetches that data they will need to be able to see 中国 instead.

In this blog, I want to show you how we solved this issue from start to finish.

First off, I’ve mentioned a little about angular-translate, the AngularJS module used to easily localise your app for multiple languages. It’s very quick and easy to setup, simple type the following into your terminal.

npm install --save-dev angular-translate

Once installed add the module to your existing Angular app just like any other module.

You can see a quick example of how the module looks and works here.

var app = angular.module('at', ['pascalprecht.translate']);
app.config(function ($translateProvider) {
    $translateProvider.translations('en', {
     'LOGIN': 'Login',
     'LOGOUT': 'Logout',
     'REGISTER': 'Register',
     'CREATE': 'Create an account',
     'OK': 'Ok',
     'EDIT': 'Edit',
     'SAVE': 'Save',
     'CONTINUE': 'Continue',
     'CANCEL': 'Cancel',
     'BACK': 'Back'
    });
    $translateProvider.translations('zh', {
     'LOGIN': '登录',
     'LOGOUT': '登出',
     'REGISTER': '注册',
     'CREATE': '创建新帐号',
     'OK': '成功',
     'EDIT': '保存',
     'SAVE': '保存',
     'CONTINUE': '继续',
     'CANCEL': '取消',
     'BACK': '返回'
    });
    $translateProvider.preferredLanguage('en');
});

Now whenever you want to translate a string you can use

{{ 'LOGIN' | translate }}

If the preferred language is ‘en’ then LOGIN will be replaced with the string ‘Login’. If the preferred language is ‘zh’ then LOGIN will be replaced with ‘登录’. Simple.

This is all fine and well until things get a little more complicated. We also use the preferred language to load language specific files. For example, we have two files we use when selecting a country.

country_en.json is a long file so I’ll just show a snippet of what it looks like here

{
   "Americas": [{
     "code": "AW",
     "name": "Aruba"
   },{
     "code": "AI",
     "name": "Anguilla"
   },{
     "code": "AR",
     "name": "Argentina"
   }...]
}...

We also have country_zh.json which is loaded when the preferred language is ‘zh’ or Chinese. It looks like this

{
   "美洲": [{
     "code": "AW",
     "name": "阿鲁巴"
   },{
     "code": "AI",
     "name": "安圭拉"
   },{
     "code": "AR",
     "name": "阿根廷"
   }...]
}...

Our app uses the preferred language variable to determine which file is loaded and rendered to the screen, the user’s selection is then saved to be sent to our server for storage. Before the data is sent we have an intermediary stage that converts Chinese text to its English equivalent. This is how we’ve set up our factories.

app.factory('FILES', function ($http, lang) {
   return functions = {
     en_zh: function () {
       return $http.get('json/converters/en-zh.json');
     },

     zh_en: function () {
       return $http.get('json/converters/zh-en.json');
     },

     countryList: function () {
       return $http.get('json/country_' + lang + '.json'); 
     }
   };
});

and

app.factory('TRANSLATE', function (FILES) {

   return {
       toEnglish: function (key, value) {

         return FILES.zh_en().then(function (data) {
           var data = data.data[0];
           var result = {};
 
           for(var i in data){
             if(i == value){
               result[key] = data[i];
               return result;
             }
           }
           result[key] = value;
           return result;
         });
       },
       toChinese: function (key, value) {

         return FILES.en_zh().then(function (data) {
           var data = data.data[0];
           var result = {};

           for(var i in data){
             if(i == value){
               result[key] = data[i];
               return result;
             }
           }
           result[key] = value;
           return result;
       });
     }
   }
});

To convert either toEnglish or toChinese we can simply add the TRANSLATE module to a function like so.

function editProfile(data, callback, TRANSLATE, $q, $http) {

   var promises = [];
   var url = 'http://www.example.com/';

   for (var i in data){
     var promise = TRANSLATE.toEnglish(i, data[i]);
     promises.push(promise);
   }

   $q.all(promises).then(function(results){
     var data = {};

     for(var i in results){
       Object.assign(data, results[i]);
     }

     var req = {
       method: 'POST',
       url: url,
       headers: {
         "Content-Type": 'application/json'
       },
       withCredentials: true,
       data: data
     };

     $http(req).success(function (result) {
       callback(result);
     });
   })
}

The example above is when a user updates their profiles, their data is sent to this function as form data. The data is sent through toEnglish. This iterates through the file zh_en.json and if it finds a match the data will be updated to English.

// zh_en.json
{
  '阿鲁巴': 'Aruba',
  '安圭拉': 'Anguilla',
  '阿根廷': 'Argentina'
}

Once all date has been through the process the results will be sent to the server using Angular’s $http method.

You can see from the code above that this process can be used to fetch English strings and convert them to Chinese as well if the user is using the app with the Chinese language setting.

// en_zh.json
{
  'Aruba': '阿鲁巴',
  'Anguilla': '安圭拉',
  'Argentina': '阿根廷'
}

Choosing a university: An exploratory analysis

At ICAN, we are developing the application that will be a powerful tool at the students’ hands when they are choosing their university studies. Wouldn’t be magical for a student to get access to the university options that best match their interests?
Before getting deeper into this idea, we decided to take inspiration from students’ own needs and opinions about universities. By creating and distributing a questionnaire, we received responses from students from all over the world and aggregated them in order to get an idea about their opinions.
The students who participated in our research are current UK-based students studying on different degree levels. As the research is running at a limited time period, we gathered around 300 responses, from which half of them remained after cleaning inaccurate or irrelevant data. Some interesting results from our exploratory analysis are presented below.
First of all, we asked the students to rank in order of importance 5 factors (University Ranking, Job opportunities, Fees, Location and Agents’ recommendation) on which they rely when choosing a university. We included standard factors but also the Agents’ recommendation factor, as many of the students studying at UK are overseas (such as China) who tend to use private agencies to apply for a university course. Here is what we found out:
Choosing a university: Average score for each factor

Choosing a university: Average score for each factor

Students clearly consider more the University Ranking and the Job opportunities when choosing their studies over the Location or the Agents’ Recommendation. In order to realize how big is this difference, we count the times each of the factors was ranked in the first place only. The following bar chart assures our previous finding that the University Ranking is the first factor considering by students.
choosing_by_factor

Choosing a university: Times each factor has been ranked first

Then, we were curious to see how satisfied students from different continents are whey they start studying at a UK university. As the majority of our data was from two continents, Europe and Asia, our analysis is restricted to these two. Seven domains of the university were included in the questionnaire and respondents had to score them from 1 to 6, with 1 being the least satisfied and the 6 the most satisfied score. From the following chart, we found out that European students are overall more satisfied in each single domain about their quality of studies.
europe_vs_asia

European VS Asian students: Satisfaction score for each domain

In the case of Asian students, we had another issue to count in our analysis: More than 70% of Asian students that appeared to our data filled in that they delivered their university application through an agent service. To this end, we thought that it would be useful to separate the ones who applied by an agent service and the ones applied by themselves in order to interpret the low scores in terms of satisfaction got from Asian students. In the next bar chart, one can see the scores reported by the two Asian students’ groups. Indeed, Asian students applied by agents were less satisfied when it comes to core university domains, like Courses, Material, Lecturers and Working opportunities. Only social domains, like Life, and Networking gained higher scores by students who used an agent service. It seems that an agent service by itself is not capable to offering an accurate recommendation about which university a student should choose.

agent_vs_diy

Asian students: Satisfaction score when applying via an Agent service or by themselves

 

Finally, it is important to note that choosing a studying field that best matches a student’s own needs and profile is a multivariate decision. However, the results of our research have shown some interesting findings when it comes to students and university choice. The limits of this research are of course not restricted to this. Data gathering is continuous in our work and it gives us the ability to go on with presenting more data reports.

Scraping the web with Erlang

One of the things we do at ICAN is scraping websites. In my particular case, I had to write a scraper to serve some information to our users to fulfil one of the requirements for one of our customers.

There is no particular technology to scrape a website. In other words, (almost) any programming language can be used for this purpose. The only thing you need is the ability to fetch the source code of a website and the ability to manipulate strings. I decided to use Erlang and there were several reasons to make this choice. Not only that I am pretty comfortable with the language and I had to write this scraper in a very short amount of time. Also some of its features made me pick this language instead of any of other available options like, for example, Java.

Erlang is a soft real-time non-pure functional programming language which has been widely used in the field of communications. It is not a particularly fast language (compared to, for example, C or C++), but it has quite a lot of nice features that made it a good choice for this task. One of these features emerges from its own design. Erlang is an implementation of the so called Actor Model. This makes it particularly suitable for concurrent and parallel programming. In this paradigm, computations are carried out by “actors”. Actors are individual entities that do not share memory (they have their own memory spaces) and communicate to each other passing messages. This means that there are no shared data structures and therefore computations made by one of these actors do not have side-effects in other actors. The consequence is clear: there is no need of locking mechanisms to do parallel computations.

Another nice feature derived from the actor model is the tolerance to failures. Since there is no shared data structures and due to the independence of these actors respect to the others, the failure of one of these actors has no consequences to the others. Therefore, in the event of a failure the program can continue even though with reduced power. Besides, Erlang comes with an implemented supervision facility that allows you to write fault-tolerant programs very easily. You only have to monitor these actors (processes in Erlang terminology), so in the case of failure the monitor would recover the actor automatically.

Erlang is not particularly good at string manipulation. In fact, there is not such type in Erlang (strings are just lists of integers). However, it is particularly good at binary streams manipulation. Indeed, they can be almost manipulated as if they were strings, at least for the ASCII charset. And for other charsets like ISO/IEC 8859-1 (Latin-1) or UTF-8 it is still rather convenient (although their support is still a little bit limited). Nonetheless, nice features from functional programming like lists/binaries comprehension and pattern-matching certainly easies the process. Besides, Erlang has a powerful built-in library for regular expressions that works with both strings and binaries. This also makes it an attractive choice to build a scraper.

Yet to carry out the operations in the source of the websites, it is necessary to fetch the source code in the first place. In this area, Erlang is also a winner. The Erlang/OTP environment includes a http and a ssl client libraries that makes this process trivial.

For these reasons, I used Erlang to write an scraper has the following features:
– It is concurrent (it uses a workers pool to process the different web pages required).
– It is fault tolerant (it has a monitor process that supervises the operation of the workers and master so in the event of failure it recovers them and tries to finish the task for at least five times before giving up).
– It has support for both Chinese and English languages.

And all this is made in less than 500 lines of pure Erlang code, without the need of any third-party external libraries.

In the future this scraper will be replaced by another one written in a “less exotic” language. Mostly, for the sake of the maintainability (not many Erlang programmers out there). But not because Erlang cannot do the job.

Source: http://turnoff.us/geek/advanced-species/