On several occasions, I’ve had the experience where, after a survey has been sent for translation (and we were assured no further changes would be made), the inevitable “last” few modifications have followed soon thereafter.  Such changes could be as a result of mistakes made in the design of the original survey or based on feedback received from survey pilots.  In other cases, stakeholders who hadn’t been involved early on in the process suddenly want to add their feedback.

Short of sending surveys for complete re-translation every time a change is made, maintaining synchronisation between translations is difficult and frustrating – not to mention costly. This complexity increases exponentially with the number of additional languages.

Our approach

To overcome the logistical nightmares which often accompany multi-lingual surveys, we focused on three key areas.

Firstly, we built the survey designer to give users the freedom to design in any language and easily switch from one to another. The key point is that the structure remains intact, regardless of the language you’re working with. Logic, question order and constraints remain unaffected by the language. Only the text shown to the fieldworker changes.

Secondly, we decided to take a very practical view on the actual translation process. While it’s true that at some point in the future (watch this space) we may provide “fancier” and more collaborative mechanisms for translation, for now we settled on the concept of translation tables.

A translation table is simply an Excel export of the text used in the survey. There is a source language (e.g. English) and a destination language (e.g. isiZulu). The spreadsheet produced by the system, contains the source language for the translator as well as a cell for the translated equivalent. This file is sent to a translator who opens it and simply fills in the blanks. Upon returning the file to you, all that remains is to upload the file and the translated text will be imported and stored for the specified language.

Should a question change, the translation table is simply regenerated and sent to the translator. All their previous translations will be provided with highlighted cells indicating where their input is needed. Once their changes have been made, uploading the file overwrites the previous translation text.

The final step we took was to allow the generation of translations for any source and destination combination. Assume you have a survey which, for example, needs to be available in 3 languages: English, Afrikaans and German. You may have a translator who can speak English and Afrikaans and another who can speak Afrikaans and German. The translation functionality in Mobile Researcher allows you to export a translation table for English-to-Afrikaans and, once the Afrikaans translation has been performed, a table for Afrikaans-to-English.

Translation is an area we are planning to give a lot more attention to in the future. I think we have some really neat ideas which I’ll share with you in future posts. Hopefully as things stand currently, you’ll find the existing functionality a huge help.

In the first stage of the field research process, the metrics are usually determined – the question “What data must be collected?” is answered.  Once this has been decided on, the surveys (aka questionnaires or forms – depending on function) are designed.  The goals which survey designers should be cognisant of during this process are:

• Surveys are structured in an unambiguous manner which does not make their responses subject to fieldworker bias or (mis)interpretation.  Factors such as localised terminology, advanced logic and respondent fatigue come into play.
• The collected data is easy (or at the very least possible) to analyse.  Tables are a common favourite amongst designers who are used to deploying surveys on paper because they use very little space – but they can create tremendous challenges when trying to unravel them at a later stage.

In the majority of cases, the design process involves multiple stakeholders (such as principal investigators, consultants, ethics committees, etc) and the design tools used are often run-of-the-mill word processors (such as Microsoft Word or OpenOffice).  Countless versions (and I use the term version rather loosely here) are emailed from one stakeholder to the next with limited regard for change control. 

Without a dedicated survey design tool, asking questions in a consistent fashion and the way in which the logic is described to the fieldworker can vary significantly – even within the same survey.  That’s bad news for data integrity.

If a change has to be made to a survey (and that occurs rather frequently) such as the addition of a new question, or removal of an unwanted one, it often necessitates a thorough check of the entire survey to ensure that skip logic references and numbering still make sense.

Our approach

When developing the survey designer component of Mobile Researcher, we decided to keep the existing conceptual model of a designing a survey on paper.  Designers lay out their surveys section by section and can select from a variety of question types.  We felt that it was a better idea to pass the complexity of survey logic on to the designer rather than leave it to fieldworkers to follow instructions.

Instead of writing out instructions which fieldworkers need to interpret or follow, the survey workflow (logic) is built into the design of the survey itself. This logic is not actually shown to the fieldworker but is evaluated and executed by the mobile application on their phone.  The fieldworker is only ever presented with one question at a time.

Using a consistent approach to survey design improves efficiency and reduces the likelihood of errors being introduced by confused fieldworkers.  A caveat to the approach is that survey designers need to think clearly about how they’d like the survey to flow.  Although the designer is intended to be intuitive to use, there is no substitute for logical thought.

Somewhat obviously, the challenges differ somewhat depending on the nature of the data collection exercise but, in general, there is a great deal of commonality.  In a series of discussions on the challenges of paper-based field research, I’ll use the scenario whereby a fieldworker interviews a respondent to collect the necessary information (interviewer administered survey).

In a typical field research exercise, there are several key processes which occur in a reasonably sequential fashion.  An obvious observation is that each process has outputs or deliverables which feed into and impact on subsequent processes and thus the final outcome.

Some of the processes and activities which I will cover in separate posts include:

• Survey design
• Translation
• Deployment
• Conduction
• Collection
• Capture
• Cleaning and processing
• Analysis
• Archive
• Quality control
• Privacy and confidentiality
• Management of distributed resources

Each process can pose significant obstacles and expense when making use of paper-based techniques.  By replacing paper with a suitable technology alternative, we can go a long way to improving the efficiency and quality of field research and data collection.