Jump to ContentJump to Main Navigation
Developing Cross Cultural Measurement$

Thanh V. Tran

Print publication date: 2009

Print ISBN-13: 9780195325089

Published to Oxford Scholarship Online: May 2009

DOI: 10.1093/acprof:oso/9780195325089.001.0001

Show Summary Details
Page of

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2019. All Rights Reserved. An individual user may print out a PDF of a single chapter of a monograph in OSO for personal use.  Subscriber: null; date: 23 October 2019

Adopting or Adapting Existing Instruments Adopting or Adapting Existing Instruments

Adopting or Adapting Existing Instruments Adopting or Adapting Existing Instruments

Chapter:
(p.31) 4 Adopting or Adapting Existing Instruments
Source:
Developing Cross Cultural Measurement
Author(s):

Thanh V. Tran

Publisher:
Oxford University Press
DOI:10.1093/acprof:oso/9780195325089.003.0004

Abstract and Keywords

Both adopting and adapting existing research instruments often require the translation of the selected instrument from a source language to a target language. Cross-cultural translation is one of the major tasks in cross-cultural research. The task of translation becomes more challenging when an instrument is translated into two or more target languages simultaneously. The process of adopting and adapting existing research instruments involves (1) reviewing existing cross-cultural translation approaches and offering the reader practical guidelines; (2) employing a multilevel translation process encompassing back-translation, expert evaluation, cognitive interviews, focus group evaluation, and field evaluation; and (3) evaluating the translation of the adopted or adapted instruments.

Keywords:   cross-cultural translation, multilevel translation process, committee translation, translation evaluation, translation biases

Cross-Cultural Translation and Related Issues

Both adopting and adapting existing research instruments often require the translation of the selected instrument from a source language to a target language. Cross-cultural translation is one of the major tasks in cross-cultural research. The task of translation becomes more challenging when an instrument is translated into two or more target languages simultaneously.

This chapter will (1) review existing cross-cultural translation approaches and offer the reader with practical guidelines; (2) present a multilevel translation process encompassing back translation, expert evaluation, cognitive interviews, focus group evaluation, and field evaluation; and (3) offer a guide for best practices in selecting translators to perform cross-cultural translation. Several examples will be presented to illustrate potential biases in cross-cultural translation and cross-cultural data analysis.

Researchers have employed a variety of translation approaches, and it appears that no single approach has become universal. In cross-cultural psychology, Brislin, Lonner, and Thorndike (1973) recommended that cross-cultural translation should involve back-translation, bilingual techniques, committee approach, and pretest. More specifically, (p.32) back-translation is a process wherein an instrument is translated from its original language to a different language, and the translated version of the instrument is translated back to the original language to assure conceptual equivalence.

Maneesriwongul and Dixon (2004) reviewed 47 studies that involved cross-cultural translation and concluded that there is a lack of consensus on the standards of cross-cultural translation. There are three common approaches of cross-cultural translation that have been used: (1) forward-only translation approach is a one-way translation from the source language to a target language. This approach is not recommended because of the lack of reliability and validity evaluation; (2) forward translation with testing approach is stronger, but it also does not address the issue of cross-cultural validity and reliability; and (3) back translation approach appears to be stronger than the previous two approaches, but it tends to emphasize the literal translation or linguistic equivalence, which does not warrant cross-cultural equivalence.

Harkness (2003) has offered a more desirable approach of survey questionnaire translation process called Translation, Review, Adjudication, Pre-testing and Documentation (TRAPD). This approach is a committee-based approach that involves translators, reviewers, and adjudicator. Committee or team approaches of translation have been recognized as a more effective approach compared with other approaches (Guillemin, Bombardier, & Beaton, 1993; Harkness, Pennell & Schoua-Glusber, 2004). The translation can be performed via parallel translations or split translations. In parallel translations, translators work independently before submitting their translation for committee review and evaluation. When two or more nations and groups share the same language, each group can translate a part of the whole research instrument (i.e., split translation). The committee approach and TRADP process do not address the issue of gender representatives in either translation or evaluation. In this book, gender representative is required in questionnaire construction and translation process. Harkness (2003) has suggested that translators are “skilled practitioners who have received training on translating questionnaires,” (p. 36). Skilled translators should have adequate knowledge on the research topic and population to perform valid translation. It would be difficult for a translator who has his/her formal training in engineering to perform a translation of a research questionnaire for a study of depression. This should also be (p.33) applied to the reviewers of the translation of the questionnaire and those who adjudicate the final translation of the questionnaire. Harness (2003) recommends the use of “team approaches” for cross-cultural translation of research instruments. The key advantage of this approach is that team members whose diverse backgrounds will help the team to determine the best translation outcomes (Guillemin, Bombardier, & Beaton, 1993).

Figure 4.1 illustrates the cross-cultural translation process from the source language (e.g., English) to the target language (e.g., Vietnamese). Because there is no gold standard for cross-cultural translation procedure, this book combines the previously used cross-cultural translation procedures into the following schematic description. This translation process is time-consuming and requires the researchers to carefully select

Adopting or Adapting Existing Instruments Adopting or Adapting Existing Instruments

Figure 4.1 Cross-Cultural Translation Process

(p.34) advisory committees and translators and to use different methods of translation evaluation to achieve quality cross-cultural translation of an instrument.

The five-step flowchart depicted in Figure 4.1 illustrates a comprehensive procedure for cross-cultural translation involving one target group.

Advisory Committee

The first step in the cross-cultural translation procedure is to form an advisory committee that will work with the research team to select an appropriate instrument or a survey questionnaire that can be used to collect data for the research project. This advisory committee should be comprised of professionals who are trained in the area of the research interest and who understand the nature and scope of the research problems or questions concerning the research population. The rationale for these selection criteria is straightforward—only individuals who are trained in the area of research interest can help the research team to select the appropriate instruments or measures for the research purposes. These individuals also need to be culturally and linguistically competent. If the research project involves more than two groups, then each group should have at least one female and one male expert representative serving on the advisory committee. This gender representative principle should apply for all translation activities. More than one expert in each group is called for to assure the diversity of voice and perspectives. Throughout the book, the author emphasizes the representation of both genders in all key instrument development because gender is a complex variable encompassing sex, socialization, identity, and communication (Cameron, 1988).

Forward and Backward Translation

The second step is to recruit and hire competent bilingual translators. The research team should work with the advisory committee to screen, hire, and provide adequate training on key aspects of the research project to the selected translators. For each linguistic group, there should be at least four translators, two females and two males, who work independently to perform translation and back-translation from the source (p.35) language (e.g., English) to the target language (e.g., Vietnamese) and vice versa. The translators must have adequate understanding of the research aims and key research concepts to perform a valid translation. Translators must be instructed to avoid verbatim translation but emphasize the comparability of the concepts and ideas between the source and target languages.

Evaluation

The third step is to evaluate the translation. The research team should employ multi-evaluation methods, including expert appraisal and review (evaluation committee), cognitive interviews, focus groups, and pilot testing. An independent group of experts or the same advisory committee should evaluate the translation and back-translation version of the translated instrument. This group should meet with the translators to clarify and verify their translations. The following evaluation matrix in Table 4.1 can be used for the expert group evaluation. The evaluation matrix allows the research team to collect both quantitative and qualitative evaluation information. Each evaluator is asked to respond “yes” or “no” to each evaluation criterion and to provide a brief explanation for the rating. Simultaneously, the team should conduct cognitive interviews and focus groups as the integral parts of the evaluation process.

Language Clarity

Committee members evaluate the use of words and syntax of the translated item and its back-translation.

Appropriateness

Committee members determine whether the translated items are culturally appropriate in both language and meaning for the target population.

Difficulty

Committee members determine whether the translated items are difficult for prospective respondents or participants to understand and to respond.

Table 4.1 Translation Evaluation Matrix

Table 4.1

(p.36)

(p.37) Relevance

Committee members determine whether the translated items are culturally relevant to the participants’ experiences in real-life situations.

Cognitive Interviews

As mentioned earlier, cognitive interviews should be used to evaluate the research questionnaire (Rothgeb, Willis & Forsyth, 2005). The cognitive interview is a form of in-depth interview that allows the prospective research subjects to express their feelings toward the research instrument in terms of its appropriateness, usefulness, and meanings and to explain how they understand the questionnaire and how they respond to the questionnaire. The research team works with the evaluation committee to recruit a representative sample of at least 6 to 12 prospective research subjects for each target group to participate in the cognitive interviews. The sample of size recommended here is similar to the sample sizes recommended for other types of qualitative research (Morgan, 1988). Think-aloud and verbal probing approaches of cognitive interviewing should be used concurrently. The interviewers begin the interviews by reading the translated item to the participants and asking them to say whatever comes to their minds as they listen to the item. Subsequently, the interviewers should use the criteria in the evaluation matrix (see Table 4.1) to probe for more information. Digital recording should be used as the means of data collection.

Interviewers must be trained to have a clear understanding of the purpose of the research project, the meanings of the research questionnaire, and its items. Interviewers must possess skills to probe for in-depth answers and encourage the participants to reveal their thoughts and feelings concerning the translated items of the questionnaire. Interviewers should probe the participants regarding four aspects of the quality of the translation: language clarity, cultural appropriateness, difficulty, and relevance. Clarity refers to the use of words and syntax of the items. Appropriateness refers to the suitability of the translated items to the research participants’ culture and values. Difficulty refers to the participants’ cognitive ability to respond or react to each translated item. Relevance refers to the connection of the translated items to respondents’ real-life experiences within their cultural context.

(p.38) Focus Groups

Focus groups can be conducted simultaneously with cognitive interviews as a part of the evaluation process. The research team should conduct at least three focus groups: one female group, one male group, and one combined gender group. The single-sex group allows a more comfortable atmosphere for the participants to bring up issues that are sex-related before they can be discussed in both-gender groups. Each linguistic or cultural group should have a minimum of six participants (Fayers & Machin, 2007). Having within- and between-gender-group and culture-group meetings sounds complicated, but they are desirable because these meetings will generate rich information necessary for a comprehensive assessment of cross-cultural equivalence. Group members should receive the complete translation of the questionnaire at least 1 week prior to the focus group meeting. Focus group participants are asked to review and discuss the quality of the translation using the four quality criteria presented in Table 4.1. All members must have an equal opportunity to discuss their ideas concerning the translation. The moderator must ensure the participation of each member during the focus group meetings. Discussions should be audio- or video-recorded. Each meeting should be no more than 2 hours, because the participants will be more likely to lose focus and interest in longer meetings.

Data Analysis and Synthesis

Evaluation data compiled from committee evaluation, cognitive interviews, and focus groups are transcribed and edited for an overall evaluation. Translators, advisory committees, and the researchers will work together to revise and refine the research instrument or survey questionnaire for a field pilot testing.

Pilot Testing

The purpose of the pilot testing is to establish the feasibility of the research instrument, evaluate the quality of interviewing methods, and assess the sources of missing data and other aspects of the implementation of the research instrument. Structured pilot interviews via survey (p.39) are conducted for reliability and validity evaluations. If possible, telephone interviews, face-to-face interviews, and mail surveys can be used simultaneously to improve the validity of the translation. A reliable and valid translated questionnaire should produce similar data regardless of the data collection methods. The sample size of the pilot survey test is determined by the size of the instruments and resources. From the factor analysis perspective, each item of a scale requires the minimum number of 5 to 10 subjects (Fayers & Machin, 2007). If the questionnaire consists of several standardized scales, then the team can use the scale with the largest number of items or questions as the guide to determine sample size. Random sampling is always a desirable method to draw a sample, but it is often not feasible. Therefore, the research team should make every effort to recruit individuals from diverse backgrounds of the target population to participate in the pilot test.

Finally, once data from the pilot tests are compiled and analyzed, the evaluation committee will review the results and work with the research team to finalize the translation. The research team will combine data collected from expert groups, cognitive interviews, focus groups, and pilot survey testing to finalize the research instrument for data collection. The process of translation is time-consuming and can be expensive. However, the benefits of having a valid and reliable translation outweigh the cost and time that the researchers have to spend to achieve the most desirable translated instrument. Researchers should continue to update and refine the translations of cross-cultural instruments to catch up with the changes of languages, culture, and communication.

Cross-Cultural Translation Issues and Biases

There are several issues and biases concerning cross-cultural translation, such as the quality of translated instruments and potential problems in using secondary data analysis for cross-cultural comparisons. These issues and biases often are the results of the attempts to replicate existing measurements that were developed in a source language such as English for studies among people whose primary languages are not English. Biases are also the results of pooling data from various sources for global comparisons. Some of the potential biases in cross-cultural translation (p.40) and analysis can be avoided with careful planning and implementation of translation procedures and analysis.

Multilingual Translation

When two or more linguistic groups are involved in a comparative study, the cross-cultural translation process becomes more complicated. The procedure illustrated in Figure 4.1 must be modified to achieve optimum translation outcomes. The composition of the “Advisory Committee” and the “Evaluation Committee” must include individuals from all participating groups. Members of these committees should be able to communicate through a common language. For example, if a researcher plans to study depression among Vietnamese and Russian immigrants in the United States, then the evaluation committee members are expected to be bilingual (English–Vietnamese and English–Russian).

Recruit and Train Translators

In addition to language proficiency, prospective translators are expected to have knowledge of the research field and the culture of the target population. They should be trained to have a good understanding of the research aims and the meanings of the research variables. Translators must also have good communication and listening skills to work with other translators and the research team. When there are no available trained and experienced translators, the team must design a special training program to prospective translators. This training program should include cultural characteristics, linguistic requirements such as terminologies related to the research topics, and substantive knowledge of the research area.

Recruit Subjects for Evaluation

Prospective research subjects or participants from the comparative cultural groups or communities are recruited to participate in the evaluation process of the translation. These individuals need to have good language skills in their own language and critical thinking ability to evaluate the quality and accuracy of the translation. They are people who can represent their own ethnic or cultural groups. It is expected that these (p.41) individuals have a good understanding of the research aims, the underlying purposes of the research instruments, and an ability to speak for themselves and on behalf of their communities.

Recruit and Train Interviewers for Translation Evaluation

Interviewers are recruited and trained to conduct cognitive interviews for the evaluation of the translation. They must go through rigorous training, including instruction on how to read the questionnaire clearly and probe for appropriate information, record and maintain quality data, and respect and protect participants’ confidentiality.

Verbatim Translation

When a researcher uses an instrument developed in the language that is different from the language of the target population, verbatim translation of the selected research instrument is not recommended. Equivalence of language translation often confuses respondents from a different culture. For example, in the CESD scale, there is the item “I felt that I was just as good as other people.” This item appears to be straightforward and should be easily translated to other languages, but it turns out that when it is translated into Russian and Vietnamese, the item has very poor reliability. Looking back on our translation process, we did not pay as much attention to this item as we did with other difficult item such as, “I felt that I could not shake off the blues even with help from my family or friends.” The translation team gave more effort in translating this item than to the item “I felt that I was just as good as other people.” To avoid verbatim translation of those items that appear to be straightforward, the translators should give equal attention to all items. In the case of the item “I felt that I could not shake of the blues even with help from my family or friends,” two groups of translators (three females and three males) worked independently to translate the difficult items of the questionnaire. The two groups met with the Principal Investigator (P.I.) to have an open discussion on the similarities and differences of the group translation to arrive at a consensus. The group discussed meanings of the difficult items in English and their translated meanings in Vietnamese. The final translated items did not have language equivalence but did have cultural and conceptual equivalence. Table 4.2 demonstrates the reliability (p.42)

Table 4.2 Corrected Item–Total Correlation of “Shake off the Blues” and “As Good as other People”

Vietnamese

Russian

Shake off the blues

0.634

0.675

As Good as Other People

0.013

0.044

of these two items in the Russian and Vietnamese sample. Differences in “Corrected Item–Total” correlation between different cultural groups is a sign of poor cross-cultural comparability (van de Vijver & Leung, 1997).

Single Translator

The use of a single translator is not recommended because of the lack of validity and reliability checks for the quality and accuracy of the translation (Maneesriwongul & Dixon, 2004). This bias becomes even more serious when the researcher has no language skills and is foreign to the target population.

Translators Without Appropriate Training Background

Translators who have no training background relevant to the research questions will not be able to perform valid and reliable translation of the research instrument. For example, a translator who is fluent in the language of the target research population, but who has no training in public health or social work, would not be able to produce a quality translation of the research instrument designed to collect data on health and mental health problems.

Failure to Evaluate the Validity and Reliability After Translation

One should never assume that if an instrument has been well-developed for one particular group, it is sufficient to translate it and replicate it in another group without a reaffirmation of its validity and reliability. It is recommended that researchers always evaluate the factor structure or configuration of the scale and its internal consistency once it is translated to the target language.

(p.43) Inconsistency of Translation Procedure

When cross-cultural comparison is performed on a variable that has been translated from one original language to different target languages, one should be cognizant of the consistency of the translation procedures. Inconsistency of the translation procedures will impact the equivalence of reliability and validity of the measurement of the research variables, and the results could be severely biased. Table 4.3 illustrates the differences of internal consistency as the result of inconsistent translation procedures. Each of the studies presented in Table 4.3 employs different procedures of cross-cultural translation.

The statistics presented in Table 4.3 suggest that the three items of the CESD Scale have somewhat weak cross-cultural equivalence. The “depressed” item has cross-cultural equivalence between the Chinese and Russian samples. The “I felt lonely” item has a similar corrected item–total correlation between the Russian and Vietnamese samples. The “I felt sad” item also has a similar corrected item–total correlation between the Russian and Vietnamese samples. When we examined the equivalence of Cronbach's α-coefficients across the four samples, the only two samples that exhibited similar Cronbach's α-coefficients are the Chinese and the Japanese. Nevertheless, the corrected inter-item correlation of the “I felt Sad” item appears to be different between the two samples.

Different reliability statistics presented in Table 4.3 could be either cultural, methodological, or both. Cultural variations are more difficult to recognize than methodological variations. From the methodological perspective, these studies employed different translation procedures. The Japanese survey used “an extensive translation, back translation, retranslation, and pilot testing process in order to ensure a very high degree of comparability,” (Sugisawa et al., 2002, p. 791; Liang et al., 2005).

Table 4.3 Corrected Item–Total Correlation of Three CESD Items

CESD Items

Chinese (n = 175)

Japanese (n = 2119)

Russian (n = 299)

Vietnamese (n = 339)

I felt depressed

0.587

0.483

0.591

0.790

I felt lonely

0.472

0.565

0.637

0.674

I felt sad

0.507

0.524

0.712

0.710

Cronbach's α

0.697

0.705

0.798

0.853

(p.44) The Russian and Chinese surveys used one professional bilingual translator, committee evaluation, and pilot testing to ensure comparability (Tran, Aroian, Balsam & Conway, 2000; Wu, Tran & Amjad, 2004). Overall, the translation procedure of the Japanese survey was more desirable than the Russian and Chinese surveys. The Vietnamese studies used a similar approach of translation as the Japanese survey; however, it emphasized the use of equal numbers of female and male translators to avoid gender biases in translation (Tran, Ngo & Conway, 2003). These statistics suggest that one should be cautious in using the composite scores of these items for the purpose of cross-cultural comparisons, especially when one compares the meaning of these items across groups.

Inconsistency in Research Designs and Data Collection

When pooling data from different studies for statistical comparisons, researchers must account for the variations of designs and data collection methods. The results in Table 4.3 also suggest that the variation of study designs can affect the reliability and validity of the research instrument.

Cultural Variation Within the Same Linguistic Population

Researchers could be mistaken in assuming that language is the only marker of cultural similarities or differences. Sharing the same language does not warrant an equivalence of measurement. Table 4.4 presents the corrected item–total correlation of the five items that were designed to measure “negative” feelings or negative well-being among three groups of Hispanic elderly individuals (Davis, 1997). Although the respondents are members of three ethnic groups that share similar languages (Spanish and English), the statistics presented in Table 4.4 suggest that the selected items of negative well-being exhibit variation in reliability.

The statistics presented in Table 4.4 indicate that there is a variation of reliability of the items across the three ethnic groups that share similar languages. These three major groups of Hispanics do not share the same culture. They have different immigration histories and are very diverse in terms of family values, religion, and socio-economic backgrounds (Bean & Gillian, 2003; U.S. Census Bureau, 2001). This suggests that researchers should not assume that similarity of languages is equivalent to similarity of measurement. It is always a good practice to verify (p.45)

Table 4.4 Corrected Item–Total Correlation of Five Bradburn's Negative Items

Negative Items

Survey (n = 2235)

Cuban (n = 692)

Mexican (n = 757)

Puerto Rican (n = 358)

Restless

0.480

0.432

0.459

0.570

Lonely

0.526

0.451

0.540

0.528

Bored

0.532

0.430

0.523

0.598

Depressed

0.574

0.536

0.566

0.648

Upset

0.294

0.243

0.292

0.261

Cronbach's α

0.722

0.665

0.718

0.755

the equivalence of the measurement of the variables used among the comparative groups, even if they share a language.

Different Language of Interview Within a Cultural Group

Language of interview can influence the reliability of an instrument even if it is used within the same cultural group. Respondents of The National Survey of Hispanic Elderly could be interviewed in English or in Spanish. A small number of respondents chose to be interviewed in English (n = 308), and the majority were interviewed in Spanish. It should be noted that the survey instrument was developed in English and translated into Spanish. Table 4.5 contains the reliability analysis of five negative item scales.

Although three of five items appeared to have similar corrected item–total correlation, the item ”Restless” had a poorer correlation in the sample of English interviews, whereas the item “Upset” had a poorer

Table 4.5 Corrected Item–Total Correlation of Five Bradburn's’ Negative Items between English and Spanish

Negative Items

English Interviews (n = 308)

Spanish Interviews (n = 1991)

Restless

0.369

0.496

Lonely

0.580

0.518

Bored

0.524

0.533

Depressed

0.523

0.581

Upset

0.370

0.284

Cronbach's α

0.722

0.724

(p.46) corrected item–total correlation in the sample of Spanish interviews. Researchers can control for this problem by making sure that interviewers in different languages follow the same protocols. Languages of interview can be considered as a covariate in multivariable analysis, such as multiple regression analysis or other multivariate statistical procedures.

This chapter provides an overview of existing translation procedures and illustrates a multilevel translation process and procedure. The author used existing cross-cultural data to demonstrate potential biases of cross-cultural translation. The quality of the translation of adopted or adapted instruments is determined by its reliability and validity. The researchers can use the pilot evaluation data to decide whether an instrument is ready to be implemented. The techniques to assess the cross-cultural reliability and validity of the research instruments will be explained and illustrated in Chapter 6 and 7. Chapter 5 is devoted to the process of developing and constructing new cross-cultural research instruments.