Best Practices For Questionnaire Design: Tips And Guidelines

What is a questionnaire?

Describe about good practice in questionnaire design?

A questionnaire is an instrument for conducting research. The instrument contains a set of questions which prompts a person to collect information from the respondents. The questionnaire is designed for the purpose of generating a statistical analysis of the responses. The questionnaire though can be used to generate other data also (Alexander, Entwisle & Dauber, 2003).

It was emphasised by Brace (2004) that the importance of the questions is successful for the communication process particularly in conducting a marketing research. The questionnaire should be in tune with the respondents who different in terms of language, age, gender and education. The questions must be so designed such that the there is no negative responses or non-responses from the respondents (Full-Length Donor History Questionnaire, 2015). It should be seen that the respondents do not misinterpret or lie while answering the questions.

A good practise method of the questionnaire improves the quality of the questionnaire to a great extent and it helps to avoid the pitfalls. This paper will try to explain the good practices of that can be used to design the questionnaire in a sequential manner (Mymarketresearchmethods.com, 2014). It is to be kept in mind that if the questions are not carefully designed then the best method to gather, and analyse the data later in the research will not be able to compensate the bad design. This can lead to the questions not being useful for the purpose of the survey (Yuen).

According to a research undertaken by Holbrook et al (2006) indicated that the questions should be kept as short as possible. This should be done so that the respondent’s comprehension increases. It was said by Brislin (1986) that the maximum number of words in the question should be 16 while another work by Oppenheim (1992) recommends that the number of words should be 20 per sentence and the question can contain more than one sentence (Jurges, 2015). According to a report by Jabien (1987) it was said that longer questions can generate more accurate report as they try to convey the notion that the task has some importance and that it deserves some serious effort.

The grammatical complexities of the questionnaire should be kept to a minimum so that the respondents are able to answer to the questions appropriately. The questions should be framed in the active voice rather than the passive voice. There are certain guidelines like the use of pronouns should be less and nouns should be used as much as possible. These guidelines should be followed so that the mental capacity of the respondents is freed and they can think about the answers.

The cognitive load on the respondents can be reduced by using specific terms instead of general terms. The more complex questions should be broken down into simpler questions illustrates a certain behavioural illustrations. The use of vague words such as maybe, perhaps, probably should be avoided. A study by Foddy (1993) and Belson (1981) says that the use of hypothetical questions in the questionnaire should be avoided. Since it tries to find the complex behaviours of the respondents the researcher should not give any hypothetical questions (Research & Consultation Guidelines -Questionnaires, 2015). It is recommended that alternative scenario and vignettes should be used to generate the responses which are external to the realm of the present or the past.

Question length

A double barrelled question is that the statements or the questions contain two types of verbs or two different types of concepts. It was said by Brislin (1986) that it is detrimental to obtain a accurate response if the researcher uses a double barrelled questions which uses the use of two concepts in the same question. Examples of double barrelled question “do you find time to go through the newspaper every day?” here it is seen that the question have two separate statements such as “does the person have the time” and the second question is that “does the person goes through the newspaper everyday” (Survey Research, 2015). In this case it is seen that the statement has two separate ideas put in one statement. It can be seen that the researcher is not sure about which area he wants to gather the information on. The researcher may not be aware of the fact that the question is a double barrelled question (Formulating Survey Questions, 2015).

It should be kept in mind that the questions should not be poorly worded and it poses a threat on the respondents; as said by Foddy (1993). He also adds that by using too many difficult words in the questions or in the instructions can make the respondents feel that they are either uneducated or stupid. This can often lead to respondent not answering to the question. Often it is seen that the respondents feel a social barrier in responding to certain type of questions such as whether they smoke or drink. The respondents often choose to answer the question which is favourable to the society. Foddy (1993) also states that there are certain issues like medical diagnosis related to the mental or genital diagnosis often leads the respondents in giving a false answer (Kaminska, 2015). It is done so as to avoid any embarrassing situation or to avoid being recognised in the society.

According to Brace (2004), the researcher should use indirect ways to generate the information from the respondents so that they feel free to answer sensitive issues such as crime, drug use or adultery. A question can be asked like “what do you believe other people might think about the use of drugs?” in this type of questions, it is assumed that the respondents answer their own views by putting it others mouth (Leitz, 2015). Brace (2004) also points that the respondents’ propensity to give more desirable answers is by asking them question that will familiarize him with the situation. This can be done by using an introductory question like “are you aware of the....?”

The use of the negative words in the questionnaire should be avoided since those type of questions take a longer time to process. The use of negative words often increases the chances of making mistakes for the respondents. Foddy (1993) said that the use of the words like ‘no/not’ often leads to the confusion for the respondents (Colosi, 2015). For example a question like- “what do you think that the conservationist must not be uncooperative with the government?” this type of questions should be avoided. In place of such questions it should be given that “should the conservationist be more cooperative with the government?” this will reduce the chances of the respondents to spend more time assessing the answers of the questions. It was often seen that the respondents have the tendency to indicate to a ‘no’ option rather than indicating to a ‘yes’.

Grammar

The order of the question is an important thing in a questionnaire. The cause problems and often threatens the truthfulness of the results. The pattern or the order of the question may affect the whole layout of the questionnaire or may often affect the part of the questionnaire. The relevant questions should be put together or serially one after the other. There are a set of general questions and a set of specific questions (Leung, 2001). It is seen that the general question is more suitably placed before the specific question. It is such because the specific question is aimed at finding out the more detailed aspect of the study being undertaken. For example questions like “the degree of happiness in general life” and “the degree of happiness in the marriage”. Here the previous question is the general question and the second part of the question is the specific question which is aimed at finding out the specific range of happiness in being married (Leitz, 2015).

It must be noted that the demographic questions like the name, age, education qualification, marital status and the income level of the respondents should be given at the end of the questionnaire. According to Gaskell et al. (1994) this should not be given at the beginning of the questionnaire so that the respondents do not feel negative about the disclosure of the personal information which in turn can impact the answering behaviour of the respondents (Colosi, 2015).

The responses are the second most important area to be considered. The responses include the answer given by the respondents. The responses generated are based on the various categories or the scales that are being used as part of the research. It is also the work of the researcher to encode the answers given by the respondents (Leitz, 2015).

It is the part of the researcher to find out whether the respondents will be able to answer all the questions; or the respondents who have little knowledge should be scanned out and they are not asked the same questions. The problem arises when the interviewer asks the respondents questions in a face to face interview or over a telephonic interview. The interviewer often finds it difficult whether they should include a “don’t know” option in the questionnaire (Ro.uow.edu.au, 2015). In a study conducted by Schumann and Pressser (1996), it was seen that the “don’t know” option has increased from 22% to 25% when the questions were asked explicitly. The questions with which the respondents were less familiar were more prone to answer the “don’t know” option and vice versa. When the questions were not explicitly answered the answers shifted from the “don’t know” to either agree or disagree (McColl, Thomas & Jacoby, 2015).

Some of the responses of the questions should try to achieve the opinion of the respondents. In questions like “what is your opinion about the new policy?” would generate a specific opinion of the individual respondent. It can also be seen from these questions that whether the respondent is ignorant about the responses (Krosnick & Presser, 2015).

Simplicity and specificity

It was stated by a number of authors that a 5-point or a 7-point scale is used to generate the responses and are most commonly used. It was said by Dawes (2008) that by using those scales it is easier to find the responses as the scales can be rescaled and it facilitates comparison. The 7- point scale is more frequently used than the 5-point scale, as it offers more differentiation of the responses and is more reliable. It was also seen that with the usage of a more scaled responses, the degree of getting a satisfactory answer also increases. It is also used when an abstract response is to be generated from the respondents.

The researcher has to decide whether he should include an even set of options or an odd set of options while setting the questionnaire. Giving an odd number or responses gives the researcher an option to indicate to the “neutral” option which lies in the middle of the answer set. The neutral option lets the respondent not to indicate or commit to a favourable option (Wording for rating scales, 2015). Researchers have shown that the middle option attracts 6-23% of the respondents when they are given the chance. It was also seen that the responses shifted to the “don’t know” when the middle option was removed. It was also seen that the options such as “somewhat liberal” or “somewhat conservative” drew more attention even though the middle “neutral” option was not provided (Instructional Assessment Resources, 2015).

The problem that most of the researcher faces is that, whether they should use a unipolar scale or to use a bipolar scale. The unipolar scale uses scale marking from 0 to 10 and the bipolar scale uses +5 to -5 options in the scale. There are other scales such as the verbal scales which includes options like, strongly disagree, disagree, neither disagree nor agree, strongly agree, and agree (Menold & Kaczmirek, 2015). It was seen in a research study that the respondents have the tendency of choosing  scale of more than 5 in the unipolar scale whereas a positive rating in the bipolar scale.

Foddy (1993) has shown in a study that a number of response options effects arise in the questionnaire’s responses. Those effects can be recency or primary effects. There are other effects like the effect of the shifting frame of references of the individuals. The primary effects are the assumptions that the individuals will select the earlier options in the questionnaire while the recency effect is that the respondents will select the later responses of the options being provided (Booth, Colomb & Williams, 2008). The shifting frame of references refers to the psychology of the individual and depends on whether he has indicated to an option earlier or not (Pasmore, Woodman & Shani, 2010).

Conclusion

Researchers who are responsible for making the questionnaire will find the above considerations to be useful. At first the objectives of the study must be laid out. The questionnaire will need to review the previous researches made in the past (Stone). After the grammar, language, simplicity of the questionnaire is maintained, the task of the researcher is to hold the research process and measure the success of the questionnaire as well as the research.>

References

Alexander, K., Entwisle, D., & Dauber, S. (2003). On the success of failure. Cambridge, UK: Cambridge University Press.

Booth, W., Colomb, G., & Williams, J. (2008). The craft of research. Chicago: University of Chicago Press.

Colosi, L. (2015). Designing an Effective Questionnaire (1st ed.). Retrieved from https://www.human.cornell.edu/pam/outreach/parenting/research/upload/Designing-20an-20Effective-20Questionnaire.pdf

Colosi, R. (2015). Negatively Worded Questions Cause Respondent Confusion (1st ed.). Retrieved from https://www.amstat.org/sections/SRMS/Proceedings/y2005/Files/JSM2005-000508.pdf

Formulating Survey Questions. (2015) (1st ed.). Retrieved from https://www.crsllc.org/assets/resource-3-formulating-survey-questions.pdf

Full-Length Donor History Questionnaire. (2015) (1st ed.). Retrieved from https://www.fda.gov/downloads/BiologicsBloodVaccines/BloodBloodProducts/ApprovedProducts/LicensedProductsBLAs/BloodDonorScreening/UCM213552.pdf

Instructional Assessment Resources. (2015) (1st ed.). Retrieved from https://www.utexas.edu/academic/ctl/assessment/iar/teaching/plan/method/survey/responseScale.pdf

Jurges, H. (2015). Interview, Module and Question Length (1st ed.). Retrieved from https://www.share-project.org/uploads/tx_sharepublications/Methodology_Ch8.pdf

Kaminska, O. (2015). Understanding Sources of Social Desirability Bias in Different Modes: (1st ed.). Retrieved from https://www.iser.essex.ac.uk/research/publications/working-papers/iser/2013-04.pdf

Krosnick, J., & Presser, S. (2015). Question and Questionnaire Design (1st ed.). Retrieved from https://pprg.stanford.edu/wp-content/uploads/2010-Handbook-of-Survey-Research.pdf

Leitz, P. (2015). Research into Questionnaire Design (1st ed.). Retrieved from https://www.cse.edu/dotAsset/134306.pdf

Leung, W. (2001). How to design a questionnaire (1st ed.). Retrieved from https://www.dpcdsb.org/NR/rdonlyres/138975AC-B110-4D1E-902F-81C8E69BF9A0/107060/How_to_design_a_questionnaire.pdf

McColl, E., Thomas, L., & Jacoby, A. (2015). Design and use of questionnaires: a review of best practice applicable to surveys (1st ed.). Retrieved from https://www.journalslibrary.nihr.ac.uk/__data/assets/pdf_file/0006/64833/FullReport-hta5310.pdf

Menold, N., & Kaczmirek, L. (2015). A literature review on constructing answer formats (1st ed.). Retrieved from https://file:///C:/Users/User/Downloads/Contribution258.pdf

Mymarketresearchmethods.com,. (2014). Survey Design Best Practices. Retrieved 7 March 2015, from https://www.mymarketresearchmethods.com/survey-design-best-practices/

Pasmore, W., Woodman, R., & Shani, A. (2010). Research in organizational change and development. Bingley, UK: Emerald.

Research & Consultation Guidelines -Questionnaires. (2015) (1st ed.). Retrieved from https://www.kirklees.gov.uk/community/yoursay/Questionnaires.pdf

Ro.uow.edu.au,. (2015). Questionnaire design and systematic literature reviews. Retrieved 7 March 2015, from https://ro.uow.edu.au/cgi/viewcontent.cgi?article=1124&context=ahsri

Stone, D H. 'Design A Questionnaire.'. BMJ 307.6914 (1993): 1264-1266. Web.

Survey Research. (2015) (1st ed.). Retrieved from https://www.sagepub.in/upm-data/54137_Chapter_8.pdf

Wording for rating scales. (2015) (1st ed.). Retrieved from https://www.uwex.edu/ces/4h/evaluation/documents/Wordingforratingscales.pdf

Yuen, Raymond Wai Pong. 'Question Design Tips For Questionnaire'. SSRN Journal n. pag. Web.