[MUSIC] I want to talk now about some key considerations in questionnaire design. In fact, questionnaire design is as much of an art as a science. Most students learn questionnaire design not so much from classes but by actually joining ongoing research projects. In which questionnaires are being designed and then looking at them and learning about the process in a hands on way. That said, I can at least outline a few key principles to keep in mind if you design a questionnaire. One is that you need to minimize the amount of thinking that a respondent has to do to answer the question. Basically, as much as possible, the questions should be clear, concise, short. They should not require additional explanation. Hopefully the respondent, upon hearing the question foreseeing it will immediately know exactly what is needed from them without further interpretation or explanation. Each item in a questionnaire, that is each question should only seek to measure one underlying dimension. Don't mix apples and oranges. So if you are interested in collecting information about age. Then only ask about age. If you're asking about some attitude about a particular topic, the question should not refer to any other topics. You also want to avoid too much reading or listening. So it may be that respondents will tune out If they have to read too much or listen to too much, there is a few special exceptions, but for the most part the questions again should be short. Make sure that the questions are relevant to the respondent's education, experience, knowledge and so forth. Ask questions that the respondent is capable of answering. So one particular problem that I've seen is that sometimes, people that are new to questionnaire design, they may ask respondents not about their personal situation, but about their opinion of what people like them feel. So rather than asking a respondent, what they think of a particular social topic. They may ask a respondent what people like them think about that social topic, in other words, asking the respondent to speak on behalf of some larger group. Now, that's not normally a useful approach, because a respondent may actually not know what other people that make up their group are actually thinking. More technical issues, that questionnaires typically use skip patterns. To ensure that respondents are only asked questions, that are specifically relevant to their experience. So in a survey, that includes a sample that has both men and women, if there are questions about previous pregnancies normally a skip pattern would be used according to which men who complete the survey would be directed away from answering questions about pregnancy because obviously they wouldn’t be able to answer them in a meaningful way. There's also some issues that come up in the ordering of questions, some general principles. The ordering of questions is important. Most surveys begin by putting the most straightforward questions about non-sensitive issues, at the beginning of the survey. This also helps build some trust or rapport with the interviewee because the questions are fairly innocuous and straightforward. Questionnaires then therefore begin with collection of information about social demographic or economic characteristics of the interviewee. And then questionnaires save the most sensitive questions for the end of the survey. By which time, hopefully, the interviewer has established a relationship with the interviewee and the interviewee has opened up a bit become more trusting. The ordering of questions is especially important when it comes to attitudinal questions, for example on opinion polls. We know that the ordering of questions about attitudes for particular topics may influence the results that the respondents give. I want to talk a little bit about online surveys. They've gotten a lot of attention. Now, you've probably, at some point seen some online survey since you've been browsing the web. You may have been invited to complete an online survey. Again quite often, they are perhaps used more for entertainment than for anything else. Perhaps a news website may invite readers to complete a very brief survey indicating their attitude about a particular poll. Now, online surveys do seem attractive in that they promise to allow the collection of an enormous amount of information very quickly. So we see sometimes online surveys with hundreds of thousands, or even millions of responses. That come in very quickly. The problem is that we really don't know how representative an online population is, whether you can generalize from anything that you learn online to some other real population out there in the world. You think about what kind of people spend a lot of time on the internet and might even have the time free to complete an online survey. There are proposed solutions especially when it comes to academic research involving online surveys, involving re-waiting and so forth. So, for example if an online survey Includes questions about the demographic characteristics of the respondent, their age, their sex, their race, and so forth. People have suggested that these can be used to reweight the answers to make the respondents on the online survey, the data representative of some larger population. Again, that's controversial, because online users, even if you are able to re-weight the numbers to make it appear representative of the larger population, the people that are online may still be different from the quote real population in important ways. What's most promising, especially for academic research, are hybrid approaches in which subjects perhaps are initially contacted in person, or by mail and perhaps even interviewed and then follow up is conducted online. Or subjects are recruited with a piece of mail from an official address perhaps especially for a government survey that explains the survey and then gives a web address for the respondents to go to complete the survey. There, because we have some control over the sample, we can select the people we invite to participate in the survey according to traditional sampling techniques. And then the online portion is basically a convenience for the people that have been selected and agree to participate in the interview, then we have a lot less problem with representativeness, than an online survey that consists of simply putting up a web page, and then asking people to click on a link to complete a survey, again, with no attempt at traditional sampling. So, this sort of online hybridization is especially important and potentially useful for longitudinal surveys. So, there are longitudinal surveys that are hybrids, where perhaps at regular intervals, perhaps a few years apart, there are in person surveys where interviewers go to visit people at their homes, and they have long questionnaires but maybe takes several hours to complete. But then in the time between those in person visits, the respondents are invited to complete, perhaps brief update questionnaires online. Perhaps providing information about recent activities like employment or their relationships status. So again, this integration of detailed data collection in person and then more sporadic and less detailed data collection online in between those in person visits is a very promising approach for the future of longitudinal surveys. So hopefully, I've at least introduced you to some of the basic considerations the practical considerations you'll have to keep in mind if you're actually going to conduct a survey. Assuming that you figured out how you're going to do your sampling and you need to start thinking about how you're going to design your questionnaire and what sort of format the survey will be in.
