In the 1970s, there was a television series called Columbo. This was about a detective played by Peter Falk, and the detective was famous for being a little bit rumpled, he wore a overcoat no matter what the weather was, and he was very absent-minded. But he was also an incredibly shrewd detective, and one of the things about the show was that almost more than half of every show was basically him interviewing or what was likely to be the murder suspect in each episode, and eventually getting to a point where he was able to extract the information he needed from the suspect to the point where he could actually find enough evidence to convict the person. Now, what's interesting about Columbo was that almost in every episode, there was a point at which he seemed to be over with the interview and then he would say, "Oh, by the way, just one more thing," and he would ask some critical question that really caused the entire thing to unravel for the murder suspect. What's interesting about Columbo is not so much that we would necessarily want to imitate his style, but he was very good at combining the science of interviewing with the art of interviewing. He did a lot of preparation for each of his interviews and he did all of the background things that detectives need to do to make sure that they have the evidence that they need, but at the same time he was very skilled at establishing rapport with the person he was interviewing, and basically finding out different ways in which the interviewee would give up important information. So, that's exactly the kind of thing that we'd like to do also in our user needs assessments. Now, interviewing is actually both a science and an art. On the one hand, there's a science of actually understanding how to put together an interview protocol, how to ask certain kinds of questions, but it also requires a certain amount of improvisation during the interviewing process itself. In this video, we'll be focusing primarily on the science of the interview protocol. In a later video, we'll talk about how to actually adjust and to improvise during the interview itself. One of the things that you want to think about is what kind of questions you want to ask, and perhaps, the biggest issue in semi-structured interviews is to make sure that you're asking open-ended questions as opposed to close-ended ones. So open-ended questions are those where you leave enough room for the interview participant to provide a lot of interesting detail. You want to ask how and why questions. Why did you do a certain thing? How did you do a certain thing. Instead of yes and no questions like, do you like certain feature X? Similarly, you can ask things like, how did you go about doing this the previous time you engaged with the product? As opposed to, how many times a week do you do a certain activity? Because that's, again, a question in which you'll have a very short answer. You can also ask, can you tell me about a specific instance? And that's, again, different from, what is your favorite color, for example, where, again, the answer is going to be very short. So throughout the idea with open-ended questions is to ask questions in which you really engage the interview participant in a way that they feel compelled to tell you as much as they might feel about a particular product or service. So one thing you'll notice is at the top of this slide, I've labeled open-ended questions to be good questions and close-ended questions to be not-so-good questions. That's because close-ended questions aren't necessarily bad. There will certainly be times when you want to ask closed-ended questions and get at short answers, but for the most part you want to ask open-ended questions that encourage the interview participant to reveal more detail and that'll allow them to tell a little bit more of a story about their interaction with the product or service. Another thing about the questions that you want to put in your interview protocol are that you want to get at specific and concrete information as opposed to abstracted and generalized information. Now, this might not make immediate sense. After all, eventually, what you want to do is come up with abstracted and generalized knowledge that you can use as a way to design future products or services. However, your job as the researcher is actually to do this abstraction and generalization from the more concrete and specific information that interview participants give you. So, during the interview, what you really want to do is focus on as much concrete, specific information as you can. One way to do this is to ask questions about a specific instance when the interview participant has used a product or service. So you can ask questions like, tell me about the last time that you did X, or can you think of a specific instance when you did this and can you tell us more about that? You want to avoid questions where you necessarily ask them, what do you normally do when-? or what is the typical way in which you do something? Because what this does is it causes the interview participant to generalize from their experience and sometimes, people are not very good at generalizing accurately. Again, the generalization is your job as a researcher, and so what you really want to do is start with the concrete, specific information. Now, sometimes it's useful to ask the generalized question towards the end of a section of the interview because people do often have ideas about why they do certain things and why they tend to do them on a regular basis. But again, you want to make sure that you're also covering the specific and the concrete as well. Again, the question is not between good and bad questions, but questions which are good as well as questions which you might allow yourself to ask but which you'd like to suppress in favor of spending more time on the more concrete and specific questions. A third issue with questions in an interview protocol is to try to think of ways in which you can ask questions that are not leading and which are not judgmental. So what do I mean by this? Non-leading questions are questions where you don't express any kind of opinion about whether the different ways in which the interview participant might answer are a good or bad. You might ask, what option do you prefer, as opposed to saying, do you prefer X over Y? Because that suggests maybe that you're expecting them to prefer X over Y. You also might want to ask, what do you think was the reason for a certain kind of behavior, as opposed to, was it because of a bad process that the task was difficult? The second question actually seems to suggest that the task itself was difficult and there was something wrong with it, and so you're encouraging people to think something that they may not necessarily believe without their prompting. Another category is, how did you try to address a particular problem, as opposed to, why didn't you use feature Z? The problem with the second question again is that it's a little bit judgmental. It's suggesting maybe that the interview participant should have known that they should have used that feature and that can sometimes make them defensive, which is not what you want during the interview process. To the extent possible, you want people to feel extremely open and free, and to connect with you in such a way that they can tell you things that they might not otherwise tell other people. The other thing you want to think about in an interview protocol is the overall organization and flow of questions. Now, for a one hour interview, I would suggest that you want to prepare something between about six or ten questions. If they're good open-ended questions, you'll find that even ten questions is a lot. Then what you want to do is think about how those questions are organized. You'll want to put questions that are about similar themes together so that it allows the interview participant to maintain their stream of thought, and then within each clustered theme, you want to organize them so that they make some sense. Maybe the questions are arranged in some chronological order with respect to a process, they might have a larger narrative in mind, or they might go from, let's say, broad questions to narrow questions or vice versa. Similarly, you'll want to take the existing clusters and organize them in some way that makes sense. Again, you could organize them in chronological, narrative or other ways. One thing that I recommend is also saving any sensitive questions or questions where you anticipated that the interview participant might become a little bit uncomfortable towards the end, so that you can maintain the momentum with the interview, and by the time you get to those sensitive questions, the interview participant is accustomed to giving you answers that are interesting to you. Once you have the core questions, you'll want to add follow-up questions to each of them. It's useful if you organize these with bulleted lists where the follow-up questions occur underneath each core question. You can ask different kinds of follow-up questions where you're asking for more detail as well as prompts for specific information. Sometimes, you can anticipate that users have challenges with certain aspects of a product or service but they may not necessarily mention them the first time they answer one of your open-ended questions. If that happens, you'll want to make sure that you have follow-up questions where you specifically ask about the feature X or about service Y so that you can make sure that you're still getting all of the information you need for your overarching question. Sometimes, it also helps if you repeat certain questions. So for example, if a person tells you about one instance when they used a product, you might say, well, can you tell me about another instance, maybe the time immediately before that, and so on. So these are the kinds of follow-up questions you'll want to ask. It's really useful if you think through the kind of responses that you might get to one of your core questions and then design your follow-up questions to be in response to those kinds of answers. It's useful also for your follow-up questions to be ordered in a certain kind of way and, again, you'll probably not ask all of the follow-up questions that you list, but having them there is very helpful during the interview itself. Some final thoughts about the interview protocol. Again, remember that the goal of the protocol is for you to prepare and for you to have a good sense of what the interview is going to be about. Writing out the questions is very useful because during the interview itself, you have to think about a large number of things and you'll need to improvise. Just like with playing music, if you've memorized the basic fundamentals, then it's much easier to improvise later on. So one thing I recommend is for you to memorize the overall flow of the protocol. You don't have to memorize the wording of every question but you do want to make sure that you understand what questions are there. An ideal interview takes place in such a way that you don't even have to refer to your protocol during the interview itself because you understand the protocol so well. Again, let me remind you that you will not be asking all of the questions in the protocol one after the other. That's more like a survey. Again, what we're trying to get at is a conversational style during the interview process itself so that your interview participants feel comfortable and relaxed, and are open to providing the information that you're seeking. Finally, this video comes with some handouts that you'll want to take a look at for how to design interview protocols. It also contains templates for the parts of the interview protocol. They are pretty standard such as the introduction and the conclusion. You can, of course, edit them and revise them in a way that fits your tone and the way that you speak.
