Participating in a user test can be a strange and sometimes stressful situation for some of your participants. When you're hosting or moderating a user test, it's your responsibility to set a tone that will help your participants feel comfortable. One of the main things that you're trying to accomplish in setting a comfortable tone is establishing rapport with your participants. This basically means you want to establish a relationship where the participants feel like they can trust you and they want to share information with you and that they can perform naturally in front of you. Some of the things that you can do to help establish that rapport, is first of all, be enthusiastic, have a positive energy when greeting your participants and when introducing them to the test. Show some excitement about what you're doing to show them that their participation is valuable to you. You also want to be human. You want to get to know them. Ask them some questions. Ask them how their trip into the meeting space was. Ask them if they need a drink of water or a cup of coffee or anything else that you might have to offer. When describing different aspects of the user test protocol as you step through it, be as clear as possible so that they understand what it is that you're asking of them, what it is that you're hoping to get out of it, and what it is that you want them to do. After you've greeted your participant and gotten them water and help them to get comfortable, you want to go through a few things to just get the test going off on the right foot. You want to introduce yourself. Perhaps you've had interactions with them over email and things like that, but they may not remember you or who you are or what it is that you're trying to do. If there's anybody else attending the session, maybe somebody that's helping to you run the equipment, or do the recording, or helping to take notes, introduce them as well. It's pretty weird to have other people sitting in the room that haven't been introduced to the participant. State the goals of the test. Again, you may have communicated this previously through an email or through a phone call, but it's good to go through it again so that they remember what it is that you're trying to do. Explain your role and the roles of other people in the room so that they understand what it is that you're doing and also explain their role. So, what is your role anyway? Well, as the moderator of a user test, you are facilitating the evaluation. It's really the participant who's performing the evaluation of the system and you're really there to make sure that the evaluation goes smoothly. You are observing the test. So, you can take the data from their evaluation and feed it back into the design of the system. You are interested in an honest assessment of the system and this is important to communicate to the participant. So, what is the participants role? Well, they are helping you evaluate the system. In a sense, the two of you together are testing the system. You're on the same side and you're both trying to figure out if the system works for what they need to do or not. Their participation is entirely voluntary. They're there to help you out and it's important to remember that that is their role is as a helper to you. They are not being tested and we'll come back to this. This is very important to communicate to them. This is not a test of their ability. This is a test of the system. Because this is voluntary, they can stop at anytime and they can decline to answer any questions or perform any tasks that you ask of them. Let me say that last part again. Their participation is entirely voluntary. They are not being tested. They can stop at anytime and they can decline to answer any questions or perform any tasks. It's very important to keep in mind that participating in a user test can be frustrating and stressful in some cases. People may end up feeling stupid, or embarrassed, or angry. You need to do everything you can to make sure that they don't feel like they're being tested that their abilities and their knowledge isn't what's being evaluated, but it's the system that's being evaluated and they're helping you do it. An important aspect of conducting a user test is making sure that participants know what it is they're being asked to do and that they've agreed to participate based on that understanding. The principle of informed consent is a basic ethical principle of research that involves human subjects which is what you're going to be doing if you're conducting a user test. This principle dictates that participants need to know the purpose of the study, what the study consists of, what their rights are as a participant, and what you will be doing with the data that you collect about them. As a test moderator, you need to make sure that they understand all of this, and in many cases, you're going to need to actually document their consent usually by asking them to sign a consent form. Here's an example of what such a consent form might look like. "Thank you for participating in today's study. The purpose of this study is whatever. In today's session I will ask you to perform some tasks using the system that we're evaluating and, afterwards, I'll ask you some questions about your experience. To help with my analysis, I will record audio and video of today's session. This data will only be viewed by members of the research/ products/ design team whatever is the best description of what your team is, and all information about your participation will remain confidential. Your participation in this usability study is voluntary. You can withdraw your consent and stop participation at any time. You can also decline to perform particular tasks or answer particular questions. I understand the information on this form and have agreed to participate in the study name and date." In an earlier lecture, we talked about some of the biases that participants and researchers can bring to bear when interacting through a user test. One class of biases that we talked about are called demand characteristics which was the tendency of participants to try to give a researcher what they think they're looking for. You can mitigate the impact of demand characteristics by clearly stating the purpose of the test. This way your participants don't have to guess what you're looking for because you've told them. You should also state clearly that you want their honest opinion and that you're not invested in the outcome of the test. This will communicate to them that you want to know what they really think and really feel and that they're not going to please you by telling you what they think you want to hear. When users are performing the tasks that you've assigned, it's important to set expectations about what kinds of interactions you will have with them during the performance of the tasks. Especially when using think aloud, it can be a little bit awkward for users to understand what kinds of interactions you're expecting and what kinds of interactions they can expect you to have with them. It's important to plan out in advance what you're going to do when users complete a task, when users make mistakes, when users decide to give up, or when users really screw things up and maybe put the system into a state where they can't continue. By planning these out in advance, you can communicate to your participants what the ground rules of the interaction are going to be during the task portion of the user test. So, you might say things like, tell me when you believe you've completed the task. It's actually important not to have you as the moderator be the one who decides when they finished a task, but for them to tell you when they think they've completed the task because you might actually give away too much by telling them, "No, you're not quite done," or, "Okay, great. You stop there. You've already done it." You might say something like, "If you decide to give up on a task, let me know that you want to move on to the next task." Again, so that you're not in the position of trying to guess whether they've given up or whether they've decided to move on, let them be the ones who decide that and communicate that to you. You might also say something like, "I won't offer any help or hint while you perform the tasks. My goal is to see how you would approach these tasks without any help." You might add, "And we can answer any questions that you have after the tasks are done." We also talked earlier about post-test interviews in particular debriefing where we follow up on things that happened during the tasks to get more information about places where users encountered problems or places where participants had questions. One thing that you can do is develop a note taking protocol that will help you flag particular things for follow up later. So, you might just write down the task where something happened and a word or two that will remind you what you want to ask later. But even with a very streamlined note-taking protocol, it can be challenging to moderate the test and also take note of any kind at the same time. There's just too many things to keep track of. If you have somebody else who is helping you conduct the test, which is a good idea if you can do it, you can use a technique I call tag-team debriefing. The idea here is that one person is actually moderating the first part of the test all the way up through the part where the participant accomplishes the task, and the other person is taking notes and noting down questions that they might like to ask in the debrief. After the user has completed the tasks, you actually switch moderators and the note taker becomes the person conducting the interview. If you're going to use this technique, you want to make sure that participant is prepared for it. You've just built all of this rapport, they've gotten comfortable with you, and now you're introducing them to somebody else. So, during the introductions at the very beginning, you want to make sure that the participant is expecting this and that they get to know the other person who's going to be talking to them later as well. As we said in earlier lectures, it's important to leave time at the end for wrapping up so that you're not in a hurry at the end. Be sure to leave time to ask the participant if there's anything else that they'd like to tell you, and leave time to either give the incentive that you've promised them, a gift card, compensation or whatever it is, or discuss how you will get that incentive to them later. Of course, make sure you say, "Thank you for your time." Moderating user tests is a skill. It's a skill that comes more naturally to some than others, but in all cases, it can be learned and it gets easier over time as you conduct more and more user tests. By planning ahead, remaining relaxed and positive, and building rapport with your participants, you can ensure that they have a positive experience of the test session and that they give you the data you need in order to make your tests successful.
