Welcome back. In the last lecture, we came back to the concepts of personas and gave a little bit more detail about developing them. This lecture walks through examples of developing personas for a system you've seen earlier, the MovieLens system. So, a little bit of a history about this. This is going to be unusual persona development but an unusual and interesting way. The original MovieLens system as it was created never had any user interface design done. It was a strange situation where a site that already existed, called EachMovie, was shutting down, and shutting down quickly, and we had the opportunity to take it over If we could build it. And so the directive was clone it, and clone the interface as best we can, and we got something built in a month. That was no time to study users. That was no time to do anything. But none of us ever believed that was the right way to build the site. It was sort of an emergency and an opportunity. So over time we've conducted substantial user research, and it's a nice example to look at synthesizing personas for MovieLens, because we have so many different types of data to work from. We have surveys studies, we have interviews, we have usage data logs, we even have data we collected in research studies. And in this lecture we're going to bring these together to form a couple of personas. We'll also take those going forward or one of them in particular going forward as we look at use cases and even into task and scenario development. So let's take a look at some of these sources of user research. Perhaps the most common one for an existing site is usage data, and we're going to talk about both aggregate and user-level data. If you have user-level data you can look at correlations and what behaviors happen coincident with other behaviors much like we saw in our last lecture looking at particular doctors that might have similar patterns in their surgery. We did a bunch of work with web surveys, and I'm going to take you through one of those in a second. We did a little bit of telephone survey work, to get a little bit more depth. We have our history of user support contacts, mostly emails. And then other data collected in prior studies and I'm going to come back to the idea of external data. So here's some example from survey data that we did. In this case this survey was conducted as part of a larger study. And we started by asking people why they used our system. That seems like really a useful thing. And what you'll noticed is that we had about 330 responses and almost 60% of them said, their main reason for using MovieLens was to view recommendations. That's good to know but what really shocked us was that about 30% of them said, their main reason for using our system was to rate movies. This was a shock to us. It required follow up. We never thought of rating movies as something people wanted to do. In fact when you saw that redesign that we talked about earlier in this course. We talked about how do we get away from forcing people to rate movies. But in fact the study said that for some people rating movies is where the fun is. And that's something that causes us to think hard about the fact that we have different kinds of users and different kinds of usage. When we asked people why they rate movies, again the majority was to improve recommendations. But there were a significant number, about a hundred of them, where their first choice was to either keep a list for themselves, or because rating itself was fun. And these were popular second and third choices as well, and so suddenly our thinking had to develop. We asked people what they thought of the system, that's lovely, the people who stick around in the system like it. That wasn't that useful. We've asked them a lot of other questions. Whether the system is getting better or worse for them. We asked them what they thought the system was worth. This was a study done with economists and they like to reduce everything to dollars. We also asked how often they do different things in the system. From several times a week to less than once a month to never. And we'd see that all sorts of activities, there were certain people who did them and certain people who really did not. And that gave us some information. We also looked at their beliefs on different things. You could see these are on sort of fun scales, where we can look at agree, or agree plus don't know, or just look at disagree, and mostly people's opinions were positive. The one thing we found out was people mostly don't write reviews anywhere else. Well, we can also look at very specific data. This is a set of graphs we pulled up for the last month of activity. On our left we have log in count. And the vast majority of our users log in fewer than five times in a month. But there's a significant chunk that log in more. And we're going to take a look at that in a bit more detail in one second. We have tags. Most people don't tag. But a bunch of people tag significantly. Ratings, the distribution's much broader. Again, most people don't put in a huge number of ratings in a month, fewer than 25. Well that shouldn't be so surprising. 25 is our quartile, that's a lot of movies frankly. Sorry 25 is the three quarter mark. For somebody to have 25 new movies to rate in a month either they're going back or they see a lot of movies but the median person rated at about 6 movies a month. That's an interesting number to take a look at. We actually can pull that up in the form of a spreadsheet, which I'm going to get you here. And what you're going to see here, I'm going to take you all the way up to the top. Is the data that we've dumped out by user sorted by the number of times they logged in in the last month. And it shows some interesting patterns. It shows that we have some really aggressively wonderful super users. The kind of person who's coming in six times a day, four times a day, three times a day, twice a day. But these people aren't all the same. Some of them love to tag movies, many of them never tag a movie during the month. Some of them rate a movie not even every time they log in. Some of them rate several movies every time they log in. As we go down we see people log in less, but some of them have even more activity. This person logged in 55 times in a month and rated over 2000 movies. With detail we could look up, is this a new user who is just getting going? Is this somebody who's just insane? >> [LAUGH] >> In a good way. We could say the same thing about this person who rated almost 2000 movies in 45 logins. But of course we also have to look further down. There's a pretty smooth slope but as we keep going, we'll see that there's people at relatively small number of logins who rate a lot. There's people who don't rate much at all, and the vast majority of the people who came in this month Logged in once, and did very little. So knowing that gives us another way of looking at people in clusters. Let me add some other data to this. We did some phone survey data. We actually hired an undergraduate to contact a bunch of our users. A while ago, and find out what mattered to them, why they used the system, what they thought about how the system worked. And it was interesting, the biggest reasons we heard from the phone survey were that people liked our system because they believed it was independent and, therefore, trustworthy. We had no advertising, we weren't selling them anything. That was a big deal. That's come up in other forms of input as well. It comes up in surveys. It comes up in unsolicited comments. It comes up in some reviews people have written of our site. We learned that many people who use the site regularly don't understand or really even care how the system works. They're not technology people. One of them actually assumed we sat around in a room and came up with good recommendations and then put them up on the site. Didn't even understand that it was trying to be personalized. We were disappointed that our communication failed but happy that they trusted us that well thinking it was just our opinion. There's a wide variety of used context we've learned about, some people use this as a planning tool, some people use it more as a reflecting tool. Let me think about the movies I've enjoyed, some people make active decisions, some people just use it to browse for fun. And almost all of the people we talk to use this as one of many tools when they're selecting a movie. Some don't use the recommendations at all, and almost nobody uses movie lenses, the only source of information when selecting a movie. >> Yeah I think Joe, what I've heard you say that is really interesting in this example is how there's different lenses on the users. Each different method of gathering data gives you different types of information, all of which can be useful. You can do interviews and find out about people's motivations, goals, their attitudes towards the system. You can do log data and find out about usage patterns and let's you say there's super users versus casual users, different types of usage. It's really interesting to take all those kinds of data and put them together to get a rich picture of different types of users. >> Exactly. And I'm going to even overload us with just a little bit more data. We did a study where we gathered personality data on our users. And we discovered not really a great surprise, that MovieLens users tended to be significantly more introverted than the population as a whole. I say it's not a big surprise because given the choice of going out with people to see a movie or talk about movies or sitting on your computer screen using MovieLens to think about movies. The introvert is probably less likely to favor going out and talking with other people and more likely to favor spending time alone. We got behavioral data. We found some systematic differences between people who use the system but never see the movies we recommend. They don't seem to take recommendations, they often don't even ask for recommendations, and people who do take recommendations. We found out that some people view this as a social system and care about the other people that are there and other people don't care about or even want to interact with social features. And we've had different ones over the years. And maybe the last thing is that in repeated studies, we have found that people care more about MovieLens as a concept, a site or a community than about the people in the site. And so if we ask people to help MovieLens, they're willing to do work. If we tell them I'll help them, we get less of a response. I think people don't believe us, if we say it'll help them. And if we tell them, it'll help others on the site, that doesn't work as well as just helping the site. So we do have a lot of data but there's data we don't have. Our demographics are relatively poor. We can at a times have gotten some location data from IP addresses that can be useful but even though we've done some collection of gender and age data, it's pretty noisy and pretty sketchy. We also have little data on social movie consumption. I mean do people go out by themselves are they watching at home are they watching with friends. We could do a separate survey on this and we may at some point but this brings up the point that sometimes you can find external sources of data. So we could get data from some of the research companies that issue reports mostly for businesses that subscribed. Talking about what are movie ticket sales like. How many tickets are bought, sold pairs larger groups. How many tickets are bought in advance versus bought right at the theater at that time. How much of movie watching happens on streaming versus cable television versus rental of still-existing DVDs or tapes versus in theaters and broadcast? All of that is data we might be able to fold in if we needed it for our design. So let's move towards a set of personas. Much as Loren showed you last time we need to identify some dimensions that we might try to cluster people along. Here we have too many people to cluster them by just spotting them. We're going to cluster them by looking at aggregate patterns. And we identified five dimensions we thought where relatively important on what we saw. Usage level ranging from the super-users, who come in, once a day or more, through average users, infrequent users. And the one timers which are a significant portion of our users. That at times we care a lot about and in some features we might not care about at all because one-timers never use them. What about their underlying movie fandom? What's become clear is that we have some people here who are super-fans who've seen thousands and thousands of movies and know a huge amount. We have people who are sort of enthusiastic movie fans, but not devoting their lives to it. Average movie goers, and we do get a bunch of users we've discovered who are technology curious, but don't really care that much about movies themselves. They're more interested in this as an artifact of recommendation, and so that's something we need to know about. In movie tastes, there's this continuum between mainstream and eclectic. That matters for us particularly because people with mainstream tastes, we can do a very good job for right away because we don't have to model their tastes. The average of our community is pretty mainstream. People with eclectic tastes will get more benefit from our site in the long run, because it's a personal movie recommender site. But we need to get a lot more work from them to get to that benefit. Because there isn't one eclectic taste, every eclectic taste is somewhat different. Second to last, we have usage goals. It's clear we have some people who are here for getting recommendations and decision support. Some people are there for personal logging and reflection. Some people, but a small number, intend Really care about self-expression and influence. How do I say what I think and help other people find good movies or steer them towards what I think is good. And these are not a continuum, these are different attributes that each user might have different waitings among the different goals. And finally, there is a geographic issue that comes up frequently. That because we focus on US movie releases, the North American movie market, which interestingly isn't just in North America. There are parts around the world where North American English-language movies tend to get first releases. Is a market that's much more likely to have a certain type of usage than people who are in a country. Where the North American English language films are not the dominant content. Even if they are released and often subtitled or dubbed. And so we've noticed that users from other countries tend to be different particularly other countries where English isn't the first language. >> Yeah I love the title of this slide towards the set of personas because it reminds me that you can't just design your system. Or if you design your system for one kind of user and one kind of usage you're probably going to be shutting out a bunch of people. And knowing that you typically need a set of personas is a strong reminder to keep that in mind. >> But it's also a reminder that we don't want to just take all these combinations. This is not about factorial design where we say well let's find a non-North American who's a super fan, who's a whatever. Our goal is to capture the relevant design dimensions in reasonable combinations, that's what these clusters are. But too many personas is bad, for a site like this, we might have three, we might have four, we're not going to have 10 or 20. Let alone all of the combinations and so what we try to do is get the patterns that are really common. And the ones that seem to be interesting and important even if they're not common now. Because the reason they may be not common is because we haven't designed well for them. So let's try some combination ideas and then we'll refine a couple of them into personas. So we might have the superfan superuser, who really enjoys expression and reflection in the system. That's the sort of person we know we have a lot of. We might have the newbie explorer looking for useful information who likes movies, tends to find stuff on the web. And is trying to find out whether this is a site that would make sense, that's something we get a lot of people who show up. We don't always retain them very well, so that might be an important persona. We may have the once-a-week movie watcher, sort of the typical person who sees movies regularly, but not obsessively. Who values unbiased and maybe a little bit of quirky advice to help find movies that she or he wouldn't see all the time. No, we've had some people in and then maybe the important category but it's not a huge category is the Budding film critic. The person who wants to use movie lens as a platform to help share opinions and influence others. And at various times we've made writing critiques a significant or not significant part of the movie lens experience as we've engaged with or disengaged from this type of user. So let's try to refine some of these and I'm going to start with the idea of Victor, why Victor? Because you have to have a name, you don't give a person a name you don't have an identity and Victor just seemed like a good name for somebody whose a real movie fan. 52 years old, super fan of movies, an active MovieLens user from Seattle. We know that Victor is in the North American movie market, seen over 5,000 movies rated more than 3,500 of them on MovieLens. The other 1,500 either hasn't rated yet or maybe are movies we don't even have in our database. In a typical week he watches five to seven movies, sometimes in theater, sometimes at home. Including most new releases and lots of older films, because frankly it's hard to watch five to seven new releases every week. Victor uses MovieLense several times a week or more, mostly to record the movies he watched and what he thought of them. He actually has a collection of tags he uses to help him organize his movie experiences, which is a pretty cool thing to do. When asked why Victor uses MovieLens, he says it's unbiased, it's not for profit. In fact, he actually frequently volunteers for our experiments to support the community. When asked about recommendations, he confesses that he's almost never seen a recommendation from the site that he cared about. He already knew about the movies but he occasionally looks at them anyways to see how well it models his tastes. Honestly, he doesn't actually think that MovieLens has a hard job modeling his tastes he just generally likes movies. So if we say it's a good movie he would say yeah I probably would agree with that. So let's reflect on this for a second. I got it a picture for Victor. Now, I picked this picture up from the free for commercial use section of Flickr, and I want you to remember couple of things. There is no Victor, we just made Victor up, he's not real yet. But much like doctor Frankenstein, when we put our energy into it, we're going to treat Victor like he's real. And we're going to get our developers and our designers to treat Victor like he's real. Everything we said here, the quotes, are made up. The data is not made up, it's a composite from the research that we've gathered. Victor is anchored in truth but a synthetic composite and we're going to treat him as real. >> Yeah and we can say our designers can say remember Victor likes to rate movies to keep track of what he seen. He likes to tag them because he likes to organize his own set of movies. And he likes what we're doing because we're unbiased, we don't take ads, we're non commercial, so let's keep Victor in mind. Let's be wary of putting in advertisements let's make sure it's easy to rate and tag. Those are the kinds of things that having Victor helps us do. >> And if somebody suggests that you know what we might be able to improve our prediction value if we get rid of user created tags. Somebody on the team is going to say, wait a minute, Victor could care less about prediction quality. Victor cares a lot about tags to organize and the Victors of the site are the people who are putting in all this data. Maybe that's not a good design decision, so let's do one more example. So Anna, Anna is a 34-year old potter. If you're not familiar with the term, this is somebody who creates pottery, clay pots and art objects, plates and other things from clay. She has a quirky taste in movies. In fact, she rarely sees movies with her husband and children, but more often watches them with artist friends or on her own. She's not a major film buff, maybe she watches a movie once a week or so. And she found MovieLens from the recommendation of a friend who said it was helpful in finding movie she had liked and the friend had quirky taste too so she tried it out. So Anna has been using MovieLens about two years and has been pretty happy with it. She rates movies to improve her recommendations but has never done anything else with the site, doesn't care to tag, doesn't try any of the social features or other things. When asked about the site, she indicated she uses it occasionally, mostly to help pick among movies she hasn't heard of. She appreciates that the site helps her find the best choices among movies that might be showing at any time. And that it's unbiased, which matches her artistic values. So as we reflect on Anna, we found another Flickr picture. And Anna we can tell just looking at the picture that she really wants to be guided towards a movie that would match her tastes. We're going to use Anna as we go forward in our next lecture and we look at some of the used cases. So I'm not going to reflect much more detail on Anna other than to say, you'll see her again soon and with that, we're going to wrap up our examples of personas. See you shortly. >> Yep, we'll see you next time.
