Hi. In this lecture, we're gonna talk about a very simple class of models that helps us make sense of data and these are known as categorical models. In a categorical model, what you do is you basically bin reality into different categories, and you hope these categories help you make better sense of the data. That they explain some of the variation in the data. I wanna start out by just describing what a categorical model's like and we'll talk about how they can help us make sense of data. So let me give an example. A long time ago, over a decade ago I was at a conference in Amazon, which is a company that, you know, sells all sorts of stuff over the web, [laugh] right? Had, was just going public. And there was a discussion whether Amazon was a good investment or not. So one person who was a Wall Street investor said, you know, I think it's a horrible investment. If you think of Amazon all it is, is it's just a delivery company. Like they just got a big warehouse, you order stuff, they deliver it. The margins in that industry are really small. Like there's already UPS and EPX, [laugh] UPS and FedEx and DHL and all those sorta places. I just don't think there's any money in it. Now, another person said, you know what, I'm gonna put Amazon in a very different box. I'm gonna put this box over here in a box that says information, because I think its part of the new information economy. They're gonna gather all this information about what consumers want. It's all going to be centrally held. They're gonna be worth a ton of money. Now it turns out if you put Amazon in this information box, you probably would have invested in it, and you'd have made a lot money. If you put Amazon in this delivery box, you wouldn't have invested in it, and you wouldn't have made a lot of money. So which box you use [inaudible] how you categorize things affects how you think about things. And, again, what sort of decisions you make. So this leads to a phrase that one of my friends who's a physcholy, a psychologist [laugh] once said, lump to live. What my friend meant is this, is that, we create these lumps, these boxes, these categories in order to make sense of the world. So I look out there in the street and I see a vehicle. I don't say oh it looks like a 1997 Ford F150 pickup truck, right? Instead I just say truck or I just say car. Or if I look at a piece of furniture I just say it's a dresser. I don't say it's an 1874 Chip N Dale dresser. I don't completely break it down. I just put things in categories. And these [inaudible]. They're short cuts, right? They help us make sense of the world. And let's think about why we model again, right? One of the reasons we model is to help us decide, strategize and design. Right. So one of the reasons we lump is it helps us make quicker faster decisions where we just put things in categories and say this is something I like, this is something I don't like, this is something that's risky, this is something that's not risky. Let me give some examples. Give some fun examples. So the first one is let's suppose you?re a kid and you gotta decide what am I gonna eat and what am I not gonna eat. Well one sorta categorization you might use is the green categorization. So you might say anything that's green. Proxy screen. A grasshopper's green, asparagus is green. All these things are green, right? Everything else, bananas, those are yellow. Candy bars, brown, orange, they're orange. Pears, pears can be green, but we'll assume they're yellow. And strawberries, they're red. These other things aren't green. And so your rule could be, I'm gonna eat anything that's not green and I won't eat anything that's green. And so that rule will keep you safe from things like grasshoppers and asparagus. Right, so that's a rule you might follow. Now it's not an optimal rule because you might run into a green pear and that green pear might be something you'd really like but if you've been avoiding green things, you may decide, well not gonna risk it. So that the same, an example of a simple rule. Let's now show how you can use a rule like that to make sense of data. So now let's suppose I've got a bunch of data here, and these are different food items, and what you've got in this column right here are calories. So this is how many calories there are in each of these, these food items. So what I want to do is I'm trying, I wanna make sense of why do some things have a lot of calories and some things not have a lot of calories. And so I've got this list of items. Well the first thing that I need to try to make sense of is how much variation is there in this data. Well to understand how much variation there is, first I've got to find out, sort of what's the average value? And then variation tells me, how far are things, on average, from that value? So if I add all this up, I've got a 100+250 that's 350, 440, 550, 900 right? So we've got 900 divided by five, so that means the mean here Is 180, so on average, everything in this group has about 180 calories. And I want to ask some things are higher. Right? This is 350 and some things are lower, this is 90. I want some understanding of how much variation is in that data. So one way to do that we just subtract the mean from everything. So if we take 100 minus 180 that's gonna be minus 80. 250 minus 80 that's gonna be 70. 90 minus 180 that's minus 90, right? 110 minus 180 is minus 70 and 350 minus 80 is 170. Well if I add all these things up I'm gonna get minus 80, plus 70, minus 90, minus 170, plus 70, it's be zero because it's gonna be the same as a mean. So what I need is I need all these differences to be positive. So one thing I can do is I can just take the absolute value. Of all these things. Right? And then I could add up the absolute value. And I could get 80 plus 70 is 150. Plus 90 is 240. Plus 70 is 310. Plus one 70 is 480. So we could say, the total difference from the mean is 480. But what we do in statistics, is we tend to do something different. We actually tend to take the difference and square it. And the reason we square it is really twofold. One is that again it makes everything positive, which is what it did before. And the other thing is that it amplifies larger deviations. Because what we'd really like to do is prevent those huge deviations. This is gonna amplify large deviations. So if I look at the pair, I would have 100 minus 180, which is 80 squared which is 6,400. So that's how much variation there would be. So that's the, how much the. Difference from the pair, pair to the mean squared. And if [inaudible] for the cake, I'm gonna get 250-180 which is 70. And if I square that, right, I'm gonna get 4900. Now I could do this for everything. All of them right? So for the pear, I get sixty four hundred, for the cake I get forty nine hundred, for the apple eighty one, for the banana forty nine, for the pie, twenty eight [laugh], thousand, nine hundred. So this is again, a long way from the mean, and you square it at a huge effect, so square amplifies larger mistakes. Now if I add up all these numbers, I'm gonna get fifty three twenty. That's what we call a total variation. So I plotted that data, this tells me sort of, how much variation is in that data, what I'd like to do is keep. [inaudible] categories [inaudible] that reduces that variation that somehow explains why something are high and some things are low. So what's the obvious. Categorization. The obvious categorization here is that pears and apples and bananas are fruit and cakes and pies, right, are desserts. So let's create a fruit category and a dessert category. So in the fruit, I've got one thing that's 90, one thing that's 100 and one thing that's 110. And in the dessert category, I've got one thing that's 250 and one thing that's 350. So let's look at them in more detail. I've got 90,100, 110, the mean Is gonna be 100 here, right? The average is also 100. What's the total variation? Well, 90 minus 100. Is just ten, so if I square that, I get 100. 100 minus 100. Right. Is zero, so if I square that, I get zero. And one ten minus 100. Is also ten, so if I square that, I get 100. So the total variation here is just gonna be 100+100 or 200. So now what I've done is I've got a mean of 100, and a total variation of 200. And now, if I go to this case, the mean is gonna be 300, right, for the desserts. And what's the total variation? Well, for the cake, it's 250-300, which is 50 squared, which is 2500. And for the pie, it's 350-300, which is also 50 squared. Which is 2500 [inaudible]. Add those up, I get 5000. Alright, so let's clean this up a little bit. So what I did is by creating two categories, a fruit category and a dessert category, I now have a mean in the fruit category of 100 and a variation of 200, and a mean in the dessert category of 300 and a variation of 5000. Now one [inaudible] I started out with, right? When I had all the stuff together. I had a mean of 180 and I had a variation. Of 53,200. Now look at how much my variation has gone down. It went from 53,000 to 5,000. So here's to the idea. These categories substantially reduce the amount of variation I have left over. So think of the variation as what's unexplained. So initially I say look I can just say things on average of 180 calories and we've got 53,000 units of variation that's [laugh] unexplained. Now I say wait a minute, I'm gonna create a categorical model that says there's fruit in desserts and fruits have few calories than desserts. And you can say we?ll look it appears to be the case. Fruits have a mean of 100. Desserts have a mean of 300 and the variation in the fruits is only 200 and the variation in the desserts is 5,000. So I've reduced variation a ton. What we want is we want a formal measure of how much we've reduced variation. That's actually fairly simple, right? So that a total variation of 5300, fruit variation is 200, dessert variation is 5000. So that gives me 5200. So 53,000 start and I get 5200 left. So what we wanna ask is how much did I explain. That's the question. How much of that radiation did I explain? Well, then I started out with 53,000, right 200. And now I only have 5,200 left over and so the amount I explained is just 53,000 minus 5,000 right which is 48,000 right. >> Over 53, two. So the percentage of [inaudible] I explained was 48,000 divided by 53,000 which is a huge amount. And I can write this more simply as just one minus the amount I didn't, that's left over. One minus 5,200 over 53,000. So, right? Because that's just a simpler way to do it. And so when I get that the amount of [inaudible] I explained was 90 thou- 90 percent. So 90.2%. So that's how much of that variation I explained. This is equal to, again, that 48,000 right, divided by 53,200, the amount of variation that I explained. Now, formally, this is called the R squared. So this is the [inaudible], the percentage of variation that I explained just by that simple categorization. So, if the R squared is near one. That means I explained almost all the variation, so the model explains a lot. Right? If the R squared is near zero, that means I didn't explain any of the variation really, and the model doesn't explain very much at all. Now the better the model, the more R large R squared it'll have. But depending on, there could be so much variation in the data that even a great model. Only has an r squared of five or ten%. There also could be situations where the thing you're trying to explain in pretty understandable and a good model has to have an r squared of 90%. So there's no fixed rules whether, you know, what a good r squared is. It depends what the data looks like. But with a class, you know sort of class of models, or you know that's [inaudible] data class, you can sorta figure out this is a good model, this is a bad model. Based on experience. Let's push this a little bit further. We had, you know, fruit and desserts, right? Those are our two categories. But if I had, you know, a whole kitchen worth of food, it may be the case, that, like, I'd wanna have more categories. So I might create a vegetable category and a grains category. And then I could put everything in one of these four boxes. So one of the differences between sort of experts and nonexperts is experts tend to have more boxes. They also tend to put the right things in the right boxes, so they tend to have useful boxes. So if you want to be good at sort of predicting things or understanding how the world works, what you have to have is a lot of categories and you have to have those categories be the right categories. They've got to explain a lot of the variation. And we can measure how much of the variation it explains, your model explains, by using that R squared. One last point. Even if you explain a lot of variation, it doesn't mean you've got a good model. Let's go back to the schools case. So suppose I'm trying to figure out what makes a good school, what really leads to a good school performance. So I try all sorts of different boxes. I look at schools that spend a lot of money versus schools that don't spend a lot of money. Schools that have small class sizes and big class sizes. Schools that are big and schools that are small, right? And nothing really seems to explain too much of the variation. And then I create a box that I call the equestrian box. And I put all the schools in here that have equestrian teams. And I find, oh my goodness, every school with an equestrian team is great. Well, the thing is, that doesn't mean that the equestrian team made the school good, right? So statisticians make a distinction between correlation. Which is, is there a statistical relationship between having an equestrian team and being a good school? And causation, did the equestrian team cause the school to be good? So remember when we draw, when you think about putting it in this box. Like, this box right here is a bunch of good outcomes. And this box here has mostly bad outcomes. That doesn't necessarily mean that the thing that created this box, if it's the equestrian box. Is the reason that the schools were good. It could be that there's some other reason. So why would you have an equestrian team? Well you only have an equestrian team if you had a lot of money. And you probably also only have an equestrian team if you have a lot of parental involvement, things like that. Like a lot of support from the community. And you only have an equestrian team if you have a lot of open space. So it could be that having an equestrian team is a proxy for things like money, parental involvement, open space, right, those sorts of things that actually do make a school good. So even if your boxes work, that's no guarantee that they're actually the cause of why it works. Okay, so what do we have? We have one way, [inaudible] the simplest model you can possibly have is a character model. You can say, Amazon is a delivery company. Amazon's an information company. You can say things are either fruits or desserts. Right? And by creating these boxes it can help you sort of explain the variation in data. What we saw then is a simple way to measure how good a categorization is, is how much of that you explained the percentage of the variation you explained and that's what we called R squared, right? R squared was just take all the variation that was there and then ask how much was left how much was left over. That means and then we subtract those two that tells you how much you explained. So you ask what percentage of all that variation do you. Let me start over. Okay, what have we learned. What we've learned is this. We've learned that the simplest kind of model you can have is just a category based model, right where you just sorta lump the world in different categories, and you place your data in different boxes depending on what different data it is. So that could be information companies versus delivery companies. That could be fruits and desserts, right? And in doing that, what you could do is you can reduce the amount of variation you see in the data. So there's a total variation which is sorta of just like how much unexplained variation there was out there in the world, by putting it in boxes you organize it in such a way that you reduce the variation. The amount at which you reduce the variation is what we call the r squared. That's the percent of variation explained. And the more variation explained the better your categorization is. Of course if you create more boxes you can explain more of the variation. Where we're going next, if you've got linear models, which in effect can create a different box for each value of x, our dependent variable. Okay, thanks.
