So you're probably wondering what was in Bell Core's pragmatic chaos or the Ensembles algorithm that made them work so well. Well, turns out that there's really hundreds and hundreds ingredient algorithms that are really blended together in order to make it work as well as it did. So here we're not going to go through all that. We're going to illustrate the main ideas. Again, there's hundreds of ingredient algorithms. We're going to focus on certainly two of the main ideas that are, kind of form the baseline in order to get there. Two of the main themes that they used, and, the first one of them is the baseline predictor. It's, no pun intended. I said before, forming the baseline. This does form the baseline for what's called the neighborhood model, and so basically, in the baseline predictor, we're going to just look at trying to use some sort of biases, HDL users, in order to make a prediction. So this is going to be something like user movie biases. So, is this user an easy or harsh user, is the movie well received, not well received and modify the predictions accordingly. And then the neighborhood method, we're actually going to do sort of, similiarities. So we're going to say like, user To user or movie to movie similarities. So for instance if Charlie and Bob tend to like all the same movies then we can say that if Charlie's going to like some movie that Bob probably likes it too. then at the same time movies could be like one another so if you may have two different movies two different Disney movies for instance Like Aladdin and the Lion King, right, and users tend to either like or not like them, so there, you can therefore say that if one user likes one movie he's probably going to like the other one. And, so, now let's look at our example that we're going to be using to illustrate this. We'll stick to one example just to make it easier. And so, in our example, we have six users which we label A through F down the rows again, users down the rows and, we have five movies one through five, which we use Roman numerals to denote. the reason I did letters and then Roman numerals was so you didn't get confused with them and the actual rating values which will use just, numbers, one through five to do that. And so this, this table is a lot more dense than is necessary. We said before that Netflix really was a very sparse data set, alright so many entries are just not filled in. out of the even a hundred million entries that there are, there's just many, many more users than movies than there are entries. When you consider all the different pairs that could be filled out on the table. And, so, in this, in this case right here, you know? We have all, except for 5 of these, filled. So, you know? Really, we have, you know, quite a lot of the data. and that's not going to be the case in real life. But it'll, this will suffice to illustrate the, the main ideas. And, So we're going to section our data as well. again, we have our training data, which are the. the what the numbers right here that aren't that are just written in black so 5, 4, 2, 4, 4 and so on. Then the test data we've, we've unbolden them and colored them blue. So for instance 4 is in the test set. 2 is in the test set. 5, this 3 value and this 4 value over here they're all in the test set. So what users see rates movie 1 we do know the value. But we're going to hold that off and make that part of the test set in similar with what user D, rates would be too. So, we're not going to use those when rating algorithm, we're just going to use them when we compare in RMSC in the predictions. And then there's unknown ratings, which we just don't have, because the users haven't watched them or rated the movies yet. For instance, user E and movie 2, he hasn't rated that. user c, movie 3, this hasn't been rated. And so on.
