Welcome back. Just like we discussed, flexibility versus interpretability, in statistics, we're going to have a lot of things called trade-offs. A common saying in statistics is, there's no free lunch. You can't get everything you want for free in statistics. You want to launch, you got to work for it in terms of that trade-off. In terms of flexibility and interpretability, there is that trade-off, you have to pick where you want to be sitting in that space. Here, there's also another trade-off in terms of bias and variance. Variance, in general, is thought about a spread, bias, in general, is spoken about in terms of accuracy of a model. Now, here we're using, notice something else about this formula, which is that instead of using y_i and x_i, we're using y sub 0 and x sub 0, y_, x_0. Going forward in this course, we'll generally talk about, x_1 through x_n, y_1 through y_n as a training set. We use that data to inform a model and then with that model, we can go forward with other things. When we talk about x_ 0 and y_0, we're generally talking about either real data or test data. When it comes to Mean Squared Error, what you don't want to do is, you don't want to train on training data and then evaluate Mean Squared Error of that training data. It doesn't really tell you how well your model translates to new data. It's really important that when you train using a model, you evaluate it on new data because if you evaluate it on the same data you used to train it, it's not a good evaluation of how that model actually works. In general here, we're just introducing some new concepts which are x_0, y_0, which is test data. In general, you want to train a model on training data and then evaluate it on unseen test data and that's what we're doing here, generally speaking. It's just a good concept to get used to now in the beginning because we'll see it later throughout the course, when we deal with training sets and test sets. Here we have bias-variance trade-off and we're dealing with test. This is test data. We are dealing with test or expected test, MSE here. This gets decomposed into the variance, plus the bias squared, plus here, is the variance of the error term. Now if you remember back, not too long ago, we discussed that this is independent of x. This is irreducible error and we can't touch this. Right now we can't really deal with this at all, and we won't really be able to in the ways that we can deal with this reducible error. Now, the variance of our model here is inherently non-negative and bias squared is also non-negative. We're not playing around with a positive and a negative term to balance, we're playing around with two non-negative terms, and so if we increase one, we have to decrease the other in turn, to compensate. Variance here deals with the amount our model will change with new training data. Again, that's just an intuitive understanding of this part, this variance to our model we'll say. Again, we're dealing with test data here. We're saying, the variance can be thought about as, if I feed this model new training data, how will that affect it? If I feed it newer and different training data over and over again, how is that going to change my model when I get to apply to test data? In general, the more flexible your model is, the more variants that you're going to have to deal with. That's usually the tradeoff. Again, just like interpretability and flexibility, there's a little bit of wiggle room here as you go up or down that curve. Here, with bias, that is, it's biased squares, so it's definitely non-negative. Where bias comes into play is that in real life, when you deal with real data, the models that actually connect the inputs to the output, those models in real life are usually extremely complicated, to the point that estimating them will always lose something. You'll always lose something estimating a real-life model with even a complicated model, it doesn't matter. Real-life data is extremely complicated, if you're trying to bring it down to one model, you're going to lose something at some point. It can be intuitively thought about as the error when approximating some real-life complicated data with some pretty much guaranteed simpler model. That's what bias can be thought about as. In general, if you have a more flexible model, you have a lower bias. That makes a little bit of sense because remember, more flexibility is usually a more intense, complicated model, because that will lead to less interpretability, which also makes sense. The more flexible your model is, the more complex your model is, the closer it is to your real-life complicated data, and so you have less bias. This is the bias-variance trade-off and like I said, in this course or in statistics in general, you should get used to seeing these trade-offs. You're going to have to make a lot of these decisions as an actual Data Scientist.