But the worst of those is the ordinary least squares solution.
And the IRLS solutions are actually better than ordinary least squares,
in terms of false positive.
Now, here the distance, the multi-rate outlier removal, is performing better.
But that's really because the procedure that was used to remove outliers
was exactly the procedure that's used to generate the outliers.
So, in that case, it can be a win.
But, on average, the multivariate outlier removal isn't a good idea unless
you really know that it's performing well and is valid for your data.
So, IRLS here is better.
And then, finally, in the bottom, we're seeing
the case where there really are true effects, but there are also outliers.
And IRLS is the most powerful.
Here, it's the bi-square rooting function when true effects are present.
And ordinary squares is the worst.
So, ordinary squares is not very robust to outliers in terms of power.