So far, in observational studies, we've been assuming that unconfoundedness holds and that's key for all the stuff that we've done. Of course, it did hold with the experiments. But in observational studies, we've been assuming this. The thing is the assumption is untestable. The reason is just it's fundamental for each observation. We see Y1 or we see Y0, but we don't see both. In the sequel, we'll discuss some other assumptions that one might make without assuming unconfoundedness. But for now, we're going to focus on how one might assess this assumption and account for some departures from it. So this module be concerned with that. So, several approaches to assess unconfoundedness have been put forth. One approach, primarily the Rosenbaum, still looks at additional outcomes known to be unaffected by the treatment. So, imagine an outcome unaffected by the treatment, you have to know or you'd have to know exactly how it's affected, and let's just call them control outcomes. Now, if a comparison between the treatment and control group suggests that treatment affects a control outcome and you know that the control outcome is unaffected by treatment, then something's happened that's wrong, either a type 1 error has occurred or the unconfoundedness assumption is incorrect. So, what this suggests is that an investigator only needs to think of some control outcomes and check whether or not these appear to be affected by a treatment. Oh, sounds great, pretty simple, pretty straight forward, but you need to be much more careful than that. Okay. It's intuitively appealing idea, but let's look at it closer. A pre-treatment covariate cannot be affected by the treatment, so you could use any old pre-treatment covariate. That's not quite right. The covariate must be one that is not a confounder. If you believe the covariate might be necessary to make unconfoundedness hold, well, you can't use that covariate as a control outcome, you need to use it as a confounder. So, second, recall that if the unconfoundedness assumption fails given a set of covariates X, that means you have unmeasured variables that are associated both with the potential outcome and the treatment assignment. So, if the unmeasured variables are related to treatment assignment Z but not related to the control outcome, we shouldn't expect to see a significant difference between the treatment group and control group means on the control outcomes. That would lead us to the possibly false conclusion that treatment assignment is unconfounded. Well, always with respect to the outcome of interest. So, what that means is that in order for this approach to be useful, the investigator needs to choose a control outcome that is related to the variables he/she thinks maybe unmeasured confounders. That in turn means that the investigators should have some idea of what variables he has failed to measure in the analysis. This is why investigators will often, if possible, choose something like a measure of the outcome itself, only one that is measured prior to treatment, like a pretest or something. But that said, sometimes these pre-treatment measures of outcomes are confounders. So, assessing unconfoundedness in this way is very straightforward if you have chosen a good control outcome, but choosing a good control outcome is not so straight forward. Well, once you've got one, you can test the unconfoundedness holds, subject to the caveats above, just by using the control outcomes and seeing whether the parameter of interest is zero. Second approach, also prominent in some of Rosenbaum's work, uses a second control group. So, let's let Wi be one of the following three c1, c2, control group1, control group 2, or t, treatment, and that's going to tell us what i receives. Now, if treatment assignment is unconfounded, given X, then the control group, outcome Y0, is independent of W given X. Therefore, it is independent of W given X and the fact that W is a control. What that means is that the observed outcome Y, which is then equal to Y0, should satisfy Y as independent of W given X and W, W being a control. So now, the difference is we've replaced Y0s independent of W given X above with observed Y is independent of W given and the W is in a control group. W was the control group, one of the two. So then, the investigator can test to see if the conditional distribution of Y is the same in the two control groups, which it should be, that's what the assumption says, and you can use a regression of Y on X and W, and the coefficient for W should not be significantly different from zero. Well, again, several caveats are in order. If the two control groups are very similar, as would be the case. For example, if you took a single control group and split it into two groups, well, the failure to reject the null hypothesis that Y and W are conditionally independent given X, which is what we're testing, that wouldn't really be evidence for the unconfoundedness assumption. I mean, we just expect that to happen. Now, more generally, in order to be useful, the two control groups should be different with respect to the distribution of the potentially unobserved confounders. That's when it's going to be most useful. Now, as before, some notion of the possible confounders is therefore needed in order to choose the groups so that testing for differences between them with respect to the outcome would actually constitute evidence for or against the hypothesis that treatment assignment is unconfounded. Second, you'll notice from the outset that we assumed that the observations had they been placed in the control group one or control group two, they're going to be the same. That's often overlooked, but that's not innocuous either. So, you can imagine if you're testing the effect of a pill on cholesterol level and the first control group is in Canada, the second in the United States, the potential outcomes for the subject could differ in the two countries due to differences in medical care systems and/or differences in the diets that a subject might have in the two different contexts. As another example, if outcomes are measured with systematic error that differs in the two control groups, again, the responses could differ in the two groups. Now, I'd like to go on and talk about something a little bit different. So, assessing unconfoundedness so far was sort of, how can we actually ask if the unconfoundedness assumption holds? Now, sensitivity analysis is a bit different. It says, what if the unconfoundedess assumption doesn't hold? How would that affect the conclusions of the study? Maybe it would affect the conclusions a lot. Maybe it wouldn't affect them very much. But to quantify this and make some useful headway, you have to say something about how far away the observational study is from meeting this assumption. Because intuitively, if you're very close to unconfoundedness, it shouldn't matter. But if you're far away, that's when it might matter. But then you have to say, what does it mean to be far away, and how far away, and quantify that. So that's what's difficult about this. Now, let's start with the observation that if unconfoundedness doesn't hold, that's because in addition to the measured confounders, there are one or more unmeasured confounders that haven't been considered or taken into account, at least. Now, I'm going to talk about Rosenbaum's version of sensitivity analysis with randomization based inference because he does very, very nice job. He asked how far the treatment assignment process deviates from the unconfoundedness assumption. I'm going to take up just the simplest case. Again, I'm just trying to give you guys a flavor of things, and I've given you references so that you can take this up in further by yourselves. So, I want to take up the simplest case, which is the so-called paired randomized experiment. So, I'm going to have capital i pairs. Now, we saw this way back when we talked about such kinds of experiments, but if the unconfoundedness assumption held, each member of pair i would have probability 0.5 of being the treated unit. If the assumption doesn't hold, the unobserved confounders are associated with the outcomes in the treatment assignment process; and the probability would not be 0.5. Let's get a little notation in here. So Pi 1 and Pi 2 are the probability that the first member of the pair with covariates X1 and unobserved confounder u1 takes up treatment, and the probability that the second member of the pair, similarly covariates X2 and confounder u2, takes up treatment. You will notice that X2 is equal to X1 because this is a paired randomized experiment. Now, let's suppose that unit one in the pair is actually twice as likely as unit two, or vice versa unit two twice as likely as unit one, to be the treated union. Then, the probability that unit 1 is treated and 2 is not is at least one third and at most two thirds. So now, we can consider all possible treatment assignments with the first member of the pair treated with the probability between one-third and two-thirds, and we can calculate the p-value for the null hypothesis of no effect. That's going to yield the range of p-values. It's going to be a maximum and a minimum. The minimum p-value of course is going to be less than the p-value, if each unit had probability 0.5 being treated, but the maximum p-value will be larger. As we increase from twice as likely to three times as likely, you can see that the range of the one-third, two-thirds, that that's going to expand when we go to three times as likely. The minimum p-value is going to continue to decrease. The maximum p value will increase. So, this provides a quantitative measure of how far the study would have to deviate from the ideal case or the paired randomized experiment in order to change our conclusion because you're going to look at the maximum p-value. You can also of course test the null hypothesis not just the constant of a zero effect, but you can test for a constant but non-zero effect in this framework, just like we did in Module 2. So, that's Rosenbaum's sort of starting point and notion for how to do a sensitivity analysis. Very, very nice. Now, the methodology above extends to studies in which a treated subject is matched with more than one control and Rosenbaum worked on this extensively and has a lot of nice work. There are certainly other approaches. Suppose we want to estimate a quantity such as the average treatment effect the value X, where X is set of confounders. Then the following formula just quantifies the bias right, that's the bias in the estimate. Now, if treatment assignment is unconfounded given X, the bias is 0, otherwise it's not. So let's look at this expression up there and let's break it down. So, the bias can be written therefore in this way, so that we see that it's the expectation of Y given X and Z equals 1. U has some value U. U as being the confounder, and then of course we have to consider the difference between the distribution of U just given X and given X and also that Z is 1. So you can see that for a sensitivity analysis based on decomposition above that hinges on the extent to which the distribution of a confounder is different in the treatment control groups. There's some work on this, Vanderweele and Arah, for example and there's also this review paper that I've cited. So, finally I'll talk just very, very briefly about another thing called bounds, which were developed by Robins and subsequently Manski that somehow take into consideration the worst-case scenario in terms of some deviations from the unconfoundedness assumption. Unfortunately, the bounds are often too wide to be very useful in practice. With further assumptions, you can tighten the bounds but then you have to say something about how reasonable the assumptions are of course. If they're not, well, that may not be very useful either. Richardson, Hudgens, Gilbert, and Fine have a review of the literature which I'll refer you to at this point. Now, to conclude, so let's review. We've introduced some of the main ideas and methods underlying the statistical literature on causal inference, and we focused on the identification and estimation of quantities such as the sample average treatment effect, finite population average treatment effect, and average treatment effect, and some related quantities such as the average treatment effect at X and the average effective treatment on the treated, for example. So, now we typically assume throughout that the unconfoundedness and stable unit treatment value assumptions hold. Although we focused on the case of a binary treatment, multiple treatments even a continuous treatment are easily handled in theory, I say in theory. It can also be handled in practice. So I've referred you to an article by Lopez and Gutman in statistical science for review in this literature. Now while we briefly introduce some methods for examining the unconfoundedness assumption, we didn't discuss methodology that's not based on this assumption. In the second part, we shall look at one method, instrumental variables that can sometimes be used when the unconfoundedness assumption is not plausible. Another method for dealing with this issue is to use so-called fixed effects regression models, in which special data structures for panel or clustered data are used to take into account unobserved confounders without actually measuring them. Technically, although we've highlighted the unconfoundedness assumption, we've also been assuming that each of the units can be exposed to either the treatment or its absence. In other words, the propensity score is strictly between 0 and 1. When treatment assignment is based on a cutoff, this is no longer the case. Imagine something like a reading program and for students who don't read very well, and of course, the students who do read well, you don't need the program, and so you'd only assign to students who are below some threshold. So you get this kind of thing. This is come to be called the regression discontinuity design and it requires a different treatment than that which we have given this far. In addition, there are a lot of questions and causal inference that lead to estimands other than those we have studied. For example, scientists are often interested in the pathways through which a cause effects and outcome. Thus, you want to know the effect of one or more intermediate variables on the outcome. But even a treatment assignment is unconfounded, the intermediate variables will not in general be unconfounded with respect to the outcome. Now, statisticians have studied this problem, there are some twin literatures on mediation and principle stratification, which are very useful, important literatures. Another important area is longitudinal causal inference. Here treatments are assigned sequentially and may depend on previous assignments, responses, and covariates. For example, if you give a person a pill and then you want to see what to do next, then at time 1, you might record how they did. Did they improve? Did they not improve? Et cetera. And then at time 1, you might want to assign them to no longer take the pill, continue to take the pill, take a different pill, et cetera. The way in which you do that could depend upon not just the response at time 1, but it could depend on covariates, et cetera. It could of course often will depend upon how they did. So taking into account these consideration, it turns out is not trivial at all but very important. Finally, we shall also briefly take up the case where the stable unit treatment value assumption doesn't hold and that can often not hold in social science types of setups where the treatment assignment that one unit gets may effect the outcome that another unit gets. That's the idea, and social networks that occurs in that kind of context as well, that's called interference. There's another component which we will discuss two stable unit treatment value assumption as well, but that's the main thing, and let's focus over fair amount of literature these days.