This course covers commonly used statistical inference methods for numerical and categorical data. You will learn how to set up and perform hypothesis tests, interpret p-values, and report the results of your analysis in a way that is interpretable for clients or the public. Using numerous data examples, you will learn to report estimates of quantities in a way that expresses the uncertainty of the quantity of interest. You will be guided through installing and using R and RStudio (free statistical software), and will use this software for lab exercises and a final project. The course introduces practical tools for performing data analysis and explores the fundamental concepts necessary to interpret and report results for both categorical and numerical data
Multiple comparisons
Loading...
En provenance du cours de Duke University
Inferential Statistics
1117 notes
À partir de la leçon
Inference for Comparing Means
Welcome to Week Three of the course! This week we will introduce the t-distribution and comparing means as well as a simulation based method for creating a confidence interval: bootstrapping. If you have questions or discussions, please use this week's forum to ask/discuss with peers.
Rencontrer les enseignants
Mine Çetinkaya-RundelAssociate Professor of the Practice
Department of Statistical Science
So far, we've introduced ANOVA as a method for
comparing many means to each other concurrently.
Finding a statistically significant result at the end of an ANOVA,
however, only tells us that at least one pair of means are different, but
not which pair of means are different.
In this video, we set out to answer this follow up question, and when doing so,
we're going to discuss how to control the Type I error rate that would be
inflated by doing many paralyzed tests in the quest for
identifying groups whose means are significantly different from each other.
Remember that to determine whether two means are different from each other,
we use t tests.
And with each test that you do, you incur a probability of doing a Type I error.
The probability of committing a Type I error is the significance level
of the test, which is often set to 5%.
So when you do multiple tests, you're going to be
inflating your Type I error rate, which is an undesirable outcome.
Thankfully, there is a simple solution.
Use a modified significance level that is lower than the original significance
level for these pairwise tests, so that the overall Type I error rate for
the series of tests you have to do can still be held at the original low rate.
Testing many pairs of groups is often called multiple comparisons, and
a common modification that we use when doing
multiple comparisons is the Bonferroni correction,
which uses a more stringent significance level for each of the pairwise tests.
More specifically,
we adjust our alpha by dividing it by the number of comparisons being considered.
For example, if you have four groups in your ANOVA, and it does yield
a significant result, then you need to compare group one to group two,
two to three, three to four, so on and so forth.
But counting these out is somewhat tedious and error prone, so
we usually use a shortcut formula for determining this value and
then use this value to adjust the significance level.
So the Bonferroni correction is a two step process.
First, find the number of comparisons as k times k minus one divided by two,
and then correct your original alpha by this level,
as alpha divided by the number of comparisons.
Let's practice this.
The social class variable we have been considering has four levels.
That is, k = 4, and if the original alpha was .05, what
is going to be the modified significance level for the multiple comparisons test?
Each group needs to be compared to three other groups,
resulting in a total of 12 comparisons.
But, if you have already compared group one to group two, say middle class to
lower class, you don't need to go back around and compare lower to middle again.
So, the total number of comparisons can be cut in half.
Hence, the formula k times k minus one over two,
which results in six total comparisons, and
we can use this number to correct our significance level down to 0.0083.
This is the significance double we're going to use for the pairwise comparisons,
to see if two pairs of means are different from each other.
There are a couple other considerations when doing
these multiple comparisons after ANOVA.
First, it's related to the constant variance condition.
Since for
ANOVA, we need to meet this condition, we need to now rethink the standard error and
the degrees of freedom to be used in the multiple comparisons test.
And, of course, now we have a new modified significance level that we're going to
compare to the p-values of these tests to determine significance.
So what are these consistent standard error and
degrees of freedoms we need to use?
The formula for the standard error should seem familiar to you,
except that instead of the individual group variances,
we're actually using mean squared error from the ANOVA table.
Remember, the mean's squared error is actually the average within group
variance, so
we're still getting at the same thing, the individual group variances, but now,
we have a consistent measure that we can use for all of the tests.
If, indeed, the constant variance condition is satisfied,
this value should be very close to your group variances anyway.
And the consistent degrees of freedom is going to be the DF
error from the ANOVA output, as opposed to the minimum of the sample sizes minus one,
from the two groups that we're comparing.
So let's put all of this information together.
Pick one of the pairs of groups and do the comparison.
Is there a significant difference between the average vocabulary scores
between middle and lower class Americans?
Our hypotheses are, in the null hypothesis, there is no difference,
the averages are equal, and the averages are different for
the alternative hypothesis.
We calculate our T-score as the difference between the two group means,
minus zero, our null value, divided by the standard error,
calculating the mean square error from the ANOVA output.
That is 6.76 for the average middle class score, minus 5.07 for
the lower class score, divided by the mean square error,
which yields a point estimate of 1.69, and
a standard error of 0.315, for an overall T-score of 5.365.
And the degrees of freedom comes from the ANOVA table, 791.
Let's clean up our plate here, write down our test statistic and
the degrees of freedom, and
before we can finally get to the p-value, remember to always draw the curve.
So here is our T-distribution with 791 degrees of freedom, which by the way,
remember, is going to be just like the normal distribution at this point,
because of the really high degrees of freedom.
And a T-score that's so high,
that's five standard deviations away from the center,
is going to result in a really, tiny, tail area, because it's really unusual
to get an observation that's more than five standard deviations from the mean.
It's also clear from the sketch with the really skinny
tail areas when you get that far from the center.
We can also use r for this.
We use the pt function, and remember to multiply one of your tail areas by
2 to account for the 2-sided alternative hypothesis, and
also remember that the significance level we're going to use for
this test is that modified significance level we calculated earlier.
So even though that was very conservative and stringent, because we
have a pretty tiny p-value, we actually can reject the null hypothesis again.
Explorer notre catalogue
Rejoignez-nous gratuitement et obtenez des recommendations, des mises à jour et des offres personnalisées.
Coursera propose un accès universel à la meilleure formation au monde,
en partenariat avec des universités et des organisations du plus haut niveau, pour proposer des cours en ligne.
© 2018 Coursera Inc. Tous droits réservés.
