Explore
For Enterprise
Join for Free
Log In

Parcourir
Rechercher
For Enterprise
Connexion
Inscrivez-vous gratuitement

Example: Chi-Squared Test

Loading...

Practical Predictive Analytics: Models and Methods

Université de Washington

4.1 (290 évaluations) | 30K étudiants inscrits

Cours 2 de 4 dans Science des données à grande échelle Spécialisation

Statistical experiment design and analytics are at the heart of data science. In this course you will design statistical experiments and analyze the results using modern methods. You will also explore the common pitfalls in interpreting statistical arguments, especially those associated with big data. Collectively, this course will help you internalize a core set of practical and effective machine learning methods and concepts, and apply them to solve some real world problems. Learning Goals: After completing this course, you will be able to: 1. Design effective experiments and analyze the results 2. Use resampling methods to make clear and bulletproof statistical arguments without invoking esoteric notation 3. Explain and apply a core set of classification methods of increasing complexity (rules, trees, random forests), and associated optimization methods (gradient descent and variants) 4. Explain and apply a set of unsupervised learning concepts and methods 5. Describe the common idioms of large-scale graph analytics, including structural query, traversals and recursive queries, PageRank, and community detection

Visualiser le programme de cours

Compétences que vous apprendrez

Random Forest, Predictive Analytics, Machine Learning, R Programming

Avis

4.1 (290 évaluations)

5 stars
132 ratings
4 stars
98 ratings
3 stars
31 ratings
2 stars
17 ratings
1 star
12 ratings

PV

Nov 12, 2015

The topic the professor covers are awesome. Going from statistics to machine learning is something very awesome about this course

CC

Jul 20, 2016

Nive that the course covered a broad range of topics.\n\nAnd good to get pushed to do some kaggle competition and peer review.

À partir de la leçon

Practical Statistical Inference

Learn the basics of statistical inference, comparing classical methods with resampling methods that allow you to use a simple program to make a rigorous statistical argument. Motivate your study with current topics at the foundations of science: publication bias and reproducibility.

Outliers and Rank Transformation3:24

Example: Chi-Squared Test3:39

Bad Science Revisited: Publication Bias4:47

Effect Size4:26

Meta-analysis5:05

Fraud and Benford's Law4:09

Intuition for Benford's Law2:50

Benford's Law Explained Visually3:50

Enseigné par

Bill Howe
Director of Research

Transcription

I want to give you another example of how to apply these resampling methods in a different kind of statistical inference context. So the Chi-Squared Test is used for categorical data. So for example, what is the relationship between health and wealth, where we define health in terms of three categories, poor, middle class, and rich, and wealth in terms of, let's say, sick or healthy? And we perhaps do a survey to get tallies of each one of these categories. And we wanna try to understand the relationship here. The chi-squared test is used when we're trying to measure the relationship between two variables or the difference in observation from the expected distribution. So if we tabulate these results like we have here. We have the measured count of those who reported being both sick and poor is 20, and so on. How do we compute the expected value that we're gonna measure the difference from? Well if these two variables, health and wealth, are independent, then we'd expect the same ratio of sick to healthy exhibited in the poor to be the same ratio of sick to healthy in middle and the same ratio of sick to healthy in rich, that it would have no affect. And so what we can do is, count up the total number of overall subjects we have, 110, and take the ratio of sick over 110. There's 46 sick over 110 total. And then multiply that by the number of poor, and that's the expected number of both poor and sick, which is 18.48, and we can repeat that for each one of the combinations of variables. And then subtract the expected value from our measured value to get a marginal, a residual rather, and then square that and then divide it again by the expected value, and that's the chi squared statistic. And then we add that all up for the overall case, the overall experiment. The overall data set, in order to get this drive statistic. And then this is the statistics we want to do the significance test on. So how do we do that? Well, we talked about shuffling the labels and we talked about boot strap resampling methods. Here what you can notice is the calculation depends on the marginal values here. The total number of sick, the total number of poor, the total in the middle and so on. So what you want to do is design a relabeling, a shuffle process that will keep the marginals the same, but should still change the value in order to allow you to do the test. So the way to think about this is that each observation has a single label rather than two labels. It has the sick/poor label, or the sick/middle label, or the sick/rich label, or the healthy/poor label, or the healthy/middle label, or the healthy/rich label, and sort of concatenate these two labels together. Then you can do a shuffle test as usual. And the marginals will then necessarily stay the same. The total number of poor will stay the same and the total number of healthy will stay the same and so on. One other thing I should mention about the Chi-Squared test is it's only valid if every category has at least ten observations in it, it has fewer than that then it's invalid to use. The sample won't be distributed in Chi-Squared.

Explorer notre catalogue

Rejoignez-nous gratuitement et obtenez des recommendations, des mises à jour et des offres personnalisées.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. Tous droits réservés.

Disponible sur Google Play

Coursera
À propos
Direction
Carrières
Catalogue
Certificats
Certificats MasterTrack™
Diplômes
For Enterprise
For Government
Pour le campus

Communauté
Learners
Partners
Developers
Beta Testers
Translators

Se connecter
Blog
Facebook
LinkedIn
Twitter
Instagram
YouTube
Blog Tech

Plus
Conditions
Confidentialité
Aide
Accessibilité
Presse
Contact
Répertoire
Filiales