Analysis of Variance ANOVA

Loading...

En provenance du cours de IBM

Data Analysis with Python

403 notes

IBM

Data Analysis with Python

403 notes

Cours 7 sur 9 dans Specialization IBM Data Science Professional Certificate

Learn how to analyze data using Python. This course will take you from the basics of Python to exploring many different types of data. You will learn how to prepare data for analysis, perform simple statistical analyses, create meaningful data visualizations, predict future trends from data, and more! Topics covered: 1) Importing Data sets 2) Cleaning the Data 3) Data frame manipulation 4) Summarizing the Data 5) Building machine learning Regression models 6) Building data pipelines Data Analysis with Python will be delivered through lecture, lab, and assignments. It includes following parts: Data Analysis libraries: will learn to use Pandas, Numpy and Scipy libraries to work with a sample dataset. We will introduce you to pandas, an open-source library, and we will use it to load, manipulate, analyze, and visualize cool datasets. Then we will introduce you to another open-source library, scikit-learn, and we will use some of its machine learning algorithms to build smart models and make cool predictions. If you choose to take this course and earn the Coursera course certificate, you will also earn an IBM digital badge. LIMITED TIME OFFER: Subscription is only $39 USD per month for access to graded materials and a certificate.

À partir de la leçon

Exploratory Data Analysis

Exploratory Data Analysis1:20

Descriptive Statistics4:39

GroupBy in Python3:20

Analysis of Variance ANOVA3:58

Correlation2:29

Correlation - Statistics2:37

Rencontrer les enseignants

Joseph Santarcangelo
Ph.D., Data Scientist at IBM
IBM Developer Skills Network

In this video, we'll study analysis of variance.

Assume that we want to analyze a categorical variable and

see the correlation among different categories.

For example, consider the car data set.

The question we may ask is how different categories of the make feature,

as a categorical variable, has impact on the price?

The diagram shows the average price of different vehicle makes.

We do see a trend of increasing princes as we move right along the graph.

But which category in the make feature has the most, and

which one has the least impact on the car price prediction?

To analyze categorical variables,

such as the make variable, we could use a method such as the ANOVA method.

ANOVA is a statistical test that stands for analysis of variance.

ANOVA can be used to find a correlation between different groups of a categorical

variable.

According to the car data set, we can use ANOVA to see if there is any difference in

mean price for the different car make, such as Subaru and Honda.

The ANOVA test returns two values, the F-test score and the p-value.

The F-test calculates the ratio of variation between the group's mean over

the variation within each of the sample groups.

The p-value shows whether the obtained result is statistically significant.

Without going too deep into the details,

the F-test calculates the ratio of variation between groups' means

over the variation within each of the sample group means.

This diagram illustrates a case where the F-test score would be small.

Because as we can see, the variation of the prices in each group of data

is way larger than the differences between the average values of each group.

Looking at this diagram, assume that Group 1 is Honda and Group 2 is Subaru.

Both are the make feature categories.

Since the F-score is small,

the correlation between price as the target variable and the groupings is weak.

In the second diagram, we see a case where the F-test score would be large.

The variation between the averages of the two groups is comparable to the variations

within the two groups.

Assume that Group 1 is Jaguar and Group 2 is Honda.

Both are the make feature categories.

Since the F-score is large, that's the correlation is strong in this case.

Getting back to our example, the bar chart shows the average price for

different categories of the make feature.

As we can see from the bar chart, we expect a small F-score between Hondas and

Subaru because there is a small difference between the average prices.

On the other hand, we can expect a large F-value between Hondas and Jaguars.

Because the differences between the prices is very significant.

However from this chart, we do not know the exact variances.

So let's perform an ANOVA test to see if our intuition is correct.

In the first line,

we extract the make and price data, then we'll group the data by different makes.

The ANOVA test can be performed in Python using the f_oneway method as

the built-in function of the scipy package.

We pass in the price data of the two car make groups that we want to compare, and

it calculates the ANOVA results.

The results confirm what we guessed at first.

The prices between Hondas and Subarus are not significantly different,

as the F-test score is less than 1 and p-value is larger than 0.05.

We can do the same for Honda and Jaguar.

The prices between Hondas and

Jaguars are significantly different since the F-score is very large.

F equals 401, and the p-value is larger than 0.05.

All in all, we can say that there is a strong correlation between a categorical

variable and other variables,

if the ANOVA test gives us a large F-test value and a small p-value.

Explorer notre catalogue

Rejoignez-nous gratuitement et obtenez des recommendations, des mises à jour et des offres personnalisées.

Coursera propose un accès universel à la meilleure formation au monde, en partenariat avec des universités et des organisations du plus haut niveau, pour proposer des cours en ligne.

© 2018 Coursera Inc. Tous droits réservés.

Télécharger dans l'App Store

Disponible sur Google Play