Analysis of Variance ANOVA

Loading...

En provenance du cours de IBM

Data Analysis with Python

1644 notes

IBM

Data Analysis with Python

1644 notes

Cours 6 sur 9 dans Specialization IBM Data Science Professional Certificate

Learn how to analyze data using Python. This course will take you from the basics of Python to exploring many different types of data. You will learn how to prepare data for analysis, perform simple statistical analysis, create meaningful data visualizations, predict future trends from data, and more! Topics covered: 1) Importing Datasets 2) Cleaning the Data 3) Data frame manipulation 4) Summarizing the Data 5) Building machine learning Regression models 6) Building data pipelines Data Analysis with Python will be delivered through lecture, lab, and assignments. It includes following parts: Data Analysis libraries: will learn to use Pandas, Numpy and Scipy libraries to work with a sample dataset. We will introduce you to pandas, an open-source library, and we will use it to load, manipulate, analyze, and visualize cool datasets. Then we will introduce you to another open-source library, scikit-learn, and we will use some of its machine learning algorithms to build smart models and make cool predictions. If you choose to take this course and earn the Coursera course certificate, you will also earn an IBM digital badge. LIMITED TIME OFFER: Subscription is only $39 USD per month for access to graded materials and a certificate.

À partir de la leçon

Exploratory Data Analysis

Exploratory Data Analysis1:20

Descriptive Statistics4:39

GroupBy in Python3:20

Correlation2:29

Correlation - Statistics2:37

Analysis of Variance ANOVA3:58

Rencontrer les enseignants

Joseph Santarcangelo
Ph.D., Data Scientist at IBM
IBM Developer Skills Network

In this video, we'll study analysis of variance.

Assume that we want to analyze

a categorical variable and see

the correlation among different categories.

For example, consider the car dataset.

The question we may ask is how different categories

the make feature as

a categorical variable has impact on the price?

The diagram shows the average price

of different vehicle makes.

We do see a trend of

increasing prices as we move right along the graph,

but which category in the Make Feature has the most and

which one has the least impact

on the car price prediction?

To analyze categorical variables

such as the make variable,

we can use a method such as the ANOVA method.

ANOVA is a statistical test that

stands for Analysis of Variance.

ANOVA can be used to find the correlation

between different groups of a categorical variable.

According to the car dataset,

we can use ANOVA to see if there is any difference in

mean price for the different car

makes such a Subaru and Honda.

The ANOVA test returns two values,

the F-test score and the p-value.

The F-test calculates the ratio of variation between

the groups mean over

the variation within each of the sample groups.

The p-value shows whether

the obtained result is statistically significant.

Without going too deep into the details,

the F-test calculates the ratio

of variation between groups

means over the variation

within each of the sample group means.

This diagram illustrates a case where

the F-test score would be small because as we can see,

the variation of the prices in each group of data is

way larger than the differences

between the average values of each group.

Looking at this diagram,

assume that group one is Honda and group two is Subaru,

both are the Make Feature categories.

Since the F-score is small,

the correlation between price as

the target variable and the groupings is weak.

In the second diagram,

we see a case where the F-test score would be large.

The variation between the averages of the two groups is

comparable to the variations within the two groups.

Assume that group one is Jaguar and group two is Honda,

both are the Make Feature categories.

Since the F-score is large,

thus the correlation is strong in this case.

Getting back to our example,

the bar chart shows

the average price for different

categories, the Make Feature.

As we can see from the bar chart,

we expect a small F-score between Honda's and

Subaru because there is

a small difference between the average prices.

On the other hand,

we can expect a large F-value between Honda's and

Jaguars because the differences

between the prices is very significant.

However, from this chart,

we do not know the exact variances.

So let's perform an ANOVA test

to see if our intuition is correct.

In the first line, we extract the make and price data.

Then, we'll group the data by different makes.

The ANOVA test can be performed in Python using

the f_oneway method as

the built-in function of the SAI PI package.

We pass in the price data of

the two car make groups that we want to compare,

and it calculates the ANOVA results.

The results confirm what we guessed at first.

The prices between Hondas and

Subarus are not significantly different,

as the F-test score is less than one,

and p-value is larger than 0.05.

We can do the same for Honda and Jaguar.

The prices between Hondas and Jaguars are

significantly different since the F-score is very large.

F equals 401, and the p-value is larger than 0.05.

All in all, we can say that there's

a strong correlation between a categorical variable and

other variables if the ANOVA test gives us

a large F-test value and a small p-value.

Explorer notre catalogue

Rejoignez-nous gratuitement et obtenez des recommendations, des mises à jour et des offres personnalisées.

Coursera propose un accès universel à la meilleure formation au monde, en partenariat avec des universités et des organisations du plus haut niveau, pour proposer des cours en ligne.

© 2019 Coursera Inc. Tous droits réservés.

Télécharger dans l'App Store

Disponible sur Google Play