Overgeneralization and Sample Bias

Loading...

En provenance du cours de Duke University

Data Visualization and Communication with Tableau

1715 notes

Duke University

Data Visualization and Communication with Tableau

1715 notes

Cours 3 sur 5 dans Specialization Excel to MySQL: Analytic Techniques for Business

One of the skills that characterizes great business data analysts is the ability to communicate practical implications of quantitative analyses to any kind of audience member. Even the most sophisticated statistical analyses are not useful to a business if they do not lead to actionable advice, or if the answers to those business questions are not conveyed in a way that non-technical people can understand. In this course you will learn how to become a master at communicating business-relevant implications of data analyses. By the end, you will know how to structure your data analysis projects to ensure the fruits of your hard labor yield results for your stakeholders. You will also know how to streamline your analyses and highlight their implications efficiently using visualizations in Tableau, the most popular visualization program in the business world. Using other Tableau features, you will be able to make effective visualizations that harness the human brain’s innate perceptual and cognitive tendencies to convey conclusions directly and clearly. Finally, you will be practiced in designing and persuasively presenting business “data stories” that use these visualizations, capitalizing on business-tested methods and design principles.

À partir de la leçon

Your Communication Toolbox: Visualizations, Logic, and Stories

Welcome to week 4! This week you will become a master at getting people to agree with your data-driven business recommendations as you learn to deliver a compelling business presentation. You’ll learn about the insight from the intersection of visualization science and decision science, and what this means for you as a data analyst, who seeks to design a compelling and effective business presentations. If you intend to affect people’s decisions, you need to influence where they look. This week we will review a set of tools and concepts you can use to optimize your visualizations and your presentation style. You will soon be a master at getting people to agree with your data-driven business recommendations! <p>By the end of this week, you will be able to:<p><ul><li>Tell stories with data</li> <li>Design effective slide presentations to showcase your data story, and</li> <li> Deliver compelling business presentations</ul></li><p> Remember to refer back to the Study Guide: Designing and Delivering Effective Presentations. You will also complete a graded quiz. <p>As always, if you have any questions, post them to the <b>Discussions</b>.<p> To get started, please begin with the video “Using Visualization to Influence Business Decisions.”<p> I hope you enjoy this week's materials!

The Best Stress-Testers are Teams3:11

Overgeneralization and Sample Bias5:46

Misinterpretations Due to Lack of Controls3:52

Correlation Does Not Equal Causation6:03

How Correlations Impact Business Decisions8:36

Rencontrer les enseignants

Daniel Egger
Executive in Residence and Director, Center for Quantitative Modeling
Pratt School of Engineering, Duke University
Jana Schaich Borg
Assistant Research Professor
Social Science Research Institute

In this video, we are going to talk about how overgeneralization and

sample bias can undermine the logic of your data stories.

Overgeneralization happens when you assume what you are seeing in your dataset

is what you would see if you looked any other dataset meant to assess the same

information, despite the fact that your data is very small or

sometimes it's selected subset.

You will be especially tempted to commit the logical fallacy of overgeneralization

when the sample you are looking at in your data set is small or highly biased.

In our data analysis of data related jobs, for

example, one of the first things you should be wondering is

are data related salaries the same for US citizens as they are for non US citizens?

After all, our salary information is highly biased in that it only reflects

salaries offered to non US citizens.

We can hypothesize, and

try to test whether these salaries will be the same offered to US citizens, but

we shouldn't assume it, especially because there might be other factors related to US

and non-US job applicants that can independently affect offered salaries.

One such factor is gender.

It turns out that there are many more skilled visas submitted for

men than for women.

This means that the median salary for women who are US citizens,

might look very different than the salaries we saw in our data sample.

It's tempting to think that if you just have a big enough data set,

you should be able to overcome most types of sampling bias.

Unfortunately, as anybody who analyzes smartphone data will tell you,

large sample sizes can't completely save you from bias either.

Some of the biggest data sets people analyze these days comes from

smartphone data.

However, smartphones are disproportionately owned by wealthier

and younger people.

That means that any time you use smartphone data,

the information you learn will be biased towards younger and wealthier groups.

If you aren't aware of this, it can lead to poor decisions and

inaccurate predictions.

As some real life examples,

many groups tried to analyze Twitter data around the time of Hurricane Katrina and

Hurricane Sandy to determine whether it would be possible to analyze tweets to get

real-time updates about a disaster zone during natural crises.

It was found that models based purely on tweets will not reflect the on

the ground realities following disasters, because of what are called data shadows or

lack of data from areas with lower average socio-economic status.

Similar data shadows caused problems when the city of Boston used accelerometer data

in combination with GPS data from smartphones to identify potholes.

The program initially did really well at identifying potholes in high income areas,

but missed a lot of potholes in lower income areas.

Of course,

one of the most difficult aspects of overgeneralization is that it can take

a lot of detective work to figure out that your sample is biased in the first place.

Often, teams don't start digging into the details about where

their data set came from until the predictions start to fail.

Here's Elena Grewal again, manager in data science team from Airbnb telling us about

an example of this happening with Airbnb's type of data, and

sharing their thoughts about how to catch mistakes caused by overgeneralization.

>> Often the mistakes are not necessarily analytical mistakes,

but they're cases where the data was behaving in a way that

wasn't actually the way you assumed it to behave.

And so we have many cases like that where you think, oh okay,

we're looking at this table of data that has all of the listings in San Fransisco,

all the homes on Airbnb in San Fransisco.

And then we realize that actually this is a subset of all the listings, and so

our conclusions are not what we had originally thought.

And that happens quite often, and so

I think being kind of single-mindedly paranoid about your data quality is

actually one of the biggest ways that you can prevent mistakes.

I think about it as developing a data intuition, there's this sense that you

have that something's not quite right, and that comes with experience for

sure, where you get really familiar with the data that you're looking at,

and you can say wait, I know that on average the percentage is this.

Why is it different in my table?

Something is wrong.

And that's definitely very important for helping to prevent mistakes.

>> Another way over generalization can get you into trouble is when you have

a lot of missing data in your data set.

Often you'll find there are rows in your data that have entries in some columns but

not in all the columns.

A common way to deal with that is to completely take those rows out of your

data set whenever you do analyses that require entries in the columns

with missing data.

Then you move forward with your analysis as if you never knew any of

the data was missing.

Sometimes that's okay, but other times the missing data

all comes from the same group, perhaps because there is something wrong with how

data is collected from Android phones, for example.

So all the data from Android phone users is missing.

When this is the case, removing the missing data from the data set you

are analyzing will systematically bias your sample.

As a consequence, the results you get might be different then if you had a more

representative sample.

Here are some tips to help you avoid falling into the overgeneralization trap.

First always,

always ask a lot of questions about how your data was collected.

Listen for hints about how the collection methods might have biased your data, and

look for ways to test the data you have for

bias against specific demographic groups.

Second, always check how many data points you have in all the groups

you're looking at.

If you don't have many data points from a specific group, subcategory, or

time point, don't put much weight on the effects you see there.

Third, if you do have a lot of data,

split your full data set into three to five random subsets.

See if you observe the same effects in each one as you see in

the group as a whole.

If not, take caution when interpreting the results from the group as a whole.

It's likely that the effect is either due to chance, isn't that large, or

is only found in a certain subset of your data that you should track down and

characterize.

Fourth, before removing outliers or rows with some missing data from your analysis,

always examine whether there are any characteristics that seem common and or

unique to those outliers or rows.

If so, you will likely wanna try to collect more data with those

characteristics to fill in what you will then exclude.

At the very least you will wanna be aware how you are biasing your results when you

remove those entries from your data set.

Explorer notre catalogue

Rejoignez-nous gratuitement et obtenez des recommendations, des mises à jour et des offres personnalisées.

Coursera propose un accès universel à la meilleure formation au monde, en partenariat avec des universités et des organisations du plus haut niveau, pour proposer des cours en ligne.

© 2018 Coursera Inc. Tous droits réservés.

Télécharger dans l'App Store

Disponible sur Google Play