Apprendre>
santé>
Psychologie>
Mindware: Critical Thinking for the Information Age cours>
Base Rate

Loading...

Base Rate

You will learn about the kinds of systematic errors we make when trying to predict the future. You will learn about regression to the mean and why you should assume that extreme values on a variable will be less extreme when next observed. You will learn how to think about observations in terms of true score plus error. You will learn about the concept of base rate and why it must be taken into account when estimating probabilities of specific events.

Most professions these days require more than general intelligence. They require in addition the ability to collect, analyze and think about data. Personal life is enriched when these same skills are applied to problems in everyday life involving judgment and choice. This course presents basic concepts from statistics, probability, scientific methodology, cognitive psychology and cost-benefit theory and shows how they can be applied to everything from picking one product over another to critiquing media accounts of scientific research. Concepts are defined briefly and breezily and then applied to many examples drawn from business, the media and everyday life. What kinds of things will you learn? Why it’s usually a mistake to interview people for a job. Why it’s highly unlikely that, if your first meal in a new restaurant is excellent, you will find the next meal to be as good. Why economists regularly walk out of movies and leave restaurant food uneaten. Why getting your picture on the cover of Sports Illustrated usually means your next season is going to be a disappointment. Why you might not have a disease even though you’ve tested positive for it. Why you’re never going to know how coffee affects you unless you conduct an experiment in which you flip a coin to determine whether you will have coffee on a given day. Why it might be a mistake to use an office in a building you own as opposed to having your office in someone else’s building. Why you should never keep a stock that’s going down in hopes that it will go back up and prevent you from losing any of your initial investment. Why it is that a great deal of health information presented in the media is misinformation.

Visualiser le programme de cours

Compétences que vous apprendrez

Sunk Costs, Cost–Benefit Analysis, Cognitive Bias, Decision-Making, Data Analysis, Statistical Inference

Enseigné par

Richard E. Nisbett
Theodore M. Newcomb Distinguished University Professor

Transcription

So let's talk about base rate.

It's a concept that's critical to many prediction problems.

Again, we'll start this segment with a quiz.

Imagine that a physician has administered a test for

prostate cancer to a very large number of men in their 50s.

He finds that if a man has cancer,

there's an 80% chance that the test will reveal it.

That is to say, an 80% chance that the man will test positive for cancer.

Charles who is 53 years old has taken the test, and it comes out positive.

So, what's the probability that Charles has prostate cancer?

The answer is,

we can't possibly know because we also have to know the probability

that a man will test positive for cancer when he doesn't have it.

And in fact all most medical tests

do produce such false positives they are called.

So let's say that you know that the false positive rate is 10%,

10% of men who test positive do not in fact have prostate cancer.

So the question is what is the probability that

Charles has prostate cancer given that he tested positive.

We still can't know because we also have to know the base rate for

prostate cancer in the population in question.

Let's say the base rate is 1% 1% of all

men in their 50s have prostate cancer.

That's the base rate.

So now what is the probability that Charles has prostate cancer

given that he's tested positive for it.

We can actually now answer the question although it's complicated.

To make it simpler, let's think about a population of a 1,000 men.

And the statistics I'm about to show you were not real ones,

they're just chosen to make it reasonably easy to figure out the probability

that someone has a disease given that the test says the person does.

So it's another old friend here, a two-by-two table.

We have some people who get a positive test.

Yes, some people who do not test positive.

And we have some people, who have cancer and some people who don't.

And we're trying to predict the likelihood that someone has cancer

when the test says he does.

So we got a 1,000 people.

We know that 10 of these people have, that's 1 in a 100, have cancer and

8 of them have been correctly diagnosed with it, that's 80% of 10 which is 8.

Two of them have been incorrectly diagnosed.

They have cancer but it doesn't show up.

Then we know that total of 990 men don't have cancer.

We know that 10% of those men are going to get false positives, so 10% of 990 is 99.

Now, how do we find out whether Charlie actually has prostate cancer?

Well, we divide the number of men who have prostate cancer and

test positive by the total number of men who test positive.

That's 8 plus 99 and the rate there is

.008/(.008+.099) or

a numbers 8 / 107 Which is 7.5%.

So Charles who was panicked when he heard a test that correctly

identifies man who have prostate cancer with 80% accuracy.

And he has been identified with having prostate cancer.

He's going to be relieved when he finds out that the actual probability that he

has cancer is not 80%, it's only 7.5%.

So if you're given a positive diagnosis for

some disease, here's what you need to know.

You must know the percent of people who have the disease who will

correctly test positive.

The percent of people who do not have the disease who will incorrectly test positive

and the overall percent of people who have the disease, that's the base rate.

Then you divide the number of positives who have the disease

by the total number of positives.

If you were unable to figure out the probability that Charles has the disease

given those three numbers,

the good news is that most physician can't figure that out either.

But of course that's also a bad news because many of those man who

are false positives are going to have biopsies which involve some risk and

some discomfort so testing has its dangers.

A decision about whether to test depends on knowing what fraction of men who test

positive, and do in fact have the disease would have died of it.

And in fact the very high fraction of men who test positive with currently available

techniques, would not have died of the disease.

As a consequence,

the government is now recommending against routine testing for prostate cancer.

The exact same logic prompted the government

recently to recommend that there should not be routine testing for

breast cancer for women below the age of 40.

Now, I should say both of this recommendation were met with anger and

disbelief on the part of many people.

It's safe to say, however,

that most such people were not sufficiently aware of the implications

when the disease has a low base rate of occurrence, it's uncommon.

The disease has a high false positive rate.

Further diagnostic procedures pose problems.

And even if you have the disease, you might not die from it.

Now lets look at another kind of problem.

A cab was involved in a hit and run accident at night.

Two cab companies, the Green and the Blue operate in the city.

You are given the following data.

85% of the cabs in the city are Green and 15% are Blue.

A witness identified the cab as Blue.

The court tested the reliability of the witness under the circumstances

that existed on the night of the accident and concluded that the witness correctly

identified each one of the two colors 80% of the time and failed 20% of the time.

What is the probability that the cab involved in the accident was Blue

rather than Green?

Please say your answer out loud, or write it down.

And now a similar problem, panel truck problem.

A panel truck was involved in a hit-and-run accident at dawn

in an alley behind a restaurant.

The only panel trucks that come down that alley are the white

ones of the Pacific Bread Company and the yellow ones of the Mountain Milk Company.

You are given the following data.

The white Pacific trucks cause slightly more than five times as many panel

truck accidents in the city as the yellow Mountain Trucks.

A witness identified the truck as yellow.

The court tested the reliability of the witness under the circumstances that

existed in the early morning hours of the accident and concluded

that the witness correctly identified each one of the two colors 80% of the time.

And failed 20% of the time.

What is the probability that the truck involved in the accident was

a Yellow Mountain truck rather than a White Pacific truck?

Okay, so I hope you didn't say, for the cab problem,

that there's an 80% chance the cab is blue.

because the base rate for cabs, blue cabs, versus green cabs is only 15%.

So you have to take that into account, somehow, in the problem.

To see that, suppose the percent of cabs that are blue was one half of 1%.

Very rare, in other words.

So 99.5% of the cabs are green.

And the witness is wrong 20% of the time.

This means that a very high fraction of the time that the witness says the cab is

blue, it's actually green.

In fact, nearly every time that the witness says it's blue, it's green.

So you have to take into account the prior odds for blue, or the base rate for blue.

The actual probability that the cab is blue,

given that the witness says the cab is blue, is a little tricky to calculate,

but it's actually the same procedure that we used for the prostate cancer diagnosis.

Let's assume that there are a total of a 100 cabs in the city.

We're going to set up a two by two table.

So the witness says the cab is blue, or

he says it's green, and the cab is in fact blue or in fact green.

And we know that 15 of these cabs are blue, and 85 are green.

We know that 12 of the cabs that the witness says are blue are blue,

because he's right 80% of the time.

80% of 15 is 12.

And we know in 68% of these incidents,

when the witness says the cab is green, it is green.

That's 80% of 85.

And we know that 17 times when the witness says it's blue, it's actually green.

And three times when the witness says it's green, it's actually blue.

So how do we figure out the likelihood

that that cab is blue given that the witness said that it's blue?

We know that we need to divide the number of times that the witness says it's blue,

and the cab is blue by number of times that the witness says it's blue,

and it's blue.

And the number of times the witness says it's blue, but it's green.

That's 12/29, which is 0.41.

So, given that a witness with 80% accuracy says

that a cab was blue when only 15% of them were blue is 41%.

So the fact that the witness said it's blue

doesn't even get us up to having our best guess be it was blue.

If that's what the base rate is.

Because of the weakness of our grasp on the relevance of base rate,

we place too much confidence in information about a particular case,

relative to the base rate for the kind of event we're trying to predict.

Information about a particular case is sometimes called

individuating information.

This gets us in big trouble when that information is fallible.

But even when people know the information is fallible,

they tend to ignore the base rate.

In trials, when the witness points to someone he saw fleetingly at dusk and

says, that's the man,

such testimony is usually treated as if it's almost surely true.

Even when the numbers tell us it's probably false.

Well, I'm betting that you made more use of the base rate for

the panel truck problem than you did for the cab problem.

We're actually pretty good at taking base rate information into account when there's

a causal interpretation.

When you hear that white trucks cause more than five times as many accidents as

yellow trucks, this creates an image of hellions behind the wheel of

white trucks playing bumper cars in the city streets.

And we don't forget that when the witness says that the offending truck was yellow.

Now we'll look at a problem that will deal with some pretty familiar seeming events.

A college senior that we'll call David L was admitted to two colleges.

One was a large public university and one was a small private college.

The two schools were about equally expensive, equally selective, and

about equally far from home.

David had several friends at each college.

His friends at the large university liked it.

They found it intellectually stimulating and they had good friends there.

His friends at the small college were not so happy.

They didn't find it a very stimulating place to be and

they weren't crazy about either their teachers or their fellow students.

David went to both schools for a day.

At the large university, he was given the brushoff by a couple of teachers

he wanted to see, the student he met didn't seem terribly interesting,

and he just didn't like the feel of the place.

At the small college, he met several students who were bright and

fun to be with, a couple of professors who took a personal interest in him, and

he left feeling upbeat.

Which school do you think David L should attend?

Why?

The great majority of University of Michigan students with no statistics say,

he outta go to the large university.

Why is that?

Well, because he's choosing for himself, not his friends.

Can you think of a principle that might suggests this reasoning is dubious?

I hope you're thinking about the law of large numbers.

His friends have spent scores, if not hundreds of days, at their school.

He spent one day.

His sample size is one, compared with hundreds, for the total, for his friends.

We know that sample values can be pretty far from population values

if they're small.

And intuitively, you know that you could meet a few atypical students and

encounter a few atypical professors.

And you might be off your game.

You've got a cold or a spat with your parents.

The whole college might be a bit off.

Maybe it's a particularly bleak day during exam time.

On the other hand, maybe the school just won the play-off in basketball.

And it's true, he's not his friends, but, and now I'm going to tell you one of

the most important things a social psychologist can tell you, and

that is, if you're like most people, then,

like most people, you think you're not like most people.

But you are.

If everybody you know liked the movie, odds are strong that you will too.

And by the way, your friends are not most people.

They're people you selected as friends,

partly because of shared interests and values.

So your sample of friends is biased to be more like you than most people are.

And that bias means you should be heavily influenced by their experiences.

In short, David L has extremely good base rate data.

The firsthand information David L has could be misleading.

Just for interest's sake, I'll tell you that I actually live my base rates.

Some of my friends think I'm a little far gone on that but

I don't judge books by their cover.

I basically don't read books unless they've been highly recommended by

people I trust.

Same is true for movies.

Let me tell you about how I made my decision to come to

the University of Michigan.

I was at a school on the East Coast, a good school and

I was asked by folks at Michigan, would I come to interview?

Everything I knew about the University of Michigan and

the psych department there and the City of Ann Arbor was good.

I made up my mind to go, if I got an offer.

And I made up my mind not to let myself be influenced by the visit.

A good thing too, because my visit happened in February,

very cold, patchy, dirty snow on the ground.

The Chairman and

his executive committee spoke mostly about baseball during my interview.

The dean, obviously didn't hear a word I said.

One of the least interesting and

pleasant people in the department monopolized my time.

Fortunately, I stuck to my guns and went to the University of Michigan.

As it turns out, I actually never heard the word baseball again.

Football is a different matter, no baseball.

I found out that the Dean was in the midst of an incredible student crisis.

There was a demonstration that was feared it would become violent.

So it's hardly surprisingly that the Dean was not mentally present.

And I was able to avoid pretty easily that guy who wasn't so

interesting and pleasant.

I want to make one last useful point about base rates for you.

in July 1997, the proposed new

Scottish Parliament building was estimated to cost 40 million pounds.

Two years later, the budget had become 109 million.

That estimated cost increased twice in 2002, ending the year 295 million pounds.

It rose three times more in 2003, reaching 375 million pounds.

The building was finally completed in 2004,

way behind the schedule at a final cost of 431 million.

A 2005 study examined rail projects undertaken worldwide for

the 30 year period beginning in 1969.

In more than 90% of the cases,

the number of passengers projected to use the system was overestimated.

Even though these passengers shortfalls were widely publicized,

forecasts did not improve over those 30 years.

On average, planners overestimated how many people would use the new rail

projects by 106% and the average cost overrun with 45%.

There is no evidence that that changed over the course of 30 years.

Rail planners didn't bother to take into account this dismal base rate.

You might say those are government projects and

even government's biggest fans don't tend to praise government for

timeliness or thriftiness or efficiency of planning.

How about ordinary citizens?

How do they do at planning projects?

How well do you think most Americans do in

estimating the cost of kitchen remodeling projects?

Do you think they overestimate and

they're pleasantly surprised at how little the kitchen cost?

Maybe they hit it on the nose, or underestimate by 10%,

30%, 50%, 70%?

A survey of homeowners who had remodeled their kitchens found that on average,

they had expected the job to cost $19,000.

In fact, they ended up having an average of $39,000.

They underestimated by more than 100%.

How about you and me?

Do the papers and

reports that you write get produced in the amount of time that you anticipated?

Mine don't.

Why not?

I sit down and look over my notes and plans and

carefully estimate how much time each part of the project will take.

I say, let's see, the introduction just covers standard stuff I've done before,

a half day is enough.

Methods are always a breeze to write, let's say a day at most.

Results could be a little tricky, there are some things I don't know

exactly how am I going to handle, let's say a day and a half and

then discussions are never big deal, they're never very long.

So I'm going to say, three and a half days, four days tops.

And how long does a paper usually take after those calculations?

Nine or ten days, if I'm lucky.

What I'm I doing wrong?

I'm just paying attention to the things I know have to be done and I can't see

around the corners to the myriad little things that are going to crop up.

I can't find that reference by Snerdly.

My colleague never finished one of the analysis and

I don't know how to do it myself really.

In short, I pay attention only to the individuating information about this

particular case and I'm never going to see all the things that produced delays.

How could I do better?

Pay attention to my own based rate for paper writing projects.

At the very least, after I do the calculations and

come up with the estimated time, I should triple it.

Do I do this?

Not really.

I usually just say vaguely that I'm often surprised, so I realize it might

take longer than I think but secretly, I think this time will be different.

It usually hasn't.

So do what I say here, not what I do.

I have learned about other people's foibles.

When a tradesman says a project will take two days, I triple it in my mind so

that I won't be unpleasantly surprised.

When I set deadlines for other people, I make them a week or two earlier than I

actually need the job done, so I'm not likely to be caught in a bind.

Now, that should be as easy for you as it is for me.

Set deadlines for other people with an eye to your safety and convenience.

The next lesson is on cognitive biases.

That's the mental procedures that we use to understand the world and

how those procedures can sometimes go awry.

À propos de Coursera

Cours, Spécialisations et Diplômes en ligne enseignés par des enseignants du plus haut niveau provenant des meilleurs universités et établissements d'enseignement du monde.

Join a community of 40 million learners from around the world

Earn a skill-based course certificate to apply your knowledge

Gain confidence in your skills and further your career

Coursera propose un accès universel à la meilleure formation au monde, en partenariat avec des universités et des organisations du plus haut niveau, pour proposer des cours en ligne.

© 2019 Coursera Inc. Tous droits réservés.

Télécharger dans l'App Store

Disponible sur Google Play