What About Big Data?

Loading...

En provenance du cours de Johns Hopkins University

The Data Scientist’s Toolbox

15554 notes

Johns Hopkins University

The Data Scientist’s Toolbox

15554 notes

Cours 1 sur 10 dans Specialization Data Science

In this course you will get an introduction to the main tools and ideas in the data scientist's toolbox. The course gives an overview of the data, questions, and tools that data analysts and data scientists work with. There are two components to this course. The first is a conceptual introduction to the ideas behind turning data into actionable knowledge. The second is a practical introduction to the tools that will be used in the program like version control, markdown, git, GitHub, R, and RStudio.

À partir de la leçon

Week 3: Conceptual Issues

The Week 3 lectures focus on conceptual issues behind study design and turning data into knowledge. If you have trouble or want to explore issues in more depth, please seek out answers on the forums. They are a great resource! If you happen to be a superstar who already gets it, please take the time to help your classmates by answering their questions as well. This is one of the best ways to practice using and explaining your skills to others. These are two of the key characteristics of excellent data scientists.

Types of Questions9:09

What is Data?5:15

What About Big Data?4:15

Experimental Design15:59

Rencontrer les enseignants

Jeff Leek, PhD
Associate Professor, Biostatistics
Bloomberg School of Public Health
Roger D. Peng, PhD
Associate Professor, Biostatistics
Bloomberg School of Public Health
Brian Caffo, PhD
Professor, Biostatistics
Bloomberg School of Public Health

So, a question that often comes up when you

talk about data science is, what about big data?

So big data is obviously different to different people.

So, big data for somebody without a

computer might be 1,000 numbers, but data for,

big data for somebody with access to Amazon

EC2 might be enormous, much, much, much larger.

So, how much data is there?

So, this is an info graphic that says that there in 2001 there

will be 1.8 zetabytes that were created, which is a gigantic amount of data.

But realistically, only a tiny fraction of that data can be used

to answer any specific question that you might have in your mind.

So the question that keeps coming up is what about big data?

So, you often think, see things like, you know, big

data and cloud management and cloud and big data tend to

go together and that's because for some data sets they're so

big that you can't analyze them on your local laptop computer.

So, it really depends on your perspective.

So, this is one of the very first hard drives.

so, created by IBM and so it's it was able to contain much less data

than you could even store right now on your computer or even on your cell phone.

And so what this suggests is that over

time as technology increases, big data will change.

So, one way to solve the big data problem is to just sort

of wait until the hardware catches up with the size of the data.

But most questions that you're trying to answer don't necessarily have the big data

component that necessitates the need of huge

numbers of computers, although sometimes it does.

So why is big data such a big deal now?

Well this is only one example of that.

So, there was an experiment run by Stanley Milgram where he

took 296 individuals and he, what he tried to do was basically

send them a letter and ask them to send a letter from

someone they knew and so forth until they went a specific address.

And so, 64 of these such chains came back, so 64 out of 296.

And from that they found out that there were about 5.2 people in between

the person that originally got the letter

and the person that finally received the letter.

And this is where this sort of six degrees of separation sort of came about.

And so what ended up happening is, is that people nowadays can

collect much more data than they could before and much more cheaply.

So the, these investigators took an instant messaging network and

they looked at 30 billion conversations between 240 million people.

And then they performed a similar sort of experiment to

try to identify how far apart people were, and they

looked at the sort of to analyze the same question

that was looked at with just 64 email chains before.

And they found that the average path length was actually 6.6, so they sort

of upgraded the six points, the six

degrees of separation to seven degrees of separation.

So, what's the take home message here?

The take home message is that it's now possible to collect much

more data much more cheaply than it was before and to analyze it.

But the question is, is how much of that data is useful

for answering the question that you're sort of involved, you're involved in.

This is sort of a tongue in cheek, suppose

it says, don't use Hadoop, your data isn't that big.

So Hadoop is another of these buzzwords you frequently hear around big data, and

it is an incredibly powerful and useful

technique, if your data is very, very large.

But if you have enough memory on your computer, or you have a powerful enough

computer, you can analyze a large number of,

data sets, that you, that are important and

relevant to you without having to get

into the more complicated techniques that are induced

by going, scaling up to massive sized data

sets like those analyzed by Google and Facebook.

Big or small data that you need, regardless of

the size of the data, you need the right data.

So, the data might not contain the answer.

The combination of some data and an aching desire for an answer does not

ensure that a reasonable answer can be extracted from a given body of data.

This was a quote by John Toupee, one of the first data scientists.

And I would add to it, no matter how big the data are.

So, even if you have gigantic data, it might not be big enough

to be able to answer your question if it's not the right data.

Explorer notre catalogue

Rejoignez-nous gratuitement et obtenez des recommendations, des mises à jour et des offres personnalisées.

Coursera propose un accès universel à la meilleure formation au monde, en partenariat avec des universités et des organisations du plus haut niveau, pour proposer des cours en ligne.

© 2018 Coursera Inc. Tous droits réservés.

Télécharger dans l'App Store

Disponible sur Google Play