In this course, you will learn the science behind how digital images and video are made, altered, stored, and used. We will look at the vast world of digital imaging, from how computers and digital cameras form images to how digital special effects are used in Hollywood movies to how the Mars Rover was able to send photographs across millions of miles of space. The course starts by looking at how the human visual system works and then teaches you about the engineering, mathematics, and computer science that makes digital images work. You will learn the basic algorithms used for adjusting images, explore JPEG and MPEG standards for encoding and compressing video images, and go on to learn about image segmentation, noise removal and filtering. Finally, we will end with image processing techniques used in medicine. This course consists of 7 basic modules and 2 bonus (non-graded) modules. There are optional MATLAB exercises; learners will have access to MATLAB Online for the course duration. Each module is independent, so you can follow your interests.
2 - Image Processing and HIV (Part II) - Duration 16:30
Loading...
En provenance du cours de Duke University
Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital
373 notes
Duke University
373 notes
À partir de la leçon
Medical imaging
This is a bonus unit. Enjoy it. Image processing has been very successful in medical imaging, and we will use examples from HIV and brain research to illustrate the importance of image processing in solving societal problems. We will describe the basic tools in these exciting applications, from the acquisition to the analysis.
Rencontrer les enseignants
Guillermo SapiroProfessor
Electrical and Computer Engineering
>> That was our first step. >> Now, we get a lot of these volumes.
>> So, we basically go into our three-dimensional data, that we got from
the tomograms. >> We found, so we have the
three-dimension data. >> We go and look for all the areas
with the virus, and we segment out those spikes, or envelopes.
>> Sometimes that's, that done by hand. >> Basically, somebody goes and places
a box around each one of them. >> Sometimes you can do that
automatically if you have a model of what, more or less, you expect.
>> Then you travel around your image with that model.
>> And you do, a distance at every place.
>> And if the distance is small, you say, oh.
>> It looks like there is 1 of these envelopes here.
>> It's very similar to what we did, we did with non local means.
>> In non local means, we were looking for similar patches.
>> Now, the patch is a model that you have and with it, you go wrong.
>> So, either one of those ways, either manual or with this semi-automatic, you
get this volumes. >> This volumes can be like a
100x100x100 pixels representing, basically, the envelope inside this spike
that we have. >> Now remember, these are noisy,
they're not all the same. >> We need to group them, we need to
align them, and that's what we're going to basically do.
>> We are going to align them first, and there still very noisy.
>> So the alignment is not going to be great because you're trying to align
things that are very, very noisy. >> but it's a good step.
>> You can actually, only if you want, align those that are.
>> Basically, it put a very, very high threshold and very high level of
confidence in the alignment. >> And you say I'm going to align them.
>> Only those that basically after I align, the distance is almost zero.
>> They're almost identical. >> So, then I have more confidence that
I have than. >> Good alignment.
>> Once again alignment is you start and just rotate them until the distance
that we show in the previous slides becomes the smallest possible.
>> Once you have them aligned, then classification becomes a bit easier.
>> So you say okay now I have them all aligned.
>> I have them all in a common space. >> Now I want to group them to try to
only put in the same group those envelopes that basically have the same 3
dimensional shape. >> And there are many ways of doing
this classification or clustering that we need there.
>> But let me just illustrate to you one example.
>> So you have all of them aligned. >> And now you have the distance, the
distance that basically led you to the smallest raw, the smallest alignment
rotation. >> So you have, you align them, you
rotate them, you align them, you have the distance.
>> And now, you basically, how do you group them, you.
>> Group all of them. >> So you pick one, and you say, hey,
bring me all my friends. >> Who are my friends? Don't have very
small distance to me. >> That's 1 group.
>> Then you pick another one. >> And say, bring me all my friends.
>> And who are the friends? Those that I go a very small distance.
>> So you basically group them. >> Once you group.
>> If you did everything right, you can basically average them.
>> And we know that the average would reduce the noise.
>> That's what we taught before. >> We explained when we were doing
image denoising. >> So you can kind of denoise them a
tiny bit. >> And after you denoise them a tiny
bit, you can say, hey, let's try to align them again.
>> Now, there are a bit more clean images.
>> So we are going to align them again. >> We are going to cluster again.
>> We are going to align them again. >> And we are going to cluster them
again. >> And you repeat that for a number of
iterations until you're satisfied. >> Either that things are not changing
too much. >> Or basically you say, this is the
best I can do, so I'm going to stick with that, and that's a process.
>> With the final distance, with alignment, clustering, alignment,
clustering, and the big thing here, once again, and we're going to come again and
again to that, if we have a lot of this >> Small boxes.
>> Now, I want to reiterate to you, all this is done in three dimensions.
>> These 100x100x100 cubes, we have like 4,000 of them.
>> Sometimes we have 10, 20,000 of them.
>> So, a lot of computations. >> We need to align everybody with
everybody. >> We need to compare everybody to
everybody. >> We need to do that many, many times.
>> You can speed up these using different, basically, tricks.
>> One is for this particular case, going to Fourier domain.
>> Normally these things are implemented in GPU's and, and very, very
efficient implementations to make them run in hours and not in weeks.
>> So that's what we have that more challenging, the most challenging part,
computationally is this alignment part because we have to basically try all
these angles, and that's normally done in the, in the Fourier domain.
>> For these types of examples, as we say, because of the missing wedge and
because of the, the speed. >> Now, you have an algorithm.
>> You have, you've developed this image processing algorithm, and you're
going to use it to detect the three-dimensional shape of the envelope
in the HIV virus. >> But then you say to yourself, how do
I know if my algorithm works? Nobody has ever seen the envelope at the resolution
that I want to see it. >> So, how can I validate? And this is
very important when you're talking about medical.
>> Imaging. >> Because you don't want to make
mistake and mislead, let's say, a complete vaccine development because you
made a computational mistake. >> So, what you do in most image
processing, but in particular when you're talking with medical imaging, is you go
slow. >> First, you start from what are
called phantoms. >> You create three dimension shapes
that look like viruses from your prior knowledge.
>> You add noise. >> You project them.
>> You make missing wedges. >> So you make them like they were
acquiring the microscope. >> And you run your algorithm for that.
>> You know the grand truth, because you created it.
>> You hope. >> To reconstruct.
>> And to you don't finish that, you don't move up.
>> So, you solve this. >> You say, great.
>> I put different structures. >> Phantoms I created.
>> I rotate, add noise, add missing wedge.
>> I got them back. >> I'm happy.
>> The next step, because it's almost impossible to do a perfect simulation.
>> Of everything that is going in the microscope, is you take particles that,
for some reason, we know their structure. >> Maybe because they are larger than
the HIVs, maybe because they are more rigid.
>> And they could be acquired with other technologies.
>> And one of them is what's called the GroEL, the GroEL.
>> So, you go into your microscope, you put GroEL, you run your algorithm.
>> You get the 3D shape that everybody knows is the right shape.
>> You say, okay, this is as much validation as I can do.
>> Now, I'm ready to do HIV. >> Even here, you can do a lot of
validation. >> For example.
>> You can take these 4000 samples and drop 500 of them.
>> Do your computations. >> Then drop a different 500 at random.
>> Do your computations. >> And you expect to get very similar
results. >> So, still here you have to do a lot
of validation. >> But this is a regular progress:
phantoms, no data, and then you go into the unknown.
>> Which is the HIV. >> Remember, we are looking for these
tiny spikes here, and which are, we're talking about basically, arms strokes
that's what basically, the size is that we are talking.
>> Now, let me reiterate to you why this is so important in this schematic.
>> These are what we have been seeing, let me show that again.
>> This is basically a true example from this prior tomography, one slice,
and now, this is a schematic of that. >> Now, as I said, when it goes and
tries to latch itself into the membrane, it actually tries to, basically, grab
from something, which is called a CD4, here.
>> And then, the virus has these two. >> Components that are the gp120,
basically this red, and the gp41, this blue.
>> And it's like, you know, you could think maybe as a flower that is opening
and then trying to latch. >> Into the CD4.
>> And once again, it's very important to understand how this confirmation is
latching, how it's contacting. >> How it's basically managing to grab
the CD4 and are there any changes happening when that's.
>> Going on. >> Now, this has been very, very
elusive. >> It's very, very hard to find, but
this is the importance. >> Again, we have this schematic.
>> This latching, this contact, is critical for the virus to go into the
host cell. >> So that's a very important step.
>> Here. >> And we work, if I was lucky to work
on this with people at NIH, at the National Institute of Health.
>> And I'm going to show you some results of these, and much more research
has been done on this area. >> But this is important and enough
>> To illustrate the example for this image processing class.
>> And normally, what you do, is, you basically have the virus in different
states. >> And then you try to look at this 3D
shape in different states. >> And for the particular examples I'm
going to show to you. >> We have 400 viruses.
>> From them, we got 4,000 of these spikes and envelopes under different
states. >> So, what's called unliganded with a
b12 complex and with other eh, eh, complexes.
>> So, just different states. >> We want to concentrate on the image
processing part. >> This is what you see, this is your
raw data. >> You put the virus, you did the
cryotomography, you basically put all the tiles together, this is what you see, and
you say, oh boy, what am I going to do with that?
>> But, you remember that you just developed an algorithm that is working
for other things. >> So you plug your algorithm into
that. >> You start, you find the envelopes,
you do the alignment, classification, alignment classification.
>> And then, you start seeing progress. >> You start from basically, this is
kind of the very beginning. >> Nothing looks okay.
>> If you align things which are completely non, if you average things
which are completely non-aligned, you just get balls.
>> You know, it just, everything going in every direction.
>> Then things start getting better, better, better, better, better, better.
>> You're, things start to appear, and, you know, you start seeing shapes.
>> You didn't see anything, and after iterations, you start seeing shape.
>> Because you're progressively improving your alignment, improving your
classification, and then we know that if we average things that are the same, the
noise goes down and then we can get the shapes that we wanted.
>> So we go from this, which kind of has it there, so we can see
>> Them here. >> And that's where you go and you
either automatically manually as we say, you pick all these particles.
>> And then you can get your reconstruction.
>> this is actually obtained from the tomography.
>> And what we really want, so we are starting to see the 3D.
>> This is just one of these. >> It doesn't have to be exactly this
one, but it's one of them. >> You see that, and you see the
envelopes. >> And basically, you average and
classify those envelopes. >> And here is what you see.
>> You have the membrane that you were expecting.
>> And, I'm going to just run this video.
>> And this is the shape. >> This is the envelope.
>> That you were looking for. >> And then you tried to fit pieces
that, for all the reasons are known, and you see, oh, this is one of my
validations. >> Nobody has seen this in its entire
composition, but they have seen pieces of them.
>> So let's just try to bring them. >> And see if we can basically fit them
properly. >> So this, is the envelopes that you
see here. >> How you go from here to here? Smart
image processing. >> Once you have done that, you can
start doing the biology, and this is what the people from biology teach us, that is
the important part. >> Why are we doing this? We cannot
explain, because now, I can look at the HIV envelope, under different states, and
basically see that their shapes are different.
>> Under different states, and then you can start making biological conclusions
about that. >> And say under certain confirmation
this is what happen under different state when it's actually latching into the CD4.
>> It's changing shape. >> And that's extremely important for
the vaccine development people. >> And then the interaction starts, and
then they ask you more image processing questions and then you try to address
that. >> And that's what we basically is,
illustrate here with this schematic, that basically there is a change in shape, a
change in conformation. >> When there is CD4 binding.
>> And that's what we illustrate here with this change in the arms, kind of an
open. >> you can see these with image
processing. >> You couldn't see that with your
naked eye. >> This is 1 of the really exciting
areas, where image processing actually helps you see something you couldn't see.
>> Before. >> When we talked, for example, about
activity classification, that was to do in the computer something that the human
knows how to do. >> Here is to do something that the
human cannot do. >> We cannot see the 3D shape just from
those projections or from those tomography 3D reconstructions.
>> We need to do this. >> Image processing.
>> I hope you had fun, this is an exciting area, and this is just the tip
of the iceberg of research in image processing, and viruses, and microscopy.
>> But it's an illustrative example about the power of image processing to
solve. >> Real progress.
>> We're going to see more in the next video.
>> Looking forward to seeing you then. >> Thank you.
Explorer notre catalogue
Rejoignez-nous gratuitement et obtenez des recommendations, des mises à jour et des offres personnalisées.
Coursera propose un accès universel à la meilleure formation au monde,
en partenariat avec des universités et des organisations du plus haut niveau, pour proposer des cours en ligne.
© 2018 Coursera Inc. Tous droits réservés.
