0:03

And the method we introduce

Â here is called, As-Rigid-As-Possible Shape Manipulation.

Â The goal of this work is to move and deform 2D shapes

Â on the screen, as if manipulating real object.

Â So in the real world, you can pick up an object, with your

Â both hands, and you can manipulate it,

Â moving around, rotate, and pull, or squash.

Â And also you can, you know, shake his head or wave his hand and so on.

Â So you can do many things to interact with it, using your hands on the real world.

Â That's what we do, on a computer.

Â 0:42

And there are a couple of previous works

Â shape, manipulation shape deformation, but they have couple problems.

Â The most popular approach is space walk, using a simple function you can can deform

Â the space, and then you can deform the images in, in embedded in the space.

Â So this is very efficient for compute.

Â However, the resulting deformation is like swapping

Â space and the result is not very realistic.

Â And the other possibility is use physics,

Â or simply meant mass spring water and others.

Â And this is a kind of simulation with real world, so can be very realistic.

Â While this is very slow to converge and can

Â be very unstable, and also cannot handle contradictory configuration.

Â So,

Â 1:36

So, again let me fast describe of the current state of the art.

Â So, this is typical drawing generator.

Â So you can, do the basic things.

Â You can, draw something

Â 1:51

And then move it around.

Â So, you can do it.

Â And you can rotate, and you can also scale them.

Â However, [UNKNOWN] if you want to, like, shake his head or swing

Â his arm, it's suddenly become very

Â difficult and you have to redraw everything,

Â 2:10

basically.

Â And, here is a system we implemented.

Â So basically the same, you draw three dimensional illustration,

Â however, after you finish drawing you can use push pin tool, so you

Â put pin here, and then you can manipulate them, by using these pins.

Â So, if you pull this, you can pull his, his ears.

Â Or you can you can, you know, shake his head, or make him run this way.

Â So you can, deform the illustration very [UNKNOWN].

Â And this is very useful for making animation.

Â Example if you draw this kind of character.

Â 2:51

And then again, this is just two dimensional nothing special drawing.

Â But as soon as I put push pin, then you, you

Â can make them deform, as if this is a physical object.

Â And then, you can also press leg hold, [SOUND] to make motion.

Â And then, in this way.

Â So you can easily make him run.

Â And also you can drag him around, and

Â you can get an interesting animation very quickly.

Â Traditionally, if you want to make this kind

Â of animation, you have to draw many, many drawings.

Â But here you can just draw something, and

Â they're moving around, and you will get an animation.

Â And here's an example.

Â Again, nothing special.

Â Just 2D drawing.

Â But we can press push pin, and press record,

Â and then you can get this kind of animation.

Â A kick, a kick.

Â And then you can draw something else here, [SOUND] and press record.

Â And you can get this kind of animation.

Â So of course, this is not designed for professional animations, but

Â we can quickly generate interesting stories with this kind of technique.

Â Let me show you one more example.

Â So, suppose you have a is here, have a worm here.

Â And then that's a frog fill.

Â And then let's and then let's put push pins,

Â and then for a select code, and one and two.

Â [SOUND] So in this way you can make

Â a crawling worm very easily.

Â And then of course, it's a computer so you can multiply

Â the main very quickly, and you can make an army of worms.

Â 4:53

And so far, I only used hand drawings.

Â But [UNKNOWN] is basically just a shape to formation.

Â And you can apply the same technique for images,

Â so this is again just a two dimensional image, photograph.

Â 5:09

But if you put push pins, then it starts to move as if there is a physical 3D

Â object here, and then you're going to generate interesting animations.

Â [SOUND] And so far, I have been showing just mouse

Â operations, but if you have two hands, you can do more.

Â For example here, you have two handed mouse,

Â and then, two handed mice, so, then you

Â can grab screen with your two hands and

Â then moving around a little bit and deform.

Â 5:44

And if you have a multi touch device, here we use Sony

Â computer science research laboratories smartskin system,

Â and it can detect your fingertips.

Â And you can directly grab object on the screen,

Â and manipulating them as if you have a, physical object.

Â 6:22

So you can use both hands.

Â Also, two people can work together.

Â [SOUND] And a little bit more.

Â So, so far, I showed the deformation of a lesion but shape but

Â you can also apply the same technique for the deformation of a car.

Â So suppose, you have a cob this way and you

Â can grab a cob and then deform it this way.

Â And if you pull more and more, and you get a larger region will be deformed.

Â And then you pull more, and you can deform the entire shape.

Â [SOUND] And finally, this is not very

Â essential, but we, another convenient operation is.

Â smoothing.

Â So this smoothing will remove small noise, and then after

Â you rubbing, then you will eventually get a very beautiful shape.

Â So by combining this pulling and smoothing, you will get a very

Â professional looking smooth illustration, just by starting from a rough sketch.

Â [SOUND] So okay.

Â So now, let me briefly describe the algorithm behind this technique.

Â So here this is our input.

Â And then this is output.

Â A traditional approach is as I said is physical stimulation.

Â So, moving the handles, when it tries to deform.

Â 7:48

Say a little bit by little, by converting fours.

Â In acceleration.

Â And so, for this kind of a step by step computation and it can be very slow.

Â Here what we propose is a kind of instant computation.

Â Or final results from this kind of input information.

Â So input information is initial lets the shape.

Â And handle positions.

Â [SOUND] And then target locations of handles.

Â And then we compute the final result.

Â And what we do is minimize shape distortion, satisfying constraints.

Â So let me describe a little bit more.

Â So again, input is coordinates of handles and new handle positions.

Â And they output this coordinated mesh vertices or free vertices.

Â So these as I input on the system copies all these other vertices.

Â And what we do is we minimize the distortion of triangles.

Â So there are many possible mesh particle positions.

Â But one particular shape, minimize distortion of individual triangle.

Â Here, in this blue, blue triangle, corresponds to

Â this triangle, and then we try to find the

Â shapes that minimize the deformation or distortion of these

Â triangles, so that's a problem we want to solve.

Â 9:07

And mathematically, this is defined like this.

Â So what rate is to minimize, this function, so this function

Â is what is triangle t, we compute the formation.

Â For distortion, depending on the given by a products position.

Â And then we aggregate [UNKNOWN] distortions.

Â And then, we try to find mesh [UNKNOWN] positions that minimize this energy.

Â 9:38

And what we want is energy function.

Â Specific definition of it that holds no cost.

Â For translation and rotation.

Â Translation means moving around, and rotation is rotation.

Â So these are called alleged deformation because

Â there is no deformation to this guy.

Â And in this kind of alleged deformation, there should be no energy.

Â However, if you scale, make bigger or smaller.

Â Or stretch or press here, if you cause this kind

Â deformation to triangle, it should be a cause of energy.

Â So we need very particular energy or cost

Â function or distortion [UNKNOWN] that satisfies this [UNKNOWN].

Â Another important requirement is that E.

Â Energy, or cost, should be quadra, quadratic in you.

Â Which means if u, if this energy is

Â a quadratic to u, then its derivative is linear.

Â So you can solve instantly by solving si, simultaneous linear equations.

Â So that's what we want.

Â 10:51

So okay, ideally we want the [UNKNOWN] functional, and that causes zero energy

Â for translation or rotation, but [UNKNOWN] energy for scale, stretch or shear.

Â Unfortunately, we don't [UNKNOWN].

Â So there's no such energy in this world for quadratic energy.

Â So we therefore combine two complementary sub-optimal energies.

Â So here's a description.

Â So ideally, we want to have this.

Â But in this world, what we have is only.

Â 11:27

Two kinds, so E1 and E2.

Â So E1 is basically, similarity transform.

Â It supports translation and rotation, but also scales and then

Â causes no energy but it can detect stretch and shear, so this is kind of lose energy.

Â And another E2 is more strict energy.

Â It only allows translations, no rotation, no scaling, no stretch, no shear.

Â And then we can have quadratic function.

Â So E1 is a little bit too relaxed.

Â E2 is to a little bit strict.

Â So we combine these two.

Â So what we do is fast apply E1.

Â And their scale is wrong and they're waiting www E2 to fix the scale.

Â So, here's what we want.

Â So as far as the input positions, we fast

Â obtain intermediate result, like this one, by using another E1.

Â 12:21

And, in this case E1.

Â Appropriate handles minimize the, shearing and scaling and

Â scaling [ and stretching, but scaling is allowed.

Â So you get this kind of inflation in this, you know left arm.

Â 12:41

Correct size or individual triangle and then we

Â use energy two, to get the correct answer.

Â So that's the algorithm and, refer to the paper for the details of these energies.

Â [SOUND] So original paper is published

Â is As-Rigid-As-Possible Shape Manipulation, [UNKNOWN] file.

Â And they [UNKNOWN] serious paper to see the details of the energy definition.

Â And as I said, previous existing technique is popular [UNKNOWN] space walk.

Â And representative work is called feature based image metamorphosis.

Â Published was published in 1992.

Â And our technique is using segregation of triangular elements.

Â And this work is inspired by

Â As-Rigids-Possible Shape interpolation technique published in 2000.

Â And also, our work is related to recent works on shape deformation.

Â And I recommend that you to take a look at

Â this [UNKNOWN] paper called on linear variational surface deformation methods.

Â Published in, 2008.

Â [INAUDIBLE]

Â