However, what you can do is sample the function at different pairs of Xs.

What that means is I can provide the function with an X1, an X2 and

see what the value of the function is at that X1 and X2.

And if I could do this with every X1 and

X2, it would be very easy to see what the maximum of the function was.

I would just choose the X1 and X2 that yielded the greatest value of F.

However, this is often infeasible

especially if we have a very high dimensional space of inputs.

So what does one do?

One possibility would be to choose a bunch of random pairs of X1 and

X2, and see which one yielded the highest path.

So maybe we would choose an X1, X2 that yielded that F right there.

Maybe one that yielded that F, maybe one over there,

maybe one there, maybe one there.

And then at the end of the day,

once we have chosen a finite number of random X1, X2 pairs.

We would just say okay, which of those pairs yielded the highest value of F?

And we'll say that's the maximum.

However, that's actually not going to work very well and

you can probably see why that's true based on this drawing.

When you're just randomly sampling,

you're using no strategy to try to figure out where the maximum might be.

You're just choosing randomly, so can we do something a little bit better?

Instead, we're going to use a method called gradient ascent and

how gradient ascent works is that you pick a starting point.

And then instead of just randomly moving to any other X1,

X2, you instead move to an X1 X2 pair that is in your little neighborhood.

So, you only move to points that are nearby.

So this is very similar to what you would do if you were

maybe in a foggy city or in a city at night where you couldn't see very well.

And you were task with finding the highest point of the city.

You obviously, can't randomly jump around to take different

samples of different points of the city Instead you can only move locally.

All right, so now that you're constrained to only move locally,

to only be able to move to locations very nearby to where you currently are.

What's the well, if I were trying to find a highest point in the city,

what I would do is I would always walk in the steepest direction.