0:00

Now, I want to continue with the Fokker-Planck Equation in log

space that we derived in the last video, and which I'll show you here once again,

along with the typical form of the potential that can arise for such problem.

Such potentials with the metal stable state and

decaying potential behind the barrier are in fact, well studied in physics,

where they describe tons of differences in distance phenomena.

In particular they describe the famous problem of

a decay of a metastable state in statistical physics,

which was solved by Kramer in 1940s.

It is for this reason that this problem is sometimes called a Kramer Escape problem.

We will talk about it in a sec but first let's discuss something more

conventional with the Fokker-Planck equation,

namely the existence of a steady state and time independent solution.

This problem is easy to solve, at least formerly.

So for a steady state the left-hand side of the Fokker-Plank equation

should vanish.

But the right-hand side of this equation is a total derivative

of a quantity called the probability current.

So if the left-hand side vanishes, we can express the Fokker-Plank

equation as a conservation of the probability current.

And the in first stationary state, also called the ground

state if the state exists then the current should vanish.

And this simple argument leads to a suggestive status

state solution in the form shown in equation 51 here.

2:13

Here such a distribution is derived from a physics motivated model.

Now, this distribution has a normalization factor Z in front of it.

And this part is a bit tricky but also the most interesting one.

The point is that a steady state density only exists if it's normalizeable

which requires that the normalization constant Z should be finite and positive.

But if there is escape from a metastable state through tunnel link,

the system cannot have a genuine steady state.

What can be obtained instead in such a system would be a Quasi-stationary state

that might be very long lived in fact, but still will decay in a very long time.

And the time it takes for

such state to decay is driven by the height of the pillar and the noise level.

The existence of such process shows up in divergence or

zeros of the normalization factor Z.

That is called a partition function in physics.

And it turns out that such a scape and tunneling problems can also be

described using the language of quantum mechanics.

This can be done starting with the Fokker-Planck equation as follows.

First, we change the dependent variable and

try the unknown density pure white as shown in equation 52 as

a product of the exponential factor and a new unknown function K(y).

And please note that the exponential factor here is a square root of

the steady-state distribution.

Now if we plug such answers into the Fokker-Planck Equation, we get another

equation for the newer non-function K(y), and this is shown in equation 53.

This equation is actually the same as the Schrodinger equation of quantum mechanics,

where sigma squared plays the role of the Planck constant.

In quantum mechanics, the Schrodinger equation is the main equation of dynamics,

and it replaces the Newtonian, Lagrangian, and Hamiltonian methods for

the classical mechanics to the setting of quantum domains.

But in the original Schrodinger equation, there is an additional imaginary unit,

i, in the left-hand side in front of the time derivative.

And here there is none.

So this equation can be interpreted as a Schrodinger equation in imaginary time.

So as a result, we find that the classical stochastic system is

mathematically equivalent to quantum mechanics in imaginary time,

and a variance sigma square playing the role of the Planck constant.

The Hamiltonian for the result in Schrodinger equation for

this case is shown in equation 54.

5:15

And it turns out that the Schrodinger equation has many,

many interesting properties.

And one of them is called supersymmetry or SUSY as physicists call it.

Not only Schrodinger equation has supersymmetry, but

this one obtained from the classical stochastic dynamics does have it.

An the supersymmetry of the Schrodinger equation is based on the observation that

are Hamiltonian H can be factorized as a product of two operators,

A and A+, as shown in equations 55, and 56 here.

These operators are sometimes called the supercharge generators.

And the function, U prime is called the superpotential.

Now if we have these generators we can swap the order and

get a new Hamiltonian H+ shown in equation 57.

And the most interesting thing about this pair of Hamiltonians H and

H+ is that they have the generate spectra for all states excluding

the lowest energy state with zero energy, if such state exists.

And these can be seen using the simple chain of

transformations shown in equation 58.

If Si n is an eigenstate of H, who is an eigenvalue En,

then we can form a new state, A times this state, and this state

will be an eigenstate of the SUSY partner Hamiltonian H+ with the same energy.

And this means that all eigenstates of H with non zero energy

should be the degenerate with eigenstates of H+.

And now SUSY can be unbroken or spontaneously broken.

If it's unbroken, a ground state with zero energy exists.

On the other hand, if the energy of the ground state is larger than zero,

supersymmetry is broken.

And it turns out that mechanisms of breaking supersymmetry in

quantum mechanics and quantum field theory are the same mechanisms

that lead to tunneling and escape from metastable potential.

So how the escape looks like.

If we go back to the language of classical statistical physics,

then the process is described as an event when due to thermal fluctuations,

a particle gets enough energy to jump over the barrier.

And the probability of such event will be obtained as a product

of two factors, the Arrhenius factor B and pre-factor A.

The Arrhenius factor B is shown here in equation 60.

And its exponential in parameter Eb that gives the height of the barrier.

So if a barrier is very high then the actual escape probability can be

very tiny.

And vice versa, if a barrier is not too high, or

the energy of a particle is such that it's near the top of the barrier

then the escape probability might become quite noticeable.

And the remaining pre-factor A can also be computed for

one dimensional diffusion it turns out that this factor is proportional to

the frequency of oscillations, omega near the bottom of the potential well.

That's shown in equation 61 here.

It turns out that the same expression can also

be obtained from an equivalent quantum mechanical formulation.

And in this case it turns out that tunneling can also be described by

the laws of classical mechanics.

But applied in imaginary time.

In imaginary time, the kinetic energy becomes negative, and

the action becomes imaginary.

As you can see, if you look again at our equation,

for, which I did here as equation 62, for your convenience.

So for this case, the expression in the square root in there

integral is negative and therefore the action itself is imaginary.

But because the weight of the action is I times S,

this produces exponentially suppressed tunneling in quantum mechanics.

And finally, a few more words about the tunneling effect.

This effect is non perturbative as we said, so

it cannot be obtained as an expansion in small values of parameters kappa and

g around a model with a trivial vacuum x=0.

It turns out if we still start developing such

perturbative schemes they become divergence series and

the origin of these divergence of perturbative series and

tunneling turns out to be the same.

A mechanism for this is similar to divergence of Quantum Electro-Dynamics

that was discovered by Freeman Dyson in 1950s.

You can read more about such problems since statistical physics and

quantum mechanics in your weekly reading quiz for this week.

And for now we just want to summarize.

So we saw that reinforcement learning and inverse reinforcement learning can be used

not only to compute specific numbers in finals, but also to construct new models.

And we presented one such simple model for market dynamics that is inspired or

kind of derived from reinforcement learning in our previous course.

Now, in this week we took another look at the same model and

found the need to extend it by introducing the cubic non-linearity.

And this falls from the analysis of behavior of the model and

is needed for stability.

This cubic non-linearity can also probably be derived directly from

the enforcement learning approach.

But here we take a simpler route and just added this to our phenomenological grounds

using arguments based on asymptotic analysis on electricity and symmetries.

And all these are common are useful tools in physics.

In particular symmetries play a major role in determining characters or

phase transitions between different phases of matter.

Here we use such sort of analysis in a similar way to using

prior Bayesian statistics.

This is something that is not directly in the data but

should hold anyway based on some more general arguments.

In the course project for this course you will analyze the model that you

did in your previous course, but this time with keeping a non-zero value of g.

And hence with keeping a cubic non linearity.

And this will be it for this lesson.

And see you the next week.