Hello. The topic of the today's lecture is the differential games.

The first part is devoted to the study of

some preliminary information or

the approaches of how to solve differential games.

The second part is devoted to

the non-cooperative differential games of n players,

where the main question is of how to model the behavior of

players in processes where they have individual preferences,

or each player has his own payoff function.

The third part is devoted to

the topic of cooperative differential games,

where the question is of how to

allocate the maximum joint payoff of players in the game.

So, that the cooperation would

be beneficial for all of the participants.

Let's start with the example called optimization of

advertising costs and consider a market.

On the market, there is a company

who tries to maximize its revenue.

Its revenue mainly depends on the market share.

The only tool that can be used in order to increase

the market share is the advertising.

So, the company can control the advertising costs.

Let's suppose that the company wants to make

a plan for advertising for one year.

Then the question is,

of how it would allocate the advertising costs,

when company need to spend more money on advertising and when not.

In order to construct a mathematical model for this process,

the first thing we need to do is to

define the optimal control problem.

Let's suppose that we have a dynamical system.

In our case, it is a company.

The dynamics of the system is defined by

the system of differential equations or motion equations (2).

The solution of this motion equation is the function x(t),

which defines the state of the game.

In our case, under the state of the game,

we can understand the market share of the company.

The right hand side of the system of differential equations,

also depends on the function u(t),

which has a control function or in

our case, the volume of advertising.

For the control function u(t),

we will consider a class of functions u(t,x).

So, the functions that depend on

t - the time instant and x - the state of the game.

Also, we will suppose that

the conditions of existence, uniqueness and

prolongability of the system of

differential equations (2) for each of such function exist.

In the similar way,

we already defined the control function

or the strategy of the players in one of the previous sections.

When we consider a multi-stage game with perfect information,

the strategy of the player was a mapping

that for each vertex

from the set of personal moves of the player i,

assigns the next vertex on the graph.

It is important that we define

the strategy as a function of any vertex.

In the same way, we do it in here.

We define a strategy

or a control function u(t,x) for any time instant t

and for any state x(t).

So, for any function u(t,x)

or for any advertising expenses,

the right-hand side of the system of

differential equations is different.

Then as a result,

the trajectory of the system or the function x(t) is different.

For a different function x(t),

and for different control function,

we have the different,

values of the functional (1).

The optimal control problem

is to find the control function u(t,x),

that maximizes the value of the functional (1).

In our case, the function (1) could be

the profits or the revenue of the company.

In here, we also suppose that the functions f(t),

g(t) and q(t) are differentiable.

Let's construct an optimal control problem

for advertising costs model.

On the slide, the formula (3)

defines the functional that we need to maximize,

which is a revenue of

the company on the interval [0,T],

which depends on the state of

the game or the state function x(t) on this period,

and on the advertising expenses.

So, for if we fix the market share of the company,

and then we try to change the advertising costs,

then of course the advertising costs are higher

than the value of the functional is lower.

The formula (4) defines

the differential equation or

the motion equation for this dynamical system.

The right-hand sign of this differential equation depends on

the market share in the current time instant,

and also depends on the marketing expenses.

But the question is of how to find the optimal control

or how to find a function u(t,x),

that would maximize the functional (3).

In order to do that,

we can use a several classical approaches.

The first one is dynamic programming principle

or the Bellman equation.

The second one that we can use is called the maximum principle

or the Pontryagin's maximum principle,

but we will use the first one.

Because in the differential games,

this is the approach that is more widely used.

Why? Because the Bellman equation is

a sufficient condition for the optimal control.

So, if there is a solution

for a Bellman equation,

then we say that our solution is optimal.

With maximum principle, we can find

a solution for a much wider class of problems,

but it is only the necessary condition.

So, we would need to check

the solution once again and prove that it is sufficient.

So, in general, in differential games,

people use the dynamic programming principle.

But it has some disadvantages and we will talk about that later.

So, what is the dynamic programming principle?

Suppose that we know the optimal control in

the problem defined on the interval [t0,T].

We also can define the corresponding trajectory.

Let's denote the optimal control as a u*(t,x),

and the corresponding trajectory as x*(t).

Then, the truncation of the optimal control u*(t,x)

on the subproblem defined on the interval [t',T],

would be also optimal

in the problem starting at time instant t' and

in the position x*(t').

In the position on the optimal trajectory.

This is true for any truncated interval.

According to this statement,

we can define the procedure

to find the optimal solution of the control problem.