| Russian Mathematical Surveys |
|
|
|
|
|
This article is cited in 2 scientific papers (total in 2 papers) Minimax solutions of Hamilton–Jacobi equations in dynamic optimization problems for hereditary systems M. I. Gomoyunovab, N. Yu. Lukoyanovab a Krasovskii Institute of Mathematics and Mechanics, Ural Branch of the Russian Academy of Sciences b Ural Federal University
Bibliographic databases:
     1. IntroductionHamilton–Jacobi equations are now usually understood as first-order partial differential equations. In classical mechanics a Hamilton–Jacobi equation occurs, for example, as an equation for the generating function in the theory of canonical transformations of Hamiltonian systems (see, for example, [43], Chap. IV, and [4], § 47) and also as an equation holding for the action as a function of time and the position at the end of motion of a mechanical system (see, for example, [104], § 47). In a similar way a Hamilton–Jacobi equation occurs in variational calculus as an equation satisfied by the minimum in the simplest variational problem as a function of the upper limit of integration and the boundary value at the right-hand endpoint (see, for example, [45], § 19, and [177], § 26). The development of the modern theory of Hamilton–Jacobi equations is largely related to extremal problems in dynamics such as optimal control problems and differential games. We recall briefly how a Hamilton–Jacobi equation occurs in one canonical optimal control problem (for greater detail, see, for example, [177], § 8, and [107], § 5.1). We consider a dynamical system whose motion is described by the differential equation
$$
\begin{equation}
\dot{y}(\tau)= f(\tau, y(\tau), u(\tau)),\qquad \tau \in [0, T],\quad y(\tau) \in \mathbb{R}^n,\quad u(\tau) \in P \subset \mathbb{R}^{n_P},
\end{equation}
\tag{1.1}
$$
where $\tau$ is the current time, $T > 0$ is a fixed terminal instant, $y(\tau)$ is the state of the system at $\tau$, $\dot{y}(\tau)={\rm d}y(\tau)/{\rm d}\tau$ is the rate of change in this state, $u(\tau)$ is the current control action and $P$ is a compact set describing the geometric constraints on its value. We assume that an initial position $(t, x) \in [0, T] \times \mathbb{R}^n$ of the system (1.1) is prescribed, where $t$ is the initial instant and $x$ is the initial state at this instant. Thus, we obtain the initial condition $y(t) = x$ for differential equation (1.1). An admissible (program) control on the interval $[t, T]$ is an arbitrary (Lebesgue-)measurable function $u \colon [t, T] \to P$. We let $\mathcal{U}(t)$ denote the set of all admissible controls. By choosing a control $u(\,{\cdot}\,) \in \mathcal{U}(t)$ we need to minimize the quality index
$$
\begin{equation}
J(t, x, u(\,{\cdot}\,))= \sigma(y(T)) - \int_{t}^{T} \chi(\tau, y(\tau), u(\tau)) \, {\rm d} \tau,
\end{equation}
\tag{1.2}
$$
where the function $y \colon [t, T] \to \mathbb{R}^n$ is the motion of system (1.1) generated by $u(\,{\cdot}\,) \in \mathcal{U}(t)$ from an initial position $(t, x)$. More precisely, $y(\,{\cdot}\,)$ is a Lipschitz- continuous function (that is, $y(\,{\cdot}\,) \in \operatorname{Lip}([t, T]; \mathbb{R}^n)$) satisfying the equality $y(t) = x$ and, for almost every $\tau \in [t, T]$, satisfying the differential equation (1.1) for the control $u(\,{\cdot}\,)$ we have chosen. The functions
$$
\begin{equation*}
f \colon [0, T] \times \mathbb{R}^n \times P \to \mathbb{R}^n,\quad \sigma \colon \mathbb{R}^n \to \mathbb{R},\quad \chi \colon [0, T] \times \mathbb{R}^n \times P \to \mathbb{R}
\end{equation*}
\notag
$$
are prescribed and have some properties similar to those described below (see conditions $(2.\mathrm{DG}.1)$–$(2.\mathrm{DG}.4)$ in § 2). We consider the optimal result in problem (1.1), (1.2):
$$
\begin{equation}
\rho(t, x)= \inf_{u(\,{\cdot}\,) \in \mathcal{U}(t)} J(t, x, u(\,{\cdot}\,)).
\end{equation}
\tag{1.3}
$$
When the initial position $(t, x)$ is unfixed, we have the optimal result function $\rho \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$, which is also called the value function. By definition, this function has the following property, which is called the dynamic programming principle: for any initial position $(t, x) \in [0, T) \times \mathbb{R}^n$ and any instant $\vartheta \in (t, T]$,
$$
\begin{equation}
\rho(t, x)= \inf_{u(\,{\cdot}\,) \in \mathcal{U}(t)} \biggl( \rho(\vartheta, y(\vartheta)) - \int_{t}^{\vartheta} \chi(\tau, y(\tau), u(\tau)) \, {\rm d} \tau \biggr).
\end{equation}
\tag{1.4}
$$
When the value function $\rho$ turns out to be continuously differentiable, by the formula for differentiation of a composite function, for almost every $\tau \in [t, T]$ we have
$$
\begin{equation}
\begin{aligned} \, \nonumber \frac{{\rm d}}{{\rm d} \tau} \rho(\tau, y(\tau))& = \frac{\partial \rho}{\partial t} (\tau, y(\tau))+ \biggl\langle \frac{\partial \rho}{\partial x} (\tau, y(\tau)), \dot{y}(\tau) \biggr\rangle \\ & = \frac{\partial \rho}{\partial t} (\tau, y(\tau))+ \biggl\langle \frac{\partial \rho}{\partial x}(\tau, y(\tau)), f(\tau, y(\tau), u(\tau)) \biggr\rangle, \end{aligned}
\end{equation}
\tag{1.5}
$$
where $\langle\,\cdot\,{,}\,\cdot\,\rangle$ is the scalar product in $\mathbb{R}^n$. Based on (1.4) and (1.5), we conclude that the function $\rho$ satisfies the following Hamilton–Jacobi equation (which is also called Bellman’s equation or the Hamilton–Jacobi–Bellman equation):
$$
\begin{equation}
\frac{\partial \rho}{\partial t} (t, x) + H\biggl( t, x, \frac{\partial \rho}{\partial x}(t, x) \biggr)= 0,\qquad (t, x) \in (0, T) \times \mathbb{R}^n,
\end{equation}
\tag{1.6}
$$
where the function $H \colon [0, T] \times \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$, called the Hamiltonian, is defined by
$$
\begin{equation}
H(t, x, s)= \min_{u \in P} \bigl( \langle s, f(t, x, u) \rangle + \chi(t, x, u) \bigr),\qquad t \in [0, T], \quad x, s \in \mathbb{R}^n.
\end{equation}
\tag{1.7}
$$
In fact, for each point $(t, x) \in (0, T) \times \mathbb{R}^n$ we infer that
$$
\begin{equation*}
\begin{aligned} \, 0& = \lim_{\delta \to 0^+}\,\inf_{u(\,{\cdot}\,) \in \mathcal{U}(t)} \frac{1}{\delta} \int_{t}^{t + \delta} \biggl( \frac{\partial \rho}{\partial t} (\tau, y(\tau)) \\ & \qquad + \biggl\langle \frac{\partial \rho}{\partial x}(\tau, y(\tau)), f(\tau, y(\tau), u(\tau)) \biggr\rangle - \chi(\tau, y(\tau), u(\tau)) \biggr) \, {\rm d} \tau \\ & = \frac{\partial \rho}{\partial t} (t, x) + H\biggl( t, x, \frac{\partial \rho}{\partial x}(t, x) \biggr). \end{aligned}
\end{equation*}
\notag
$$
By (1.3) the function $\rho$ also satisfies the right-end boundary condition Hence the value function $\rho$ is a solution of the Cauchy problem for the Hamilton–Jacobi equation (1.6) with boundary condition (1.8). A result which is converse in a certain sense and is called sometimes the verification theorem, is important for applications. To be precise, if a continuously differentiable function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ is a solution of the Cauchy problem (1.6), (1.8) with Hamiltonian (1.7), then this function coincides with the value function $\rho$ in the optimal control problem (1.1), (1.2). In this case an optimal feedback control strategy, making it possible to generate $\varepsilon$-optimal controls for any a priori fixed $\varepsilon > 0$, can be constructed using extremal aiming in the direction of the gradient $\partial \varphi / \partial x$ of the function $\varphi$:
$$
\begin{equation}
\begin{gathered} \, U^\circ(\tau, y)\in \operatorname*{arg\,min}_{u \in P} \biggl( \biggl\langle \frac{\partial \varphi}{\partial x} (\tau, y), f(\tau, y, u) \biggr\rangle - \chi(\tau, y, u) \biggr), \\ (\tau, y) \in [0, T] \times \mathbb{R}^n. \end{gathered}
\end{equation}
\tag{1.9}
$$
For more details about optimal feedback control strategies, see § 2.5. We emphasize that the above approach to the optimal control problem (1.1), (1.2), which is based on the dynamic programming principle and leads to Hamilton–Jacobi equations, is only one of the possible approaches. It is aimed directly at constructing optimal feedback control strategies. Another well-known approach is related to Pontryagin’s maximum principle [147]. In this case the emphasis is put on seeking an optimal program control for a fixed initial position $(t, x)$. For the relationship between the Hamilton–Jacobi equation and Pontryagin’s maximum principle in the optimal control problem (1.1), (1.2), see, for example, [40], Theorem 8.1, and also [168]. Dynamic programming constructions become of fundamental importance in the theory of differential games, in which control problems are formalized in the presence of obstacles or counteraction. In problems of this type, in choosing the current control action it is significant to use additional information on the progress of the process. Just like above for the optimal control problem (1.1), (1.2), we illustrate the role of the Hamilton–Jacobi equations by the following differential zero-sum game of two players. Assume that the motion of a dynamical system is described by the differential equation
$$
\begin{equation}
\begin{gathered} \, \dot{y}(\tau)= f(\tau, y(\tau), u(\tau), v(\tau)), \\ \tau \in [0, T],\quad y(\tau) \in \mathbb{R}^n,\quad u(\tau) \in P \subset \mathbb{R}^{n_P},\quad v(\tau) \in Q \subset \mathbb{R}^{n_Q}, \end{gathered}
\end{equation}
\tag{1.10}
$$
where $u(\tau)$ and $v(\tau)$ are the current control actions of the first and second players, respectively, and an initial position $(t, x) \in [0, T] \times \mathbb{R}^n$ is fixed. We let $\mathcal{U}(t)$ and $\mathcal{V}(t)$ denote the sets of admissible (program) controls of the players. The first (second) player is aimed at minimizing (maximizing) the quality index
$$
\begin{equation}
J(t, x, u(\,{\cdot}\,), v(\,{\cdot}\,))= \sigma(y(T)) - \int_{t}^{T} \chi(\tau, y(\tau), u(\tau), v(\tau)) \, {\rm d} \tau.
\end{equation}
\tag{1.11}
$$
A non-anticipating strategy (a quasi-strategy in another terminology) of the first player is a mapping $\boldsymbol{\alpha} \colon \mathcal{V}(t) \to \mathcal{U}(t)$ with the following property (which is often called the property of non-anticipation or physical feasibility): for any functions $v_1(\,{\cdot}\,),v_2(\,{\cdot}\,) \in \mathcal{V}(t)$ and instant $\vartheta \in [t, T]$, if $v_1(\tau) = v_2(\tau)$ for almost all $\tau \in [t, \vartheta]$, then $\boldsymbol{\alpha}(v_1(\,{\cdot}\,))(\tau) = \boldsymbol{\alpha}(v_2(\,{\cdot}\,))(\tau)$ for almost all $\tau \in [t, \vartheta]$. Let $\boldsymbol{\mathcal{A}}(t)$ be the set of all strategies of this type. We consider the lower value of the game (in another terminology, the optimal guaranteed result of the first player in the class of quasistrategies $\boldsymbol{\mathcal{A}}(t)$)
$$
\begin{equation}
\rho_-(t, x)= \inf_{\boldsymbol{\alpha} \in \boldsymbol{\mathcal{A}}(t)}\, \sup_{v(\,{\cdot}\,) \in \mathcal{V}(t)} J\bigl(t,x,\boldsymbol{\alpha}(v(\,{\cdot}\,))(\,{\cdot}\,),v(\,{\cdot}\,)\bigr).
\end{equation}
\tag{1.12}
$$
The upper value of the game (the optimal guaranteed result of the second player in the class $\boldsymbol{\mathcal{B}}(t)$ of quasi-strategies $\boldsymbol{\beta} \colon \mathcal{U}(t) \to \mathcal{V}(t)$) is defined symmetrically by
$$
\begin{equation}
\rho_+(t, x)= \sup_{\boldsymbol{\beta} \in \boldsymbol{\mathcal{B}}(t)}\, \inf_{u(\,{\cdot}\,) \in \mathcal{U}(t)} J \bigl(t, x, u(\,{\cdot}\,), \boldsymbol{\beta}(u(\,{\cdot}\,))(\,{\cdot}\,) \bigr).
\end{equation}
\tag{1.13}
$$
In the case when the quantities (1.12) and (1.13) are equal, the differential game under consideration is said to have value For the differential game (1.10), (1.11), the dynamic programming principle is expressed by the following equalities, valid for all $(t, x) \in [0, T) \times \mathbb{R}^n$ and $\vartheta \in (t, T]$ (see, for example, [36], Theorem 3.1, and also [41], Chap. XI, Theorem 5.1, and [176], Theorem 3.3.5):
$$
\begin{equation}
\begin{aligned} \, \rho_-(t, x)& = \inf_{\boldsymbol{\alpha} \in \boldsymbol{\mathcal{A}}(t)}\, \sup_{v(\,{\cdot}\,) \in \mathcal{V}(t)}\biggl(\rho_-(\vartheta, y(\vartheta)) - \int_{t}^{\vartheta} \chi \bigl( \tau, y(\tau), \boldsymbol{\alpha} (v(\,{\cdot}\,))(\tau), v(\tau) \bigr) \, {\rm d} \tau \biggr), \\ \rho_+(t, x)& = \sup_{\boldsymbol{\beta} \in \boldsymbol{\mathcal{B}}(t)}\, \inf_{u(\,{\cdot}\,) \in \mathcal{U}(t)}\biggl(\rho_+(\vartheta, y(\vartheta)) - \int_{t}^{\vartheta} \chi \bigl( \tau, y(\tau), u(\tau), \boldsymbol{\beta}(u(\,{\cdot}\,))(\tau) \bigr) \, {\rm d} \tau \biggr). \end{aligned}
\end{equation}
\tag{1.14}
$$
Assume that the function $\rho_- \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ is continuously differentiable. Then, on the basis of (1.5), from the first equality in (1.14) we derive (see, for example, [41], Chap. XI, Lemma 6.2)
$$
\begin{equation*}
\begin{aligned} \, 0& = \lim_{\delta \to 0^+}\,\inf_{\boldsymbol{\alpha} \in \boldsymbol{\mathcal{A}}(t)}\,\sup_{v(\,{\cdot}\,) \in \mathcal{V}(t)} \frac{1}{\delta} \int_{t}^{t + \delta} \biggl( \frac{\partial \rho}{\partial t} (\tau, y(\tau)) \\ & \qquad + \biggl\langle \frac{\partial \rho}{\partial x} (\tau, y(\tau)), f \bigl( \tau, y(\tau), \boldsymbol{\alpha}(v(\,{\cdot}\,)) (\tau), v(\tau) \bigr) \biggr\rangle \\ &\qquad- \chi\bigl( \tau, y(\tau), \boldsymbol{\alpha}(v(\,{\cdot}\,))(\tau),v(\tau)\bigr)\biggr)\, {\rm d}\tau \\ & = \frac{\partial \rho}{\partial t} (t, x) + H_- \biggl( t, x, \frac{\partial \rho}{\partial x}(t, x) \biggr),\qquad (t, x) \in (0, T) \times \mathbb{R}^n, \end{aligned}
\end{equation*}
\notag
$$
where
$$
\begin{equation}
\begin{gathered} \, H_-(t, x, s)= \max_{v \in Q}\,\min_{u \in P} \bigl( \langle s, f(t, x, u, v) \rangle - \chi(t, x, u, v) \bigr), \\ t \in [0, T], \quad x, s \in \mathbb{R}^n. \end{gathered}
\end{equation}
\tag{1.15}
$$
Thus, the function $\rho_-$ satisfies the Hamilton–Jacobi equation (1.6) with Hamiltonian (1.15). In a similar way, if $\rho_+ \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ is continuously differentiable, then by the second equality in (1.14) it satisfies (1.6) with Hamiltonian
$$
\begin{equation}
\begin{gathered} \, H_+(t, x, s)= \min_{u \in P}\,\max_{v \in Q} \bigl(\langle s, f(t, x, u, v) \rangle - \chi(t, x, u, v) \bigr), \\ t \in [0, T], \quad x, s \in \mathbb{R}^n. \end{gathered}
\end{equation}
\tag{1.16}
$$
Equation (1.6) with the Hamiltonian (1.15) (or (1.16)) is also called Isaacs’s equation or the Hamilton–Jacobi–Bellman–Isaacs equation. According to (1.12) and (1.13), the functions $\rho_-$ and $\rho_+$ satisfy the boundary condition (1.8). If Isaacs’s condition ([72], § 2.4; it is also called the saddle-point condition in a small game [96], p. 79)
$$
\begin{equation}
H_-(t, x, s)= H_+(t, x, s),\qquad t \in [0, T],\quad x, s \in \mathbb{R}^n,
\end{equation}
\tag{1.17}
$$
holds, then the value function $\rho$ must satisfy (1.6) with the Hamiltonian
$$
\begin{equation}
H(t, x, s)= H_- (t, x, s)= H_+ (t, x, s),\qquad t \in [0, T],\quad x, s \in \mathbb{R}^n.
\end{equation}
\tag{1.18}
$$
The converse is also true (in this connection see, for example, [96], § 15, and [162], § 11.5): if there is a continuously differentiable function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ satisfying this equation and the boundary condition (1.8), then the differential game has value $\rho(t, x) = \varphi(t, x)$, $(t,x) \in [0,T] \times \mathbb{R}^n$. In this case, by extremal aiming in the direction of the gradient of $\varphi$ we can construct optimal feedback control strategies of the form
$$
\begin{equation}
\begin{gathered} \, \begin{aligned} \, U^\circ(\tau, y)& \in \operatorname*{arg\,min}_{u \in P}\, \max_{v \in Q} \biggl( \biggl\langle \frac{\partial \varphi}{\partial x} (\tau, y), f(\tau, y, u, v) \biggr\rangle - \chi(\tau, y, u, v) \biggr), \\ V^\circ(\tau, y)& \in \operatorname*{arg\,max}_{v \in Q}\, \min_{u \in P}\biggl( \biggl\langle \frac{\partial \varphi} {\partial x} (\tau, y), f(\tau, y, u, v) \biggr\rangle - \chi(\tau, y, u, v) \biggr), \end{aligned} \\ (\tau, y) \in [0, T] \times \mathbb{R}^n, \end{gathered}
\end{equation}
\tag{1.19}
$$
which allow each player, regardless of the opponent’s control, to guarantee that the corresponding value of the quality index (1.11) is not worse than the game value $\rho$, with any prescribed accuracy $\varepsilon > 0$ (for more details, see § 2.5). For details concerning the derivation of Hamilton–Jacobi equations in variational calculus, optimal control, and differential games, see, for example, [153], § § 3.1–3.3. We emphasize that a key assumption in the above arguments that makes it possible to use Hamilton–Jacobi equations to study problems in optimal control and differential games and to construct optimal control strategies of the form (1.9) and (1.19) is the existence of a continuously differentiable solution of the Cauchy problem (1.6), (1.8) or, which is equivalent, the continuous differentiability of the value function. However, as a rule, the Cauchy problem for nonlinear Hamilton–Jacobi equations has no classical (continuously differentiable) global solution. This can be the case even when the Hamiltonian $H$ and boundary function $\sigma$ are infinitely differentiable (see an example in [162], § 1.4). At the same time there are meaningful problems in optimal control theory and differential games in which the value function is not smooth. In particular, this leads to the fact that in problems of this type formulae (1.9) and (1.19) cannot be applied directly to the construction of optimal control strategies, since the gradient $\partial \varphi (\tau, y) / \partial x$ may not exist at some points $(\tau, y)$. These facts have in many aspects prompted the development of the theory of generalized (non-smooth) solutions of Hamilton–Jacobi equations that, in particular, must satisfy the following natural conditions:
There are several approaches to the concept of generalized solution of various boundary value problems for Hamilton–Jacobi equations (see, for example, [101], [81], [159], [109], [27], [89], [160], and [168]). In our opinion, two approaches stand out: the minimax approach and the viscosity approach. In fact, they have received the greatest development. The minimax approach is based on the theory of positional differential games (see, for instance, [99], [163], [96], [100], and [90]) and can be regarded as a development of the unification constructions in this theory [94], [95] (also see [100], § 10.5, and [171], [167], and [174]). Within this approach [159], [160], [162] a generalized (minimax) solution is defined as a function satisfying a pair of non-local conditions that express the properties of the weak invariance of its super- and subgraph under so-called characteristic differential inclusions. In the infinitesimal form these conditions are expressed by inequalities for lower and upper directional derivatives of the minimax solution. The resulting pair of differential inequalities can be regarded as a generalization of the Hamilton–Jacobi equation to the non-smooth case. The viscosity approach [27], [25] (also see, for instance, [109], [8], [41], and [173]) is conceptually related to the vanishing viscosity method and comparison theorems in mathematical physics. According to this approach, a generalized (viscosity) solution is defined as a function satisfying a pair of conditions involving smooth underlying (test) functions. Expressing these conditions in an infinitesimal form leads to a generalization of the Hamilton–Jacobi equation in the form of inequalities for sub- and supergradients of the viscosity solution. Though the concepts of minimax and viscosity solutions are based on a different background, it is important that they turn out to be equivalent under rather general assumptions (see, for example, [110], [166], [160], [42], [20], and [162]). Furthermore, in the minimax approach special attention is paid to applications of the theory of generalized solutions of Hamilton–Jacobi equations to developing non-smooth methods for the construction of optimal feedback control strategies. We also note a slightly isolated ingenious approach [89], [134] to the definition of generalized solutions of Hamilton–Jacobi equations which is based on constructions in idempotent analysis [133]. The relationship between this approach and the minimax one was considered, for example, in [151]. In this survey we discuss the development of the theory of Hamilton–Jacobi equations for so-called hereditary dynamical systems. One feature of systems of this type is that the rate of change of their state depends not only on the current position, like in the ordinary (classical) case, but also on the full path travelled, that is, the history of the motion. The Hamilton–Jacobi equations under consideration play for hereditary systems a role similar to the role of (1.6) in motion control problems for ordinary differential systems of the form (1.10). Most attention is paid to the minimax approach to the concept of generalized solution of the Cauchy problems for equations of this type and its relationship with the viscosity approach. We present methods for designing optimal feedback control strategies with memory of the motion history that are based on the constructions discussed here. A major part of the review is devoted to the results deduced for hereditary dynamical systems whose motion is described by retarded functional differential equations of the following form (see, for example, [88] and [71]): Here the map $f \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}^n$, where $h > 0$ and $\operatorname{C}([- h, T]; \mathbb{R}^n)$ is the space of continuous functions from $[-h, T]$ to $\mathbb{R}^n$, has the non-anticipation property: for any functions $y_1(\,{\cdot}\,),y_2(\,{\cdot}\,)\in \operatorname{C}([- h, T]; \mathbb{R}^n)$ and instant $\tau \in [0, T]$ the equality $y_1(\xi) = y_2(\xi)$ for all $\xi \in [-h, \tau]$ implies that
$$
\begin{equation*}
f(\tau, y_1(\,{\cdot}\,))= f(\tau, y_2(\,{\cdot}\,)).
\end{equation*}
\notag
$$
In other words, this property means that, instead of the full function $y(\,{\cdot}\,)$, the right-hand side of (1.20) depends only on $y(\xi)$ for $\xi \in [- h, \tau]$, that is, on the history of the motion $y(\,{\cdot}\,)$ to the current instant $\tau$. In what follows systems of the form (1.20) are called time-delay systems. Typical examples of systems of this type, which often arise in applications, are:
We also present some results concerning the Hamilton–Jacobi equations for more general systems than (1.20), which are described by functional differential equations of neutral type in Hale’s form (see, for example, [70], [2], [88], and [71]):
$$
\begin{equation}
\frac{{\rm d}}{{\rm d} \tau} \bigl( y(\tau) - g(\tau, y(\,{\cdot}\,)) \bigr)= f(\tau, y(\,{\cdot}\,)),
\end{equation}
\tag{1.24}
$$
where the mapping $g \colon [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n) \to \mathbb{R}^n$ has a non-anticipation property similar to the above one for $f$. In addition, we will consider so-called fractional-order systems (see, for example, [152], [136], [146], [82], and [32]) described by differential equations of the form
$$
\begin{equation}
(^{\mathrm C\!} D^\alpha y)(\tau)= f(\tau, y(\tau)),
\end{equation}
\tag{1.25}
$$
which are closely related to neutral-type systems (1.24). Here $(^{\mathrm C\!} D^\alpha y)(\tau)$ is the Caputo fractional derivative of order $\alpha \in (0, 1)$ defined by (see, for example, [82], § 2.4, and [32], Chap. 3)
$$
\begin{equation}
(^{\mathrm C\!} D^\alpha y)(\tau)= \frac{1}{\Gamma(1 - \alpha)}\, \frac{{\rm d}}{{\rm d} \tau} \int_{0}^{\tau} \frac{y(\xi) - y(0)}{(\tau - \xi)^\alpha} \, {\rm d} \xi,
\end{equation}
\tag{1.26}
$$
where $\Gamma$ is the gamma function. Topics related to Hamilton–Jacobi equations and boundary value problems for equations of this type that arise, for example, in time-optimal problems, control problems with infinite horizon, problems with state constraints, or control problems for stochastic dynamical systems are beyond the scope of this review. Concluding the introductory part, we note that this paper contains quite a lot of various conditions and properties. To make it easier to navigate through them we choose a special system of notation. More precisely, as a rule, the notation for a particular condition consists of three parts: the first part is the number of the section in which the condition appears for the first time; the second part is a letter abbreviation intended to stress the object this condition applies to; the third part is an order number within the relevant group of conditions. For example, condition $(2.\mathrm{CP}.3)$ is stated in § 2, refers to the Cauchy problem studied in that section, and has order number 3. 2. Minimax and viscosity solutions of Hamilton–Jacobi equationsIn this section we consider the Cauchy problem for the Hamilton–Jacobi equation
$$
\begin{equation}
\frac{\partial \varphi}{\partial t} (t, x) + H\biggl(t,x,\frac{\partial \varphi}{\partial x} (t, x) \biggr)= 0,\qquad (t, x) \in (0, T) \times \mathbb{R}^n,
\end{equation}
\tag{2.1}
$$
with the right-end boundary condition The function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ is unknown. The function $H \colon [0, T] \times \mathbb{R}^n \times \mathbb{R}^n \to\mathbb{R}$, which is called the Hamiltonian, and the boundary function $\sigma \colon \mathbb{R}^n \to \mathbb{R}$ are known and satisfy the following conditions.
Troughout what follows $\|\cdot\|$ is the Euclidean norm in $\mathbb{R}^n$. Conditions $(2.\mathrm{CP}.1)$–$(2.\mathrm{CP}.3)$ are rather general. In the introduction we indicated the relationship between a Cauchy problem of the form (2.1), (2.2) (see (1.6), (1.8), and (1.18)) and the differential game (1.10), (1.11). These results are certainly true when the functions $f \colon [0, T] \times \mathbb{R}^n \times P \times Q \to \mathbb{R}^n$, $\sigma \colon \mathbb{R}^n \to \mathbb{R}$, and $\chi \colon [0, T] \times \mathbb{R}^n \times P \times Q \to \mathbb{R}$ satisfy the following conditions, which are typical in optimal control theory and differential games.
Under conditions $(2.\mathrm{DG}.1)$–$(2.\mathrm{DG}.3)$ the Hamiltonians $H_-$ and $H_+$ in (1.15) and (1.16) satisfy the conditions $(2.\mathrm{CP}.1)$–$(2.\mathrm{CP}.3)$. Thus, conditions $(2.\mathrm{CP}.1)$–$(2.\mathrm{CP}.3)$ are natural, at least in the context of applications to control problems. To date, the theory of minimax and viscosity solutions of the Cauchy problem (2.1), (2.2) has been developed quite thoroughly. Below we present only a few basic concepts and results in this theory. Their development for the case of hereditary dynamical systems is the subject of our survey. 2.1. Definition of a minimax solutionThe concept of a minimax solution of the Cauchy problem (2.1), (2.2) is related to constructions in the theory of positional differential games. It was proved in this theory (in this connection, see [96], § 29, [90], § 9, and also [99], § 18, [163], Chap. III, § 2, and [100], § 3.1) that under conditions $(2.\mathrm{DG}.1)$–$(2.\mathrm{DG}.4)$ the differential game (1.10), (1.11) has a value such that the value function is the only continuous function $\rho \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ satisfying the boundary condition (1.8) and the following two conditions.
Conditions $(2.\mathrm{US}.1)$ and $(2.\mathrm{VS}.1)$ are called the conditions of $u$- and $v$-stability of the function $\rho$, respectively. Note that these conditions are fulfilled by the dynamic programming principle (see (1.14)). In addition, unification of differential games has been carried out (see [94], [95], and also [100], § 10.5): it has been shown that we can switch from the game (1.10), (1.11) to an equivalent differential game, in the sense of the coincidence of the value functions, which are specified by the Hamiltonian $H$ (see (1.18)) and the function $\sigma$. As a consequence, if $H$ and $\sigma$ coincide in two games of the form (1.10), (1.11), then the value functions also coincide; in this case, the specific form of the functions $f$ and $\chi$ and the sets $P$ and $Q$ does not play any role. In particular, following unification constructions, we can express properties $(2.\mathrm{US}.1)$ and $(2.\mathrm{VS}.1)$ of $\rho$ as follows.
Hence the value function can be characterized as the unique continuous function $\rho \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ satisfying the boundary condition (1.8), which is specified by the function $\sigma$, and preserving the stability properties $(2.\mathrm{US}.2)$ and $(2.\mathrm{VS}.2)$, which are specified by the Hamiltonian $H$. The following definition of a minimax solution of the Cauchy problem (2.1), (2.2) is based on the above reasoning (in this connection, see [162], § 6.3). We consider a multivalued mapping
$$
\begin{equation*}
[0, T] \times \mathbb{R}^n \times \mathbb{R}^n \ni (\tau, y, s) \mapsto E_0(\tau, y, s) \subset \mathbb{R}^n \times \mathbb{R}
\end{equation*}
\notag
$$
defined for all $\tau \in [0, T]$ and $y,s \in \mathbb{R}^n$ by
$$
\begin{equation}
E_0(\tau, y, s)= \bigl\{ (f, \chi) \in \mathbb{R}^n \times \mathbb{R} \colon \|f\| \leqslant c (1 + \|y\|),\, \chi = \langle s, f \rangle - H(\tau, y, s) \bigr\},
\end{equation}
\tag{2.4}
$$
where, like in (2.3), the number $c$ is the same as in $(2.\mathrm{CP}.2)$. Given $(t, x) \in [0, T] \times \mathbb{R}^n$ and $s \in \mathbb{R}^n$, we let $\operatorname{Sol}(t, x, E_0, s)$ denote the set of solutions of the differential inclusion
$$
\begin{equation}
(\dot{y}(\tau), \dot{z}(\tau))\in E_0(\tau, y(\tau), s),\qquad \tau \in [t, T], \quad (y(\tau), z(\tau)) \in \mathbb{R}^n \times \mathbb{R},
\end{equation}
\tag{2.5}
$$
satisfying the initial conditions that is, the set of pairs $(y(\,{\cdot}\,), z(\,{\cdot}\,)) \in \operatorname{Lip}([t, T]; \mathbb{R}^n) \times \operatorname{Lip}([t, T]; \mathbb{R})$ satisfying equalities (2.6) and, for almost all $\tau \in [t, T]$, inclusion (2.5). Sometimes, the differential inclusion (2.5) is said to be characteristic, while elements of the set $\operatorname{Sol}(t, x, E_0, s)$ are called generalized characteristics of the Hamilton–Jacobi equation (2.1). Definition 1. An upper minimax solution of the Cauchy problem (2.1), (2.2) is a lower semicontinuous function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ that satisfies the boundary condition and has the following property.
Definition 2. A lower minimax solution of the Cauchy problem (2.1), (2.2) is an upper semicontinuous function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ that satisfies the boundary condition and has the following property.
Definition 3. A minimax solution of the Cauchy problem (2.1), (2.2) is a function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ that is an upper and a lower minimax solution of this problem at the same time. Note that conditions $(2.\mathrm{MS}.1_+)$ and $(2.\mathrm{MS}.1_-)$, respectively, reflect the property of weak invariance of the super- and subgraph of the function $\varphi$ with respect to the characteristic differential inclusion (2.5). We can show that it is possible to combine conditions $(2.\mathrm{MS}.1_+)$ and $(2.\mathrm{MS}.1_-)$ in defining a minimax solution of the Cauchy problem (2.1), (2.2). More precisely ([162], Theorem 6.4), a continuous function $\varphi \colon [0,T] \times \mathbb{R}^n \to \mathbb{R}$ is a minimax solution if and only if it satisfies the boundary condition (2.2) and has the following property.
In terms of generalized characteristics, (2.11) assumes the form
$$
\begin{equation*}
\varphi(\vartheta, y(\vartheta)) - z(\vartheta)= \varphi(t, x),\qquad \vartheta \in [t, T];
\end{equation*}
\notag
$$
in this sense, property $(2.\mathrm{MS})$ expresses the weak invariance of the graph of the minimax solution with respect to the characteristic differential inclusion (2.5). 2.2. Well-posednessA key point in the theory of minimax solutions is to prove the existence and uniqueness of a minimax solution of the Cauchy problem (2.1), (2.2). It is conceptually related to proving the existence of a game value (the alternative theorem) in the theory of positional differential games. The result is that the Cauchy problem (2.1), (2.2) has a unique minimax solution under conditions $(2.\mathrm{CP}.1)$–$(2.\mathrm{CP}.3)$ ([162], Theorem 8.1). The scheme of the proof can be divided into two parts. In the first ([162], Theorem 8.2) it is shown that the lower and upper closures of the lower envelope of the family of all functions $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ satisfying conditions $(2.\mathrm{MS}.1_+)$ and (2.7), respectively, specify an upper and a lower minimax solution, $\varphi_+^\circ$ and $\varphi_-^\circ$, such that
$$
\begin{equation*}
\varphi_+^\circ(t, x)\leqslant \varphi_-^\circ(t, x),\qquad (t, x) \in [0, T] \times \mathbb{R}^n.
\end{equation*}
\notag
$$
In the second part ([162], Theorem 7.3) the so-called comparison principle is proved, which states that
$$
\begin{equation*}
\varphi_+(t,x)\geqslant \varphi_-(t, x),\qquad (t, x) \in [0, T] \times \mathbb{R}^n,
\end{equation*}
\notag
$$
for any upper and lower minimax solutions $\varphi_+$ and $\varphi_-$. The philosophy of Lyapunov functions is used here. An appropriate function is, for example, as follows:
$$
\begin{equation}
\nu_\varepsilon(\tau, w)= \frac{e^{- \lambda \tau} - \varepsilon}{\varepsilon} \sqrt{\varepsilon^4 + \|w\|^2}\,,\qquad (\tau, w) \in [0, T] \times \mathbb{R}^n,
\end{equation}
\tag{2.12}
$$
where $\lambda > 0$ is a number from $(2.\mathrm{CP}.3)$ chosen for an appropriate bounded set $W \subset \mathbb{R}^n$, and $\varepsilon > 0$ is a small parameter. In addition, it turns out (in this connection, see [160], Theorem 4.3) that the minimax solution of the Cauchy problem (2.1), (2.2) depends continuously on the Hamiltonian $H$ and the boundary function $\sigma$ in the following sense. Assume that for each $k \in \mathbb{N} \cup \{0\}$ we have a Hamiltonian $H_k$ and a boundary function $\sigma_k$ satisfying conditions $(2.\mathrm{CP}.1)$–$(2.\mathrm{CP}.3)$, where the constant $c$ in $(2.\mathrm{CP}.2)$ is independent of $k$. Also assume that for any bounded set $W \subset \mathbb{R}^n$ and any $s \in \mathbb{R}^n$ we have the convergence
$$
\begin{equation*}
|H_k(t, x, s) - H_0(t, x, s)| + |\sigma_k(x) - \sigma_0(x)|\to 0 \quad \text{as}\ \ k \to \infty
\end{equation*}
\notag
$$
which is uniform with respect to $t \in [0, T]$ and $x \in W$. For $k \in \mathbb{N} \cup \{0\}$ we consider the minimax solution $\varphi_k \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ of the Cauchy problem (2.1), (2.2) with $H = H_k$ and $\sigma = \sigma_k$. Then $\varphi_k(t, x) \to \varphi_0(t, x)$ as $k \to \infty$ uniformly with respect to $t \in [0, T]$ and $x \in W$ for any bounded set $W \subset \mathbb{R}^n$. 2.3. Characteristic complexesTo understand whether a function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ is a minimax solution of the Cauchy problem (2.1), (2.2), it is often more convenient, in place of a specific characteristic differential inclusion of the form (2.5) to verify conditions $(2.\mathrm{MS}.1_+)$ and $(2.\mathrm{MS}.1_-)$ for certain differential inclusions chosen in a special way. We present the corresponding constructions (see [162], § 6.2, and also [160], § 2.4). Throughout what follows we let $\mathcal{K}(\mathbb{R}^m)$ denote the set of non-empty convex compact subsets of $\mathbb{R}^m$. For a non-empty set $\Psi$ and a multivalued mapping
$$
\begin{equation*}
[0, T] \times \mathbb{R}^n \times \Psi \ni (\tau, y, \psi) \mapsto E(\tau, y, \psi) \subset \mathbb{R}^n \times \mathbb{R},
\end{equation*}
\notag
$$
we consider the following conditions.
When conditions $(2.\mathrm{C}.1)$–$(2.\mathrm{C}.3)$ and $(2.\mathrm{C}.4_+)$ (conditions $(2.\mathrm{C}.1)$–$(2.\mathrm{C}.3)$ and $(2.\mathrm{C}.4_-)$, respectively) hold, the pair $(\Psi, E)$ is called an upper (a lower) characteristic complex of the Hamilton–Jacobi equation (2.1). Note that conditions $(2.\mathrm{C}.1)$–$(2.\mathrm{C}.3)$ are technical, whereas $(2.\mathrm{C}.4_+)$ and $(2.\mathrm{C}.4_-)$ relate the pair $(\Psi, E)$ to the Hamiltonian $H$. Note also that the pair $(\mathbb{R}^n, E_0)$, where the multivalued mapping $E_0$ is defined by (2.4), is an upper and a lower characteristic complex of (2.1) at the same time. Let $(\Psi, E)$ be some characteristic complex, and let $(t, x) \in [0, T] \times \mathbb{R}^n$ and $\psi\in \Psi$. We denote by $\operatorname{Sol}(t, x, E, \psi)$ the set of solutions $(y(\,{\cdot}\,), z(\,{\cdot}\,))$ of the differential inclusion
$$
\begin{equation*}
(\dot{y}(\tau), \dot{z}(\tau))\in E(\tau, y(\tau), \psi),\qquad \tau \in [t, T], \quad (y(\tau), z(\tau)) \in \mathbb{R}^n \times \mathbb{R},
\end{equation*}
\notag
$$
with the initial condition (2.6). Given a function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$, we consider the following conditions.
It turns out ([162], Theorem 6.4) that if $\varphi$ is a lower semicontinuous function satisfying $(2.\mathrm{MS}.2_+)$ for some upper characteristic complex, then it also satisfies this condition for all other possible upper characteristic complexes. In a similar way, for an upper semicontinuous function $\varphi$, if condition $(2.\mathrm{MS}.2_-)$ holds for some lower characteristic complex, then this condition holds for all lower characteristic complexes. Thus, in Definitions 1 and 2 of an upper and a lower minimax solution of the Cauchy problem (2.1), (2.2) there is some freedom in the choice of appropriate characteristic complexes, which sometimes makes it possible to take certain properties of a particular Hamiltonian $H$ into account. For example, when we consider the Hamiltonian $H$ from (1.18), which occurs in the differential game (1.10), (1.11), then we can take the following upper $(Q, E_+)$ and lower $(P, E_-)$ characteristic complexes (see, for example, [99], § 11, and [160], § 14.1), which are closely related to the stability conditions $(2.\mathrm{US}.1)$ and $(2.\mathrm{VS}.1)$:
$$
\begin{equation*}
E_+ (\tau, y, v) = \operatorname{co} \bigl\{(f(\tau, y, u_\ast, v), \chi(\tau, y, u_\ast, v)) \in \mathbb{R}^n \times \mathbb{R} \colon u_\ast \in P \bigr\},
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
E_- (\tau, y, u) = \operatorname{co} \bigl\{ (f(\tau, y, u, v_\ast), \chi(\tau, y, u, v_\ast)) \in \mathbb{R}^n \times \mathbb{R} \colon v_\ast \in Q \bigr\},
\end{equation*}
\notag
$$
$$
\begin{equation*}
\tau \in [0, T], \quad y \in \mathbb{R}^n, \quad u \in P, \quad v \in Q,
\end{equation*}
\notag
$$
where $\operatorname{co}\! A$ denotes the convex hull of the set $A$. A special place in the theory of minimax solutions belongs to the Hamilton–Jacobi equations (2.1) with Hamiltonian $H$ that is positively homogeneous in the third variable, when conditions $(2.\mathrm{CP}.1)$, $(2.\mathrm{CP}.2)$, and the following conditions, which are stronger than $(2.\mathrm{CP}.3)$, hold.
Note that the homogeneity condition $(2.\mathrm{CP}.5)$ is satisfied for the Hamiltonians (1.7) and (1.18), which are related, respectively, to the optimal control problem (1.1), (1.2) and differential game (1.10), (1.11) for $\chi \equiv 0$. To study minimax solutions of the Cauchy problem (2.1), (2.2) under conditions $(2.\mathrm{CP}.1)$, $(2.\mathrm{CP}.2)$, $(2.\mathrm{CP}.4)$, and $(2.\mathrm{CP}.5)$ it is convenient to use, for example, the upper and lower characteristic complexes $(S_n, E_+)$ and $(S_n, E_-)$ (see [160], § 2):
$$
\begin{equation*}
E_+(\tau, y, s) = \bigl\{ f \in \mathbb{R}^n \colon \|f\| \leqslant \sqrt{2}\,c (1 + \|y\|),\, \langle s, f \rangle \geqslant H(\tau, y, s) \bigr\}\times \{0\}\subset \mathbb{R}^n \times \mathbb{R}
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
E_-(\tau, y, s) = \bigl\{ f \in \mathbb{R}^n \colon \|f\| \leqslant \sqrt{2}\,c (1 + \|y\|),\, \langle s, f \rangle \leqslant H(\tau, y, s) \bigr\}\times \{0\}\subset \mathbb{R}^n \times \mathbb{R},
\end{equation*}
\notag
$$
where $\tau \in [0, T]$, $y \in \mathbb{R}^n$, $s \in S_n$, and $c$ is borrowed from $(2.\mathrm{CP}.2)$. In this connection we also note the case when the Hamiltonian $H$ is originally not homogeneous but the Cauchy problem (2.1), (2.2) can be reduced to an auxiliary Cauchy problem with homogeneous Hamiltonian $\overline{H}$: to be precise, when the following conditions hold along with $(2.\mathrm{CP}.1)$.
Note that $(2.\mathrm{CP}.2)$ and $(2.\mathrm{CP}.3)$ are automatically satisfied under conditions $(2.\mathrm{CP}.6)$ and $(2.\mathrm{CP}.7)$. Note also that conditions $(2.\mathrm{CP}.6)$–$(2.\mathrm{CP}.8)$ hold for the Hamiltonian (1.18) if the functions $f$ and $\chi$ satisfy $(2.\mathrm{DG}.1)$–$(2.\mathrm{DG}.4)$ and the following additional condition: there is $c > 0$ such that
$$
\begin{equation*}
|\chi(\tau, y, u, v)|\leqslant c (1 + \|y\|),\quad \tau \in [0, T], \quad y \in \mathbb{R}^n, \quad u \in P, \quad v \in Q.
\end{equation*}
\notag
$$
In the case of conditions $(2.\mathrm{CP}.1)$ and $(2.\mathrm{CP}.6)$–$(2.\mathrm{CP}.8)$ we can associate the original Cauchy problem (2.1), (2.2) with inhomogeneous Hamiltonian $H$ with an auxiliary Cauchy problem for the Hamilton–Jacobi equation:
$$
\begin{equation}
\begin{gathered} \, \frac{\partial \overline{\varphi}}{\partial t} (t, x, z) + \overline{H}\biggl(t,x,\frac{\partial \overline{\varphi}} {\partial x}(t, x, z),\frac{\partial\overline{\varphi}} {\partial z}(t,x,z)\biggr)= 0, \\ (t, x, z) \in (0, T) \times \mathbb{R}^n \times \mathbb{R}, \end{gathered}
\end{equation}
\tag{2.13}
$$
with the right-end boundary condition
$$
\begin{equation}
\overline{\varphi}(T, x, z)= \sigma(x) - z,\qquad (x, z) \in \mathbb{R}^n \times \mathbb{R},
\end{equation}
\tag{2.14}
$$
where the Hamiltonian
$$
\begin{equation*}
\overline{H}(t, x, s, r)= \begin{cases} |r| H \biggl( t, x, \dfrac{s}{|r|} \biggr) & \text{if}\ r \ne 0, \\ H^\circ(t, x, s) & \text{if}\ r = 0 \end{cases}
\end{equation*}
\notag
$$
is homogeneous in the variable $\overline{s} = (s, r)$. The relationship between the solution $\varphi$ of the Cauchy problem (2.1), (2.2) and the solution $\overline{\varphi}$ of the Cauchy problem (2.13), (2.14) is as follows:
$$
\begin{equation*}
\overline{\varphi}(t, x, z)= \varphi(t, x) - z,\qquad (t, x, z) \in [0, T] \times \mathbb{R}^n \times \mathbb{R}.
\end{equation*}
\notag
$$
Thus, in the case under consideration, a minimax solution can be defined using $(2.\mathrm{MS}.2_-)$ and $(2.\mathrm{MS}.2_+)$, based on the upper and lower characteristic complexes $(S_{n + 1}, \overline{E}_+)$ and $(S_{n + 1}, \overline{E}_-)$ (see [160], § 3):
$$
\begin{equation*}
\begin{gathered} \, \overline{E}_+(\tau, y, \overline{s} = \bigl\{ (f, \chi) \in \mathbb{R}^n \times \mathbb{R} \colon \\ \qquad \sqrt{\|f\|^2 + \chi^2}\leqslant \sqrt{2}\,c (1 + \|y\|), \, \langle s, f \rangle + r \chi\geqslant \overline{H}(\tau, y, \overline{s}) \bigr\} \end{gathered}
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
\begin{gathered} \, \overline{E}_-(\tau, y, \overline{s}) = \bigl\{ (f, \chi) \in \mathbb{R}^n \times \mathbb{R} \colon \\ \qquad \sqrt{\|f\|^2 + \chi^2}\leqslant \sqrt{2}\,c (1 + \|y\|), \, \langle s, f \rangle + r \chi\leqslant \overline{H}(\tau, y, \overline{s}) \bigr\}, \end{gathered}
\end{equation*}
\notag
$$
where $\tau \in [0, T]$, $y \in \mathbb{R}^n$, $\overline{s}=(s,r) \in S_{n+1}$, and $c$ is borrowed from $(2.\mathrm{CP}.6)$. 2.4. Infinitesimal criteria and consistencyWe emphasize that the properties $(2.\mathrm{MS}.1_+)$ and $(2.\mathrm{MS}.1_-)$ in Definitions 1 and 2 of an upper and a lower minimax solution of the Cauchy problem (2.1), (2.2), as well as the equivalent properties $(2.\mathrm{MS}.2_+)$ and $(2.\mathrm{MS}.2_-)$, are of non-local nature, that is, to verify them at some point $(t, x)$ directly it is not sufficient to know the values of $\varphi$ in an arbitrarily small neighbourhood of this point. These properties are convenient, for example, for the proof of the well-posedness of the minimax solution. However, to understand whether these properties hold for a particular function $\varphi$, their infinitesimal forms turn out to be useful. Since it is possible in the general case that the minimax solution is not differentiable, we need to employ the apparatus of non-smooth analysis. The following assertion is true (see, for example, [162], Theorem 6.4): given a lower semicontinuous function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$, condition $(2.\mathrm{MS}.2_+)$ for an upper characteristic complex $(\Psi, E)$ is equivalent to the differential inequality
$$
\begin{equation}
\min_{(f, \chi) \in E(t, x, \psi)} \bigl( \partial_- \{\varphi (t, x) \mid 1, f \} - \chi \bigr) \leqslant 0,\quad (t, x) \in (0, T) \times \mathbb{R}^n, \quad \psi \in \Psi.
\end{equation}
\tag{2.15}
$$
In a similar way, given an upper semicontinuous function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$, condition $(2.\mathrm{MS}.2_-)$ for a lower characteristic complex $(\Psi, E)$ is equivalent to the differential inequality
$$
\begin{equation}
\max_{(f, \chi) \in E(t, x, \psi)} \bigl(\partial_+\{\varphi (t,x) \mid 1,f\}-\chi \bigr)\geqslant 0,\quad (t, x) \in (0, T) \times \mathbb{R}^n, \quad \psi \in \Psi.
\end{equation}
\tag{2.16}
$$
Here $\partial_- \{ \varphi (t, x) \mid 1, f \}$ and $\partial_+ \{ \varphi (t, x) \mid 1, f \}$ denote the lower and upper derivatives of $\varphi$ at $(t, x)$ in the direction $(1, f)$:
$$
\begin{equation}
\begin{aligned} \, \partial_- \{\varphi (t, x) \mid 1, f \}& = \liminf_{\delta \to 0^+, \, g \to f} \frac{\varphi(t + \delta, x + \delta g) - \varphi(t, x)}{\delta}\,, \\ \partial_+ \{ \varphi (t, x) \mid 1, f \}& = \limsup_{\delta \to 0^+, \, g \to f} \frac{\varphi(t + \delta, x + \delta g) - \varphi(t, x)}{\delta}\,. \end{aligned}
\end{equation}
\tag{2.17}
$$
In particular, considering the characteristic complex $(\mathbb{R}^n, E_0)$, where $E_0$ is defined by (2.4), we deduce that, under conditions $(2.\mathrm{CP}.1)$–$(2.\mathrm{CP}.3)$, the minimax solution of the Cauchy problem (2.1), (2.2) can be characterized as the unique continuous function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ satisfying the boundary condition (2.2) and the following differential inequalities, which are equivalent to $(2.\mathrm{MS}.1_+)$ and $(2.\mathrm{MS}.1_-)$ respectively:
$$
\begin{equation}
\begin{gathered} \, \min_{f \in B(c (1 + \|x\|))}\bigl( \partial_- \{\varphi(t, x) \mid 1,f\}- \langle s, f \rangle \bigr) + H(t, x, s)\leqslant 0, \\ (t, x) \in (0, T) \times \mathbb{R}^n, \quad s \in \mathbb{R}^n, \end{gathered}
\end{equation}
\tag{2.18}
$$
and
$$
\begin{equation}
\begin{gathered} \, \max_{f \in B(c(1+\|x\|))}\bigl(\partial_+ \{\varphi(t,x) \mid 1,f\} - \langle s, f \rangle \bigr) + H(t, x, s)\geqslant 0, \\ (t, x) \in (0, T) \times \mathbb{R}^n, \quad s \in \mathbb{R}^n, \end{gathered}
\end{equation}
\tag{2.19}
$$
where $B( c(1 + \|x\|))$ is the ball in $\mathbb{R}^n$ with centre at the origin and radius $c(1+\|x\|)$, and the number $c$ is from $(2.\mathrm{CP}.2)$. The above criterion makes it possible to establish easily the consistency of the minimax solution with the concept of solution in the classical sense. In fact, assume that a function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ is differentiable at some point $(t, x) \in (0, T) \times \mathbb{R}^n$. Then
$$
\begin{equation*}
\partial_- \{ \varphi (t, x) \mid 1, f \}= \frac{\partial \varphi }{\partial t}(t, x) + \biggl\langle\frac{\partial\varphi}{\partial x}(t,x),f\biggr\rangle,\qquad f \in \mathbb{R}^n,
\end{equation*}
\notag
$$
and (2.18) takes the form
$$
\begin{equation}
\frac{\partial \varphi}{\partial t} (t, x)+ H(t, x, s) - c (1 + \|x\|) \biggl\|\frac{\partial\varphi}{\partial x}(t,x)-s\biggr\|\leqslant 0,\qquad s \in \mathbb{R}^n.
\end{equation}
\tag{2.20}
$$
For $s = \partial \varphi(t, x) / \partial x$ inequality (2.20) turns to
$$
\begin{equation}
\frac{\partial \varphi}{\partial t} (t, x) + H \biggl(t,x,\frac{\partial \varphi}{\partial x} (t, x) \biggr)\leqslant 0.
\end{equation}
\tag{2.21}
$$
On the other hand, in view of condition $(2.\mathrm{CP}.2)$ inequality (2.21) ensures (2.20). In a similar way (2.19) turns out to be equivalent to the inequality
$$
\begin{equation*}
\frac{\partial \varphi}{\partial t} (t, x) + H\biggl(t,x,\frac{\partial\varphi}{\partial x}(t,x)\biggr)\geqslant 0;
\end{equation*}
\notag
$$
therefore, the pair of inequalities (2.18), (2.19) is equivalent to (2.1). Thus, first, the minimax solution satisfies the Hamilton–Jacobi equation (2.1) at all points where it is differentiable; second, if there is a continuous function $\varphi \colon [0,T] \times \mathbb{R}^n \to \mathbb{R}$ that is differentiable at all points $(t,x) \in (0,T) \times \mathbb{R}^n$ and satisfies equation (2.1) and boundary condition (2.2), then it is a minimax solution of the Cauchy problem (2.1), (2.2). In this sense the pair of differential inequalities (2.18), (2.19) can be regarded as a possible generalization of the Hamilton–Jacobi equation (2.1) to the non-smooth case. 2.5. Meaningfulness: value function and optimal strategiesIn the context of optimal control problems and differential games, the meaningfulness of the concept of a minimax solution actually follows directly from its definition, since this definition is precisely based on the characteristic properties of the value function. Namely, the following assertion is true (in this connection, see [160], Theorem 14.1, and [162], Theorem 12.4): under conditions $(2.\mathrm{DG}.1)$–$(2.\mathrm{DG}.4)$ the value function $\rho$ of the differential game (1.10), (1.11) coincides with the unique minimax solution $\varphi$ of the Cauchy problem (2.1), (2.2) with Hamiltonian (1.18). It follows that if, for a particular differential game of the form (1.10), (1.11), on the basis of some considerations we have managed to construct a continuous function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ satisfying boundary condition (2.2) and the differential inequalities (2.15) and (2.16) for some characteristic complexes corresponding to the Hamiltonian $H$ defined by (1.18), then $\varphi$ is the value function of the game. Assume that $\varphi$ satisfies the following Lipschitz condition with respect to the second variable.
In this case, once inequalities (2.15) and (2.16) have been verified, the lower and upper directional derivatives can be calculated using the following formulae, which are simpler than the definitions (2.17) (see, for example, [160], § 5):
$$
\begin{equation}
\begin{aligned} \, \partial_- \{ \varphi (t, x) \mid 1, f \} & = \liminf_{\delta \to 0^+} \frac{\varphi(t + \delta, x + \delta f) - \varphi(t, x)}{\delta}\,, \\ \partial_+ \{ \varphi (t, x) \mid 1, f \} & = \limsup_{\delta \to 0^+} \frac{\varphi(t + \delta, x + \delta f) - \varphi(t, x)}{\delta}\,. \end{aligned}
\end{equation}
\tag{2.22}
$$
Note (in this connection, see, for example, [96], § 29) that the value function of the differential game (1.10), (1.11) satisfies $(2.\mathrm{L})$ if the following additional condition holds, which is stronger than the assumption of the continuity of $\sigma$ (see $(2.\mathrm{DG}.1)$): for any bounded set $W \subset \mathbb{R}^n$ there is $\lambda > 0$ such that
$$
\begin{equation*}
|\sigma(y_1) - \sigma(y_2)| \leqslant \lambda \|y_1 - y_2\|, \qquad y_1, y_2 \in W.
\end{equation*}
\notag
$$
The fact that the value function $\rho$ of the differential game (1.10), (1.11) coincides with the minimax solution $\varphi$ of the Cauchy problem (2.1), (2.2) with Hamiltonian (1.18) can be proved in various ways. The proof usually given in the theory of minimax solutions is based on the construction of optimal feedback control strategies of the players from the minimax solution. In this case the existence of a game value is established independently. First, following [44] (also see [162], § 12.2), we present the corresponding constructions in the case when the following conditions hold for the differential game (1.10), (1.11), along with $(2.\mathrm{DG}.1)$ and $(2.\mathrm{DG}.4)$.
Note that condition $(2.\mathrm{DG}.5)$ implies $(2.\mathrm{DG}.2)$ and $(2.\mathrm{DG}.3)$. Conditions $(2.\mathrm{DG}.5)$ and $(2.\mathrm{DG}.6)$ ensure, in particular, the existence of $\kappa \geqslant 0$ such that the minimax solution $\varphi$ satisfies the estimate
$$
\begin{equation}
|\varphi(t, x)| \leqslant \kappa (1 + \|x\|), \qquad (t, x) \in [0, T] \times \mathbb{R}^n.
\end{equation}
\tag{2.23}
$$
We borrow $\lambda$ from $(2.\mathrm{DG}.5)$ and take $\varepsilon > 0$ satisfying the inequality
$$
\begin{equation}
\frac{e^{-\lambda T}-\varepsilon}{\varepsilon}> \kappa;
\end{equation}
\tag{2.24}
$$
in accordance with (2.12), we consider the Lyapunov function $\nu_\varepsilon$ used in the proof of the comparison principle for minimax solutions. For each point $(\tau,y) \in [0,T] \times \mathbb{R}^n$ we choose a vector $z_\varepsilon^{(u)}(\tau,y)$ so that
$$
\begin{equation*}
z_\varepsilon^{(u)}(\tau, y)\in \operatorname*{arg\,min}_{z \in \mathbb{R}^n}(\varphi(\tau, z) + \nu_\varepsilon(\tau, y - z)).
\end{equation*}
\notag
$$
In view of the continuity of $\varphi$ and $\nu_\varepsilon$ this is possible by (2.23) and (2.24). We set
$$
\begin{equation*}
s_\varepsilon^{(u)}(\tau, y)= \frac{\partial \nu_\varepsilon}{\partial w} \bigl(\tau, y - z_\varepsilon^{(u)}(\tau, y)\bigr)= \frac{e^{- \lambda \tau} - \varepsilon}{\varepsilon \sqrt{\varepsilon^4 + \|y - z_\varepsilon^{(u)}(\tau, y)\|^2}} \bigl(y - z_\varepsilon^{(u)}(\tau, y)\bigr).
\end{equation*}
\notag
$$
The vector $s_\varepsilon^{(u)}(\tau, y)$ is interpreted as the generalized gradient of the function $\varphi$ at the point $(\tau, y)$ and is sometimes called the quasi-gradient. This is partly owing to the fact that if a function $\mathbb{R}^n \ni x \mapsto \varphi(\tau, x) \in \mathbb{R}$ turns out to be continuously differentiable in a neighbourhood of $y$, then $s_\varepsilon^{(u)}(\tau, y) \to \partial \varphi(\tau, y) / \partial x$ as $\varepsilon \to 0^+$. Thus, replacing the gradient $\partial \varphi(\tau, y) / \partial x$, which possibly does not exist in the general case, by the quasi-gradient $s_\varepsilon^{(u)}(\tau, y)$ in the first formula in (1.19), we arrive at the following rule for constructing the control strategy of the first player:
$$
\begin{equation}
\begin{gathered} \, U^\circ_\varepsilon(\tau,y)\in \operatorname*{arg\,min}_{u \in P}\, \max_{v \in Q}\bigl(\langle s_\varepsilon^{(u)}(\tau,y),f(\tau,y,u,v)\rangle- \chi(\tau, y, u, v) \bigr), \\ (\tau, y) \in [0, T] \times \mathbb{R}^n. \end{gathered}
\end{equation}
\tag{2.25}
$$
In a similar way we obtain the control strategy of the second player:
$$
\begin{equation}
\begin{gathered} \, V^\circ_\varepsilon (\tau, y)\in \operatorname*{arg\,max}_{v \in Q}\, \min_{u \in P}\bigl(\langle s_\varepsilon^{(v)}(\tau,y),f(\tau,y,u,v)\rangle - \chi(\tau, y, u, v) \bigr), \\ (\tau, y) \in [0, T] \times \mathbb{R}^n, \end{gathered}
\end{equation}
\tag{2.26}
$$
where
$$
\begin{equation*}
s_\varepsilon^{(v)}(\tau, y)=-\frac{\partial \nu_\varepsilon}{\partial w} \bigl(\tau, y - z_\varepsilon^{(v)}(\tau, y)\bigr)\ \ \text{and}\ \ z_\varepsilon^{(v)}(\tau, y)\in \operatorname*{arg\,max}_{z \in \mathbb{R}^n} \bigl(\varphi(\tau, z) - \nu_\varepsilon(\tau, y - z)\bigr).
\end{equation*}
\notag
$$
To describe the optimality properties of the strategies $U^\circ_\varepsilon$ and $V^\circ_\varepsilon$, we introduce some auxiliary concepts and notation (for more details, see, for example, [96], § § 7 and 8, and [90], § § 5 and 6). Assume that we have an initial position $(t, x) \in [0, T]$ and a partition $\Delta = (\tau_j)_{j \in \{1,\dots, k + 1\}}$ of the interval $[t, T]$: $\tau_1 = t$, $\tau_j < \tau_{j + 1}$ for all $j \in \{1,\dots, k\}$, $\tau_{k+1}=T$. We let $J(t,x,U^\circ_\varepsilon,\Delta,v(\,{\cdot}\,))$ denote the value of the quality index (1.11) realized in the case when, during the game, the first player forms a piecewise constant feedback control $u(\,{\cdot}\,) \in \mathcal{U}(t)$ based on the partition $\Delta$ according to the rule
$$
\begin{equation}
u(\tau)= U^\circ_\varepsilon(\tau_j, y(\tau_j)),\qquad \tau \in [\tau_j, \tau_{j + 1}),\quad j \in \{1,\dots k\},
\end{equation}
\tag{2.27}
$$
while the second player uses a control $v(\,{\cdot}\,) \in \mathcal{V}(t)$. We define the guaranteed result of the control strategy $U^\circ_\varepsilon$ of the first player by
$$
\begin{equation}
\rho^{(u)} (t, x, U^\circ_\varepsilon)= \lim_{\delta \to 0^+}\, \sup_{\Delta \in \Pi_\delta(t)}\,\sup_{v(\,{\cdot}\,) \in \mathcal{V}(t)} J(t, x, U^\circ_\varepsilon, \Delta, v(\,{\cdot}\,)),
\end{equation}
\tag{2.28}
$$
where $\Pi_\delta(t)$ is the set of partitions $\Delta$ of the interval $[t, T]$ with diameter below $\delta$. In a similar way we introduce the guaranteed result of the control strategy $V^\circ_\varepsilon$ of the second player:
$$
\begin{equation}
\rho^{(v)} (t, x, V^\circ_\varepsilon)= \lim_{\delta \to 0^+}\, \inf_{\Delta \in \Pi_\delta(t)}\,\inf_{u(\,{\cdot}\,) \in \mathcal{U}(t)} J(t, x, u(\,{\cdot}\,), V^\circ_\varepsilon, \Delta),
\end{equation}
\tag{2.29}
$$
where $J(t, x, u(\,{\cdot}\,), V^\circ_\varepsilon, \Delta)$ is the value of the quality index (1.11) corresponding to the control $u(\,{\cdot}\,) \in \mathcal{U}(t)$ of the first player and the control $v(\,{\cdot}\,) \in \mathcal{V}(t)$ of the second player, which is formed by the equality
$$
\begin{equation}
v(\tau)= V^\circ_\varepsilon(\tau_j, y(\tau_j)),\qquad \tau \in [\tau_j, \tau_{j + 1}),\quad j \in \{1,\dots, k\}.
\end{equation}
\tag{2.30}
$$
Based immediately on the definition of the minimax solution $\varphi$ and the properties of the Lyapunov function $\nu_\varepsilon$, we can show (see [44] and also [162], Theorem 12.3) that
$$
\begin{equation}
\limsup_{\varepsilon \to 0^+}\rho^{(u)}(t,x,U^\circ_\varepsilon)\leqslant \varphi(t, x)\leqslant \liminf_{\varepsilon \to 0^+} \rho^{(v)}(t, x, V^\circ_\varepsilon)
\end{equation}
\tag{2.31}
$$
for each position $(t, x) \in [0, T] \times \mathbb{R}^n$. On the other hand, we have
$$
\begin{equation}
\rho^{(v)} (t, x, V^\circ_\varepsilon) \leqslant \rho_- (t, x) \leqslant \rho^{(u)} (t, x, U^\circ_\varepsilon),
\end{equation}
\tag{2.32}
$$
where $\rho_- (t, x)$ is the lower game value (1.12). In fact, to verify the first inequality in (2.32) we must consider the case when the first player uses an arbitrary quasi-strategy $\boldsymbol{\alpha} \in \boldsymbol{\mathcal{A}}(t)$, while the second player forms a control $v(\,{\cdot}\,) \in \mathcal{V}(t)$ according to (2.30). The second inequality in (2.32) follows from the fact that the mapping associating each control $v(\,{\cdot}\,) \in \mathcal{V}(t)$ of the second player with the control $u(\,{\cdot}\,) \in \mathcal{U}(t)$ of the first player formed by (2.27) is a quasi-strategy of the first player. We can similarly establish that
$$
\begin{equation}
\rho^{(v)} (t, x, V^\circ_\varepsilon)\leqslant \rho_+ (t, x)\leqslant \rho^{(u)} (t, x, U^\circ_\varepsilon),
\end{equation}
\tag{2.33}
$$
where $\rho_+ (t, x)$ is the upper game value (1.13). By (2.31)–(2.33), first, we have hence the differential game has value $\rho(t,x)=\varphi(t,x)$. Second, we obtain
$$
\begin{equation}
\lim_{\varepsilon \to 0^+} \rho^{(u)} (t, x, U^\circ_\varepsilon) = \rho(t,x)=\lim_{\varepsilon \to 0^+}\rho^{(v)}(t,x,V^\circ_\varepsilon),
\end{equation}
\tag{2.34}
$$
which reflects the optimality properties of the strategies $U^\circ_\varepsilon$ and $V^\circ_\varepsilon$. We can abandon conditions $(2.\mathrm{DG}.5)$ and $(2.\mathrm{DG}.6)$, additional to $(2.\mathrm{DG}.1)$–$(2.\mathrm{DG}.4)$, if the rules (2.25) and (2.26) for constructing the control strategies of the players are modified. The distinction is that now the constructions depend on the prescribed initial position $(t_0, x_0) \in [0, T] \times \mathbb{R}^n$ (or, in a more general case, on an arbitrary bounded set of points $(t_0, x_0)$ in $[0, T] \times \mathbb{R}^n$). For any $t \in [t_0, T]$ we consider a compact set $W(t)$ of points $x \in \mathbb{R}^n$ for each of which there exists a function $y(\,{\cdot}\,) \in \operatorname{Lip}([t_0,t];\mathbb{R}^n)$ such that
$$
\begin{equation*}
y(t_0)= x_0\quad\text{and}\quad \|\dot{y}(\tau)\| \leqslant c(1+\|y(\tau)\|)\quad \text{for a. e. } \tau \in [t_0,t],\qquad y(t) = x,
\end{equation*}
\notag
$$
where $c$ is from condition $(2.\mathrm{DG}.2)$. Note that the family of sets $W(t)$, $t \in [t_0, T]$, has the following property of strong invariance with respect to motions of the dynamical system (1.10): for any motion $y(\,{\cdot}\,)$ of this system from the position $(t, x)$, where $t \in [t_0, T]$ and $x \in W(t)$, which is generated by controls $u(\,{\cdot}\,) \in \mathcal{U}(t)$ and $v(\,{\cdot}\,) \in \mathcal{V}(t)$ of the players, we have $y(\tau) \in W(\tau)$, $\tau \in [t, T]$. Taking account of the fact that $W(t) \subset W(T)$ for $t \in [t_0, T]$, we choose $\lambda$ for $W(T)$ in accordance with $(2.\mathrm{DG}.3)$ and consider the Lyapunov function $\nu_\varepsilon$ for $\varepsilon > 0$ (see (2.12)). We obtain strategies $\overline{U}^\circ_\varepsilon$ and $\overline{V}^\circ_\varepsilon$ by using (2.25) and (2.26) with the only exception that now, when the vectors $z_\varepsilon^{(u)}(\tau, y)$ and $z_\varepsilon^{(v)}(\tau, y)$ are specified, the corresponding minimum and maximum are considered not on the whole of $\mathbb{R}^n$ but on the compact set $W(\tau)$:
$$
\begin{equation*}
z_\varepsilon^{(u)}(\tau, y) \in \operatorname*{arg\,min}_{z \in W(\tau)} (\varphi(\tau, z) + \nu_\varepsilon(\tau, y - z))
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
z_\varepsilon^{(v)}(\tau, y) \in \operatorname*{arg\,max}_{z \in W(\tau)} (\varphi(\tau, z) - \nu_\varepsilon(\tau, y - z)).
\end{equation*}
\notag
$$
Now, repeating in fact the proof of (2.31) we can show that
$$
\begin{equation*}
\limsup_{\varepsilon \to 0^+} \rho^{(u)} (t_0,x_0,\overline{U}^\circ_\varepsilon)\leqslant \varphi(t_0,x_0)\leqslant \liminf_{\varepsilon \to 0^+} \rho^{(v)}(t_0,x_0,\overline{V}^\circ_\varepsilon).
\end{equation*}
\notag
$$
Just like above, this implies the existence of the game value $\rho(t_0, x_0)$, the equality $\rho(t_0, x_0) = \varphi(t_0, x_0)$, and also the optimality of the strategies $\overline{U}^\circ_\varepsilon$ and $\overline{V}^\circ_\varepsilon$ for the initial position $(t_0, x_0)$:
$$
\begin{equation*}
\lim_{\varepsilon \to 0^+}\rho^{(u)}(t_0,x_0,\overline{U}^\circ_\varepsilon)= \rho(t_0, x_0)=\lim_{\varepsilon \to 0^+}\rho^{(v)} (t_0,x_0,\overline{V}^\circ_\varepsilon).
\end{equation*}
\notag
$$
2.6. Viscosity solutionsAs already noted in the introduction, the minimax approach to defining a generalized solution of the Hamilton–Jacobi equations is closely related to another intensively developing approach, called the viscosity approach (see, for example, [27] and [25], and also [8], [41], and [173]). We give a definition of a viscosity solution of the Cauchy problem (2.1), (2.2). Definition 4. An upper viscosity solution of the Cauchy problem (2.1), (2.2) is a continuous function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ that satisfies the boundary condition (2.7) and has the following property.
Definition 5. A lower viscosity solution of the Cauchy problem (2.1), (2.2) is a continuous function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ that satisfies the boundary condition (2.9) and has the following property.
Definition 6. A viscosity solution of the Cauchy problem (2.1), (2.2) is a function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ that is both an upper and a lower viscosity solution of this problem. Note that the functions $\psi$ in $(2.\mathrm{V}_+)$ and $(2.\mathrm{V}_-)$ are sometimes called underlying or test functions. Under conditions $(2.\mathrm{CP}.1)$–$(2.\mathrm{CP}.3)$ a viscosity solution of the Cauchy problem (2.1), (2.2) exists and is unique (see, for example, [30], Theorem VI.1). In the theory of viscosity solutions existence is usually proved by the vanishing viscosity method (see, for example, [109], [27], [25], [157], and also [101] and [173], § 1.3). However, if the theory of minimax solutions is used, then this follows from the fact that a minimax solution is a viscosity solution. Consider indeed a minimax solution $\varphi$ of the Cauchy problem (2.1), (2.2). We can show, for example, that it has property $(2.\mathrm{V}_+)$. Assume that the difference $\varphi-\psi$ attains a local minimum at some point $(t, x) \in (0, T) \times \mathbb{R}^n$. Owing to property $(2.\mathrm{MS})$, for $s = \partial \psi(t, x) / \partial x$ there exists a function $y(\,{\cdot}\,) \in \operatorname{Lip}([t,T];\mathbb{R}^n)$ satisfying conditions (2.3) for $\vartheta = T$ and equality (2.11). Since
$$
\begin{equation*}
\varphi(t, x) - \psi(t, x)\leqslant \varphi(\vartheta, y(\vartheta)) - \psi(\vartheta, y(\vartheta)),\qquad \vartheta \in [t, t + \delta],
\end{equation*}
\notag
$$
for some number $\delta \in (0, T - t]$, by the choice of $y(\,{\cdot}\,)$ we have
$$
\begin{equation*}
\psi(\vartheta, y(\vartheta)) - \psi(t, x)\leqslant \biggl\langle \frac{\partial \psi}{\partial x}(t, x), y(\vartheta) - x \biggr\rangle- \int_{t}^{\vartheta} H \biggl( \xi, y(\xi), \frac{\partial \psi}{\partial x} (t, x) \biggr) \, {\rm d}\xi
\end{equation*}
\notag
$$
for all $\vartheta \in [t, t + \delta]$. It follows from the differentiability of $\psi$ at $(t, x)$ that inequality (2.35) is true. Like in the case of minimax solutions, the verification of the uniqueness of a viscosity solution consists in the proof of the comparison principle, that is, the fact that each upper viscosity solution $\varphi_+$ dominates each lower viscosity solution $\varphi_-$ (see, for example, [173], Theorem 2.5, and [176], Theorem 2.5.3, under conditions $(2.\mathrm{CP}.1)$–$(2.\mathrm{CP}.3)$). Usually, one reasons by contradiction and uses the idea of doubling the number of variables: for $(t, x),(\tau, y) \in [0, T] \times \mathbb{R}^n$ one considers the difference $\varphi_- (t, x) - \varphi_+ (\tau, y)$. To this difference one adds some specially chosen correction terms, among which the penalty functions
$$
\begin{equation}
-\frac{(t-\tau)^2}{\delta}\quad\text{and}\quad -\frac{\|x-y\|^2}{\varepsilon}\,,
\end{equation}
\tag{2.36}
$$
where $\delta > 0$ and $\varepsilon > 0$ are small parameters, play an important role. Based on the search for maximum points of the resulting function in appropriate subsets of points $(t, x, \tau, y)$ in the space $[0, T] \times \mathbb{R}^n \times [0, T] \times \mathbb{R}^n$, a pair of test functions is constructed, using which we obtain a contradiction. In the infinitesimal form conditions $(2.\mathrm{V}_+)$ and $(2.\mathrm{V}_-)$, defining viscosity solutions, can be expressed in terms of inequalities for sub- and supergradients. Namely (see, for example, [125], Theorem 1.1, and also [173], § 1.2.3), for a continuous function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ condition $(2.\mathrm{V}_+)$ is equivalent to the differential inequality
$$
\begin{equation}
a+ H(t, x, s)\leqslant 0,\qquad (a, s) \in D_- \varphi(t, x), \quad (t, x) \in (0, T) \times \mathbb{R}^n,
\end{equation}
\tag{2.37}
$$
while $(2.\mathrm{V}_-)$ is equivalent to the differential inequality
$$
\begin{equation}
a+ H(t, x, s)\geqslant 0,\qquad (a, s) \in D_+ \varphi(t, x), \quad (t, x) \in (0, T) \times \mathbb{R}^n.
\end{equation}
\tag{2.38}
$$
Here $D_- \varphi(t, x)$ and $D_+ \varphi(t, x)$ denote the sub- and superdifferential of $\varphi$ at the point $(t, x)$, respectively:
$$
\begin{equation}
\begin{aligned} \, D_- \varphi(t,x)& = \biggl\{(a,s) \in \mathbb{R} \times \mathbb{R}^n \colon \\ & \qquad \limsup_{(\tau, y) \to (t, x)} \frac{\varphi(\tau, y) - \varphi(t,x)-a(\tau - t) - \langle s, y - x \rangle}{|\tau - t| + \|y - x\|} \leqslant 0 \biggr\}, \\ D_+ \varphi(t,x)& = \biggl\{(a,s) \in \mathbb{R} \times \mathbb{R}^n \colon \\ & \qquad \liminf_{(\tau, y) \to (t, x)} \frac{\varphi(\tau, y) - \varphi(t,x)-a(\tau-t)-\langle s,y-x\rangle}{|\tau-t|+\|y-x\|} \geqslant 0 \biggr\}. \end{aligned}
\end{equation}
\tag{2.39}
$$
Thus, in the framework of the viscosity approach, the Hamilton–Jacobi equation (2.1) is generalized to the non-smooth case by replacing it by the pair of inequalities (2.37), (2.38). Since the minimax solution of the Cauchy problem (2.1), (2.2) is a viscosity solution of this problem, from the uniqueness of a viscosity solution we conclude that the minimax and viscosity solutions coincide. In addition, there is a stronger result ([162], Theorem 6.4) which establishes the equivalence of the properties used in the definitions of minimax and viscosity solutions: under conditions $(2.\mathrm{CP}.1)$–$(2.\mathrm{CP}.3)$ the differential inequalities (2.18) and (2.37) are equivalent for each lower semicontinuous function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$; the differential inequalities (2.19) and (2.38) are equivalent for each upper semicontinuous function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$. We comment briefly on the proof of the first assertion. The fact that (2.37) implies (2.18) can easily be verified directly on the basis of the definitions of the lower directional derivative (2.17) and subdifferential (2.39). A key result, using which it is possible to prove the converse, is the following very subtle property of the subdifferential ([161], Theorem 1.1; also see [160], Lemma 6.4, and [162], § A6): if
$$
\begin{equation*}
\partial_- \{\varphi(t_0,x_0) \mid 1,f\}> 0,\qquad f \in F,
\end{equation*}
\notag
$$
for a lower semicontinuous function $\varphi \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$, a point $(t_0, x_0) \in (0, T) \times \mathbb{R}^n$, and a set $F \in \mathcal{K}(\mathbb{R}^n)$, then there exists a point $(t, x)$ in an arbitrarily small neighbourhood of $(t_0, x_0)$ at which $\varphi$ has a subgradient $(a, s) \in D_-\varphi (t, x)$ such that Note that this property can be obtained as an infinitesimal version of the multidimensional non-smooth generalization of the finite increment formula [19], [20], [86] (also see [162], § A6, and [21], Chap. 3, Theorem 2.6).
3. Time-delay systemsAn answer to the question of equations playing the role of Hamilton–Jacobi equations for time-delay systems (1.20) is closely related to dynamic programming constructions in control problems for systems of this type. The formalization of these constructions is based on the functional interpretation of time-delay systems [92] (also see, for example, [71]), according to which their evolution is considered in an appropriate function space of motion histories. Two main approaches can conventionally be distinguished here. The first is based on switching directly to the description of the evolution of time-delay systems by ordinary differential equations in a function state space (see, for example, [92], § 29, [154], [155], and [71], § 7.11, as well as [12], Chap. 4). For simplicity we explain the main idea without going into detail, by taking an example of a system with concentrated delay (see (1.21)) of the form
$$
\begin{equation}
\dot{y}(\tau) = f_1(\tau, y(\tau), y(\tau - h)),\qquad \tau \in [0, T], \quad y(\tau) \in \mathbb{R}^n.
\end{equation}
\tag{3.1}
$$
We let $y_\tau(\,{\cdot}\,)$ denote the motion history $y(\,{\cdot}\,)$ of this system on the interval $[\tau-h,\tau]$ shifted to the unified interval $[-h,0]$:
$$
\begin{equation*}
y_\tau(\theta) = y(\tau + \theta),\qquad \theta \in [- h, 0].
\end{equation*}
\notag
$$
It is natural to consider the evolution of $y_\tau(\,{\cdot}\,)$ with $\tau$ in the space $\operatorname{C}([- h, 0]; \mathbb{R}^n)$. Then we arrive at the differential equation
$$
\begin{equation}
\dot{\boldsymbol{y}}[\tau]=\boldsymbol{f}_1(\tau,\boldsymbol{y}[\tau]),\qquad \tau \in [0, T], \quad \boldsymbol{y}[\tau] \in \operatorname{C}([-h,0];\mathbb{R}^n),
\end{equation}
\tag{3.2}
$$
where
$$
\begin{equation*}
\boldsymbol{f}_1(\tau, \boldsymbol{y}[\tau])(\theta)= \begin{cases} \dfrac{{\rm d} \boldsymbol{y}[\tau](\theta)}{{\rm d} \theta}\ & \text{if}\ \theta \in [- h, 0), \\ f_1(\tau,\boldsymbol{y}[\tau](0),\boldsymbol{y}[\tau](-h)) & \text{if}\ \theta = 0. \end{cases}
\end{equation*}
\notag
$$
The relationship between solutions of (3.1) and (3.2) is given by the equalities
$$
\begin{equation*}
\boldsymbol{y}[\tau]= y_\tau(\,{\cdot}\,)\quad\text{and}\quad y(\tau)= \boldsymbol{y}[\tau](0).
\end{equation*}
\notag
$$
Note that for each $\tau \in [0, T]$ the domain of the unbounded operator $\boldsymbol{y} \mapsto \boldsymbol{f}_1(\tau, \boldsymbol{y})$ is a subset of $\operatorname{C}([- h, 0]; \mathbb{R}^n)$ consisting of continuously differentiable functions $\boldsymbol{y} \colon [- h, 0] \to \mathbb{R}^n$ whose derivative at the point $\theta = 0$ coincides with $f_1(\tau, \boldsymbol{y}(0), \boldsymbol{y}(- h))$. As a result, time-delay systems are included in a wider class of dynamical systems which, in particular, covers systems described by partial differential equations. When this approach is consistently applied to control problems for time-delay systems, we arrive at Hamilton–Jacobi equations in the space of continuous functions with Frechét derivatives. Quite a lot of research works are devoted to Hamilton–Jacobi equations in abstract Banach spaces with Frechét derivatives and develop the theory of generalized solutions (in the viscosity sense) of equations of this type. We limit ourselves here to referring to the series of seven papers that began with [28] and [29], to the papers [17], [74], [172], [31], [87], [148], and [169], and to the monographs [7], [105], and [37]. The greatest progress has been achieved in the case of Hilbert spaces. As basic applications of this theory, control problems for systems whose dynamics is described by partial differential equations are considered. There are also several works where control problems for different time-delay systems are studied using results obtained in this theory (see, for example, [156], [9], [18], [38], and [39]). Nevertheless, this approach does not make it possible to cover fully time-delay systems of the general form (1.20). One reason is that the development of a full-fledged theory of generalized solutions of Hamilton–Jacobi equations in the space of continuous functions has failed. The difficulties arising are largely due to the bad differentiability properties of the norm in this space. On the other hand the space of continuous functions is natural for the description of the space of states of time-delay systems (in this connection see, for example, [92], [71], and also [175]). Our survey is devoted to another approach, in which we do not explicitly switch to the description of time-delay systems by ordinary differential equations in a function space. In this approach the properties of the optimal result (game value) under shifts along possible trajectories of the system are investigated directly. In this case it is possible to consider the hereditary nature of time-delay systems in more details and to cover the general class of systems (1.20). When this approach is used, the classical machinery of Frechét derivatives turns out to be inconvenient; therefore, we have to consider special concepts of differentiability of functionals of the motion history and use derivatives adequate to these concepts. For example, so-called coinvariant derivatives are suitable [111] (also see [123], § 2). The ideas of invariant and coinvariant differentiation of functionals and the terms invariant and coinvariant derivatives themselves were originally proposed in [83], in connection with the second Lyapunov method in stability problems for time-delay systems. Subsequently, derivatives of this type found applications to other branches of the theory of functional differential equations (see, for example, [84] and [85]). Among close concepts of derivatives, we note Clio derivatives [5] and also horizontal and vertical derivatives [33] (see § 3.9 for details). Thus, this approach leads to a new class of Hamilton–Jacobi equations, which is under consideration in this section. 3.1. Hamilton–Jacobi equations with coinvariant derivativesWe consider the space $[0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ with metric
$$
\begin{equation}
\begin{gathered} \, \operatorname{dist} \bigl( (t_1, x_1(\,{\cdot}\,)), (t_2, x_2(\,{\cdot}\,)) \bigr) = |t_1 - t_2| + \|x_1(\,{\cdot}\,) - x_2(\,{\cdot}\,)\|_\infty, \\ (t_1, x_1(\,{\cdot}\,)), (t_2, x_2(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n), \end{gathered}
\end{equation}
\tag{3.3}
$$
where $\|\cdot\|_\infty$ is the uniform norm in $\operatorname{C}([- h, T]; \mathbb{R}^n)$:
$$
\begin{equation}
\|x(\,{\cdot}\,)\|_\infty= \max_{\tau \in [- h, T]} \|x(\tau)\|,\qquad x(\,{\cdot}\,) \in \operatorname{C}([- h, T]; \mathbb{R}^n).
\end{equation}
\tag{3.4}
$$
Given a pair $(t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n)$, we let $\operatorname{Lip}(t, x(\,{\cdot}\,))$ denote the set of functions $y(\,{\cdot}\,) \in \operatorname{C}([- h, T]; \mathbb{R}^n)$ that satisfy the equality and are Lipschitz continuous on the interval $[t, T]$. Definition 7. A functional $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ is coinvariantly differentiable (ci-differentiable) at a point $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ if there exist $\partial_t \varphi(t, x(\,{\cdot}\,)) \in \mathbb{R}$ and $\nabla \varphi(t, x(\,{\cdot}\,)) \in \mathbb{R}^n$ such that The term coinvariant is intended to emphasize that, first, the quantities $\partial_t \varphi(t, x(\,{\cdot}\,))$ and $\nabla \varphi(t, x(\,{\cdot}\,))$ are independent of $y(\,{\cdot}\,) \in \operatorname{Lip}(t, x(\,{\cdot}\,))$ (‘invariance’) and, second, the variables $\tau$ and $y(\,{\cdot}\,)$ vary simultaneously and consistently (the prefix co). We comment on some distinctions between ci-differentiability and the classical concepts of differentiability of functionals. First we note that only functions $y(\,{\cdot}\,)$ in $\operatorname{Lip}(t,x(\,{\cdot}\,))$ are considered to be admissible increments of the argument $x(\,{\cdot}\,)$. Hence, by (3.5) the values of $x(\,{\cdot}\,)$ on $[- h, t]$ do not vary; since $y(\,{\cdot}\,)$ is a Lipschitz function on $[t, T]$, only the difference $\tau-t$ is taken as the argument of the infinitesimal $o(\,{\cdot}\,)$. In addition, the linear part of the increment in (3.6) has a special form that takes account only of the value of $y(\,{\cdot}\,)$ at the point $\tau$. Therefore, in particular, the ci-gradient $\nabla \varphi(t, x(\,{\cdot}\,))$ is an element of the finite-dimensional space $\mathbb{R}^n$. Finally, we note separately that ci-differentiability is closely related to the non- anticipation property of functionals. Recall that a functional $\varphi \colon [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n) \to \mathbb{R}$ is called non-anticipating if for $t \in [0, T)$ and $x(\,{\cdot}\,),y(\,{\cdot}\,) \in \operatorname{C}([-h,T];\mathbb{R}^n)$ equality (3.5) implies that $\varphi(t, x(\,{\cdot}\,)) = \varphi(t, y(\,{\cdot}\,))$. It turns out that if $\varphi$ is ci-differentiable at the points $(t,x(\,{\cdot}\,)),(t,y(\,{\cdot}\,)) \in [0,T) \times \operatorname{C}([-h,T];\mathbb{R}^n)$ satisfying (3.5), then
$$
\begin{equation*}
\varphi(t, x(\,{\cdot}\,))=\varphi(t, y(\,{\cdot}\,)),\ \partial_t\varphi(t, x(\,{\cdot}\,))=\partial_t\varphi(t,y(\,{\cdot}\,)),\ \text{and}\ \nabla \varphi(t, x(\,{\cdot}\,))=\nabla\varphi(t, y(\,{\cdot}\,)).
\end{equation*}
\notag
$$
In fact, let $f \in \mathbb{R}^n$, and consider a function $y^{(f)}(\,{\cdot}\,) \in \operatorname{Lip}(t, x(\,{\cdot}\,))$ such that $y^{(f)}(\tau) = x(t) + f (\tau - t)$, $\tau \in [t, T]$. Note that $y^{(f)}(\,{\cdot}\,) \in \operatorname{Lip}(t, y(\,{\cdot}\,))$. For all $\tau \in (t, T]$ we have
$$
\begin{equation*}
\begin{gathered} \, \varphi(\tau, y^{(f)}(\,{\cdot}\,)) - \varphi(t, x(\,{\cdot}\,)) = \partial_t \varphi(t, x(\,{\cdot}\,)) (\tau - t) \\ \qquad{}+\langle \nabla \varphi(t,x(\,{\cdot}\,)),f\rangle(\tau-t)+o_1(\tau-t) \end{gathered}
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
\begin{gathered} \, \varphi(\tau, y^{(f)}(\,{\cdot}\,)) - \varphi(t, y(\,{\cdot}\,)) = \partial_t \varphi(t, y(\,{\cdot}\,)) (\tau - t) \\ \qquad{}+\langle\nabla\varphi(t,y(\,{\cdot}\,)),f\rangle(\tau-t)+o_2(\tau-t). \end{gathered}
\end{equation*}
\notag
$$
Subtracting the second equality from the first and then taking the limit as $\tau \to t^+$, we obtain $\varphi(t,x(\,{\cdot}\,))=\varphi(t,y(\,{\cdot}\,))$. Consequently,
$$
\begin{equation*}
\begin{aligned} \, &\partial_t\varphi(t,x(\,{\cdot}\,))+ \langle\nabla\varphi(t,x(\,{\cdot}\,)),f\rangle+\frac{o_1(\tau-t)}{\tau-t} \\ &\qquad=\partial_t \varphi(t, y(\,{\cdot}\,))+ \langle\nabla\varphi(t,y(\,{\cdot}\,)),f\rangle+\frac{o_2(\tau-t)}{\tau-t} \end{aligned}
\end{equation*}
\notag
$$
for all $\tau \in (t, T]$. Passing to the limit as $\tau \to t^+$ in this equality and bearing in mind that $f$ is arbitrary, we conclude that $\partial_t \varphi(t, x(\,{\cdot}\,))=\partial_t \varphi(t, y(\,{\cdot}\,))$ and $\nabla \varphi(t, x(\,{\cdot}\,)) = \nabla \varphi(t, y(\,{\cdot}\,))$. Thus, we deduce that if a functional $\varphi$ is ci-differentiable at all points $(t,x(\,{\cdot}\,)) \in [0,T) \times\operatorname{C}([-h,T];\mathbb{R}^n)$, then $\varphi$ and its ci-derivatives
$$
\begin{equation}
\partial_t\varphi \colon [0,T) \times \operatorname{C}([-h,T];\mathbb{R}^n) \to \mathbb{R} \quad\text{and}\quad \nabla \varphi \colon [0, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}^n
\end{equation}
\tag{3.7}
$$
are automatically non-anticipating. A functional $\varphi \colon [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n) \to \mathbb{R}$ is called ci-smooth if it is continuous, ci-differentiable at all points $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n)$, and its ci-derivatives (3.7) are continuous. We emphasize that the class of ci-smooth functionals is rather wide and includes, for example, functionals of the form
$$
\begin{equation*}
\varphi(t, x(\,{\cdot}\,))= a(t, x(t)) + \int_{- h}^{t} b(\tau, x(\tau)) \, {\rm d} \tau,\qquad (t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n),
\end{equation*}
\notag
$$
where the function $a \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ is continuously differentiable and the function $b \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ is continuous. The ci-derivatives of such a functional $\varphi$ at a point $(t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{C}([-h,T];\mathbb{R}^n)$ are calculated by the formulae
$$
\begin{equation*}
\partial_t \varphi(t,x(\,{\cdot}\,))=\frac{\partial a}{\partial t} (t, x(t)) + b(t, x(t))\quad\text{and}\quad \nabla \varphi(t, x(\,{\cdot}\,))= \frac{\partial a}{\partial x} (t, x(t)).
\end{equation*}
\notag
$$
On the other hand consider the functional
$$
\begin{equation}
\varphi(t, x(\,{\cdot}\,))= \|x(\cdot \wedge t)\|_\infty^2,\qquad (t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([-h,T];\mathbb{R}^n),
\end{equation}
\tag{3.8}
$$
where $a \wedge b = \min\{a, b\}$ for all $a,b \in \mathbb{R}$, and therefore
$$
\begin{equation}
x(\tau \wedge t)= \begin{cases} x(\tau) & \text{if}\ \tau \in [- h, t], \\ x(t) & \text{if}\ \tau \in (t, T]. \end{cases}
\end{equation}
\tag{3.9}
$$
We can verify directly that this functional is ci-differentiable at points $(t, x(\,{\cdot}\,)) \in [- h, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ such that $\|x(\cdot \wedge t)\|_\infty > \|x(t)\|$ or $\|x(\cdot \wedge t)\|_\infty = 0$; in this case $\partial_t \varphi(t, x(\,{\cdot}\,)) = 0$ and $\nabla \varphi(t, x(\,{\cdot}\,)) = 0$. However, the functional (3.8) is not ci-differentiable at other points. In fact, reasoning by contradiction we assume that for some point $(t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{C}([-h,T];\mathbb{R}^n)$ such that $\|x(\cdot \wedge t)\|_\infty = \|x(t)\| > 0$ there exist $\partial_t \varphi = \partial_t \varphi(t, x(\,{\cdot}\,)) \in \mathbb{R}$ and $\nabla \varphi = \nabla \varphi(t, x(\,{\cdot}\,)) \in \mathbb{R}^n$ such that
$$
\begin{equation}
\|y(\cdot \wedge \tau)\|_\infty^2 - \|x(\cdot \wedge t)\|_\infty^2= \partial_t\varphi(\tau-t)+\langle\nabla\varphi,y(\tau)-x(t)\rangle+o(\tau-t)
\end{equation}
\tag{3.10}
$$
for each $y(\,{\cdot}\,) \in \operatorname{Lip}(t, x(\,{\cdot}\,))$ and all $\tau \in (t, T]$. Choosing $y(\,{\cdot}\,) = x(\cdot \wedge t)$ in (3.10) we obtain $\partial_t\varphi=0$. Taking account of this equality and considering (3.10) for the function $y(\,{\cdot}\,) \in \operatorname{Lip}(t, x(\,{\cdot}\,))$ defined by $y(\tau) = x(t) - x(t) (\tau - t)$, $\tau \in [t, T]$, we infer the relation $\langle \nabla \varphi, x(t) \rangle = 0$. Substituting the function $y(\,{\cdot}\,) \in \operatorname{Lip}(t, x(\,{\cdot}\,))$ such that $y(\tau) = x(t) + x(t) (\tau - t)$, $\tau \in [t, T]$, into (3.10) we arrive at the equality $\|x(t)\| = 0$, which contradicts the above assumption. Coinvariant derivatives are convenient due to the fact that the total derivative of a ci-smooth functional along the motion of a time-delay system (1.20) is written in the form familiar from the standard case (in this connection see, for example, [84], Theorem 7.1.1, and [123], Lemma 2.1). More precisely, the following assertion is an analogue of the standard differentiation rule for ci-smooth functionals (see (1.5)). Proposition 1. Assume that a functional $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ is ci- smooth and a point $(t, x(\,{\cdot}\,))\,{\in}\,[0,T)\times\operatorname{C}([-h,T];\mathbb{R}^n)$, a function $y(\,{\cdot}\,)\,{\in}\,\operatorname{Lip}(t,x(\,{\cdot}\,))$, and a number $\vartheta \in (t, T)$ are fixed. Then $\mu(\tau) = \varphi(\tau, y(\,{\cdot}\,))$, $\tau \in [t, \vartheta]$, is a Lipschitz function and for a. e. $\tau \in [t, \vartheta]$. In fact, in view of the continuity of the mappings $\partial_t \varphi$ and $\nabla \varphi$, we can choose a number $M \geqslant 0$ such that $|\partial_t \varphi(\tau, y(\,{\cdot}\,))| \leqslant M$ and $\|\nabla \varphi(\tau, y(\,{\cdot}\,))\| \leqslant M$ for all $\tau \in [t, \vartheta]$. We consider the Lipschitz constant $\lambda > 0$ of the function $y(\,{\cdot}\,)$ on the interval $[t,T]$ and set $L=(1+\lambda) M$. For each $\tau \in [t, \vartheta]$ the functional $\varphi$ is ci-differentiable at the point $(\tau, y(\,{\cdot}\,))$ and $y(\,{\cdot}\,) \in \operatorname{Lip}(\tau, y(\,{\cdot}\,))$. Therefore, according to (3.6), we have
$$
\begin{equation}
\frac{\mu(\tau + \delta) - \mu(\tau)}{\delta}= \partial_t \varphi(\tau, y(\,{\cdot}\,))+ \biggl\langle \nabla \varphi(\tau, y(\,{\cdot}\,)), \frac{y(\tau + \delta) - y(\tau)}{\delta} \biggr\rangle + \frac{o(\delta)}{\delta}\,
\end{equation}
\tag{3.12}
$$
for each $\delta \in (0,T-\tau)$. Hence
$$
\begin{equation*}
\biggl|\,\limsup_{\delta \to 0^+} \frac{\mu(\tau+\delta)-\mu(\tau)}{\delta}\biggr|\leqslant L,\qquad \tau \in [t,\vartheta].
\end{equation*}
\notag
$$
Since the function $\mu(\,{\cdot}\,)$ is continuous because the functional $\varphi$ is, this inequality implies, by Dini’s theorem (see, for example, [16], Chap. 4, Theorem 1.2), that $\mu(\,{\cdot}\,)$ satisfies the Lipschitz condition with constant $L$. It remains to note that, for all $\tau \in (t, \vartheta)$ such that the derivatives $\dot{\mu}(\tau)$ and $\dot{y}(\tau)$ exist, equality (3.11) is obtained by taking the limit in (3.12) as $\delta \to 0^+$. The subject of discussion in this subsection is the Hamilton–Jacobi equation with ci-derivatives
$$
\begin{equation}
\begin{gathered} \, \partial_t \varphi(t, x(\,{\cdot}\,)) + H \bigl( t, x(\,{\cdot}\,), \nabla \varphi(t, x(\,{\cdot}\,)) \bigr)= 0, \\ (t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n), \end{gathered}
\end{equation}
\tag{3.13}
$$
and the Cauchy problem for this equation with right-end boundary condition
$$
\begin{equation}
\varphi(T, x(\,{\cdot}\,))= \sigma(x(\,{\cdot}\,)),\qquad x(\,{\cdot}\,) \in \operatorname{C}([- h, T]; \mathbb{R}^n).
\end{equation}
\tag{3.14}
$$
The non-anticipating functional $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ is unknown. The functional $H \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \times \mathbb{R}^n \to \mathbb{R}$, which is called the Hamiltonian, and the boundary functional $\sigma \colon \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ are prescribed. For each $s \in \mathbb{R}^n$ the functional
$$
\begin{equation}
[0, T] \times \operatorname{C}([-h, T]; \mathbb{R}^n) \ni (t, x(\,{\cdot}\,)) \mapsto H(t, x(\,{\cdot}\,), s) \in \mathbb{R}
\end{equation}
\tag{3.15}
$$
is assumed to be non-anticipating. To emphasize that, unlike the classical Hamilton-Jacobi equations (2.1), the sought-for quantity in (3.13) is not a function defined on the finite-dimensional space $[0, T] \times \mathbb{R}^n$ but a non-anticipating functional on $[0, T] \times \operatorname{C}([-h, T]; \mathbb{R}^n)$, equations of the form (3.13) are sometimes said to be functional or non-anticipating. In addition, equations of this type are often called hereditary, since by virtue of the non-anticipation conditions, for each $t \in [0, T]$ the Hamiltonian $H$ and the unknown functional $\varphi$ depend only on the restriction of $x(\,{\cdot}\,)$ to the interval $[-h,t]$: in applications to control problems (see § 3.2) this fact is interpreted as the dependence on the path travelled, that is, the history of motion of the system to the moment of time $t$. The term path-dependent is commonly used in this case. The Cauchy problem (3.13), (3.14) is studied under the following conditions, which are natural analogues of $(2.\mathrm{CP}.1)$–$(2.\mathrm{CP}.3)$.
Note that condition $(3.\mathrm{CP}.3)$ yields automatically the non-anticipation property of the functional (3.15) for each $s \in \mathbb{R}^n$. 3.2. Differential games for time-delay systemsWe describe how Cauchy problems of the form (3.13), (3.14) are related to differential games for time-delay systems (1.20). We consider a differential game in which the motion of the dynamical system is described by retarded-type functional differential equations of the form
$$
\begin{equation}
\begin{gathered} \, \dot{y}(\tau)= f(\tau, y(\,{\cdot}\,), u(\tau), v(\tau)), \\ \tau \in [0, T],\quad y(\tau) \in \mathbb{R}^n,\quad u(\tau) \in P,\quad v(\tau) \in Q, \end{gathered}
\end{equation}
\tag{3.16}
$$
with the initial condition (3.5) specified by a pair $(t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n)$, which plays the role of the initial position of system (3.16). By choosing the controls $u(\tau)$ the first player seeks to minimize the quality index
$$
\begin{equation}
J(t, x(\,{\cdot}\,), u(\,{\cdot}\,), v(\,{\cdot}\,))= \sigma(y(\,{\cdot}\,)) - \int_{t}^{T} \chi(\tau, y(\,{\cdot}\,), u(\tau), v(\tau)) \, {\rm d} \tau.
\end{equation}
\tag{3.17}
$$
The second player is aimed at maximizing this index by choosing the controls $v(\tau)$. The following conditions, which are natural generalizations of $(2.\mathrm{DG}.1)$–$(2.\mathrm{DG}.3)$, are assumed to hold.
Note that $(3.\mathrm{DG}.3)$ implies the non-anticipation property of the maps
$$
\begin{equation}
\begin{aligned} \, {} [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n)\ni (\tau, y(\,{\cdot}\,)) & \mapsto f(\tau, y(\,{\cdot}\,), u, v) \in \mathbb{R}^n, \\ [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n)\ni (\tau, y(\,{\cdot}\,)) & \mapsto \chi(\tau, y(\,{\cdot}\,), u, v) \in \mathbb{R} \end{aligned}
\end{equation}
\tag{3.18}
$$
for any fixed $u \in P$ and $v \in Q$. For all $(t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n)$, $u(\,{\cdot}\,) \in \mathcal{U}(t)$, and $v(\,{\cdot}\,) \in \mathcal{V}(t)$ conditions $(3.\mathrm{DG}.1)$–$(3.\mathrm{DG}.3)$ ensure the existence and uniqueness of a motion of the system (3.16) satisfying the initial condition (3.5), that is, of a function $y(\,{\cdot}\,) \in \operatorname{Lip}(t, x(\,{\cdot}\,))$ satisfying equation (3.16) with $u(\,{\cdot}\,)$ and $v(\,{\cdot}\,)$ at almost every $\tau \in [t, T]$. We consider the lower value of the differential game (3.16), (3.17):
$$
\begin{equation}
\begin{gathered} \, \rho_-(t,x(\,{\cdot}\,))=\inf_{\boldsymbol{\alpha} \in \boldsymbol{\mathcal{A}}(t)}\,\sup_{v(\,{\cdot}\,) \in \mathcal{V}(t)} J \bigl(t, x(\,{\cdot}\,), \boldsymbol{\alpha}(v(\,{\cdot}\,))(\,{\cdot}\,), v(\,{\cdot}\,) \bigr), \\ (t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T]; \mathbb{R}^n). \end{gathered}
\end{equation}
\tag{3.19}
$$
We emphasize that (3.19) defines a functional $\rho_- \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$, which is non-anticipating by construction. This functional satisfies the following equality, expressing the the dynamic programming principle in the game (3.16), (3.17):
$$
\begin{equation}
\begin{gathered} \, \rho_-(t, x(\,{\cdot}\,))= \inf_{\boldsymbol{\alpha} \in \boldsymbol{\mathcal{A}}(t)}\,\sup_{v(\,{\cdot}\,) \in \mathcal{V}(t)} \biggl( \rho_-(\vartheta, y(\,{\cdot}\,)) - \int_{t}^{\vartheta} \chi \bigl( \tau, y(\,{\cdot}\,), \boldsymbol{\alpha}(v(\,{\cdot}\,))(\tau), v(\tau) \bigr) \, {\rm d} \tau \biggr), \\ (t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n), \quad \vartheta \in (t, T]. \end{gathered}
\end{equation}
\tag{3.20}
$$
It turns out that if $\rho_-$ is ci-smooth, then it satisfies the Hamilton–Jacobi equation (3.13) with Hamiltonian
$$
\begin{equation*}
\begin{gathered} \, H_- (t, x(\,{\cdot}\,), s)= \max_{v \in Q}\,\min_{u \in P} \bigl( \langle s, f(t, x(\,{\cdot}\,), u, v) \rangle - \chi(t, x(\,{\cdot}\,), u, v) \bigr), \\ (t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n), \quad s \in \mathbb{R}^n. \end{gathered}
\end{equation*}
\notag
$$
We can verify this using formula (3.11) for the derivative of a ci-smooth functional along the motion and equality (3.20). In this case the corresponding arguments actually repeat the proof of a similar assertion in the classical situation of differential games for systems described by ordinary differential equations (see § 1). In a similar way, for the upper value functional of the game
$$
\begin{equation}
\begin{gathered} \, \rho_+(t, x(\,{\cdot}\,))= \sup_{\boldsymbol{\beta} \in \boldsymbol{\mathcal{B}}(t)}\,\inf_{u(\,{\cdot}\,) \in \mathcal{U}(t)} J \bigl(t, x(\,{\cdot}\,), u(\,{\cdot}\,), \boldsymbol{\beta}(u(\,{\cdot}\,))(\,{\cdot}\,) \bigr), \\ (t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n), \end{gathered}
\end{equation}
\tag{3.21}
$$
we obtain equation (3.13) with Hamiltonian
$$
\begin{equation*}
\begin{gathered} \, H_+(t, x(\,{\cdot}\,), s)= \min_{u \in P} \max_{v \in Q} \bigl( \langle s, f(t, x(\,{\cdot}\,), u, v) \rangle - \chi(t, x(\,{\cdot}\,), u, v) \bigr), \\ (t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n), \quad s \in \mathbb{R}^n. \end{gathered}
\end{equation*}
\notag
$$
Note that, according to (3.19) and (3.21), the functionals $\rho_-$ and $\rho_+$ satisfy the boundary condition (3.14). Assume that the saddle-point condition in a small game also holds.
Then we arrive at the following assertion concerning the value functional
$$
\begin{equation*}
\rho(t, x(\,{\cdot}\,))=\rho_-(t,x(\,{\cdot}\,))=\rho_+(t, x(\,{\cdot}\,)),\qquad (t, x(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n),
\end{equation*}
\notag
$$
of the differential game (3.16), (3.17). Theorem 1. If the value functional $\rho$ of the differential game (3.16), (3.17) under conditions $(3.\mathrm{DG}.1)$–$(3.\mathrm{DG}.4)$ is ci-smooth, then it satisfies the Hamilton–Jacobi equation (3.13) with Hamiltonian Note that under conditions $(3.\mathrm{DG}.1)$–$(3.\mathrm{DG}.4)$ the Hamiltonian (3.22) satisfies $(3.\mathrm{CP}.1)$–$(3.\mathrm{CP}.3)$. It is important that, like for ordinary differential systems, the following assertion holds, along with Theorem 1, for time-delay systems (see, for example, [115], Theorem 3.1, and [123], Theorem 3.1). Theorem 2. Assume that conditions $(3.\mathrm{DG}.1)$–$(3.\mathrm{DG}.4)$ hold and there exists a ci-smooth functional $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ satisfying the Hamilton–Jacobi equation (3.13) with Hamiltonian (3.22) and the boundary condition (3.14). Then the differential game (3.16), (3.17) has the value In view of Proposition 1 the proof of Theorem 2 actually repeats again the proof of a similar assertion in the classical case of differential games for ordinary differential systems. We explain the optimality of the strategies (3.23) using the example of the control strategy $U^\circ$ of the first player (in this connection see (2.28) and (2.34)): for any $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([-h,T];\mathbb{R}^n)$ and $\zeta>0$ there is $\delta>0$ such that, for any partition $\Delta = (\tau_j)_{j \in \{1,\dots, k + 1\}} \in \Pi_\delta(t)$ of the interval $[t, T]$ and any control $v(\,{\cdot}\,) \in \mathcal{V}(t)$ of the second player, the first player, by forming their feedback control in accordance with the step-by-step rule
$$
\begin{equation}
u(\tau)= U^\circ(\tau_j, y(\,{\cdot}\,)),\qquad \tau \in [\tau_j,\tau_{j + 1}), \quad j \in \{1,\dots, k\},
\end{equation}
\tag{3.24}
$$
ensures the inequality
$$
\begin{equation*}
J(t, x(\,{\cdot}\,), U^\circ, \Delta, v(\,{\cdot}\,))\leqslant \rho(t, x(\,{\cdot}\,)) + \zeta.
\end{equation*}
\notag
$$
Note that we can use the rule (3.24) because of the non-anticipation property of the strategy $U^\circ$: $U^\circ(\tau_j, y(\,{\cdot}\,))$ depends only on the values $y(\tau)$, $\tau \in [- h, \tau_j]$, already realized to time $\tau_j$, that is, on the history of the motion $y(\,{\cdot}\,)$ of the system before that instant. In this sense, control strategies of this form are sometimes called strategies with memory of motion history. Theorem 2 gives a solution of the differential game (3.16), (3.17) in the case when the Cauchy problem (3.13), (3.14) with Hamiltonian (3.22) has a ci-smooth solution. However, like in the case of classical Hamilton–Jacobi equations, this problem can have no ci-smooth solutions, and we thus need to consider appropriate generalized solutions. Below, according to the scheme in § 2, we discuss the results deduced in the framework of the minimax and viscosity approaches to generalized solutions of the Cauchy problem (3.13), (3.14). 3.3. Definition of a minimax solutionConsider the multivalued mapping
$$
\begin{equation*}
[0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \times \mathbb{R}^n \ni (\tau, y(\,{\cdot}\,), s) \mapsto E_0(\tau, y(\,{\cdot}\,), s) \subset \mathbb{R}^n \times \mathbb{R}
\end{equation*}
\notag
$$
that is defined for all $\tau \in [0, T]$, $y(\,{\cdot}\,) \in \operatorname{C}([- h, T]; \mathbb{R}^n)$, and $s \in \mathbb{R}^n$ by
$$
\begin{equation}
\begin{aligned} \, \nonumber E_0(\tau, y(\,{\cdot}\,), s) & = \bigl\{(f, \chi) \in \mathbb{R}^n \times \mathbb{R} \colon \\ & \qquad \|f\| \leqslant c (1 + \|y(\cdot \wedge \tau)\|_\infty), \, \chi = \langle s, f \rangle - H(\tau, y(\,{\cdot}\,), s) \bigr\}, \end{aligned}
\end{equation}
\tag{3.25}
$$
where the number $c$ is from $(3.\mathrm{CP}.2)$. Note that, for each $s \in \mathbb{R}^n$, the mapping
$$
\begin{equation*}
[0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n) \ni (\tau,y(\,{\cdot}\,)) \mapsto E_0(\tau,y(\,{\cdot}\,),s) \subset \mathbb{R}^n \times \mathbb{R}
\end{equation*}
\notag
$$
is non-anticipating. For $(t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ and $s \in \mathbb{R}^n$ we consider the set $\operatorname{Sol}(t, x(\,{\cdot}\,), E_0, s)$ of solutions $(y(\,{\cdot}\,), z(\,{\cdot}\,))$ of the retarded-type functional differential inclusion
$$
\begin{equation}
(\dot{y}(\tau), \dot{z}(\tau))\in E_0(\tau, y(\,{\cdot}\,), s),\qquad \tau \in [t, T], \quad (y(\tau), z(\tau)) \in \mathbb{R}^n \times \mathbb{R},
\end{equation}
\tag{3.26}
$$
that satisfy the initial condition
$$
\begin{equation}
y(\tau)= x(\tau),\quad z(\tau)= 0,\qquad \tau \in [- h, t].
\end{equation}
\tag{3.27}
$$
We let $\Phi_{\operatorname{C}}$, $\Phi_{\operatorname{LSC}}$, and $\Phi_{\operatorname{USC}}$ denote the sets of non-anticipating functionals $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n)\to \mathbb{R}$ that are continuous, lower semicontinuous, and upper semicontinuous, respectively. Definition 8. An upper minimax solution of the Cauchy problem (3.13), (3.14) is a functional $\varphi \in \Phi_{\operatorname{LSC}}$ that satisfies the boundary condition and has the following property.
Definition 9. A lower minimax solution of the Cauchy problem (3.13), (3.14) is a functional $\varphi \in \Phi_{\operatorname{USC}}$ that satisfies the boundary condition and has the following property.
Definition 10. A minimax solution of the Cauchy problem (3.13), (3.14) is a functional $\varphi \in \Phi_{\operatorname{C}}$ that is both an upper and a lower minimax solution of this problem. Note that properties $(3.\mathrm{MS}.1_+)$ and $(3.\mathrm{MS}.1_-)$ express the $u$- and $v$-stability of the value functional of the differential game (3.16), (3.17) in the unified way (see, for example, [121], Propositions 1 and 2). These properties appeared in the works [98], [137], [102], [138], and [131] (also see [100], § 11), devoted to the development of the theory of positional differential games for time-delay systems. We set
$$
\begin{equation}
\begin{aligned} \, \nonumber Y(t, x(\,{\cdot}\,))& = \bigl\{ y(\,{\cdot}\,) \in \operatorname{Lip}(t, x(\,{\cdot}\,)) \colon \\ & \qquad \|\dot{y}(\tau)\| \leqslant c(1 + \|y(\cdot \wedge \tau)\|_\infty) \text{ for a. e. } \tau \in [t, T] \bigr\}, \end{aligned}
\end{equation}
\tag{3.32}
$$
where $(t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n)$ and $c$ is from $(3.\mathrm{CP}.2)$. For a functional $\varphi \colon [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n) \to \mathbb{R}$ we consider the following condition.
Note that condition $(3.\mathrm{MS})$ can be reformulated as follows: for all $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([-h, T]; \mathbb{R}^n)$ and $s\,{\in}\,\mathbb{R}^n$ there exists a pair $(y(\,{\cdot}\,), z(\,{\cdot}\,)) \in \operatorname{Sol}(t, x(\,{\cdot}\,), E_0, s)$ such that
$$
\begin{equation*}
\varphi(\vartheta,y(\,{\cdot}\,))-z(\vartheta)=\varphi(t,x(\,{\cdot}\,)),\qquad \vartheta \in [t, T].
\end{equation*}
\notag
$$
The following assertion holds true ([114], Proposition 6.1; also see [123], Proposition 5.2). Proposition 2. Assume that conditions $(3.\mathrm{CP}.1)$ and $(3.\mathrm{CP}.2)$ hold and functional (3.15) is non-anticipating for each $s \in \mathbb{R}^n$. Then a functional $\varphi \in \Phi_{\operatorname{C}}$ satisfies conditions $(3.\mathrm{MS}.1_-)$ and $(3.\mathrm{MS}.1_+)$ if and only if it satisfies $(3.\mathrm{MS})$. In particular, it follows from Proposition 2 that a minimax solution of the Cauchy problem (3.13), (3.14) can be defined as a non-anticipating continuous functional $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ satisfying conditions (3.14) and $(3.\mathrm{MS})$. 3.4. Well-posedness of the minimax solutionLike in the case of the classical Hamilton–Jacobi equations, the verification of the existence and uniqueness of a minimax solution of the Cauchy problem (3.13), (3.14) consists of two parts. The first part is proving the existence of an upper and a lower minimax solution $\varphi_+^\circ$ and $\varphi_-^\circ$ of the Cauchy problem (3.13), (3.14) such that
$$
\begin{equation*}
\varphi_+^\circ (t, x(\,{\cdot}\,))\leqslant \varphi_-^\circ (t, x(\,{\cdot}\,)),\qquad (t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n).
\end{equation*}
\notag
$$
This proof (see [114], Lemmas 7.1–7.5, and also [123], § 6.1) actually repeats the reasoning ([162], Theorem 8.2) in the classical case of the Cauchy problem (2.1), (2.2) except for some technical details related, among other things, to the properties of the sets of solutions of retarded-type functional differential inclusions. The second part consists in verifying the comparison principle: for any upper and lower minimax solutions $\varphi_+$ and $\varphi_-$ of the Cauchy problem (3.13), (3.14) we have
$$
\begin{equation}
\varphi_+ (t, x(\,{\cdot}\,))\geqslant \varphi_-(t, x(\,{\cdot}\,)),\qquad (t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n).
\end{equation}
\tag{3.33}
$$
Here it becomes necessary to construct an appropriate ci-smooth Lyapunov–Krasovskii functional (an analogue of the Lyapunov function (2.12)). In [114] (also see [123], § 6) the Cauchy problem (3.13), (3.14) was studied under conditions $(3.\mathrm{CP}.1)$, $(3.\mathrm{CP}.2)$, and the following condition, which is stronger than $(3.\mathrm{CP}.3)$.
In the context of applications to control problems for time-delay systems condition $(3.\mathrm{CP}.4)$ makes it possible to cover only the case of distributed delays (see (1.23)). It has been shown that under conditions $(3.\mathrm{CP}.1)$, $(3.\mathrm{CP}.2)$, and $(3.\mathrm{CP}.4)$ an appropriate Lyapunov–Krasovskii functional is
$$
\begin{equation}
\nu_\varepsilon(\tau, w(\,{\cdot}\,))= \frac{e^{- 2 \lambda (\tau - t)} - \varepsilon}{\varepsilon}\biggl( \varepsilon^4 + \|w(\tau)\|^2+ 2 \lambda \int_{- h}^{\tau} \|w(\xi)\|^2 \, {\rm d} \xi \biggr)^{1/2},
\end{equation}
\tag{3.34}
$$
where $(\tau, w(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n)$, $\varepsilon > 0$ is a small parameter, $\lambda$ is chosen for the set $Y(t, x(\,{\cdot}\,))$ in accordance with $(3.\mathrm{CP}.4)$ (see (3.32)), and $(t, x(\,{\cdot}\,))$ is the point at which inequality (3.33) is verified. In [121] conditions $(3.\mathrm{CP}.1)$ and $(3.\mathrm{CP}.2)$ and the following condition, which is weaker than $(3.\mathrm{CP}.4)$ but, as before, stronger than $(3.\mathrm{CP}.3)$, were considered.
In applications to control problems condition $(3.\mathrm{CP}.5)$ makes it possible to deal with distributed and constant concentrated delays alike (see (1.21) and (1.23)); however, it does not cover, for example, the case of variable delay (see (1.22)). Under conditions $(3.\mathrm{CP}.1)$, $(3.\mathrm{CP}.2)$, and $(3.\mathrm{CP}.5)$ we can prove the comparison principle by using the Lyapunov–Krasovskii functional
$$
\begin{equation}
\begin{aligned} \, \nonumber \nu_\varepsilon(\tau, w(\,{\cdot}\,))& = \frac{e^{- (\mathcal{J} + 3) \lambda (\tau + h)}}{\varepsilon}\biggl( \varepsilon^4 + \|w(\tau)\|^2+ 2 \lambda \sum_{j = 1}^\mathcal{J} \int_{\tau - \vartheta_j}^{\tau} \|w(\xi)\|^2 \, {\rm d} \xi \biggr)^{1/2} \\ & \qquad + \frac{e^{- (\mathcal{J} + 3) \lambda (\tau + h)} - \varepsilon}{\varepsilon}\biggl( \varepsilon^4 + \|w(\tau)\|^2+ 2 \lambda \int_{- h}^{\tau} \|w(\xi)\|^2 \, {\rm d} \xi \biggr)^{1/2}. \end{aligned}
\end{equation}
\tag{3.35}
$$
Here $\lambda$ is chosen for the set $Y(t, x(\,{\cdot}\,))$ in accordance with $(3.\mathrm{CP}.5)$. As we can see, the constructions of the Lyapunov–Krasovskii functionals (3.34) and (3.35) are closely related to the corresponding conditions $(3.\mathrm{CP}.4)$ and $(3.\mathrm{CP}.5)$ for the Lipschitz property of the Hamiltonian $H$ in the variable $x(\,{\cdot}\,)$. Therefore, to prove the comparison principle under the most general condition $(3.\mathrm{CP}.3)$, which involves the uniform norm, it is natural to consider Lyapunov–Krasovskii functionals based on the functional (3.8). However, the implementation of this idea encounters significant difficulties caused primarily by the poor ci-differentiability properties of the latter (in particular, this functional is not ci-smooth). Generally speaking, we can abstract away from the specific form of one or another condition for the Lipschitz property of the Hamiltonian $H$ in the variable $x(\,{\cdot}\,)$ and formulate requirements for a Lyapunov–Krasovskii functional that make it possible to prove the comparison principle using this functional in accordance with the scheme in [114], Lemma 7.7, (also see [123], Lemma 6.7). Note that these requirements are close to conditions $(H.4)^\prime$ in [26] and $(A.4)$ in [162], § 9.2, in the case of the classical Hamilton–Jacobi equations. Lemma 1. Assume that conditions $(3.\mathrm{CP}.1)$ and $(3.\mathrm{CP}.2)$ hold and the functional (3.15) is non-anticipating for all $s \in \mathbb{R}^n$. Assume that for each point $(t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{C}([-h,T];\mathbb{R}^n)$ there is $\varepsilon_0 > 0$ and a Lyapunov–Krasovskii functional $\nu_\varepsilon \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ depending on the parameter $\varepsilon \in (0, \varepsilon_0]$ such that the following conditions are satisfied:
Then all upper and lower minimax solutions $\varphi_+$ and $\varphi_-$ of the Cauchy problem (3.13), (3.14) satisfy inequality (3.33). In conditions $(3.\mathrm{c})$ and $(3.\mathrm{d})$ the set $Y(t, x(\,{\cdot}\,))$ is defined by (3.32). Note that the Lyapunov–Krasovskii functionals (3.34) and (3.35) satisfy conditions $(3.\mathrm{a})$–$(3.\mathrm{d})$ if the Lipschitz conditions $(3.\mathrm{CP}.4)$ and $(3.\mathrm{CP}.5)$ hold, respectively. If $(3.\mathrm{CP}.3)$ holds, then they can fail the key condition $(3.\mathrm{d})$. We give a proof of Lemma 1. Let $(t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n)$. If $t = T$, then inequality (3.33) holds true by the boundary conditions (3.28) for $\varphi_+$ and (3.30) for $\varphi_-$; therefore, we assume in what follows that $t < T$. Let $\varepsilon \in (0, \varepsilon_0]$. For any $\tau \in [t, T]$ and $y_1(\,{\cdot}\,),y_2(\,{\cdot}\,) \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ we consider the set $F_\varepsilon(\tau, y_1(\,{\cdot}\,), y_2(\,{\cdot}\,))$ of triples $(f_1, f_2, \chi) \in \mathbb{R}^n \times \mathbb{R}^n \times \mathbb{R}$ such that
$$
\begin{equation}
\|f_1\|\leqslant c (1 + \|y_1(\cdot \wedge \tau)\|_\infty),\quad \|f_2\|\leqslant c (1 + \|y_2(\cdot \wedge \tau)\|_\infty),
\end{equation}
\tag{3.36}
$$
and
$$
\begin{equation}
\begin{aligned} \, \nonumber &\bigl|\chi-\langle\nabla\nu_\varepsilon(\tau,w(\,{\cdot}\,)),f_1-f_2\rangle \\ & \quad + H \bigl( \tau, y_1(\,{\cdot}\,), \nabla \nu_\varepsilon(\tau, w(\,{\cdot}\,)) \bigr)- H\bigl(\tau,y_2(\,{\cdot}\,), \nabla\nu_\varepsilon(\tau,w(\,{\cdot}\,))\bigr)\bigr|\leqslant \varepsilon, \end{aligned}
\end{equation}
\tag{3.37}
$$
where $c$ is borrowed from condition $(3.\mathrm{CP}.2)$ and $w(\,{\cdot}\,) = y_2(\,{\cdot}\,) - y_1(\,{\cdot}\,)$. Note that the multivalued mapping $(\tau, y_1(\,{\cdot}\,), y_2(\,{\cdot}\,)) \mapsto F_\varepsilon(\tau, y_1(\,{\cdot}\,), y_2(\,{\cdot}\,))$ is non-anticipating. We let $\operatorname{Sol}_\varepsilon$ denote the set of solutions $(y_1(\,{\cdot}\,), y_2(\,{\cdot}\,), z(\,{\cdot}\,))$ of the differential inclusion
$$
\begin{equation*}
\begin{gathered} \, (\dot{y}_1(\tau), \dot{y}_2(\tau), \dot{z}(\tau))\in F_\varepsilon(\tau, y_1(\,{\cdot}\,), y_2(\,{\cdot}\,)), \\ \tau \in [t, T],\quad (y_1(\tau), y_2(\tau), z(\tau)) \in \mathbb{R}^n \times \mathbb{R}^n \times \mathbb{R}, \end{gathered}
\end{equation*}
\notag
$$
that satisfy the initial conditions
$$
\begin{equation}
y_1(\tau)= y_2(\tau)= x(\tau)\quad\text{and}\quad z(\tau)= \varphi_+(t, x(\,{\cdot}\,)) - \varphi_-(t, x(\,{\cdot}\,)),\qquad \tau \in [- h, t].
\end{equation}
\tag{3.38}
$$
This set is non-empty and compact in $\operatorname{C}([- h, T]; \mathbb{R}^n \times \mathbb{R}^n \times \mathbb{R})$. To prove the lemma it suffices to establish that for all $\varepsilon \in (0, \varepsilon_0]$ there exists a triple of functions $(y_1^\varepsilon(\,{\cdot}\,), y_2^\varepsilon(\,{\cdot}\,), z^\varepsilon(\,{\cdot}\,)) \in \operatorname{Sol}_\varepsilon$ such that
$$
\begin{equation}
z^\varepsilon(T)\geqslant \varphi_+(T, y_1^\varepsilon(\,{\cdot}\,)) - \varphi_-(T, y_2^\varepsilon(\,{\cdot}\,)).
\end{equation}
\tag{3.39}
$$
In fact, the inclusion $(y_1^\varepsilon(\,{\cdot}\,), y_2^\varepsilon(\,{\cdot}\,), z^\varepsilon(\,{\cdot}\,)) \in \operatorname{Sol}_\varepsilon$ yields the inequality
$$
\begin{equation}
\begin{aligned} \, \nonumber \dot{z}^\varepsilon(\tau)& \leqslant -\langle \nabla \nu_\varepsilon(\tau, w^\varepsilon(\,{\cdot}\,)), \dot{w}^\varepsilon(\tau) \rangle+ H \bigl(\tau,y_2^\varepsilon(\,{\cdot}\,), \nabla \nu_\varepsilon(\tau, w^\varepsilon(\,{\cdot}\,)) \bigr) \\ & \qquad -H\bigl(\tau, y_1^\varepsilon(\,{\cdot}\,), \nabla\nu_\varepsilon(\tau,w^\varepsilon(\,{\cdot}\,))\bigr)+\varepsilon \end{aligned}
\end{equation}
\tag{3.40}
$$
for almost every $\tau \in [t, T]$. Here $w^\varepsilon(\,{\cdot}\,) = y_2^\varepsilon(\,{\cdot}\,) - y_1^\varepsilon(\,{\cdot}\,)$. We consider the function $\mu_\varepsilon(\tau) = \nu_\varepsilon(\tau, w^\varepsilon(\,{\cdot}\,)) + z^\varepsilon(\tau) - \varepsilon (\tau - t)$, $\tau \in [t, T]$. The functional $\nu_\varepsilon$ is ci-smooth. Thus, the function $\mu_\varepsilon(\,{\cdot}\,)$ is continuous, and by Proposition 1 it satisfies the Lipschitz condition on each interval $[t, \vartheta]$, where $\vartheta \in (t, T)$. In addition, by (3.40) and $(3.\mathrm{d})$ we have
$$
\begin{equation*}
\dot{\mu}_\varepsilon(\tau)= \partial_t \nu_\varepsilon(\tau, w^\varepsilon(\,{\cdot}\,))+ \langle \nabla \nu_\varepsilon(t, w^\varepsilon(\,{\cdot}\,)), \dot{w}^\varepsilon(\tau) \rangle + \dot{z}^\varepsilon(\tau) - \varepsilon \leqslant 0
\end{equation*}
\notag
$$
for almost every $\tau \in [t,T]$. We have $\mu_\varepsilon(t) \geqslant \mu_\varepsilon(T)$; in view of the initial condition (3.38) and $(3.\mathrm{b})$ it follows that
$$
\begin{equation*}
\nu_\varepsilon(T,w^\varepsilon(\,{\cdot}\,))+z^\varepsilon(T)-\varepsilon(T-t) \leqslant \varepsilon+\varphi_+(t,x(\,{\cdot}\,))-\varphi_-(t,x(\,{\cdot}\,)).
\end{equation*}
\notag
$$
The functionals $\varphi_+$ and $\varphi_-$ satisfy the boundary conditions (3.28) and (3.30), respectively; hence $z^\varepsilon(T) \geqslant \sigma(y_1^\varepsilon(\,{\cdot}\,)) - \sigma(y_2^\varepsilon(\,{\cdot}\,))$ according to (3.39). As a result, we obtain the estimate
$$
\begin{equation}
\nu_\varepsilon(T, w^\varepsilon(\,{\cdot}\,)) + \sigma(y_1^\varepsilon(\,{\cdot}\,)) - \sigma(y_2^\varepsilon(\,{\cdot}\,)) \leqslant \varepsilon (1 + T - t) + \varphi_+(t, x(\,{\cdot}\,)) - \varphi_-(t, x(\,{\cdot}\,))
\end{equation}
\tag{3.41}
$$
for all $\varepsilon \in (0,\varepsilon_0]$. The set $Y(t, x(\,{\cdot}\,))$ is compact, and the boundary functional $\sigma$ is continuous. Therefore, there is $K > 0$ such that $|\sigma(y(\,{\cdot}\,))| \leqslant K$ for $y(\,{\cdot}\,) \in Y(t, x(\,{\cdot}\,))$. Thus, we derive from (3.41) that
$$
\begin{equation*}
\nu_\varepsilon(T,y_2^\varepsilon(\,{\cdot}\,)-y_1^\varepsilon(\,{\cdot}\,)) \leqslant 2K+\varepsilon_0(1+T-t)+\varphi_+(t,x(\,{\cdot}\,))- \varphi_-(t, x(\,{\cdot}\,)),\qquad \varepsilon \in (0, \varepsilon_0].
\end{equation*}
\notag
$$
Hence $\|y_2^\varepsilon(\,{\cdot}\,) - y_1^\varepsilon(\,{\cdot}\,)\|_\infty \to 0$ as $\varepsilon \to 0^+$ by condition $(3.\mathrm{c})$. Finally, estimate (3.41), since the functional $\nu_\varepsilon$ is non-negative, implies the inequality
$$
\begin{equation*}
\sigma(y_1^\varepsilon(\,{\cdot}\,)) - \sigma(y_2^\varepsilon(\,{\cdot}\,)) \leqslant \varepsilon (1 + T - t) + \varphi_+(t, x(\,{\cdot}\,)) - \varphi_-(t, x(\,{\cdot}\,)),\qquad \varepsilon \in (0, \varepsilon_0].
\end{equation*}
\notag
$$
Passing to the limit as $\varepsilon \to 0^+$ we arrive at the required inequality (3.33). Thus, it remains to show that for each $\varepsilon \in (0,\varepsilon_0]$ there exists a triple of functions $(y_1^\varepsilon(\,{\cdot}\,), y_2^\varepsilon(\,{\cdot}\,), z^\varepsilon(\,{\cdot}\,)) \in \operatorname{Sol}_\varepsilon$ satisfying (3.39). For $\tau \in [t,T]$ we consider the set
$$
\begin{equation*}
M_\varepsilon(\tau)= \bigl\{ (y_1(\,{\cdot}\,), y_2(\,{\cdot}\,), z(\,{\cdot}\,)) \in \operatorname{Sol}_\varepsilon \colon z(\tau) \geqslant \varphi_+(\tau, y_1(\,{\cdot}\,)) - \varphi_-(\tau, y_2(\,{\cdot}\,)) \bigr\}.
\end{equation*}
\notag
$$
Note that $M_\varepsilon(t) \ne \varnothing$ by the initial condition (3.38). We set
$$
\begin{equation}
\tau_\varepsilon= \max \bigl\{ \tau \in [t, T] \colon M_\varepsilon(\tau) \ne \varnothing \bigr\}.
\end{equation}
\tag{3.42}
$$
The maximum in (3.42) is attained, since $\operatorname{Sol}_\varepsilon$ is compact and the functionals $\varphi_+$ and $\varphi_-$ are lower and upper semicontinuous, respectively. We have to prove the equality $\tau_\varepsilon = T$. Reasoning by contradiction we assume that $\tau_\varepsilon < T$. Fix
$$
\begin{equation}
(y_1(\,{\cdot}\,), y_2(\,{\cdot}\,), z(\,{\cdot}\,)) \in M_\varepsilon(\tau_\varepsilon).
\end{equation}
\tag{3.43}
$$
We introduce the notation $s=\nabla\nu_\varepsilon(\tau_\varepsilon,y_1(\,{\cdot}\,)- y_2(\,{\cdot}\,))$. The functional $\varphi_+$ is non-anticipating and lower semicontinuous. Therefore (see, for example, [123], Proposition 5.1), owing to $(3.\mathrm{MS}.1_+)$ there exists a pair $(y_+(\,{\cdot}\,), z_+(\,{\cdot}\,)) \in \operatorname{Sol}(\tau_\varepsilon, y_1(\,{\cdot}\,), E_0, s)$ such that
$$
\begin{equation}
\varphi_+(\tau, y_+(\,{\cdot}\,)) - z_+(\tau)\leqslant \varphi_+(\tau_\varepsilon, y_1(\,{\cdot}\,)),\qquad \tau \in [\tau_\varepsilon, T].
\end{equation}
\tag{3.44}
$$
In a similar way, for the functional $\varphi_-$, owing to $(3.\mathrm{MS}.1_-)$, there exists a pair $(y_-(\,{\cdot}\,), z_-(\,{\cdot}\,)) \in \operatorname{Sol}(\tau_\varepsilon,y_2(\,{\cdot}\,),E_0,s)$ such that
$$
\begin{equation}
\varphi_-(\tau, y_-(\,{\cdot}\,)) - z_-(\tau)\geqslant \varphi_-(\tau_\varepsilon,y_2(\,{\cdot}\,)),\qquad \tau \in [\tau_\varepsilon,T].
\end{equation}
\tag{3.45}
$$
Taking account of the definitions (3.25) and (3.36), (3.37) of the multivalued mappings $E_0$ and $F_\varepsilon$, from the continuity of the Hamiltonian $H$ and the ci-gradient $\nabla \nu_\varepsilon$, we conclude that there exists $\delta \in (0, T - \tau_\varepsilon)$ such that
$$
\begin{equation}
(\dot{y}_+(\tau), \dot{y}_-(\tau), \dot{z}_+(\tau) - \dot{z}_-(\tau)) \in F_\varepsilon(\tau, y_+(\,{\cdot}\,), y_-(\,{\cdot}\,))
\end{equation}
\tag{3.46}
$$
for almost every $\tau \in [\tau_\varepsilon,\tau_\varepsilon + \delta]$. We consider the functions $y^\ast_\pm (\,{\cdot}\,) = y_\pm(\cdot \wedge \tau_\varepsilon + \delta)$ and
$$
\begin{equation*}
z^\ast(\tau)= \begin{cases} z(\tau) & \text{for}\ \tau \in [- h, \tau_\varepsilon], \\ z(\tau_\varepsilon) + z_+(\tau) - z_-(\tau) & \text{for}\ \tau \in (\tau_\varepsilon, \tau_\varepsilon + \delta], \\ z(\tau_\varepsilon) + z_+(\tau_\varepsilon + \delta) - z_-(\tau_\varepsilon + \delta)+ \displaystyle\int_{\tau_\varepsilon+ \delta}^{\tau} H^\ast(\xi) \, {\rm d} \xi & \text{for}\ \tau \in (\tau_\varepsilon + \delta, T], \end{cases}
\end{equation*}
\notag
$$
where
$$
\begin{equation*}
H^\ast(\xi)= H\bigl(\xi,y^\ast_-(\xi),\nabla\nu_\varepsilon(\xi,w^\ast(\,{\cdot}\,))\bigr) -H \bigl(\xi,y^\ast_+(\xi),\nabla\nu_\varepsilon(\xi,w^\ast(\,{\cdot}\,))\bigr)
\end{equation*}
\notag
$$
and $w^\ast(\,{\cdot}\,) = y^\ast_-(\,{\cdot}\,) - y^\ast_+(\,{\cdot}\,)$. According to (3.43) and (3.46), we have $(y^\ast_+(\,{\cdot}\,), y^\ast_-(\,{\cdot}\,), z^\ast(\,{\cdot}\,)) \in \operatorname{Sol}_\varepsilon$. Thus, in view of the non-anticipation property of the functionals $\varphi_+$ and $\varphi_-$ it follows from (3.43)–(3.45) that
$$
\begin{equation*}
\begin{aligned} \, z^\ast(\tau_\varepsilon + \delta)& \geqslant \varphi_+(\tau_\varepsilon, y_1(\,{\cdot}\,)) - \varphi_-(\tau_\varepsilon, y_2(\,{\cdot}\,)) + z_+(\tau_\varepsilon + \delta) - z_-(\tau_\varepsilon + \delta) \\ & \geqslant \varphi_+(\tau_\varepsilon + \delta, y_+(\,{\cdot}\,)) - \varphi_-(\tau_\varepsilon + \delta, y_-(\,{\cdot}\,)) \\ & = \varphi_+(\tau_\varepsilon + \delta, y^\ast_+(\,{\cdot}\,)) - \varphi_-(\tau_\varepsilon + \delta, y^\ast_-(\,{\cdot}\,)); \end{aligned}
\end{equation*}
\notag
$$
hence we have the inclusion $(y^\ast_+(\,{\cdot}\,), y^\ast_-(\,{\cdot}\,), z^\ast(\,{\cdot}\,)) \in M_\varepsilon(\tau_\varepsilon + \delta)$, which contradicts the definition (3.42) of $\tau_\varepsilon$. The proof of Lemma 1 is complete. Thus, to obtain the comparison principle (3.33) under $(3.\mathrm{CP}.1)$–$(3.\mathrm{CP}.3)$, it suffices to find a Lyapunov–Krasovskii functional satisfying conditions $(3.\mathrm{a})$–$(3.\mathrm{d})$ in Lemma 1 under these assumptions. We present a construction of such a functional. Following [180], we consider the functional
$$
\begin{equation}
\begin{gathered} \, \gamma (\tau, w(\,{\cdot}\,)) = \begin{cases} \displaystyle \frac{\bigl( \|w(\cdot \wedge \tau)\|_\infty^2 - \|w(\tau)\|^2 \bigr)^2}{\|w(\cdot \wedge \tau)\|_\infty^2} + \|w(\tau)\|^2 & \text{if}\ \|w(\cdot \wedge \tau)\|_\infty > 0, \\ 0 & \text{if}\ \|w(\cdot \wedge \tau)\|_\infty = 0, \end{cases} \\ (\tau,w(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n). \end{gathered}
\end{equation}
\tag{3.47}
$$
By Lemma 2.3 in [180] we have
$$
\begin{equation}
\begin{gathered} \, \varkappa\|w(\cdot \wedge\tau)\|_\infty^2\leqslant \gamma(\tau,w(\,{\cdot}\,)) \leqslant 2 \|w(\cdot \wedge \tau)\|_\infty^2, \\ (\tau,w(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n), \end{gathered}
\end{equation}
\tag{3.48}
$$
where $\varkappa = (3 - \sqrt{5}\,) / 2$. In addition (also see [67], Appendix B), the functional $\gamma$ is ci-smooth; in this case $\partial_t \gamma(\tau, w(\,{\cdot}\,)) = 0$ and
$$
\begin{equation*}
\nabla \gamma(\tau, w(\,{\cdot}\,))= \begin{cases} \biggl(2 - 4\,\dfrac{\|w(\cdot \wedge \tau)\|_\infty^2 - \|w(\tau)\|^2}{\|w(\cdot \wedge \tau)\|_\infty^2} \biggr) w(\tau) & \text{if}\ \|w(\cdot \wedge \tau)\|_\infty > 0, \\ 0 & \text{if}\ \|w(\cdot \wedge \tau)\|_\infty = 0 \end{cases}
\end{equation*}
\notag
$$
for all $(\tau, w(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n)$. We also note the estimate
$$
\begin{equation*}
\|\nabla \gamma(\tau, w(\,{\cdot}\,))\|\leqslant 2 \|w(\tau)\|,\qquad (\tau,w(\,{\cdot}\,)) \in [0,T) \times \operatorname{C}([-h,T];\mathbb{R}^n).
\end{equation*}
\notag
$$
The functional $\gamma$ is a regularization of the functional $\varphi$ in (3.8): the functionals $\gamma$ and $\varphi$ are equivalent in the sense of inequalities (3.48); however, unlike $\varphi$, the functional $\gamma$ is ci-smooth. Let $(t,x(\,{\cdot}\,))\in[0,T)\times\operatorname{C}([-h,T];\mathbb{R}^n)$. For the set $Y(t,x(\,{\cdot}\,))$ (see (3.32)) we choose $\lambda$ in accordance with condition $(3.\mathrm{CP}.3)$, borrow $\varkappa$ from (3.48), set $\varepsilon_0 = e^{- \lambda T / \varkappa} / \sqrt{\varkappa}$, and introduce the Lyapunov–Krasovskii functional
$$
\begin{equation}
\begin{gathered} \, \nu_\varepsilon(\tau, w(\,{\cdot}\,))= \frac{e^{- \lambda \tau / \varkappa} - \varepsilon \sqrt{\varkappa}}{\varepsilon} \sqrt{\varepsilon^4 + \gamma(\tau, w(\,{\cdot}\,))}\,, \\ (\tau,w(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n), \end{gathered}
\end{equation}
\tag{3.49}
$$
for each $\varepsilon \in (0, \varepsilon_0]$. On the basis of the above properties of the functional $\gamma$ we can verify directly (see [67], § 4.1) that the functional $\nu_\varepsilon$ satisfies conditions $(3.\mathrm{a})$–$(3.\mathrm{d})$ in Lemma 1. We thus deduce the following result (see [67], Theorem 1). Theorem 3. Assume that conditions $(3.\mathrm{CP}.1)$–$(3.\mathrm{CP}.3)$ hold. Then a minimax solution of the Cauchy problem (3.13), (3.14) exists and is unique. We also note that, in connection with the existence and uniqueness problems for minimax solutions, other conditions specifying the Lipschitz properties of the Hamiltonian $H$ with respect to the variable $x(\,{\cdot}\,)$ can be considered. For example, a Lipschitz property of a slightly different form from $(3.\mathrm{CP}.3)$–$(3.\mathrm{CP}.5)$ was treated in [10]. Regarding the Cauchy problem (3.13), (3.14), we can formulate this condition as follows.
$$
\begin{equation*}
H(t, x(\,{\cdot}\,), s)= \langle s, x(t - h) \rangle,\quad H(t,x(\,{\cdot}\,),s)=\biggl\langle s,x\biggl(\frac{t}{2}\biggr)\biggr\rangle,
\end{equation*}
\notag
$$
or
$$
\begin{equation*}
H(t, x(\,{\cdot}\,), s)= \biggl\langle s,\int_{-h}^{t}x(\tau)\, {\rm d}\tau\biggl\rangle
\end{equation*}
\notag
$$
satisfy $(3.\mathrm{CP}.3)$ but do not satisfy $(3.\mathrm{CP}.6)$; thus, conditions $(3.\mathrm{CP}.3)$ do not imply $(3.\mathrm{CP}.6)$. Theorem 3 also eliminates certain ‘inconvenience’ in Definition 10 (of a minimax solution) related to the fact that $c$ (see the definition (3.25) of the multivalued mapping $E_0$) can be chosen on the basis of $(3.\mathrm{CP}.2)$ in an ambiguous way. Nevertheless, under conditions $(3.\mathrm{CP}.1)$–$(3.\mathrm{CP}.3)$ the minimax solution of the Cauchy problem (3.13), (3.14) is independent of this choice. In fact, fix numbers $c_1 > c_2 > 0$ such that condition $(3.\mathrm{CP}.2)$ holds for each of them. It is clear that a minimax solution corresponding to the smaller number $c_2$ is a minimax solution corresponding to the larger number $c_1$. Therefore, by Theorem 3 the minimax solutions corresponding to $c_1$ and $c_2$ must coincide. Starting from Theorem 3 and the comparison principle and reasoning, for example, in accordance with the scheme in Theorem 2 in [97] (also see [123], Theorem 9.1), we can establish the following property of the continuous dependence of the minimax solution $\varphi$ of the Cauchy problem (3.13), (3.14) on variations of the Hamiltonian $H$ and the boundary functional $\sigma$. Assume that for each $k \in \mathbb{N} \cup \{0\}$ we have
$$
\begin{equation*}
H_k \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \times \mathbb{R}^n \to \mathbb{R}\quad\text{and}\quad \sigma_k \colon\operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}
\end{equation*}
\notag
$$
satisfying conditions $(3.\mathrm{CP}.1)$–$(3.\mathrm{CP}.3)$, where the constant $c$ in $(3.\mathrm{CP}.2)$ is independent of $k$. Assume that for each compact set $W \subset \operatorname{C}([- h, T]; \mathbb{R}^n)$ and each $s \in \mathbb{R}^n$ the following convergence holds uniformly with respect to $t \in [0, T]$ and $x(\,{\cdot}\,) \in W$:
$$
\begin{equation*}
|H_k(t,x(\,{\cdot}\,),s)-H_0(t,x(\,{\cdot}\,),s)|+|\sigma_k(x(\,{\cdot}\,))- \sigma_0(x(\,{\cdot}\,))| \to 0\quad \text{as}\ \ k \to \infty.
\end{equation*}
\notag
$$
For each $k \in \mathbb{N} \cup \{0\}$ we let $\varphi_k \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ denote the minimax solution of the Cauchy problem (3.13), (3.14) with $H = H_k$ and $\sigma = \sigma_k$. Theorem 4. Under the above conditions, for any compact set $W \subset \operatorname{C}([- h, T]; \mathbb{R}^n)$ the convergence $\varphi_k(t, x(\,{\cdot}\,)) \to \varphi_0(t, x(\,{\cdot}\,))$ takes place uniformly with respect to $t \in [0, T]$ and $x(\,{\cdot}\,) \in W$ as $k \to \infty$. Theorem 4 can be used, among other things, in the development of methods for an approximate construction of the minimax solution of the Cauchy problem (3.13), (3.14), namely, as regards the substantiation of the transition from the infinite-dimensional argument $x(\,{\cdot}\,)$ of the sought-for solution $\varphi$ to a finite- dimensional one. This idea was implemented, for example, in [97] and [117] (also see [123], § 10) on the basis of a piecewise linear approximation of the argument $x(\,{\cdot}\,)$. As a consequence, a scheme of approximation of the minimax solution of the Cauchy problem (3.13), (3.14) by minimax solutions of appropriate Cauchy problems for the classical Hamilton–Jacobi equations in finite-dimensional spaces, albeit of growing dimension, was proposed. These constructions were further developed in [78]. We also note that close approximation schemes for the minimax solution of the Cauchy problem (3.13), (3.14) were developed in [66]. In this case retarded-type functional differential inclusions involved in the definition of a minimax solution were approximated by ordinary differential inclusions of large dimension (in this connection see, for example, [93], [150], [103], and also [125]). To conclude this subsection, we draw the reader’s attention to the fact that, although the functional $\gamma$ in (3.47), playing a key role in the proof of Theorem 3, is ci-smooth, for fixed $\tau \in [0, T]$ the functional $\operatorname{C}([- h, T]; \mathbb{R}^n) \ni w(\,{\cdot}\,) \mapsto \gamma(\tau, w(\,{\cdot}\,)) \in \mathbb{R}$ is not Gâteaux-differentiable. Points of non-differentiability are, for example, the points $w(\,{\cdot}\,) \in \operatorname{C}([- h, T]; \mathbb{R}^n)$ such that $\|w(\cdot \wedge \tau)\|_\infty > \|w(\tau)\|$ and the quantity $\|w(\cdot \wedge \tau)\|_\infty = \max_{\xi \in [- h, \tau]} \|w(\xi)\|$ is attained at two distinct values $\xi_1,\xi_2 \in [- h, \tau)$. We can prove this, for example, by following the scheme from [6] (Chap. XI, § 4, the lemma). In particular, the above observation emphasizes once again a certain specificity of the concept of ci-differentiability. 3.5. Characteristic complexesBy analogy with the classical Cauchy problem (2.1), (2.2), apart from the characteristic differential inclusion (3.26), other appropriate functional differential inclusions of retarded type can also be used to define a minimax solution of the Cauchy problem (3.13), (3.14). More precisely, an upper characteristic complex for the Hamilton–Jacobi equation (3.13) is a pair $(\Psi, E)$ of a non-empty set $\Psi$ and a multivalued mapping
$$
\begin{equation*}
[0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \times \Psi \ni (\tau, y(\,{\cdot}\,), \psi)\mapsto E(\tau, y(\,{\cdot}\,), \psi) \subset \mathbb{R}^n \times \mathbb{R}
\end{equation*}
\notag
$$
such that the following conditions hold.
A lower characteristic complex is a pair $(\Psi, E)$ satisfying conditions $(3.\mathrm{C}.1)$–$(3.\mathrm{C}.3)$ and the following condition.
Like in the classical case, the pair $(\mathbb{R}^n,E_0)$, where $E_0$ is defined by (3.25), is both an upper and a lower characteristic complex of equation (3.13). We let $\operatorname{Sol}(t, x(\,{\cdot}\,), E, \psi)$ denote the set of solutions $(y(\,{\cdot}\,), z(\,{\cdot}\,))$ of the retarded- type functional differential inclusion
$$
\begin{equation*}
(\dot{y}(\tau), \dot{z}(\tau))\in E(\tau, y(\,{\cdot}\,), \psi),\qquad \tau \in [t, T], \quad (y(\tau), z(\tau)) \in \mathbb{R}^n \times \mathbb{R},
\end{equation*}
\notag
$$
with initial condition (3.27) and consider the following conditions for a functional $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$.
Proposition 3. Assume that conditions $(3.\mathrm{CP}.1)$–$(3.\mathrm{CP}.3)$ hold. If the functional $\varphi \in \Phi_{\operatorname{LSC}}$ satisfies $(3.\mathrm{MS}.2_+)$ for some upper characteristic complex $(\Psi_+^\ast, E_+^\ast)$, then it satisfies this condition for all upper characteristic complexes $(\Psi_+, E_+)$. In a similar way, if a functional $\varphi \in \Phi_{\operatorname{USC}}$ satisfies $(3.\mathrm{MS}.2_-)$ for some lower characteristic complex $(\Psi_-^\ast, E_-^\ast)$, then it satisfies this condition for all lower characteristic complexes $(\Psi_-,E_-)$. We comment on the proof of the first part of Proposition 3. We fix $c > 0$ such that condition $(3.\mathrm{C}.3)$ for both complexes $(\Psi_+^\ast, E_+^\ast)$ and $(\Psi_+, E_+)$ and condition $(3.\mathrm{CP}.2)$ hold at the same time. Then, owing to [123], Theorem 5.1 (also see [67], Proposition 3), we infer that for $\varphi \in \Phi_{\operatorname{LSC}}$ condition $(3.\mathrm{MS}.2_+)$ for $(\Psi_+^\ast, E_+^\ast)$ implies this condition for the characteristic complex $(\mathbb{R}^n, E_0)$ (see (3.25)). On the other hand, following the scheme presented in [52], Proposition 1, and using properties $(3.\mathrm{a})$–$(3.\mathrm{d})$ of the functional $\nu_\varepsilon$ defined by (3.49) we can verify directly that condition $(3.\mathrm{MS}.2_+)$ for $(\mathbb{R}^n, E_0)$ yields the same condition for $(\Psi_+, E_+)$. In particular, Proposition 3 makes it possible to conclude that Theorem 3 generalizes the results in [112] and [97] (also see [123], §§ 7 and 8) concerning the existence and uniqueness of a minimax solution in the case when the Hamiltonian is positively homogeneous in the third variable and also in the case when the Cauchy problem (3.13), (3.14) can be reduced to an auxiliary Cauchy problem with homogeneous Hamiltonian (in this connection see § 2.4 for a discussion of similar cases). Note that the proof of the comparison principle for minimax solutions is slightly simplified in the above cases: in particular, there is no need to construct a ci-smooth Lyapunov–Krasovskii functional. 3.6. Infinitesimal criteria and consistencyTo express conditions $(3.\mathrm{MS}.1_+)$ and $(3.\mathrm{MS}.1_-)$, involved in the definitions of an upper and lower minimax solution of the Cauchy problem (3.13), (3.14), in the infinitesimal form it is convenient to use the following derivatives of functionals in multivalued directions, which are compatible with the ci-differentiability techniques in the natural way (see, for example, [113] and also [123], § 11). For a functional $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ the lower and upper right derivatives at a point $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ in a multivalued direction $F \in \mathcal{K}(\mathbb{R}^n)$ are, respectively, the quantities
$$
\begin{equation}
\begin{aligned} \, d_- \{\varphi(t, x(\,{\cdot}\,)) \mid F \}& = \lim_{\varepsilon \to 0^+}\, \inf_{y(\,{\cdot}\,) \in \Omega(t, x(\,{\cdot}\,), F, \varepsilon)}\, \liminf_{\delta \to 0^+}\frac{\varphi(t + \delta, y(\,{\cdot}\,)) - \varphi(t, x(\,{\cdot}\,))}{\delta}\,, \\ d_+ \{\varphi(t, x(\,{\cdot}\,)) \mid F \}& = \lim_{\varepsilon \to 0^+}\, \sup_{y(\,{\cdot}\,) \in \Omega(t, x(\,{\cdot}\,), F, \varepsilon)}\, \limsup_{\delta \to 0^+}\frac{\varphi(t + \delta, y(\,{\cdot}\,)) - \varphi(t, x(\,{\cdot}\,))}{\delta}\,, \end{aligned}
\end{equation}
\tag{3.50}
$$
where
$$
\begin{equation}
\Omega(t, x(\,{\cdot}\,), F, \varepsilon)= \bigl\{ y(\,{\cdot}\,) \in \operatorname{Lip}(t, x(\,{\cdot}\,)) \colon\dot{y}(\tau) \in [F]^\varepsilon \text{ for a. a. } \tau \in [t, T] \bigr\}
\end{equation}
\tag{3.51}
$$
and $[F]^\varepsilon$ is the closed $\varepsilon$-neighbourhood of the set $F$ in $\mathbb{R}^n$. The result reads as follows (see, for example, [114], Theorem 8.1, and also [123], Theorem 12.1). Theorem 5. Assume that conditions $(3.\mathrm{CP}.1)$ and $(3.\mathrm{CP}.2)$ hold and the functional (3.15) is non-anticipating for each $s \in \mathbb{R}^n$. Then for any functional $\varphi \in \Phi_{\operatorname{LSC}}$ condition $(3.\mathrm{MS}.1_+)$ is equivalent to the differential inequality In (3.52) and (3.53), $d_\mp \{\varphi(t, x(\,{\cdot}\,)) - \langle s, x(t) \rangle \mid B(c (1 + \|x(\cdot \wedge t)\|_\infty)) \}$ denote the corresponding derivatives at $(t,x(\,{\cdot}\,))$ of the auxiliary functional
$$
\begin{equation*}
\psi(\tau,y(\,{\cdot}\,))=\varphi(\tau,y(\,{\cdot}\,))- \langle s,y(\tau)\rangle,\quad (\tau, y(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n),
\end{equation*}
\notag
$$
in the multivalued direction equal to the ball $B(c (1 + \|x(\cdot \wedge t)\|_\infty))$, where $c$ is borrowed from $(3.\mathrm{CP}.2)$. In connection with infinitesimal analogues of $(3.\mathrm{MS}.2_+)$ and $(3.\mathrm{MS}.2_-)$ for arbitrary characteristic complexes $(\Psi, E)$, see [123], § 13.2. According to (3.50) and (3.51), calculating the derivatives $d_\mp \{\varphi(t, x(\,{\cdot}\,)) \mid F \}$ requires an analysis of the behaviour of $\varphi$ along all functions $y(\,{\cdot}\,)$ in the infinite- dimensional set $\Omega(t,x(\,{\cdot}\,),F,\varepsilon)$. Therefore, the verification of the differential inequalities (3.52) and (3.53) is, generally speaking, a rather difficult problem. Nevertheless, in some particular cases the derivatives $d_\mp\{\varphi(t,x(\,{\cdot}\,))\mid F\}$ can be calculated using convenient formulae. First of all, we note that if a functional $\varphi$ is ci-differentiable at $(t,x(\,{\cdot}\,))$, then (see, for example, [113], Proposition 1, and also [123], Proposition 11.1)
$$
\begin{equation}
\begin{aligned} \, d_- \{\varphi(t, x(\,{\cdot}\,)) \mid F \}& = \partial_t \varphi(t, x(\,{\cdot}\,)) + \min_{f \in F} \langle \nabla \varphi(t, x(\,{\cdot}\,)), f \rangle, \\ d_+ \{\varphi(t, x(\,{\cdot}\,)) \mid F \}& = \partial_t \varphi(t, x(\,{\cdot}\,)) + \max_{f \in F} \langle \nabla \varphi(t, x(\,{\cdot}\,)), f \rangle. \end{aligned}
\end{equation}
\tag{3.54}
$$
In addition, [113] and [118] (also see [123], § § 11.1, 11.3) contain formulae for the calculation of the derivatives $d_\mp \{\varphi(t, x(\,{\cdot}\,)) \mid F\}$ under the assumption that $\varphi$ is piecewise ci-smooth or is an envelope of some family of ci-smooth functionals, which is typical in the context of applications to control problems for time-delay systems. We also note that the calculation of the derivatives $d_\mp \{\varphi(t, x(\,{\cdot}\,)) \mid F \}$ is simpler when $\varphi$ satisfies the following Lipschitz condition in the variable $x(\,{\cdot}\,)$.
In this case, by [122], Lemma 1, we have
$$
\begin{equation}
\begin{aligned} \, d_- \{\varphi(t, x(\,{\cdot}\,)) \mid F \}& = \min_{f \in F} \partial_- \{\varphi(t, x(\,{\cdot}\,)) \mid f \}, \\ d_+ \{\varphi(t, x(\,{\cdot}\,)) \mid F \}& = \max_{f \in F} \partial_+ \{\varphi(t, x(\,{\cdot}\,)) \mid f \}. \end{aligned}
\end{equation}
\tag{3.55}
$$
Here $\partial_- \{\varphi(t, x(\,{\cdot}\,)) \mid f \}$ and $\partial_+ \{\varphi(t, x(\,{\cdot}\,)) \mid f \}$ denote the lower and upper right derivatives of $\varphi$ at $(t, x(\,{\cdot}\,))$ in the single-valued direction $f$, respectively, that is,
$$
\begin{equation}
\begin{aligned} \, \partial_- \{\varphi(t, x(\,{\cdot}\,)) \mid f \}& = \liminf_{\delta \to 0^+} \frac{\varphi(t + \delta, y^{(f)}(\,{\cdot}\,)) - \varphi(t, x(\,{\cdot}\,))}{\delta}\,, \\ \partial_+ \{\varphi(t, x(\,{\cdot}\,)) \mid f \}& = \limsup_{\delta \to 0^+} \frac{\varphi(t + \delta, y^{(f)}(\,{\cdot}\,)) - \varphi(t, x(\,{\cdot}\,))}{\delta}\,, \end{aligned}
\end{equation}
\tag{3.56}
$$
where the function $y^{(f)}(\,{\cdot}\,) \in \operatorname{Lip}(t, x(\,{\cdot}\,))$ is defined by $y^{(f)}(\tau) = x(t) + f(\tau - t)$, $\tau \in [t, T]$. Thus, to calculate $d_\mp \{\varphi(t, x(\,{\cdot}\,)) \mid F \}$ under condition $(3.\mathrm{L})$ it actually suffices to consider only functions $y^{(f)}(\,{\cdot}\,)$ parametrized by the finite-dimensional parameter $f \in F$. In terms of the derivatives (3.56) inequalities (3.52) and (3.53), which characterize minimax solutions of the Cauchy problem (3.13), (3.14), assume the form of inequalities (2.18) and (2.19) for the classical Hamilton–Jacobi equation (2.1) (see below for inequalities (3.62)). We also note that if $\varphi$ is specified by a function $\widehat{\varphi} \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}$ so that $\varphi(t,x(\,{\cdot}\,))=\widehat{\varphi}(t,x(t))$, $t \in [0, T]$, $x(\,{\cdot}\,) \in \operatorname{C}([- h, T]; \mathbb{R}^n)$, then condition $(3.\mathrm{L})$ on $\varphi$ implies condition $(2.\mathrm{L})$ on $\widehat{\varphi}$; therefore, it follows from (2.22) and (3.56) that
$$
\begin{equation*}
\partial_\mp \{\varphi(t, x(\,{\cdot}\,)) \mid f \}= \partial_\mp \{\widehat{\varphi}(t, x(t)) \mid 1, f \},\qquad (t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{C}([-h,T];\mathbb{R}^n),
\end{equation*}
\notag
$$
for each vector $f \in \mathbb{R}^n$. Finally, we emphasize that equalities (3.55) need not hold without some additional assumptions on the properties of $\varphi$. For example, consider the functional
$$
\begin{equation*}
\varphi (t, x(\,{\cdot}\,))= x(t) - 2 x\biggl(\frac{t}{2}\biggr),\qquad t \in [0, 1], \quad x(\,{\cdot}\,) \in \operatorname{C}([0, 1]; \mathbb{R}).
\end{equation*}
\notag
$$
It is non-anticipating and continuous and, in addition, has the Lipschitz property in $x(\,{\cdot}\,)$ with respect to the uniform norm in the following form: for all $t \in [0, 1]$ and $x_1(\,{\cdot}\,),x_2(\,{\cdot}\,) \in \operatorname{C}([0, 1]; \mathbb{R})$ we have
$$
\begin{equation*}
|\varphi (t, x_1(\,{\cdot}\,)) - \varphi (t, x_2(\,{\cdot}\,))|\leqslant 3 \|x_1(\cdot \wedge t) - x_2(\cdot \wedge t)\|_\infty.
\end{equation*}
\notag
$$
Nevertheless, the second equality in (3.55) does not hold at the point $(t_0=0, x_0(\,{\cdot}\,) \equiv 0) \in [0, 1] \times \operatorname{C}([0,1];\mathbb{R})$. In fact, on the one hand we obtain $\partial_+ \{\varphi(t_0, x_0(\,{\cdot}\,)) \mid f \} = 0$ for any single-valued direction $f \in \mathbb{R}$, since $\varphi(t_0+\delta,y^{(f)}(\,{\cdot}\,))=\varphi(t_0,x_0(\,{\cdot}\,))=0$, $\delta \in (0, 1]$. On the other hand we have the estimate $d_+ \{\varphi(t_0, x_0(\,{\cdot}\,)) \mid [- 1, 1] \} > 0$ for the multivalued direction $F = [- 1, 1]$. To verify this, we introduce the notation $\delta_1 = 1$ and $\delta_k = 3/ 2^k$ for all $k \in \mathbb{N} \setminus \{1\}$ and consider the function
$$
\begin{equation*}
y(\tau)= \begin{cases} 0 & \text{for}\ \tau = 0, \\ -\dfrac{1}{2^{k - 1}} + \tau& \text{if}\ \tau \in (\delta_{k + 1},\delta_k]\ \text{ for even}\ k \in \mathbb{N}, \\ \dfrac{1}{2^{k - 1}}-\tau& \text{if}\ \tau \in (\delta_{k + 1},\delta_k]\ \text{for odd}\ k \in \mathbb{N}. \end{cases}
\end{equation*}
\notag
$$
Note that $\varphi(t_0 + \delta_k, y(\,{\cdot}\,)) = 1 / 2^{k - 1}$ for each even $k \in \mathbb{N}$. Consequently, taking account of the inclusion $y(\,{\cdot}\,) \in \Omega(t_0, x_0(\,{\cdot}\,),[-1,1],0)$ we infer that
$$
\begin{equation*}
d_+ \{\varphi(t_0, x_0(\,{\cdot}\,)) \mid [- 1, 1] \}\geqslant \limsup_{k \to \infty}\frac{\varphi(t_0+\delta_k,y(\,{\cdot}\,))}{\delta_k} \geqslant \frac{2}{3}> 0.
\end{equation*}
\notag
$$
In view of condition $(3.\mathrm{CP}.2)$ and (3.54) we can conclude from Theorem 5 that the concept of a minimax solution of the Cauchy problem (3.13), (3.14) is consistent with the concept of a solution of this problem in the classical sense (in this connection also see [123], Proposition 12.1). Proposition 4. Assume that conditions $(3.\mathrm{CP}.1)$ and $(3.\mathrm{CP}.2)$ hold and the functional (3.15) is non-anticipating for each $s \in \mathbb{R}^n$. Then the following hold: (i) if a continuous functional $\varphi \colon [0, T] \times \operatorname{C}([-h, T]; \mathbb{R}^n) \to \mathbb{R}$ is ci-differentiable at each point $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ and satisfies the Hamilton–Jacobi equation (3.13) and the boundary condition (3.14), then it is a minimax solution of the Cauchy problem (3.13), (3.14); (ii) if a minimax solution of the Cauchy problem (3.13), (3.14) is ci-differentiable at some point $(t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{C}([-h,T];\mathbb{R}^n)$, then it satisfies the Hamilton–Jacobi equation (3.13) at this point. Thus, the pair of differential inequalities (3.52), (3.53) can be interpreted as a generalization of the Hamilton–Jacobi equation (3.13) to the non-smooth case. To conclude this subsection we discuss the natural question whether the concept of minimax solution of the Cauchy problem (3.13), (3.14) is a natural extension of the concept of minimax solution of the Cauchy problem for the classical Hamilton–Jacobi equations (see § 2). Assume that in the Cauchy problem (3.13), (3.14) the Hamiltonian $H$ and boundary functional $\sigma$ have the form
$$
\begin{equation}
\begin{gathered} \, H(t, x(\,{\cdot}\,), s)= \widehat{H}(t, x(t), s)\quad\text{and}\quad \sigma(x(\,{\cdot}\,))= \widehat{\sigma}(x(T)), \\ t \in [0, T], \quad x(\,{\cdot}\,) \in \operatorname{C}([- h, T]; \mathbb{R}^n), \quad s \in \mathbb{R}^n, \end{gathered}
\end{equation}
\tag{3.57}
$$
for some functions $\widehat{H} \colon [0, T] \times \mathbb{R}^n \times \mathbb{R}^n \to \mathbb{R}$ and $\widehat{\sigma} \colon \mathbb{R}^n \to \mathbb{R}$. We consider the Cauchy problem for the Hamilton–Jacobi equation
$$
\begin{equation}
\frac{\partial \widehat{\varphi}}{\partial t} (t, x) + \widehat{H}\biggl( t, x, \frac{\partial \widehat{\varphi}} {\partial x} (t, x) \biggr)= 0,\qquad (t, x) \in (0, T) \times \mathbb{R}^n,
\end{equation}
\tag{3.58}
$$
with the right-end boundary condition
$$
\begin{equation}
\widehat{\varphi}(T, x)= \widehat{\sigma}(x),\qquad x \in \mathbb{R}^n.
\end{equation}
\tag{3.59}
$$
From $(3.\mathrm{CP}.1)$–$(3.\mathrm{CP}.3)$ we obtain that the functions $\widehat{H}$ and $\widehat{\sigma}$ satisfy conditions $(2.\mathrm{CP}.1)$–$(2.\mathrm{CP}.3)$, which ensure the existence and uniqueness of the minimax solution $\widehat{\varphi} \colon [0, T]\times \mathbb{R}^n \to \mathbb{R}$ of the Cauchy problem (3.58), (3.59). In particular, the function $\widehat{\varphi}$ is continuous and satisfies the boundary condition (3.59) and the condition $(2.\mathrm{MS})$ for $\widehat{H}$. Then we can verify directly that the functional
$$
\begin{equation}
\varphi(t, x(\,{\cdot}\,))= \widehat{\varphi}(t, x(t)),\qquad (t, x(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n),
\end{equation}
\tag{3.60}
$$
is non-anticipating and continuous and satisfies the boundary condition (3.14) and condition $(3.\mathrm{MS})$. As a result, in view of Theorem 3 we arrive at the following assertion. Proposition 5. Assume that conditions $(3.\mathrm{CP}.1)$–$(3.\mathrm{CP}.3)$ hold for the Cauchy problem (3.13), (3.14) and that the Hamiltonian $H$ and boundary functional $\sigma$ have the form (3.57). Then the minimax solution $\varphi$ of this problem is defined by (3.60). In particular, we infer that if the minimax solution $\widehat{\varphi}$ of the Cauchy problem (3.58), (3.59) is differentiable at some point $(t, \widehat{x}) \in (0, T) \times \mathbb{R}^n$, then, according to (3.60), the minimax solution $\varphi$ of the Cauchy problem (3.13), (3.14) is ci-differentiable at all points $(t, x(\,{\cdot}\,))$ such that $x(\,{\cdot}\,) \in \operatorname{C}([- h, T]; \mathbb{R}^n)$ and $x(t) = \widehat{x}$. In this case we have
$$
\begin{equation*}
\partial_t \varphi(t, x(\,{\cdot}\,))= \frac{\partial \widehat{\varphi}}{\partial t} (t, x(t))\quad\text{and}\quad \nabla \varphi(t, x(\,{\cdot}\,))= \frac{\partial \widehat{\varphi}}{\partial x} (t, x(t)).
\end{equation*}
\notag
$$
Thus, we can say that the Hamilton–Jacobi equations (3.13) with ci-derivatives are a generalization of the classical Hamilton–Jacobi equations (2.1) with partial derivatives to the case when the Hamiltonian $H$ and required solution $\varphi$ are non-anticipating functionals. 3.7. Meaningfulness of the minimax solutionRegarding the differential game (3.16), (3.17), the minimax solution $\varphi$ of the Cauchy problem (3.13), (3.14) with Hamiltonian (3.22) makes it possible in the general (non-smooth) case to construct optimal feedback control strategies of the players with memory of motion history. For an initial position $(t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ we obtain the Lyapunov–Krasovskii functional $\nu_\varepsilon$ from (3.49), $\varepsilon \in (0, \varepsilon_0]$, and consider the following non- anticipating control strategies of the first and second players:
$$
\begin{equation*}
U^\circ_\varepsilon(\tau, y(\,{\cdot}\,)) \in \operatorname*{arg\,min}_{u \in P}\,\max_{v \in Q} \bigl( \langle s_\varepsilon^{(u)}(\tau, y(\,{\cdot}\,)), f(\tau, y(\,{\cdot}\,), u, v) \rangle-\chi(\tau, y(\,{\cdot}\,),u,v)\bigr)
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
\begin{gathered} \, V^\circ_\varepsilon(\tau, y(\,{\cdot}\,)) \in \operatorname*{arg\,max}_{v \in Q}\,\min_{u \in P} \bigl( \langle s_\varepsilon^{(v)}(\tau, y(\,{\cdot}\,)), f(\tau, y(\,{\cdot}\,), u, v) \rangle - \chi(\tau, y(\,{\cdot}\,),u,v)\bigr), \\ (\tau,y(\,{\cdot}\,)) \in [0,T) \times \operatorname{C}([-h,T];\mathbb{R}^n), \end{gathered}
\end{equation*}
\notag
$$
where
$$
\begin{equation*}
\begin{aligned} \, s_\varepsilon^{(u)}(\tau, y(\,{\cdot}\,)) & = \nabla \nu_\varepsilon \bigl( \tau, y(\,{\cdot}\,) - z_\varepsilon^{(u)}(\cdot \mid \tau, y(\,{\cdot}\,)) \bigr), \\ z_\varepsilon^{(u)}(\cdot \mid \tau, y(\,{\cdot}\,)) & \in \operatorname*{arg\,min}_{z(\,{\cdot}\,) \in Y(t, x(\,{\cdot}\,))} \bigl( \varphi(\tau, z(\,{\cdot}\,)) + \nu_{\varepsilon}(\tau, y(\,{\cdot}\,) - z(\,{\cdot}\,)) \bigr), \\ s_\varepsilon^{(v)}(\tau, y(\,{\cdot}\,)) & = - \nabla \nu_\varepsilon \bigl( \tau, y(\,{\cdot}\,) - z_\varepsilon^{(v)}(\cdot \mid \tau, y(\,{\cdot}\,)) \bigr), \\ z_\varepsilon^{(v)}(\cdot \mid \tau, y(\,{\cdot}\,)) & \in \operatorname*{arg\,max}_{z(\,{\cdot}\,) \in Y(t, x(\,{\cdot}\,))} \bigl( \varphi(\tau, z(\,{\cdot}\,)) - \nu_{\varepsilon}(\tau, y(\,{\cdot}\,) - z(\,{\cdot}\,)) \bigr), \end{aligned}
\end{equation*}
\notag
$$
and the set $Y(t,x(\,{\cdot}\,))$ is borrowed from (3.32). We can choose non-anticipating $U^\circ_\varepsilon(\tau, y(\,{\cdot}\,))$ and $V^\circ_\varepsilon(\tau, y(\,{\cdot}\,))$ by the non-anticipation property of the maps (3.18) and also $\varphi$, $\nu_\varepsilon$, and $\nabla \nu_\varepsilon$. By analogy with (2.28) and (2.29) (also see the comment after Theorem 2) we define the guaranteed results of the control strategies $U^\circ_\varepsilon$ and $V^\circ_\varepsilon$ of the first and second players as follows:
$$
\begin{equation}
\begin{aligned} \, \rho^{(u)} (t, x(\,{\cdot}\,), U^\circ_\varepsilon)& =\lim_{\delta \to 0^+}\, \sup_{\Delta \in \Pi_\delta(t)}\,\sup_{v(\,{\cdot}\,) \in \mathcal{V}(t)} J(t, x(\,{\cdot}\,), U^\circ_\varepsilon, \Delta, v(\,{\cdot}\,)), \\ \rho^{(v)} (t, x(\,{\cdot}\,), V^\circ_\varepsilon)& = \lim_{\delta \to 0^+}\, \inf_{\Delta \in \Pi_\delta(t)}\,\inf_{u(\,{\cdot}\,) \in \mathcal{U}(t)} J(t, x(\,{\cdot}\,), u(\,{\cdot}\,), V^\circ_\varepsilon, \Delta). \end{aligned}
\end{equation}
\tag{3.61}
$$
Taking account of the fact that the functional $\nu_\varepsilon$ satisfies conditions $(3.\mathrm{a})$–$(3.\mathrm{d})$ in Lemma 1 and following the scheme of reasoning from [123], Theorem 15.1 (also see [116], Theorem 1, and [121], Theorem 2), we can verify the following assertion. Theorem 6. Under conditions $(3.\mathrm{DG}.1)$–$(3.\mathrm{DG}.4)$ the differential game (3.16), (3.17) has a value, and the value functional $\rho$ coincides with the minimax solution $\varphi$ of the Cauchy problem (3.13), (3.14) with Hamiltonian (3.22), that is, In addition, the control strategies $U^\circ_\varepsilon$ and $V^\circ_\varepsilon$ of the first and second players are optimal: Note that in the construction of $U^\circ_\varepsilon$ and $V^\circ_\varepsilon$ we can use any other functional satisfying conditions $(3.\mathrm{a})$–$(3.\mathrm{d})$ instead of $\nu_\varepsilon$ from (3.49). In the solution of particular differential games of the form (3.16), (3.17) a successful choice of such a functional can significantly simplify the calculations (see, for example, [116]). As a consequence of Theorem 6, we deduce that the value functional of the differential game (3.16), (3.17) has all properties of the minimax solution of the Cauchy problem (3.13), (3.14) with Hamiltonian (3.22). In particular, by Proposition 4 the value functional satisfies the Hamilton–Jacobi equation at all points where it is ci-differentiable, which refines Theorem 1. On the other hand, to find out whether one or another functional $\varphi$ is a value functional, it suffices to verify any of the conditions characterizing the minimax solution of this problem. For example, if $\varphi \in \Phi_{\operatorname{C}}$ satisfies the Lipschitz condition $(3.\mathrm{L})$ and the boundary condition (3.14), then, according to § 3.6 (also see [122], Theorem 3), it suffices to verify the differential inequalities
$$
\begin{equation}
\begin{gathered} \, \begin{aligned} \, \min_{f \in B(c (1 + \|x(\cdot \wedge t)\|_\infty))} \bigl( \partial_- \{\varphi(t, x(\,{\cdot}\,)) \mid f\} - \langle s, f \rangle \bigr)+ H(t, x(\,{\cdot}\,), s)& \leqslant 0, \\ \max_{f \in B(c (1 + \|x(\cdot \wedge t)\|_\infty))} \bigl( \partial_+ \{\varphi(t, x(\,{\cdot}\,)) \mid f\} - \langle s, f \rangle \bigr) + H(t, x(\,{\cdot}\,), s)& \geqslant 0, \end{aligned} \\ (t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n), \quad s \in \mathbb{R}^n, \end{gathered}
\end{equation}
\tag{3.62}
$$
where $\partial_\mp \{\varphi(t, x(\,{\cdot}\,)) \mid f\}$ are the directional derivatives (3.56), $H$ is the Hamiltonian (3.22), and $c$ is from $(3.\mathrm{DG}.2)$. In this connection we note ([122], Theorem 2) that the value functional of the differential game (3.16), (3.17) satisfies the Lipschitz condition $(3.\mathrm{L})$ if assumptions $(3.\mathrm{DG}.1)$, $(3.\mathrm{DG}.2)$, and $(3.\mathrm{DG}.4)$ and also the following conditions hold.
Note that the condition $(3.\mathrm{DG}.5)$ implies $(3.\mathrm{DG}.3)$; using it we can treat only distributed and constant concentrated delays (see (1.21) and (1.23)). 3.8. Viscosity solutionIn building the theory of minimax solutions of the Cauchy problem (3.13), (3.14) an important trick in the investigation of the properties of functionals $\varphi$ is that it is always possible to limit ourselves to considering only the values $\varphi(t,x(\,{\cdot}\,))$ at functions $x(\,{\cdot}\,)$ in appropriately chosen compact subsets of the space $\operatorname{C}([- h, T]; \mathbb{R}^n)$. When the viscosity approach to the concept of a generalized solution of this problem is developed, there arise additional difficulties in this regard, because instead of compact subsets in $\operatorname{C}([- h, T]; \mathbb{R}^n)$ we have to deal with bounded closed ones, on which continuous bounded functionals $\varphi$ need not attain extremum values any longer. Similar obstacles also occur in the theory of viscosity solutions of Hamilton–Jacobi equations in Banach spaces with Frechét derivatives. One method for dealing with them is to use an appropriate smooth variational principle, more precisely, an assertion that, given a lower semicontinuous, bounded below functional $\varphi$, makes it possible, roughly speaking, to find an arbitrarily small Frechét-differentiable functional $\psi$ such that the sum $\varphi + \psi$ must attain its minimum. In this connection we note, for example, the variational principles due to Stegall [158], Borwein and Preiss [14], and Deville, Godefroy, and Zizler [31] (also see [15], Theorems 2.5.3, 2.5.7, and 6.3.5). However, in assertions of this type a key assumption ensuring the smoothness of the small additional term $\psi$ is usually one or another property concerning the smoothness of the norm in the relevant Banach space. In particular, this does not make it possible to cover the case of the space $\operatorname{C}([-h,T];\mathbb{R}^n)$. Another method for dealing with the obstacles under discussion was indicated in [156], where it was proposed to modify the concept of viscosity solution by introducing an appropriate sequence of expanding compact subsets such that the closure of their union covers the whole space. In [120] and [119] this modification was taken as the basis for the following definition of a viscosity solution of the Cauchy problem (3.13), (3.14). For each $k \in \mathbb{N}$ consider a compact set $W_k$ consisting of functions $x(\,{\cdot}\,) \in \operatorname{Lip}([-h,T];\mathbb{R}^n)$ such that
$$
\begin{equation*}
\|x(- h)\|\leqslant k\quad\text{and}\quad \|\dot{x}(\tau)\| \leqslant kc(1+\|x(\cdot \wedge \tau)\|_\infty)\quad \text{for a. a. } \tau \in [- h, T],
\end{equation*}
\notag
$$
where $c$ is from $(3.\mathrm{CP}.2)$. Note that $W_k \subset W_{k + 1}$, $k \in \mathbb{N}$, the closure of the union of the $W_k$ over all $k \in \mathbb{N}$ coincides with the whole of $\operatorname{C}([- h, T]; \mathbb{R}^n)$, and for each $k \in \mathbb{N}$ the set $W_k$ is strongly invariant in the following sense: for any point $(t, x(\,{\cdot}\,)) \in [0, T] \times W_k$ we have $Y(t, x(\,{\cdot}\,)) \subset W_k$, where the set $Y(t, x(\,{\cdot}\,))$ is defined by (3.32). Definition 11. An upper viscosity solution of the Cauchy problem (3.13), (3.14) is a functional $\varphi \in \Phi_{\operatorname{C}}$ that satisfies the boundary condition (3.28) and has the following property.
Definition 12. A lower viscosity solution of the Cauchy problem (3.13), (3.14) is a functional $\varphi \in \Phi_{\operatorname{C}}$ that satisfies the boundary condition (3.30) and has the following property.
Definition 13. A viscosity solution of the Cauchy problem (3.13), (3.14) is a functional $\varphi \in \Phi_{\operatorname{C}}$ that is both an upper and a lower viscosity solution of this problem. Like in the case of the classical Hamilton–Jacobi equations (2.1) we can verify that, under conditions $(3.\mathrm{CP}.1)$–$(3.\mathrm{CP}.3)$, the minimax solution of the Cauchy problem (3.13), (3.14), which exists and is unique by Theorem 3, is a viscosity solution of this problem. In this case the strong invariance of the sets $W_k$, $k \in \mathbb{N}$, turns out to be important. Thus, the following result is true ([119], Theorem 1). Theorem 7. Under conditions $(3.\mathrm{CP}.1)$–$(3.\mathrm{CP}.3)$ there exists at least one viscosity solution of the Cauchy problem (3.13), (3.14). Again, the verification of the uniqueness of the viscosity solution of the Cauchy problem (3.13), (3.14) consists in proving the comparison principle: for any upper and lower viscosity solutions $\varphi_+$ and $\varphi_-$ of this problem
$$
\begin{equation}
\varphi_+ (t, x(\,{\cdot}\,))\geqslant \varphi_- (t, x(\,{\cdot}\,)),\qquad (t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n).
\end{equation}
\tag{3.64}
$$
Here we must construct an appropriate ci-smooth penalty functional (an analogue of the function $\|x-y\|^2$ in (2.36)). As shown in [119], Theorem 2, when $(3.\mathrm{CP}.1)$, $(3.\mathrm{CP}.2)$, and $(3.\mathrm{CP}.4)$ hold, we can take the functional
$$
\begin{equation}
\begin{gathered} \, \nu(t, x(\,{\cdot}\,), \tau, y(\,{\cdot}\,)) = (t - \tau)^2 + \|x(t) - y(\tau)\|^2 + \int_{- h}^{T} \|x(\xi \wedge t) - y(\xi \wedge \tau)\|^2 \, {\rm d} \xi, \\ (t, x(\,{\cdot}\,)), (\tau, y(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n). \end{gathered}
\end{equation}
\tag{3.65}
$$
In addition, a generalization of this result to the case when conditions $(3.\mathrm{CP}.1)$, $(3.\mathrm{CP}.2)$, and $(3.\mathrm{CP}.5)$ hold was deduced in [121], Theorem 4. In this case the following functional was constructed on the basis of the ideas in [91] on the construction of Lyapunov–Krasovskii functionals in stability problems for time-delay systems:
$$
\begin{equation}
\begin{gathered} \, \begin{aligned} \, & \nu(t,x(\,{\cdot}\,),\tau,y(\,{\cdot}\,))=(t-\tau)^2+\|x(t)-y(\tau)\|^2 \\ &\qquad +\sum_{j=1}^\mathcal{J}\int_{t-\vartheta_j}^{T} \|x(\xi \wedge t) - y(\xi \wedge \tau)\|^2 \, {\rm d} \xi + \int_{- h}^{T}\|x(\xi \wedge t)-y(\xi \wedge \tau)\|^2 \, {\rm d}\xi, \end{aligned} \\ (t, x(\,{\cdot}\,)), (\tau, y(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([-h,T];\mathbb{R}^n). \end{gathered}
\end{equation}
\tag{3.66}
$$
More precisely, the following assertion is valid. Theorem 8. Under conditions $(3.\mathrm{CP}.1)$, $(3.\mathrm{CP}.2)$, and $(3.\mathrm{CP}.5)$ inequality (3.64) holds for any upper and lower viscosity solutions $\varphi_+$ and $\varphi_-$ of the Cauchy problem (3.13), (3.14). We prove Theorem 8. The closure of the union of $W_k$, $k \in \mathbb{N}$, coincides with $\operatorname{C}([- h, T]; \mathbb{R}^n)$; the functionals $\varphi_+$ and $\varphi_-$ are continuous. Therefore, it suffices to fix $k \in \mathbb{N}$ and show that inequality (3.64) holds for all $(t, x(\,{\cdot}\,)) \in [0, T] \times W_k$. Reasoning by contradiction, we assume that
$$
\begin{equation}
\max_{(t, x(\,{\cdot}\,)) \in [0, T] \times W_k} \bigl( \varphi_-(t, x(\,{\cdot}\,))-\varphi_+(t,x(\,{\cdot}\,))\bigr)> 0.
\end{equation}
\tag{3.67}
$$
We fix a number $\alpha > 0$ satisfying $4 T \alpha \leqslant b$, where $b$ is the value of the maximum in (3.67). We consider the functional
$$
\begin{equation*}
\begin{gathered} \, \begin{aligned} \, \Psi(t, x(\,{\cdot}\,), \tau, y(\,{\cdot}\,))& = \varphi_- (t, x(\,{\cdot}\,)) - \varphi_+ (\tau, y(\,{\cdot}\,)) \\ & \qquad - \alpha (2 T - t - \tau)- \frac{(t - \tau)^2}{\delta}- \frac{\nu(t, x(\,{\cdot}\,), \tau, y(\,{\cdot}\,))}{\varepsilon}\,, \end{aligned} \\ (t, x(\,{\cdot}\,)), (\tau, y(\,{\cdot}\,)) \in [0, T] \times W_k, \end{gathered}
\end{equation*}
\notag
$$
where $\delta > 0$ and $\varepsilon > 0$ are small parameters and $\nu$ is the functional (3.66). We choose points $(t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,))$, $(\tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,)) \in [0, T] \times W_k$ from the condition
$$
\begin{equation}
\Psi(t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,), \tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,)) = \max_{(t, x(\,{\cdot}\,)), (\tau, y(\,{\cdot}\,)) \in [0, T] \times W_k} \Psi(t, x(\,{\cdot}\,), \tau, y(\,{\cdot}\,)).
\end{equation}
\tag{3.68}
$$
Note that $\Psi(t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,), \tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,)) \geqslant \Psi(T, x(\,{\cdot}\,), T, x(\,{\cdot}\,)) = 0$ for any function $x(\,{\cdot}\,) \in W_k$. Hence
$$
\begin{equation}
\nu (t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,), \tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,)) \leqslant C_1 \varepsilon,\quad ( t_\varepsilon^\delta - \tau_\varepsilon^\delta )^2 \leqslant C_1 \delta.
\end{equation}
\tag{3.69}
$$
In what follows $C_i$, $i \in \{1,\dots\}$, denote positive constants independent of $\varepsilon$ and $\delta$. In view of the definition (3.66) of $\nu$ and the compactness of the set $W_k$, from the first inequality in (3.69) we derive
$$
\begin{equation}
\| x_\varepsilon^\delta(\cdot \wedge t_\varepsilon^\delta) - y_\varepsilon^\delta(\cdot \wedge \tau_\varepsilon^\delta) \|_\infty + |t_\varepsilon^\delta - \tau_\varepsilon^\delta|\to 0 \quad \text{as}\ \ \varepsilon \to 0^+;
\end{equation}
\tag{3.70}
$$
by virtue of the non-anticipation property and the continuity of $\varphi_+$ it follows that
$$
\begin{equation}
\begin{aligned} \, \nonumber &|\varphi_+(t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,))- \varphi_+(\tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,))| \\ &\qquad=|\varphi_+(t_\varepsilon^\delta, x_\varepsilon^\delta (\cdot \wedge t_\varepsilon^\delta))- \varphi_+(\tau_\varepsilon^\delta, y_\varepsilon^\delta(\cdot \wedge \tau_\varepsilon^\delta))| \to 0 \quad\text{as}\ \ \varepsilon \to 0^+. \end{aligned}
\end{equation}
\tag{3.71}
$$
Since $\Psi(t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,), \tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,)) \geqslant \Psi(t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,), t_\varepsilon^\delta,x_\varepsilon^\delta(\,{\cdot}\,))$, relations (3.70) and (3.71) yield
$$
\begin{equation}
\frac{\nu( t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,), \tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,))}{\varepsilon} \to 0\quad \text{as}\ \ \varepsilon \to 0^+.
\end{equation}
\tag{3.72}
$$
We have
$$
\begin{equation*}
\Psi(t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,), \tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,)) \geqslant \Psi(t,x(\,{\cdot}\,),t,x(\,{\cdot}\,)),\qquad (t, x(\,{\cdot}\,)) \in [0, T] \times W_k.
\end{equation*}
\notag
$$
Taking account of the choice of $\alpha$, the convergence (3.71), and the boundary conditions (3.28) and (3.30) for $\varphi_+$ and $\varphi_-$, respectively, we infer that $t_\varepsilon^\delta < T$ and $\tau_\varepsilon^\delta < T$ for all sufficiently small $\varepsilon > 0$. Now we introduce the functional
$$
\begin{equation*}
\begin{gathered} \, \psi_-(t, x(\,{\cdot}\,))= \varphi_+ (\tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,))+\alpha(2T-t-\tau_\varepsilon^\delta)+ \frac{(t - \tau_\varepsilon^\delta)^2}{\delta}+\frac{\nu(t, x(\,{\cdot}\,), \tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,))}{\varepsilon}\,, \\ (t, x(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n). \end{gathered}
\end{equation*}
\notag
$$
This functional is ci-smooth, and we have
$$
\begin{equation*}
\partial_t \psi_-(t, x(\,{\cdot}\,)) = - \alpha + \frac{2 (t - \tau_\varepsilon^\delta)}{\delta} + \frac{2 (t - \tau_\varepsilon^\delta)}{\varepsilon} - \frac{1}{\varepsilon} \sum_{j = 1}^\mathcal{J} \|x(t - \vartheta_j) - y(t - \vartheta_j)\|^2
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
\nabla \psi_-(t, x(\,{\cdot}\,)) = \frac{2}{\varepsilon} \biggl(\bigl(1 + (\mathcal{J} + 1)(T - t)\bigr) x(t)- y_\varepsilon^\delta(\tau_\varepsilon^\delta) - (\mathcal{J} + 1) \!\int_{t}^{T}\!\! y_\varepsilon^\delta(\xi \wedge \tau_\varepsilon^\delta) \, {\rm d} \xi \biggr)
\end{equation*}
\notag
$$
for all $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ such that $t \leqslant \tau_\varepsilon^\delta + \vartheta_j$, $j \in \{1,\dots, \mathcal{J}\}$. Owing to (3.68), the difference $\varphi_- - \psi_-$ attains its maximum in $[0, T] \times W_k$ at the point $(t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,))$; thus, by condition $(3.\mathrm{VS}.1_-)$ for $\varphi_-$ we obtain
$$
\begin{equation}
\partial_t\psi_-(t_\varepsilon^\delta,x_\varepsilon^\delta(\,{\cdot}\,))+ H\bigl(t_\varepsilon^\delta,x_\varepsilon^\delta(\,{\cdot}\,), \nabla \psi_-(t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,))\bigr) \geqslant 0.
\end{equation}
\tag{3.73}
$$
On the other hand, for the ci-smooth functional
$$
\begin{equation*}
\begin{gathered} \, \psi_+(\tau, y(\,{\cdot}\,))= \varphi_- (t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,))- \alpha (2 T - t_\varepsilon^\delta-\tau) - \frac{(t_\varepsilon^\delta - \tau)^2}{\delta}- \frac{\nu(t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,), \tau, y(\,{\cdot}\,))}{\varepsilon}\,, \\ (\tau, y(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n), \end{gathered}
\end{equation*}
\notag
$$
we have
$$
\begin{equation*}
\partial_t \psi_+(\tau, y(\,{\cdot}\,)) = \alpha + \frac{2 (t_\varepsilon^\delta - \tau)}{\delta}+ \frac{2 (t_\varepsilon^\delta - \tau)}{\varepsilon}\,,
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
\begin{gathered} \, \nabla \psi_+(\tau, y(\,{\cdot}\,)) = \frac{2}{\varepsilon} \biggl(x_\varepsilon^\delta(t_\varepsilon^\delta)- \bigl(1+(\mathcal{J}+1)(T-\tau)\bigr)y(\tau) \\ \qquad{}+(\mathcal{J}+1) \int_{\tau}^{T} x_\varepsilon^\delta (\xi \wedge t_\varepsilon^\delta) \, {\rm d} \xi \biggr) \end{gathered}
\end{equation*}
\notag
$$
for all $(\tau, y(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ such that $\tau \geqslant t_\varepsilon^\delta - \vartheta_j$, $j \in \{1,\dots, \mathcal{J}\}$; according to condition $(3.\mathrm{VS}.1_+)$ on $\varphi_+$, we arrive at the inequality
$$
\begin{equation}
\partial_t\psi_+(\tau_\varepsilon^\delta,y_\varepsilon^\delta(\,{\cdot}\,))+ H\bigl(\tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,), \nabla\psi_+(\tau_\varepsilon^\delta,y_\varepsilon^\delta(\,{\cdot}\,))\bigr) \leqslant 0.
\end{equation}
\tag{3.74}
$$
Hence, in view of (3.70) it follows from (3.73) and (3.74) that
$$
\begin{equation}
2\alpha\leqslant H(t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,), p_\varepsilon^\delta)- H(\tau_\varepsilon^\delta, y_\varepsilon^\delta (\,{\cdot}\,), q_\varepsilon^\delta)-\frac{1}{\varepsilon} \sum_{j = 1}^\mathcal{J} \|x(t_\varepsilon^\delta - \vartheta_j) - y(t_\varepsilon^\delta - \vartheta_j)\|^2
\end{equation}
\tag{3.75}
$$
for all sufficiently small $\varepsilon > 0$ and all $\delta > 0$, where
$$
\begin{equation*}
p_\varepsilon^\delta =\frac{2}{\varepsilon}\biggl(\bigl(1+(\mathcal{J}+1) (T-t_\varepsilon^\delta)\bigr)x(t_\varepsilon^\delta)-y_\varepsilon^\delta (\tau_\varepsilon^\delta) - (\mathcal{J} + 1) \int_{t_\varepsilon^\delta}^{T} y_\varepsilon^\delta(\xi \wedge \tau_\varepsilon^\delta) \, {\rm d}\xi\biggr)
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
q_\varepsilon^\delta = \frac{2}{\varepsilon} \biggl( x_\varepsilon^\delta (t_\varepsilon^\delta)-\bigl(1+(\mathcal{J}+1) (T-\tau_\varepsilon^\delta)\bigr)y(\tau_\varepsilon^\delta)+ (\mathcal{J}+1)\int_{\tau_\varepsilon^\delta}^{T} x_\varepsilon^\delta(\xi \wedge t_\varepsilon^\delta) \, {\rm d}\xi\biggr).
\end{equation*}
\notag
$$
The functional $\nu$ is continuous, the set $W_k$ is compact, and, in addition, all functions in this set satisfy the Lipschitz condition with the same constant. Therefore,
$$
\begin{equation}
\| p_\varepsilon^\delta - q_\varepsilon^\delta \|\leqslant C_2 \frac{\nu_\varepsilon^\delta}{\varepsilon}\, \quad\text{and}\quad \|p_\varepsilon^\delta\|\leqslant C_2 \frac{\sqrt{\nu_\varepsilon^\delta}}{\varepsilon}\,,
\end{equation}
\tag{3.76}
$$
where $\nu_\varepsilon^\delta = \nu (t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,), \tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,))$. Then from condition $(3.\mathrm{CP}.2)$ we obtain the inequality
$$
\begin{equation}
|H(\tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,), p_\varepsilon^\delta)- H(\tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,), q_\varepsilon^\delta)|\leqslant C_3 \frac{\nu_\varepsilon^\delta}{\varepsilon}\,.
\end{equation}
\tag{3.77}
$$
In addition, by condition $(3.\mathrm{CP}.5)$ for $W_k$ we have
$$
\begin{equation}
\begin{aligned} \, \nonumber & | H (t_\varepsilon^\delta, x_\varepsilon^\delta(\,{\cdot}\,), p_\varepsilon^\delta)- H (t_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,), p_\varepsilon^\delta) | \\ \nonumber & \qquad \leqslant C_4 (1 + \|p_\varepsilon^\delta\|) \biggl(\sqrt{\nu_\varepsilon^\delta}+ \sum_{j = 1}^\mathcal{J} \|x(t_\varepsilon^\delta - \vartheta_j) - y(t_\varepsilon^\delta - \vartheta_j)\| \biggr) \\ \nonumber & \qquad \leqslant C_5 \biggl(\sqrt{\nu_\varepsilon^\delta} + \frac{\nu_\varepsilon^\delta}{\varepsilon} + \sum_{j = 1}^\mathcal{J} \|x(t_\varepsilon^\delta - \vartheta_j) - y(t_\varepsilon^\delta - \vartheta_j)\| \\ & \qquad\qquad + \frac{\sqrt{\nu_\varepsilon^\delta}}{\varepsilon} \sum_{j = 1}^\mathcal{J} \|x(t_\varepsilon^\delta - \vartheta_j) - y(t_\varepsilon^\delta - \vartheta_j)\| \biggr). \end{aligned}
\end{equation}
\tag{3.78}
$$
As a result, combining (3.75), (3.77), and (3.78) and taking account of the inequality
$$
\begin{equation*}
C_5 \frac{\sqrt{\nu_\varepsilon^\delta}}{\varepsilon} \sum_{j = 1}^\mathcal{J} \|x(t_\varepsilon^\delta - \vartheta_j) - y(t_\varepsilon^\delta - \vartheta_j)\|\leqslant C_6 \frac{\nu_\varepsilon^\delta}{\varepsilon} + \frac{1}{\varepsilon} \sum_{j = 1}^\mathcal{J} \|x(t_\varepsilon^\delta - \vartheta_j) - y(t_\varepsilon^\delta - \vartheta_j)\|^2,
\end{equation*}
\notag
$$
we arrive at the estimate
$$
\begin{equation*}
\begin{aligned} \, 2\alpha &\leqslant |H(t_\varepsilon^\delta,y_\varepsilon^\delta(\,{\cdot}\,), p_\varepsilon^\delta)- H (\tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,), p_\varepsilon^\delta)| \\ & \qquad + C_7 \biggl( {\textstyle \sqrt{\nu_\varepsilon^\delta}} + \frac{\nu_\varepsilon^\delta}{\varepsilon} + \sum_{j = 1}^\mathcal{J} \|x(t_\varepsilon^\delta - \vartheta_j) - y(t_\varepsilon^\delta - \vartheta_j)\| \biggr). \end{aligned}
\end{equation*}
\notag
$$
In view of the first inequality in (3.69) and also relations (3.70) and (3.72) we can choose $\varepsilon > 0$ to be sufficienly small so that
$$
\begin{equation}
\alpha\leqslant |H(t_\varepsilon^\delta,y_\varepsilon^\delta(\,{\cdot}\,), p_\varepsilon^\delta)- H (\tau_\varepsilon^\delta, y_\varepsilon^\delta(\,{\cdot}\,), p_\varepsilon^\delta) |
\end{equation}
\tag{3.79}
$$
for each $\delta > 0$. The Hamiltonian $H$ is continuous, the set $W_k$ is compact, and $\|p_\varepsilon^\delta\| \leqslant C_8 / \varepsilon$ by virtue of the second inequality in (3.76). Hence, owing to the second inequality in (3.69), the right-hand side of (3.79) tends to zero as $\delta \to 0^+$, which contradicts the choice of $\alpha > 0$. The proof of Theorem 8 is complete. As a consequence, we see that, under conditions $(3.\mathrm{CP}.1)$, $(3.\mathrm{CP}.2)$, and $(3.\mathrm{CP}.5)$, the viscosity solution of the Cauchy problem (3.13), (3.14) exists, is unique, and coincides with the minimax solution of this problem. In particular, the viscosity solution has all properties established above for the minimax solution. On the other hand it is worthy of noting that the problem of the equivalence of conditions $(3.\mathrm{MS}.1_-)$ and $(3.\mathrm{VS}.1_-)$ (conditions $(3.\mathrm{MS}.1_+)$ and $(3.\mathrm{VS}.1_+)$, respectively), which are involved in the definitions of minimax and viscosity solutions, remains open (for example, for functionals $\varphi \in \Phi_{\operatorname{C}}$). By analogy with the theory of minimax solutions, to prove the comparison principle for viscosity solutions of the Cauchy problem (3.13), (3.14) under the more general Lipschitz condition $(3.\mathrm{CP}.3)$ it is natural to attempt to construct an appropriate penalty functional $\nu$ starting from the ci-smooth functional $\gamma$ defined by (3.47). For this purpose, we need to ‘extend’ the definition of this functional to the case of doubled variables $(t, x(\,{\cdot}\,))$ and $(\tau, y(\,{\cdot}\,))$. The most direct way to this ‘extension’ leads to the functional
$$
\begin{equation}
\begin{aligned} \, \nonumber & \nu(t, x(\,{\cdot}\,), \tau, y(\,{\cdot}\,)) \\ & \qquad =\frac{\bigl(\|x(\cdot \wedge t)-y(\cdot \wedge \tau)\|_\infty^2- \|x(t)-y(\tau)\|^2\bigr)^2}{\|x(\cdot \wedge t)- y(\cdot \wedge \tau)\|_\infty^2}+\|x(t)-y(\tau)\|^2 \end{aligned}
\end{equation}
\tag{3.80}
$$
for $\|x(\cdot \wedge t) - y(\cdot \wedge \tau)\|_\infty > 0$ and $\nu(t, x(\,{\cdot}\,), \tau, y(\,{\cdot}\,)) = 0$ otherwise. However, it turns out that for a fixed point $(\tau,y(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n)$, the functional
$$
\begin{equation*}
[0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \ni (t, x(\,{\cdot}\,)) \mapsto \nu(t, x(\,{\cdot}\,), \tau, y(\,{\cdot}\,)) \in \mathbb{R}
\end{equation*}
\notag
$$
may not be ci-differentiable at some points $(t, x(\,{\cdot}\,))$ for $t < \tau$ (see [69], Example 4.1). Thus we cannot use the functional $\nu$ from (3.80) directly in the scheme of the proof of Theorem 8. The approach concerning modifications of the concept of a viscosity solution in the spirit of Definition 13 was developed, for example, in [77], [10], [79], and [78]. Some attempts to cover the general Lipschitz condition $(3.\mathrm{CP}.3)$ were made in [180]. There are also rather many studies using similar technique and its further generalizations to construct a theory of viscosity solutions of hereditary Hamilton–Jacobi equations arising in control problems for stochastic time-delay systems. We limit ourselves to referring to [139], [170], [34], and [35]. We also note the paper [11], which is devoted to developing the vanishing viscosity method for path-dependent Hamilton–Jacobi equations. In addition, several works proposing other definitions of viscosity solutions of path-dependent Hamilton–Jacobi equations, which are more consistent with the definition of a viscosity solution of classical Hamilton–Jacobi equations (see Definition 6), appeared recently. In the papers in the first group (see, for example, [178], [179], [182], [142], and [144]) a transition from the space of continuous functions to the wider space of piecewise continuous functions is made, and only functionals satisfying certain additional continuity requirements and/or growth estimates are under consideration. This makes it possible to prove the uniqueness of the corresponding relevant viscosity solutions using methods of, in fact, finite-dimensional optimization. In works in the second group (see, for example, [23], [181], [24], and [183]), to substantiate the uniqueness of a viscosity solution, special smooth variational principles are proved (see § 3.9), where the smoothness is understood precisely in the sense of ci-differentiation or its analogues. These principles are based on an abstract variant of the Borwein–Preiss variational principle in complete metric spaces ([106], Theorem 1; also see, for example, [15], Theorem 2.5.2) and an appropriate choice of a smooth ‘gauge-type’ functional in an appropriate way. In particular, the functional $\gamma$ from (3.47) was used for these purposes in [23], [181], and [183]. To conclude this subsection, we consider another approach to the construction of the theory of viscosity solutions of the Cauchy problem (3.13), (3.14) in which the definition of a viscosity solution is based on appropriate analogues of known criteria of viscosity solutions of the classical Hamilton–Jacobi equations in the form of inequalities (2.37) and (2.38) for sub- and supergradients. In this case, to verify the existence and uniqueness the equivalence of relevant definitions of minimax and viscosity solutions is established and then the results from the theory of minimax solutions (see Theorem 3) are used. Following [123], § 14, we define the ci-sub- and ci-superdifferentials of a functional $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ at a point $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ by
$$
\begin{equation*}
\begin{gathered} \, D_- \varphi(t, x(\,{\cdot}\,)) = \biggl\{(a, s) \in \mathbb{R} \times \mathbb{R}^n \colon \forall\,y(\,{\cdot}\,) \in \operatorname{Lip}(t,x(\,{\cdot}\,)) \\ \quad\;\;\, \liminf_{\tau \to t^+}\frac{\varphi(\tau, y(\,{\cdot}\,)) - \varphi(t, x(\,{\cdot}\,)) - a(\tau - t) - \langle s, y(\tau)\! - x(t) \rangle}{\tau - t}\geqslant 0 \biggr\}\end{gathered}
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
\begin{gathered} \, D_+ \varphi(t, x(\,{\cdot}\,)) = \biggl\{ (a, s) \in \mathbb{R} \times \mathbb{R}^n \colon \forall\,y(\,{\cdot}\,) \in \operatorname{Lip}(t, x(\,{\cdot}\,)) \\ \quad\;\;\, \limsup_{\tau \to t^+}\frac{\varphi(\tau, y(\,{\cdot}\,)) - \varphi(t, x(\,{\cdot}\,)) - a(\tau - t) - \langle s, y(\tau)\! - x(t) \rangle}{\tau - t}\leqslant 0 \biggr\}. \end{gathered}
\end{equation*}
\notag
$$
Note that if $\varphi$ is ci-differentiable at $(t,x(\,{\cdot}\,))$, then
$$
\begin{equation*}
D_- \varphi (t, x(\,{\cdot}\,)) = \bigl\{ (a, s) \in \mathbb{R} \times \mathbb{R}^n \colon a\leqslant \partial_t \varphi(t, x(\,{\cdot}\,)),\, s= \nabla \varphi(t, x(\,{\cdot}\,)) \bigr\},
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
D_+ \varphi(t, x(\,{\cdot}\,)) = \bigl\{ (a, s) \in \mathbb{R} \times \mathbb{R}^n \colon a\geqslant \partial_t \varphi(t, x(\,{\cdot}\,)),\, s= \nabla \varphi(t, x(\,{\cdot}\,)) \bigr\}.
\end{equation*}
\notag
$$
Definition 14. A viscosity solution of the Cauchy problem (3.13), (3.14) is a functional $\varphi \in \Phi_{\operatorname{C}}$ satisfying the boundary condition (3.14) and the differential inequalities and The following result is true (see [123], Proposition 14.1, and also [69], Lemma 4.4). Proposition 6. Assume that conditions $(3.\mathrm{CP}.1)$ and $(3.\mathrm{CP}.2)$ hold and the functional (3.15) is non-anticipating for each $s \in \mathbb{R}^n$. Then for any functional $\varphi \in \Phi_{\operatorname{LSC}}$, inequality (3.52) implies (3.81). In a similar way, for any functional $\varphi \in \Phi_{\operatorname{USC}}$ inequality (3.53) implies (3.82). As a consequence, in accordance with Theorem 5, we deduce that a minimax solution of the Cauchy problem (3.13), (3.14) is a viscosity solution of this problem in the sense of Definition 14. The question of whether the converse is true is more subtle and has not been exhaustively studied. An answer to this question was partially given in [130], [142], and [144]. In that case it was necessary to switch from the space $\operatorname{C}([- h, T]; \mathbb{R}^n)$ to the space of piecewise continuous functions and to impose additional restrictions on the class of functionals $\varphi$ under consideration, among other things. Below we dwell more thoroughly on the results in this area [69], in which it is not required to extend the space $\operatorname{C}([- h, T]; \mathbb{R}^n)$. We consider the following properties of lower and upper semicontinuity of a functional $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$, which are expressed in terms of the functional $\nu$ defined by (3.65).
The set of functionals $\varphi$ that have property $(3.\mathrm{LSC})$ (property $(3.\mathrm{USC})$) is denoted by $\Phi_{\operatorname{LSC}}^\ast$ ($\Phi_{\operatorname{USC}}^\ast$, respectively). Note that any $\varphi \in \Phi_{\operatorname{LSC}}^\ast$ is automatically non-anticipating (in this connection see [24], Remark 2.1) and lower semicontinuous (with respect to the metric $\operatorname{dist}$ defined by (3.3)), that is, $\Phi_{\operatorname{LSC}}^\ast \subset \Phi_{\operatorname{LSC}}$. In a similar way $\Phi_{\operatorname{USC}}^\ast \subset \Phi_{\operatorname{USC}}$. The following assertion is valid [69], Theorem 2.1. Theorem 9. Assume that conditions $(3.\mathrm{CP}.1)$ and $(3.\mathrm{CP}.2)$ hold and the functional (3.15) is non-anticipating for each $s \in \mathbb{R}^n$. Then if a functional $\varphi \in \Phi_{\operatorname{LSC}}^\ast$ satisfies the differential inequality (3.81), then it satisfies condition $(3.\mathrm{MS}.1_-)$. In a similar way, for a functional $\varphi \in \Phi_{\operatorname{USC}}^\ast$ the differential inequality (3.82) implies the fulfillment of the condition $(3.\mathrm{MS}.1_+)$. Thus, for a functional $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ that has the additional continuity properties $(3.\mathrm{LSC})$ and $(3.\mathrm{USC})$, Definition 10 of a minimax solution and Definition 14 of a viscosity solution of the Cauchy problem (3.13), (3.14) are equivalent. As a consequence, we immediately derive from Theorem 3 that under conditions $(3.\mathrm{CP}.1)$–$(3.\mathrm{CP}.3)$ the viscosity solution is unique in the class of functionals $\Phi_{\operatorname{LSC}}^\ast \cap \Phi_{\operatorname{USC}}^\ast$. Furthermore, we can show (in this connection see [69], § 3.3, for optimal control problems) that $\rho \in \Phi_{\operatorname{LSC}}^\ast \cap \Phi_{\operatorname{USC}}^\ast$ for the value functional $\rho$ of the differential game (3.16), (3.17) if the following conditions hold in addition to $(3.\mathrm{DG}.1)$, $(3.\mathrm{DG}.2)$, and $(3.\mathrm{DG}.4)$.
Condition $(3.\mathrm{DG}.7)$ is stronger than $(3.\mathrm{DG}.3)$; it makes it possible to cover only the cases of distributed and constant concentrated delays (see (1.21) and (1.23)), while condition $(3.\mathrm{DG}.8)$ ensures the continuity of the functional $\sigma$. As a result, from Theorem 6 we conclude that, under the above conditions, the Cauchy problem (3.13), (3.14) with Hamiltonian (3.22) has a viscosity solution in the sense of Definition 14 which belongs to the set $\Phi_{\operatorname{LSC}}^\ast \cap \Phi_{\operatorname{USC}}^\ast$, is unique, and coincides with the value functional $\rho$. We comment on the proof of the first part of Theorem 9. Since, in general, the reasoning is carried out in accordance with the scheme of the verification of a similar result for the classical Hamilton-Jacobi equations (see, for example, [162], Theorem 4.3), we dwell only on the most significant distinctions. For $\varphi \colon [0,T] \times \operatorname{C}([-h,T]; \mathbb{R}^n) \to \mathbb{R}$, $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([-h,T];\mathbb{R}^n)$, and $F \in \mathcal{K}(\mathbb{R}^n)$ we consider the quantity (in this connection also see [20], § 4)
$$
\begin{equation*}
\begin{aligned} \, & d_-^\ast\{ \varphi(t, x(\,{\cdot}\,)) \mid F \} \\ &\qquad=\lim_{\delta \to 0^+}\,\inf\biggl\{\frac{\varphi(\tau,y(\,{\cdot}\,))- \varphi(t, x(\,{\cdot}\,))}{\tau - t} \colon\tau \in (t, t + \delta], \, y(\,{\cdot}\,) \in \Omega(t, x(\,{\cdot}\,), F, \delta) \biggr\}, \end{aligned}
\end{equation*}
\notag
$$
where the set $\Omega(t, x(\,{\cdot}\,), F, \delta)$ is defined by (3.51). Along with $d_-\{\varphi(t,x(\,{\cdot}\,)) \mid F\}$ in (3.50), the quantity $d_-^\ast \{ \varphi(t, x(\,{\cdot}\,)) \mid F \}$ can also be interpreted as some lower right derivative of $\varphi$ at $(t, x(\,{\cdot}\,))$ in the multivalued direction $F$. By analogy with Theorem 5 we can verify that for a functional $\varphi \in \Phi_{\operatorname{LSC}}$ to satisfy condition $(3.\mathrm{MS}.1_-)$ it suffices that it satisfies the differential inequality
$$
\begin{equation}
\begin{gathered} \, d_-^\ast \bigl\{ \varphi(t, x(\,{\cdot}\,)) - \langle s, x(t) \rangle \mid B(c (1 + \|x(\cdot \wedge t)\|_\infty)) \bigr\} + H(t, x(\,{\cdot}\,), s) \leqslant 0, \\ (t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{C}([-h,T];\mathbb{R}^n),\quad s \in \mathbb{R}^n. \end{gathered}
\end{equation}
\tag{3.83}
$$
A key role in substantiating the implication from (3.81) to (3.83) for $\varphi \in \Phi_{\operatorname{LSC}}^\ast$ is played by the following property of the ci-subdifferential ([69], Lemma 4.3; this is an analogue of a property of the subdifferential in [161], Theorem 1.1). Lemma 2. Assume that $\varphi \in \Phi_{\operatorname{LSC}}^\ast$, $(t_0,x_0(\,{\cdot}\,)) \in [0,T) \times \operatorname{C}([-h,T];\mathbb{R}^n)$, $F \in \mathcal{K}(\mathbb{R}^n)$, and Then for each positive number $\varepsilon $ there are $(t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{C}([-h,T];\mathbb{R}^n)$ and $(a, s) \in D_-\varphi (t, x(\,{\cdot}\,))$ such that $\operatorname{dist}((t, x(\,{\cdot}\,)), (t_0, x_0(\,{\cdot}\,))) \leqslant \varepsilon$ and Lemma 2 is proved using techniques related to the substantiation of multidimensional non-smooth generalizations of the finite increment formula [19], [20], and [86] (also see [162], § A6, and [21], Chap. 3, Theorem 2.6) and partially adapted to the context of applications to Hamilton–Jacobi equations with ci-derivatives in [142] and [144]. Among other things, constructions of penalty functionals based on the functional $\nu$ from (3.65) are used again. Note that this does not make it possible to abandon fully the additional assumption $(3.\mathrm{LSC})$ of lower semicontinuity. In addition, in comparison with the finite-dimensional case, the distinction is that there, like in [20], [21], and [86], we must use smooth variational principles. With this aim in view, the following fact was proved in [69], Lemma 4.1. Lemma 3. Assume that a non-empty bounded closed set $W \subset [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ such that $t < T$ for all $(t, x(\,{\cdot}\,)) \in W$ and a non-anticipating lower semicontinuous functional $\varphi \colon W \to \mathbb{R}$ bounded below are fixed. Then for each $\varepsilon > 0$ there exists a non-anticipating functional $\psi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ and a point $(t_\ast, x_\ast(\,{\cdot}\,)) \in W$ such that
Proving Lemma 3 is in its turn based on an abstract variant of the Borwein–Preiss variational principle ([106], Theorem 1). In this case the gauge-type functional
$$
\begin{equation*}
\begin{gathered} \, \omega(t, x(\,{\cdot}\,), \tau, y(\,{\cdot}\,))= \begin{cases} (t - \tau)^2 + \gamma(t, x(\,{\cdot}\,) - y(\cdot \wedge \tau)) & \text{if}\ t \geqslant \tau, \\ a & \text{if}\ t < \tau, \end{cases} \\ (t, x(\,{\cdot}\,)), (\tau, y(\,{\cdot}\,)) \in [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n), \end{gathered}
\end{equation*}
\notag
$$
is used, where $\gamma$ is the functional (3.47) and the positive number $a$ is chosen in an appropriate way based on $T$ and $W$. In particular, the sought-for functional $\psi$ has the form of a series:
$$
\begin{equation*}
\psi(t, x(\,{\cdot}\,))= \delta \sum_{i = 0}^{\infty} \frac{1}{2^i}\, \omega(t, x(\,{\cdot}\,), \tau_i, y_i(\,{\cdot}\,)),\qquad (t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n),
\end{equation*}
\notag
$$
for appropriately chosen $\delta > 0$ and points $(\tau_i, y_i(\,{\cdot}\,)) \in W$, $i \in \mathbb{N} \cup \{0\}$. Finally, comparing Definitions 13 and 14 of viscosity solutions of the Cauchy problem (3.13), (3.14) we note that, for example, the differential inequality (3.81) can equivalently be expressed as follows in terms of smooth underlying functionals ([69], Proposition 3.3). Proposition 7. Let $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ be a non-anticipating functional. Then $\varphi$ satisfies the differential inequality (3.81) if and only if the following condition holds: for any point $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ and each ci-smooth functional $\psi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$, if for each function $y(\,{\cdot}\,) \in \operatorname{Lip}(t, x(\,{\cdot}\,))$ there is $\delta \in (0, T - t]$ such that Nevertheless, the problem of the relationship between condition $(3.\mathrm{VS}.1_-)$ and inequality (3.81) (or between condition $(3.\mathrm{VS}.1_+)$ and inequality (3.82)) remains open. 3.9. Additional commentsIn this subsection we discuss briefly the relationship between the formalization used in the paper and some other possible approaches (concerning the choice of a function space and the derivatives used) developed in the theory of path-dependent Hamilton–Jacobi equations. Recall that in this paper, following, for example, [120], [119], [121], [122], and [67], we consider the Cauchy problem (3.13), (3.14) in the set $[0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ and assume that the unknown functional
$$
\begin{equation*}
\varphi \colon [0,T] \times\operatorname{C}([-h,T];\mathbb{R}^n) \to \mathbb{R}
\end{equation*}
\notag
$$
is non-anticipating and at least continuous with respect to the metric $\operatorname{dist}$ defined by (3.3). Another widely used approach (see, for example, [139], [34], [35], [10], [23], [11], and [24]) consists in considering the pseudometric
$$
\begin{equation}
\operatorname{dist}_0\bigl((t_1,x_1(\,{\cdot}\,)),(t_2,x_2(\,{\cdot}\,))\bigr)= |t_1-t_2|+\|x_1(\cdot \wedge t_1)-x_2(\cdot \wedge t_2)\|_\infty
\end{equation}
\tag{3.84}
$$
in the set $[0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n)$. Note that a functional $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ is continuous with respect to the pseudometric $\operatorname{dist}_0$ if and only if it is non- anticipating and continuous with respect to $\operatorname{dist}$. Thus, the above results do not change if the Cauchy problem (3.13), (3.14) is considered on the set $[0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ with pseudometric $\operatorname{dist}_0$, without indicating the non-anticipation property. Furthermore, applying the standard procedure for switching from a pseudometric space to the induced metric space of equivalence classes (see, for example, [80], Chap. 4, Theorem 15), we can switch from the space $[0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n)$ with pseudometric $\operatorname{dist}_0$ to the space
$$
\begin{equation*}
\operatorname{G}= \bigcup_{t \in [0, T]} \bigl( \{t\} \times \operatorname{C}([-h,t];\mathbb{R}^n)\bigr)=\bigl\{(t,r(\,{\cdot}\,)) \colon t \in [0,T], \, r(\,{\cdot}\,) \in \operatorname{C}([-h,t];\mathbb{R}^n)\bigr\}
\end{equation*}
\notag
$$
with metric
$$
\begin{equation*}
\operatorname{dist}_\ast\bigl((t_1,r_1(\,{\cdot}\,)),(t_2,r_2(\,{\cdot}\,))\bigr) = |t_1 - t_2| + \|r_1(\cdot \wedge t_1) - r_2(\cdot \wedge t_2)\|_\infty.
\end{equation*}
\notag
$$
In this case, a non-anticipating functional $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ can be identified with the functional $\psi \colon \operatorname{G} \to \mathbb{R}$ defined by
$$
\begin{equation*}
\psi(t, r(\,{\cdot}\,))= \varphi(t, r(\cdot \wedge t)),\qquad (t, r(\,{\cdot}\,)) \in \operatorname{G}.
\end{equation*}
\notag
$$
Then we conclude that $\varphi$ is continuous with respect to the pseudometric $\operatorname{dist}_0$ (or, which is the same, is non-anticipating and continuous in the metric $\operatorname{dist}$) if and only if $\psi$ is continuous in the metric $\operatorname{dist}_\ast$. Hence the Cauchy problem (3.13), (3.14) can also be reformulated in the space $\operatorname{G}$ with metric $\operatorname{dist}_\ast$, when the Hamiltonian $H$ is defined on triples $(t, r(\,{\cdot}\,), s)$ and the required functional acts from $\operatorname{G}$ to $\mathbb{R}$. Such a formalization was used, for example, in [77], [79], [180], [78], [69], and [183]. We also note that, for example, in [97], [111]–[115], [118], and [123], the Cauchy problem (3.13), (3.14) was considered on the set $\operatorname{G}$ but with another metric, namely, the metric
$$
\begin{equation*}
\begin{aligned} \, & \operatorname{dist}_{\textrm{H}} \bigl( (t_1, r_1(\,{\cdot}\,)), (t_2, r_2(\,{\cdot}\,)) \bigr) \\ & \qquad = \max \bigl\{ \operatorname{dist}_{\textrm{H}}^\ast \bigl( (t_1, r_1(\,{\cdot}\,)), (t_2, r_2(\,{\cdot}\,)) \bigr), \operatorname{dist}_{\textrm{H}}^\ast \bigl( (t_2, r_2(\,{\cdot}\,)), (t_1, r_1(\,{\cdot}\,)) \bigr) \bigr\}, \end{aligned}
\end{equation*}
\notag
$$
where
$$
\begin{equation*}
\begin{aligned} \, & \operatorname{dist}_{\textrm{H}}^\ast \bigl( (t_1, r_1(\,{\cdot}\,)), (t_2, r_2(\,{\cdot}\,)) \bigr) \\ & \qquad = \max_{\tau_1 \in [- h, t_1]}\,\min_{\tau_2 \in [- h, t_2]} \bigl( |\tau_1 - \tau_2|^2 + \|r_1(\tau_1) - r_2(\tau_2)\|^2 \bigr)^{1 / 2}. \end{aligned}
\end{equation*}
\notag
$$
At its core, this metric is the Hausdorff distance between the graphs of continuous functions $r_1 \colon [- h, t_1] \to \mathbb{R}^n$ and $r_2 \colon [- h, t_2] \to \mathbb{R}^n$ as compact subsets of $\mathbb{R} \times \mathbb{R}^{n}$. According, for example, to [153], Chap. 2, the metrics $\operatorname{dist}_{\textrm{H}}$ and $\operatorname{dist}_\ast$ are not equivalent in the strong sense but induce the same topology in $\operatorname{G}$. This means that if $(t_i, r_i(\,{\cdot}\,)) \in \operatorname{G}$, $i \in \mathbb{N} \cup \{0\}$, then $\operatorname{dist}_{\textrm{H}} ((t_i, r_i(\,{\cdot}\,)), (t_0, r_0(\,{\cdot}\,))) \to 0$ as $i \to \infty$ if and only if $\operatorname{dist}_\ast((t_i,r_i(\,{\cdot}\,)),(t_0,r_0(\,{\cdot}\,))) \to 0$ as $i \to \infty$. Correspondingly, any functional $\psi\colon\operatorname{G} \to \mathbb{R}$ is continuous with respect to $\operatorname{dist}_{\textrm{H}}$ if and only if it is continuous with respect to $\operatorname{dist}_\ast$. These arguments make it possible to link the results presented in our survey with the ones in the above studies. The Hamilton–Jacobi equation (3.13) under consideration employs coinvariant (ci-) derivatives, which are defined by (3.6). As already noted above, the very concept of invariant and coinvariant derivatives of functionals was introduced in [83]; the relevant technique of invariant differential calculus and its applications to problems in the theory of functional differential equations were discussed in detail, for example, in [84] and [85]. It is worthy of noting that functionals of the form
$$
\begin{equation*}
[0,T] \times \mathbb{R}^n \times \operatorname{PC}([-h,T];\mathbb{R}^n) \ni (t,z,x(\,{\cdot}\,)) \mapsto \Phi(t,z,x(\,{\cdot}\,)) \in \mathbb{R}
\end{equation*}
\notag
$$
were studied in [84] and [85]. Here $\operatorname{PC}([- h, T]; \mathbb{R}^n)$ is the set of piecewise continuous (from the right) functions $x \colon [-h, T] \to \mathbb{R}^n$. For such a functional $\Phi$ the ci-derivative $\partial \Phi(t, z, x(\,{\cdot}\,)) \in \mathbb{R}$ at a point $(t, z, x(\,{\cdot}\,)) \in [0, T) \times \mathbb{R}^n \times \operatorname{PC}([- h, T]; \mathbb{R}^n)$ is defined by
$$
\begin{equation*}
\partial \Phi(t, z, x(\,{\cdot}\,))= \lim_{\delta \to 0^+} \frac{\Phi(t + \delta, z, y(\,{\cdot}\,))-\Phi(t, z, x(\,{\cdot}\,))}{\delta},
\end{equation*}
\notag
$$
provided that this limit exists and is independent of $y(\,{\cdot}\,)\,{\in}\operatorname{Lip}(t,z,x(\,{\cdot}\,))$, where $\operatorname{Lip}(t, z, x(\,{\cdot}\,))$ is the set of functions $y(\,{\cdot}\,) \in \operatorname{PC}([- h, T]; \mathbb{R}^n)$ such that $y(\tau)=x(\tau)$, $\tau \in [-h,t)$, $y(t)=z$, and $y(\,{\cdot}\,)$ is a Lipschitz function in $[t, T]$. With respect to the finite-dimensional variable $z$ the ordinary gradient $\partial \Phi(t,z,x(\,{\cdot}\,)) / \partial z \in \mathbb{R}^n$ is considered. Note that in this approach $\Phi$ must be defined at piecewise continuous functions $x(\,{\cdot}\,)$ and, unlike (3.6), it cannot be extended directly to functionals defined only at continuous functions $x(\,{\cdot}\,)$. Nevertheless, considering, for a functional $\Phi$, the functional
$$
\begin{equation*}
\varphi(t, x(\,{\cdot}\,))= \Phi(t, x(t), x(\,{\cdot}\,)),\qquad (t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n),
\end{equation*}
\notag
$$
we have the equalities
$$
\begin{equation*}
\partial_t\varphi(t,x(\,{\cdot}\,))=\partial\Phi(t,x(t),x(\,{\cdot}\,)),\quad \nabla\varphi(t,x(\,{\cdot}\,))= \frac{\partial\Phi}{\partial z}(t,x(t),x(\,{\cdot}\,))
\end{equation*}
\notag
$$
under rather natural assumptions. In view of the above relationship the quantities $\partial_t \varphi(t, x(\,{\cdot}\,))$ and $\nabla \varphi(t, x(\,{\cdot}\,))$ defined by (3.6) are called ci-derivatives as before. We also note that a close approach to differentiation of non-anticipating functionals was proposed in [5], where the corresponding derivatives were called Clio- derivatives. It must be noted separately that path-dependent Hamilton–Jacobi equations are often (see, for example, [170], [180], [181], [23], [24], and [183]) considered with horizontal and vertical derivatives [33] (see also [22]). We give the corresponding definitions. Let $\operatorname{D}([- h, T]; \mathbb{R}^n)$ be the space of so-called càdlàg functions $x \colon [- h, T] \to \mathbb{R}^n$ (that is, right-continuous functions that have a limit from the left at each point $t \in (-h,T]$) with the norm
$$
\begin{equation*}
\|x(\,{\cdot}\,)\|_\infty= \sup_{\tau \in [- h, T]} \|x(\tau)\|,\qquad x(\,{\cdot}\,) \in \operatorname{D}([- h, T]; \mathbb{R}^n).
\end{equation*}
\notag
$$
We consider the set $[0, T] \times \operatorname{D}([- h, T]; \mathbb{R}^n)$ endowed with the pseudometric $\operatorname{dist}_0$ defined by (3.84). The horizontal derivative $\partial^{\rm H}\Phi(t,x(\,{\cdot}\,)) \in \mathbb{R}$ of a functional $\Phi \colon [0,T] \times \operatorname{D}([-h,T];\mathbb{R}^n) \to \mathbb{R}$ at a point $(t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{D}([-h,T]; \mathbb{R}^n)$ is defined by
$$
\begin{equation*}
\partial^{\rm H} \Phi(t, x(\,{\cdot}\,))= \lim_{\delta \to 0^+} \frac{\Phi(t+\delta, x(\cdot \wedge t))-\Phi(t,x(\,{\cdot}\,))}{\delta}\,.
\end{equation*}
\notag
$$
The vertical derivative of $\Phi$ at a point $(t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{D}([-h,T]; \mathbb{R}^n)$ is defined as the vector $\partial^{\rm V} \Phi(t, x(\,{\cdot}\,)) = (\partial_1^{\rm V} \Phi(t, x(\,{\cdot}\,)), \dots, \partial_n^{\rm V} \Phi(t, x(\,{\cdot}\,))) \in \mathbb{R}^n$, where
$$
\begin{equation*}
\partial_i^{\rm V} \Phi(t, x(\,{\cdot}\,))= \lim_{\delta \to 0} \frac{\Phi(t, x(\,{\cdot}\,) + \delta e_i 1_{[t, T]}(\,{\cdot}\,)) - \Phi(t, x(\,{\cdot}\,))}{\delta}\,,\qquad i \in \{1,\dots, n\}.
\end{equation*}
\notag
$$
Here $e_i \in \mathbb{R}^n$, $i \in \{1,\dots, n\}$, is the standard orthonormal basis in the space $\mathbb{R}^n$ and $1_{[t, T]}(\,{\cdot}\,) \in \operatorname{D}([- h, T]; \mathbb{R})$ is the characteristic function of the interval $[t, T]$. We note again that vertical derivatives cannot be defined directly for functionals $\varphi \colon [0, T] \times \operatorname{C}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ (in this connection see, for example, [24]). The following result is an analogue of Proposition 1 (see, for example, [24], Theorem 2.1, and also [180], Theorem 2.6). Proposition 8. Assume that a continuous functional $\Phi \colon [0, T] \times \operatorname{D}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ has a horizontal derivative $\partial^{\rm H} \Phi(t, x(\,{\cdot}\,))$ at each point $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{D}([- h, T]; \mathbb{R}^n)$ and a vertical derivative $\partial^{\rm V} \Phi(t, x(\,{\cdot}\,))$ at each point $(t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{D}([-h,T];\mathbb{R}^n)$ such that the mappings As a consequence, if a functional $\Phi \colon [0, T] \times \operatorname{D}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ satisfies the assumptions of Proposition 8, then the restriction $\varphi$ of this functional to the space $[0,T] \times \operatorname{C}([-h,T];\mathbb{R}^n)$ is ci-smooth and, in addition,
$$
\begin{equation*}
\partial_t \varphi(t, x(\,{\cdot}\,))= \partial^{\rm H} \Phi(t, x(\,{\cdot}\,)),\quad \nabla \varphi(t,x(\,{\cdot}\,))=\partial^{\rm V} \Phi(t, x(\,{\cdot}\,))
\end{equation*}
\notag
$$
for all $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{C}([- h, T]; \mathbb{R}^n)$. This makes it possible to compare the results described in this survey with ones in the theory of Hamilton–Jacobi equations with horizontal and vertical derivatives. We also note that there is a certain relationship between Frechét derivatives and horizontal and vertical derivatives for non-anticipating functionals (see, for example, [75]). Nevertheless, in our opinion it remains unclear how the theory of Hamilton–Jacobi equations in infinite-dimensional spaces with Frechét derivatives can be applied to the Cauchy problem (3.13), (3.14) under the assumptions $(3.\mathrm{CP}.1)$–$(3.\mathrm{CP}.3)$. Finally, we note that when derivatives of non-anticipating functionals are introduced in one or another way, it is important to have a formula of the form (3.11) or (3.85). Therefore, definitions of relevant derivatives are sometimes (see, for example, [139], [34], [35], and [10]) based on formulae of this type directly. 4. Neutral-type systemsIn this section we touch upon the development of the theory of Hamilton–Jacobi equations for neutral-type systems (1.24), which are more general than time-delay systems (1.20). We consider a differential game such that the motion of the corresponding dynamical system is described by neutral-type functional differential equations in the Hale form (see, for example, [70], [2], [88], and [71])
$$
\begin{equation}
\begin{gathered} \, \frac{{\rm d}}{{\rm d} \tau} \bigl( y(\tau) - g(\tau, y(\,{\cdot}\,)) \bigr)= f(\tau, y(\,{\cdot}\,), u(\tau), v(\tau)), \\ \tau \in [0, T], \quad y(\tau) \in \mathbb{R}^n, \quad u(\tau) \in P, \quad v(\tau) \in Q, \end{gathered}
\end{equation}
\tag{4.1}
$$
with the initial condition (3.5) specified by the initial position $(t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n)$. The aim of the first (second) player is to minimize (maximize, respectively) the quality index
$$
\begin{equation}
J= \sigma(y(\,{\cdot}\,)) - \int_{t}^{T}\chi(\tau,y(\,{\cdot}\,), u(\tau), v(\tau)) \, {\rm d} \tau.
\end{equation}
\tag{4.2}
$$
A distinction from the differential game (3.16), (3.17) for a time-delay system is that the subtrahend $g(\tau, y(\,{\cdot}\,))$ occurs on the left-hand side of the equation of motion (4.1). Since this subtrahend is under the derivative sign, the function $x(\,{\cdot}\,)$ in the initial position $(t, x(\,{\cdot}\,))$ must be Lipschitz continuous. In this connection we assume that the mapping $g \colon [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n) \to \mathbb{R}^n$ satisfies the following local Lipschitz condition.
Throughout what follows
$$
\begin{equation*}
\begin{aligned} \, Z_\mu& = \bigl\{ y(\,{\cdot}\,) \in \operatorname{Lip}([- h, T]; \mathbb{R}^n) \colon \\ & \qquad \|y(\,{\cdot}\,)\|_\infty \leqslant \mu, \, \|y(\tau_1) - y(\tau_2)\|\leqslant \mu |\tau_1 - \tau_2|,\, \tau_1, \tau_2 \in [- h, T] \bigr\} \end{aligned}
\end{equation*}
\notag
$$
and $\|\cdot\|_\infty$ is the uniform norm in the space $\operatorname{Lip}([- h, T]; \mathbb{R}^n)$ (see (3.4)). It follows from $(4.\mathrm{DG}.1)$ that the map $g$ is non-anticipating. In addition, $g(\tau, y(\,{\cdot}\,))$ is independent of the values of $y(\,{\cdot}\,)$ on the interval $(\tau - h_0, T]$ and, in particular, of $y(\tau)$, which implies automatically that equation (4.1) becomes solved with respect to the derivative $\dot{y}(\tau)$. Note that one or another condition of this type is rather often imposed when neutral-type functional differential equations are considered (see, for example, [70] and [71], § 2.7, for the condition of atomicity at zero, or [2], § 1.2, for the condition of strict throw-back in derivative). We also assume that the following conditions hold.
We let $X_\ast$ denote the set of points $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{Lip}([- h, T]; \mathbb{R}^n)$ at which all coordinates $g_i \colon [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$, $i \in \{1,\dots, n\}$, of the map $g = (g_1,\dots,g_n)$ in (4.1) are ci-differentiable and set
$$
\begin{equation*}
\partial_t g(t, x(\,{\cdot}\,))= \bigl( \partial_t g_1 (t, x(\,{\cdot}\,)),\dots, \partial_t g_n (t, x(\,{\cdot}\,)) \bigr)\in \mathbb{R}^n,\qquad (t, x(\,{\cdot}\,)) \in X_\ast.
\end{equation*}
\notag
$$
By condition $(4.\mathrm{DG}.1)$, for each function $x(\,{\cdot}\,) \in \operatorname{Lip}([- h, T]; \mathbb{R}^n)$, for almost every $t \in [0, T)$ we have
$$
\begin{equation*}
(t, x(\,{\cdot}\,)) \in X_\ast,\quad \partial_tg(t,x(\,{\cdot}\,))=\frac{{\rm d}}{{\rm d}t}g(t,x(\,{\cdot}\,)),\quad \nabla g_i (t, x(\,{\cdot}\,))= 0, \ \ i \in \{1,\dots, n\}.
\end{equation*}
\notag
$$
For example, let $n = 1$, and let
$$
\begin{equation*}
g(\tau, y(\,{\cdot}\,))= \widehat{g}(\tau, y(\tau - h)),\qquad (\tau, y(\,{\cdot}\,)) \in [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}),
\end{equation*}
\notag
$$
for some function $\widehat{g} \colon [0, T] \times \mathbb{R} \to \mathbb{R}$, which corresponds to the case of a constant concentrated delay on the left-hand side of system (4.1). If $\widehat{g}$ is differentiable and $\partial \widehat{g}(\tau, y) / \partial y \ne 0$ for all $(\tau, y) \in [0, T] \times \mathbb{R}$, then the set $X_\ast$ consists of pairs $(t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{Lip}([-h,T];\mathbb{R})$ such that $x(\,{\cdot}\,)$ has a right derivative $\dot{x}^+ (t - h)$ at the point $t - h$. In this case we have
$$
\begin{equation}
\partial_t g(t, x(\,{\cdot}\,))= \frac{\partial \widehat{g}}{\partial \tau} (t, x(t - h))+ \frac{\partial \widehat{g}}{\partial y} (t, x(t - h)) \dot{x}^+(t - h).
\end{equation}
\tag{4.3}
$$
We associate the differential game (4.1), (4.2) with the Cauchy problem for the Hamilton–Jacobi equation with ci-derivatives
$$
\begin{equation}
\begin{gathered} \, \partial_t \varphi(t, x(\,{\cdot}\,))+ \langle \partial_t g(t, x(\,{\cdot}\,)),\nabla\varphi(t,x(\,{\cdot}\,))\rangle+ H \bigl( t, x(\,{\cdot}\,), \nabla \varphi(t, x(\,{\cdot}\,)) \bigr)= 0, \\ (t, x(\,{\cdot}\,)) \in X_\ast, \end{gathered}
\end{equation}
\tag{4.4}
$$
with the right-end boundary condition
$$
\begin{equation}
\varphi(T, x(\,{\cdot}\,))= \sigma(x(\,{\cdot}\,)),\qquad x(\,{\cdot}\,) \in \operatorname{Lip}([- h, T]; \mathbb{R}^n).
\end{equation}
\tag{4.5}
$$
Here, by analogy with (3.22), the Hamiltonian $H$ is defined by
$$
\begin{equation}
\begin{gathered} \, \begin{aligned} \, H(t, x(\,{\cdot}\,),s)& = \max_{v \in Q}\,\min_{u \in P} \bigl( \langle s, f(t, x(\,{\cdot}\,), u, v) \rangle - \chi(t, x(\,{\cdot}\,), u, v) \bigr) \\ & = \min_{u \in P}\,\max_{v \in Q} \bigl( \langle s, f(t, x(\,{\cdot}\,), u, v) \rangle - \chi(t, x(\,{\cdot}\,), u, v) \bigr), \end{aligned} \\ (t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n), \quad s \in \mathbb{R}^n. \end{gathered}
\end{equation}
\tag{4.6}
$$
The main distinction of (4.4) from the Hamilton–Jacobi equations (3.13), which correspond to control problems for time-delay systems, is the occurrence of the new term $\langle \partial_t g(t, x(\,{\cdot}\,)), \nabla\varphi(t,x(\,{\cdot}\,))\rangle$. As we can see from the example of (4.3), the map $X_\ast \ni (t, x(\,{\cdot}\,)) \mapsto \partial_t g(t, x(\,{\cdot}\,)) \in \mathbb{R}^n$ is not necessarily continuous, so this term is singled out. In addition, since the map $\partial_t g$ is defined only on the set $X_\ast$, equation (4.4) itself can only be considered on this set. However, despite this fact, the unknown in the Cauchy problem (4.4), (4.5) is a non-anticipating functional $\varphi$ defined on the whole space $[0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n)$. Like in (3.19) and (3.21), at each point $(t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{Lip}([-h,T];\mathbb{R}^n)$ we define the lower and upper values $\rho_-(t,x(\,{\cdot}\,))$ and $\rho_+(t,x(\,{\cdot}\,))$ of the differential game (4.1), (4.2). If these quantities are equal, then the game has value
$$
\begin{equation*}
\rho(t, x(\,{\cdot}\,))=\rho_-(t, x(\,{\cdot}\,))=\rho_+(t, x(\,{\cdot}\,)).
\end{equation*}
\notag
$$
The following assertion is true (in this connection see [140], Theorem 2). Theorem 10. Assume that conditions $(4.\mathrm{DG}.1)$–$(4.\mathrm{DG}.3)$ hold. Then the value functional $\rho \colon [0,T] \times \operatorname{Lip}([-h,T]; \mathbb{R}^n) \to \mathbb{R}$ of the differential game (4.1), (4.2) satisfies the Hamilton–Jacobi equation (4.4) with Hamiltonian (4.6) at all points $(t,x(\,{\cdot}\,)) \in X_\ast$ where it is ci-differentiable. The specificity of neutral-type systems also manifests itself through the fact that the value functional $\rho$ is not ci-smooth even in the simplest examples (see, for example, [140], § 6). Therefore, to deduce a result similar to Theorem 2 and adequate for systems of this type we replace the assumption of the ci-smoothness of the solution $\varphi \colon [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ of the Cauchy problem (4.4), (4.5) by the following weaker conditions.
Note that conditions $(4.\mathrm{S}.1)$ and $(4.\mathrm{S}.2)$ ensure that the functional $\varphi$ and also (see § 3.1) the maps $\partial_t \varphi \colon X_\ast \to \mathbb{R}$ and $\nabla \varphi \colon X_\ast \to \mathbb{R}^n$ are non-anticipating. In addition, by virtue of $(4.\mathrm{S}.3)$ and the above properties of the set $X_\ast$ we can consider the extension
$$
\begin{equation*}
\Upsilon \colon [0, T) \times \operatorname{Lip}([- h, T]; \mathbb{R}^n) \to \mathbb{R}^n
\end{equation*}
\notag
$$
of the ci-gradient $\nabla \varphi$, defined at $(t, x(\,{\cdot}\,)) \in ([0, T) \times \operatorname{Lip}([- h, T]; \mathbb{R}^n)) \setminus X_\ast$ by continuity, that is,
$$
\begin{equation*}
\Upsilon(t, x(\,{\cdot}\,))= \lim_{\tau \to t^+} \nabla \varphi(\tau, x(\cdot \wedge t)),
\end{equation*}
\notag
$$
where the limit is calculated over the points $(\tau, x(\cdot \wedge t)) \in X_\ast$. The map $\Upsilon$ is also non-anticipating. The following result is true (in this connection see [140], Theorem 1, and also [124], Theorem 1). Theorem 11. Assume that conditions $(4.\mathrm{DG}.1)$–$(4.\mathrm{DG}.3)$ hold and there is a functional $\varphi \colon [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ that satisfies the smoothness conditions $(4.\mathrm{S}.1)$–$(4.\mathrm{S}.3)$, the Hamilton–Jacobi equation (4.4) with Hamiltonian (4.6), and the boundary condition (4.5). Then the differential game (4.1), (4.2) has value $\rho(t, x(\,{\cdot}\,)) = \varphi(t, x(\,{\cdot}\,))$, $(t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n)$, and the non-anticipating control strategies of the first and second players constructed by extremal aiming in the direction of the extension $\Upsilon$ of the ci-gradient of $\varphi$, namely, and
$$
\begin{equation*}
\begin{gathered} \, V^\circ(\tau, y(\,{\cdot}\,)) \in \operatorname*{arg\,max}_{v \in Q}\, \min_{u \in P} \bigl( \langle \Upsilon(\tau, y(\,{\cdot}\,)), f(\tau, y(\,{\cdot}\,), u, v) \rangle - \chi(\tau, y(\,{\cdot}\,), u, v) \bigr), \\ (\tau,y(\,{\cdot}\,)) \in [0,T) \times \operatorname{Lip}([-h,T];\mathbb{R}^n), \end{gathered}
\end{equation*}
\notag
$$
are optimal. It is worthy of noting that there is still no full-fledged theory of generalized (minimax, viscosity) solutions of the Cauchy problem (4.4), (4.5) under conditions $(4.\mathrm{DG}.1)$ and $(4.\mathrm{DG}.2)$ on $g$ and conditions similar to $(3.\mathrm{CP}.1)$–$(3.\mathrm{CP}.3)$ on the Hamiltonian $H$ and the boundary functional $\sigma$. First we indicate the results obtained in this area under the following stronger conditions, which make it possible to take account of neutral-type systems with constant concentrated delays in applications.
Definition 15. A minimax solution of the Cauchy problem (4.4), (4.5) is a functional $\varphi \colon [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ that is non-anticipating and continuous, satisfies the boundary condition (4.5), and has the following properties.
The multivalued mapping $E_0$ in (4.7) is defined by (3.25). We note that for $g \equiv 0$ this definition is naturally consistent with Definition 10. We also note that properties $(4.\mathrm{MS}.1_+)$ and $(4.\mathrm{MS}.1_-)$ express the $u$- and $v$-stability of the value functional of the differential game (4.1), (4.2) (see, for example, [132] and also [65] and [140]). The main result is as follows ([143], Theorem 3). Theorem 12. Under conditions $(4.\mathrm{CP}.1)$–$(4.\mathrm{CP}.3)$ a minimax solution of the Cauchy problem (4.4), (4.5) exists and is unique. The basic difficulty in the proof of Theorem 12 is in verifying the relevant comparison principle ([143], Lemma 4): if a non-anticipating lower semicontinuous functional $\varphi_+ \colon [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ with property $(4.\mathrm{MS}.1_+)$ and a non-anticipating upper semicontinuous functional $\varphi_- \colon [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ with property $(4.\mathrm{MS}.1_-)$ satisfy the estimates
$$
\begin{equation*}
\varphi_+(T, x(\,{\cdot}\,))\geqslant \sigma(x(\,{\cdot}\,))\geqslant \varphi_-(T, x(\,{\cdot}\,)),\qquad x(\,{\cdot}\,) \in \operatorname{Lip}([- h, T]; \mathbb{R}^n),
\end{equation*}
\notag
$$
then
$$
\begin{equation*}
\varphi_+(t, x(\,{\cdot}\,))\geqslant \varphi_-(t, x(\,{\cdot}\,)),\qquad (t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{Lip}([-h,T];\mathbb{R}^n).
\end{equation*}
\notag
$$
This can be verified using the scheme of the proof of Lemma 1 presented above. In this case one uses the Lyapunov–Krasovskii functional
$$
\begin{equation*}
\begin{gathered} \, \begin{aligned} \, & \nu_\varepsilon(\tau, w_1(\,{\cdot}\,), w_2(\,{\cdot}\,)) \\ & \qquad = \frac{e^{- \lambda_\ast \tau} - \varepsilon}{\varepsilon} \biggl( \bigl( \varepsilon^4 + \bigl\|w_1(\tau) - g(\tau, w_1(\,{\cdot}\,)) - \bigl( w_2(\tau) - g(\tau, w_2(\,{\cdot}\,)) \bigr) \bigr\| \bigr)^{1 / 2} \\ & \qquad\qquad + 2 \lambda_H \int_{\tau - h}^{\tau} \biggl( 1 - \frac{2 \lambda_g (\xi -\tau)}{h} \biggr) \|w_1(\xi) - w_2(\xi)\| \, {\rm d} \xi \biggr), \end{aligned} \\ \tau \in [0, T], \quad w_1(\,{\cdot}\,),w_2(\,{\cdot}\,) \in \operatorname{Lip}([-h,T];\mathbb{R}^n), \end{gathered}
\end{equation*}
\notag
$$
where $\varepsilon > 0$ is a small parameter, the positive numbers $\lambda_g $ and $\lambda_H $ correspond to a specially chosen set $W \subset \operatorname{Lip}([- h, T]; \mathbb{R}^n)$ in accordance with conditions $(4.\mathrm{CP}.1)$ and $(4.\mathrm{CP}.3)$, respectively, and $\lambda_\ast = 4 \lambda_H + 2 \lambda_g / h$. We also note ([143], Theorems 1 and 2) the consistency of the concept of minimax solution of the Cauchy problem (4.4), (4.5) with the concept of solution of this problem in the classical sense. More precisely, on the one hand, if a functional $\varphi \colon [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ satisfies the smoothness conditions $(4.\mathrm{S}.1)$–$(4.\mathrm{S}.3)$, the Hamilton–Jacobi equation (4.4), and the boundary condition (4.5), then $\varphi$ is a minimax solution of the Cauchy problem (4.4), (4.5). On the other hand the minimax solution satisfies the Hamilton–Jacobi equation (4.4) at all points $(t, x(\,{\cdot}\,)) \in X_\ast$ at which it is ci-differentiable. The greatest progress in the development of the theory of minimax solutions of the Cauchy problem (4.4), (4.5) has been achieved in the case when the Hamiltonian $H$ satisfies the condition of positive homogeneity in the third variable (in this connection see § 2.3). Assume that the condition $(4.\mathrm{CP}.2)$ and the following conditions hold.
We consider the multivalued mappings
$$
\begin{equation*}
F_+(\tau, y(\,{\cdot}\,), s) = \bigl\{ f \in \mathbb{R}^n \colon \|f\| \leqslant \sqrt{2}\,c (1 + \|y(\cdot \wedge \tau)\|_\infty), \, \langle s, f \rangle \geqslant H(\tau, y(\,{\cdot}\,), s) \bigr\}
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
\begin{gathered} \, F_-(\tau, y(\,{\cdot}\,), s) = \bigl\{ f \in \mathbb{R}^n \colon \|f\| \leqslant \sqrt{2}\,c (1 + \|y(\cdot \wedge \tau)\|_\infty), \, \langle s, f \rangle \leqslant H(\tau, y(\,{\cdot}\,), s) \bigr\}, \\ \tau \in [0, T], \quad y(\,{\cdot}\,) \in \operatorname{Lip}([-h,T];\mathbb{R}^n),\quad s \in S_n, \end{gathered}
\end{equation*}
\notag
$$
where $c$ is from $(3.\mathrm{CP}.2)$ (see condition $(4.\mathrm{CP}.2)$). Under conditions $(4.\mathrm{CP}.2)$ and $(4.\mathrm{CP}.4)$–$(4.\mathrm{CP}.6)$ a minimax solution of the Cauchy problem (4.4), (4.5) can be defined as follows. Definition 16. A minimax solution of the Cauchy problem (4.4), (4.5) is a functional $\varphi \colon [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ that is non-anticipating and continuous, satisfies the boundary condition (4.5), and has the following properties.
Note that a minimax solution in the sense of Definition 16 is also a minimax solution in the sense of Definition 15. The converse result for the Cauchy problem (4.4), (4.5) requires further investigations. The following result holds (see [141], Theorem 1, and also [126], Theorem 1). Theorem 13. Under conditions $(4.\mathrm{CP}.2)$ and $(4.\mathrm{CP}.4)$–$(4.\mathrm{CP}.6)$, a minimax solution of the Cauchy problem (4.4), (4.5) in the sense of Definition 16 exists and is unique. In addition, the minimax solution of the Cauchy problem (4.4), (4.5) depends continuously on variations of the map $g$, the Hamiltonian $H$, and the boundary functional $\sigma$ (see [141], Theorem 2, and also [126], Theorem 2) and, like in the case of Definition 15, is consistent with the concept of solution in the classical sense (see [141], Theorems 3 and 4, and also [126], Propositions 1 and 2). In [127] conditions $(4.\mathrm{MS}.2_+)$ and $(4.\mathrm{MS}.2_-)$ in Definition 16 were expressed in the infinitesimal form. In that case, for a functional $\varphi \colon [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ the following lower and upper right derivatives at a point $(t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{Lip}([-h,T];\mathbb{R}^n)$ in a direction $(z(\,{\cdot}\,),F) \in \operatorname{Lip}([t,T];\mathbb{R}^n) \times \mathcal{K}(\mathbb{R}^n)$ were to be considered:
$$
\begin{equation*}
\begin{gathered} \, d_- \{\varphi(t, x(\,{\cdot}\,)) \mid z(\,{\cdot}\,), F \} \\ \qquad =\lim_{\varepsilon \to 0^+}\,\inf_{y(\,{\cdot}\,) \in \Omega(t, x(\,{\cdot}\,), z(\,{\cdot}\,), F, \varepsilon)}\, \liminf_{\delta \to 0^+}\frac{\varphi(t + \delta, y(\,{\cdot}\,)) - \varphi(t, x(\,{\cdot}\,))}{\delta}\,, \end{gathered}
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
\begin{gathered} \, d_+ \{\varphi(t, x(\,{\cdot}\,)) \mid z(\,{\cdot}\,), F \} \\ \qquad =\lim_{\varepsilon \to 0^+}\,\sup_{y(\,{\cdot}\,) \in \Omega(t, x(\,{\cdot}\,), z(\,{\cdot}\,), F, \varepsilon)}\, \limsup_{\delta \to 0^+}\frac{\varphi(t + \delta, y(\,{\cdot}\,)) - \varphi(t, x(\,{\cdot}\,))}{\delta}\,, \end{gathered}
\end{equation*}
\notag
$$
where
$$
\begin{equation*}
\Omega(t, x(\,{\cdot}\,), z(\,{\cdot}\,), F, \varepsilon)= \bigl\{ y(\,{\cdot}\,) \in \operatorname{Lip}(t, x(\,{\cdot}\,)) \colon \dot{y}(\tau) - \dot{z}(\tau) \in [F]^\varepsilon \text{ for a. e. } \tau \in [t, T] \bigr\}.
\end{equation*}
\notag
$$
The result is as follows (in this connection see [127], Theorem 1). Theorem 14. Under conditions $(4.\mathrm{CP}.2)$, $(4.\mathrm{CP}.4)$–$(4.\mathrm{CP}.6)$, for each non- anticipating continuous functional $\varphi \colon [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ condition $(4.\mathrm{MS}.2_+)$ is equivalent to the differential inequality The function $z(\cdot \,|\,t,x(\,{\cdot}\,)) \in \operatorname{Lip}([t,T];\mathbb{R}^n)$ in (4.8) and (4.9) is defined by
$$
\begin{equation*}
z(\tau \mid t, x(\,{\cdot}\,))= g(\tau, x(\cdot \wedge t)),\qquad \tau \in [t, T].
\end{equation*}
\notag
$$
Thus, a non-anticipating continuous functional $\varphi \colon [0, T] \times \operatorname{Lip}([-h,T];\mathbb{R}^n) \to \mathbb{R}$ is a minimax solution of the Cauchy problem (4.4), (4.5) in the sense of Definition 16 if and only if it satisfies the boundary condition (4.5) and the differential inequalities (4.8) and (4.9). We specify the differential inequalities (4.8) and (4.9) in the case when conditions $(4.\mathrm{CP}.1)$, $(4.\mathrm{CP}.2)$, and $(4.\mathrm{CP}.6)$ and the following assumptions, which are stronger than $(4.\mathrm{CP}.5)$ and the continuity of the boundary functional $\sigma$, hold for the Cauchy problem.
Under the above conditions the minimax solution $\varphi$ of the Cauchy problem (4.4), (4.5) satisfies ([129], Lemma 6.4) the following Lipschitz condition in the variable $x(\,{\cdot}\,)$.
Note that the condition $(4.\mathrm{L})$ implies the nonanticipation property of $\varphi$. Under condition $(4.\mathrm{L})$ we have the relations ([127], Theorem 1)
$$
\begin{equation*}
d_- \{\varphi(t, x(\,{\cdot}\,)) \mid z(\,{\cdot}\,), F \} =\min_{f \in F}\partial_-\{\varphi(t,x(\,{\cdot}\,))\mid z(\,{\cdot}\,),f\}
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
d_+ \{\varphi(t, x(\,{\cdot}\,)) \mid z(\,{\cdot}\,), F \} =\max_{f \in F}\partial_+\{\varphi(t,x(\,{\cdot}\,)) \mid z(\,{\cdot}\,),f\}
\end{equation*}
\notag
$$
for all $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{Lip}([- h, T]; \mathbb{R}^n)$ and $(z(\,{\cdot}\,), F) \in \operatorname{Lip}([t, T]; \mathbb{R}^n) \times \mathcal{K}(\mathbb{R}^n)$. Here $\partial_- \{\varphi(t, x(\,{\cdot}\,)) \mid z(\,{\cdot}\,), f \}$ and $\partial_+ \{\varphi(t, x(\,{\cdot}\,)) \mid z(\,{\cdot}\,), f \}$ are the lower and upper right derivatives of $\varphi$ at the point $(t, x(\,{\cdot}\,))$ in the direction $(z(\,{\cdot}\,), f)$, that is,
$$
\begin{equation*}
\partial_- \{\varphi(t, x(\,{\cdot}\,)) \mid z(\,{\cdot}\,), f \} = \liminf_{\delta \to 0^+}\frac{\varphi(t + \delta, y^{(z(\,{\cdot}\,), f)} (\,{\cdot}\,)) - \varphi(t, x(\,{\cdot}\,))}{\delta}
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
\partial_+ \{\varphi(t, x(\,{\cdot}\,)) \mid z(\,{\cdot}\,), f \} = \limsup_{\delta \to 0^+}\frac{\varphi(t + \delta, y^{(z(\,{\cdot}\,), f)} (\,{\cdot}\,)) - \varphi(t, x(\,{\cdot}\,))}{\delta}\,,
\end{equation*}
\notag
$$
where the function $y^{(z(\,{\cdot}\,), f)}(\,{\cdot}\,) \in \operatorname{Lip}(t, x(\,{\cdot}\,))$ is defined by
$$
\begin{equation*}
y^{(z(\,{\cdot}\,),f)}(\tau)=x(t)+z(\tau)-z(t)+f(\tau-t),\qquad \tau \in (t, T].
\end{equation*}
\notag
$$
Hence we arrive at the following assertion ([129], Theorem 4.1). Theorem 15. Under conditions $(4.\mathrm{CP}.1)$, $(4.\mathrm{CP}.2)$, $(4.\mathrm{CP}.6)$–$(4.\mathrm{CP}.8)$ a continuous functional $\varphi \colon [0, T] \times \operatorname{Lip}([- h, T]; \mathbb{R}^n) \to \mathbb{R}$ is a minimax solution of the Cauchy problem (4.4), (4.5) in the sense of Definition 16 if and only if it satisfies the boundary condition (4.5), the Lipschitz condition $(4.\mathrm{L})$, and the differential inequalities Regarding the differential game (4.1), (4.2), the meaningfullness of Definition 16 of a minimax solution of the Cauchy problem (4.4), (4.5) under various assumptions was studied in [124], [68], and [128]. Among other things, appropriate methods, based on the minimax solution, for the construction of optimal control strategies of the players were proposed and the coincidence of this solution with the value functional was established. Some results concerning the viscosity approach to a generalized solution of the Cauchy problem (4.4), (4.5) and its relationship with the minimax approach were deduced in [145]. In this case the problem was considered over the space of piecewise Lipschitz functions. To conclude this section, we note that the development of the theory of differential games and the theory of relevant Hamilton–Jacobi equations for neutral-type systems of general form, for example, of the form
$$
\begin{equation*}
\dot{y}(\tau)=f(\tau,y(\tau),y(\tau-h),\dot{y}(\tau-h),u(\tau),v(\tau))
\end{equation*}
\notag
$$
is of particular interest.
5. Fractional-order systemsThe machinery of coinvariant differentiation of non-anticipating functionals has turned out to be useful for the construction of the theory of Hamilton–Jacobi equations for fractional-order systems (1.25). This section is devoted to the results obtained in this area. 5.1. Hamilton–Jacobi equations with fractional coinvariant derivativesLet $\alpha \in (0, 1)$. Following, for example, [152], Definition 2.3 we consider the space $\operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$ of functions $x \colon [0, T] \to \mathbb{R}^n$ each of which is representable in the form
$$
\begin{equation}
x(\tau)= x(0) + (I^\alpha f)(\tau),\qquad \tau \in [0, T],
\end{equation}
\tag{5.1}
$$
with its own measurable essentially bounded function $f \colon [0, T] \to \mathbb{R}^n$, where $(I^\alpha f)(\tau)$ is the Riemann–Liouville fractional integral of order $\alpha$ of the function $f(\,{\cdot}\,)$ at a point $\tau$ (see [152], Definition 2.1):
$$
\begin{equation}
(I^\alpha f)(\tau)= \frac{1}{\Gamma(\alpha)} \int_{0}^{\tau} \frac{f(\xi)}{(\tau - \xi)^{1 - \alpha}}\,{\rm d} \xi.
\end{equation}
\tag{5.2}
$$
In view of the fact that $\operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \subset \operatorname{C}([0, T]; \mathbb{R}^n)$, the space $\operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$ is endowed with the uniform norm $\|\cdot\|_\infty$ (see (3.4)). By [152], Theorem 2.4, each function $x(\,{\cdot}\,) \in \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$ has a Caputo fractional derivative $(^{\mathrm C\!} D^\alpha x)(\tau) = f(\tau)$ for almost every $\tau \in [0, T]$ (see Definition (1.26)), where $f(\,{\cdot}\,)$ is from (5.1). In particular,
$$
\begin{equation}
x(\tau)= x(0)+\frac{1}{\Gamma(\alpha)} \int_{0}^{\tau} \frac{(^{\mathrm C\!} D^\alpha x)(\xi)}{(\tau-\xi)^{1-\alpha}}\, {\rm d}\xi,\qquad \tau \in [0,T].
\end{equation}
\tag{5.3}
$$
For each point $(t, x(\,{\cdot}\,)) \in [0,T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$, we consider the set
$$
\begin{equation*}
\operatorname{AC}^\alpha(t, x(\,{\cdot}\,))=\bigl\{ y(\,{\cdot}\,) \in \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \colon y(\tau) = x(\tau), \, \tau \in [0,t]\bigr\}.
\end{equation*}
\notag
$$
Definition 17. A functional $\varphi \colon [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \to \mathbb{R}$ is coinvariantly differentiable (ci-differentiable) of order $\alpha$ at a point $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$ if there exist $\partial_t^\alpha \varphi(t, x(\,{\cdot}\,)) \in \mathbb{R}$ and $\nabla^\alpha \varphi(t, x(\,{\cdot}\,)) \in \mathbb{R}^n$ such that Note that if we take formally $\alpha=1$ and agree that $(^{\mathrm C\!} D^\alpha y)(\,{\cdot}\,)=\dot{y}(\,{\cdot}\,)$ in this case, then Definition 17 of ci-differentiability of order $\alpha$ turns to Definition 7 of ordinary ci-differentiability. In addition, like in the case of ordinary ci-differentiability (see § 3.1), it turns out that if a functional $\varphi \colon [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \to \mathbb{R}$ is ci-differentiable of order $\alpha$ at all points $(t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{AC}^\alpha([0,T];\mathbb{R}^n)$, then it and its ci-derivatives of order $\alpha$
$$
\begin{equation}
\partial_t^\alpha \varphi \colon [0, T) \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \to \mathbb{R}\quad\text{and}\quad \nabla^\alpha \varphi \colon [0, T) \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \to \mathbb{R}^n
\end{equation}
\tag{5.4}
$$
are non-anticipating. Furthermore, a functional $\varphi \colon [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \to \mathbb{R}$ is called ci-smooth of order $\alpha$ if it is continuous and ci-differentiable of order $\alpha$ at all points $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$ and the maps (5.4) are continuous. A key property of functionals ci-smooth of order $\alpha$ is the following analogue of Proposition 1 (see [48], Lemma 9.2). Proposition 9. Assume that a functional $\varphi \colon [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \to \mathbb{R}$ is ci- smooth of order $\alpha$, and fix a point $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$, a function $y(\,{\cdot}\,) \in \operatorname{AC}^\alpha(t, x(\,{\cdot}\,))$, and a number $\vartheta \in (t, T)$. Then the function $\mu(\tau) = \varphi(\tau, y(\,{\cdot}\,))$, $\tau \in [t, \vartheta]$, satisfies $\mu(\,{\cdot}\,) \in \operatorname{Lip}([t,\vartheta];\mathbb{R})$ and We present two examples of functionals ci-smooth of order $\alpha$. Given a functional $\psi \colon [0, T] \times \operatorname{C}([0, T]; \mathbb{R}^n) \to \mathbb{R}$, set
$$
\begin{equation}
\varphi(t, x(\,{\cdot}\,))=\psi(t,(I^{1-\alpha} x)(\,{\cdot}\,)),\qquad (t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{AC}^\alpha([0,T];\mathbb{R}^n),
\end{equation}
\tag{5.5}
$$
where $(I^{1 - \alpha} x)(\,{\cdot}\,)$ is the Riemann–Liouville fractional integral of order $1 - \alpha$ of $x(\,{\cdot}\,)$ (see (5.2)). Then $\varphi$ is ci-differentiable of order $\alpha$ at a point $(t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{AC}^\alpha([0,T];\mathbb{R}^n)$ if and only if $\psi$ is ci-differentiable in the sense of Definition 7 at the point $(t,(I^{1-\alpha} x)(\,{\cdot}\,)) \in [0,T) \times \operatorname{C}([0,T];\mathbb{R}^n)$. In this case
$$
\begin{equation*}
\partial_t^\alpha \varphi(t, x(\,{\cdot}\,))= \partial_t\psi(t,(I^{1-\alpha} x)(\,{\cdot}\,))\quad\text{and} \quad \nabla^\alpha \varphi(t, x(\,{\cdot}\,))= \nabla \psi(t, (I^{1 - \alpha} x)(\,{\cdot}\,)).
\end{equation*}
\notag
$$
In particular, starting from any ci-smooth functional $\psi$, we can define a functional $\varphi$ ci-smooth of order $\alpha$ by (5.5). The second example actually says that the dependence of the solution of a differential equation with Caputo fractional derivative of the form (1.25) on the initial data $(t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$ is ci-smooth of order $\alpha$. More precisely, assume that the function $f \colon [0, T] \times \mathbb{R}^n \to \mathbb{R}^n$ in (1.25) is continuously differentiable and there is $c > 0$ such that $\|f(\tau, y)\| \leqslant c (1 + \|y\|)$ for all $\tau \in [0, T]$ and $y \in \mathbb{R}^n$. Consider the functional
$$
\begin{equation*}
\varphi(t, x(\,{\cdot}\,))= \sigma(y(T)),\qquad (t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{AC}^\alpha([0,T];\mathbb{R}^n),
\end{equation*}
\notag
$$
where the function $\sigma \colon \mathbb{R}^n\to\mathbb{R}$ is continuously differentiable and $y(\,{\cdot}\,)\in\operatorname{AC}^\alpha(t, x(\,{\cdot}\,))$ is a function satisfying the differential equation (1.25) for almost all $\tau \in [t, T]$. Then $\varphi$ is ci-smooth of order $\alpha$, and the corresponding derivatives can be obtained as solutions of some integral equations (in this connection see [56], Theorem 3.1, and also [59], Corollary 9.1). On the other hand, very simple functionals are not necessarily ci-differentiable of order $\alpha$. For example, fix a vector $\ell \in \mathbb{R}^n \setminus \{0\}$ and consider the functional
$$
\begin{equation*}
\varphi(t, x(\,{\cdot}\,))= \langle \ell, x(t) \rangle,\qquad (t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([0,T];\mathbb{R}^n).
\end{equation*}
\notag
$$
Note that this functional is ci-differentiable in the sense of Definition 7 at all points $(t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{C}([0,T];\mathbb{R}^n)$, and we have $\partial_t \varphi (t, x(\,{\cdot}\,)) = 0$ and $\nabla \varphi(t, x(\,{\cdot}\,)) = \ell$. We show that $\varphi$ is not ci-differentiable of order $\alpha$ at any point $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$. Reasoning by contradiction, we assume that $\varphi$ is ci-differentiable of order $\alpha$ at some point $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$. For each vector $f \in \mathbb{R}^n$ we choose a function $y^{(f)}(\,{\cdot}\,) \in \operatorname{AC}^\alpha(t, x(\,{\cdot}\,))$ so that $(^{\mathrm C\!} D^\alpha y^{(f)})(\tau) = f$ for almost every $\tau \in [t,T]$. Note that, according to (5.3),
$$
\begin{equation*}
y^{(f)}(\tau)= x(0) + \frac{1}{\Gamma(\alpha)} \int_{0}^{t} \frac{(^{\mathrm C\!} D^\alpha x)(\xi)}{(\tau - \xi)^{1 - \alpha}} \, {\rm d} \xi+ \frac{f (\tau - t)^\alpha}{\Gamma(\alpha + 1)}\,,\qquad \tau \in [t, T], \quad f \in \mathbb{R}^n.
\end{equation*}
\notag
$$
Assume that $f,g \in \mathbb{R}^n$ satisfy the condition $\langle \ell,f-g \rangle \ne 0$. Then we have
$$
\begin{equation*}
\varphi(\tau, y^{(f)}(\,{\cdot}\,)) - \varphi(\tau, y^{(g)}(\,{\cdot}\,))= \frac{(\tau-t)^\alpha\langle \ell,f-g\rangle}{\Gamma(\alpha+1)}\,,\qquad \tau \in [t,T].
\end{equation*}
\notag
$$
At the same time, by the assumption of ci-differentiability of order $\alpha$ there is a vector $\nabla^\alpha \varphi(t, x(\,{\cdot}\,)) \in \mathbb{R}^n$ such that
$$
\begin{equation*}
\varphi(\tau, y^{(f)}(\,{\cdot}\,)) - \varphi(\tau, y^{(g)}(\,{\cdot}\,))=(\tau-t) \langle\nabla^\alpha\varphi(t,x(\,{\cdot}\,)),f-g\rangle+o(\tau-t),\qquad \tau \in (t, T].
\end{equation*}
\notag
$$
Thus, we obtain
$$
\begin{equation*}
\frac{(\tau - t)^\alpha \langle \ell, f - g \rangle}{\Gamma(\alpha + 1)}= (\tau - t) \langle \nabla^\alpha \varphi(t, x(\,{\cdot}\,)), f - g \rangle + o(\tau - t),\qquad \tau \in (t, T].
\end{equation*}
\notag
$$
Dividing both sides of this equality by $(\tau - t)^\alpha$ and passing to the limit as $\tau \to t^+$, we infer the relation $\langle\ell,f-g\rangle=0$, which contradicts the choice of $f$ and $g$. We note here that the question of ci-differentiability of different orders of the same functional was treated in [51], Proposition 5. The subject of discussion in this section is the Cauchy problem for the Hamilton–Jacobi equation with ci-derivatives of order $\alpha$,
$$
\begin{equation}
\begin{gathered} \, \partial_t^\alpha \varphi(t, x(\,{\cdot}\,)) + H \bigl( t, x(t), \nabla^\alpha \varphi(t, x(\,{\cdot}\,)) \bigr)= 0, \\ (t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{AC}^\alpha([0,T];\mathbb{R}^n), \end{gathered}
\end{equation}
\tag{5.6}
$$
with right-end boundary condition
$$
\begin{equation}
\varphi(T, x(\,{\cdot}\,))= \sigma(x(\,{\cdot}\,)),\qquad x(\,{\cdot}\,) \in \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n).
\end{equation}
\tag{5.7}
$$
We seek a functional $\varphi \colon [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \to \mathbb{R}$ with the non-anticipation property. The following conditions are assumed to hold for the Hamiltonian $H$ and the boundary functional $\sigma$.
5.2. Differential games for fractional-order systemsWe consider a differential game in which the motion of the dynamical system is described by a differential equation with Caputo fractional derivative of the form (see, for example, [152], [136], [146], [82], and [32])
$$
\begin{equation}
\begin{gathered} \, (^{\mathrm C\!} D^\alpha y)(\tau)= f(\tau, y(\tau), u(\tau), v(\tau)), \\ \tau \in [0, T],\quad y(\tau) \in \mathbb{R}^n,\quad u(\tau) \in P,\quad v(\tau) \in Q, \end{gathered}
\end{equation}
\tag{5.8}
$$
with the initial condition which is specified by an initial position $(t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$, and let the quality index have the form
$$
\begin{equation}
J(t, x(\,{\cdot}\,), u(\,{\cdot}\,), v(\,{\cdot}\,))=\sigma(y(\,{\cdot}\,)) - \int_{t}^{T} \chi(\tau, y(\tau), u(\tau), v(\tau)) \, {\rm d} \tau.
\end{equation}
\tag{5.10}
$$
We assume that the following conditions hold.
For any initial position $(t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$ and controls $u(\,{\cdot}\,) \in \mathcal{U}(t)$ and $v(\,{\cdot}\,) \in \mathcal{V}(t)$ of the first and second players, respectively, condition $(5.\mathrm{DG}.1)$ (see conditions $(2.\mathrm{DG}.1)$–$(2.\mathrm{DG}.3)$) ensures the existence and uniqueness of a motion of system (5.8) satisfying the initial condition (5.9), namely, of a function $y(\,{\cdot}\,) \in \operatorname{AC}^\alpha(t, x(\,{\cdot}\,))$ that, together with $u(\,{\cdot}\,)$ and $v(\,{\cdot}\,)$, satisfies (5.8) for almost every $\tau \in [t, T]$. Note that this function $y(\,{\cdot}\,)$ coincides with the unique solution in the space $\operatorname{C}([0, T]; \mathbb{R}^n)$ of Volterra’s integral equation
$$
\begin{equation}
y(\tau)= a(\tau \mid t, x(\,{\cdot}\,))+ \frac{1}{\Gamma(\alpha)} \int_{t}^{\tau} \frac{f(\xi, y(\xi), u(\xi), v(\xi))} {(\tau - \xi)^{1 - \alpha}} \, {\rm d} \xi,\qquad \tau \in [t, T],
\end{equation}
\tag{5.11}
$$
which satisfies condition (5.9). Here
$$
\begin{equation}
a(\tau \mid t, x(\,{\cdot}\,))=\begin{cases} x(\tau)& \text{if}\ \tau \in [0, t], \\ x(0) + \dfrac{1}{\Gamma(\alpha)}\displaystyle\int_{0}^{t} \dfrac{(^{\mathrm C\!} D^\alpha x)(\xi)}{(\tau-\xi)^{1-\alpha}} \, {\rm d} \xi& \text{if}\ \tau \in (t, T]. \end{cases}
\end{equation}
\tag{5.12}
$$
The function $a(\cdot \mid t, x(\,{\cdot}\,))$ can alternatively be defined as a function $a(\,{\cdot}\,) \in \operatorname{AC}^\alpha(t, x(\,{\cdot}\,))$ such that $(^{\mathrm C\!} D^\alpha a)(\tau) = 0$ for almost every $\tau \in [t, T]$; in this sense it is an analogue of the function $x(\cdot \wedge t)$ in (3.9). In contrast to ordinary differential equations, the Volterra integral operator in the equivalent integral equation (5.11) has the kernel $1 / (\Gamma(\alpha) (\tau - \xi)^{1 - \alpha})$, which is weakly singular in addition. In particular, this has the effect that fractional-order systems have the property of heredity (in this connection see, for example, [32], Remark 4.6). Therefore (see [48], § 4), for an adequate formalization of dynamic programming principles, control problems for systems of this type must consider, as initial data at an intermediate instant $t \in (0, T]$, not only the state $y(t)$ of the system at this instant but also the complete history of the motion $y(\,{\cdot}\,)$ to this instant. Nevertheless, it is worthy of noting that in some works (see, for example, [76] and [149]) attempts were made to apply the dynamic programming principle without taking this feature into account. We briefly dwell on the relationship between fractional-order systems and neutral- type systems. First, note that if we apply the map
$$
\begin{equation*}
g(\tau, y(\,{\cdot}\,))= y(\tau) - \bigl( I^{1 - \alpha} (y(\,{\cdot}\,) - y(0)) \bigr) (\tau),\qquad (\tau,y(\,{\cdot}\,)) \in [0,T] \times\operatorname{Lip}([0,T];\mathbb{R}^n),
\end{equation*}
\notag
$$
then, according to (1.26) and (5.2), the differential equation with Caputo fractional derivative (5.8) becomes formally a neutral-type functional differential equation in the Hale form (4.1). However, since the quantity $g(\tau, y(\,{\cdot}\,))$ actually depends on $y(\tau)$, the map $g$ does not satisfy the key condition $(4.\mathrm{DG}.1)$ (also see the discussion after the formulation of this condition). In particular, this indicates that equation (5.8) is not solved with respect to the first derivative $\dot{y}(\tau)$. Second, we can change the variables in (5.8) and define a new unknown function $w(\,{\cdot}\,) \in \operatorname{Lip}([0, T]; \mathbb{R}^n)$ by
$$
\begin{equation*}
w(\tau)=\bigl(I^{1-\alpha}(y(\,{\cdot}\,)-y(0))\bigr)(\tau),\qquad \tau \in [0, T].
\end{equation*}
\notag
$$
Then, in view of the equality ([152], Theorem 2.4)
$$
\begin{equation*}
y(\tau)= y(0) + (^{\mathrm C\!} D^{1 - \alpha} w)(\tau),\qquad \tau \in [0, T],
\end{equation*}
\notag
$$
from (1.26) we conclude that it is possible to switch from the differential game for the fractional-order system (5.8), (5.10) to an equivalent differential game for the system
$$
\begin{equation}
\dot{w}(\tau)=f\bigl(\tau,x(0)+ (^{\mathrm C\!} D^{1-\alpha}w)(\tau),u(\tau),v(\tau)\bigr),\qquad \tau \in [t,T],
\end{equation}
\tag{5.13}
$$
and the quality index
$$
\begin{equation}
J= \sigma \bigl( x(0) + (^{\mathrm C\!} D^{1 - \alpha} w)(\,{\cdot}\,) \bigr)- \int_{t}^{T} \chi\bigr( \tau, x(0) + (^{\mathrm C\!} D^{1 - \alpha} w)(\tau), u(\tau), v(\tau) \bigr) \, {\rm d} \tau.
\end{equation}
\tag{5.14}
$$
This game has formally the form of the differential game (3.16), (3.17) for a time- delay system. However, since the mapping
$$
\begin{equation*}
[0, T] \times \operatorname{Lip}([0, T]; \mathbb{R}^n) \ni (\tau, w(\,{\cdot}\,)) \mapsto (^{\mathrm C\!} D^{1 - \alpha} w)(\tau) \in \mathbb{R}^n
\end{equation*}
\notag
$$
is not continuous, condition $(3.\mathrm{DG}.1)$, for example, is in general not satisfied for the differential game (5.13), (5.14); therefore, the results in § 3 do not apply directly to this game (hence to the original game (5.8), (5.10) either). On the other hand, since ([152], Theorem 2.2)
$$
\begin{equation*}
(^{\mathrm C\!} D^{1 - \alpha} w)(\tau)= \frac{1}{\Gamma(\alpha)} \int_{0}^{\tau} \frac{\dot{w}(\xi)}{(\tau - \xi)^{1 - \alpha}} \, {\rm d} \xi\quad \text{for a. a. } \tau \in [0,T],
\end{equation*}
\notag
$$
the right-hand side of the differential equation (5.13) depends explicitly on the values of the derivative $\dot{w}(\xi)$ of the sought-for function for $\xi \in [0,\tau]$; thus, this equation must rather be classified as a neutral-type functional differential equation of general form. As mentioned in § 4 above, the theory of differential games and the theory of Hamilton–Jacobi equations are currently not developed for systems described by equations of this type. We nevertheless note that the above reasoning was used in [49] (see also [47] and [61]) to develop approximation schemes for the differential game (5.8), (5.10) by use of differential games for time-delay systems. For each point $(t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$ we consider the lower and upper values $\rho_-(t, x(\,{\cdot}\,))$ and $\rho_+(t, x(\,{\cdot}\,))$ of the differential game (5.8), (5.10) that are defined by analogy with (3.19) and (3.21). Recall that in the case when these quantities are equal, the game has value $\rho(t,x(\,{\cdot}\,))=\rho_-(t,x(\,{\cdot}\,))=\rho_+(t,x(\,{\cdot}\,))$. The following theorem ([153], Theorem 1) is true, which is largely a consequence of Proposition 9. Theorem 16. Assume that conditions $(5.\mathrm{DG}.1)$ and $(5.\mathrm{DG}.2)$ hold for the differential game (5.8), (5.10). Assume that there is a functional $\varphi \colon [0, T] \times\operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \to \mathbb{R}$ ci-smooth of order $\alpha$ and satisfying the Hamilton–Jacobi equation (5.6) with Hamiltonian (1.18) and the boundary condition (5.7). Then this game has value Note that on the basis of Theorem 16 we can, for example, construct solutions of linear quadratic problems in optimal control and differential games for fractional-order systems [60], [64], [63]. 5.3. Minimax solutionsIn this subsection we present results obtained in developing the minimax approach to the concept of a generalized solution of the Cauchy problem (5.6), (5.7). Definition 18. A minimax solution of the Cauchy problem (5.6), (5.7) is a functional $\varphi \colon [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \to \mathbb{R}$ that is non-anticipating and continuous, satisfies the boundary condition (5.7), and has the following properties.
The multivalued mapping $E_0$ in (5.15) was defined in (2.4). The main result is as follows ([155], Theorem 8.1). Theorem 17. Under conditions $(5.\mathrm{CP}.1)$ and $(5.\mathrm{CP}.2)$ the Cauchy problem (5.6), (5.7) has a unique minimax solution. In general, Theorem 17 can be proved in accordance with the scheme in Theorem 8.1 in [162], which is based on the properties of sets of solutions of differential inclusions with Caputo fractional derivatives that were established in [50]. The most significant difficulty in this case is constructing an appropriate Lyapunov–Krasovskii functional that makes it possible to prove the comparison principle for minimax solutions of the Cauchy problem (5.6), (5.7). We present a construction of such a functional that was proposed in [55]. We fix a bounded set $W \subset \mathbb{R}^n$ and choose a number $\lambda$ for it in accordance with condition $(2.\mathrm{CP}.3)$ (see $(5.\mathrm{CP}.1)$). We consider the functional
$$
\begin{equation*}
\nu_\ast(\tau, w(\,{\cdot}\,))= \frac{e^{- \lambda_\ast \tau}}{m} \sum_{i = 1}^{m} \nu_i (\tau, w(\,{\cdot}\,)),\qquad (\tau,w(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([0,T];\mathbb{R}^n).
\end{equation*}
\notag
$$
Here $m \in \mathbb{N}$ is chosen from the condition $\alpha \in [1 / 2^m, 1 / 2^{m - 1})$,
$$
\begin{equation*}
\begin{aligned} \, \lambda_\ast & = \frac{\mu_m^2 \Gamma(\alpha + \beta_m + 1) \Gamma(1 - \alpha) T^{2 \alpha + 2 \beta_m - 1}} {2\Gamma(1-\alpha-\beta_m)\Gamma(\alpha+2\beta_m)}\,e^{\mu_1 T^\alpha}, \\ \nu_1(\tau, w(\,{\cdot}\,))& = \frac{1}{\Gamma(1 - \alpha)} \int_{0}^{\tau} \frac{\exp(-\mu_1(\tau-\xi)^\alpha)\|w(\xi)-w(0)\|^2} {(\tau-\xi)^\alpha} \, {\rm d}\xi, \\ \nu_i(\tau,w(\,{\cdot}\,))&=\frac{1}{\Gamma(1-\alpha-\beta_i)\Gamma(\beta_i)} \\ &\ \ \times\!\!\int_{0}^{\tau}\frac{\exp(-\mu_i(\tau-\xi)^{\alpha+\beta_i})} {(\tau - \xi)^{\alpha + \beta_i}}\!\int_{0}^{\xi} \frac{\|w(\eta)-w(0)\|^2}{(\xi-\eta)^{1-\beta_i}}\,{\rm d}\eta \, {\rm d}\xi,\ \ i \in \{2,\dots, m\}, \end{aligned}
\end{equation*}
\notag
$$
where
$$
\begin{equation*}
\mu_1= 4 m \Gamma(1 - \alpha) \lambda \ \text{and}\ \mu_{i + 1}= \frac{\mu_i^2 \Gamma(\alpha + \beta_i + 1) \Gamma(1 - \alpha - \beta_{i + 1})}{2 \Gamma(1 - \alpha - \beta_i)}\,,\ i \in \{1,\dots, m - 1\},
\end{equation*}
\notag
$$
and $\beta_i = (2^{i-1} - 1) \alpha$. We set $\varepsilon_0 = 2 e^{- (\lambda + \lambda_\ast / 2) T}$ and define a Lyapunov–Krasovskii functional for $\varepsilon \in (0,\varepsilon_0]$ in the form
$$
\begin{equation}
\nu_\varepsilon(\tau, w(\,{\cdot}\,))= \frac{e^{- \lambda \tau}}{\varepsilon} \sqrt{\varepsilon^4 + \nu_\ast(\tau, w(\,{\cdot}\,))}\,,\quad (\tau,w(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([0,T];\mathbb{R}^n)
\end{equation}
\tag{5.17}
$$
(in this connection see (2.12)), and also define the following auxiliary mappings, which play in a certain sense the role of the ci-derivative of order $\alpha$ with respect to $t$ and the ci-gradient of order $\alpha$ of this functional, respectively:
$$
\begin{equation}
\begin{aligned} \, p_\varepsilon(\tau, w(\,{\cdot}\,))& = -\frac{\lambda e^{- \lambda \tau}}{\varepsilon} \sqrt{\varepsilon^4 + \nu_\ast(\tau, w(\,{\cdot}\,))}- \frac{2 \lambda e^{- (\lambda + \lambda_\ast) \tau}}{\varepsilon}\, \frac{\|w(\tau) - w(0)\|^2}{\sqrt{\varepsilon^4 + \nu_\ast(\tau, w(\,{\cdot}\,))}}\,, \\ s_\varepsilon(\tau, w(\,{\cdot}\,))& = \frac{e^{- (\lambda + \lambda_\ast) \tau}}{\varepsilon}\, \frac{w(\tau)-w(0)}{\sqrt{\varepsilon^4+\nu_\ast(\tau,w(\,{\cdot}\,))}}\,, \end{aligned}
\end{equation}
\tag{5.18}
$$
where $(\tau,w(\,{\cdot}\,)) \in [0,T] \times \operatorname{C}([0,T];\mathbb{R}^n)$. The mappings $\nu_\varepsilon$, $p_\varepsilon$, and $s_\varepsilon$ have ([55], Lemma 7.7) the following properties, which are similar to conditions $(3.\mathrm{a})$–$(3.\mathrm{d})$ in Lemma 1.
These properties turn out to suffice for the substantiation of the comparison principle for minimax solutions of the Cauchy problem (5.6), (5.7) by following the scheme of the proof of Lemma 1 above. We emphasize that the last assertion in $(5.\mathrm{a})$ is verified on the basis of an estimate ([46], Corollary 4.2) for the fractional derivative of order $\alpha$ of the squared norm of a function $w(\,{\cdot}\,) \in \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$. More precisely, for $r(\tau) = \|w(\tau) - w(0)\|^2$, $\tau \in [0, T]$, we have $r(\,{\cdot}\,) \in \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$ and
$$
\begin{equation*}
(^{\mathrm C\!} D^\alpha r)(\tau)=\frac{1}{\Gamma(1-\alpha)}\,\frac{{\rm d}}{{\rm d}\tau} \int_{0}^{\tau}\frac{\|w(\xi)-w(0)\|^2}{(\tau-\xi)^\alpha} \, {\rm d}\xi \leqslant 2\bigl\langle w(\tau) - w(0), (^{\mathrm C\!} D^\alpha w)(\tau)\bigr\rangle
\end{equation*}
\notag
$$
for almost every $\tau \in [0, T]$. Estimates of this form were originally derived in [3] and [1] for absolutely continuous functions $w(\,{\cdot}\,)$. As a supplement to Theorem 17, we can show (in this connection see [51], § 6.2) that the minimax solution of the Cauchy problem (5.6), (5.7) depends continuously on variations in the differentiation order $\alpha$, the Hamiltonian $H$, and the boundary functional $\sigma$. In addition, like in § 2.3, we can introduce the concept of characteristic complex of the Hamilton–Jacobi equation (5.6) and prove ([52], Proposition 2), using the Lyapunov–Krasovskii functional $\nu_\varepsilon$ defined by (5.17), that in conditions $(5.\mathrm{MS}.1_+)$ and $(5.\mathrm{MS}.1_-)$ in Definition 18 we can take arbitrary upper and lower characteristic complexes, respectively, instead of the standard characteristic complex $(\mathbb{R}^n, E_0)$. In the infinitesimal form conditions $(5.\mathrm{MS}.1_+)$ and $(5.\mathrm{MS}.1_-)$ can be expressed as follows ([52], Proposition 4 and Corollary 2; also see [51], Theorem 3). Given a functional $\varphi \colon [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \to \mathbb{R}$, we define its lower and upper right derivatives of order $\alpha$ at a point $(t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{AC}^\alpha([0,T];\mathbb{R}^n)$ in a multivalued direction $F \in \mathcal{K}(\mathbb{R}^n)$:
$$
\begin{equation*}
d^\alpha_- \{\varphi(t,w(\,{\cdot}\,)) \mid F\} =\lim_{\varepsilon \to 0^+}\, \inf_{y(\,{\cdot}\,) \in \Omega^\alpha(t,x(\,{\cdot}\,),F,\varepsilon)}\, \liminf_{\delta \to 0^+}\frac{\varphi(t+\delta,y(\,{\cdot}\,))- \varphi(t,x(\,{\cdot}\,))}{\delta}
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
d^\alpha_+\{\varphi(t,w(\,{\cdot}\,)) \mid F\} =\lim_{\varepsilon \to 0^+}\, \sup_{y(\,{\cdot}\,) \in \Omega^\alpha(t, x(\,{\cdot}\,), F, \varepsilon)}\, \limsup_{\delta \to 0^+}\frac{\varphi(t + \delta, y(\,{\cdot}\,)) - \varphi(t, x(\,{\cdot}\,))}{\delta}\,,
\end{equation*}
\notag
$$
where
$$
\begin{equation*}
\Omega^\alpha(t, x(\,{\cdot}\,), F, \varepsilon) = \bigl\{ y(\,{\cdot}\,) \in \operatorname{AC}^\alpha(t, x(\,{\cdot}\,)) \colon (^{\mathrm C\!} D^\alpha y)(\tau) \in [F]^\varepsilon \text{ for a. a. } \tau \in [t, T] \bigr\}.
\end{equation*}
\notag
$$
Theorem 18. Assume that condition $(5.\mathrm{CP}.1)$ holds. Then for a non-anticipating lower semicontinuous functional $\varphi \colon [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \to \mathbb{R}$ condition $(5.\mathrm{MS}.1_+)$ is equivalent to the differential inequality The first terms on the left-hand sides of (5.19) and (5.20) are the corresponding derivatives of the auxiliary functional
$$
\begin{equation*}
\psi(\tau, y(\,{\cdot}\,))= \varphi(\tau, y(\,{\cdot}\,)) - \int_{0}^{\tau} \langle s,(^{\mathrm C\!} D^\alpha y)(\xi) \rangle \, {\rm d}\xi,\quad (\tau,y(\,{\cdot}\,)) \in [0,T] \times \operatorname{AC}^\alpha([0,T];\mathbb{R}^n),
\end{equation*}
\notag
$$
at $(t, x(\,{\cdot}\,))$ in the multivalued direction equal to the ball $B ( c(1 + \|x(t)\|))$, where $c$ is from $(2.\mathrm{CP}.2)$ (see condition $(5.\mathrm{CP}.1)$). Assume that a functional $\varphi \colon [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \to \mathbb{R}$ satisfies the following Lipschitz condition in the variable $x(\,{\cdot}\,)$.
Then $\varphi$ is automatically non-anticipating and ([62], Lemma 2) we have
$$
\begin{equation*}
d_-^\alpha \{\varphi(t, x(\,{\cdot}\,)) \mid F \} = \min_{f \in F} \partial_-^\alpha \{\varphi(t, x(\,{\cdot}\,)) \mid f \}
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
d_+^\alpha \{\varphi(t, x(\,{\cdot}\,)) \mid F \} = \max_{f \in F} \partial_+^\alpha \{\varphi(t, x(\,{\cdot}\,)) \mid f \}
\end{equation*}
\notag
$$
for all $(t,x(\,{\cdot}\,)) \in [0,T) \times{} \operatorname{AC}^\alpha([0,T];\mathbb{R}^n)$ and $F \in \mathcal{K}(\mathbb{R}^n)$. Here $\partial_-^\alpha \{\varphi(t, x(\,{\cdot}\,)) \mid f \}$ and $\partial_+^\alpha \{\varphi(t, x(\,{\cdot}\,)) \mid f \}$ are the lower and upper right derivatives of order $\alpha$ of $\varphi$ at the point $(t, x(\,{\cdot}\,))$ in the single-valued direction $f$:
$$
\begin{equation*}
\partial_-^\alpha\{\varphi(t,x(\,{\cdot}\,)) \mid f\} =\liminf_{\delta \to 0^+} \frac{\varphi(t+\delta,y^{(f)}(\,{\cdot}\,))-\varphi(t,x(\,{\cdot}\,))}{\delta}
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
\partial_+^\alpha\{\varphi(t,x(\,{\cdot}\,)) \mid f\} =\limsup_{\delta \to 0^+} \frac{\varphi(t+\delta,y^{(f)}(\,{\cdot}\,))-\varphi(t,x(\,{\cdot}\,))}{\delta}\,,
\end{equation*}
\notag
$$
where the function $y^{(f)}(\,{\cdot}\,) \in \operatorname{AC}^\alpha(t,x(\,{\cdot}\,))$ is defined by the equality $(^{\mathrm C\!} D^\alpha y^{(f)})(\tau) = f$ for almost all $\tau \in [t, T]$. Thus, under condition $(5.\mathrm{L})$ the differential inequalities (5.19) and (5.20) take the simpler form
$$
\begin{equation}
\begin{gathered} \, \begin{aligned} \, \min_{f \in B(c (1 + \|x(t)\|))}\bigl( \partial_-^\alpha \{\varphi(t, x(\,{\cdot}\,)) \mid f\} - \langle s, f \rangle \bigr) + H(t, x(t), s)& \leqslant 0, \\ \max_{f \in B(c (1 + \|x(t)\|))} \bigl( \partial_+^\alpha \{\varphi(t, x(t)) \mid f\} - \langle s, f \rangle \bigr)+ H(t, x(\,{\cdot}\,), s)& \geqslant 0, \end{aligned} \\ (t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n), \quad s \in \mathbb{R}^n. \end{gathered}
\end{equation}
\tag{5.21}
$$
We also note that if a functional $\varphi$ is ci-differentiable of order $\alpha$ at a point $(t,x(\,{\cdot}\,))$, then by [51], Proposition 7,
$$
\begin{equation*}
d_-^\alpha \{\varphi(t, x(\,{\cdot}\,)) \mid F \} = \partial_t^\alpha \varphi(t, x(\,{\cdot}\,)) + \min_{f \in F} \langle \nabla^\alpha \varphi(t, x(\,{\cdot}\,)), f \rangle
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
d_+^\alpha \{\varphi(t, x(\,{\cdot}\,)) \mid F \} = \partial_t^\alpha \varphi(t, x(\,{\cdot}\,)) + \max_{f \in F} \langle \nabla^\alpha \varphi(t, x(\,{\cdot}\,)), f \rangle.
\end{equation*}
\notag
$$
In particular, this and Theorem 18 imply the consistency of the minimax solution of the Cauchy problem (5.6), (5.7) with the concept of solution in the classical sense (in this connection also see [51], Theorem 4, and [55], Theorem 6.1 and Proposition 6.2). Theorem 19. Assume that conditions $(5.\mathrm{CP}.1)$ and $(5.\mathrm{CP}.2)$ hold for the Cauchy problem (5.6), (5.7). Then the following assertions are true: (i) if a continuous functional $\varphi \colon [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \to \mathbb{R}$ is ci-differentiable of order $\alpha$ at each point $(t, x(\,{\cdot}\,)) \in [0, T) \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$ and satisfies the Hamilton–Jacobi equation (5.6) and the boundary condition (5.7), then it is a minimax solution of the Cauchy problem (5.6), (5.7); (ii) if a minimax solution of the Cauchy problem (5.6), (5.7) is ci-differentiable of order $\alpha$ at some point $(t,x(\,{\cdot}\,)) \in [0,T) \times \operatorname{AC}^\alpha([0,T];\mathbb{R}^n)$, then it satisfies equality (5.6) at this point. We turn back to the differential game (5.8), (5.10). We consider a minimax solution $\varphi$ of the corresponding Cauchy problem (5.6), (5.7) with Hamiltonian (1.18). For an initial position $(t, x(\,{\cdot}\,)) \in [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$ we define the set
$$
\begin{equation}
\begin{aligned} \, \nonumber Y^\alpha(t, x(\,{\cdot}\,))& = \bigl\{ y(\,{\cdot}\,) \in \operatorname{AC}^\alpha(t, x(\,{\cdot}\,)) \colon \\ & \qquad \|(^{\mathrm C\!} D^\alpha y)(\tau)\| \leqslant c (1 + \|y(\tau)\|) \text{ for a. a. } \tau \in [t, T] \bigr\}, \end{aligned}
\end{equation}
\tag{5.22}
$$
where $c$ is borrowed from condition $(2.\mathrm{DG}.2)$ (see $(5.\mathrm{DG}.1)$). As the set $Y^\alpha(t, x(\,{\cdot}\,))$ is compact in the space $\operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$, we can choose a bounded set $W \subset \mathbb{R}^n$ such that for each $y(\,{\cdot}\,) \in Y^\alpha(t, x(\,{\cdot}\,))$ we have $y(\tau) \in W$, $\tau \in [t, T]$. Using (5.17) and (5.18) we define the corresponding mappings $\nu_\varepsilon$, $p_\varepsilon$, and $s_\varepsilon$ for $\varepsilon \in (0,\varepsilon_0]$. We consider non-anticipating control strategies of the following form of the first and second players:
$$
\begin{equation}
\begin{gathered} \, \begin{aligned} \, U^\circ_\varepsilon(\tau, y(\,{\cdot}\,))& \in \operatorname*{arg\,min}_{u \in P}\,\max_{v \in Q} \bigl( \langle s_\varepsilon^{(u)}(\tau, y(\,{\cdot}\,)), f(\tau, y(\tau), u, v) \rangle - \chi(\tau, y(\tau), u, v) \bigr), \\ V^\circ_\varepsilon(\tau, y(\,{\cdot}\,))& \in \operatorname*{arg\,max}_{v \in Q}\,\min_{u \in P} \bigl( \langle s_\varepsilon^{(v)}(\tau, y(\,{\cdot}\,)), f(\tau, y(\tau), u, v) \rangle - \chi(\tau, y(\tau), u, v) \bigr), \end{aligned} \\ (\tau, y(\,{\cdot}\,)) \in [0, T) \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n), \end{gathered}
\end{equation}
\tag{5.23}
$$
where
$$
\begin{equation*}
\begin{aligned} \, s_\varepsilon^{(u)}(\tau, y(\,{\cdot}\,))& = s_\varepsilon \bigl( \tau, y(\,{\cdot}\,) - z_\varepsilon^{(u)}(\cdot \mid \tau, y(\,{\cdot}\,)) \bigr), \\ z_\varepsilon^{(u)}(\cdot \mid \tau, y(\,{\cdot}\,)) & \in \operatorname*{arg\,min}_{z(\,{\cdot}\,) \in Y^\alpha(t, x(\,{\cdot}\,))}\bigl( \varphi(\tau, z(\,{\cdot}\,)) + \nu_\varepsilon(\tau, y(\,{\cdot}\,) - z(\,{\cdot}\,)) \bigr), \end{aligned}
\end{equation*}
\notag
$$
and
$$
\begin{equation*}
\begin{aligned} \, s_\varepsilon^{(v)}(\tau, y(\,{\cdot}\,))& = - s_\varepsilon \bigl( \tau, y(\,{\cdot}\,) - z_\varepsilon^{(v)}(\cdot \mid \tau, y(\,{\cdot}\,)) \bigr), \\ z_\varepsilon^{(v)}(\cdot \mid \tau, y(\,{\cdot}\,))& \in \operatorname*{arg\,max}_{z(\,{\cdot}\,) \in Y^\alpha(t, x(\,{\cdot}\,))}\bigl( \varphi(\tau, z(\,{\cdot}\,)) - \nu_\varepsilon(\tau, y(\,{\cdot}\,) - z(\,{\cdot}\,)) \bigr). \end{aligned}
\end{equation*}
\notag
$$
The following assertion is true ([53], Theorem 2). Theorem 20. Under conditions $(5.\mathrm{DG}.1)$ and $(5.\mathrm{DG}.2)$ the differential game (5.8), (5.10) has a value. The value functional $\rho$ of this game coincides with the minimax solution $\varphi$ of the Cauchy problem (5.6), (5.7) with Hamiltonian (1.18), that is, $\rho(t, x(\,{\cdot}\,)) = \varphi(t, x(\,{\cdot}\,))$ for $(t,x(\,{\cdot}\,)) \in [0,T] \times \operatorname{AC}^\alpha([0,T];\mathbb{R}^n)$. The strategies $U^\circ_\varepsilon$ and $V^\circ_\varepsilon$ are optimal: In (5.24) the guaranteed results $\rho^{(u)} (t, x(\,{\cdot}\,), U^\circ_\varepsilon)$ and $\rho^{(v)} (t, x(\,{\cdot}\,), V^\circ_\varepsilon)$ of the control strategies $U^\circ_\varepsilon$ and $V^\circ_\varepsilon$ of the first and second players, respectively, for the differential game (5.8), (5.10) are defined similarly to (3.61) (also see (2.28) and (2.29)). Assume that the following addition condition, which is stronger than $(5.\mathrm{DG}.2)$, holds.
In this case the value functional $\rho$ satisfies the Lipschitz condition $(5.\mathrm{L})$ (in this connection see [62], Lemma 1). Thus, we arrive at the following criterion. Theorem 21. Assume that conditions $(5.\mathrm{DG}.1)$ and $(5.\mathrm{DG}.3)$ hold for the differential game (5.8), (5.10). Then a functional $\varphi \colon [0,T] \times \operatorname{AC}^\alpha([0,T];\mathbb{R}^n) \to \mathbb{R}$ is a value functional for this game if and only if $\varphi$ is continuous and satisfies the Lipschitz condition $(5.\mathrm{L})$, the boundary condition (5.7), and the differential inequalities (5.21) with Hamiltonian $H$ from (1.18). Note that it was proved in [59] for optimal control problems for fractional-order systems that under some additional assumptions the value functional is differentiable of order $\alpha$ in finite-dimensional directions (for a similar result for optimal control problems for ordinary differential systems, see, for example, [164], [165], and [168]). On this basis it is possible, first, to describe ([59], Theorem 10.1) a non- smooth rule for the construction of optimal control strategies that is simpler than (5.23) and, second, to establish ([58], Theorem 3) the relationship between the Hamilton–Jacobi equation (5.6) and the relevant Pontryagin maximum principle (see, for example, [13] and [108]), which is similar to the one known in the classical case (see, for example, [40], Theorem 8.1, and also [168]). In particular, the last result is another confirmation that ci-derivatives of order $\alpha$ are an adequate tool for the study of control problems for fractional-order systems. 5.4. Viscosity solutionsIn [54], following the constructions in [156], [120], [119], and [121], elements of the viscosity approach to a generalized solution of the Cauchy problem (5.6), (5.7) were developed. For each $k \in \mathbb{N}$ consider the compact set $W_k$ consisting of functions $x(\,{\cdot}\,) \in \operatorname{AC}^\alpha([0,T];\mathbb{R}^n)$ such that
$$
\begin{equation*}
\|x(0)\| \leqslant k\quad\text{and}\quad \|(^{\mathrm C\!} D^\alpha x)(\tau)\|\leqslant k c (1 + \|x(\tau)\|)\quad \text{for a. e. } \tau \in [0, T],
\end{equation*}
\notag
$$
where $c$ is from condition $(2.\mathrm{CP}.2)$ (see $(5.\mathrm{CP}.1)$). Note that $W_k \subset W_{k + 1}$, $k \in \mathbb{N}$, the union of $W_k$ over $k \in \mathbb{N}$ coincides with $\operatorname{AC}^\alpha([0, T]; \mathbb{R}^n)$, and for any $(t, x(\,{\cdot}\,)) \in [0, T] \times W_k$ and $y(\,{\cdot}\,) \in Y^\alpha(t, x(\,{\cdot}\,))$ (see (5.22)) we have $y(\,{\cdot}\,) \in W_k$. Definition 19. A viscosity solution of the Cauchy problem (5.6), (5.7) is a functional $\varphi \colon [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n) \to \mathbb{R}$ that is non-anticipating and continuous, satisfies the boundary condition (5.7), and has the following properties.
The following result holds ([54], Corollary 5.1). Theorem 22. Assume that assumptions $(5.\mathrm{DG}.1)$ and $(5.\mathrm{DG}.3)$ hold. Then the value functional of the differential game (5.8), (5.10) is the unique viscosity solution of the Cauchy problem (5.6), (5.7) with Hamiltonian (1.18) that satisfies the Lipschitz condition $(5.\mathrm{L})$. The greatest difficulty in the verification of Theorem 22 is to prove the comparison principle for viscosity solutions satisfying $(5.\mathrm{L})$. It is generally speaking proved using the scheme of the above proof of Theorem 8. The main distinction is that the following functional, which is ci-smooth of order $\alpha$ with respect to both $(t, x(\,{\cdot}\,))$ and $(\tau, y(\,{\cdot}\,))$, is used to construct appropriate penalty functionals (an analogue of the function $\|x - y\|^2$ in (2.36) in the classical case and also of the functionals (3.65) and (3.66) in the case of time-delay systems):
$$
\begin{equation}
\begin{gathered} \, \begin{aligned} \, & \nu_\varepsilon(t, x(\,{\cdot}\,), \tau, y(\,{\cdot}\,)) \\ &\quad\!= \bigl( \varepsilon^{2 / (q - 1)} + \|a(T \mid t, x(\,{\cdot}\,)) - a(T \mid \tau, y(\,{\cdot}\,))\|^2 \bigr)^{q / 2} \\ & \qquad + \int_{0}^{T} \frac{\bigl( \varepsilon^{2 / (q - 1)} + \|a(\xi \mid t, x(\,{\cdot}\,)) - a(\xi \mid \tau, y(\,{\cdot}\,))\|^2 \bigr)^{q / 2}} {(T - \xi)^{(1 - \alpha - \beta) q}} \, {\rm d} \xi - C \varepsilon^{q / (q - 1)}, \end{aligned} \\ (t, x(\,{\cdot}\,)), (\tau, y(\,{\cdot}\,)) \in [0, T] \times \operatorname{AC}^\alpha([0, T]; \mathbb{R}^n), \end{gathered}
\end{equation}
\tag{5.25}
$$
where $\varepsilon > 0$ is a small parameter, $q = 2 / (2 - \alpha) \in (1, 2)$, the positive number $\beta $ satisfies $\beta < 1 - \alpha$ and $\beta < \alpha / 2$, the functions $a(\cdot \mid t, x(\,{\cdot}\,))$ and $a(\cdot \mid \tau, y(\,{\cdot}\,))$ are defined by (5.12), and $C = 1 + T^{1 - (1 - \alpha - \beta) q} / (1 - (1 - \alpha - \beta) q)$. In this case the additional Lipschitz condition $(5.\mathrm{L})$ is significant and makes it possible, by analogy with Lemma 7.6 in [144], to obtain certain boundedness properties for ci-gradients of order $\alpha$ of test functionals $\psi$ to be constructed. Note that, under the additional Lipschitz condition $(5.\mathrm{DG}.3)$, the functional $\nu_\varepsilon$ in (5.25) can also be used to construct optimal non-anticipating control strategies of the players in the differential game (5.8), (5.10) (in this connection see [57]).
Citation in format AMSBIB
\Bibitem{GomLuk24} |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Contact us: |
Terms of Use
|
Registration to the website |
Logotypes |
|