On the existence and uniqueness of a generalized solution of the Protter problem for $(3+1)$-D Keldysh-type equations

Abstract

A $(3+1)$-dimensional boundary value problem for equations of Keldysh type (the second kind) is studied. Such problems for equations of Tricomi type (the first kind) or for the wave equation were formulated by M.H. Protter (1954) as multidimensional analogues of Darboux or Cauchy-Goursat plane problems. Now, it is well known that Protter problems are not correctly set, and they have singular generalized solutions, even for smooth right-hand sides. In this paper an analogue of the Protter problem for equations of Keldysh type is given. An appropriate generalized solution with possible singularity is defined. Results for uniqueness and existence of such a generalized solution are obtained. Some a priori estimates are stated.

1 Introduction

In the present paper we consider an analogue of the Protter problems for $(3+1)$-D Keldysh-type equations. For $m \in \mathbf{R}$, $0< m<2$, we study some boundary value problems (BVPs) for the weakly hyperbolic equation

$$ L_{m}[u]\equiv u_{x_{1}x_{1}}+u_{x_{2}x_{2}}+u_{x_{3}x_{3}} - \bigl(t^{m}u_{t} \bigr)_{t}=f(x,t), $$

(1.1)

expressed in Cartesian coordinates $(x,t)=(x_{1},x_{2},x_{3},t) \in \mathbf{R}^{4}$ in a simply connected region

$$\Omega_{m}:= \biggl\{ ( x,t ) : 0< t< t_{0}, \frac{2}{2-m} t^{\frac{2-m}{2}}< \sqrt{x_{1}^{2}+x_{2}^{2}+x_{3}^{2}}< 1- \frac{2}{2-m} t^{\frac{2-m}{2}} \biggr\} , $$

bounded by the ball $\Sigma_{0}:=\{(x,t): t=0, \sqrt{x_{1}^{2}+x_{2}^{2}+x_{3}^{2}}<1\}$, centered at the origin $O=(0,0,0,0)$ and by two characteristic surfaces of equation (1.1)

$$\begin{aligned}& \Sigma_{1}^{m}:= \biggl\{ (x,t):0< t< t_{0}, \sqrt {x_{1}^{2}+x_{2}^{2}+x_{3}^{2}}=1- \frac {2}{2-m} t^{\frac{2-m}{2}} \biggr\} , \\& \Sigma_{2}^{m}:= \biggl\{ (x,t): 0< t< t_{0}, \sqrt {x_{1}^{2}+x_{2}^{2}+x_{3}^{2}}= \frac{2}{2-m} t^{\frac{2-m}{2}} \biggr\} , \end{aligned}$$

where $t_{0}:= ( \frac{2-m}{4} ) ^{\frac{2}{2-m}}$.

In this work we are interested in finding sufficient conditions for the existence and uniqueness of a generalized solution of the following problem.

Problem PK

Find a solution to equation (1.1) in $\Omega_{m}$ that satisfies the boundary conditions

$$ u|_{\Sigma_{1}^{m}}=0; \quad\quad t^{m}u_{t} \rightarrow0, \quad \text{as } t\to+0. $$

The adjoint problem to PK is as follows.

Problem $\mathbf {PK}^{\ast }$

Find a solution to the self-adjoint equation (1.1) in $\Omega_{m}$ that satisfies the boundary conditions

$$u|_{\Sigma_{2}^{m}}=0;\quad \quad t^{m}u_{t}\rightarrow0,\quad \text{as } t\to+0. $$

First, we present a brief historical overview here and provide an extensive list of references.

Protter arrived at similar problems, but for Tricomi-type equations, while studying BVPs which describe transonic flows in fluid dynamics. It is well known that most important boundary value problems that, in the case of linear mixed-type equations, appear in hodograph plane for two-dimensional transonic potential flows are the classical Tricomi, Frankl’, and Guderley-Morawetz problems. The first two for flows in nozzles and jets and the third one as an approximation in flows about airfoils. For such connections, see the paper of Morawetz [1]. About sixty years ago Murray Protter [2] stated a multidimensional variant of the famous Guderley-Morawetz plane problem for hyperbolic-elliptic equations that had been studied by Morawetz [3], Lax and Phillips [4]. This problem now is known as the Protter-Morawetz problem. A result for uniqueness was obtained by Aziz and Schneider [5] in the case of Frankl-Morawetz problem. However, the multidimensional case is rather different, and there is no general understanding of the situation. Even the question of well posedness is not completely resolved. For different statements of multidimensional Darboux-type problems or some related Protter-Morawetz problems for mixed-type equations, see [1, 6–13]. Some Tricomi problems for the Lavrent’ev-Bitsadze equation are studied in [14–16]. On the other hand, different problems for elliptic-hyperbolic equations of Keldysh type have specific applications in plasma physics, optics, and analysis on projective spaces (see Otway [17, 18] and Otway and Marini [19]). Various statements of problems for mixed equations of Tricomi or Keldysh type can be found in Oleı̌nik and Radkevič [20], Nakhushev [21], and several applications of such problems in the study of transonic flows are described in Chen [22], Čanić and Keyfitz [23]. Let us also mention some results in the thermodynamic theory of porous elastic bodies given in [24, 25]. In order to analyze the spatial behavior of solutions, some appropriate estimates and similar procedures are used there.

In relation to the mixed-type problems, Protter also formulated and studied some BVPs in the hyperbolic part of the domain for the wave equation [26] and degenerated hyperbolic (or weakly hyperbolic) equations of Tricomi type [2]. In that case the Protter problems are multidimensional analogues of the plane Darboux or Cauchy-Goursat problems (see Kalmenov [27] and Nakhushev [28]). The equations are considered in $(3+1)$-D domain, bounded by two characteristic surfaces and noncharacteristic plane region. The data are prescribed on one characteristic and on a noncharacteristic boundary part. Protter considered [2, 26] Tricomi-type equations or the wave equation ($m \in\mathbf{R}$, $m \geq0$)

$$ \tilde{L}_{m}[u]:= t^{m}[u_{x_{1}x_{1}}+u_{x_{2}x_{2}}+u_{x_{3}x_{3}}]- u_{tt}=f(x,t) $$

(1.2)

in the domain

$$ \tilde{\Omega}_{m}:= \biggl\{ ( x_{1},x_{2},x_{3},t ) : t>0, \frac{2}{m+2} t^{\frac{m+2}{2}}< \sqrt{x_{1}^{2}+x_{2}^{2}+x_{3}^{2}}< 1- \frac{2}{m+2} t^{\frac{m+2}{2}} \biggr\} , $$

bounded by $\Sigma_{0}$ and two characteristics surfaces of (1.2)

$$\begin{aligned}& \tilde{\Sigma}_{1}^{m}= \biggl\{ t>0, \sqrt {x_{1}^{2}+x_{2}^{2}+x_{3}^{2}}=1- \frac{2}{m+2} t^{\frac{m+2}{2}} \biggr\} , \\& \tilde{\Sigma}_{2}^{m}= \biggl\{ t>0, \sqrt {x_{1}^{2}+x_{2}^{2}+x_{3}^{2}}= \frac{2}{m+2} t^{\frac{m+2}{2}} \biggr\} . \end{aligned}$$

He proposed four problems, known now as Protter problems.

Protter problems

Find a solution of equation (1.2) in the domain $\tilde{\Omega}_{m}$ with one of the following boundary conditions:

$$\begin{aligned}& P1:\quad u|_{\Sigma_{0}\cup\tilde{\Sigma}_{1}^{m}}=0, \quad \quad P1^{\ast}:\quad u|_{\Sigma_{0}\cup\tilde{\Sigma}_{2}^{m}}=0; \\& P2:\quad u|_{\tilde{\Sigma}_{1}^{m}}=0,\quad\quad u_{t}|_{\Sigma_{0}}=0,\quad\quad P2^{\ast}:\quad u|_{\tilde{\Sigma}_{2}^{m}}=0,\quad\quad u_{t}|_{\Sigma_{0}}=0. \end{aligned}$$

The boundary conditions in problem P1^∗ (respectively P2^∗) are the adjoint boundary conditions to problem P1 (respectively P2) for (1.2) in $\tilde{\Omega}_{m}$.

It turns out that instead of both boundary conditions given in problems P1 on $\tilde{\Sigma}_{1}^{m}$, $\Sigma_{0}$ and in P2 on $\tilde{\Sigma}_{2}^{m}$, $\Sigma_{0}$ for the Tricomi-type equation (1.2), in the case of Keldysh-type equation (1.1), they are reduced to only one boundary condition on the characteristic $\Sigma_{1}^{m}$ and a condition on the growth of possible singularity of the derivative $u_{t}$ as $t \to+0$.

We mention some known results for Protter problems in the Tricomi case that make the investigation of such problems interesting and reasonable. Garabedian [29] obtained a result for the uniqueness of classical solution to problem P1 for the wave equation (i.e., equation (1.2) with $m=0$). It is interesting that contrary to their plane analogues, the 3-D Protter problems are not well posed (see [30, 31] and the monograph of Bitsadze [32]). The reason is that the adjoint homogeneous problems P1^∗ and P2^∗ have an infinite number of linearly independent nontrivial classical solutions. On the other hand, the unique generalized solutions of 3-D problems P1 and P2 could have strong power-type singularity on the $\tilde{\Sigma}^{m} _{2}$ even for smooth right-hand sides (see [31, 33, 34]). Behavior of the singular solutions to 3-D problems P1 and P2 is studied in [35, 36]. Such results are announced for the 4-D case as well [37]. Didenko [38] studied problems P1 and P1^∗ for the Tricomi equation ($m = 1$) in the symmetric case. Aldashev [39] studied some multidimensional analogues of Protter problems for equation (1.2), but he did not mention any possible singular solutions.

These known results for Protter problems for Tricomi-type equations and many interesting applications of different boundary value problems for equations of Keldysh type motivate us to study problems PK and $\mathit {PK}^{\ast}$ and to try to find out new effects that appear. In [40] ill-posedness of 3-D Protter problems for Keldysh-type equations in the frame of classical solvability is discussed, and the results for uniqueness of quasi-regular solutions are obtained. Existence and uniqueness of generalized solutions to problem PK in that case are obtained in [41], and some singular generalized solutions are announced in [42].

In [31, 33] we study Protter problems for Tricomi-type equations. For 3-D Keldysh-type equation in [43], we formulate a new Protter problem and announce some results for the existence and uniqueness of a generalized solution in the case $0< m<1$. In [44] we announce analogical results for $(3+1)$-D equations of Keldysh type in a more general case $0< m<4/3$ and claim the existence of infinitely many classical smooth solutions of the $(3+1)$-D homogeneous problem $\mathit {PK}^{\ast}$. Now, in the present paper we work in the case $0< m<4/3$. Using an appropriate Riemann-Hadamard function, we find an exact integral representation of the generalized solution and prove the results announced in [44]. To avoid an infinite number of necessary conditions in the frame of classical solvability, we give a notion of a generalized solution to problem PK which can have some singularity at the point O. In order to deal successfully with the encountered difficulties for $\varepsilon\in(0,1)$, we introduce the region

$$\Omega_{m,\varepsilon}:=\Omega_{m} \cap \bigl\{ \vert x\vert > \varepsilon \bigr\} , $$

where $\vert x\vert =\sqrt{x_{1}^{2}+x_{2}^{2}+x_{3}^{2}}$.

We give the following definition of a generalized solution of problem PK in the case $0< m<4/3$.

Definition 1.1

We call a function $u(x,t)$ a generalized solution of problem PK in $\Omega_{m}$, $0< m<\frac{4}{3}$, for equation (1.1) if:

• 1. $u,u_{x_{j}} \in C(\bar{\Omega}_{m} \setminus O)$, $j=1,2,3$, $u_{t} \in C(\bar{\Omega}_{m} \setminus\bar{\Sigma}_{0})$;

• 2. $u|_{\Sigma_{1}^{m}}=0$;

• 3. For each $\varepsilon\in(0,1)$ there exists a constant $C(\varepsilon)>0$ such that in $\Omega_{m,\varepsilon}$

  $$ \bigl\vert u_{t}(x,t) \bigr\vert \leq C( \varepsilon)t^{-\frac{3m}{4}}; $$

  (1.3)

• 4. The identity

  $$\begin{aligned} \int_{\Omega_{m}} \bigl\{ t^{m} u_{t}v_{t}-u_{x_{1}}v_{x_{1}}- u_{x_{2}}v_{x_{2}}-u_{x_{3}}v_{x_{3}}-fv \bigr\} \,dx_{1}\,dx_{2}\,dx_{3}\,dt =0 \end{aligned}$$

  (1.4)

  holds for all v from

  $$V_{m}:= \bigl\{ v(x,t):v \in C^{2}(\bar{\Omega}_{m}), v|_{\Sigma_{2}^{m}} =0, v \equiv0 \text{ in a neighborhood of }O \bigr\} . $$

Remark 1.1

We mention that all the first derivatives of the generalized solutions of 3-D Protter problems in the Tricomi case can have singularity on the boundary of the domain (see [31, 33]). Actually, this fact corresponds to the analogical situation in a 2-D case of the Darboux problem (see [27]). While in the Keldysh case, according to Definition 1.1, the derivative $u_{t}$ can be unbounded when $t\to+0$, but $u_{x_{1}}$, $u_{x_{2}}$ and $u_{x_{3}}$ are bounded in each $\bar{\Omega}_{m,\varepsilon}$, $\varepsilon >0$.

In this paper, first, we prove results for the uniqueness of a generalized solution to problem PK.

Theorem 1.1

If $m \in(0,\frac{4}{3})$, then there exists at most one generalized solution of problem PK in $\Omega_{m}$.

Further, we use the three-dimensional spherical functions $Y_{n}^{s}(x)$ with $n \in\mathbf{N} \cup\{0\}$; $s=1,2, \ldots,2n+1$. The functions $Y_{n}^{s}(x)$ are defined usually on the unit sphere $S^{2}:=\{(x_{1},x_{2},x_{3}):x_{1}^{2}+x_{2}^{2}+x_{3}^{2}=1\}$, and $Y_{n}^{s}$ form a complete orthonormal system in $L_{2}(S^{2})$ (see [45]). For convenience of discussions that follow, we extend the spherical functions out of $S^{2}$ radially, keeping the same notation for the extended functions $Y_{n}^{s}(x):=Y_{n}^{s}(x/\vert x\vert )$ for $x \in\mathbf {R}^{3}\setminus\{0\}$.

Let the right-hand side function $f(x,t)$ of equation (1.1) be fixed as a “harmonic polynomial” of order l with $l \in\mathbf{N} \cup\{0\}$, and it has the following representation:

$$ f(x,t)=\sum_{n=0}^{l}\sum _{s=1}^{2n+1}f_{n}^{s} \bigl( \vert x\vert ,t \bigr)Y_{n}^{s}(x) $$

(1.5)

with some coefficients $f_{n}^{s}(\vert x\vert ,t)$.

In this special case we give an existence result as well.

Theorem 1.2

Let $m \in(0,\frac{4}{3})$. Suppose that the right-hand side function $f(x,t)$ has the form (1.5) and $f \in C^{1}(\bar{\Omega}_{m})$. Then the unique generalized solution $u(x,t)$ of problem PK in $\Omega_{m}$ exists and has the form

$$ u(x,t)=\sum_{n=0}^{l}\sum _{s=1}^{2n+1}u_{n}^{s} \bigl( \vert x\vert ,t \bigr)Y_{n}^{s}(x). $$

(1.6)

Remark 1.2

Actually, when the right-hand side function $f(x,t)$ has the form (1.5) in Theorem 1.2, we find explicit representations for the functions $u_{n}^{s}(\vert x\vert ,t)$ in (1.6). These representations involve appropriate hypergeometric functions.

In the case when the right-hand side function $f(x,t)$ has the form (1.5), we give an a priori estimate for the generalized solution of problem PK in $\Omega_{m}$ as well.

Theorem 1.3

Let the conditions in Theorem  1.2 be fulfilled. Then the unique generalized solution of problem PK in $\Omega_{m}$ has the form (1.6) and satisfies the a priori estimate

$$ \bigl\vert u(x ,t) \bigr\vert \leq c \Bigl( \max _{\bar{\Omega}_{m}} \vert f\vert \Bigr) \vert x\vert ^{-l-1}, $$

(1.7)

with a constant $c>0$ independent of f.

Estimate (1.7) shows the maximal order of possible singularity at point O, when the right-hand side function $f(x,t)$ is a “harmonic polynomial” of fixed order l. We will point out that a similar a priori estimate for generalized solutions to 3-D Protter problem P1 in the Tricomi case is obtained in [36], while an estimate from below in this case is given in [31].

The present paper contains the introduction and five more sections. In Section 2, the Protter problem PK is considered in a model case when the right-hand side function $f(x,t)$ of equation (1.1) is fixed as a “harmonic polynomial” (1.5) of order l. In that case we formulate the 2-D boundary value problems $\mathit {PK}_{1}$ and $\mathit {PK}_{2}$, corresponding to the $(3+1)$-D problem PK. We give a notion for a generalized solution of Cauchy-Goursat problem $\mathit {PK}_{2}$, and in Section 3, using the Riemann-Hadamard function associated to this problem, we find an integral representation for a generalized solution. Further, we obtain existence and uniqueness results for a generalized solution of problem $\mathit {PK}_{2}$. Actually, this is the essential result in this paper and has the most difficult proof. Using the results of the previous section, in Section 4 we prove the main results in this paper, i.e., Theorem 1.1, Theorem 1.2 and Theorem 1.3. In Appendix A we give the main properties of the Riemann-Hadamard function associated to the Cauchy-Goursat problem $\mathit {PK}_{2}$. In Appendix B some auxiliary results, needed for the study of the generalized solution to problem $\mathit {PK}_{2}$, are proven.

2 Two-dimensional Cauchy-Goursat problems corresponding to problem PK

Using spherical coordinates $(r,\theta, \varphi, t)\in \mathbf{R}^{4}$, $0 \leq\theta< \pi$, $0 \leq\varphi< 2 \pi$, $r>0$ with

$$x_{1}=r \sin\theta\cos\varphi, \quad\quad x_{2}=r \sin\theta\sin \varphi, \quad\quad x_{3}=r \cos\theta $$

problem PK can suitably be treated. Written in the new coordinates, equation (1.1) becomes

$$ L_{m}u=\frac{1}{r^{2}} \bigl(r^{2} u_{r } \bigr)_{r }+ \frac{1}{r^{2} \sin\theta} (\sin\theta u_{\theta})_{\theta} +\frac{1}{r ^{2} \sin^{2} \theta}u_{\varphi\varphi}- \bigl(t^{m}u_{t} \bigr)_{t}=f. $$

(2.1)

We consider equation (2.1) in the region

$$\Omega_{m}= \biggl\{ (r,\theta, \varphi, t):0< t< t_{0}, 0 \leq\theta< \pi, 0 \leq\varphi< 2 \pi, \frac {2}{2-m}t^{\frac{2-m}{2}} < r < 1-\frac{2}{2-m}t^{\frac{2-m}{2}} \biggr\} , $$

bounded by the following surfaces:

$$\begin{aligned}& \Sigma_{0}= \bigl\{ (r,\theta, \varphi, t):t=0, 0 \leq\theta< \pi, 0 \leq\varphi< 2 \pi, r< 1 \bigr\} , \\& \Sigma_{1}^{m}= \biggl\{ (r,\theta, \varphi, t):t>0, 0 \leq \theta< \pi, 0 \leq\varphi< 2 \pi, r=1-\frac{2}{2-m}t^{\frac{2-m}{2}} \biggr\} , \\& \Sigma_{2}^{m}= \biggl\{ (r,\theta, \varphi, t):t>0, 0 \leq\theta< \pi, 0 \leq\varphi< 2 \pi, r=\frac{2}{2-m}t^{\frac{2-m}{2}} \biggr\} . \end{aligned}$$

Problem PK becomes the following one: find a solution to equation (2.1) with the boundary conditions

$$u|_{\Sigma_{1}^{m} } =0; \quad\quad t^{m} u_{t} \to0,\quad \text{as }t \to+0. $$

The two-dimensional spherical functions, expressed in terms of θ and φ in the traditional definition (see [45]), are $Y_{n}^{s}(\theta,\varphi):=Y_{n}^{s}(x)$, $x \in S^{2}$, $n \in\mathbf{N} \cup\{0\}$, $s=1,2,\ldots,2n+1$, and satisfy the differential equation

$$ \frac{1}{\sin\theta}\frac{\partial}{\partial \theta} \biggl(\sin\theta \frac{\partial}{\partial\theta}Y_{n}^{s} \biggr)+\frac{1}{\sin^{2} \theta} \frac{\partial^{2}}{\partial\varphi^{2}}Y_{n}^{s}+n(n+1)Y_{n}^{s}=0. $$

In the special case when the right-hand side function $f(x,t)$ of equation (2.1) has the form

$$ f(r ,\theta, \varphi,t)=f_{n}^{s}(r,t)Y_{n}^{s}( \theta, \varphi), $$

we may look for a solution of the form

$$ u(r ,\theta, \varphi,t)=u_{n}^{s}(r,t)Y_{n}^{s}( \theta, \varphi) $$

with unknown coefficient $u_{n}^{s}(r,t)$.

For the coefficients $u_{n}^{s}(r ,t)$ which correspond to the right-hand sides $f_{n}^{s}(r ,t)$, we obtain the 2-D equation

$$\begin{aligned} u_{rr}+\frac{2}{r}u_{r}- \bigl(t^{m}u_{t } \bigr)_{t}-\frac{n(n+1)}{r ^{2}}u=f \end{aligned}$$

in the domain

$$G_{m }= \biggl\{ (r ,t):0< t< t_{0}, \frac{2}{2-m}t^{\frac{2-m}{2}}< r < 1-\frac{2}{2-m}t^{\frac{2-m}{2}} \biggr\} , $$

which is bounded by the segment $S_{0} =\{( r ,t): 0 < r <1, t=0\}$ and the characteristics

$$\begin{aligned}& S_{1}^{m}:= \biggl\{ (r ,t):0< t< t_{0}, r =1-\frac{2}{2-m}t^{\frac{2-m}{2}} \biggr\} , \\& S^{m}_{2}:= \biggl\{ (r ,t):0< t< t_{0}, r = \frac{2}{2-m}t^{\frac{2-m}{2}} \biggr\} . \end{aligned}$$

In this case, for $u(r ,t)$, the 2-D problem corresponding to PK is the problem

$$ \mathit {PK}_{1} \quad \textstyle\begin{cases} u_{rr}+\frac{2}{r}u_{r}-(t^{m}u_{t })_{t}-\frac{n(n+1)}{r ^{2}}u=f(r,t) \quad\text{in } G_{m }, \\ u|_{S_{1}^{m}}=0; \quad\quad t^{m} u_{t} \to0,\quad \text{as }t \to+0. \end{cases} $$

Remark 2.1

When the right-hand side function $f(x,t)$ has the form (1.5), it is enough to take test functions $v \in V_{m}$ in the identity (1.4) to have the form $v=w(\vert x\vert ,t)Y_{n}^{s}(x)$, $n \in\mathbf{N} \cup\{0\}$, $s=1,2,\ldots,2n+1$ and

$$\begin{aligned} w \in V_{m}^{(1)}:= \bigl\{ w(r,t): w \in C^{2}( \bar{G}_{m}), w|_{S_{2}^{m}}=0, w \equiv0 \text{ in a neighborhood of }(0,0) \bigr\} . \end{aligned}$$

The generalized solution of problem $\mathit {PK}_{1}$ is defined as follows.

Definition 2.1

We call a function $u(r,t)$ a generalized solution of problem $\mathit {PK}_{1}$ in $G_{m}$ ($0< m<\frac{4}{3}$) if:

• 1. $u, u_{r} \in C(\bar{G}_{m}\setminus(0,0))$, $u_{t}\in C(\bar {G}_{m}\setminus\bar{S}_{0})$;

• 2. $u|_{S_{1}^{m}}=0$;

• 3. For each $\varepsilon\in(0,1)$ there exists a constant $C(\varepsilon)>0$ such that the estimates

  $$\bigl\vert u_{t}(r,t) \bigr\vert \leq C(\varepsilon)t^{-\frac{3m}{4}} $$

  hold in $G_{m,\varepsilon}:=G_{m} \cap\{r> \varepsilon\}$;

• 4. The identity

  $$ \int_{G_{m}} \biggl\{ u_{r}v_{r}-t^{m} u_{t}v_{t}+\frac{n(n+1)}{r^{2}}uv+fv \biggr\} r^{2} \,dr \,dt =0 $$

  (2.2)

  holds for all $v \in V_{m}^{(1)}$.

Substituting the new characteristic coordinates

$$ \xi=1-r -\frac{2}{2-m}t^{\frac{2-m}{2}}, \quad \quad\eta=1-r + \frac{2}{2-m}t^{\frac{2-m}{2}} $$

(2.3)

and the new functions

$$ \begin{aligned} &U(\xi,\eta)=r(\xi,\eta) u \bigl(r (\xi,\eta ),t(\xi, \eta ) \bigr), \\ &V(\xi,\eta)=r(\xi,\eta) v \bigl(r (\xi,\eta),t(\xi,\eta) \bigr), \\ &F(\xi,\eta)=\frac{1}{8}(2-\xi-\eta)f \bigl(r (\xi,\eta),t(\xi,\eta ) \bigr), \end{aligned} $$

(2.4)

from problem $\mathit {PK}_{1}$, we get the 2-D Cauchy-Goursat problem

where

$$D:= \bigl\{ (\xi,\eta):0< \xi< \eta< 1 \bigr\} \subset\mathbf{R}^{2}, $$

and the parameter $\beta=\frac{ m}{ 2(2-m)}\in(0,1)$ since $0< m<\frac{4}{3}$.

The generalized solution of problem $\mathit {PK}_{2}$ is defined as follows.

Definition 2.2

We call a function $U(\xi,\eta )$ a generalized solution of problem $\mathit {PK}_{2}$ in D ($0<\beta<1$) if:

• 1. $U, U_{\xi}+U_{\eta} \in C(\bar{D} \setminus(1,1))$, $U_{\xi}-U_{\eta} \in C(\bar{D}\setminus \{\eta=\xi\})$;

• 2.
  $$ U(0,\eta)=0; $$

  (2.7)

• 3. For each $\varepsilon\in(0,1)$ there exists a constant $C(\varepsilon)>0$ such that

  $$ \bigl\vert (U_{\xi}-U_{\eta}) (\xi,\eta) \bigr\vert \leq C(\varepsilon) (\eta-\xi)^{-\beta}\quad \text{in }\bar{D}_{\varepsilon} \setminus\{\eta=\xi\}, $$

  (2.8)

  where $D_{\varepsilon}:=D \cap\{\xi< 1-\varepsilon\}$;

• 4. The identity

  $$ \int_{D}(\eta-\xi)^{2 \beta} \biggl\{ U_{\xi}V_{\eta}+U_{\eta}V_{\xi} +\frac{2n(n+1)}{(2-\xi-\eta)^{2}}UV+2FV \biggr\} \,d\xi \,d\eta=0 $$

  (2.9)

  holds for all

  $$V \in V^{(2)}:= \bigl\{ V(\xi,\eta): V \in C^{2}(\bar{D}), V (\xi,1)= 0, V \equiv0 \text{ in a neighborhood of }(1,1) \bigr\} . $$

3 Existence and uniqueness of a generalized solution to the Cauchy-Goursat plane problem $\mathit {PK}_{2}$

In this section we prove the existence and uniqueness of a generalized solution to problem $\mathit {PK}_{2}$. In order to do this, we use the Riemann-Hadamard function associated to problem $\mathit {PK}_{2}$ to find an integral representation for a generalized solution of this problem in D. According to Gellerstedt [46] and the results of Nakhushev mentioned in the book of Smirnov [47], this function has the form

$$ \Phi(\xi,\eta; \xi_{0} ,\eta_{0} ) = \textstyle\begin{cases} \Phi^{+}(\xi,\eta; \xi_{0},\eta_{0} ), & \eta> \xi_{0}, \\ \Phi^{-}(\xi,\eta; \xi_{0},\eta_{0} ), & \eta< \xi_{0}, \end{cases} $$

(3.1)

for $(\xi_{0},\eta_{0}) \in D$ and $(\xi,\eta) \in \bar{T} \cup\bar{\Pi} \setminus\{\eta=\xi_{0}\}$, where

$$\begin{aligned} T:= \bigl\{ (\xi,\eta): 0 < \xi< \eta< \xi_{0} \bigr\} , \quad\quad \Pi:= \bigl\{ ( \xi,\eta): 0 < \xi< \xi_{0}, \xi_{0} < \eta< \eta_{0} \bigr\} . \end{aligned}$$

The Riemann-Hadamard function $\Phi(\xi,\eta;\xi_{0},\eta_{0})$ should have the following main properties (see [46, 47]):

• (i) The function Φ as a function of $(\xi_{0},\eta_{0})$ satisfies

  $$ \begin{aligned}[b] E_{\xi_{0},\eta_{0}}[\Phi]&:= \frac{\partial^{2} \Phi }{\partial\xi_{0}\, \partial \eta_{0}}+\frac{\beta}{\eta_{0}-\xi_{0}} \biggl( \frac{\partial \Phi}{\partial\xi_{0}}-\frac{\partial\Phi}{\partial \eta_{0}} \biggr)-\frac{n(n+1)}{(2-\xi_{0}-\eta_{0})^{2}}\Phi \\ &=0 \quad\text{in } D, \eta\neq\xi_{0} \end{aligned} $$

  (3.2)

  and with respect to the first pair of variables $(\xi,\eta)$

  $$ \begin{aligned}[b] E^{\ast}_{\xi,\eta}[\Phi]&:= \frac{\partial^{2} \Phi }{\partial\xi\,\partial \eta}- \frac{\partial}{\partial\xi} \biggl(\frac{\beta \Phi}{\eta-\xi} \biggr)+ \frac{\partial}{\partial\eta} \biggl( \frac{\beta \Phi}{\eta-\xi} \biggr)-\frac{n(n+1)}{(2-\xi-\eta)^{2}}\Phi \\ &=0 \quad\text{in } D, \eta\neq \xi_{0}; \end{aligned} $$

  (3.3)

• (ii) $\Phi^{+}(\xi_{0},\eta_{0};\xi_{0},\eta_{0})=1$;

• (iii) $\Phi^{+}(\xi,\eta_{0};\xi_{0},\eta_{0})= ( \frac{\eta_{0}-\xi}{\eta_{0}-\xi_{0}} )^{\beta}$;

• (iv) $\Phi^{+}(\xi_{0},\eta;\xi_{0},\eta_{0})= ( \frac{\eta-\xi_{0}}{\eta_{0}-\xi_{0}} )^{\beta}$;

• (v) The jump of the function Φ on the line $\{\eta=\xi_{0}\}$ is

  $$\begin{aligned}{} [ [\Phi] ] :=& \lim_{\delta\to +0} \bigl\{ \Phi^{-}(\xi,\xi_{0}-\delta;\xi_{0}, \eta_{0})- \Phi^{+}(\xi,\xi_{0}+\delta; \xi_{0},\eta_{0}) \bigr\} \\ =& \cos(\pi\beta) \lim_{\delta\to +0} \bigl\{ \Phi^{+}( \xi,\xi_{0}+\delta;\xi_{0},\xi_{0}+\delta) \Phi^{+}(\xi_{0},\xi_{0}+\delta; \xi_{0},\eta_{0}) \bigr\} \\ =& \cos(\pi\beta) \biggl(\frac{\xi_{0}-\xi}{\eta_{0}-\xi_{0}} \biggr)^{\beta}; \end{aligned}$$

• (vi) $\Phi^{-}$ vanishes on the line $\{\eta=\xi\}$ of power 2β.

Actually, the function $\Phi^{+}$ is the Riemann function for equation (2.5).

Remark 3.1

In the case $\boldsymbol{0<\beta<1/2}$ and $\boldsymbol{F(\xi,\eta)=(\eta-\xi)^{-4 \beta}f(\xi,\eta)}$, where $f \in C(\bar{D})$, a generalized solution of problem $\mathit {PK}_{2}$ has an explicit integral representation (see [46] and [47]). We find an integral representation in the case $\boldsymbol{0<\beta<1}$ and $F\in C(\bar{D})$ using the properties of the Riemann-Hadamard function $\Phi(\xi,\eta;\xi_{0},\eta_{0})$. The existence of a function $\Phi(\xi,\eta;\xi_{0},\eta_{0})$ with properties $\mbox{(i)} \div\mbox{(vi)}$ is shown in Appendix A (see also [44]).

Theorem 3.1

Let $0<\beta<1$ and $F\in C(\bar{D})$. Then each generalized solution of problem $\mathit {PK}_{2}$ in D has the following integral representation:

$$ U (\xi_{0},\eta_{0}) = \int_{0}^{\xi_{0}} \int_{\xi}^{\eta_{0}}F(\xi,\eta) \Phi(\xi,\eta; \xi_{0},\eta_{0}) \,d\eta \,d\xi. $$

(3.4)

Proof

Let $U(\xi,\eta)$ be a generalized solution of problem $\mathit {PK}_{2}$ in D. For any arbitrary function $\psi(\xi,\eta)$ from $C_{0}^{\infty}(D)$, we have $\psi\in V^{(2)}$, and from (2.9) we obtain the identity

$$ \int_{D}(\eta-\xi)^{2 \beta} \biggl\{ U_{\xi\eta}+ \frac{\beta}{\eta-\xi}(U_{\xi}-U_{\eta}) -\frac{n(n+1)}{(2-\xi-\eta)^{2}}U-F \biggr\} \psi \,d\xi \,d\eta=0, $$

where $U_{\xi\eta}$ is the weak derivative of U. Therefore

$$U_{\xi\eta}=F +\frac{n(n+1)}{(2-\xi-\eta)^{2}}U-\frac{\beta}{\eta-\xi}(U_{\xi}-U_{\eta }) \in C(D) $$

since $F, U, U_{\xi}-U_{\eta} \in C(D)$. From this it follows that $U_{\xi\eta}$ is a classical derivative of U and $U(\xi,\eta)$ satisfies the differential equation (2.5) in D in a classical sense.

Now, using the properties of the Riemann-Hadamard function $\Phi(\xi,\eta;\xi_{0},\eta_{0})$, we obtain the integral representation (3.4) for a generalized solution of problem $\mathit {PK}_{2}$ by integrating the identity

$$\Phi(\xi,\eta;\xi_{0},\eta_{0}) E_{\xi,\eta} \bigl[U( \xi,\eta) \bigr] -U(\xi,\eta) E^{\ast}_{\xi,\eta} \bigl[\Phi(\xi, \eta; \xi_{0},\eta_{0}) \bigr] = F(\xi,\eta) \Phi (\xi,\eta; \xi_{0},\eta_{0}) $$

over a triangle

$$T_{\delta}:= \bigl\{ (\xi,\eta):0 < \xi< \xi_{0}-2 \delta, \xi+ \delta< \eta< \xi_{0} - \delta \bigr\} $$

and then over the rectangle

$$\Pi_{\delta}:= \bigl\{ (\xi,\eta): 0 < \xi< \xi_{0}-2 \delta, \xi_{0} + \delta< \eta< \eta_{0} \bigr\} $$

with $\delta>0$ small enough, and finally letting $\delta\to0$. □

Theorem 3.1 claims the uniqueness of a generalized solution to problem $\mathit {PK}_{2}$. Next, we prove that if $F \in C^{1}(\bar{D})$ and $U(\xi,\eta)$ is a function defined by (3.4) in D, then $U(\xi,\eta)$ is a generalized solution to problem $\mathit {PK}_{2}$ in D. In order to do this, we introduce the notation

$$M_{F}:=\max \Bigl\{ \max_{\bar{D}_{0}}\vert F\vert , \max_{\bar {D}_{0}}\vert F_{\xi}+F_{\eta} \vert \Bigr\} , $$

and we mention that, according to Lemma A.1 (see Appendix A below), the Riemann-Hadamard function $\Phi(\xi,\eta;\xi_{0},\eta_{0})$ can be decomposed in the following way:

$$\Phi(\xi,\eta;\xi_{0},\eta_{0})= H(\xi,\eta; \xi_{0},\eta_{0})+G(\xi,\eta;\xi_{0}, \eta_{0}), $$

where $H(\xi,\eta;\xi_{0},\eta_{0})$ is the Riemann-Hadamard function (A.12) associated to problem $\mathit {PK}_{2}$ in the case $n=0$ and $G(\xi,\eta;\xi_{0},\eta_{0})$ is an additional term. Therefore we can rewrite representation (3.4) in the form

$$ U(\xi_{0},\eta_{0})=U^{H}( \xi_{0},\eta_{0})+U^{G}(\xi_{0}, \eta_{0}), $$

(3.5)

where

$$ U^{H}(\xi_{0},\eta_{0}):= \int_{0}^{\xi_{0}} \int_{\xi}^{\eta_{0}}F(\xi,\eta) H (\xi,\eta; \xi_{0},\eta_{0}) \,d\eta \,d\xi $$

(3.6)

and

$$ U^{G}(\xi_{0},\eta_{0}):= \int_{0}^{\xi_{0}} \int_{\xi}^{\eta_{0}}F(\xi,\eta) G (\xi,\eta; \xi_{0},\eta_{0}) \,d\eta \,d\xi. $$

(3.7)

Firstly, we will study the function $U^{H}(\xi_{0},\eta_{0})$. To do this, we use the estimates for some integrals involving function $H(\xi,\eta;\xi_{0},\eta_{0})$ obtained in Appendix B.

Theorem 3.2

Let $0 < \beta<1$ and $F \in C^{1}(\bar{D})$. Then, for the function $U^{H}(\xi_{0},\eta_{0})$, we have $U^{H}, U^{H}_{\xi_{0}}+U^{H}_{\eta_{0}}\in C(\bar{D}\setminus(1,1))$, $U^{H}_{\eta_{0}}\in C(\bar{D}\setminus\{\eta_{0}=\xi_{0}\})$ and the following estimates hold:

$$\begin{aligned}& \bigl\vert U^{H}(\xi_{0}, \eta_{0}) \bigr\vert \leq K_{1} M_{F} \xi_{0}\quad \textit{in } \bar{D}\setminus(1,1), \\& \bigl\vert U^{H}_{\xi_{0}}+U^{H}_{\eta_{0}} \bigr\vert (\xi_{0},\eta_{0}) \leq K_{1} M_{F} \eta_{0} \quad \textit{in } \bar{D}\setminus(1,1), \\& \bigl\vert U^{H}_{\eta_{0}}(\xi_{0}, \eta_{0}) \bigr\vert \leq K_{1} M_{F} \xi_{0}(\eta_{0}-\xi_{0})^{-\beta} \quad \textit{in } \bar{D}\setminus\{\eta_{0}=\xi_{0}\}, \end{aligned}$$

where $K_{1}>0$ is a constant independent of F.

Proof

Step 1. From (3.6) and (B.1) from Lemma B.1 (see Appendix B) we obtain

$$ \bigl\vert U^{H}(\xi_{0}, \eta_{0}) \bigr\vert \leq M_{F} \int_{0}^{\xi_{0}} \int_{\xi }^{\eta_{0}} H (\xi,\eta;\xi_{0}, \eta_{0}) \,d\eta \,d\xi =M_{F}I^{1}( \xi_{0},\eta_{0}) \leq k_{1} M_{F} \xi_{0}. $$

Step 2. Differentiating (3.6) with respect to $\eta_{0}$ and using (B.4) from Lemma B.2, we obtain

$$ \begin{aligned} \bigl\vert U^{H}_{\eta_{0}}( \xi_{0},\eta_{0}) \bigr\vert &= \biggl\vert \int_{0}^{\xi_{0}} \int_{\xi}^{\eta_{0}}F(\xi,\eta) H_{\eta_{0}} (\xi, \eta;\xi_{0},\eta_{0}) \,d\eta \,d\xi+ \int_{0}^{\xi_{0}}F(\xi,\eta_{0}) \frac{(\eta_{0}-\xi)^{\beta}}{(\eta_{0}-\xi_{0})^{\beta}} \,d\xi \biggr\vert \\ & \leq M_{F} \biggl\{ \int_{0}^{\xi_{0}} \int_{\xi}^{\eta_{0}} \bigl\vert H_{\eta_{0}} (\xi, \eta;\xi_{0},\eta_{0}) \bigr\vert \,d\eta \,d\xi+ \int_{0}^{\xi_{0}} \biggl(\frac{\eta_{0}-\xi}{\eta_{0}-\xi_{0}} \biggr)^{\beta} \,d\xi \biggr\} \\ &= M_{F} \biggl\{ I^{2}(\xi_{0}, \eta_{0})+ \int_{0}^{\xi_{0}} \biggl(\frac{\eta_{0}-\xi}{\eta_{0}-\xi_{0}} \biggr)^{\beta} \,d\xi \biggr\} \\ &\leq M_{F}(k_{2}+1) \xi_{0}( \eta_{0}-\xi_{0})^{-\beta}. \end{aligned} $$

Step 3. According to Remark A.1, the derivatives $H_{\xi_{0}}^{+}$, $H_{\xi_{0}}^{-}$ have singularities of order $\vert \eta-\xi_{0}\vert ^{-1}$ on the line $\{\eta=\xi_{0}\}$. Gellerstedt [46] and Moiseev [48] consider the case $n= 0$ and suggest differentiating (3.6) after appropriate substitutions of variables. In that way one can find integral representations for the first derivatives of the solution which do not involve the first derivatives of function H. In order to do this, following Moiseev [48], we introduce new variables

$$ \tilde{\xi}:=\frac{\xi_{0}-\xi}{\eta_{0}-\xi_{0}}, \quad\quad \tilde{\eta}:= \frac{\eta_{0}-\eta}{\eta_{0}-\xi_{0}}. $$

(3.8)

We define

$$\tilde{H}^{+}(\tilde{\xi},\tilde{\eta}):=H^{+}(\xi,\eta; \xi_{0},\eta_{0}), \quad\quad \tilde{H}^{-}(\tilde{\xi}, \tilde{\eta}):=H^{-}(\xi,\eta;\xi_{0},\eta_{0}), $$

from (A.12) we obtain

$$\begin{aligned}& \tilde{H}^{+} (\tilde{\xi},\tilde{\eta})= (1-\tilde{ \eta}+\tilde{\xi})^{\beta}F \biggl(\beta,1-\beta,1;\frac{\tilde{\xi} \tilde{\eta}}{1-\tilde{\eta}+\tilde{\xi}} \biggr),\quad\tilde{\eta}< 1, \\& \tilde{H}^{-}(\tilde{\xi},\tilde{\eta}) = \frac{k (1-\tilde{\eta}+\tilde{\xi}) ^{2\beta}}{\tilde{\xi}^{\beta}\tilde{\eta}^{\beta}} F \biggl(\beta,\beta,2\beta;\frac{1-\tilde{\eta}+\tilde{\xi}}{\tilde{\xi} \tilde{\eta}} \biggr),\quad\tilde{\eta}>1. \end{aligned}$$

Then we have

$$\begin{aligned} &U^{H}(\xi_{0}, \eta_{0}) \\ &\quad =(\eta_{0}-\xi_{0})^{2} \int_{0}^{\frac{\xi_{0}}{\eta_{0}-\xi_{0}}} \biggl\{ \int_{0}^{1+\tilde{\xi}}F \bigl(\xi_{0}-( \eta_{0}-\xi_{0})\tilde{\xi},\eta _{0}-( \eta_{0}-\xi_{0})\tilde{\eta} \bigr) \tilde{H }(\tilde{\xi}, \tilde{\eta}) \,d\tilde{\eta} \biggr\} \,d\tilde{\xi}, \end{aligned} $$

and

$$ \begin{aligned} & \bigl(U^{H}_{\xi_{0}}+U^{H}_{\eta_{0}} \bigr) (\xi_{0},\eta_{0}) \\ &\quad = (\eta_{0}-\xi_{0})^{2} \int_{0}^{\frac{\xi_{0}}{\eta_{0}-\xi_{0}}} \int_{0}^{1+\tilde{\xi}}(F_{\xi}+F_{\eta}) \bigl(\xi_{0}-(\eta_{0}-\xi_{0})\tilde{\xi}, \eta_{0}-(\eta_{0}-\xi_{0})\tilde{\eta} \bigr) \tilde{H} (\tilde{\xi},\tilde{\eta}) \,d\tilde{\eta} \,d\tilde{\xi} \\ &\quad\quad{} +(\eta_{0}-\xi_{0}) \int_{0}^{\frac{\eta_{0}}{\eta_{0}-\xi_{0}}}F \bigl(0,\eta_{0}-( \eta_{0}-\xi_{0})\tilde{\eta} \bigr) \tilde{H} \biggl( \frac{\xi_{0}}{\eta_{0}-\xi_{0}},\tilde{\eta} \biggr) \,d\tilde{\eta}. \end{aligned} $$

Now the inverse transform of (3.8) gives

$$ \begin{aligned} \bigl(U^{H}_{\xi_{0}}+U^{H}_{\eta_{0}} \bigr) (\xi_{0},\eta_{0})&= \int_{0}^{\xi_{0}} \int_{\xi}^{\eta_{0}}(F_{\xi}+F_{\eta}) (\xi,\eta) H (\xi,\eta;\xi_{0},\eta_{0}) \,d\eta \,d\xi \\ &\quad{} + \int_{0}^{\eta_{0}}F(0,\eta)H(0,\eta;\xi_{0}, \eta_{0}) \,d\eta. \end{aligned} $$

Now (B.1) from Lemma B.1 and (B.6) from Lemma B.3 give

$$\bigl\vert \bigl(U^{H}_{\xi_{0}}+U^{H}_{\eta_{0}} \bigr) (\xi_{0},\eta_{0}) \bigr\vert \leq M_{F} \bigl\{ I^{1}(\xi_{0},\eta_{0})+I^{3}( \xi_{0},\eta_{0}) \bigr\} \leq M_{F}(k_{1}+k_{3}) \eta_{0}. $$

 □

Theorem 3.3

Let the conditions in Theorem  3.2 be fulfilled. Then for the function $U^{G}(\xi_{0},\eta_{0})$ we have $U^{G}, U^{G}_{\xi_{0}}, U^{G}_{\eta_{0}} \in C(\bar{D}\setminus(1,1))$, and the following estimates hold in $\bar{D}\setminus(1,1)$:

$$\begin{aligned}& \bigl\vert U^{G}(\xi_{0}, \eta_{0}) \bigr\vert \leq K_{2}M_{F} \xi_{0}(2-\xi_{0}-\eta_{0})^{-n}, \end{aligned}$$

(3.9)

$$\begin{aligned}& \bigl\vert U^{G}_{\xi_{0}}(\xi_{0}, \eta_{0}) \bigr\vert \leq K_{2}M_{F} \xi_{0}(2-\xi_{0}-\eta_{0})^{-n-1}, \end{aligned}$$

(3.10)

$$\begin{aligned}& \bigl\vert U^{G}_{\eta_{0}}(\xi_{0}, \eta_{0}) \bigr\vert \leq K_{2}M_{F} \xi_{0}(2-\xi_{0}-\eta_{0})^{-n-1}, \end{aligned}$$

(3.11)

where $K_{2}>0$ is a constant independent of F.

Proof

Using estimates (A.26) and (A.27), from (3.7) we obtain estimate (3.9):

$$ \begin{aligned} \bigl\vert U^{G}(\xi_{0}, \eta_{0}) \bigr\vert &= \biggl\vert \int_{0}^{\xi_{0}} \int_{\xi}^{\xi_{0}}F(\xi,\eta) G^{-}(\xi,\eta; \xi_{0},\eta_{0}) \,d\eta \,d\xi \\ &\quad{} + \int_{0}^{\xi_{0}} \int_{\xi_{0}}^{\eta_{0}}F(\xi,\eta) G^{+}(\xi,\eta; \xi_{0},\eta_{0}) \,d\eta \,d\xi \biggr\vert \\ &\leq C_{G}M_{F}\xi_{0} \biggl\{ \frac{1}{2}\xi_{0}(2-\xi_{0}-\eta_{0})^{-n}+( \eta_{0}-\xi_{0})^{1-\beta} \biggr\} \\ &\leq K_{2}M_{F}\xi_{0}(2-\xi_{0}- \eta_{0})^{-n}. \end{aligned} $$

Now we calculate

$$U^{G}_{\xi_{0}}(\xi_{0},\eta_{0})= \int_{0}^{\xi_{0}} \int_{\xi}^{\eta_{0}}F(\xi,\eta ) G_{\xi_{0}}(\xi, \eta,\xi_{0},\eta_{0}) \,d\eta \,d\xi. $$

Here we do not have integrals on the boundaries because $Y=0$ on the line $\{\xi=\xi_{0}\}$, and the function $G(\xi,\eta,\xi_{0},\eta_{0})$ has no jump on the line $\{\eta=\xi_{0}\}$ (see Appendix A). Applying estimates (A.30) and (A.31) to this integral, we have

$$ \begin{aligned} \bigl\vert U^{G}_{\xi_{0}}( \xi_{0},\eta_{0}) \bigr\vert &\leq\frac{M_{F}C_{G}}{(2-\xi_{0}-\eta_{0})^{n+1}} \int_{0}^{\xi_{0}} \int_{\xi}^{\xi_{0}} (\xi_{0}- \eta)^{-\beta} \,d\eta \,d\xi \\ &\quad{} +\frac{M_{F}C_{G}}{2-\xi_{0}-\eta_{0}} \int_{0}^{\xi_{0}} \int_{\xi_{0}}^{\eta_{0}}(\eta-\xi_{0})^{-\beta} \,d\eta \,d\xi \\ &\leq\frac{M_{F} C_{G}}{(2-\xi_{0}-\eta_{0})^{n+1}} \bigl(I_{1}^{1}+2^{n}I_{2}^{1} \bigr). \end{aligned} $$

Now (B.2) and (B.3) from Lemma B.1 give estimate (3.10). Further, we calculate

$$U^{G}_{\eta_{0}}(\xi_{0},\eta_{0})= \int_{0}^{\xi_{0}} \int_{\xi}^{\eta_{0}}F(\xi,\eta ) G_{\eta_{0}}(\xi, \eta;\xi_{0},\eta_{0}) \,d\eta \,d\xi, $$

where we used that $Y=0$ on the line $\eta=\eta_{0}$. Analogously, applying estimates (A.28) and (A.29), which are even better than (A.30) and (A.31), to the last integral for the derivative $G_{\eta_{0}}$, we obtain estimate (3.11). □

As a direct consequence of Theorem 3.2 and Theorem 3.3, in view of $U=U^{H}+U^{G}$, we have the following theorem.

Theorem 3.4

Let $0 < \beta<1$ and $F \in C^{1}(\bar{D})$. Then, for the function $U(\xi,\eta)$ defined by (3.4), we have $U, U_{\xi}+U_{\eta} \in C(\bar{D} \setminus(1,1))$, $U_{\eta} \in C(\bar{D}\setminus \{\eta=\xi\})$ and for some constant $K_{3}>0$ the estimates below hold

$$ \begin{gathered} \bigl\vert U(\xi,\eta) \bigr\vert \leq K_{3}M_{F}\xi(2-\xi-\eta)^{-n} \quad\textit{in }\bar{D} \setminus(1,1), \\ \bigl\vert (U_{\xi}+U_{\eta}) (\xi,\eta) \bigr\vert \leq K_{3}M_{F}(2-\xi-\eta)^{-n-1} \quad\textit{in } \bar{D} \setminus (1,1), \\ \bigl\vert U_{\eta}(\xi,\eta) \bigr\vert \leq K_{3}M_{F} \xi(\eta-\xi)^{-\beta}(2-\xi- \eta)^{-n-1} \quad\textit{in } \bar{D}\setminus\{ \eta=\xi\}. \end{gathered} $$

(3.12)

Now, we are able to prove the following existence result.

Theorem 3.5

Let $0 < \beta<1$ and $F \in C^{1}(\bar{D})$. Then there exists one and only one generalized solution to problem $\mathit {PK}_{2}$ in D, which has integral representation (3.4), and it satisfies estimates (3.12).

Proof

Let $U(\xi,\eta)$ be the function known from Theorem 3.4. Therefore $U, U_{\xi}+U_{\eta} \in C(\bar{D} \setminus(1,1))$, $U_{\eta} \in C(\bar{D}\setminus \{\eta=\xi\})$, and it satisfies estimates (3.12) in Definition 2.2. But in view of (3.12) it is obvious that condition (2.7) and estimate (2.8) hold.

To prove that $U(\xi,\eta)$ satisfies identity (2.9) in Definition 2.2, we need several steps as follows.

Step 1. We prove that $U(\xi,\eta)$ satisfies the differential equation (2.5) in a classical sense and $\frac{\partial}{\partial\eta} (U_{\xi} ) \in C(D)$.

(1.i) Following Smirnov [47], we find another representation formula for the function $U^{H}(\xi,\eta)$. Let us introduce the function

$$ R_{0}(\xi,\eta;\xi_{0},\eta_{0}):= \textstyle\begin{cases} R_{0}^{+}(\xi,\eta;\xi_{0},\eta_{0}), &\eta>\xi_{0},\\ R_{0}^{-}(\xi,\eta;\xi_{0},\eta_{0}), &\eta< \xi_{0}, \end{cases} $$

where

$$ \begin{gathered} R_{0}^{+}(\xi,\eta;\xi_{0}, \eta_{0}) := \biggl(\frac{\eta_{0}-\eta}{\eta_{0}-\xi_{0}} \biggr)^{\beta} \biggl( \frac{\eta_{0}-\eta}{\eta_{0}-\xi} \biggr)^{1-\beta}F_{1} \biggl(1-\beta,\beta,1- \beta,2;\frac{\eta_{0}-\eta}{\eta_{0}-\xi_{0}},\frac{\eta_{0}-\eta}{\eta_{0}-\xi} \biggr), \\ R_{0}^{-}(\xi,\eta;\xi_{0},\eta_{0}) :=\gamma \biggl(\frac{\eta-\xi}{\xi_{0}-\xi} \biggr)^{\beta} \biggl(\frac{\eta-\xi }{\eta_{0}-\xi} \biggr)^{\beta}F_{1} \biggl(\beta,\beta,\beta,1+2 \beta; \frac{\eta-\xi}{\xi_{0}-\xi},\frac{\eta-\xi}{\eta_{0}-\xi} \biggr). \end{gathered} $$

Here $\gamma=- \frac{\Gamma(\beta)}{\Gamma(1-\beta) \Gamma(1+2 \beta )}$ and $F_{1}(a,b_{1},b_{2},c;x,y)$ is the hypergeometric function (A.8) of two variables (see Appendix A).

In [47] the case $0<\beta<1/2$ is considered, but here we find that in a more general case $0<\beta<1$ the function $R_{0}(\xi,\eta;\xi_{0},\eta_{0})$ solves

$$ \begin{gathered} \frac{\partial R_{0}}{\partial\eta}=-(\eta- \xi)^{-1}H( \xi,\eta;\xi_{0},\eta_{0})\quad\text{for }(\xi, \eta) \in\Pi\cup T, \\ R_{0}|_{\eta=\eta_{0}}=0,\quad\quad R_{0}|_{\eta=\xi}=0, \end{gathered} $$

(3.13)

where $(\xi_{0},\eta_{0}) \in D$ and $H(\xi,\eta;\xi_{0},\eta_{0})$ is function (A.12).

Using (3.13), integration by parts and

$$\bigl[R_{0}^{+} - R_{0}^{-} \bigr]\big|_{\eta=\xi_{0}}= \frac{1}{\beta} $$

leads to the integral representation

$$ \begin{aligned}[b] U^{H}(\xi_{0}, \eta_{0}) &:= \int_{0}^{\xi_{0}} \int_{\xi}^{\eta_{0}}\frac{\partial}{\partial\eta} \bigl[(\eta-\xi)F( \xi,\eta) \bigr] R_{0} (\xi,\eta;\xi_{0}, \eta_{0}) \,d\eta \,d\xi \\ &\quad{} +\frac{1}{\beta} \int_{0}^{\xi_{0}}(\xi_{0}-\xi)F(\xi, \xi_{0}) \,d\xi. \end{aligned} $$

(3.14)

(1.ii) Differentiating (3.14) we obtain that $U^{H}$ satisfies the differential equation

$$ \bigl(U^{H}_{\xi_{0}} \bigr)_{\eta_{0}}+ \frac{\beta}{\eta_{0}-\xi_{0}} \bigl( U^{H}_{ \xi_{0}}- U^{H}_{\eta_{0}} \bigr)=F(\xi_{0},\eta_{0}), $$

(3.15)

where all derivatives are in a classical sense and they are continuous in D.

(1.iii) Since $H(\xi,\eta;\xi_{0},\eta_{0})$ satisfies the differential equation (3.2) with $n=0$ and $\Phi =H+G$ satisfies (3.2) with $n \geq0$ for the difference $G=\Phi- H$, we obtain

$$G_{\xi_{0}\eta_{0}}+\frac{\beta}{\eta_{0}-\xi_{0}} ( G_{ \xi_{0}}- G_{\eta_{0}} )- \frac{n(n+1)}{(2-\xi_{0}-\eta_{0})^{2}}G=\frac{n(n+1)}{(2-\xi_{0}-\eta_{0})^{2}}H. $$

Now, using integral representation (3.7) for $U^{G}(\xi_{0},\eta _{0})$, we calculate

$$ \begin{aligned}[b] & \bigl(U^{G}_{\xi_{0}} \bigr)_{\eta_{0}}+\frac{\beta}{\eta_{0}-\xi_{0}} \bigl( U^{G}_{ \xi_{0}}- U^{G}_{\eta_{0}} \bigr)-\frac{n(n+1)}{(2-\xi_{0}-\eta_{0})^{2}}U^{G} \\ &\quad = \int_{0}^{\xi_{0}} \int_{\xi}^{\eta_{0}}F(\xi,\eta) \biggl[G_{\xi_{0}\eta_{0}}+ \frac{\beta}{\eta_{0}-\xi_{0}} ( G_{ \xi_{0}}- G_{\eta_{0}} ) \\ &\quad\quad{} -\frac {n(n+1)}{(2-\xi_{0}-\eta_{0})^{2}}G \biggr](\xi,\eta;\xi_{0},\eta_{0}) \,d\eta \,d\xi \\ &\quad =\frac{n(n+1)}{(2-\xi_{0}-\eta_{0})^{2}} \int_{0}^{\xi_{0}} \int_{\xi}^{\eta_{0}}F(\xi,\eta) H(\xi,\eta; \xi_{0},\eta_{0}) \,d\eta \,d\xi \\ &\quad =\frac{n(n+1)}{(2-\xi_{0}-\eta_{0})^{2}}U^{H}, \end{aligned} $$

(3.16)

where all derivatives are in a classical sense and they are continuous in D.

(1.iv) Since $U=U^{H}+U^{G}$, summing up equations (3.15) and (3.16), we obtain the differential equation

$$(U_{\xi_{0}} )_{\eta_{0}}+\frac{\beta}{\eta_{0}-\xi_{0}} ( U_{ \xi_{0}}- U_{\eta_{0}} )-\frac{n(n+1)}{(2-\xi_{0}-\eta_{0})^{2}}U=F(\xi_{0},\eta_{0}) $$

in a classical sense. But, since $F, U, U_{\xi_{0}}-U_{\eta_{0}} \in C(D)$, it follows that $(U_{\xi_{0}} )_{\eta _{0}}\in C(D)$.

Step 2. We will prove that identity (2.9) holds for all $V(\xi,\eta) \in V^{(2)}$.

(2.i) Let $V(\xi,\eta) \in V^{(2)}$ and in addition $V(\xi,\eta)\equiv0$ in a neighborhood of $\{\eta=\xi\}$ and in a neighborhood of $\{\eta=1\}$. From Step 1 we know that $U(\xi,\eta)$ satisfies the differential equation (2.5), where all derivatives are in a classical sense, continuous in D. Let us consider

$$ I_{V}:= \int_{D}(\eta-\xi)^{2 \beta} \biggl\{ U_{\xi}V_{\eta}+U_{\eta}V_{\xi} +\frac{2n(n+1)}{(2-\xi-\eta)^{2}}UV+2FV \biggr\} \,d\xi \,d\eta. $$

(3.17)

Now we integrate by parts in $I_{V}$ in the following way:

• in the term $U_{\xi}V_{\eta}$, we move the derivative from $V_{\eta}$ to $U_{\xi}$ and obtain the term $(U_{\xi} )_{\eta}V$:

  $$ \int_{D}(\eta-\xi)^{2 \beta}U_{\xi}V_{\eta} \,d\xi \,d\eta= - \int_{D}(\eta-\xi)^{2 \beta} \biggl[ (U_{\xi} )_{\eta}+\frac{2\beta}{\eta-\xi}U_{\xi} \biggr]V \,d\xi \,d\eta; $$

  (3.18)

• in the term $U_{\eta}V_{\xi}$, we move the derivative from $U_{\eta}$ to $V_{\xi}$ and obtain the term $U (V_{\xi} )_{\eta}$:

  $$\int_{D}(\eta-\xi)^{2 \beta}U_{\eta}V_{\xi} \,d\xi \,d\eta= - \int_{D}(\eta-\xi)^{2 \beta} \biggl[ (V_{\xi} )_{\eta}+\frac{2\beta}{\eta-\xi}V_{\xi} \biggr]U \,d\xi \,d\eta. $$

There are not integrals on the boundary of D because $U(0,\eta)=0$, $V(\xi,\eta)\equiv0$ in a neighborhood of $\{\eta=\xi\}$ and in a neighborhood of $\{\eta=1\}$.

• since $V \in C^{2}(\bar{D})$, we have $(V_{\xi} )_{\eta}= (V_{\eta } )_{\xi}$;

• in the term $(V_{\eta} )_{\xi}U$, we move the derivatives from $(V_{\eta} )_{\xi}$ to U and obtain the term $(U_{\xi} )_{\eta}V$:

  $$ \begin{aligned}[b] \int_{D}(\eta-\xi)^{2 \beta}U_{\eta}V_{\xi} \,d\xi \,d\eta&= - \int_{D}(\eta-\xi)^{2 \beta} \biggl[ (V_{\eta} )_{\xi}+\frac{2\beta}{\eta-\xi}V_{\xi} \biggr]U \,d\xi \,d\eta \\ &= \int_{D}(\eta-\xi)^{2 \beta} \biggl[U_{\xi}V_{\eta}- \frac{2\beta}{\eta-\xi}(V_{\xi}+V_{\eta})U \biggr] \,d\xi \,d\eta \\ &=- \int_{D}(\eta-\xi)^{2 \beta} \biggl[ (U_{\xi} )_{\eta}-\frac{2\beta}{\eta-\xi}U_{\eta} \biggr]V \,d\xi \,d\eta. \end{aligned} $$

  (3.19)

Again there are not integrals on the boundary of D, and putting (3.18) and (3.19) into (3.17), we get

$$ I_{V}= -2 \int_{D}(\eta-\xi)^{2 \beta} \biggl\{ (U_{\xi} )_{\eta}+\frac{\beta}{\eta-\xi}(U_{\xi}-U_{\eta}) - \frac{n(n+1)}{(2-\xi-\eta)^{2}}U-F \biggr\} V \,d\xi \,d\eta=0. $$

(3.20)

(2.ii) Let $V(\xi,\eta) \in V^{(2)}$ and $\Psi(s)$ be a function having the properties $\Psi(s) \in C^{\infty}(\mathbf{R}^{1})$, $\Psi(s)=1$ for $s\geq2$, $\Psi(s)=0$ for $s\leq1$. If $k, l \in\mathbf{N}$, then according to (2.i) (see (3.17) and (3.20)) for the functions

$$V_{k,l}(\xi,\eta):=V(\xi,\eta)\Psi \bigl(k [1-\eta] \bigr)\Psi \bigl(l [ \eta-\xi] \bigr) $$

identity (2.9) holds. Therefore we have

$$ \begin{aligned}[b] 0&= \int_{D}(\eta-\xi)^{2 \beta} \biggl\{ U_{\xi}V_{\eta}+U_{\eta}V_{\xi} +\frac{2n(n+1)}{(2-\xi-\eta)^{2}}UV+2FV \biggr\} \\ &\quad{}\times \Psi \bigl(k [1-\eta] \bigr) \Psi \bigl(l [ \eta-\xi] \bigr) \,d\xi \,d\eta \\ &\quad{} + \int_{D}l(\eta-\xi)^{2 \beta} \{U_{\xi}-U_{\eta} \}\Psi \bigl(k [1-\eta] \bigr)\Psi' \bigl(l [\eta-\xi] \bigr) V \,d \xi \,d\eta \\ &\quad{} - \int_{D}k(\eta-\xi)^{2 \beta}U_{\xi} \Psi' \bigl(k [1-\eta] \bigr) \Psi \bigl(l [\eta-\xi] \bigr)V \,d\xi \,d\eta \\ &=: I_{1,kl}+I_{2,kl}+I_{3,kl}. \end{aligned} $$

(3.21)

Obviously, $I_{1,kl} \to I_{V}$, as $k, l \to\infty$.

We know that $V \equiv0$ in a neighborhood of $(0,0)$ and $\operatorname {supp}\Psi'(l[\eta-\xi])$ is contained in $\{1\leq l[\eta-\xi]\leq2 \}$. Using estimate (3.12) we find that on $\operatorname {supp}\Psi'(l[\eta-\xi])$ the functions

$$W_{k,l}(\xi,\eta):=l(\eta-\xi)^{2 \beta} \{U_{\xi}-U_{\eta} \}\Psi \bigl(k [1-\eta] \bigr)\Psi' \bigl(l [\eta-\xi] \bigr) V $$

satisfy the estimates

$$ \bigl\vert W_{k,l}(\xi,\eta) \bigr\vert \leq \mathit{const.} ( \eta-\xi)^{\beta-1}. $$

(3.22)

Since, obviously, the sequence ${W_{k,l}}$ converges pointwise almost everywhere to zero and it is dominated by a Lebesgue integrable function in D for $0<\beta<1$ (see (3.22)). Thus, according to the Lebesgue dominated convergence theorem, $I_{2,kl} \to0$ as $k, l \to \infty$.

Since $V(\xi,1)=0$, we have

$$k \bigl\vert V(\xi,\eta) \bigr\vert \bigl\vert \Psi' \bigl(k [1-\eta] \bigr) \bigr\vert = k(1-\eta) \bigl\vert V_{\eta}(\xi, \tilde{ \eta}) \bigr\vert \bigl\vert \Psi' \bigl(k [1-\eta] \bigr) \bigr\vert \leq c_{v}, $$

where $c_{v}$ is a constant and $\eta<\tilde{\eta}<1$. Therefore $I_{3,kl} \to0$ as $k, l \to\infty$.

Thus, letting $k, l \to\infty$ in (3.21), we obtain that identity (2.9) holds for $V \in V^{(2)}$. Consequently, the function $U(\xi,\eta)$ is a generalized solution to problem $\mathit {PK}_{2}$. □

4 Proof of the main results

In this section we give the proofs of Theorem 1.1, Theorem 1.2 and Theorem 1.3 formulated in Section 1.

Proof of Theorem 1.1

Let and $u_{1}$ and $u_{2}$ be two generalized solutions of problem PK in $\Omega_{m}$. Then the function $u:=u_{1}-u_{2}$ solves the homogeneous problem PK. We will show that the Fourier expansion

$$u(r ,\theta, \varphi ,t)=\sum_{n=0}^{\infty} \sum_{s=1}^{2n+1}u_{n}^{s}(r,t)Y_{n}^{s}( \theta, \varphi) $$

has zero Fourier-coefficients

$$\begin{aligned} u_{n}^{s}(r ,t):= \int_{0}^{\pi} \int_{0}^{2\pi} u(r,\theta,\varphi,t)Y_{n}^{s}( \theta,\varphi)\sin\theta \,d\varphi \,d\theta \end{aligned}$$

in $G_{m}$, i.e., $u \equiv0$ in $\Omega_{m }$.

For u we know that the identity

$$\begin{aligned} \int_{\Omega_{m}} \bigl\{ t^{m} u_{t}v_{t}-u_{x_{1}}v_{x_{1}}-u_{x_{2}}v_{x_{2}}-u_{x_{3}}v_{x_{3}} \bigr\} \,dx_{1}\,dx_{2}\,dx_{3}\,dt =0 \end{aligned}$$

(4.1)

holds for all test functions $v(x,t)=w(r,t)Y_{n}^{s}(x)$ described in Remark 2.1. Therefore from (4.1) we derive

$$ \int_{G_{m}} \biggl\{ u_{n,r}^{s}w_{r}-t^{m} u_{n,t}^{s}w_{t}+\frac{n(n+1)}{r^{2}}u_{n}^{s}w \biggr\} r^{2} \,dr \,dt =0 $$

(4.2)

for all $w(r,t)\in V_{m}^{(1)}$ (see Definition 2.1), $n \in\mathbf{N}\cup\{0\}$, $s=1,2,\ldots, 2n+1$. Since $u(x,t)$ satisfies conditions (1), (2) and (3) in Definition 1.1, the functions $u_{n}^{s}(r,t)$ satisfy conditions (1), (2) and (3) in Definition 2.1, and therefore they are generalized solutions of problem $\mathit {PK}_{1}$.

Using (2.3) we see that the functions $V(\xi,\eta):=r(\xi,\eta)w(r(\xi,\eta),t(\xi,\eta)) \in V^{(2)}$. Now from (4.2) we obtain that for the functions $U_{n}^{s}(\xi,\eta):=r(\xi,\eta)u_{n}^{s}(r(\xi,\eta),t(\xi,\eta))$ the identity

$$ \int_{D}(\eta-\xi)^{2 \beta} \biggl\{ U_{n,\xi}^{s}V_{\eta}+U_{n,\eta }^{s}V_{\xi} +\frac{2n(n+1)}{(2-\xi-\eta)^{2}}U_{n}^{s}V \biggr\} \,d\xi \,d\eta=0 $$

holds for all $V(r,t)\in V^{(2)}$ (see Definition 2.2), $n \in\mathbf{N}\cup\{0\}$, $s=1,2,\ldots, 2n+1$. The functions $U_{n}^{s}(\xi,\eta)$ satisfy conditions (1), (2) and (3) in Definition 2.2 and, consequently, $U_{n}^{s}(\xi,\eta)$ are generalized solutions of the 2-D homogeneous problem $\mathit {PK}_{2}$. Theorem 3.1 gives $U_{n}^{s}(\xi,\eta) \equiv0$ in D. Therefore $u_{n}^{s}(r,t)\equiv0$ in $G_{m}$ and thus $u=u_{1}-u_{2} \equiv0$ in $\Omega_{m}$. □

Proof of Theorem 1.2

From Theorem 1.1 it follows that there exists at most one generalized solution of problem PK in $\Omega_{m}$. Since $f(x,t)$ has the form (1.5), we look for a generalized solution of the form (1.6), i.e.,

$$u(x,t)=\sum_{n=0}^{l}\sum _{s=1}^{2n+1}u_{n}^{s} \bigl( \vert x\vert ,t \bigr)Y_{n}^{s}(x). $$

To find such a solution means to find functions $u_{n}^{s}(r,t)$ that satisfy the identities

$$\begin{aligned} \int_{G_{m,\varepsilon}} \biggl[u_{n,r }^{s}v_{r }-t^{m} u_{n,t}^{s}v_{t} +\frac{n(n+1)}{r^{2}}u_{n}^{s}v+f_{n}^{s}v \biggr]r^{2} \,dr \,dt =0 \end{aligned}$$

for all $v \in V_{m}^{(1)}$ and satisfy the corresponding conditions (1), (2) and (3) in Definition 2.1. In view of (2.3) to find such functions means to find functions

$$U_{n}^{s}(\xi,\eta)=r(\xi,\eta)u_{n}^{s} \bigl(r(\xi,\eta),t(\xi,\eta) \bigr), $$

such that for $F_{n}^{s}(\xi,\eta):=\frac{1}{4}r(\xi,\eta)f_{n}^{s}(r(\xi,\eta),t(\xi,\eta))$ the identity

$$ \int_{D}(\eta-\xi)^{2 \beta} \biggl\{ U_{n,\xi}^{s}V_{\eta}+U_{n,\eta }^{s}V_{\xi} +\frac{2n(n+1)}{(2-\xi-\eta)^{2}}U_{n}^{s}V+2F_{n}^{s}V \biggr\} \,d\xi \,d\eta=0 $$

holds for all $V(\xi,\eta)=r(\xi,\eta)v(r(\xi,\eta),t(\xi,\eta)) \in V^{(2)}$ and satisfies the corresponding conditions (1), (2) and (3) in Definition 2.2. Theorem 3.5 gives the existence of such functions $U^{s}_{n}(\xi,\eta)$ which are generalized solutions of problem $\mathit {PK}_{2}$ in D. In that way we found functions $u_{n}^{r}(r,t)=r^{-1}U_{n}^{s}(\xi(r,t),\eta(r,t))$ which are generalized solutions of problem $\mathit {PK}_{1}$ in $G_{m}$. Therefore the function $u(x ,t)$, given by (1.6), is a generalized solution of problem PK in $\Omega_{m}$. □

Proof of Theorem 1.3

Theorem 1.1 and Theorem 1.2 claim the existence and uniqueness of generalized solutions $u(x ,t)$ of problem PK in $\Omega_{m}$, which has the form (1.6). Using (2.3) for functions $U_{n}^{s}(\xi,\eta)=r(\xi,\eta)u_{n}^{s}(r(\xi,\eta),t(\xi,\eta))$ and $F_{n}^{s}(\xi,\eta)=\frac{1}{4}r(\xi,\eta)f_{n}^{s}(r (\xi,\eta),t(\xi ,\eta))$, we obtain the 2-D problem $\mathit {PK}_{2}$. According to Theorem 3.5, estimates (3.12) hold, and in view of (3.6) and (3.7) we see that the estimate for $\vert U_{n}^{s}(\xi,\eta)\vert$ holds with ${ ( \max_{\bar{G}_{m}} \vert f_{n}^{s}\vert )}$ instead of $M_{F}$:

$$\bigl\vert U_{n}^{s}(\xi,\eta) \bigr\vert \leq K \Bigl( \max_{\bar{G}_{m}} \bigl\vert f_{n}^{s} \bigr\vert \Bigr) (2-\xi-\eta)^{-n} $$

with a constant $K>0$ independent of $f_{n}^{s}$. That implies

$$\bigl\vert u_{n}^{s}(r,t) \bigr\vert \leq2^{-n}K \Bigl(\max_{\bar{G}_{m}} \bigl\vert f_{n}^{s} \bigr\vert \Bigr) r^{-n-1}. $$

Therefore in view of (1.6), summing up over n and s, we get the desired estimate (1.7). □

Remark 4.1

It is interesting that in the case $0< m<1$ problem PK for the Keldysh-type equation (1.1) can be formally reduced to problem P2 for the Tricomi-type equation (1.2) with power of degeneration $m_{1}:=m/(1-m)>0$ and the right-hand side function, which vanishes on $\Sigma_{0}$ like $t^{m_{1}}$. That implies many differences between the investigation and behavior of the solution to the obtained problem and the usual Protter problem P2. However, in the present paper we study the $(3+1)$-D problem PK in a more general case when $\boldsymbol{0< m<4/3}$.

Appendices

Appendix 1: The Riemann-Hadamard function $\Phi(\xi,\eta,\xi_{0},\eta_{0})$

Firstly, to aid the reader, we briefly recall some known properties of the hypergeometric function of Gauss $F(a,b,c;\zeta)$ that we will use.

If $c \ne0,-1,-2,\ldots$ , then

$$ F(a,b,c;\zeta) := \sum_{i=0}^{\infty}\frac{(a)_{i} (b)_{i} }{i! (c)_{i} } \zeta^{i}, $$

(A.1)

with $(a)_{i} =\Gamma(a+i)/\Gamma(a)$, where Γ is the Euler gamma function of Euler. For $i\in\mathbf{N}$, one has $(a)_{i}=a(a+1)\cdots (a+i-1)$, $(a)_{0}=1$.

The series (A.1) converges absolutely for $\zeta\in \mathbf{C}$ with $\vert \zeta \vert < 1$ and also for $\vert \zeta \vert = 1$ if $\operatorname{Re}(c-a-b) > 0$. If $-1 < \operatorname{Re}(c-a-b) <0$, then the series converges conditionally for $\vert \zeta \vert =1$ with $\zeta\neq1$.

We mention the following properties of the hypergeometric function (see [31, 49, 50]):

$$ F(a,b,c;\zeta)=\frac{\Gamma(c)}{\Gamma(a)\Gamma(c-a)} \int_{0}^{1} t^{a-1} (1-t)^{c-a-1} (1-\zeta t)^{-b} \,dt $$

(A.2)

for $\zeta\in\mathbf{C}$, $0<\operatorname{Re}(a) < \operatorname{Re}(c)$, $\vert \arg(1-\zeta )\vert <\pi$.

In the case $c-a-b>0$:

$$ \bigl\vert F(a,b,c;\zeta) \bigr\vert \leq \mathit{const.}, \quad\quad F(a,b,c;1) = \frac{\Gamma(c)\Gamma(c-a-b)}{\Gamma(c-a)\Gamma(c-b) }; $$

(A.3)

resp. $c-a-b<0$:

$$ F(a,b,c;\zeta) = (1-\zeta)^{c-a-b} F(c-a,c-b,c;\zeta) $$

(A.4)

and

$$ \bigl\vert F(a,b,c;\zeta) \bigr\vert \leq \mathit{const.}(1-\zeta )^{c-a-b}; $$

(A.5)

resp. $c-a-b=0$: For each $\alpha>0$, there exists a constant $c(\alpha)>0$ such that

$$\begin{aligned}& \bigl\vert F(a,b,c;\zeta \bigr\vert \leq c(\alpha) (1-\zeta )^{-\alpha }, \end{aligned}$$

(A.6)

$$\begin{aligned}& \frac{d}{d\zeta} F(a,b,c;\zeta) = \frac{ab}{c} F(a+1,b+1,c+1;\zeta). \end{aligned}$$

(A.7)

The hypergeometric function of two variables is defined by

$$ F_{1}(a,b_{1},b_{2},c;x,y):=\sum _{i=0}^{\infty}\sum _{j=0}^{\infty} \frac{(a)_{i+j}(b_{1})_{j}(b_{2})_{i}}{(c)_{i+j}i!j!}x^{j}y^{i}. $$

(A.8)

The series converges absolutely for $x,y \in\mathbf{C}$ with $\vert x\vert <1$, $\vert y\vert <1$ (for more properties of $F_{1}$, see [49], pp.224-228).

Now, in the case $n \in\mathbf{N}\cup\{0\}$, we construct the following Riemann-Hadamard function of the form (3.1) associated to problem $\mathit {PK}_{2}$: For $(\xi_{0},\eta_{0}) \in D$

$$ \begin{gathered} {\Phi^{+} = \biggl(\frac{\eta-\xi}{\eta_{0}-\xi_{0} } \biggr)^{\beta}F_{3}(\beta,n+1,1-\beta,-n,1;X,Y)}, \quad \eta> \xi_{0}, \\ {\Phi^{-} = k \biggl(\frac{\eta-\xi}{\eta_{0}-\xi_{0} } \biggr)^{\beta}X^{-\beta} H_{2} \biggl(\beta,\beta,-n,n+1,2\beta; \frac{1}{X},-Y \biggr)}, \quad \eta< \xi_{0}, \end{gathered} $$

(A.9)

where

$$\begin{aligned}& k= \frac{\Gamma(\beta)}{\Gamma(1-\beta)\Gamma(2\beta)}, \\& X = X(\xi,\eta,\xi_{0},\eta_{0}):= \frac{(\xi_{0}-\xi) (\eta_{0}-\eta) }{(\eta- \xi) (\eta_{0} - \xi_{0}) }, \\& Y=Y(\xi,\eta,\xi_{0},\eta_{0}):=-\frac{(\xi_{0}-\xi)(\eta_{0}-\eta)}{(2-\xi-\eta )(2-\xi_{0}-\eta_{0})}. \end{aligned}$$

Here $F_{3}(a_{1},a_{2},b_{1},b_{2},c;x,y)$ is the Appell series

$$ F_{3}(a_{1},a_{2},b_{1},b_{2},c;x,y):= \sum_{i=0}^{\infty}\sum _{j=0}^{\infty} \frac{(a_{1})_{j}(a_{2})_{i}(b_{1})_{j}(b_{2})_{i}}{(c)_{i+j}i!j!}x^{j}y^{i} $$

(A.10)

which converges absolutely for $x,y \in\mathbf{C}$ with $\vert x\vert <1$, $\vert y\vert <1$ (see [49], pp.224-228) and $H_{2}(a_{1},a_{2},b_{1},b_{2},c;x,y)$ is the Horn series

$$ H_{2}(a_{1},a_{2},b_{1},b_{2},c;x,y):= \sum_{i=0}^{\infty}\sum _{j=0}^{\infty} \frac{(a_{1})_{j-i}(a_{2})_{j}(b_{1})_{i}(b_{2})_{i}}{(c)_{j}i!j!}x^{j}y^{i} $$

(A.11)

which converges absolutely for $x,y \in\mathbf{C}$ with $\vert x\vert <1$, $\vert y\vert (1+\vert x\vert )<1$ (see [49], pp.224-228).

We mention that for $(\xi_{0},\eta_{0})\in D$ we have $\vert X\vert <1$ in Π̄ and $1/\vert X\vert <1$ in T̄, while $\vert Y\vert <1$ in Π̄ but $\vert Y\vert$ could be greater than 1 in T. However, the function Φ is well defined because $n \in\mathbf{N}\cup\{0\}$, since $b_{1}=-n$, and we have a finite sum with respect to i in the function $H_{2}$ (see (A.11)), which appears in (A.9). We will fix all these properties a little bit later.

Let, for $(\xi_{0},\eta_{0}) \in D$ and $(\xi,\eta) \in\bar{T} \cup\bar{\Pi} \setminus\{\eta=\xi_{0}\}$, us introduce the functions

$$ H(\xi,\eta; \xi_{0} ,\eta_{0} ) = \textstyle\begin{cases} H^{+}(\xi,\eta; \xi_{0},\eta_{0} ), & \eta> \xi_{0}, \\ H^{-}(\xi,\eta; \xi_{0},\eta_{0} ), & \eta< \xi_{0}, \end{cases} $$

(A.12)

where

$$ \begin{gathered} H^{+}(\xi,\eta;\xi_{0}, \eta_{0}) = \biggl(\frac{\eta-\xi}{\eta_{0}-\xi_{0} } \biggr)^{\beta}F( \beta,1-\beta,1;X), \\ H^{-}(\xi,\eta;\xi_{0},\eta_{0}) = k \biggl( \frac{\eta-\xi}{\eta_{0}-\xi_{0} } \biggr)^{\beta}X^{-\beta} F \biggl(\beta,\beta,2 \beta;\frac{1}{X} \biggr) \end{gathered} $$

and

$$ G(\xi,\eta; \xi_{0} ,\eta_{0} ) = \textstyle\begin{cases} G^{+}(\xi,\eta; \xi_{0} ,\eta_{0} ) , &\eta> \xi_{0}, \\ G^{-} (\xi,\eta; \xi_{0} ,\eta_{0} ), & \eta< \xi_{0}, \end{cases} $$

(A.13)

where

$$\begin{aligned}& {G^{+}(\xi,\eta;\xi_{0}, \eta_{0}):= \biggl(\frac{\eta-\xi }{\eta_{0}-\xi_{0} } \biggr)^{\beta}\sum _{i=1}^{n} c_{i} Y^{i} F(\beta, 1-\beta,i+1; X)}, \end{aligned}$$

(A.14)

$$\begin{aligned}& {G^{-}(\xi,\eta;\xi_{0},\eta_{0}):=k \biggl( \frac{\eta-\xi }{\eta_{0}-\xi_{0} } \biggr)^{\beta}X^{-\beta}\sum _{i=1}^{n} d_{i} Y^{i} F \biggl(\beta-i, \beta,2\beta; \frac{1}{X} \biggr)}, \\& c_{i}:=\frac{(n+1)_{i}(-n)_{i}}{i! i!}, \quad\quad d_{i}:=\frac{(n+1)_{i}(-n)_{i}}{(1-\beta)_{i} i!}. \end{aligned}$$

(A.15)

Now we prove the following important lemma.

Lemma A.1

The function $\Phi(\xi,\eta;\xi_{0},\eta_{0})$ has the following decomposition:

$$ \Phi(\xi,\eta;\xi_{0},\eta_{0})=H(\xi,\eta; \xi_{0},\eta_{0})+G(\xi,\eta;\xi_{0}, \eta_{0}). $$

(A.16)

Proof

(i) In view of (A.10) we have

$$F_{3}(\beta,n+1,1-\beta,-n,1;X,Y)=\sum_{i=0}^{n} \sum_{j=0}^{\infty}\frac{(\beta)_{j}(1-\beta)_{j}(n+1)_{i}(-n)_{i}}{(1)_{i+j} i! j!}X^{j}Y^{i}. $$

Since $(1)_{i+j}=(i+j)!=i! (i+1)_{j}$ for $i,j \in\mathbf{N} \cup \{0\}$, we obtain from (A.1) and (A.9)

$$\begin{aligned} \Phi^{+}(\xi,\eta;\xi_{0}, \eta_{0}) &= \biggl(\frac{\eta-\xi }{\eta_{0}-\xi_{0} } \biggr)^{\beta}\Biggl\{ \sum_{j=0}^{\infty}\frac{(\beta)_{j}(1-\beta)_{j}}{(1)_{j} j!}X^{j} + \sum_{i=1}^{n} c_{i}Y^{i} \sum_{j=0}^{\infty}\frac{(\beta)_{j}(1-\beta)_{j}}{(i+1)_{j} j!}X^{j} \Biggr\} \\ &= \biggl(\frac{\eta-\xi}{\eta_{0}-\xi_{0} } \biggr)^{\beta}\Biggl\{ F(\beta,1-\beta,1;X) + \sum_{i=1}^{n} c_{i}Y^{i} F(\beta,1-\beta,i+1;X) \Biggr\} \\ &=H^{+}(\xi,\eta;\xi_{0},\eta_{0})+G^{+}( \xi,\eta;\xi_{0},\eta_{0}). \end{aligned} $$

(ii) In view of (A.11) we have

$$H_{2} \biggl(\beta,\beta,-n,n+1,2 \beta;\frac{1}{X},Y \biggr):= \sum_{i=0}^{n}\sum _{j=0}^{\infty} \frac{(\beta)_{j-i}(\beta)_{j}(-n)_{i}(1-n)_{i}}{(2 \beta)_{j}i!j!}X^{-j}(-Y)^{i}. $$

We mention that for $0<\beta<1$ and $i,j \in\mathbf{N} \cup\{0\}$

$$(\beta)_{j-i}=\frac{\Gamma(\beta+j-i)}{\Gamma(\beta)}=\frac{\Gamma(\beta -i)}{\Gamma(\beta)}( \beta-i)_{j}, $$

and using the relation $\Gamma(z)\Gamma(1-z)=\frac{\pi}{\sin(\pi z)}$ we calculate

$$\begin{aligned} (\beta)_{j-i}(1-\beta)_{i}&=( \beta-i)_{j}\frac{\Gamma(\beta-i)\Gamma(1-\beta +i)}{\Gamma(\beta)\Gamma(1-\beta)} \\ &=(\beta-i)_{j}\frac{\sin(\beta\pi)}{\sin((\beta-i)\pi)} \\ &=(-1)^{i}(\beta-i)_{j}. \end{aligned} $$

Now, from (A.1) and (A.9) we obtain

$$\begin{aligned} \Phi^{-}(\xi,\eta;\xi_{0}, \eta_{0}) &= k \biggl(\frac{\eta-\xi }{\eta_{0}-\xi_{0} } \biggr)^{\beta}X^{-\beta} \Biggl\{ \sum_{j=0}^{\infty} \frac{(\beta)_{j}(\beta)_{j}}{(2 \beta)_{j} j!}\frac{1}{X^{j}} + \sum_{i=1}^{n} d_{i}Y^{i} \sum_{j=0}^{\infty} \frac{(\beta-i)_{j}(\beta)_{j}}{(2 \beta)_{j} j!}\frac {1}{X^{j}} \Biggr\} \\ &=k \biggl(\frac{\eta-\xi}{\eta_{0}-\xi_{0} } \biggr)^{\beta}X^{-\beta} \Biggl\{ F \biggl(\beta,\beta,2\beta;\frac{1}{X} \biggr) + \sum _{i=1}^{n} d_{i}Y^{i} F \biggl( \beta-i,\beta,2 \beta;\frac{1}{X} \biggr) \Biggr\} \\ &=H^{-}(\xi,\eta;\xi_{0},\eta_{0})+G^{-}( \xi,\eta;\xi_{0},\eta_{0}). \end{aligned} $$

 □

We mention here that function (A.9) is closely connected to the Riemann-Hadamard function announced in [51], p.25, example 7, which is associated to a Cauchy-Goursat problem for an equation connected with (2.5) with some appropriate substitutions. Actually, the function $H(\xi,\eta; \xi_{0} ,\eta_{0} )$ is the Riemann-Hadamard function associated to problem $\mathit {PK}_{2}$ in the case $n=0$ (see Gellerstedt [46] and Smirnov [47]). It is well known that the function $H(\xi,\eta; \xi_{0} ,\eta_{0} )$ has the properties $\mbox{(i)} \div\mbox{(vi)}$ listed in Section 3. It is not difficult to check that in the case $n \geq0$ function $\Phi(\xi,\eta; \xi_{0} ,\eta_{0} )$ has the same properties. Using the systems of differential equations that $F_{3}$ and $H_{2}$ satisfy (see [49], pp.233-234), with a straightforward calculation we check that the function $\Phi(\xi,\eta; \xi_{0} ,\eta_{0} )$ satisfies equations (3.2) and (3.3). Further, since $X(\xi_{0},\eta,\xi_{0},\eta_{0})=X(\xi,\eta_{0},\xi_{0},\eta_{0})=0$, $Y(\xi_{0},\eta,\xi_{0},\eta_{0})=Y(\xi,\eta_{0},\xi_{0},\eta_{0})=0$, we see that the function Φ has the properties (ii), (iii) and (iv). We also have $X(\xi,\xi,\xi_{0},\eta_{0})=0$, and therefore the function $G(\xi,\eta; \xi_{0} ,\eta_{0})$ vanishes on the line $\{\eta=\xi\}$ of power 2β. Therefore the function Φ has the properties (vi). Let us calculate the jump of the function Φ on the line $\{\eta=\xi_{0}\}$. We will show that the function G has no jump on the line $\{\eta=\xi_{0}\}$. Using (A.3) and the relation $\Gamma(i)=(i-1)!$ for $i \in\mathbf{N}$, we calculate

$$c_{i} F(\beta, 1-\beta,i+1; 1)=k d_{i} F(\beta-i, \beta,2 \beta; 1)=\frac{(n+1)_{i}(-n)_{i}}{i \Gamma(1-\beta+i)\Gamma(\beta+i)}. $$

In view of (A.14) and (A.15) we have

$$\begin{aligned} G^{+}(\xi,\xi_{0}; \xi_{0},\eta_{0})&=G^{-}(\xi,\xi_{0}; \xi_{0},\eta_{0}) \\ &= \biggl(\frac{\xi_{0}-\xi}{\eta_{0}-\xi_{0} } \biggr)^{\beta}\sum _{i=1}^{n} \frac{(n+1)_{i}(-n)_{i}}{i \Gamma(1-\beta+i) \Gamma(\beta+i)} Y^{i}(\xi, \xi_{0},\xi_{0},\eta_{0}). \end{aligned} $$

Therefore the jump $[[G]]=0$, and in view of (A.16) we have $[[\Phi]]=[[H]]$. Consequently, the function Φ has the property (v) since $[[H]]=\cos(\pi\beta) (\frac{\xi_{0}-\xi}{\eta_{0}-\xi_{0}} )^{\beta}$ (see Gellerstedt [46]).

1.1 1.1 The function $H(\xi,\eta,\xi_{0},\eta_{0})$

Using the properties of a hypergeometric function mentioned above and the relations

$$\begin{aligned}& 1-X = \frac{(\eta_{0}-\xi) (\eta-\xi_{0}) }{(\eta- \xi) (\eta_{0} - \xi_{0}) }, \quad\quad 1- \frac{1}{X} = \frac{(\eta_{0}-\xi) (\xi_{0}-\eta) }{(\xi_{0}- \xi) (\eta_{0} - \eta) }, \\& X_{\xi_{0}}=\frac{(\eta_{0}-\xi) (\eta_{0}-\eta) }{(\eta -\xi) (\eta_{0} - \xi_{0})^{2} }, \quad\quad X_{\eta_{0}}= \frac{(\eta-\xi_{0}) (\xi_{0}-\xi) }{(\eta-\xi) (\eta_{0} - \xi_{0})^{2} }, \end{aligned}$$

(A.17)

$$\begin{aligned}& \biggl(\frac{1}{X} \biggr)_{\xi_{0}}= \frac{(\xi-\eta_{0}) (\eta-\xi) }{(\eta_{0}-\eta) (\xi_{0} - \xi)^{2} }, \quad\quad \biggl(\frac{1}{X} \biggr)_{\eta_{0}}= \frac{(\xi_{0}-\eta) (\eta-\xi) }{(\xi_{0} - \xi)(\eta_{0}-\eta)^{2} } , \end{aligned}$$

(A.18)

we prove the following lemma.

Lemma A.2

Let $0 < \beta<1$ and $0 < \xi_{0} < \eta _{0} <1$. Then there exists a constant $C_{H}>0$ such that

$$\begin{aligned}& \bigl\vert H^{+}(\xi,\eta;\xi_{0}, \eta_{0}) \bigr\vert \leq C_{H}(\eta- \xi_{0})^{-\beta},\quad(\xi,\eta) \in\Pi, \end{aligned}$$

(A.19)

$$\begin{aligned}& \bigl\vert H^{-}(\xi,\eta;\xi_{0},\eta_{0}) \bigr\vert \leq C_{H}(\xi_{0}-\eta)^{-\beta}, \quad( \xi,\eta) \in T, \end{aligned}$$

(A.20)

$$\begin{aligned}& \bigl\vert H^{+}_{\eta_{0}}(\xi,\eta; \xi_{0},\eta_{0}) \bigr\vert \leq C_{H} \frac{(\eta-\xi_{0})^{-\beta}}{\eta_{0}-\xi_{0}}, \quad(\xi,\eta) \in\Pi, \end{aligned}$$

(A.21)

$$\begin{aligned}& \bigl\vert H^{-}_{\eta_{0}}(\xi,\eta;\xi_{0}, \eta_{0}) \bigr\vert \leq C_{H}\frac{(\xi_{0}-\eta)^{-\beta}}{\eta_{0}-\eta }, \quad (\xi,\eta) \in T. \end{aligned}$$

(A.22)

Proof

(i) Using (A.6) we find that for each $\alpha>0$ there exists a constant $c(\alpha)>0$ such that

$$ \begin{gathered} \bigl\vert H^{+}(\xi, \eta; \xi_{0},\eta_{0}) \bigr\vert \leq c(\alpha) \biggl( \frac{\eta-\xi }{\eta_{0}-\xi_{0} } \biggr)^{\beta}(1-X)^{-\alpha} \\ \hphantom{\bigl\vert H^{+}(\xi, \eta; \xi_{0},\eta_{0}) \bigr\vert }=c(\alpha)\frac{(\eta-\xi)^{\alpha +\beta}(\eta_{0}-\xi_{0})^{\alpha-\beta}}{(\eta-\xi_{0})^{\alpha}(\eta_{0}-\xi )^{\alpha}} , \\ \bigl\vert H^{-}(\xi,\eta;\xi_{0},\eta_{0}) \bigr\vert \leq c(\alpha) \frac{ (\eta-\xi) ^{2\beta}}{(\xi_{0}-\xi)^{\beta}(\eta_{0}-\eta)^{\beta} } \biggl(1-\frac{1}{X} \biggr)^{-\alpha} \\ \hphantom{\bigl\vert H^{-}(\xi,\eta;\xi_{0},\eta_{0}) \bigr\vert } =c(\alpha)\frac{(\eta-\xi)^{2\beta}(\xi_{0}-\xi)^{\alpha-\beta}(\eta _{0}-\eta)^{\alpha-\beta}}{(\eta_{0}-\xi)^{\alpha}(\xi_{0}-\eta)^{\alpha}}. \end{gathered} $$

(A.23)

From here, choosing $\alpha=\beta$, we obtain estimates (A.19), (A.20).

(ii) In view of (A.17), (A.18) for the derivatives with respect to $\eta_{0}$, using (A.4) and (A.7), we obtain

$$\begin{aligned} \bigl\vert H^{+}_{\eta_{0}} \bigr\vert &= \biggl\vert - \frac{\beta}{\eta_{0} -\xi_{0}} H^{+} +\beta(1-\beta) \biggl(\frac{\eta-\xi }{\eta_{0}-\xi_{0} } \biggr)^{\beta}X_{\eta_{0}}F(1+\beta,2-\beta,2;X) \biggr\vert \\ &=\frac{\beta(\eta- \xi)^{\beta}}{(\eta_{0} - \xi_{0})^{1+\beta}} \biggl\vert -F(\beta,1-\beta,1;X)+(1-\beta) \frac{\xi_{0}-\xi}{\eta_{0}-\xi} F(1-\beta,\beta,2;X) \biggr\vert \\ &\leq c(\alpha) \frac{(\eta-\xi)^{\beta}}{(\eta_{0}-\xi_{0})^{1+\beta} }(1-X)^{-\alpha}\leq c(\alpha) \frac{(\eta_{0}-\xi_{0})^{\alpha-\beta-1}}{(\eta-\xi_{0})^{\alpha}}, \end{aligned} $$

and

$$\begin{aligned} \bigl\vert H^{-}_{\eta_{0}} \bigr\vert &= \biggl\vert \frac{\beta H^{-}}{\eta-\eta_{0}} + \frac{k \beta(\eta-\xi) ^{2\beta }}{2(\xi_{0}-\xi)^{\beta}(\eta_{0}-\eta)^{\beta}} \biggl(\frac{1}{X} \biggr)_{\eta_{0}}F \biggl(1+\beta,1+\beta,1+2\beta;\frac{1}{X} \biggr) \biggr\vert \\ &= \frac{k\beta(\eta-\xi) ^{2\beta}}{(\xi_{0}-\xi)^{\beta}(\eta_{0}-\eta)^{1+\beta}} \biggl\vert -F \biggl(\beta,\beta,2\beta; \frac{1}{X} \biggr)+ \frac{1}{2}\frac{\eta-\xi}{\eta_{0}-\xi}F \biggl(\beta, \beta,1+2\beta;\frac{1}{X} \biggr) \biggr\vert \\ &\leq c(\alpha) \frac{ (\eta-\xi) ^{2\beta}}{(\xi_{0}-\xi)^{\beta}(\eta_{0}-\eta)^{1+\beta} } \biggl(1-\frac{1}{X} \biggr)^{-\alpha}\leq c(\alpha)\frac{(\eta_{0}-\eta)^{\alpha-\beta -1}}{(\xi_{0}-\eta)^{\alpha}}. \end{aligned} $$

Now we choose $\alpha=\beta$ to obtain the desired estimates (A.21), (A.22). □

Remark A.1

In the same manner, for the derivatives with respect to $\xi_{0}$, we obtain

$$ H^{+}_{\xi_{0}}=\beta\frac{(\eta- \xi)^{\beta}}{(\eta _{0} - \xi_{0})^{1+\beta}} \biggl[F(\beta,1-\beta,1;X)+(1-\beta)\frac{\eta_{0}-\eta}{\eta-\xi_{0}} F(1-\beta,\beta,2;X) \biggr] $$

(A.24)

and

$$ H^{-}_{\xi_{0}}=- \frac{k\beta(\eta-\xi) ^{2\beta}}{(\xi_{0}-\xi)^{1+\beta} (\eta_{0}-\eta)^{\beta}} \biggl[F \biggl(\beta,\beta,2\beta;\frac{1}{X} \biggr)+ \frac{1}{2} \frac{\eta-\xi}{\xi_{0}-\eta}F \biggl(\beta,\beta,1+2\beta;\frac{1}{X} \biggr) \biggr]. $$

(A.25)

In the case $0 <\beta< 1/2$, Smirnov [47] and Meredov [52] claim

$$ \bigl\vert H^{+} _{\xi_{0}}(\xi,\eta; \xi_{0},\eta _{0}) \bigr\vert \leq \frac{(\eta-\xi)(\eta_{0}-\xi_{0})^{-2\beta}}{(\eta_{0}-\xi) ^{1-\beta} (\eta-\xi_{0}) ^{1-\beta}}, $$

i.e., $H^{+} _{\xi_{0}}(\xi,\eta;\xi_{0},\eta_{0})$ has integrable singularity on $\{\eta=\xi_{0}\}$. As we see from (A.24) and (A.25), the derivative with respect to $\xi_{0}$ of function H has not integrable singularity on $\{\eta=\xi_{0}\}$.

1.2 1.2 The function $G(\xi,\eta,\xi_{0},\eta_{0})$

In this section we prove some properties of the function $G(\xi,\eta;\xi_{0},\eta_{0})$ defined by (A.13).

Lemma A.3

Let $0 < \beta<1$ and $0 < \xi_{0} < \eta _{0} <1$. Then there exists a constant $C_{G}>0$ such that

$$\begin{aligned}& \bigl\vert G^{+}(\xi,\eta;\xi_{0}, \eta_{0}) \bigr\vert \leq C_{G} (\eta_{0}- \xi_{0})^{-\beta},\quad(\xi,\eta) \in\Pi, \end{aligned}$$

(A.26)

$$\begin{aligned}& \bigl\vert G^{-}(\xi,\eta;\xi_{0},\eta_{0}) \bigr\vert \leq C_{G} (2-\xi_{0}-\eta_{0})^{-n}, \quad (\xi,\eta) \in T , \end{aligned}$$

(A.27)

$$\begin{aligned}& \bigl\vert G^{+}_{\eta_{0}}(\xi,\eta, \xi_{0},\eta_{0}) \bigr\vert \leq C_{G} \frac{(\eta_{0}-\xi_{0})^{-\beta}}{2-\xi_{0}-\eta_{0}},\quad(\xi,\eta)\in\Pi, \end{aligned}$$

(A.28)

$$\begin{aligned}& \bigl\vert G^{-}_{\eta_{0}}(\xi,\eta,\xi_{0}, \eta_{0}) \bigr\vert \leq C_{G}\frac{(\eta_{0}-\eta)^{-\beta}}{(2-\xi _{0}-\eta_{0})^{n+1}}, \quad( \xi,\eta)\in T, \end{aligned}$$

(A.29)

$$\begin{aligned}& \bigl\vert G^{+}_{\xi_{0}}(\xi,\eta; \xi_{0},\eta_{0}) \bigr\vert \leq { C_{G} \frac{(\eta-\xi_{0})^{-\beta}}{(2-\xi_{0}-\eta_{0}) }}, \quad(\xi,\eta)\in \Pi, \end{aligned}$$

(A.30)

$$\begin{aligned}& \bigl\vert G^{-}_{\xi_{0}}(\xi,\eta;\xi_{0}, \eta_{0}) \bigr\vert \leq {C_{G}\frac{(\xi_{0}-\eta)^{-\beta}}{(2-\xi _{0}-\eta_{0})^{n+1} } }, \quad(\xi,\eta)\in T. \end{aligned}$$

(A.31)

Proof

First, we mention that

$$Y_{\xi_{0}}=-\frac{(2-\xi-\eta_{0})(\eta_{0}-\eta)}{(2-\xi -\eta)(2-\xi_{0}-\eta_{0})^{2}}, \quad\quad Y_{\eta_{0}}=- \frac{(2-\xi_{0}-\eta)(\xi_{0}-\xi)}{(2-\xi-\eta)(2-\xi_{0}-\eta_{0})^{2}}. $$

(i) Let $(\xi,\eta) \in\Pi$. Then we have

$$ \begin{gathered} \vert X_{\xi_{0}}\vert \leq \frac{\eta_{0}-\xi}{(\eta-\xi)(\eta_{0}-\xi_{0})}, \\ \vert Y\vert < 1, \quad \quad\frac{\vert Y\vert }{\eta_{0}-\xi _{0}}\leq\frac{\vert Y\vert (\eta_{0}-\xi)}{(\eta-\xi)(\eta_{0}-\xi _{0})}\leq \frac{1}{2-\xi_{0}-\eta_{0}}, \\ \vert Y_{\xi_{0}}\vert \leq\frac{1}{2-\xi_{0}-\eta_{0}}, \vert Y_{\eta_{0}}\vert \leq\frac{2}{2-\xi_{0}-\eta_{0}}. \end{gathered} $$

(A.32)

According to (A.3), $\vert F(\beta, 1-\beta,i+1; X)\vert \leq \mathit{const.}$, $i=1,2,\ldots,n$, in expression (A.14) for $G^{+}$. Therefore estimate (A.26) holds.

With use of (A.7) we calculate the derivative with respect to $\xi_{0}$

$$\begin{aligned} G^{+}_{\xi_{0}}&= \biggl( \frac{\eta-\xi}{\eta_{0}-\xi_{0}} \biggr)^{\beta}\Biggl\{ \sum _{i=1}^{n} c_{i} \biggl[ \frac{\beta Y^{i}}{\eta_{0}-\xi_{0}} +i Y^{i-1}Y_{\xi_{0}} \biggr] F(\beta, 1- \beta,i+1; X) \\ &\quad{} + \beta(1-\beta)\sum_{i=1}^{n} \frac{c_{i}}{i+1} Y^{i} X_{\xi_{0}} F(\beta+1, 2-\beta,i+2; X) \Biggr\} \end{aligned} $$

and the derivative with respect to $\eta_{0}$

$$\begin{aligned} G^{+}_{\eta_{0}}&= \biggl( \frac{\eta-\xi}{\eta_{0}-\xi_{0} } \biggr)^{\beta}\Biggl\{ \sum _{i=1}^{n}c_{i} \biggl[ - \frac{\beta Y^{i}}{\eta_{0}-\xi_{0}}+i Y^{i-1}Y_{\eta_{0}} \biggr] F(\beta, 1- \beta,i+1; X) \\ &\quad{} + \beta(1-\beta)\sum_{i=1}^{n} \frac{c_{i}}{i+1} Y^{i} X_{\eta_{0}} F(\beta+1, 2-\beta,i+2; X) \Biggr\} . \end{aligned} $$

According to (A.3) and (A.6), for the hypergeometric functions in the expressions for $G^{+}_{\xi_{0}}$ and $G^{+}_{\eta_{0}}$, we have

$$ \begin{gathered} \bigl\vert F(\beta+1, 2-\beta,3; X) \bigr\vert \leq c( \alpha)\frac{(\eta_{0}-\xi_{0})^{\alpha} (\eta-\xi)^{\alpha} }{(\eta-\xi_{0})^{\alpha} (\eta_{0} - \xi)^{\alpha} }\leq c(\alpha) \biggl(\frac{\eta_{0}-\xi_{0}}{\eta-\xi_{0}} \biggr)^{\alpha} , \quad\alpha>0, \\ \bigl\vert F(1+\beta, 2-\beta,i+2; X) \bigr\vert \leq \mathit{const.},\quad i=2,3, \ldots,n. \end{gathered} $$

Therefore in view of (A.17) we have

$$\begin{aligned}& \bigl\vert G^{+}_{\xi_{0}} \bigr\vert \leq \frac{C_{1}(\alpha)}{(\eta_{0}-\xi _{0})^{\beta}} \biggl\{ \frac{\vert Y\vert }{\eta_{0}-\xi_{0}}+\frac{1}{2-\xi_{0}-\eta_{0}} + \frac{\vert Y\vert (\eta_{0}-\xi)}{(\eta-\xi)(\eta_{0}-\xi_{0})} \biggl(\frac{\eta_{0}-\xi_{0}}{\eta-\xi_{0}} \biggr)^{\alpha} \biggr\} , \\& \bigl\vert G^{+}_{\eta_{0}} \bigr\vert \leq \frac{C_{2}(\alpha)}{(\eta_{0}-\xi _{0})^{\beta}} \biggl\{ \frac{\vert Y\vert }{\eta_{0}-\xi_{0}}+\frac{2}{2-\xi_{0}-\eta_{0}} + \frac{\vert Y\vert (\xi_{0}-\xi)}{(\eta-\xi)(\eta_{0}-\xi_{0})} \biggl(\frac{\eta-\xi_{0}}{ \eta_{0} - \xi_{0}} \biggr)^{1-\alpha} \biggr\} . \end{aligned}$$

Now, taking $\alpha=\beta\in(0,1)$ and using (A.32), we obtain estimates (A.30) and (A.28).

(ii) Let $(\xi,\eta) \in T$. Then we have

$$ \begin{gathered} \biggl\vert \biggl( \frac{1}{X} \biggr)_{\xi_{0}} \biggr\vert \leq\frac{\eta_{0}-\xi}{(\xi_{0}-\xi)(\eta _{0}-\eta)}, \\ \vert Y\vert < \frac{1}{2-\xi_{0}-\eta_{0}}, \quad \quad\frac{\vert Y\vert }{\eta_{0}-\eta}\leq \frac{1}{2-\xi_{0}-\eta_{0}}, \\ \frac{\vert Y\vert }{\xi_{0}-\xi}\leq\frac{\vert Y\vert (\eta_{0}-\xi)}{(\xi_{0}-\xi)(\eta_{0}-\eta)}\leq\frac{1}{2-\xi_{0}-\eta_{0}}, \\ \vert Y_{\xi_{0}}\vert \leq\frac{1}{(2-\xi_{0}-\eta_{0})^{2}}, \vert Y_{\eta_{0}}\vert \leq\frac{1}{(2-\xi_{0}-\eta_{0})^{2}}. \end{gathered} $$

(A.33)

According to (A.3) $\vert F(\beta-i, \beta,2\beta ;1/X)\vert \leq \mathit{const.}$, $i=1,2,\ldots,n$, in expression (A.15) for $G^{-}$. Since

$$\biggl(\frac{\eta-\xi}{\eta_{0}-\xi_{0}} \biggr)^{\beta}X^{-\beta}= \frac{(\eta-\xi)^{2\beta}}{(\xi_{0}-\xi)^{\beta}(\eta_{0}-\eta)^{\beta}}, $$

we see that estimate (A.27) holds.

With use of (A.7) we calculate the derivative with respect to $\xi_{0}$

$$\begin{aligned} G^{-}_{\xi_{0}}&= \frac{k (\eta-\xi)^{2\beta}}{(\xi_{0}-\xi)^{\beta}(\eta_{0}-\eta)^{\beta}} \Biggl\{ \sum_{i=1}^{n}d_{i} \biggl[-\frac{\beta Y^{i}}{\xi_{0}-\xi}+i Y^{i-1} Y_{\xi_{0}} \biggr] F \biggl(\beta-i,\beta,2\beta; \frac{1}{X} \biggr) \\ &\quad{} + \frac{1}{2}\sum_{i=1}^{n} ( \beta-i)d_{i} Y^{i} \biggl(\frac{1}{X} \biggr)_{\xi_{0}} F \biggl(\beta-i+1,\beta+1,2\beta+1; \frac{1}{X} \biggr) \Biggr\} \end{aligned} $$

and the derivative with respect to $\eta_{0}$

$$\begin{aligned} G^{-}_{\eta_{0}}&= \frac{k (\eta-\xi)^{2\beta}}{(\xi_{0}-\xi)^{\beta}(\eta_{0}-\eta)^{\beta}} \Biggl\{ \sum_{i=1}^{n}d_{i} \biggl[-\frac{\beta Y^{i}}{\eta_{0}-\eta}+i Y^{i-1}Y_{\eta_{0}} \biggr] F \biggl( \beta-i,\beta,2\beta; \frac{1}{X} \biggr) \\ &\quad{} + \frac{1}{2}\sum_{i=1}^{n} ( \beta-i)d_{i} Y^{i} \biggl( \frac{1}{X} \biggr)_{\eta_{0}} F \biggl(\beta-i+1,\beta+1,2\beta+1; \frac{1}{X} \biggr) \Biggr\} . \end{aligned} $$

According to (A.3) and (A.6), for the hypergeometric functions in the expressions for $G^{-}_{\xi_{0}}$ and $G^{-}_{\eta_{0}}$, we have

$$ \begin{aligned} & \biggl\vert F \biggl(\beta, \beta+1,2\beta+1; \frac{1}{X} \biggr) \biggr\vert \leq c(\alpha)\frac{(\eta_{0}-\eta )^{\alpha} (\xi_{0}-\xi)^{\alpha} }{(\xi_{0}-\eta)^{\alpha} (\eta_{0} - \xi)^{\alpha} }\leq c( \alpha) \biggl(\frac{\eta_{0}-\eta}{\xi_{0}-\eta} \biggr)^{\alpha}, \quad \alpha>0, \\ & \bigl\vert F(\beta-i+1,1+ \beta,1+2\beta;1/X) \bigr\vert \leq \mathit{const.},\quad i=2,3, \ldots,n. \end{aligned} $$

Now using (A.18) we calculate

$$\begin{aligned} \bigl\vert G^{-}_{\xi_{0}} \bigr\vert &\leq\frac{C_{3}(\alpha)}{(\eta_{0}-\eta )^{\beta}} \biggl\{ \frac{\vert Y\vert }{\xi_{0}-\xi}+\frac{1}{(2-\xi_{0}-\eta_{0})^{2}} + \frac{\vert Y\vert (\eta_{0}-\xi)}{(\xi_{0}-\xi)(\eta_{0}-\eta)} \biggl(\frac{\eta_{0}-\eta}{\xi_{0}-\eta} \biggr)^{\alpha} \biggr\} \sum_{i=1}^{n}\vert Y\vert ^{i-1} \\ & \leq3 C_{3}(\alpha)\frac{(\eta_{0}-\eta)^{\alpha- \beta}}{(\xi_{0}-\eta )^{\alpha}} \sum _{i=1}^{n} (2-\xi_{0}- \eta_{0})^{-i-1} \end{aligned} $$

and for $0<\alpha<1$

$$\begin{aligned} \bigl\vert G^{-}_{\eta_{0}} \bigr\vert &\leq\frac{C_{4}(\alpha)}{(\eta_{0}-\eta )^{\beta}} \biggl\{ \frac{\vert Y\vert }{\eta_{0}-\eta}+\frac{1}{(2-\xi_{0}-\eta _{0})^{2}} \\ & \quad{} +\frac{\vert Y\vert (\eta-\xi)}{(\xi_{0}-\xi)(\eta_{0}-\eta)} \biggl(\frac{\xi_{0}-\eta}{\eta_{0}-\eta} \biggr)^{1-\alpha} \biggr\} \sum_{i=1}^{n}\vert Y\vert ^{i-1} \\ &\leq \frac{3C_{4}(\alpha)}{(\eta_{0}-\eta)^{\beta}}\sum_{i=1}^{n} (2-\xi _{0}-\eta_{0})^{-i-1}. \end{aligned} $$

Therefore, taking $\alpha=\beta\in(0,1)$, we obtain estimates (A.29) and (A.31). □

Appendix 2: Auxiliary results

Lemma B.1

Suppose $0< \beta< 1$ and $0 < \xi_{0} < \eta _{0} <1$. Then

$$ I^{1}(\xi_{0},\eta_{0}):= \int_{0}^{\xi_{0}} \int_{\xi}^{\eta_{0}} H (\xi,\eta;\xi_{0}, \eta_{0}) \,d\eta \,d\xi\leq k_{1} \xi_{0}. $$

(B.1)

Proof

From (A.19) and (A.20) we obtain

$$\begin{aligned} I^{1} &\leq C_{H} \biggl\{ \int_{0}^{\xi_{0}} \int_{\xi}^{\xi_{0}}(\xi_{0}- \eta)^{-\beta} \,d\eta \,d\xi+ \int_{0}^{\xi_{0}} \int_{\xi_{0}}^{\eta_{0}}(\eta-\xi_{0})^{-\beta} \,d\eta \,d\xi \biggr\} \\ &=:C_{H} \bigl\{ I_{1}^{1}+I_{2}^{1} \bigr\} . \end{aligned} $$

Now we obtain

$$ I_{1}^{1}=\frac{\xi_{0}^{2-\beta}}{(1-\beta)(2-\beta)} $$

(B.2)

and

$$ I_{2}^{1}= \frac{1}{1-\beta} \xi_{0}(\eta_{0}-\xi_{0})^{1-\beta}. $$

(B.3)

Therefore estimate (B.1) holds. □

Lemma B.2

Suppose $0< \beta< 1$ and $0 < \xi_{0} < \eta_{0} <1$. Then

$$ I^{2}(\xi_{0},\eta_{0}):= \int_{0}^{\xi_{0}} \int_{\xi}^{\eta_{0}} \bigl\vert H_{\eta_{0}} (\xi, \eta;\xi_{0},\eta_{0}) \bigr\vert \,d\eta \,d\xi\leq k_{2} \xi_{0}(\eta_{0}-\xi_{0})^{-\beta}. $$

(B.4)

Proof

From (A.21) and (A.22) we obtain

$$\begin{aligned} I^{2}&\leq C_{H} \biggl\{ \int_{0}^{\xi_{0}} \int_{\xi}^{\xi_{0}}\frac{(\xi_{0}-\eta)^{-\beta}}{\eta_{0}-\eta} \,d\eta \,d\xi + \int_{0}^{\xi_{0}} \int_{\xi_{0}}^{\eta_{0}}\frac{(\eta-\xi_{0})^{-\beta}}{\eta_{0}-\xi_{0}} \,d\eta \,d\xi \biggr\} \\ &=:C_{H} \biggl\{ I_{1}^{2}+\frac{I^{1}_{2}}{\eta_{0}-\xi_{0}} \biggr\} . \end{aligned} $$

In $I_{1}^{2}$ we substitute $\eta=\xi+(\xi_{0}-\xi)\sigma$ and, according to (A.2), we get

$$\begin{aligned} I_{1}^{2} =& \int_{0}^{\xi_{0}} \biggl[ \int_{0}^{1} (1-\sigma)^{-\beta} \biggl(1- \frac{\xi_{0} -\xi}{\eta_{0}-\xi}\sigma \biggr)^{-1} \,d\sigma \biggr] \frac{(\xi_{0}-\xi)^{1-\beta}}{\eta_{0}-\xi} \,d\xi \\ =&\frac{\Gamma (1)\Gamma(1-\beta)}{\Gamma(2-\beta)} \int_{0}^{\xi_{0}}F(1, 1,2 -\beta;\zeta) \frac{(\xi_{0}-\xi)^{1-\beta}}{\eta_{0}-\xi} \,d\xi, \end{aligned}$$

where $\zeta=\frac{\xi_{0} -\xi}{\eta_{0}-\xi}$. Since $c-a-b=-\beta <0$, according to (A.5), the hypergeometric function $\vert F\vert \leq \mathit{const.} (1-\zeta)^{-\beta}$. Therefore

$$ I_{1}^{2} \leq c_{1} ( \eta_{0}-\xi_{0})^{-\beta} \int_{0}^{\xi_{0}} \biggl(\frac{\xi_{0} -\xi}{\eta_{0}-\xi} \biggr)^{1-\beta} \,d\xi\leq c_{1} \xi_{0}( \eta_{0}-\xi_{0})^{-\beta}. $$

(B.5)

Now (B.3) and (B.5) give estimate (B.4). □

Lemma B.3

Suppose $0< \beta< 1$ and $0 < \xi_{0} < \eta_{0} <1$. Then

$$ I^{3}(\xi_{0},\eta_{0}):= \int_{0}^{\eta_{0}} H (0,\eta;\xi_{0}, \eta_{0}) \,d\eta\leq k_{3} \eta_{0}. $$

(B.6)

Proof

Using (A.23) with $\alpha=\beta$, we obtain

$$ \begin{aligned} I^{3} &\leq c(\beta)\eta_{0}^{-\beta} \biggl\{ \int_{0}^{\xi_{0}}\frac{\eta^{2\beta}}{(\xi_{0}-\eta)^{\beta}} \,d\eta+ \int_{\xi_{0}}^{\eta_{0}}\frac{\eta^{2\beta}}{(\eta-\xi_{0})^{\beta}} \,d\eta \biggr\} \\ & \leq c(\beta)\eta_{0}^{-\beta} \biggl\{ \xi_{0}^{2\beta} \int_{0}^{\xi_{0}}(\xi_{0}- \eta)^{-\beta} \,d\eta+ \eta_{0}^{2\beta} \int_{\xi_{0}}^{\eta_{0}}(\eta-\xi_{0})^{-\beta} \,d\eta \biggr\} \\ &=\frac{ c(\beta)}{1-\beta}\eta_{0}^{-\beta} \bigl\{ \xi_{0}^{1+\beta}+\eta_{0}^{2\beta}( \eta_{0}-\xi_{0})^{1-\beta} \bigr\} \\ &\leq k_{3} \eta_{0}. \end{aligned} $$

 □