Nonexistence of positive solutions for the weighted higher-order elliptic system with Navier boundary condition

Abstract

We establish a Liouville-type theorem for a weighted higher-order elliptic system in a wider exponent region described via a critical curve. We first establish a Liouville-type theorem to the involved integral system and then prove the equivalence between the two systems by using superharmonic properties of the differential systems. This improves the results in (Complex Var. Elliptic Equ. 5:1436–1450, 2013) and (Abstr. Appl. Anal. 2014:593210, 2014).

1 Introduction

In this paper, we establish a Liouville-type theorem for the weighted 2mth-order elliptic equations coupled via the Navier boundary conditions in the half-space $\mathbb{R}_{+}^{n}=\{x\in \mathbb{R}^{n}: x_{n}>0\}$:

$$\begin{aligned} \textstyle\begin{cases} (-\Delta )^{m}( \vert x \vert ^{\alpha} u(x))= \vert x \vert ^{-\beta} v^{q}\quad\text{in } \mathbb{R}^{n}_{+}, \\ (-\Delta )^{m}( \vert x \vert ^{\beta}v(x))= \vert x \vert ^{-\alpha}u^{p}\quad\text{in } \mathbb{R}^{n}_{+}, \\ u=\Delta u=\cdots =\Delta ^{m-1}u=0 \quad\text{on } \partial \mathbb{R}^{n}_{+}, \\ v=\Delta v=\cdots =\Delta ^{m-1}v=0 \quad\text{on } \partial \mathbb{R}^{n}_{+}, \end{cases}\displaystyle \end{aligned}$$

(1.1)

where m is a positive integer satisfying $0<2m<n$, $p,q\geq 1$, and $\alpha, \beta \geq 0$, which is closely related to the following integral system:

$$\begin{aligned} \textstyle\begin{cases} u(x)=C_{n}\int _{\mathbb{R}^{n}_{+}} \frac{1}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}} (\frac{1}{ \vert x-y \vert ^{n-2m}}- \frac{1}{ \vert \bar{x}-y \vert ^{n-2m}} )v^{q} (y)\,dy, \\ v(x)=C_{n}\int _{\mathbb{R}^{n}_{+}} \frac{1}{ \vert x \vert ^{\beta} \vert y \vert ^{\alpha}} (\frac{1}{ \vert x-y \vert ^{n-2m}}- \frac{1}{ \vert \bar{x}-y \vert ^{n-2m}} )u^{p} (y)\,dy, \end{cases}\displaystyle \end{aligned}$$

(1.2)

where $C_{n}>0$, and $\bar{x}=(x_{1},\ldots,x_{n-1},-x_{n})$ is the reflection of the point x about the $\partial \mathbb{R}_{+}^{n}$. Similar to some integral systems or partial differential systems, the integral system (1.2) is usually divided into three cases according to the value of the exponents $(p,q)$. We introduce the critical curve

$$\begin{aligned} \frac{1}{p+1}+\frac{1}{q+1}=\frac{n-2m+\alpha +\beta}{n} \end{aligned}$$

(1.3)

for (1.2) to determine a Liouville-type theorem.

The well-known classical Hardy–Littlewood–Sobolev inequality states that

$$\begin{aligned} \int _{\mathbb{R}^{n}} \int _{\mathbb{R}^{n}} \frac{f(x)g(y)}{ \vert x-y \vert ^{\nu}}\,dx\,dy \leq C_{l, \nu, n} \Vert f \Vert _{h} \Vert g \Vert _{l} \end{aligned}$$

for all $f\in L^{h}(R^{n})$ and $g\in L^{l}(R^{n})$, where $1< h, l<\infty$, $0<\nu <n$, and $\frac{1}{h}+\frac{1}{l}+\frac{\nu}{n}=2$. Hardy and Littlewood also introduced the double weighted inequality, which was generalized by Stein and Weiss [13]. This inequality is called the double weighted Hardy–Littlewood–Sobolev (WHLS) inequality

$$\begin{aligned} \biggl\vert \int _{\mathbb{R}^{n}} \int _{\mathbb{R}^{n}} \frac{f(x)g(y)}{ \vert x \vert ^{\tau} \vert x-y \vert ^{\nu} \vert y \vert ^{\kappa}}\,dx\,dy \biggr\vert \leq C_{ \tau, \kappa, l, \nu, n} \Vert f \Vert _{h} \Vert g \Vert _{l}, \end{aligned}$$

(1.4)

where $1< l$, $h<\infty$, $0<\nu <n$, $\tau +\kappa \geq 0$, and τ and κ satisfy $1-\frac{1}{h}-\frac{\nu}{n}<\frac{\tau}{n}<1-\frac{1}{h}$ with $\frac{1}{l}+\frac{1}{h}+\frac{\nu +\kappa +\tau}{n}=2$. To obtain the best constant in the weighted inequality (1.4), we can maximize the functional

$$\begin{aligned} J(f,g)= \int _{\mathbb{R}^{n}} \int _{\mathbb{R}^{n}} \frac{f(x)g(y)}{ \vert x \vert ^{\tau} \vert x-y \vert ^{\nu} \vert y \vert ^{\kappa}}\,dx\,dy \end{aligned}$$

under the constrains $\|f\|_{h}=\|g\|_{l}=1$. The corresponding Euler–Lagrange equations are the following system of integral equations:

$$\begin{aligned} \textstyle\begin{cases} {\lambda }_{1} hf(x)^{h-1}=\int _{\mathbb{R}^{n}} \frac{g(y)}{ \vert x \vert ^{\tau } \vert y \vert ^{\kappa } \vert x-y \vert ^{\mu}}\,dy, \\ {\lambda }_{2} lg(x)^{l-1}=C_{n}\int _{\mathbb{R}^{n}} \frac{f(y)}{ \vert x \vert ^{\kappa } \vert y \vert ^{\tau } \vert x-y \vert ^{\mu}}\,dy, \end{cases}\displaystyle \end{aligned}$$

(1.5)

where $f, g\geq 0$, $x\in \mathbb{R}^{n}$, and ${\lambda }_{1} h={\lambda }_{2} l=J(f,g)$. Let $u=c_{1} f^{h-1}$, $v=c_{2} g^{l-1}$, $p=\frac{1}{h-1}$, $q=\frac{1}{l-1}$ with $pq\neq 1$. Then by a proper choice of constants $c_{1}$ and $c_{2}$ system (1.5) becomes

$$\begin{aligned} \textstyle\begin{cases} u(x)=C_{n}\int _{\mathbb{R}^{n}} \frac{1}{ \vert x \vert ^{\tau } \vert y \vert ^{\kappa}}\frac{1}{ \vert x-y \vert ^{\mu}}v^{q} (y)\,dy, \\ v(x)=C_{n}\int _{\mathbb{R}^{n}} \frac{1}{ \vert x \vert ^{\kappa } \vert y \vert ^{\tau}}\frac{1}{ \vert x-y \vert ^{\mu}}u^{p} (y)\,dy, \end{cases}\displaystyle \end{aligned}$$

(1.6)

where $u, v\geq 0$, $0< p, q<\infty$, $0<\mu <n$, $\frac{\tau}{n}<\frac{1}{p+1}<\frac{\mu +\tau}{n}$, and $\frac{1}{p+1}+\frac{1}{q+1}=\frac{\mu +\tau +\kappa}{n}$.

Jin and Li [10] derived that the positive solution of systems (1.6) is symmetric and monotonic. In [6] and [9], they also discussed the regularity of solutions to (1.6). Lei and Lü [11] proved that system (1.6) and the related differential systems are equivalent to each other under the condition $\max \{\tau (p+1), \kappa (q+1)\}\leq n-\mu$ with $pq>1$ and $\tau, \kappa \geq 0$, and the positive locally bounded solutions are symmetric and decreasing about some axis. The Liouville-type theorem to the whole space problem was established by Ma and Chen [8]. In recent years, the nonlocal fractional Laplacian ($0< m<1$) on the whole space has received much attention from researchers. Zhuo and Li [17] had proved the nonexistence of an antisymmetric solution in the case $0< p\leq \frac{n+2m}{n-2m}$, whereas Li and Zhuo [18] have proved the consequence of systems in the case $0 < pq < 1$ or $p + 2m > 1$ and $q + 2m > 1$ with $0< p,q\leq \frac{n+2m}{n-2m}$. For more related results, see [19–23] and the references therein.

For $\alpha =\beta =0$ in system (1.2), Zhuo and Li [14] established the symmetry of solutions to an integral system, and Cao and Dai [3] obtained the nonexistence of nontrivial solutions. Zhao, Yang, and Zheng proved the nonexistence of nontrivial solutions for partial differential equations (1.1) in [15] and considered the general nonlinear source in [16].

For $\alpha,\beta \neq 0$ in system (1.2), Cao and Dai [4] obtained a Liouville-type theorem in the super- and subcritical cases under some integrability conditions by the Pohozaev-type identity of integral form, and in the critical case, they showed that a pair of positive solutions to the system is rotationally symmetric about the $x_{n}$-axis. Also, we mention the recent important works on the existence and asymptotic analysis of nontrivial solutions for some elliptic systems; see [24–28].

In the present paper, instead of (1.1), we will first establish a Liouville-type theorem for the integral system (1.2) in the supercritical case and then prove that systems (1.2) and (1.1) are equivalent by using the superharmonic properties, that is, the following two propositions.

Proposition 1

Let $(u,v)\in L^{q_{1}}(\mathbb{R}^{n}_{+})\times L^{q_{2}}(\mathbb{R}^{n}_{+})$ be a nonnegative solution of system (1.2), and let $q_{1}:=\frac{n(pq-1)}{(2m-\alpha -\beta )(1+q)}$ and $q_{2}:=\frac{n(pq-1)}{(2m-\alpha -\beta )(1+p)}$ with $p, q\ge 1$, $pq\neq 1$, and $\alpha +\beta <2m$. If

$$\begin{aligned} \frac{1}{p+1}+\frac{1}{q+1}< \frac{n-2m+\alpha +\beta}{n}, \end{aligned}$$

(1.7)

then $(u,v)\equiv (0,0)$.

Proposition 2

Let $p, q\ge 1$ with $pq\neq 1$, and let $\alpha +\beta <2m$. Then the differential system (1.1) is equivalent to the integral system (1.2).

Remark 1

Without the growth conditions

$$\begin{aligned} \bigl\vert (-\Delta )^{m-1}u \bigr\vert , \bigl\vert (-\Delta )^{m-1}v \bigr\vert =O\bigl( \vert x \vert ^{a}\bigr),\quad a\in (0,1), \vert x \vert \rightarrow \infty, \end{aligned}$$

in [12, Theorem 1], we can arrive at the same result by using the proof of Proposition 2.

Remark 2

By Proposition 2 we can show that the conclusions of [4, Theorems 1.2 and 1.3] hold for the partial differential system (1.1). Moreover, the conditions $\frac{1}{p+1}<\frac{n-2m}{2n}+\frac{\alpha}{n}$ and $\frac{1}{q+1}<\frac{n-2m}{2n}+\frac{\beta}{n}$ in [4, Theorem 1.2] are covered by condition (1.7).

Based on Propositions 1 and 2, the main result of the paper is the following theorem.

Theorem 1

Under the conditions of Proposition 1, the classical nonnegative solutions of system (1.1) must be trivial.

To prove Proposition 1, we will explore the moving plane method in integral forms by Chen, Li, and Ou [5]. For the proof of Proposition 2, we first prove the superharmonic properties of systems (1.1) and then establish the equivalence between the two systems by using a technique introduced in [7] for the scalar case of higher-order equations.

Next, we will prove Propositions 1 and 2 in Sects. 2 and 3, respectively.

2 Proof of Proposition 1

We introduce three lemmas for the integral system (1.2) as preliminaries, and let $C_{n}=1$ there for simplicity.

Denote

$$\begin{aligned} G(x,y):=\frac{1}{ \vert x-y \vert ^{n-2m}}-\frac{1}{ \vert \bar{x}-y \vert ^{n-2m}},\quad x, y \in \mathbb{R}_{+}^{n}, \end{aligned}$$

with x̄ reflecting x about the $\partial \mathbb{R}_{+}^{n}$. Let $x^{{\lambda }}=(x_{1},x_{2},\ldots,2{\lambda }-x_{n})$ be the reflection of the point x about the plane $T_{{\lambda }}=\{x\in \mathbb{R}^{n}_{+} | x_{n}={\lambda }\}$, and denote $u_{\lambda}(x)=u(x^{\lambda})$, $v_{\lambda}(x)=v(x^{\lambda})$. Define $\Sigma _{\lambda}:=\{x\in \mathbb{R}^{n}_{+} | 0< x_{n}<\lambda \}$ and $\tilde{\Sigma}_{\lambda}:=\{x^{\lambda} | x\in \Sigma _{ \lambda}\}$, $\Sigma _{\lambda}^{c}=\mathbb{R}^{n}_{+}\setminus \Sigma _{\lambda}$. The following lemma on the Green function $G(x,y)$ in $\Sigma _{\lambda}$ is known.

Lemma 2.1

([2, Lemma 2.1]) (i) For all $x, y\in \Sigma _{\lambda}$, $x\neq y$, we have

$$\begin{aligned} & G\bigl(x^{\lambda},y^{\lambda}\bigr)>\max \bigl\{ G \bigl(x^{\lambda},y\bigr),G\bigl(x,y^{\lambda}\bigr) \bigr\} , \\ & G\bigl(x^{\lambda},y^{\lambda}\bigr)-G(x,y)> \bigl\vert {G \bigl(x^{\lambda},y\bigr)-G\bigl(x,y^{\lambda}\bigr)} \bigr\vert . \end{aligned}$$

(ii) For all $x\in \Sigma _{\lambda}$, $y\in \Sigma _{\lambda}^{c}$, we have

$$\begin{aligned} G\bigl(x^{\lambda},y\bigr)>G(x,y). \end{aligned}$$

Lemma 2.2

Let $(u,v)$ be a nonnegative solution of (1.2). For all $x\in \Sigma _{\lambda}$, we have

$$\begin{aligned} &u(x)-u_{\lambda}(x)\leq \int _{\Sigma _{\lambda}}G\bigl(x^{\lambda},y^{ \lambda}\bigr) \frac{[v^{q}-v_{\lambda}^{q}](y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy, \\ &v(x)-v_{\lambda}(x)\leq \int _{\Sigma _{\lambda}}G\bigl(x^{\lambda},y^{ \lambda}\bigr) \frac{[u^{p}-u_{\lambda}^{p}](y)}{ \vert x \vert ^{\beta} \vert y \vert ^{\alpha}}\,dy. \end{aligned}$$

Proof

Since

$$\begin{aligned} &u(x)= \int _{\Sigma _{\lambda}}G(x,y) \frac{v^{q}(y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy + \int _{\Sigma _{\lambda}}G\bigl(x,y^{ \lambda}\bigr)\frac{v_{\lambda}^{q}(y)}{ \vert x \vert ^{\alpha} \vert y^{\lambda} \vert ^{\beta}} \,dy + \int _{\Sigma ^{c}_{\lambda}\setminus \tilde{\Sigma}_{\lambda}}G(x,y) \frac{v^{q}(y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy, \\ &u_{\lambda}(x)= \int _{\Sigma _{\lambda}}G\bigl(x^{\lambda},y\bigr) \frac{v^{q}(y)}{ \vert x^{\lambda} \vert ^{\alpha} \vert y \vert ^{\beta}} \,dy+ \int _{\Sigma _{ \lambda}}G\bigl(x^{\lambda},y^{\lambda}\bigr) \frac{v_{\lambda}^{q}(y)}{ \vert x^{\lambda} \vert ^{\alpha} \vert y^{\lambda} \vert ^{\beta}}\,dy \\ &\phantom{u_{\lambda}(x)=}{}+ \int _{\Sigma ^{c}_{\lambda}\setminus \tilde{\Sigma}_{\lambda}}G\bigl(x^{ \lambda},y\bigr)\frac{v^{q}(y)}{ \vert x^{\lambda} \vert ^{\alpha} \vert y \vert ^{\beta}} \,dy, \end{aligned}$$

we have by Lemma 2.1 that

$$\begin{aligned} u(x)-u_{{\lambda }}(x) \leq{}& \int _{\Sigma _{\lambda}}\bigl[G\bigl(x^{\lambda},y^{ \lambda} \bigr)-G\bigl(x,y^{\lambda}\bigr)\bigr]\frac{v^{q}(y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy\\ &{} - \int _{\Sigma _{\lambda}}\bigl[G\bigl(x^{\lambda},y^{\lambda} \bigr)-G\bigl(x,y^{\lambda}\bigr)\bigr] \frac{v_{\lambda}^{q}(y)}{ \vert x^{\lambda} \vert ^{\alpha} \vert y^{\lambda} \vert ^{\beta}}\,dy \\ &{} + \int _{\Sigma ^{c}_{\lambda}\setminus \tilde{\Sigma}_{{\lambda }}}\bigl[G(x,y)-G\bigl(x^{{ \lambda }},y\bigr)\bigr] \frac{v^{q}(y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy \\ \leq{}& \int _{\Sigma _{\lambda}}\bigl[G\bigl(x^{\lambda},y^{\lambda} \bigr)-G\bigl(x,y^{ \lambda}\bigr)\bigr] \frac{[v^{q}-v_{\lambda}^{q}](y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy \\ \leq{}& \int _{\Sigma _{\lambda}}G\bigl(x^{\lambda},y^{\lambda}\bigr) \frac{[v^{q}-v_{\lambda}^{q}](y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy. \end{aligned}$$

The second inequality can be obtained in the same way. □

In addition, we also need the weighted Hardy–Littlewood–Sobolev inequality.

Lemma 2.3

([10]) Let $1< l, m<\infty$, $0<\nu <n$, $\tau +\kappa \geq 0$, $\frac{1}{l}+\frac{1}{m}+\frac{\nu +\kappa +\tau}{n}=2$, and $1-\frac{1}{m}-\frac{\nu}{n}<\frac{\tau}{n}<1-\frac{1}{m}$. Then

$$\begin{aligned} \biggl\vert \int _{\mathbb{R}^{n}} \int _{\mathbb{R}^{n}} \frac{f(x)g(y)}{ \vert x \vert ^{\tau} \vert x-y \vert ^{\nu} \vert y \vert ^{\kappa}}\,dx\,dy \biggr\vert \leq C \Vert f \Vert _{m} \Vert g \Vert _{l} \end{aligned}$$

with $C=C(\tau, \kappa, l, \nu, n)>0$, or, equivalently,

$$\begin{aligned} \bigl\Vert Tg(x) \bigr\Vert _{\gamma}:=\sup_{ \Vert f \Vert _{m}=1} \bigl\langle Tg(x),f(x)\bigr\rangle \leq C \Vert g \Vert _{l} \end{aligned}$$

with $Tg(x)=\int _{\mathbb{R}^{n}} \frac{g(y)}{|x|^{\tau}|x-y|^{\nu}|y|^{\kappa}}\,dy$, $\frac{1}{l}+\frac{\nu +\kappa +\tau}{n}=1+\frac{1}{\gamma}$ and $\frac{1}{m}+\frac{1}{\gamma}=1$.

Now we can prove Proposition 1.

Proof of Proposition 1

We apply the moving-plane method in two steps.

1. Determine the starting position

Start from the very low end of $\mathbb{R}^{n}_{+}$, i.e., near $x_{n}=0$. We will show that for λ sufficiently small,

$$\begin{aligned} w_{{\lambda }}(x):=u(x)-u_{{\lambda }}(x)\leq 0,\qquad g_{{\lambda }}(x):=v(x)-v_{{ \lambda }}(x) \leq 0 \quad\text{a.e. in } \Sigma _{{\lambda }}. \end{aligned}$$

(2.1)

Denote

$$\begin{aligned} B_{\lambda}^{u}:=\bigl\{ x\in \Sigma _{\lambda} | w_{{\lambda }}(x)>0 \bigr\} , \qquad B_{\lambda}^{v}:=\bigl\{ x\in \Sigma _{\lambda} | g_{{ \lambda }}(x)>0 \bigr\} . \end{aligned}$$

We will prove that $B_{{\lambda }}^{u}$ and $B_{{\lambda }}^{v}$ must be of zero measure, provided that λ sufficiently small. In fact, by Lemma 2.2 with the mean value theorem we have that for sufficiently small λ and $x\in B_{\lambda}^{u}$,

$$\begin{aligned} 0&\leq w_{{\lambda }}(x) = \int _{B^{v}_{\lambda}}+ \int _{\Sigma _{ \lambda}\setminus B^{v}_{\lambda} }G\bigl(x^{\lambda},y^{\lambda}\bigr) \frac{[v^{q}-v_{\lambda}^{q}](y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy \\ &\leq \int _{B^{v}_{\lambda}}G\bigl(x^{\lambda},y^{\lambda}\bigr) \frac{[v^{q}-v_{\lambda}^{q}](y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy \\ &\leq q \int _{B^{v}_{\lambda}} \frac{[v^{q-1}(v-v_{{\lambda }})](y)}{ \vert x-y \vert ^{n-2m} \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy. \end{aligned}$$

Furthermore, by Lemma 2.3 with Hölder’s inequality and $q_{1}^{*}=\frac{q_{1}}{q_{1}-1}$

$$\begin{aligned} \Vert w_{{\lambda }} \Vert _{q_{1}, B^{u}_{\lambda}} &\leq \sup _{ \Vert f \Vert _{q_{1}^{*}}=1} \int _{B^{v}_{\lambda}} \frac{[v^{q-1}(v-v_{{\lambda }})](y)}{ \vert x-y \vert ^{n-2m} \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy \\ &=C \bigl\Vert v^{q-1}g_{{\lambda }} \bigr\Vert _{l, B^{v}_{\lambda}} \\ &\leq C \Vert v \Vert ^{q-1}_{q_{2}, B^{v}_{\lambda}} \Vert g_{{\lambda }} \Vert _{q_{2}, B^{v}_{\lambda}} \end{aligned}$$

(2.2)

with the universal constant $C>0$, where the supercritical inequality (1.7) with $p, q\ge 1$ and $\alpha +\beta <2m$ implies

$$\begin{aligned} &q_{1}=\frac{n(pq-1)}{(2m-\alpha -\beta )(1+q)}>p+1>1, \\ &q_{2}=\frac{n(pq-1)}{(2m-\alpha -\beta )(1+p)}>q+1>1, \end{aligned}$$

and

$$\begin{aligned} \frac{1}{l}=1+\frac{1}{q_{1}}-\frac{n-2m+\alpha +\beta}{n}= \frac{(2m-\alpha -\beta )(1+p)q}{n(pq-1)}< \frac{q}{q+1}< 1. \end{aligned}$$

Similarly, we have

$$\begin{aligned} \Vert g_{{\lambda }} \Vert _{q_{2}, B^{v}_{\lambda}}\leq C \Vert u \Vert ^{p-1}_{q_{1}, B^{u}_{ \lambda}} \Vert w_{{\lambda }} \Vert _{q_{1}, B^{u}_{\lambda}}. \end{aligned}$$

(2.3)

It follows from (2.2) and (2.3) that

$$\begin{aligned} \Vert w_{{\lambda }} \Vert _{q_{1}, B^{u}_{\lambda}}\leq C \Vert u \Vert ^{p-1}_{q_{1}, B^{u}_{\lambda}} \Vert v \Vert ^{q-1}_{q_{2}, B^{v}_{\lambda}} \Vert w_{{\lambda }} \Vert _{q_{1}, B^{u}_{\lambda}}. \end{aligned}$$

(2.4)

Since $(u,v)\in L^{q_{1}}(\mathbb{R}^{n}_{+})\times L^{q_{2}}(\mathbb{R}^{n}_{+})$, we can choose λ small enough such that

$$\begin{aligned} C \Vert u \Vert ^{p-1}_{q_{1}, B^{u}_{\lambda}} \Vert v \Vert ^{q-1}_{q_{2}, B^{v}_{ \lambda}}< \frac{1}{2}, \end{aligned}$$

and thus $\|w_{{\lambda }}\|_{q_{1}, B^{u}_{\lambda}}=0$ by (2.4). In the same way, $\|g_{{\lambda }}\|_{q_{2}, B^{v}_{\lambda}}=0$. This proves (2.1).

2. Move the plane to the infinity

Inequalities (2.1) provide a starting point to move the plane $T_{{\lambda }}$. We start from a neighborhood of λ and move the plane up as long as (2.1) holds. Define

$$\begin{aligned} {\lambda }_{0}:=\sup \{{\lambda } | w_{\rho}, g_{\rho}\leq 0, \rho \leq {\lambda }\ {\mathrm{for\ a.e.}}\ x \in \Sigma _{\rho}\}. \end{aligned}$$

(2.5)

We first prove that ${\lambda }_{0}=\infty$. Assume for contradiction that ${\lambda }_{0}<\infty$. We claim that

$$\begin{aligned} w_{{\lambda }_{0}}(x)=g_{{\lambda }_{0}}(x)= 0\quad {\mathrm{a.e. in }}\ \Sigma _{{\lambda }_{0}}. \end{aligned}$$

(2.6)

Otherwise, for such ${\lambda }_{0}$, e.g.,

$$\begin{aligned} E_{0}:=\bigl\{ x| g_{{\lambda }_{0}}(x)< 0, x\in \Sigma _{{\lambda }_{0}} \bigr\} \quad {\mathrm{has\ a\ positive\ measure.}} \end{aligned}$$

(2.7)

By Lemma 2.2

$$\begin{aligned} u(x)-u_{{\lambda }_{0}}(x)&\le \int _{\Sigma _{\lambda _{0}}}G\bigl(x^{ \lambda _{0}},y^{\lambda _{0}}\bigr) \frac{[v^{q}-v_{\lambda _{0}}^{q}](y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy \\ &= \int _{E_{0}}G\bigl(x^{\lambda _{0}},y^{\lambda _{0}}\bigr) \frac{[v^{q}-v_{\lambda _{0}}^{q}](y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy, \end{aligned}$$

Consequently,

$$\begin{aligned} w_{{\lambda }_{0}}(x)< 0\quad {\mathrm{a.e.\ in}}\ \Sigma _{\lambda _{0}}. \end{aligned}$$

(2.8)

Denote $\lambda _{\epsilon}:=\lambda +\epsilon$ with $\epsilon >0$ to be determined. For any small $\eta >0$, choose R sufficiently large such that

$$\begin{aligned} \int _{\mathbb{R}^{n}_{+}\setminus B_{R}(0)} \vert u \vert ^{q_{1}}(y)\,dy \leq \eta. \end{aligned}$$

It follows from Lusin’s theorem and (2.8) that for any $\delta >0$, there exists a closed set $F_{\delta}\subset E:=\Sigma _{\lambda _{0}}\bigcap B_{R}(0)$ with $m(E\setminus F_{\delta})<\delta$ such that $w_{{\lambda }_{0}}(x)<0$ and is continuous in $F_{\delta}$. Choosing $\epsilon >0$ sufficiently small, we have

$$\begin{aligned} w_{\lambda _{\epsilon}}(x)< 0 \quad\text{for all } x\in F_{\delta} \end{aligned}$$

by continuity. Denote $D_{\lambda _{\epsilon}}:=(\Sigma _{\lambda _{\epsilon}}\setminus \Sigma _{\lambda _{0}})\cap B_{R}(0)$. Then

$$\begin{aligned} B_{\lambda _{\epsilon}}^{u}\subset M:=\bigl(\mathbb{R}^{n}_{+} \setminus B_{R}(0)\bigr) \cup (E\setminus F_{\delta})\cup D_{\lambda _{\epsilon}}. \end{aligned}$$

Let R be large and δ and ϵ small such that $\int _{B_{\lambda _{\epsilon}}^{u}}|u|^{q_{1}}(y)\,dy\leq \int _{M}|u|^{q_{1}}(y)\,dy \leq \frac{1}{2}$. Similarly, $\int _{B_{\lambda _{\epsilon}}^{v}}|v|^{q_{2}}(y)\,dy\leq \frac{1}{2}$.

By (2.4) with ${\lambda }={\lambda }_{\epsilon}$ we can get

$$\begin{aligned} \Vert w_{\lambda _{\epsilon}} \Vert _{q_{1}, B^{u}_{\lambda _{\epsilon}}} \leq C \Vert u \Vert ^{p-1}_{q_{1}, B^{u}_{\lambda _{\epsilon}}} \Vert v \Vert ^{q-1}_{q_{2}, B^{v}_{\lambda _{\epsilon}}} \Vert w_{\lambda _{\epsilon}} \Vert _{q_{1}, B^{u}_{ \lambda _{\epsilon}}}\leq \frac{1}{4} \Vert w_{\lambda _{\epsilon}} \Vert _{q_{1}, B^{u}_{\lambda _{\epsilon}}}, \end{aligned}$$

which implies $\|w_{\lambda _{\epsilon}}\|_{q_{1}, B^{u}_{\lambda _{\epsilon}}} \equiv 0$. Thus

$$\begin{aligned} w_{{\lambda }_{\epsilon}}(x)\le 0\quad {\mathrm{a.e.\ in}}\ \Sigma _{{\lambda }_{ \epsilon}}, \end{aligned}$$

and, similarly,

$$\begin{aligned} g_{{\lambda }_{\epsilon}}(x)\le 0\quad {\mathrm{a.e.\ in}}\ \Sigma _{{\lambda }_{ \epsilon}}. \end{aligned}$$

This contradicts (2.7) with (2.5). Thus (2.6) holds. This yields the contradiction that $u(x)=v(x)\equiv 0$ on the plane $\{x_{n}=2\lambda _{0}\}$. We conclude that $\lambda _{0}=+\infty$, which implies that both u and v are strictly monotonically increasing with respect to $x_{n}$. Moreover, we know that $u\in L^{q_{1}}(\mathbb{R}^{n}_{+})$ and $v\in L^{q_{2}}(\mathbb{R}^{n}_{+})$and for any $a>0$,

$$\begin{aligned} &\int _{\mathbb{R}^{n}_{+}} \bigl\vert u\bigl(x',x_{n} \bigr) \bigr\vert ^{q_{1}}\,dx'\,dx_{n}\geq \int _{ \mathbb{R}^{n-1}} \int _{a}^{\infty} \bigl\vert u\bigl(x',a \bigr) \bigr\vert ^{q_{1}}\,dx_{n}\,dx', \\ &\int _{\mathbb{R}^{n}_{+}} \bigl\vert v\bigl(x',x_{n} \bigr) \bigr\vert ^{q_{2}}\,dx'\,dx_{n}\geq \int _{ \mathbb{R}^{n-1}} \int _{a}^{\infty} \bigl\vert v\bigl(x',a \bigr) \bigr\vert ^{q_{2}}\,dx_{n}\,dx', \end{aligned}$$

and hence $u(x',a)=v(x',a)=0$ for all $x'\in \mathbb{R}^{n-1}$, a contradiction. □

3 Proof of Proposition 2

Denote by $B_{R}(0):=\{x\in \mathbb{R}^{n}, |x|< R\}$ the ball of radius R centered at the origin in $\mathbb{R}^{n}$ with $B_{R}^{+}(0):=B_{R}(0)\cap \mathbb{R}^{n}_{+}$ and $\partial B_{R}^{+}(0):=\Gamma _{R}=\bar{\Gamma}_{R}\cup \widehat{\Gamma}_{R}$, the union of the flat and hemisphere parts of $\Gamma _{R}$. Let $x*:=\frac{x}{|x^{2}|}R^{2}$ be the reflection of x about $\partial B_{R}(0)$, and let

$$\begin{aligned} \tilde{G}_{R}(x,y):= \biggl(\frac{1}{ \vert x-y \vert ^{n-2}}- \frac{1}{(\frac{ \vert x \vert }{R} \vert x^{*}-y \vert )^{n-2}} \biggr)- \biggl( \frac{1}{ \vert \bar{x}-y \vert ^{n-2}}- \frac{1}{(\frac{ \vert \bar{x} \vert }{R} \vert \bar{x}^{*}-y \vert )^{n-2}} \biggr). \end{aligned}$$

We begin with the well-known lemma.

Lemma 3.1

([1, Lemma 2.1])

(i) For $x\in B_{R}^{+}(0)$, $\tilde{G}_{R}(x,y)$ satisfies the equation

$$\begin{aligned} \textstyle\begin{cases} -\Delta \tilde{G}_{R}(x,y)=\delta (x-y)&\textit{in } B_{R}^{+}(0), \\ \tilde{G}_{R}(x,y)=0&\textit{on } \partial B_{R}^{+}(0). \end{cases}\displaystyle \end{aligned}$$

(3.1)

(ii) For $x,y\in B_{R}^{+}(0)$,

$$\begin{aligned} \tilde{G}_{R}(x,y)\rightarrow G(x,y,2)=\frac{1}{ \vert x-y \vert ^{n-2}}- \frac{1}{ \vert \bar{x}-y \vert ^{n-2}} \quad \textit{as } R\rightarrow \infty. \end{aligned}$$

(3.2)

(iii) For $x\in B_{R}^{+}(0)$ and $y\in \widehat{\Gamma}_{R}$,

$$\begin{aligned} \frac{\partial \tilde{G}_{R}}{\partial \nu}(x,y)=(2-n)R \biggl(1- \frac{ \vert x \vert ^{2}}{R^{2}} \biggr) \biggl( \frac{1}{ \vert x-y \vert ^{n}}- \frac{1}{ \vert \bar{x}-y \vert ^{n}} \biggr), \end{aligned}$$

where ν is the outward unit normal vector of $\widehat{\Gamma}_{R}$.

We follow the main idea of Chen, Fang, and Li [7] to give superharmonic properties of system (1.1). This result plays a key role in the proof of Proposition 2.

Lemma 3.2

If $(u,v)$ is a positive solution of (1.1), then

$$\begin{aligned} (-\Delta )^{i}\bigl( \vert x \vert ^{\alpha }u\bigr)>0,\qquad (- \Delta )^{i}\bigl( \vert x \vert ^{\beta }v\bigr)>0,\quad i=1, \ldots,m-1, x\in \mathbb{R}^{n}_{+}. \end{aligned}$$

Proof

We make an odd extension of u and v to the whole space. Define

$$\begin{aligned} u\bigl(x', x_{n}\bigr)=-u\bigl(x', -x_{n}\bigr),\qquad v\bigl(x', x_{n}\bigr)=-v \bigl(x', -x_{n}\bigr),\quad x_{n}< 0, \end{aligned}$$

with $x'=(x_{1},\ldots,x_{n-1})$. Then $(u,v)$ satisfy

$$\begin{aligned} \textstyle\begin{cases} (-\Delta )^{m}( \vert x \vert ^{\alpha} u(x))= \vert x \vert ^{-\beta} \vert v \vert ^{q-1}v&\text{in } \mathbb{R}^{n}, \\ (-\Delta )^{m}( \vert x \vert ^{\beta}v(x))= \vert x \vert ^{-\alpha} \vert u \vert ^{p-1}u &\text{in } \mathbb{R}^{n}. \end{cases}\displaystyle \end{aligned}$$

(3.3)

Write $u_{i}(x):=(-\Delta )^{i}(|x|^{\alpha }u)$ and $v_{i}(x):=(-\Delta )^{i}(|x|^{\beta }v)$. We will prove that $u_{i}(x),v_{i}(x)>0$, $x\in \mathbb{R}^{n}_{+}$, $i=1,2,\ldots,m-1$.

Step 1. We claim that $u_{m-1}(x)\geq 0$, $x\in \mathbb{R}^{n}_{+}$. Otherwise, there exists $x_{1}\in \mathbb{R}^{n}_{+}$ such that

$$\begin{aligned} u_{m-1}(x_{1})< 0. \end{aligned}$$

(3.4)

We will deduce a contradiction by two substeps.

(i) We first claim

$$\begin{aligned} (-1)^{i}\hat{u}_{m-i}(r)>0,\quad \forall r\geq 0, i=1,2, \ldots,m-1, \end{aligned}$$

(3.5)

where $\hat{u}_{m-i}$ is the $(m-i)$th average of $u_{m-i}$.

Denote by $B_{r}(x_{1})$ the ball of radius r centered at $x_{1}$, and define the first averages of u and v on $\partial B_{r}(x_{1})$ as

$$\begin{aligned} &\bar{u}(r):=\frac{1}{ \vert \partial B_{r}(x_{1}) \vert } \int _{\partial B_{r}(x_{1})} \vert x \vert ^{ \alpha }u(x)\,ds;\qquad \bar{v}(r):=\frac{1}{ \vert \partial B_{r}(x_{1}) \vert } \int _{ \partial B_{r}(x_{1})} \vert x \vert ^{\beta }v(x)\,ds \end{aligned}$$

and

$$\begin{aligned} &\bar{u}_{i}(r):=\frac{1}{ \vert \partial B_{r}(x_{1}) \vert } \int _{\partial B_{r}(x_{1})}u_{i}(x)\,ds;\qquad \bar{v}_{i}(r):= \frac{1}{ \vert \partial B_{r}(x_{1}) \vert } \int _{\partial B_{r}(x_{1})}v_{i}(x)\,ds \end{aligned}$$

with $i=2,3,\ldots,m-1$. Then for $r>0$, we have by (3.3) that for $x\in \mathbb{R}^{n}$,

$$\begin{aligned} \textstyle\begin{cases} -\Delta \bar{u}=\bar{u}_{1}, &-\Delta \bar{v}=\bar{v}_{1}, \\ -\Delta \bar{u}_{1}=\bar{u}_{2},&-\Delta \bar{v}_{1}= \bar{v}_{2}, \\ \ldots \\ -\Delta \bar{u}_{m-2}=\bar{u}_{m-1}, &-\Delta \bar{v}_{m-2}= \bar{v}_{m-1}, \\ -\Delta \bar{u}_{m-1}=f(r), &-\Delta \bar{v}_{m-1}=g(r), \end{cases}\displaystyle \end{aligned}$$

(3.6)

where $f(r):=\overline{|x|^{-\beta}|v|^{q-1}v}$ and $g(r):=\overline{|x|^{-\alpha}|u|^{p-1}u}$. Integrate the last equation for u in (3.6) from 0 to r. Notice that $x_{1}\in \mathbb{R}^{n}_{+}$ implies that more than half of $B_{r}(x_{1})$ is contained in $\mathbb{R}^{n}_{+}$. By the odd symmetry of v with respect to $\partial \mathbb{R}^{n}_{+}$ we have

$$\begin{aligned} -r^{n-1}\bar{u}'_{m-1}(r)&= \int ^{r}_{0} s^{n-1}f(s)\,ds= \frac{1}{n\alpha (n)} \int ^{r}_{0} \int _{\partial B_{s}(x_{1})} \vert x \vert ^{- \beta} \vert v \vert ^{q-1}vd\sigma \,ds \\ &=\frac{1}{n\alpha (n)} \int _{B_{r}(x_{1})} \vert x \vert ^{-\beta} \vert v \vert ^{q-1}vdx>0, \end{aligned}$$

(3.7)

where $\alpha (n)$ denotes the surface area of the unit sphere $\partial B_{1}(0)$ in $\mathbb{R}^{n}$.

By (3.4) and (3.7) we deduce that

$$\begin{aligned} \bar{u}'_{m-1}(r)< 0 \quad\text{and} \quad \bar{u}_{m-1}(r)\leq \bar{u}_{m-1}(0)=u_{m-1}(x_{1})< 0\quad \forall r\geq 0. \end{aligned}$$

(3.8)

Then by the second to the last equation in (3.6) we have

$$\begin{aligned} -\frac{1}{r^{n-1}} \bigl(r^{n-1}\bar{u}'_{m-2}(r) \bigr)'=\bar{u}_{m-1}(r) \leq \bar{u}_{m-1}(0) \equiv -c_{0}< 0 \quad \forall r>0 \end{aligned}$$

with universal positive constant $c_{0}$, that is,

$$\begin{aligned} \bigl(r^{n-1}\bar{u}'_{m-2}(r) \bigr)'>r^{n-1}c_{0}\quad \forall r>0, \end{aligned}$$

and hence

$$\begin{aligned} \bar{u}_{m-2}(r)\geq \bar{u}_{m-2}(0)+\frac{c_{0}}{2n}r^{2}\quad \forall r>0 \end{aligned}$$

(3.9)

after integrating. So we find a suitably large $r_{1}>0$ such that $\bar{u}_{m-2}(r_{1})>0$. In view of the definition of the average, there exists $x_{2}\in (\partial B_{r_{1}}(x_{1})\cap \mathbb{R}^{n}_{+})$ such that

$$\begin{aligned} u_{m-2}(x_{2})>0. \end{aligned}$$

(3.10)

Moreover, we deduce by (3.8) that

$$\begin{aligned} u_{m-1}(x_{2})< 0. \end{aligned}$$

(3.11)

Define the second averages of u and v on $\partial B_{r}(x_{2})$:

$$\begin{aligned} &\tilde{u}(r):=\frac{1}{ \vert \partial B_{r}(x_{2}) \vert } \int _{\partial B_{r}(x_{2})} \vert x \vert ^{ \alpha }u(x)\,ds;\qquad \tilde{v}(r):=\frac{1}{ \vert \partial B_{r}(x_{2}) \vert } \int _{\partial B_{r}(x_{2})} \vert x \vert ^{\beta }v(x)\,ds, \\ &\tilde{u}_{i}(r):=\frac{1}{ \vert \partial B_{r}(x_{2}) \vert } \int _{\partial B_{r}(x_{2})}u_{i}(x)\,ds; \qquad\tilde{v}_{i}(r):= \frac{1}{ \vert \partial B_{r}(x_{2}) \vert } \int _{\partial B_{r}(x_{2})}v_{i}(x)\,ds, \end{aligned}$$

where $i=2,3,\ldots,m-1$. By (3.7) and (3.11) we have

$$\begin{aligned} \tilde{u}_{m-1}(r)\leq \tilde{u}_{m-1}(0)=u_{m-1}(x_{2})< 0,\quad r\geq 0. \end{aligned}$$

Similarly to (3.9) and (3.11), we have

$$\begin{aligned} \tilde{u}_{m-2}(r)\geq \tilde{u}_{m-2}(0)+cr^{2}=u_{m-2}(x_{2})+cr^{2}>0,\quad r\geq 0. \end{aligned}$$

Repeating the same argument to $u_{m-3}$, we also obtain the third average on $\partial B_{r}(x_{3})$:

$$\begin{aligned} \check{u}_{m-1}(r)< 0, \qquad\check{u}_{m-2}(r)>0,\qquad \check{u}_{m-3}(r)< 0,\quad r\geq 0. \end{aligned}$$

By induction we can get the claim (3.5) for the component u.

(ii) Taking the scaling transformations

$$\begin{aligned} u_{\mu}(x)=\mu ^{\frac{(2m-\alpha -\beta )(q+1)}{pq-1}}u(\mu x), \qquad v_{ \mu}(x)=\mu ^{\frac{(2m-\alpha -\beta )(p+1)}{pq-1}}v(\mu x), \end{aligned}$$

we find that $u_{\mu}$ and $v_{\mu}$ are also nonnegative solutions of (3.3). This implies that by repeating step 2 in Part I of [7, Sect. 2] a suitably large $\mu >0$ ensures

$$\begin{aligned} \hat{u}(r)\geq a_{0} (r-1)^{b_{0}},\quad r\in [1,2], \end{aligned}$$

(3.12)

with $b_{0}:=p+q+2m+n$ and $a_{0}>0$ sufficiently large.

Next, we treat the component v. Set $U^{+}=B_{\tau}(x_{m-1})\cap \mathbb{R}_{+}^{N}$ and $U^{-}=B_{\tau}(x_{m-1})\cap (\mathbb{R}^{N}\setminus \mathbb{R}_{+}^{N})$. Let $\tilde{U}^{-}$ be the reflection of $U^{-}$ with respect to the boundary $\mathbb{\partial R}_{+}^{N}$, and let $U_{\tau}=U^{+}\setminus \tilde{U}^{-}$. By Jensen’s inequality and the equations for v in (3.6), we derive that for all $0\leq r\leq 2$,

$$\begin{aligned} -\hat{v}_{m-1}(r)={}& \int _{0}^{r}\frac{1}{\tau ^{n-1}} \int _{0}^{\tau }s^{n-1}g(s)\,ds\,d \tau - \hat{v}_{m-1}(0) \\ ={}& \int _{0}^{r}\frac{1}{\tau ^{n-1}} \biggl( \int _{0}^{\tau }s^{n-1} \biggl[ \frac{1}{ \vert \partial B_{r}(x_{m-1}) \vert } \int _{\partial B_{r}(x_{m-1})} \vert x \vert ^{- \alpha} \vert u \vert ^{p-1}u(x)\,d\sigma \biggr] \,ds \biggr)\,d\tau \\ &{}-\hat{v}_{m-1}(0) \\ ={}& \int _{0}^{r}\frac{1}{\tau ^{n-1}} \biggl( \int _{0}^{\tau }s^{n-1} \frac{1}{n\alpha (n)} \int _{\partial B_{r}(x_{m-1})} \vert x \vert ^{-\alpha} \vert u \vert ^{p-1}u(x)\,d \sigma \,ds \biggr)\,d\tau -\hat{v}_{m-1}(0) \\ \geq{}& \frac{1}{n\alpha (n)(2+ \vert x_{m-1} \vert )^{\alpha}} \int _{0}^{r} \frac{1}{\tau ^{n-1}} \biggl( \int _{ B_{\tau}(x_{m-1})} \vert u \vert ^{p-1}u(x)\,dx \biggr) \,d\tau -\hat{v}_{m-1}(0) \\ ={}&\frac{1}{n\alpha (n)(2+ \vert x_{m-1} \vert )^{\alpha}} \int _{0}^{r} \frac{1}{\tau ^{n-1}} \biggl( \int _{ U_{\tau}}u^{p}(x)\,dx \biggr)\,d\tau - \hat{v}_{m-1}(0) \\ \geq{}& \frac{1}{n\alpha (n)(2+ \vert x_{m-1} \vert )^{\alpha}} \int _{0}^{r} \frac{ \vert U_{\tau} \vert }{\tau ^{n-1}} \biggl( \frac{1}{ \vert U_{\tau} \vert } \int _{ U_{ \tau}}u^{p}(x)\,dx \biggr)\,d\tau - \hat{v}_{m-1}(0) \\ \geq{}& \frac{1}{n\alpha (n)(2+ \vert x_{m-1} \vert )^{\alpha}} \int _{0}^{r} \biggl( \frac{ \vert B_{\tau}(x_{m-1}) \vert }{ \vert U_{\tau} \vert } \biggr)^{p-1} \\ &{}\times\frac{1}{\tau ^{n-1} \vert B_{\tau}(x_{m-1}) \vert ^{p-1}} \biggl( \int _{ U_{\tau}}u(x)\,dx \biggr)^{p} \,d\tau - \hat{v}_{m-1}(0) \\ \geq {}&\frac{1}{(n\alpha (n))^{p}(2+ \vert x_{m-1} \vert )^{\alpha}} \int _{0}^{r} \frac{1}{\tau ^{np-1}} \biggl( \int _{ U_{\tau}} \vert x \vert ^{-\alpha}\cdot \vert x \vert ^{ \alpha}u(x)\,dx \biggr)^{p} \,d\tau - \hat{v}_{m-1}(0) \\ \geq{}& \frac{1}{(n\alpha (n))^{p}(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}} \int _{0}^{r} \frac{1}{\tau ^{np-1}} \biggl( \int _{B_{\tau}(x_{m-1})} \vert x \vert ^{\alpha}u(x)\,dx \biggr)^{p} \,d\tau -\hat{v}_{m-1}(0) \\ \geq{}& \frac{1}{(n\alpha (n))^{p}(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}} \int _{0}^{r} \frac{1}{\tau ^{np-1}} \biggl( \int _{0}^{\tau} \int _{\partial B_{s}(x_{m-1})} \vert x \vert ^{ \alpha}u(x)\,dx\,ds \biggr)^{p} \,d\tau \\ &{} -\hat{v}_{m-1}(0) \\ ={}&\frac{1}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}} \int _{0}^{r} \frac{1}{\tau ^{np-1}} \biggl( \int _{0}^{\tau }s^{n-1}\hat{u}(s)\,ds \biggr)^{p} \,d\tau -\hat{v}_{m-1}(0). \end{aligned}$$

(3.13)

Choosing $r=2$ in (3.12) and substituting the latter into (3.13), we get

$$\begin{aligned} \hat{v}_{m-1}(r)\leq -\frac{a_{0}}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}} r^{2}+ \hat{v}_{m-1}(0). \end{aligned}$$

Thus $\hat{v}_{m-1}(r)$ must be negative whenever $a_{0}$ is large. Now we can repeat the above procedure for u with $m-1$ times recenters to deduce for the component v that

$$\begin{aligned} (-1)^{i}\hat{v}_{m-i}(r)>0,\quad r\in [0,2], i=1,2,\ldots,m-1. \end{aligned}$$

(3.14)

For any $1\leq r \leq 2$, we get by (3.12)–(3.14) that

$$\begin{aligned} -\hat{v}_{m-1}(r)&\geq \frac{1}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}} \int _{0}^{r} \frac{1}{\tau ^{np-1}} \biggl( \int _{0}^{\tau }s^{n-1}\hat{u}(s)\,ds \biggr)^{p} \,d\tau -\hat{v}_{m-1}(0) \\ &\geq \frac{1}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}} \int _{1}^{r} \frac{1}{\tau ^{np-1}} \biggl( \int _{1}^{\tau }s^{n-1}a_{0} (s-1)^{b_{0}}\,ds \biggr)^{p} \,d\tau \\ &\geq \frac{1}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}} \int _{1}^{r} \frac{1}{\tau ^{np-1}} \biggl( \frac{a_{0}}{b_{0}+n}(\tau -1)^{b_{0}+1} \tau ^{n-1} \biggr)^{p} \,d\tau ) \\ &=\frac{1}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}}\cdot \frac{a_{0}^{p}}{(n+b_{0})^{p}} \int _{1}^{r}(\tau -1)^{(b_{0}+1)p} \tau ^{1-p}\,d\tau \\ &\geq \frac{1}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}}\cdot \frac{a_{0}^{p}}{(n+b_{0})^{p}[(b_{0}+1)p+1]}(r-1)^{(b_{0}+1)p+1} r^{1-p} \\ &\geq \frac{1}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}}\cdot \frac{a_{0}^{p}}{2^{p-1}(n+b_{0})^{p}[(b_{0}+1)p+1]}(r-1)^{(b_{0}+1)p+1} \\ &\geq \frac{1}{2^{p}(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}}\cdot \frac{a_{0}^{p}}{(n+b_{0})^{p}[(b_{0}+1)p+1]}(r-1)^{(b_{0}+1)p+1}, \end{aligned}$$

(3.15)

where $\int _{1}^{r}(s-1)^{\theta}s^{\iota}\,ds\geq \frac{1}{\theta +\iota +1}(r-1)^{\theta +1}r^{\iota}$, and $\theta, \iota >0$ by an elementary calculation. This implies

$$\begin{aligned} \hat{v}_{m-1}(r)\leq -\frac{(Ma_{0})^{p}}{(2b_{0}p)^{p+1}}(r-1)^{(b_{0}+1)p+1} \end{aligned}$$

with $M^{p}:=\frac{1}{2^{p}(2+|x_{m-1}|)^{\alpha (p+1)}}$. So we get

$$\begin{aligned} -\frac{1}{r^{n-1}} \bigl(r^{n-1}\hat{v}'_{m-2}(r) \bigr)'=\hat{v}'_{m-1}(r) \leq - \frac{(Ma_{0})^{p}}{(2b_{0}p)^{p+1}}(r-1)^{(b_{0}+1)p+1}< 0. \end{aligned}$$

Integrating twice from 0 to r and from 0 to τ, by (3.14) we obtain that

$$\begin{aligned} \hat{v}_{m-2}(r)&\geq \int _{0}^{r}\frac{1}{\tau ^{n-1}} \int _{0}^{ \tau }s^{n-1}\frac{(Ma_{0})^{p}}{(2b_{0}p)^{p+1}}(s-1)^{(b_{0}+1)p+1} \,ds \,d\tau \\ &\geq \frac{(Ma_{0})^{p}}{(2b_{0}p)^{p+1}} \int _{1}^{r} \frac{1}{\tau ^{n-1}}\frac{1}{(b_{0}+1)p+n+1} \tau ^{n-1}(\tau -1)^{(b_{0}+1)p+2} \,d\tau \\ &\geq \frac{(Ma_{0})^{p}}{(2b_{0}p)^{p+1}}\frac{1}{(b_{0}+1)p+n+1} \frac{1}{(b_{0}+1)p+3}(r-1)^{(b_{0}+1)p+3} \\ &\geq \frac{(Ma_{0})^{p}}{(2b_{0}p)^{p+3}}(r-1)^{(b_{0}+1)p+3}. \end{aligned}$$

We know by repeating step 1 in Part I of [7, Sect. 2] that m is even. Continuing this way, for m even, we obtain

$$\begin{aligned} \hat{v}(r)&\geq \frac{(Ma_{0})^{p}}{(2b_{0}p)^{p+2m-1}}(r-1)^{(b_{0}+1)p+2m-1} \geq A_{0}(r-1)^{B_{0}} \end{aligned}$$

(3.16)

with $A_{0}:=c_{0}(Ma_{0})^{p}(2pb_{0})^{-p-2m}$ and $B_{0}:=2pb_{0}\geq (b_{0}+1)p+2m$.

Similarly to (3.13), by (3.5) we have

$$\begin{aligned} -\hat{u}_{m-1}(r)&\geq \int _{0}^{r}\frac{1}{\tau ^{n-1}} \int _{0}^{ \tau }s^{n-1}g(s)\,ds- \hat{u}_{m-1}(0) \\ &\geq \frac{1}{(2+ \vert x_{m-1} \vert )^{\beta (q+1)}} \int _{0}^{r} \frac{1}{\tau ^{nq-1}} \biggl( \int _{0}^{\tau }s^{n-1}\hat{v}(s)\,ds \biggr)^{q} \,d\tau. \end{aligned}$$

Together with (3.16), we obtain

$$\begin{aligned} \hat{u}_{m-1}(r)\leq -\frac{(M_{0}A_{0})^{q}}{(2B_{0}q)^{q+1}}(r-1)^{(B_{0}+1)q+1} \end{aligned}$$

with $M_{0}^{q}:=\frac{1}{2^{q}(2+|x_{m-1}|)^{\beta (q+1)}}$. By this and (3.6) we obtain

$$\begin{aligned} \hat{u}(r)\geq \frac{(M_{0}A_{0})^{q}}{(2B_{0}q)^{q+2m}}(r-1)^{(B_{0}+1)q+2m} \geq a_{1}(r-1)^{b_{1}} \end{aligned}$$

with $a_{1}:=c_{0}(M_{0}A_{0})^{q}(2B_{0}q)^{-q-2m}$ and $b_{1}:=2B_{0}q$. Then

$$\begin{aligned} b_{1}\geq (B_{0}+1)q+2m. \end{aligned}$$

(3.17)

By induction we can obtain by k steps that $\hat{u}(r)\geq a_{k}(r-1)^{b_{k}}$ with $b_{k+1}=4pqb_{k}$, $a_{k+1}= \frac{M^{pq}M_{k}^{q} a_{k}^{pq}}{(2pb_{k})^{q(p+2m)+(q+2m)}(2q)^{q+2m}}$, $k=0,1,\dots$.

Set $h:=\max \{q+2m,p+2m\}$ and $M_{k}^{q}:=\min \{\frac{1}{2^{p}(2+|x_{m-1}|)^{\alpha (p+1)}}, \frac{1}{2^{q}(2+|x_{m-1}|)^{\beta (q+1)}}\}$. Choose z such that $z\geq \frac{h(q+1)+1}{pq-1}$ and thus $b_{0}\geq c(4pq)^{pq(q+1)+z}$ with

$$\begin{aligned} c:=2^{pq+q+2(q+2m)+q(p+2m)}\bigl(2+ \vert x_{m-1} \vert \bigr)^{\beta (q+1)+\alpha q(p+1)}p^{q(p+2m)+(q+2m)}q^{q+2m}>0. \end{aligned}$$

We claim that

$$\begin{aligned} a_{k}^{pq}\geq cb_{k}^{h(q+1)+z},\quad k=0,1, \dots. \end{aligned}$$

(3.18)

Obviously, (3.18) holds for $k=0$ by choosing $a_{0}$ sufficiently large. Assume that (3.18) is true for k. We have

$$\begin{aligned} \frac{a_{k+1}^{pq}}{cb_{k+1}^{h(q+1)+z}}&=\biggl[ \frac{a_{k}^{pq}}{b_{k}^{h(q+1)}}\biggr]^{pq} \frac{1}{c(4pqb_{k})^{h(q+1)+z}} \\ &\geq \frac{b_{k}^{z(pq-1)-h(q+1)}}{c(4pq)^{pq(q+1)+z}} \\ &\geq \frac{b_{0}}{c(4pq)^{pq(q+1)+z}}. \end{aligned}$$

Therefore (3.18) holds for all integer k.

Now choose $r=2$. Then (3.17) and (3.18) yield a contradiction that

$$\begin{aligned} \hat{u}(2)\geq a_{k}\geq cb_{k}^{(pq(q+1)+z)/(pq)} \rightarrow \infty\quad \text{as } k\rightarrow \infty. \end{aligned}$$

This excludes (3.4).

Step 2. We furthermore claim that $u_{m-1}(x)>0$.

Otherwise, there exists $x_{1}\in \mathbb{R}^{n}_{+}$ such that $u_{m-1}(x_{1})=0$. Thus $-\Delta u_{m-1}(x_{1})\leq 0$, since $x_{1}$ is a local minimum of $u_{m-1}(x)$. This contradicts $-\Delta u_{m-1}(x)=u^{p}>0$, $x\in \mathbb{R}_{+}^{n}$.

Similarly, we can get $v_{m-1}(x)>0$ for $\mathbb{R}^{n}_{+}$ by similar arguments in Steps 1 and 2 for $u_{m-1}(x)$.

Step 3. We show that $u_{m-i}(x), v_{m-i}(x)>0$, $x\in \mathbb{R}^{n}_{+}$ for $i=2,3,\dots,m-1$.

Based on the positivity of $v_{m-1}(x)$, we first show that $u_{m-i}(x)>0$, $x\in \mathbb{R}^{n}_{+}$, for $i=2,3,\ldots,m-1$. Otherwise, there exists $x_{0}\in \mathbb{R}^{n}_{+}$ with $i\in \{2,3,\ldots,m-1\}$ such that

$$\begin{aligned} &u_{m-1}(x)>0, u_{m-2}(x)>0,\ldots, u_{m-i+1}(x)>0, \quad x \in \mathbb{R}^{n}_{+}, \\ &u_{m-i}(x_{0})< 0. \end{aligned}$$

(3.19)

(i) Assume that $m-i$ is even. Then $\bar{u}_{m-i}(r)$ for $x\in \mathbb{R}^{n}$ satisfies that

$$\begin{aligned} \textstyle\begin{cases} -\Delta \bar{u}=\bar{u}_{1}, \\ \cdots \\ -\Delta \bar{u}_{m-i-1}=\bar{u}_{m-i}, \\ -\Delta \bar{u}_{m-i}=\bar{u}_{m-i+1}. \end{cases}\displaystyle \end{aligned}$$

(3.20)

Integrating the last equation in (3.20), we arrive at

$$\begin{aligned} -r^{n-1}\bar{u}'_{m-i}(r)&= \int ^{r}_{0} s^{n-1} \bar{u}_{m-i+1}(s)\,ds= \frac{1}{n\alpha (n)} \int ^{r}_{0} \int _{\partial B_{s}(x_{0})}u_{m-i+1}(s)\,d \sigma \,ds \\ &=\frac{1}{n\alpha (n)} \int _{B_{r}(x_{0})}u_{m-i+1}(s)\,ds>0. \end{aligned}$$

Here we have used the odd symmetry of $u_{m-i+1}(x)$ with respect to $\partial \mathbb{R}^{n}_{+}$ and the fact that more than half of $B_{r}(x_{0})$ is contained in $\mathbb{R}^{n}_{+}$. Together with (3.19), we deduce

$$\begin{aligned} \bar{u}'_{m-i}(r)< 0 \quad\text{and} \quad \bar{u}_{m-i}(r)\leq \bar{u}_{m-i}(0)=u_{m-i}(x_{0})< 0,\quad r \geq 0. \end{aligned}$$

Then by the second to the last equation in (3.20) we have

$$\begin{aligned} -\frac{1}{r^{n-1}} \bigl(r^{n-1}\bar{u}'_{m-i-1}(r) \bigr)'=\bar{u}_{m-i}(r) \leq \bar{u}_{m-i}(0) \equiv -c_{0}< 0,\quad r>0. \end{aligned}$$

This yields

$$\begin{aligned} \bar{u}'_{m-i-1}(r)>\frac{c_{0}}{n}r, \end{aligned}$$

and hence

$$\begin{aligned} \bar{u}_{m-i-1}(r)\geq \bar{u}_{m-i-1}(0)+\frac{c_{0}}{2n}r^{2} \geq \frac{c_{0}}{2n}r^{2}+c_{1},\quad r>0. \end{aligned}$$

Continuing this way with $m-i$ even, we derive that

$$\begin{aligned} \bar{u}(r)\leq -c_{0}r^{2(m-i)}+\sum ^{m-i}_{j=1}c_{j}r^{2(m-i-j)},\quad r>0. \end{aligned}$$

(3.21)

This yields a contraction that

$$\begin{aligned} 0&< \frac{1}{n\alpha (n)} \int _{B_{r}(x_{0})} \vert x \vert ^{\alpha}u(x)\,dx= \int _{0}^{r} s^{n-1}\bar{u}(s)\,ds \\ &\leq -c_{0}r^{n+2(m-i)}+\sum^{m-i}_{j=1}c_{j} r^{n+2(m-i-j)} \\ &< 0 \quad\text{as } r \text{ is sufficiently large.} \end{aligned}$$

(ii) Assume that $m-i$ is odd.

Similarly to (3.21) with $m-i$ odd, by (3.5) we deduce

$$\begin{aligned} \hat{u}(r)\geq c_{0}r^{2(m-i)},\quad r>0. \end{aligned}$$

(3.22)

By parallel arguments for (3.13), by (3.22) we have

$$\begin{aligned} -r^{n-1}\hat{v}'_{m-1}(r)&=\frac{1}{n\alpha (n)} \int _{B_{r}(x_{m-1})} \vert x \vert ^{- \alpha} \vert u \vert ^{p-1}u(x)\,dx \\ &\geq \frac{c}{r^{np-n}}\biggl( \int ^{r}_{0} s^{n-1}\hat{u}(s)\,ds \biggr)^{p} \\ &\geq cr^{n+2p(m-i)} \end{aligned}$$

with $c:=\frac{c_{0}}{(1+|x_{m-1}|)^{\alpha (p+1)}}$ or, equivalently, $\hat{v}'_{m-1}(r)\leq -cr^{2p(m-i)+1}$, and, consequently,

$$\begin{aligned} \hat{v}_{m-1}(r)\leq -c_{0}r^{2p(m-i)+2}+ \hat{v}_{m-1}(0). \end{aligned}$$

Combining this with $v_{m-1}(x)>0$ for $x\in \mathbb{R}^{n}_{+}$, we obtain by (3.14) a contradiction that

$$\begin{aligned} 0&< \frac{1}{n\alpha (n)} \int _{B_{r}(x_{m-1})}v_{m-1}(x)\,dx= \int ^{r}_{0}s^{n-1} \hat{v}_{m-1}(s)\,ds \\ &\leq -c_{0}r^{n+2p(m-i)+2}\leq 0. \end{aligned}$$

Combining (i) and (ii), we exclude (3.19).

Similarly, we can prove that $v_{m-i}(x)>0$, $i=2,3,\ldots,m-1$, by $u_{m-1}(x)>0$ for $\mathbb{R}^{n}_{+}$. □

Proof of Proposition 2

First, we show that the classical solutions of (1.2) must solve (1.1).

When $x\in \partial R_{+}^{n}$, we have

$$\begin{aligned} \Delta ^{j}\biggl(\frac{1}{ \vert x-y \vert ^{n-2m}}\biggr)=\Delta ^{j} \biggl( \frac{1}{ \vert \bar{x}-y \vert ^{n-2m}}\biggr) \end{aligned}$$

due to $x=\bar{x}$, and thus by system (1.2) that $\Delta ^{j}u(x)=0$, $j=0,1,\ldots,m-1$. For $x\in R_{+}^{n}$, we have

$$\begin{aligned} (-\Delta )^{m}\bigl( \vert x \vert ^{\alpha}u(x)\bigr)&= \int _{\mathbb{R}^{n}_{+}}(- \Delta )^{m}\biggl(\frac{1}{ \vert x-y \vert ^{n-2m}}- \frac{1}{ \vert \bar{x}-y \vert ^{n-2m}}\biggr) \vert y \vert ^{- \beta}v^{q}(y)\,dy \\ &=C \int _{\mathbb{R}^{n}_{+}}\delta (x-y) \vert y \vert ^{-\beta}v^{q}(y) \,dy \\ &=C \vert x \vert ^{-\beta}v^{q}(x) . \end{aligned}$$

Similarly, $\Delta ^{j}v(x)=0$, $j=0,1,\ldots,m-1$, on $\partial \mathbb{R}_{+}^{n}$ and $(-\Delta )^{m}(|x|^{\beta}v(x))=C|x|^{-\alpha}u^{p}(x)$ in $\mathbb{R}_{+}^{n}$.

Next, we should prove that if $(u,v)$ is a smooth positive solution of (1.1) with $p, q\ge 1$ and $\alpha, \beta >0$, then a constant multiple of $(u,v)$ satisfies (1.2).

Rewrite the higher-order PDEs problem (1.1) as the following second-order system:

$$\begin{aligned} \textstyle\begin{cases} -\Delta u_{i}=u_{i+1}, u_{i} |_{\partial \mathbb{R}_{+}^{n}}=0,\\ \quad i=0,1, \ldots,m-1, \text{ with } u_{0}= \vert x \vert ^{\alpha }u, u_{m}= \vert x \vert ^{-\beta} v^{q}, \\ -\Delta v_{i}=v_{i+1}, v_{i} |_{\partial \mathbb{R}_{+}^{n}}=0,\\ \quad i=0,1, \ldots,m-1, \text{ with } v_{0}= \vert x \vert ^{\beta} v, v_{m}= \vert x \vert ^{-\alpha} u^{p}. \end{cases}\displaystyle \end{aligned}$$

(3.23)

On the other hand, rewrite the integral system (1.3) as

$$\begin{aligned} \textstyle\begin{cases} u_{i}=\int _{\mathbb{R}^{n}_{+}}G(x,y,2)u_{i+1}(y)\,dy,\\ \quad i=0,1,\ldots,m-1, \text{ with } u_{0}= \vert x \vert ^{\alpha }u, u_{m}= \vert x \vert ^{-\beta} v^{q}, \\ v_{i}=\int _{\mathbb{R}^{n}_{+}}G(x,y,2)v_{i+1}(y)\,dy, \\ \quad i=0,1,\ldots,m-1, \text{ with } v_{0}= \vert x \vert ^{\beta} v, v_{m}= \vert x \vert ^{-\alpha} u^{p}. \end{cases}\displaystyle \end{aligned}$$

(3.24)

By [1, Theorem 2.4] we know that

$$\begin{aligned} G(x,y,2j+2)= \int _{\mathbb{R}^{n}_{+}}G(x,y,2j)G(y,z,2)\,dz,\quad j=1, \ldots,m-1. \end{aligned}$$

This yields the equivalence between the integral systems (1.2) and (3.24).

Now let $(u_{0},\ldots,u_{m-1},v_{0},\ldots,v_{m-1})$ be a positive classical solution of (3.23). It suffices to show that $(u_{0},\ldots,u_{m-1},v_{0},\ldots,v_{m-1})$ does satisfy (3.24).

Let $(u_{0},\ldots,u_{m-1},v_{0},\ldots,v_{m-1})$ be a positive solution of (3.23). Multiply (3.23) by $\tilde{G}_{R}(x,y)$ and then integrate over $B_{R}^{+}(0)$ to get by (3.1) that

$$\begin{aligned} \textstyle\begin{cases} \int _{B^{+}_{R}}\tilde{G}_{R}(x,y)u_{i+1}(y)\,dy=u_{i}(x)+\int _{ \widehat{\Gamma}_{R}\cup \bar{\Gamma}_{R}}u_{i}(y) \frac{\partial \tilde{G}_{R}}{\partial \nu}(x,y)\,ds -\int _{ \widehat{\Gamma}_{R}\cup \bar{\Gamma}_{R}}\tilde{G}_{R} \frac{\partial u_{i}(y)}{\partial \nu}(x,y)\,ds, \\ \int _{B^{+}_{R}}\tilde{G}_{R}(x,y)v_{i+1}(y)\,dy=v_{i}(x)+\int _{ \widehat{\Gamma}_{R}\cup \bar{\Gamma}_{R}}v_{i}(y) \frac{\partial \tilde{G}_{R}}{\partial \nu}(x,y)\,ds -\int _{ \widehat{\Gamma}_{R}\cup \bar{\Gamma}_{R}}\tilde{G}_{R}(x,y) \frac{\partial v_{i}(y)}{\partial \nu}\,ds, \end{cases}\displaystyle \end{aligned}$$

which $i=0,1,\ldots,m-1$. By $\frac{\partial \tilde{G}_{R}}{\partial \nu}|_{\bar{\Gamma}_{R}}=0$ and $\tilde{G}_{R}|_{\widehat{\Gamma}_{R}\cup \bar{\Gamma}_{R}}=0$ we have

$$\begin{aligned} \textstyle\begin{cases} \int _{B^{+}_{R}}\tilde{G}_{R}(x,y)u_{i+1}(y)\,dy=u_{i}(x)+\int _{ \widehat{\Gamma}_{R}}u_{i}(y) \frac{\partial \tilde{G}_{R}}{\partial \nu}(x,y)\,ds,\quad i=0,1,\ldots,m-1, \\ \int _{B^{+}_{R}}\tilde{G}_{R}(x,y)v_{i+1}(y)\,dy=v_{i}(x)+\int _{ \widehat{\Gamma}_{R}}v_{i}(y) \frac{\partial \tilde{G}_{R}}{\partial \nu}(x,y)\,ds,\quad i=0,1,\ldots,m-1, \end{cases}\displaystyle \end{aligned}$$

(3.25)

which implies by Lemma 3.2 that

$$\begin{aligned} \int _{B^{+}_{R}}\tilde{G}_{R}(x,y)u_{i+1}(y) \,dy\leq u_{i}(x), \int _{B^{+}_{R}} \tilde{G}_{R}(x,y)v_{i+1}(y) \,dy\leq v_{i}(x),\quad i=0,1,\ldots,m-1. \end{aligned}$$

Letting $R\rightarrow \infty$, we deduce with (3.2) that

$$\begin{aligned} \int _{\mathbb{R}^{n}_{+}}G(x,y,2)u_{i+1}(y)\,dy< +\infty, \int _{ \mathbb{R}^{n}_{+}}G(x,y,2)v_{i+1}(y)\,dy< +\infty, \quad i=0,1, \dots,m-1, \end{aligned}$$

and hence there exists a sequence $R_{k}\rightarrow \infty$ such that

$$\begin{aligned} &\int _{\widehat{\Gamma}_{R_{k}}}G(x,y,2)u_{i+1}(y)\,ds\rightarrow 0, \int _{\widehat{\Gamma}_{R_{k}}}G(x,y,2)v_{i+1}(y)\,ds\rightarrow 0,\\ &\quad \text{as } k\rightarrow \infty, i=0,1,\dots,m-2. \end{aligned}$$

For fixed $x\in B^{+}_{R}(0)$, we have

$$\begin{aligned} G(x,y,2)|_{y\in \widehat{\Gamma}_{R}}= \biggl(\frac{1}{ \vert x-y \vert ^{n-2}}- \frac{1}{ \vert \bar{x}-y \vert ^{n-2}} \biggr)|_{y\in \widehat{\Gamma}_{R}}=O\biggl( \frac{y_{n}}{R^{n}}\biggr),\quad R\rightarrow \infty, \end{aligned}$$

(3.26)

and thus

$$\begin{aligned} \frac{\partial \tilde{G}_{R}}{\partial \nu}(x,y)=O\biggl( \frac{y_{n}}{R^{n+1}}\biggr),\quad R\rightarrow \infty. \end{aligned}$$

(3.27)

By (3.26) we derive

$$\begin{aligned} \begin{aligned} &\frac{1}{R_{k}^{n}} \int _{\widehat{\Gamma}_{R}}y_{n} u_{i+1}(y)\,ds \rightarrow 0, \\ &\frac{1}{R_{k}^{n}} \int _{\widehat{\Gamma}_{R}}y_{n} v_{i+1}(y)\,ds \rightarrow 0 \quad\text{as } k\rightarrow \infty, i=0,1,\ldots,m-2. \end{aligned} \end{aligned}$$

(3.28)

Similarly, there exists a sequence of $\{R_{k}\}$ such that

$$\begin{aligned} \frac{1}{R_{k}^{n+\beta}} \int _{\widehat{\Gamma}_{R}}y_{n} v^{q}(y)\,ds \rightarrow 0, \qquad\frac{1}{R_{k}^{n+\alpha}} \int _{\widehat{\Gamma}_{R}}y_{n} u^{p}(y)\,ds\rightarrow 0 \quad\text{as } k\rightarrow \infty. \end{aligned}$$

(3.29)

To show that the boundary terms in (3.25) approach 0 as $R\rightarrow \infty$, by (3.27) we only need to derive that there exists a sequence $R_{k}\rightarrow \infty$ such that

$$\begin{aligned} \begin{aligned} &\frac{1}{R_{k}^{n+1}} \int _{\widehat{\Gamma}_{R}}y_{n} u_{i+1}(y)\,ds \rightarrow 0, \\ &\frac{1}{R_{k}^{n+1}} \int _{\widehat{\Gamma}_{R}}y_{n} v_{i+1}(y)\,ds\rightarrow 0, \quad k\rightarrow \infty, i=0,1,\dots,m-2, \end{aligned} \end{aligned}$$

(3.30)

and

$$\begin{aligned} \frac{1}{R_{k}^{n+\alpha +1}} \int _{\widehat{\Gamma}_{R}}y_{n}u(y)\,ds \rightarrow 0,\qquad \frac{1}{R_{k}^{n+\beta +1}} \int _{\widehat{\Gamma}_{R}}y_{n}v(y)\,ds \rightarrow 0 \quad \text{as } k\rightarrow \infty. \end{aligned}$$

(3.31)

Obviously, (3.30) is a direct consequence of (3.28).

By Jensen’s inequality with $p\ge 1$ and (3.29) we have

$$\begin{aligned} \biggl(\frac{1}{R_{k}^{n+\alpha}} \int _{\widehat{\Gamma}_{R_{k}}}y_{n}^{ \frac{1}{p}}u(y)\,ds \biggr)^{p}&\leq \frac{1}{R_{k}^{\alpha p+n}} \int _{ \widehat{\Gamma}_{R_{k}}}{y_{n}}u^{p}(y)\,ds \\ &\leq \frac{1}{R_{k}^{\alpha +n}} \int _{\widehat{\Gamma}_{R_{k}}}{y_{n}}u^{p}(y)\,ds \rightarrow 0 \quad\text{as } R_{k}\rightarrow \infty, \end{aligned}$$

and hence

$$\begin{aligned} \frac{1}{R_{k}^{n+\alpha}} \int _{\widehat{\Gamma}_{R_{k}}}y_{n}^{ \frac{1}{p}}u(y)\,ds\rightarrow 0,\quad R_{k}\rightarrow \infty. \end{aligned}$$

(3.32)

Denote $\widehat{\Gamma}^{1}_{R_{k}}=\{y\in \widehat{\Gamma}_{R_{k}}, y_{n} \leq 1\}$ and $\widehat{\Gamma}^{2}_{R_{k}}=\{y\in \widehat{\Gamma}_{R_{k}}, y_{n}>1 \}$. Then

$$\begin{aligned} \int _{\widehat{\Gamma}_{R_{k}}}\frac{y_{n}}{R^{n+\alpha +1}_{k}}u(y)\,ds &= \int _{\widehat{\Gamma}^{1}_{R_{k}}} \frac{y_{n}}{R^{n+\alpha +1}_{k}}u(y)\,ds + \int _{\widehat{\Gamma}^{2}_{R_{k}}} \frac{y_{n}}{R^{n+\alpha +1}_{k}}u(y)\,ds \\ &\leq \int _{\widehat{\Gamma}^{1}_{R_{k}}} \frac{y_{n}^{\frac{1}{p}}}{R^{n+\alpha +1}_{k}}u(y)\,ds + \int _{ \widehat{\Gamma}^{2}_{R_{k}}}\frac{1}{R^{n+\alpha}_{k}}u(y)\,ds \\ &\leq \frac{1}{R_{k}} \int _{\widehat{\Gamma}^{1}_{R_{k}}} \frac{y_{n}^{\frac{1}{p}}}{R^{n+\alpha}_{k}}u(y)\,ds + \int _{ \widehat{\Gamma}^{2}_{R_{k}}} \frac{y_{n}^{\frac{1}{p}}}{R^{n+\alpha}_{k}}u(y)\,ds \\ &\leq \frac{2}{R_{k}^{n+\alpha}} \int _{\widehat{\Gamma}_{R_{k}}}y_{n}^{ \frac{1}{p}}u(y)\,ds, \end{aligned}$$

which vanishes as $R_{k}\rightarrow \infty$ by (3.32). This proves (3.31) for u. The argument for v is similar.

Substituting (3.30) and (3.31) into (3.25), we arrive at (3.24). □