Nonexistence of positive solutions for the weighted higher-order elliptic system with Navier boundary condition

Abstract

We establish a Liouville-type theorem for a weighted higher-order elliptic system in a wider exponent region described via a critical curve. We first establish a Liouville-type theorem to the involved integral system and then prove the equivalence between the two systems by using superharmonic properties of the differential systems. This improves the results in (Complex Var. Elliptic Equ. 5:1436–1450, 2013) and (Abstr. Appl. Anal. 2014:593210, 2014).

1 Introduction

In this paper, we establish a Liouville-type theorem for the weighted 2mth-order elliptic equations coupled via the Navier boundary conditions in the half-space $$\mathbb{R}_{+}^{n}=\{x\in \mathbb{R}^{n}: x_{n}>0\}$$:

\begin{aligned} \textstyle\begin{cases} (-\Delta )^{m}( \vert x \vert ^{\alpha} u(x))= \vert x \vert ^{-\beta} v^{q}\quad\text{in } \mathbb{R}^{n}_{+}, \\ (-\Delta )^{m}( \vert x \vert ^{\beta}v(x))= \vert x \vert ^{-\alpha}u^{p}\quad\text{in } \mathbb{R}^{n}_{+}, \\ u=\Delta u=\cdots =\Delta ^{m-1}u=0 \quad\text{on } \partial \mathbb{R}^{n}_{+}, \\ v=\Delta v=\cdots =\Delta ^{m-1}v=0 \quad\text{on } \partial \mathbb{R}^{n}_{+}, \end{cases}\displaystyle \end{aligned}
(1.1)

where m is a positive integer satisfying $$0<2m<n$$, $$p,q\geq 1$$, and $$\alpha, \beta \geq 0$$, which is closely related to the following integral system:

\begin{aligned} \textstyle\begin{cases} u(x)=C_{n}\int _{\mathbb{R}^{n}_{+}} \frac{1}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}} (\frac{1}{ \vert x-y \vert ^{n-2m}}- \frac{1}{ \vert \bar{x}-y \vert ^{n-2m}} )v^{q} (y)\,dy, \\ v(x)=C_{n}\int _{\mathbb{R}^{n}_{+}} \frac{1}{ \vert x \vert ^{\beta} \vert y \vert ^{\alpha}} (\frac{1}{ \vert x-y \vert ^{n-2m}}- \frac{1}{ \vert \bar{x}-y \vert ^{n-2m}} )u^{p} (y)\,dy, \end{cases}\displaystyle \end{aligned}
(1.2)

where $$C_{n}>0$$, and $$\bar{x}=(x_{1},\ldots,x_{n-1},-x_{n})$$ is the reflection of the point x about the $$\partial \mathbb{R}_{+}^{n}$$. Similar to some integral systems or partial differential systems, the integral system (1.2) is usually divided into three cases according to the value of the exponents $$(p,q)$$. We introduce the critical curve

\begin{aligned} \frac{1}{p+1}+\frac{1}{q+1}=\frac{n-2m+\alpha +\beta}{n} \end{aligned}
(1.3)

for (1.2) to determine a Liouville-type theorem.

The well-known classical Hardy–Littlewood–Sobolev inequality states that

\begin{aligned} \int _{\mathbb{R}^{n}} \int _{\mathbb{R}^{n}} \frac{f(x)g(y)}{ \vert x-y \vert ^{\nu}}\,dx\,dy \leq C_{l, \nu, n} \Vert f \Vert _{h} \Vert g \Vert _{l} \end{aligned}

for all $$f\in L^{h}(R^{n})$$ and $$g\in L^{l}(R^{n})$$, where $$1< h, l<\infty$$, $$0<\nu <n$$, and $$\frac{1}{h}+\frac{1}{l}+\frac{\nu}{n}=2$$. Hardy and Littlewood also introduced the double weighted inequality, which was generalized by Stein and Weiss [13]. This inequality is called the double weighted Hardy–Littlewood–Sobolev (WHLS) inequality

\begin{aligned} \biggl\vert \int _{\mathbb{R}^{n}} \int _{\mathbb{R}^{n}} \frac{f(x)g(y)}{ \vert x \vert ^{\tau} \vert x-y \vert ^{\nu} \vert y \vert ^{\kappa}}\,dx\,dy \biggr\vert \leq C_{ \tau, \kappa, l, \nu, n} \Vert f \Vert _{h} \Vert g \Vert _{l}, \end{aligned}
(1.4)

where $$1< l$$, $$h<\infty$$, $$0<\nu <n$$, $$\tau +\kappa \geq 0$$, and τ and κ satisfy $$1-\frac{1}{h}-\frac{\nu}{n}<\frac{\tau}{n}<1-\frac{1}{h}$$ with $$\frac{1}{l}+\frac{1}{h}+\frac{\nu +\kappa +\tau}{n}=2$$. To obtain the best constant in the weighted inequality (1.4), we can maximize the functional

\begin{aligned} J(f,g)= \int _{\mathbb{R}^{n}} \int _{\mathbb{R}^{n}} \frac{f(x)g(y)}{ \vert x \vert ^{\tau} \vert x-y \vert ^{\nu} \vert y \vert ^{\kappa}}\,dx\,dy \end{aligned}

under the constrains $$\|f\|_{h}=\|g\|_{l}=1$$. The corresponding Euler–Lagrange equations are the following system of integral equations:

\begin{aligned} \textstyle\begin{cases} {\lambda }_{1} hf(x)^{h-1}=\int _{\mathbb{R}^{n}} \frac{g(y)}{ \vert x \vert ^{\tau } \vert y \vert ^{\kappa } \vert x-y \vert ^{\mu}}\,dy, \\ {\lambda }_{2} lg(x)^{l-1}=C_{n}\int _{\mathbb{R}^{n}} \frac{f(y)}{ \vert x \vert ^{\kappa } \vert y \vert ^{\tau } \vert x-y \vert ^{\mu}}\,dy, \end{cases}\displaystyle \end{aligned}
(1.5)

where $$f, g\geq 0$$, $$x\in \mathbb{R}^{n}$$, and $${\lambda }_{1} h={\lambda }_{2} l=J(f,g)$$. Let $$u=c_{1} f^{h-1}$$, $$v=c_{2} g^{l-1}$$, $$p=\frac{1}{h-1}$$, $$q=\frac{1}{l-1}$$ with $$pq\neq 1$$. Then by a proper choice of constants $$c_{1}$$ and $$c_{2}$$ system (1.5) becomes

\begin{aligned} \textstyle\begin{cases} u(x)=C_{n}\int _{\mathbb{R}^{n}} \frac{1}{ \vert x \vert ^{\tau } \vert y \vert ^{\kappa}}\frac{1}{ \vert x-y \vert ^{\mu}}v^{q} (y)\,dy, \\ v(x)=C_{n}\int _{\mathbb{R}^{n}} \frac{1}{ \vert x \vert ^{\kappa } \vert y \vert ^{\tau}}\frac{1}{ \vert x-y \vert ^{\mu}}u^{p} (y)\,dy, \end{cases}\displaystyle \end{aligned}
(1.6)

where $$u, v\geq 0$$, $$0< p, q<\infty$$, $$0<\mu <n$$, $$\frac{\tau}{n}<\frac{1}{p+1}<\frac{\mu +\tau}{n}$$, and $$\frac{1}{p+1}+\frac{1}{q+1}=\frac{\mu +\tau +\kappa}{n}$$.

Jin and Li [10] derived that the positive solution of systems (1.6) is symmetric and monotonic. In [6] and [9], they also discussed the regularity of solutions to (1.6). Lei and Lü [11] proved that system (1.6) and the related differential systems are equivalent to each other under the condition $$\max \{\tau (p+1), \kappa (q+1)\}\leq n-\mu$$ with $$pq>1$$ and $$\tau, \kappa \geq 0$$, and the positive locally bounded solutions are symmetric and decreasing about some axis. The Liouville-type theorem to the whole space problem was established by Ma and Chen [8]. In recent years, the nonlocal fractional Laplacian ($$0< m<1$$) on the whole space has received much attention from researchers. Zhuo and Li [17] had proved the nonexistence of an antisymmetric solution in the case $$0< p\leq \frac{n+2m}{n-2m}$$, whereas Li and Zhuo [18] have proved the consequence of systems in the case $$0 < pq < 1$$ or $$p + 2m > 1$$ and $$q + 2m > 1$$ with $$0< p,q\leq \frac{n+2m}{n-2m}$$. For more related results, see [1923] and the references therein.

For $$\alpha =\beta =0$$ in system (1.2), Zhuo and Li [14] established the symmetry of solutions to an integral system, and Cao and Dai [3] obtained the nonexistence of nontrivial solutions. Zhao, Yang, and Zheng proved the nonexistence of nontrivial solutions for partial differential equations (1.1) in [15] and considered the general nonlinear source in [16].

For $$\alpha,\beta \neq 0$$ in system (1.2), Cao and Dai [4] obtained a Liouville-type theorem in the super- and subcritical cases under some integrability conditions by the Pohozaev-type identity of integral form, and in the critical case, they showed that a pair of positive solutions to the system is rotationally symmetric about the $$x_{n}$$-axis. Also, we mention the recent important works on the existence and asymptotic analysis of nontrivial solutions for some elliptic systems; see [2428].

In the present paper, instead of (1.1), we will first establish a Liouville-type theorem for the integral system (1.2) in the supercritical case and then prove that systems (1.2) and (1.1) are equivalent by using the superharmonic properties, that is, the following two propositions.

Proposition 1

Let $$(u,v)\in L^{q_{1}}(\mathbb{R}^{n}_{+})\times L^{q_{2}}(\mathbb{R}^{n}_{+})$$ be a nonnegative solution of system (1.2), and let $$q_{1}:=\frac{n(pq-1)}{(2m-\alpha -\beta )(1+q)}$$ and $$q_{2}:=\frac{n(pq-1)}{(2m-\alpha -\beta )(1+p)}$$ with $$p, q\ge 1$$, $$pq\neq 1$$, and $$\alpha +\beta <2m$$. If

\begin{aligned} \frac{1}{p+1}+\frac{1}{q+1}< \frac{n-2m+\alpha +\beta}{n}, \end{aligned}
(1.7)

then $$(u,v)\equiv (0,0)$$.

Proposition 2

Let $$p, q\ge 1$$ with $$pq\neq 1$$, and let $$\alpha +\beta <2m$$. Then the differential system (1.1) is equivalent to the integral system (1.2).

Remark 1

Without the growth conditions

\begin{aligned} \bigl\vert (-\Delta )^{m-1}u \bigr\vert , \bigl\vert (-\Delta )^{m-1}v \bigr\vert =O\bigl( \vert x \vert ^{a}\bigr),\quad a\in (0,1), \vert x \vert \rightarrow \infty, \end{aligned}

in [12, Theorem 1], we can arrive at the same result by using the proof of Proposition 2.

Remark 2

By Proposition 2 we can show that the conclusions of [4, Theorems 1.2 and 1.3] hold for the partial differential system (1.1). Moreover, the conditions $$\frac{1}{p+1}<\frac{n-2m}{2n}+\frac{\alpha}{n}$$ and $$\frac{1}{q+1}<\frac{n-2m}{2n}+\frac{\beta}{n}$$ in [4, Theorem 1.2] are covered by condition (1.7).

Based on Propositions 1 and 2, the main result of the paper is the following theorem.

Theorem 1

Under the conditions of Proposition 1, the classical nonnegative solutions of system (1.1) must be trivial.

To prove Proposition 1, we will explore the moving plane method in integral forms by Chen, Li, and Ou [5]. For the proof of Proposition 2, we first prove the superharmonic properties of systems (1.1) and then establish the equivalence between the two systems by using a technique introduced in [7] for the scalar case of higher-order equations.

Next, we will prove Propositions 1 and 2 in Sects. 2 and 3, respectively.

2 Proof of Proposition 1

We introduce three lemmas for the integral system (1.2) as preliminaries, and let $$C_{n}=1$$ there for simplicity.

Denote

\begin{aligned} G(x,y):=\frac{1}{ \vert x-y \vert ^{n-2m}}-\frac{1}{ \vert \bar{x}-y \vert ^{n-2m}},\quad x, y \in \mathbb{R}_{+}^{n}, \end{aligned}

with reflecting x about the $$\partial \mathbb{R}_{+}^{n}$$. Let $$x^{{\lambda }}=(x_{1},x_{2},\ldots,2{\lambda }-x_{n})$$ be the reflection of the point x about the plane $$T_{{\lambda }}=\{x\in \mathbb{R}^{n}_{+} | x_{n}={\lambda }\}$$, and denote $$u_{\lambda}(x)=u(x^{\lambda})$$, $$v_{\lambda}(x)=v(x^{\lambda})$$. Define $$\Sigma _{\lambda}:=\{x\in \mathbb{R}^{n}_{+} | 0< x_{n}<\lambda \}$$ and $$\tilde{\Sigma}_{\lambda}:=\{x^{\lambda} | x\in \Sigma _{ \lambda}\}$$, $$\Sigma _{\lambda}^{c}=\mathbb{R}^{n}_{+}\setminus \Sigma _{\lambda}$$. The following lemma on the Green function $$G(x,y)$$ in $$\Sigma _{\lambda}$$ is known.

Lemma 2.1

([2, Lemma 2.1]) (i) For all $$x, y\in \Sigma _{\lambda}$$, $$x\neq y$$, we have

\begin{aligned} & G\bigl(x^{\lambda},y^{\lambda}\bigr)>\max \bigl\{ G \bigl(x^{\lambda},y\bigr),G\bigl(x,y^{\lambda}\bigr) \bigr\} , \\ & G\bigl(x^{\lambda},y^{\lambda}\bigr)-G(x,y)> \bigl\vert {G \bigl(x^{\lambda},y\bigr)-G\bigl(x,y^{\lambda}\bigr)} \bigr\vert . \end{aligned}

(ii) For all $$x\in \Sigma _{\lambda}$$, $$y\in \Sigma _{\lambda}^{c}$$, we have

\begin{aligned} G\bigl(x^{\lambda},y\bigr)>G(x,y). \end{aligned}

Lemma 2.2

Let $$(u,v)$$ be a nonnegative solution of (1.2). For all $$x\in \Sigma _{\lambda}$$, we have

\begin{aligned} &u(x)-u_{\lambda}(x)\leq \int _{\Sigma _{\lambda}}G\bigl(x^{\lambda},y^{ \lambda}\bigr) \frac{[v^{q}-v_{\lambda}^{q}](y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy, \\ &v(x)-v_{\lambda}(x)\leq \int _{\Sigma _{\lambda}}G\bigl(x^{\lambda},y^{ \lambda}\bigr) \frac{[u^{p}-u_{\lambda}^{p}](y)}{ \vert x \vert ^{\beta} \vert y \vert ^{\alpha}}\,dy. \end{aligned}

Proof

Since

\begin{aligned} &u(x)= \int _{\Sigma _{\lambda}}G(x,y) \frac{v^{q}(y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy + \int _{\Sigma _{\lambda}}G\bigl(x,y^{ \lambda}\bigr)\frac{v_{\lambda}^{q}(y)}{ \vert x \vert ^{\alpha} \vert y^{\lambda} \vert ^{\beta}} \,dy + \int _{\Sigma ^{c}_{\lambda}\setminus \tilde{\Sigma}_{\lambda}}G(x,y) \frac{v^{q}(y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy, \\ &u_{\lambda}(x)= \int _{\Sigma _{\lambda}}G\bigl(x^{\lambda},y\bigr) \frac{v^{q}(y)}{ \vert x^{\lambda} \vert ^{\alpha} \vert y \vert ^{\beta}} \,dy+ \int _{\Sigma _{ \lambda}}G\bigl(x^{\lambda},y^{\lambda}\bigr) \frac{v_{\lambda}^{q}(y)}{ \vert x^{\lambda} \vert ^{\alpha} \vert y^{\lambda} \vert ^{\beta}}\,dy \\ &\phantom{u_{\lambda}(x)=}{}+ \int _{\Sigma ^{c}_{\lambda}\setminus \tilde{\Sigma}_{\lambda}}G\bigl(x^{ \lambda},y\bigr)\frac{v^{q}(y)}{ \vert x^{\lambda} \vert ^{\alpha} \vert y \vert ^{\beta}} \,dy, \end{aligned}

we have by Lemma 2.1 that

\begin{aligned} u(x)-u_{{\lambda }}(x) \leq{}& \int _{\Sigma _{\lambda}}\bigl[G\bigl(x^{\lambda},y^{ \lambda} \bigr)-G\bigl(x,y^{\lambda}\bigr)\bigr]\frac{v^{q}(y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy\\ &{} - \int _{\Sigma _{\lambda}}\bigl[G\bigl(x^{\lambda},y^{\lambda} \bigr)-G\bigl(x,y^{\lambda}\bigr)\bigr] \frac{v_{\lambda}^{q}(y)}{ \vert x^{\lambda} \vert ^{\alpha} \vert y^{\lambda} \vert ^{\beta}}\,dy \\ &{} + \int _{\Sigma ^{c}_{\lambda}\setminus \tilde{\Sigma}_{{\lambda }}}\bigl[G(x,y)-G\bigl(x^{{ \lambda }},y\bigr)\bigr] \frac{v^{q}(y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy \\ \leq{}& \int _{\Sigma _{\lambda}}\bigl[G\bigl(x^{\lambda},y^{\lambda} \bigr)-G\bigl(x,y^{ \lambda}\bigr)\bigr] \frac{[v^{q}-v_{\lambda}^{q}](y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy \\ \leq{}& \int _{\Sigma _{\lambda}}G\bigl(x^{\lambda},y^{\lambda}\bigr) \frac{[v^{q}-v_{\lambda}^{q}](y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy. \end{aligned}

The second inequality can be obtained in the same way. □

In addition, we also need the weighted Hardy–Littlewood–Sobolev inequality.

Lemma 2.3

([10]) Let $$1< l, m<\infty$$, $$0<\nu <n$$, $$\tau +\kappa \geq 0$$, $$\frac{1}{l}+\frac{1}{m}+\frac{\nu +\kappa +\tau}{n}=2$$, and $$1-\frac{1}{m}-\frac{\nu}{n}<\frac{\tau}{n}<1-\frac{1}{m}$$. Then

\begin{aligned} \biggl\vert \int _{\mathbb{R}^{n}} \int _{\mathbb{R}^{n}} \frac{f(x)g(y)}{ \vert x \vert ^{\tau} \vert x-y \vert ^{\nu} \vert y \vert ^{\kappa}}\,dx\,dy \biggr\vert \leq C \Vert f \Vert _{m} \Vert g \Vert _{l} \end{aligned}

with $$C=C(\tau, \kappa, l, \nu, n)>0$$, or, equivalently,

\begin{aligned} \bigl\Vert Tg(x) \bigr\Vert _{\gamma}:=\sup_{ \Vert f \Vert _{m}=1} \bigl\langle Tg(x),f(x)\bigr\rangle \leq C \Vert g \Vert _{l} \end{aligned}

with $$Tg(x)=\int _{\mathbb{R}^{n}} \frac{g(y)}{|x|^{\tau}|x-y|^{\nu}|y|^{\kappa}}\,dy$$, $$\frac{1}{l}+\frac{\nu +\kappa +\tau}{n}=1+\frac{1}{\gamma}$$ and $$\frac{1}{m}+\frac{1}{\gamma}=1$$.

Now we can prove Proposition 1.

Proof of Proposition 1

We apply the moving-plane method in two steps.

1. Determine the starting position

Start from the very low end of $$\mathbb{R}^{n}_{+}$$, i.e., near $$x_{n}=0$$. We will show that for λ sufficiently small,

\begin{aligned} w_{{\lambda }}(x):=u(x)-u_{{\lambda }}(x)\leq 0,\qquad g_{{\lambda }}(x):=v(x)-v_{{ \lambda }}(x) \leq 0 \quad\text{a.e. in } \Sigma _{{\lambda }}. \end{aligned}
(2.1)

Denote

\begin{aligned} B_{\lambda}^{u}:=\bigl\{ x\in \Sigma _{\lambda} | w_{{\lambda }}(x)>0 \bigr\} , \qquad B_{\lambda}^{v}:=\bigl\{ x\in \Sigma _{\lambda} | g_{{ \lambda }}(x)>0 \bigr\} . \end{aligned}

We will prove that $$B_{{\lambda }}^{u}$$ and $$B_{{\lambda }}^{v}$$ must be of zero measure, provided that λ sufficiently small. In fact, by Lemma 2.2 with the mean value theorem we have that for sufficiently small λ and $$x\in B_{\lambda}^{u}$$,

\begin{aligned} 0&\leq w_{{\lambda }}(x) = \int _{B^{v}_{\lambda}}+ \int _{\Sigma _{ \lambda}\setminus B^{v}_{\lambda} }G\bigl(x^{\lambda},y^{\lambda}\bigr) \frac{[v^{q}-v_{\lambda}^{q}](y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy \\ &\leq \int _{B^{v}_{\lambda}}G\bigl(x^{\lambda},y^{\lambda}\bigr) \frac{[v^{q}-v_{\lambda}^{q}](y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy \\ &\leq q \int _{B^{v}_{\lambda}} \frac{[v^{q-1}(v-v_{{\lambda }})](y)}{ \vert x-y \vert ^{n-2m} \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy. \end{aligned}

Furthermore, by Lemma 2.3 with Hölder’s inequality and $$q_{1}^{*}=\frac{q_{1}}{q_{1}-1}$$

\begin{aligned} \Vert w_{{\lambda }} \Vert _{q_{1}, B^{u}_{\lambda}} &\leq \sup _{ \Vert f \Vert _{q_{1}^{*}}=1} \int _{B^{v}_{\lambda}} \frac{[v^{q-1}(v-v_{{\lambda }})](y)}{ \vert x-y \vert ^{n-2m} \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy \\ &=C \bigl\Vert v^{q-1}g_{{\lambda }} \bigr\Vert _{l, B^{v}_{\lambda}} \\ &\leq C \Vert v \Vert ^{q-1}_{q_{2}, B^{v}_{\lambda}} \Vert g_{{\lambda }} \Vert _{q_{2}, B^{v}_{\lambda}} \end{aligned}
(2.2)

with the universal constant $$C>0$$, where the supercritical inequality (1.7) with $$p, q\ge 1$$ and $$\alpha +\beta <2m$$ implies

\begin{aligned} &q_{1}=\frac{n(pq-1)}{(2m-\alpha -\beta )(1+q)}>p+1>1, \\ &q_{2}=\frac{n(pq-1)}{(2m-\alpha -\beta )(1+p)}>q+1>1, \end{aligned}

and

\begin{aligned} \frac{1}{l}=1+\frac{1}{q_{1}}-\frac{n-2m+\alpha +\beta}{n}= \frac{(2m-\alpha -\beta )(1+p)q}{n(pq-1)}< \frac{q}{q+1}< 1. \end{aligned}

Similarly, we have

\begin{aligned} \Vert g_{{\lambda }} \Vert _{q_{2}, B^{v}_{\lambda}}\leq C \Vert u \Vert ^{p-1}_{q_{1}, B^{u}_{ \lambda}} \Vert w_{{\lambda }} \Vert _{q_{1}, B^{u}_{\lambda}}. \end{aligned}
(2.3)

It follows from (2.2) and (2.3) that

\begin{aligned} \Vert w_{{\lambda }} \Vert _{q_{1}, B^{u}_{\lambda}}\leq C \Vert u \Vert ^{p-1}_{q_{1}, B^{u}_{\lambda}} \Vert v \Vert ^{q-1}_{q_{2}, B^{v}_{\lambda}} \Vert w_{{\lambda }} \Vert _{q_{1}, B^{u}_{\lambda}}. \end{aligned}
(2.4)

Since $$(u,v)\in L^{q_{1}}(\mathbb{R}^{n}_{+})\times L^{q_{2}}(\mathbb{R}^{n}_{+})$$, we can choose λ small enough such that

\begin{aligned} C \Vert u \Vert ^{p-1}_{q_{1}, B^{u}_{\lambda}} \Vert v \Vert ^{q-1}_{q_{2}, B^{v}_{ \lambda}}< \frac{1}{2}, \end{aligned}

and thus $$\|w_{{\lambda }}\|_{q_{1}, B^{u}_{\lambda}}=0$$ by (2.4). In the same way, $$\|g_{{\lambda }}\|_{q_{2}, B^{v}_{\lambda}}=0$$. This proves (2.1).

2. Move the plane to the infinity

Inequalities (2.1) provide a starting point to move the plane $$T_{{\lambda }}$$. We start from a neighborhood of λ and move the plane up as long as (2.1) holds. Define

\begin{aligned} {\lambda }_{0}:=\sup \{{\lambda } | w_{\rho}, g_{\rho}\leq 0, \rho \leq {\lambda }\ {\mathrm{for\ a.e.}}\ x \in \Sigma _{\rho}\}. \end{aligned}
(2.5)

We first prove that $${\lambda }_{0}=\infty$$. Assume for contradiction that $${\lambda }_{0}<\infty$$. We claim that

\begin{aligned} w_{{\lambda }_{0}}(x)=g_{{\lambda }_{0}}(x)= 0\quad {\mathrm{a.e. in }}\ \Sigma _{{\lambda }_{0}}. \end{aligned}
(2.6)

Otherwise, for such $${\lambda }_{0}$$, e.g.,

\begin{aligned} E_{0}:=\bigl\{ x| g_{{\lambda }_{0}}(x)< 0, x\in \Sigma _{{\lambda }_{0}} \bigr\} \quad {\mathrm{has\ a\ positive\ measure.}} \end{aligned}
(2.7)

By Lemma 2.2

\begin{aligned} u(x)-u_{{\lambda }_{0}}(x)&\le \int _{\Sigma _{\lambda _{0}}}G\bigl(x^{ \lambda _{0}},y^{\lambda _{0}}\bigr) \frac{[v^{q}-v_{\lambda _{0}}^{q}](y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy \\ &= \int _{E_{0}}G\bigl(x^{\lambda _{0}},y^{\lambda _{0}}\bigr) \frac{[v^{q}-v_{\lambda _{0}}^{q}](y)}{ \vert x \vert ^{\alpha} \vert y \vert ^{\beta}}\,dy, \end{aligned}

Consequently,

\begin{aligned} w_{{\lambda }_{0}}(x)< 0\quad {\mathrm{a.e.\ in}}\ \Sigma _{\lambda _{0}}. \end{aligned}
(2.8)

Denote $$\lambda _{\epsilon}:=\lambda +\epsilon$$ with $$\epsilon >0$$ to be determined. For any small $$\eta >0$$, choose R sufficiently large such that

\begin{aligned} \int _{\mathbb{R}^{n}_{+}\setminus B_{R}(0)} \vert u \vert ^{q_{1}}(y)\,dy \leq \eta. \end{aligned}

It follows from Lusin’s theorem and (2.8) that for any $$\delta >0$$, there exists a closed set $$F_{\delta}\subset E:=\Sigma _{\lambda _{0}}\bigcap B_{R}(0)$$ with $$m(E\setminus F_{\delta})<\delta$$ such that $$w_{{\lambda }_{0}}(x)<0$$ and is continuous in $$F_{\delta}$$. Choosing $$\epsilon >0$$ sufficiently small, we have

\begin{aligned} w_{\lambda _{\epsilon}}(x)< 0 \quad\text{for all } x\in F_{\delta} \end{aligned}

by continuity. Denote $$D_{\lambda _{\epsilon}}:=(\Sigma _{\lambda _{\epsilon}}\setminus \Sigma _{\lambda _{0}})\cap B_{R}(0)$$. Then

\begin{aligned} B_{\lambda _{\epsilon}}^{u}\subset M:=\bigl(\mathbb{R}^{n}_{+} \setminus B_{R}(0)\bigr) \cup (E\setminus F_{\delta})\cup D_{\lambda _{\epsilon}}. \end{aligned}

Let R be large and δ and ϵ small such that $$\int _{B_{\lambda _{\epsilon}}^{u}}|u|^{q_{1}}(y)\,dy\leq \int _{M}|u|^{q_{1}}(y)\,dy \leq \frac{1}{2}$$. Similarly, $$\int _{B_{\lambda _{\epsilon}}^{v}}|v|^{q_{2}}(y)\,dy\leq \frac{1}{2}$$.

By (2.4) with $${\lambda }={\lambda }_{\epsilon}$$ we can get

\begin{aligned} \Vert w_{\lambda _{\epsilon}} \Vert _{q_{1}, B^{u}_{\lambda _{\epsilon}}} \leq C \Vert u \Vert ^{p-1}_{q_{1}, B^{u}_{\lambda _{\epsilon}}} \Vert v \Vert ^{q-1}_{q_{2}, B^{v}_{\lambda _{\epsilon}}} \Vert w_{\lambda _{\epsilon}} \Vert _{q_{1}, B^{u}_{ \lambda _{\epsilon}}}\leq \frac{1}{4} \Vert w_{\lambda _{\epsilon}} \Vert _{q_{1}, B^{u}_{\lambda _{\epsilon}}}, \end{aligned}

which implies $$\|w_{\lambda _{\epsilon}}\|_{q_{1}, B^{u}_{\lambda _{\epsilon}}} \equiv 0$$. Thus

\begin{aligned} w_{{\lambda }_{\epsilon}}(x)\le 0\quad {\mathrm{a.e.\ in}}\ \Sigma _{{\lambda }_{ \epsilon}}, \end{aligned}

and, similarly,

\begin{aligned} g_{{\lambda }_{\epsilon}}(x)\le 0\quad {\mathrm{a.e.\ in}}\ \Sigma _{{\lambda }_{ \epsilon}}. \end{aligned}

This contradicts (2.7) with (2.5). Thus (2.6) holds. This yields the contradiction that $$u(x)=v(x)\equiv 0$$ on the plane $$\{x_{n}=2\lambda _{0}\}$$. We conclude that $$\lambda _{0}=+\infty$$, which implies that both u and v are strictly monotonically increasing with respect to $$x_{n}$$. Moreover, we know that $$u\in L^{q_{1}}(\mathbb{R}^{n}_{+})$$ and $$v\in L^{q_{2}}(\mathbb{R}^{n}_{+})$$and for any $$a>0$$,

\begin{aligned} &\int _{\mathbb{R}^{n}_{+}} \bigl\vert u\bigl(x',x_{n} \bigr) \bigr\vert ^{q_{1}}\,dx'\,dx_{n}\geq \int _{ \mathbb{R}^{n-1}} \int _{a}^{\infty} \bigl\vert u\bigl(x',a \bigr) \bigr\vert ^{q_{1}}\,dx_{n}\,dx', \\ &\int _{\mathbb{R}^{n}_{+}} \bigl\vert v\bigl(x',x_{n} \bigr) \bigr\vert ^{q_{2}}\,dx'\,dx_{n}\geq \int _{ \mathbb{R}^{n-1}} \int _{a}^{\infty} \bigl\vert v\bigl(x',a \bigr) \bigr\vert ^{q_{2}}\,dx_{n}\,dx', \end{aligned}

and hence $$u(x',a)=v(x',a)=0$$ for all $$x'\in \mathbb{R}^{n-1}$$, a contradiction. □

3 Proof of Proposition 2

Denote by $$B_{R}(0):=\{x\in \mathbb{R}^{n}, |x|< R\}$$ the ball of radius R centered at the origin in $$\mathbb{R}^{n}$$ with $$B_{R}^{+}(0):=B_{R}(0)\cap \mathbb{R}^{n}_{+}$$ and $$\partial B_{R}^{+}(0):=\Gamma _{R}=\bar{\Gamma}_{R}\cup \widehat{\Gamma}_{R}$$, the union of the flat and hemisphere parts of $$\Gamma _{R}$$. Let $$x*:=\frac{x}{|x^{2}|}R^{2}$$ be the reflection of x about $$\partial B_{R}(0)$$, and let

\begin{aligned} \tilde{G}_{R}(x,y):= \biggl(\frac{1}{ \vert x-y \vert ^{n-2}}- \frac{1}{(\frac{ \vert x \vert }{R} \vert x^{*}-y \vert )^{n-2}} \biggr)- \biggl( \frac{1}{ \vert \bar{x}-y \vert ^{n-2}}- \frac{1}{(\frac{ \vert \bar{x} \vert }{R} \vert \bar{x}^{*}-y \vert )^{n-2}} \biggr). \end{aligned}

We begin with the well-known lemma.

Lemma 3.1

([1, Lemma 2.1])

(i) For $$x\in B_{R}^{+}(0)$$, $$\tilde{G}_{R}(x,y)$$ satisfies the equation

\begin{aligned} \textstyle\begin{cases} -\Delta \tilde{G}_{R}(x,y)=\delta (x-y)&\textit{in } B_{R}^{+}(0), \\ \tilde{G}_{R}(x,y)=0&\textit{on } \partial B_{R}^{+}(0). \end{cases}\displaystyle \end{aligned}
(3.1)

(ii) For $$x,y\in B_{R}^{+}(0)$$,

\begin{aligned} \tilde{G}_{R}(x,y)\rightarrow G(x,y,2)=\frac{1}{ \vert x-y \vert ^{n-2}}- \frac{1}{ \vert \bar{x}-y \vert ^{n-2}} \quad \textit{as } R\rightarrow \infty. \end{aligned}
(3.2)

(iii) For $$x\in B_{R}^{+}(0)$$ and $$y\in \widehat{\Gamma}_{R}$$,

\begin{aligned} \frac{\partial \tilde{G}_{R}}{\partial \nu}(x,y)=(2-n)R \biggl(1- \frac{ \vert x \vert ^{2}}{R^{2}} \biggr) \biggl( \frac{1}{ \vert x-y \vert ^{n}}- \frac{1}{ \vert \bar{x}-y \vert ^{n}} \biggr), \end{aligned}

where ν is the outward unit normal vector of $$\widehat{\Gamma}_{R}$$.

We follow the main idea of Chen, Fang, and Li [7] to give superharmonic properties of system (1.1). This result plays a key role in the proof of Proposition 2.

Lemma 3.2

If $$(u,v)$$ is a positive solution of (1.1), then

\begin{aligned} (-\Delta )^{i}\bigl( \vert x \vert ^{\alpha }u\bigr)>0,\qquad (- \Delta )^{i}\bigl( \vert x \vert ^{\beta }v\bigr)>0,\quad i=1, \ldots,m-1, x\in \mathbb{R}^{n}_{+}. \end{aligned}

Proof

We make an odd extension of u and v to the whole space. Define

\begin{aligned} u\bigl(x', x_{n}\bigr)=-u\bigl(x', -x_{n}\bigr),\qquad v\bigl(x', x_{n}\bigr)=-v \bigl(x', -x_{n}\bigr),\quad x_{n}< 0, \end{aligned}

with $$x'=(x_{1},\ldots,x_{n-1})$$. Then $$(u,v)$$ satisfy

\begin{aligned} \textstyle\begin{cases} (-\Delta )^{m}( \vert x \vert ^{\alpha} u(x))= \vert x \vert ^{-\beta} \vert v \vert ^{q-1}v&\text{in } \mathbb{R}^{n}, \\ (-\Delta )^{m}( \vert x \vert ^{\beta}v(x))= \vert x \vert ^{-\alpha} \vert u \vert ^{p-1}u &\text{in } \mathbb{R}^{n}. \end{cases}\displaystyle \end{aligned}
(3.3)

Write $$u_{i}(x):=(-\Delta )^{i}(|x|^{\alpha }u)$$ and $$v_{i}(x):=(-\Delta )^{i}(|x|^{\beta }v)$$. We will prove that $$u_{i}(x),v_{i}(x)>0$$, $$x\in \mathbb{R}^{n}_{+}$$, $$i=1,2,\ldots,m-1$$.

Step 1. We claim that $$u_{m-1}(x)\geq 0$$, $$x\in \mathbb{R}^{n}_{+}$$. Otherwise, there exists $$x_{1}\in \mathbb{R}^{n}_{+}$$ such that

\begin{aligned} u_{m-1}(x_{1})< 0. \end{aligned}
(3.4)

We will deduce a contradiction by two substeps.

(i) We first claim

\begin{aligned} (-1)^{i}\hat{u}_{m-i}(r)>0,\quad \forall r\geq 0, i=1,2, \ldots,m-1, \end{aligned}
(3.5)

where $$\hat{u}_{m-i}$$ is the $$(m-i)$$th average of $$u_{m-i}$$.

Denote by $$B_{r}(x_{1})$$ the ball of radius r centered at $$x_{1}$$, and define the first averages of u and v on $$\partial B_{r}(x_{1})$$ as

\begin{aligned} &\bar{u}(r):=\frac{1}{ \vert \partial B_{r}(x_{1}) \vert } \int _{\partial B_{r}(x_{1})} \vert x \vert ^{ \alpha }u(x)\,ds;\qquad \bar{v}(r):=\frac{1}{ \vert \partial B_{r}(x_{1}) \vert } \int _{ \partial B_{r}(x_{1})} \vert x \vert ^{\beta }v(x)\,ds \end{aligned}

and

\begin{aligned} &\bar{u}_{i}(r):=\frac{1}{ \vert \partial B_{r}(x_{1}) \vert } \int _{\partial B_{r}(x_{1})}u_{i}(x)\,ds;\qquad \bar{v}_{i}(r):= \frac{1}{ \vert \partial B_{r}(x_{1}) \vert } \int _{\partial B_{r}(x_{1})}v_{i}(x)\,ds \end{aligned}

with $$i=2,3,\ldots,m-1$$. Then for $$r>0$$, we have by (3.3) that for $$x\in \mathbb{R}^{n}$$,

\begin{aligned} \textstyle\begin{cases} -\Delta \bar{u}=\bar{u}_{1}, &-\Delta \bar{v}=\bar{v}_{1}, \\ -\Delta \bar{u}_{1}=\bar{u}_{2},&-\Delta \bar{v}_{1}= \bar{v}_{2}, \\ \ldots \\ -\Delta \bar{u}_{m-2}=\bar{u}_{m-1}, &-\Delta \bar{v}_{m-2}= \bar{v}_{m-1}, \\ -\Delta \bar{u}_{m-1}=f(r), &-\Delta \bar{v}_{m-1}=g(r), \end{cases}\displaystyle \end{aligned}
(3.6)

where $$f(r):=\overline{|x|^{-\beta}|v|^{q-1}v}$$ and $$g(r):=\overline{|x|^{-\alpha}|u|^{p-1}u}$$. Integrate the last equation for u in (3.6) from 0 to r. Notice that $$x_{1}\in \mathbb{R}^{n}_{+}$$ implies that more than half of $$B_{r}(x_{1})$$ is contained in $$\mathbb{R}^{n}_{+}$$. By the odd symmetry of v with respect to $$\partial \mathbb{R}^{n}_{+}$$ we have

\begin{aligned} -r^{n-1}\bar{u}'_{m-1}(r)&= \int ^{r}_{0} s^{n-1}f(s)\,ds= \frac{1}{n\alpha (n)} \int ^{r}_{0} \int _{\partial B_{s}(x_{1})} \vert x \vert ^{- \beta} \vert v \vert ^{q-1}vd\sigma \,ds \\ &=\frac{1}{n\alpha (n)} \int _{B_{r}(x_{1})} \vert x \vert ^{-\beta} \vert v \vert ^{q-1}vdx>0, \end{aligned}
(3.7)

where $$\alpha (n)$$ denotes the surface area of the unit sphere $$\partial B_{1}(0)$$ in $$\mathbb{R}^{n}$$.

By (3.4) and (3.7) we deduce that

\begin{aligned} \bar{u}'_{m-1}(r)< 0 \quad\text{and} \quad \bar{u}_{m-1}(r)\leq \bar{u}_{m-1}(0)=u_{m-1}(x_{1})< 0\quad \forall r\geq 0. \end{aligned}
(3.8)

Then by the second to the last equation in (3.6) we have

\begin{aligned} -\frac{1}{r^{n-1}} \bigl(r^{n-1}\bar{u}'_{m-2}(r) \bigr)'=\bar{u}_{m-1}(r) \leq \bar{u}_{m-1}(0) \equiv -c_{0}< 0 \quad \forall r>0 \end{aligned}

with universal positive constant $$c_{0}$$, that is,

\begin{aligned} \bigl(r^{n-1}\bar{u}'_{m-2}(r) \bigr)'>r^{n-1}c_{0}\quad \forall r>0, \end{aligned}

and hence

\begin{aligned} \bar{u}_{m-2}(r)\geq \bar{u}_{m-2}(0)+\frac{c_{0}}{2n}r^{2}\quad \forall r>0 \end{aligned}
(3.9)

after integrating. So we find a suitably large $$r_{1}>0$$ such that $$\bar{u}_{m-2}(r_{1})>0$$. In view of the definition of the average, there exists $$x_{2}\in (\partial B_{r_{1}}(x_{1})\cap \mathbb{R}^{n}_{+})$$ such that

\begin{aligned} u_{m-2}(x_{2})>0. \end{aligned}
(3.10)

Moreover, we deduce by (3.8) that

\begin{aligned} u_{m-1}(x_{2})< 0. \end{aligned}
(3.11)

Define the second averages of u and v on $$\partial B_{r}(x_{2})$$:

\begin{aligned} &\tilde{u}(r):=\frac{1}{ \vert \partial B_{r}(x_{2}) \vert } \int _{\partial B_{r}(x_{2})} \vert x \vert ^{ \alpha }u(x)\,ds;\qquad \tilde{v}(r):=\frac{1}{ \vert \partial B_{r}(x_{2}) \vert } \int _{\partial B_{r}(x_{2})} \vert x \vert ^{\beta }v(x)\,ds, \\ &\tilde{u}_{i}(r):=\frac{1}{ \vert \partial B_{r}(x_{2}) \vert } \int _{\partial B_{r}(x_{2})}u_{i}(x)\,ds; \qquad\tilde{v}_{i}(r):= \frac{1}{ \vert \partial B_{r}(x_{2}) \vert } \int _{\partial B_{r}(x_{2})}v_{i}(x)\,ds, \end{aligned}

where $$i=2,3,\ldots,m-1$$. By (3.7) and (3.11) we have

\begin{aligned} \tilde{u}_{m-1}(r)\leq \tilde{u}_{m-1}(0)=u_{m-1}(x_{2})< 0,\quad r\geq 0. \end{aligned}

Similarly to (3.9) and (3.11), we have

\begin{aligned} \tilde{u}_{m-2}(r)\geq \tilde{u}_{m-2}(0)+cr^{2}=u_{m-2}(x_{2})+cr^{2}>0,\quad r\geq 0. \end{aligned}

Repeating the same argument to $$u_{m-3}$$, we also obtain the third average on $$\partial B_{r}(x_{3})$$:

\begin{aligned} \check{u}_{m-1}(r)< 0, \qquad\check{u}_{m-2}(r)>0,\qquad \check{u}_{m-3}(r)< 0,\quad r\geq 0. \end{aligned}

By induction we can get the claim (3.5) for the component u.

(ii) Taking the scaling transformations

\begin{aligned} u_{\mu}(x)=\mu ^{\frac{(2m-\alpha -\beta )(q+1)}{pq-1}}u(\mu x), \qquad v_{ \mu}(x)=\mu ^{\frac{(2m-\alpha -\beta )(p+1)}{pq-1}}v(\mu x), \end{aligned}

we find that $$u_{\mu}$$ and $$v_{\mu}$$ are also nonnegative solutions of (3.3). This implies that by repeating step 2 in Part I of [7, Sect. 2] a suitably large $$\mu >0$$ ensures

\begin{aligned} \hat{u}(r)\geq a_{0} (r-1)^{b_{0}},\quad r\in [1,2], \end{aligned}
(3.12)

with $$b_{0}:=p+q+2m+n$$ and $$a_{0}>0$$ sufficiently large.

Next, we treat the component v. Set $$U^{+}=B_{\tau}(x_{m-1})\cap \mathbb{R}_{+}^{N}$$ and $$U^{-}=B_{\tau}(x_{m-1})\cap (\mathbb{R}^{N}\setminus \mathbb{R}_{+}^{N})$$. Let $$\tilde{U}^{-}$$ be the reflection of $$U^{-}$$ with respect to the boundary $$\mathbb{\partial R}_{+}^{N}$$, and let $$U_{\tau}=U^{+}\setminus \tilde{U}^{-}$$. By Jensen’s inequality and the equations for v in (3.6), we derive that for all $$0\leq r\leq 2$$,

\begin{aligned} -\hat{v}_{m-1}(r)={}& \int _{0}^{r}\frac{1}{\tau ^{n-1}} \int _{0}^{\tau }s^{n-1}g(s)\,ds\,d \tau - \hat{v}_{m-1}(0) \\ ={}& \int _{0}^{r}\frac{1}{\tau ^{n-1}} \biggl( \int _{0}^{\tau }s^{n-1} \biggl[ \frac{1}{ \vert \partial B_{r}(x_{m-1}) \vert } \int _{\partial B_{r}(x_{m-1})} \vert x \vert ^{- \alpha} \vert u \vert ^{p-1}u(x)\,d\sigma \biggr] \,ds \biggr)\,d\tau \\ &{}-\hat{v}_{m-1}(0) \\ ={}& \int _{0}^{r}\frac{1}{\tau ^{n-1}} \biggl( \int _{0}^{\tau }s^{n-1} \frac{1}{n\alpha (n)} \int _{\partial B_{r}(x_{m-1})} \vert x \vert ^{-\alpha} \vert u \vert ^{p-1}u(x)\,d \sigma \,ds \biggr)\,d\tau -\hat{v}_{m-1}(0) \\ \geq{}& \frac{1}{n\alpha (n)(2+ \vert x_{m-1} \vert )^{\alpha}} \int _{0}^{r} \frac{1}{\tau ^{n-1}} \biggl( \int _{ B_{\tau}(x_{m-1})} \vert u \vert ^{p-1}u(x)\,dx \biggr) \,d\tau -\hat{v}_{m-1}(0) \\ ={}&\frac{1}{n\alpha (n)(2+ \vert x_{m-1} \vert )^{\alpha}} \int _{0}^{r} \frac{1}{\tau ^{n-1}} \biggl( \int _{ U_{\tau}}u^{p}(x)\,dx \biggr)\,d\tau - \hat{v}_{m-1}(0) \\ \geq{}& \frac{1}{n\alpha (n)(2+ \vert x_{m-1} \vert )^{\alpha}} \int _{0}^{r} \frac{ \vert U_{\tau} \vert }{\tau ^{n-1}} \biggl( \frac{1}{ \vert U_{\tau} \vert } \int _{ U_{ \tau}}u^{p}(x)\,dx \biggr)\,d\tau - \hat{v}_{m-1}(0) \\ \geq{}& \frac{1}{n\alpha (n)(2+ \vert x_{m-1} \vert )^{\alpha}} \int _{0}^{r} \biggl( \frac{ \vert B_{\tau}(x_{m-1}) \vert }{ \vert U_{\tau} \vert } \biggr)^{p-1} \\ &{}\times\frac{1}{\tau ^{n-1} \vert B_{\tau}(x_{m-1}) \vert ^{p-1}} \biggl( \int _{ U_{\tau}}u(x)\,dx \biggr)^{p} \,d\tau - \hat{v}_{m-1}(0) \\ \geq {}&\frac{1}{(n\alpha (n))^{p}(2+ \vert x_{m-1} \vert )^{\alpha}} \int _{0}^{r} \frac{1}{\tau ^{np-1}} \biggl( \int _{ U_{\tau}} \vert x \vert ^{-\alpha}\cdot \vert x \vert ^{ \alpha}u(x)\,dx \biggr)^{p} \,d\tau - \hat{v}_{m-1}(0) \\ \geq{}& \frac{1}{(n\alpha (n))^{p}(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}} \int _{0}^{r} \frac{1}{\tau ^{np-1}} \biggl( \int _{B_{\tau}(x_{m-1})} \vert x \vert ^{\alpha}u(x)\,dx \biggr)^{p} \,d\tau -\hat{v}_{m-1}(0) \\ \geq{}& \frac{1}{(n\alpha (n))^{p}(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}} \int _{0}^{r} \frac{1}{\tau ^{np-1}} \biggl( \int _{0}^{\tau} \int _{\partial B_{s}(x_{m-1})} \vert x \vert ^{ \alpha}u(x)\,dx\,ds \biggr)^{p} \,d\tau \\ &{} -\hat{v}_{m-1}(0) \\ ={}&\frac{1}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}} \int _{0}^{r} \frac{1}{\tau ^{np-1}} \biggl( \int _{0}^{\tau }s^{n-1}\hat{u}(s)\,ds \biggr)^{p} \,d\tau -\hat{v}_{m-1}(0). \end{aligned}
(3.13)

Choosing $$r=2$$ in (3.12) and substituting the latter into (3.13), we get

\begin{aligned} \hat{v}_{m-1}(r)\leq -\frac{a_{0}}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}} r^{2}+ \hat{v}_{m-1}(0). \end{aligned}

Thus $$\hat{v}_{m-1}(r)$$ must be negative whenever $$a_{0}$$ is large. Now we can repeat the above procedure for u with $$m-1$$ times recenters to deduce for the component v that

\begin{aligned} (-1)^{i}\hat{v}_{m-i}(r)>0,\quad r\in [0,2], i=1,2,\ldots,m-1. \end{aligned}
(3.14)

For any $$1\leq r \leq 2$$, we get by (3.12)–(3.14) that

\begin{aligned} -\hat{v}_{m-1}(r)&\geq \frac{1}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}} \int _{0}^{r} \frac{1}{\tau ^{np-1}} \biggl( \int _{0}^{\tau }s^{n-1}\hat{u}(s)\,ds \biggr)^{p} \,d\tau -\hat{v}_{m-1}(0) \\ &\geq \frac{1}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}} \int _{1}^{r} \frac{1}{\tau ^{np-1}} \biggl( \int _{1}^{\tau }s^{n-1}a_{0} (s-1)^{b_{0}}\,ds \biggr)^{p} \,d\tau \\ &\geq \frac{1}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}} \int _{1}^{r} \frac{1}{\tau ^{np-1}} \biggl( \frac{a_{0}}{b_{0}+n}(\tau -1)^{b_{0}+1} \tau ^{n-1} \biggr)^{p} \,d\tau ) \\ &=\frac{1}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}}\cdot \frac{a_{0}^{p}}{(n+b_{0})^{p}} \int _{1}^{r}(\tau -1)^{(b_{0}+1)p} \tau ^{1-p}\,d\tau \\ &\geq \frac{1}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}}\cdot \frac{a_{0}^{p}}{(n+b_{0})^{p}[(b_{0}+1)p+1]}(r-1)^{(b_{0}+1)p+1} r^{1-p} \\ &\geq \frac{1}{(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}}\cdot \frac{a_{0}^{p}}{2^{p-1}(n+b_{0})^{p}[(b_{0}+1)p+1]}(r-1)^{(b_{0}+1)p+1} \\ &\geq \frac{1}{2^{p}(2+ \vert x_{m-1} \vert )^{\alpha (p+1)}}\cdot \frac{a_{0}^{p}}{(n+b_{0})^{p}[(b_{0}+1)p+1]}(r-1)^{(b_{0}+1)p+1}, \end{aligned}
(3.15)

where $$\int _{1}^{r}(s-1)^{\theta}s^{\iota}\,ds\geq \frac{1}{\theta +\iota +1}(r-1)^{\theta +1}r^{\iota}$$, and $$\theta, \iota >0$$ by an elementary calculation. This implies

\begin{aligned} \hat{v}_{m-1}(r)\leq -\frac{(Ma_{0})^{p}}{(2b_{0}p)^{p+1}}(r-1)^{(b_{0}+1)p+1} \end{aligned}

with $$M^{p}:=\frac{1}{2^{p}(2+|x_{m-1}|)^{\alpha (p+1)}}$$. So we get

\begin{aligned} -\frac{1}{r^{n-1}} \bigl(r^{n-1}\hat{v}'_{m-2}(r) \bigr)'=\hat{v}'_{m-1}(r) \leq - \frac{(Ma_{0})^{p}}{(2b_{0}p)^{p+1}}(r-1)^{(b_{0}+1)p+1}< 0. \end{aligned}

Integrating twice from 0 to r and from 0 to τ, by (3.14) we obtain that

\begin{aligned} \hat{v}_{m-2}(r)&\geq \int _{0}^{r}\frac{1}{\tau ^{n-1}} \int _{0}^{ \tau }s^{n-1}\frac{(Ma_{0})^{p}}{(2b_{0}p)^{p+1}}(s-1)^{(b_{0}+1)p+1} \,ds \,d\tau \\ &\geq \frac{(Ma_{0})^{p}}{(2b_{0}p)^{p+1}} \int _{1}^{r} \frac{1}{\tau ^{n-1}}\frac{1}{(b_{0}+1)p+n+1} \tau ^{n-1}(\tau -1)^{(b_{0}+1)p+2} \,d\tau \\ &\geq \frac{(Ma_{0})^{p}}{(2b_{0}p)^{p+1}}\frac{1}{(b_{0}+1)p+n+1} \frac{1}{(b_{0}+1)p+3}(r-1)^{(b_{0}+1)p+3} \\ &\geq \frac{(Ma_{0})^{p}}{(2b_{0}p)^{p+3}}(r-1)^{(b_{0}+1)p+3}. \end{aligned}

We know by repeating step 1 in Part I of [7, Sect. 2] that m is even. Continuing this way, for m even, we obtain

\begin{aligned} \hat{v}(r)&\geq \frac{(Ma_{0})^{p}}{(2b_{0}p)^{p+2m-1}}(r-1)^{(b_{0}+1)p+2m-1} \geq A_{0}(r-1)^{B_{0}} \end{aligned}
(3.16)

with $$A_{0}:=c_{0}(Ma_{0})^{p}(2pb_{0})^{-p-2m}$$ and $$B_{0}:=2pb_{0}\geq (b_{0}+1)p+2m$$.

Similarly to (3.13), by (3.5) we have

\begin{aligned} -\hat{u}_{m-1}(r)&\geq \int _{0}^{r}\frac{1}{\tau ^{n-1}} \int _{0}^{ \tau }s^{n-1}g(s)\,ds- \hat{u}_{m-1}(0) \\ &\geq \frac{1}{(2+ \vert x_{m-1} \vert )^{\beta (q+1)}} \int _{0}^{r} \frac{1}{\tau ^{nq-1}} \biggl( \int _{0}^{\tau }s^{n-1}\hat{v}(s)\,ds \biggr)^{q} \,d\tau. \end{aligned}

Together with (3.16), we obtain

\begin{aligned} \hat{u}_{m-1}(r)\leq -\frac{(M_{0}A_{0})^{q}}{(2B_{0}q)^{q+1}}(r-1)^{(B_{0}+1)q+1} \end{aligned}

with $$M_{0}^{q}:=\frac{1}{2^{q}(2+|x_{m-1}|)^{\beta (q+1)}}$$. By this and (3.6) we obtain

\begin{aligned} \hat{u}(r)\geq \frac{(M_{0}A_{0})^{q}}{(2B_{0}q)^{q+2m}}(r-1)^{(B_{0}+1)q+2m} \geq a_{1}(r-1)^{b_{1}} \end{aligned}

with $$a_{1}:=c_{0}(M_{0}A_{0})^{q}(2B_{0}q)^{-q-2m}$$ and $$b_{1}:=2B_{0}q$$. Then

\begin{aligned} b_{1}\geq (B_{0}+1)q+2m. \end{aligned}
(3.17)

By induction we can obtain by k steps that $$\hat{u}(r)\geq a_{k}(r-1)^{b_{k}}$$ with $$b_{k+1}=4pqb_{k}$$, $$a_{k+1}= \frac{M^{pq}M_{k}^{q} a_{k}^{pq}}{(2pb_{k})^{q(p+2m)+(q+2m)}(2q)^{q+2m}}$$, $$k=0,1,\dots$$.

Set $$h:=\max \{q+2m,p+2m\}$$ and $$M_{k}^{q}:=\min \{\frac{1}{2^{p}(2+|x_{m-1}|)^{\alpha (p+1)}}, \frac{1}{2^{q}(2+|x_{m-1}|)^{\beta (q+1)}}\}$$. Choose z such that $$z\geq \frac{h(q+1)+1}{pq-1}$$ and thus $$b_{0}\geq c(4pq)^{pq(q+1)+z}$$ with

\begin{aligned} c:=2^{pq+q+2(q+2m)+q(p+2m)}\bigl(2+ \vert x_{m-1} \vert \bigr)^{\beta (q+1)+\alpha q(p+1)}p^{q(p+2m)+(q+2m)}q^{q+2m}>0. \end{aligned}

We claim that

\begin{aligned} a_{k}^{pq}\geq cb_{k}^{h(q+1)+z},\quad k=0,1, \dots. \end{aligned}
(3.18)

Obviously, (3.18) holds for $$k=0$$ by choosing $$a_{0}$$ sufficiently large. Assume that (3.18) is true for k. We have

\begin{aligned} \frac{a_{k+1}^{pq}}{cb_{k+1}^{h(q+1)+z}}&=\biggl[ \frac{a_{k}^{pq}}{b_{k}^{h(q+1)}}\biggr]^{pq} \frac{1}{c(4pqb_{k})^{h(q+1)+z}} \\ &\geq \frac{b_{k}^{z(pq-1)-h(q+1)}}{c(4pq)^{pq(q+1)+z}} \\ &\geq \frac{b_{0}}{c(4pq)^{pq(q+1)+z}}. \end{aligned}

Therefore (3.18) holds for all integer k.

Now choose $$r=2$$. Then (3.17) and (3.18) yield a contradiction that

\begin{aligned} \hat{u}(2)\geq a_{k}\geq cb_{k}^{(pq(q+1)+z)/(pq)} \rightarrow \infty\quad \text{as } k\rightarrow \infty. \end{aligned}

This excludes (3.4).

Step 2. We furthermore claim that $$u_{m-1}(x)>0$$.

Otherwise, there exists $$x_{1}\in \mathbb{R}^{n}_{+}$$ such that $$u_{m-1}(x_{1})=0$$. Thus $$-\Delta u_{m-1}(x_{1})\leq 0$$, since $$x_{1}$$ is a local minimum of $$u_{m-1}(x)$$. This contradicts $$-\Delta u_{m-1}(x)=u^{p}>0$$, $$x\in \mathbb{R}_{+}^{n}$$.

Similarly, we can get $$v_{m-1}(x)>0$$ for $$\mathbb{R}^{n}_{+}$$ by similar arguments in Steps 1 and 2 for $$u_{m-1}(x)$$.

Step 3. We show that $$u_{m-i}(x), v_{m-i}(x)>0$$, $$x\in \mathbb{R}^{n}_{+}$$ for $$i=2,3,\dots,m-1$$.

Based on the positivity of $$v_{m-1}(x)$$, we first show that $$u_{m-i}(x)>0$$, $$x\in \mathbb{R}^{n}_{+}$$, for $$i=2,3,\ldots,m-1$$. Otherwise, there exists $$x_{0}\in \mathbb{R}^{n}_{+}$$ with $$i\in \{2,3,\ldots,m-1\}$$ such that

\begin{aligned} &u_{m-1}(x)>0, u_{m-2}(x)>0,\ldots, u_{m-i+1}(x)>0, \quad x \in \mathbb{R}^{n}_{+}, \\ &u_{m-i}(x_{0})< 0. \end{aligned}
(3.19)

(i) Assume that $$m-i$$ is even. Then $$\bar{u}_{m-i}(r)$$ for $$x\in \mathbb{R}^{n}$$ satisfies that

\begin{aligned} \textstyle\begin{cases} -\Delta \bar{u}=\bar{u}_{1}, \\ \cdots \\ -\Delta \bar{u}_{m-i-1}=\bar{u}_{m-i}, \\ -\Delta \bar{u}_{m-i}=\bar{u}_{m-i+1}. \end{cases}\displaystyle \end{aligned}
(3.20)

Integrating the last equation in (3.20), we arrive at

\begin{aligned} -r^{n-1}\bar{u}'_{m-i}(r)&= \int ^{r}_{0} s^{n-1} \bar{u}_{m-i+1}(s)\,ds= \frac{1}{n\alpha (n)} \int ^{r}_{0} \int _{\partial B_{s}(x_{0})}u_{m-i+1}(s)\,d \sigma \,ds \\ &=\frac{1}{n\alpha (n)} \int _{B_{r}(x_{0})}u_{m-i+1}(s)\,ds>0. \end{aligned}

Here we have used the odd symmetry of $$u_{m-i+1}(x)$$ with respect to $$\partial \mathbb{R}^{n}_{+}$$ and the fact that more than half of $$B_{r}(x_{0})$$ is contained in $$\mathbb{R}^{n}_{+}$$. Together with (3.19), we deduce

\begin{aligned} \bar{u}'_{m-i}(r)< 0 \quad\text{and} \quad \bar{u}_{m-i}(r)\leq \bar{u}_{m-i}(0)=u_{m-i}(x_{0})< 0,\quad r \geq 0. \end{aligned}

Then by the second to the last equation in (3.20) we have

\begin{aligned} -\frac{1}{r^{n-1}} \bigl(r^{n-1}\bar{u}'_{m-i-1}(r) \bigr)'=\bar{u}_{m-i}(r) \leq \bar{u}_{m-i}(0) \equiv -c_{0}< 0,\quad r>0. \end{aligned}

This yields

\begin{aligned} \bar{u}'_{m-i-1}(r)>\frac{c_{0}}{n}r, \end{aligned}

and hence

\begin{aligned} \bar{u}_{m-i-1}(r)\geq \bar{u}_{m-i-1}(0)+\frac{c_{0}}{2n}r^{2} \geq \frac{c_{0}}{2n}r^{2}+c_{1},\quad r>0. \end{aligned}

Continuing this way with $$m-i$$ even, we derive that

\begin{aligned} \bar{u}(r)\leq -c_{0}r^{2(m-i)}+\sum ^{m-i}_{j=1}c_{j}r^{2(m-i-j)},\quad r>0. \end{aligned}
(3.21)

This yields a contraction that

\begin{aligned} 0&< \frac{1}{n\alpha (n)} \int _{B_{r}(x_{0})} \vert x \vert ^{\alpha}u(x)\,dx= \int _{0}^{r} s^{n-1}\bar{u}(s)\,ds \\ &\leq -c_{0}r^{n+2(m-i)}+\sum^{m-i}_{j=1}c_{j} r^{n+2(m-i-j)} \\ &< 0 \quad\text{as } r \text{ is sufficiently large.} \end{aligned}

(ii) Assume that $$m-i$$ is odd.

Similarly to (3.21) with $$m-i$$ odd, by (3.5) we deduce

\begin{aligned} \hat{u}(r)\geq c_{0}r^{2(m-i)},\quad r>0. \end{aligned}
(3.22)

By parallel arguments for (3.13), by (3.22) we have

\begin{aligned} -r^{n-1}\hat{v}'_{m-1}(r)&=\frac{1}{n\alpha (n)} \int _{B_{r}(x_{m-1})} \vert x \vert ^{- \alpha} \vert u \vert ^{p-1}u(x)\,dx \\ &\geq \frac{c}{r^{np-n}}\biggl( \int ^{r}_{0} s^{n-1}\hat{u}(s)\,ds \biggr)^{p} \\ &\geq cr^{n+2p(m-i)} \end{aligned}

with $$c:=\frac{c_{0}}{(1+|x_{m-1}|)^{\alpha (p+1)}}$$ or, equivalently, $$\hat{v}'_{m-1}(r)\leq -cr^{2p(m-i)+1}$$, and, consequently,

\begin{aligned} \hat{v}_{m-1}(r)\leq -c_{0}r^{2p(m-i)+2}+ \hat{v}_{m-1}(0). \end{aligned}

Combining this with $$v_{m-1}(x)>0$$ for $$x\in \mathbb{R}^{n}_{+}$$, we obtain by (3.14) a contradiction that

\begin{aligned} 0&< \frac{1}{n\alpha (n)} \int _{B_{r}(x_{m-1})}v_{m-1}(x)\,dx= \int ^{r}_{0}s^{n-1} \hat{v}_{m-1}(s)\,ds \\ &\leq -c_{0}r^{n+2p(m-i)+2}\leq 0. \end{aligned}

Combining (i) and (ii), we exclude (3.19).

Similarly, we can prove that $$v_{m-i}(x)>0$$, $$i=2,3,\ldots,m-1$$, by $$u_{m-1}(x)>0$$ for $$\mathbb{R}^{n}_{+}$$. □

Proof of Proposition 2

First, we show that the classical solutions of (1.2) must solve (1.1).

When $$x\in \partial R_{+}^{n}$$, we have

\begin{aligned} \Delta ^{j}\biggl(\frac{1}{ \vert x-y \vert ^{n-2m}}\biggr)=\Delta ^{j} \biggl( \frac{1}{ \vert \bar{x}-y \vert ^{n-2m}}\biggr) \end{aligned}

due to $$x=\bar{x}$$, and thus by system (1.2) that $$\Delta ^{j}u(x)=0$$, $$j=0,1,\ldots,m-1$$. For $$x\in R_{+}^{n}$$, we have

\begin{aligned} (-\Delta )^{m}\bigl( \vert x \vert ^{\alpha}u(x)\bigr)&= \int _{\mathbb{R}^{n}_{+}}(- \Delta )^{m}\biggl(\frac{1}{ \vert x-y \vert ^{n-2m}}- \frac{1}{ \vert \bar{x}-y \vert ^{n-2m}}\biggr) \vert y \vert ^{- \beta}v^{q}(y)\,dy \\ &=C \int _{\mathbb{R}^{n}_{+}}\delta (x-y) \vert y \vert ^{-\beta}v^{q}(y) \,dy \\ &=C \vert x \vert ^{-\beta}v^{q}(x) . \end{aligned}

Similarly, $$\Delta ^{j}v(x)=0$$, $$j=0,1,\ldots,m-1$$, on $$\partial \mathbb{R}_{+}^{n}$$ and $$(-\Delta )^{m}(|x|^{\beta}v(x))=C|x|^{-\alpha}u^{p}(x)$$ in $$\mathbb{R}_{+}^{n}$$.

Next, we should prove that if $$(u,v)$$ is a smooth positive solution of (1.1) with $$p, q\ge 1$$ and $$\alpha, \beta >0$$, then a constant multiple of $$(u,v)$$ satisfies (1.2).

Rewrite the higher-order PDEs problem (1.1) as the following second-order system:

\begin{aligned} \textstyle\begin{cases} -\Delta u_{i}=u_{i+1}, u_{i} |_{\partial \mathbb{R}_{+}^{n}}=0,\\ \quad i=0,1, \ldots,m-1, \text{ with } u_{0}= \vert x \vert ^{\alpha }u, u_{m}= \vert x \vert ^{-\beta} v^{q}, \\ -\Delta v_{i}=v_{i+1}, v_{i} |_{\partial \mathbb{R}_{+}^{n}}=0,\\ \quad i=0,1, \ldots,m-1, \text{ with } v_{0}= \vert x \vert ^{\beta} v, v_{m}= \vert x \vert ^{-\alpha} u^{p}. \end{cases}\displaystyle \end{aligned}
(3.23)

On the other hand, rewrite the integral system (1.3) as

\begin{aligned} \textstyle\begin{cases} u_{i}=\int _{\mathbb{R}^{n}_{+}}G(x,y,2)u_{i+1}(y)\,dy,\\ \quad i=0,1,\ldots,m-1, \text{ with } u_{0}= \vert x \vert ^{\alpha }u, u_{m}= \vert x \vert ^{-\beta} v^{q}, \\ v_{i}=\int _{\mathbb{R}^{n}_{+}}G(x,y,2)v_{i+1}(y)\,dy, \\ \quad i=0,1,\ldots,m-1, \text{ with } v_{0}= \vert x \vert ^{\beta} v, v_{m}= \vert x \vert ^{-\alpha} u^{p}. \end{cases}\displaystyle \end{aligned}
(3.24)

By [1, Theorem 2.4] we know that

\begin{aligned} G(x,y,2j+2)= \int _{\mathbb{R}^{n}_{+}}G(x,y,2j)G(y,z,2)\,dz,\quad j=1, \ldots,m-1. \end{aligned}

This yields the equivalence between the integral systems (1.2) and (3.24).

Now let $$(u_{0},\ldots,u_{m-1},v_{0},\ldots,v_{m-1})$$ be a positive classical solution of (3.23). It suffices to show that $$(u_{0},\ldots,u_{m-1},v_{0},\ldots,v_{m-1})$$ does satisfy (3.24).

Let $$(u_{0},\ldots,u_{m-1},v_{0},\ldots,v_{m-1})$$ be a positive solution of (3.23). Multiply (3.23) by $$\tilde{G}_{R}(x,y)$$ and then integrate over $$B_{R}^{+}(0)$$ to get by (3.1) that

\begin{aligned} \textstyle\begin{cases} \int _{B^{+}_{R}}\tilde{G}_{R}(x,y)u_{i+1}(y)\,dy=u_{i}(x)+\int _{ \widehat{\Gamma}_{R}\cup \bar{\Gamma}_{R}}u_{i}(y) \frac{\partial \tilde{G}_{R}}{\partial \nu}(x,y)\,ds -\int _{ \widehat{\Gamma}_{R}\cup \bar{\Gamma}_{R}}\tilde{G}_{R} \frac{\partial u_{i}(y)}{\partial \nu}(x,y)\,ds, \\ \int _{B^{+}_{R}}\tilde{G}_{R}(x,y)v_{i+1}(y)\,dy=v_{i}(x)+\int _{ \widehat{\Gamma}_{R}\cup \bar{\Gamma}_{R}}v_{i}(y) \frac{\partial \tilde{G}_{R}}{\partial \nu}(x,y)\,ds -\int _{ \widehat{\Gamma}_{R}\cup \bar{\Gamma}_{R}}\tilde{G}_{R}(x,y) \frac{\partial v_{i}(y)}{\partial \nu}\,ds, \end{cases}\displaystyle \end{aligned}

which $$i=0,1,\ldots,m-1$$. By $$\frac{\partial \tilde{G}_{R}}{\partial \nu}|_{\bar{\Gamma}_{R}}=0$$ and $$\tilde{G}_{R}|_{\widehat{\Gamma}_{R}\cup \bar{\Gamma}_{R}}=0$$ we have

\begin{aligned} \textstyle\begin{cases} \int _{B^{+}_{R}}\tilde{G}_{R}(x,y)u_{i+1}(y)\,dy=u_{i}(x)+\int _{ \widehat{\Gamma}_{R}}u_{i}(y) \frac{\partial \tilde{G}_{R}}{\partial \nu}(x,y)\,ds,\quad i=0,1,\ldots,m-1, \\ \int _{B^{+}_{R}}\tilde{G}_{R}(x,y)v_{i+1}(y)\,dy=v_{i}(x)+\int _{ \widehat{\Gamma}_{R}}v_{i}(y) \frac{\partial \tilde{G}_{R}}{\partial \nu}(x,y)\,ds,\quad i=0,1,\ldots,m-1, \end{cases}\displaystyle \end{aligned}
(3.25)

which implies by Lemma 3.2 that

\begin{aligned} \int _{B^{+}_{R}}\tilde{G}_{R}(x,y)u_{i+1}(y) \,dy\leq u_{i}(x), \int _{B^{+}_{R}} \tilde{G}_{R}(x,y)v_{i+1}(y) \,dy\leq v_{i}(x),\quad i=0,1,\ldots,m-1. \end{aligned}

Letting $$R\rightarrow \infty$$, we deduce with (3.2) that

\begin{aligned} \int _{\mathbb{R}^{n}_{+}}G(x,y,2)u_{i+1}(y)\,dy< +\infty, \int _{ \mathbb{R}^{n}_{+}}G(x,y,2)v_{i+1}(y)\,dy< +\infty, \quad i=0,1, \dots,m-1, \end{aligned}

and hence there exists a sequence $$R_{k}\rightarrow \infty$$ such that

\begin{aligned} &\int _{\widehat{\Gamma}_{R_{k}}}G(x,y,2)u_{i+1}(y)\,ds\rightarrow 0, \int _{\widehat{\Gamma}_{R_{k}}}G(x,y,2)v_{i+1}(y)\,ds\rightarrow 0,\\ &\quad \text{as } k\rightarrow \infty, i=0,1,\dots,m-2. \end{aligned}

For fixed $$x\in B^{+}_{R}(0)$$, we have

\begin{aligned} G(x,y,2)|_{y\in \widehat{\Gamma}_{R}}= \biggl(\frac{1}{ \vert x-y \vert ^{n-2}}- \frac{1}{ \vert \bar{x}-y \vert ^{n-2}} \biggr)|_{y\in \widehat{\Gamma}_{R}}=O\biggl( \frac{y_{n}}{R^{n}}\biggr),\quad R\rightarrow \infty, \end{aligned}
(3.26)

and thus

\begin{aligned} \frac{\partial \tilde{G}_{R}}{\partial \nu}(x,y)=O\biggl( \frac{y_{n}}{R^{n+1}}\biggr),\quad R\rightarrow \infty. \end{aligned}
(3.27)

By (3.26) we derive

\begin{aligned} \begin{aligned} &\frac{1}{R_{k}^{n}} \int _{\widehat{\Gamma}_{R}}y_{n} u_{i+1}(y)\,ds \rightarrow 0, \\ &\frac{1}{R_{k}^{n}} \int _{\widehat{\Gamma}_{R}}y_{n} v_{i+1}(y)\,ds \rightarrow 0 \quad\text{as } k\rightarrow \infty, i=0,1,\ldots,m-2. \end{aligned} \end{aligned}
(3.28)

Similarly, there exists a sequence of $$\{R_{k}\}$$ such that

\begin{aligned} \frac{1}{R_{k}^{n+\beta}} \int _{\widehat{\Gamma}_{R}}y_{n} v^{q}(y)\,ds \rightarrow 0, \qquad\frac{1}{R_{k}^{n+\alpha}} \int _{\widehat{\Gamma}_{R}}y_{n} u^{p}(y)\,ds\rightarrow 0 \quad\text{as } k\rightarrow \infty. \end{aligned}
(3.29)

To show that the boundary terms in (3.25) approach 0 as $$R\rightarrow \infty$$, by (3.27) we only need to derive that there exists a sequence $$R_{k}\rightarrow \infty$$ such that

\begin{aligned} \begin{aligned} &\frac{1}{R_{k}^{n+1}} \int _{\widehat{\Gamma}_{R}}y_{n} u_{i+1}(y)\,ds \rightarrow 0, \\ &\frac{1}{R_{k}^{n+1}} \int _{\widehat{\Gamma}_{R}}y_{n} v_{i+1}(y)\,ds\rightarrow 0, \quad k\rightarrow \infty, i=0,1,\dots,m-2, \end{aligned} \end{aligned}
(3.30)

and

\begin{aligned} \frac{1}{R_{k}^{n+\alpha +1}} \int _{\widehat{\Gamma}_{R}}y_{n}u(y)\,ds \rightarrow 0,\qquad \frac{1}{R_{k}^{n+\beta +1}} \int _{\widehat{\Gamma}_{R}}y_{n}v(y)\,ds \rightarrow 0 \quad \text{as } k\rightarrow \infty. \end{aligned}
(3.31)

Obviously, (3.30) is a direct consequence of (3.28).

By Jensen’s inequality with $$p\ge 1$$ and (3.29) we have

\begin{aligned} \biggl(\frac{1}{R_{k}^{n+\alpha}} \int _{\widehat{\Gamma}_{R_{k}}}y_{n}^{ \frac{1}{p}}u(y)\,ds \biggr)^{p}&\leq \frac{1}{R_{k}^{\alpha p+n}} \int _{ \widehat{\Gamma}_{R_{k}}}{y_{n}}u^{p}(y)\,ds \\ &\leq \frac{1}{R_{k}^{\alpha +n}} \int _{\widehat{\Gamma}_{R_{k}}}{y_{n}}u^{p}(y)\,ds \rightarrow 0 \quad\text{as } R_{k}\rightarrow \infty, \end{aligned}

and hence

\begin{aligned} \frac{1}{R_{k}^{n+\alpha}} \int _{\widehat{\Gamma}_{R_{k}}}y_{n}^{ \frac{1}{p}}u(y)\,ds\rightarrow 0,\quad R_{k}\rightarrow \infty. \end{aligned}
(3.32)

Denote $$\widehat{\Gamma}^{1}_{R_{k}}=\{y\in \widehat{\Gamma}_{R_{k}}, y_{n} \leq 1\}$$ and $$\widehat{\Gamma}^{2}_{R_{k}}=\{y\in \widehat{\Gamma}_{R_{k}}, y_{n}>1 \}$$. Then

\begin{aligned} \int _{\widehat{\Gamma}_{R_{k}}}\frac{y_{n}}{R^{n+\alpha +1}_{k}}u(y)\,ds &= \int _{\widehat{\Gamma}^{1}_{R_{k}}} \frac{y_{n}}{R^{n+\alpha +1}_{k}}u(y)\,ds + \int _{\widehat{\Gamma}^{2}_{R_{k}}} \frac{y_{n}}{R^{n+\alpha +1}_{k}}u(y)\,ds \\ &\leq \int _{\widehat{\Gamma}^{1}_{R_{k}}} \frac{y_{n}^{\frac{1}{p}}}{R^{n+\alpha +1}_{k}}u(y)\,ds + \int _{ \widehat{\Gamma}^{2}_{R_{k}}}\frac{1}{R^{n+\alpha}_{k}}u(y)\,ds \\ &\leq \frac{1}{R_{k}} \int _{\widehat{\Gamma}^{1}_{R_{k}}} \frac{y_{n}^{\frac{1}{p}}}{R^{n+\alpha}_{k}}u(y)\,ds + \int _{ \widehat{\Gamma}^{2}_{R_{k}}} \frac{y_{n}^{\frac{1}{p}}}{R^{n+\alpha}_{k}}u(y)\,ds \\ &\leq \frac{2}{R_{k}^{n+\alpha}} \int _{\widehat{\Gamma}_{R_{k}}}y_{n}^{ \frac{1}{p}}u(y)\,ds, \end{aligned}

which vanishes as $$R_{k}\rightarrow \infty$$ by (3.32). This proves (3.31) for u. The argument for v is similar.

Substituting (3.30) and (3.31) into (3.25), we arrive at (3.24). □

Data Availability

No datasets were generated or analysed during the current study.

References

1. Cao, L., Chen, W.: Liouville type theorems for poly-harmonic Navier problems. Discrete Contin. Dyn. Syst. 33, 3937–3955 (2013)

2. Cao, L., Dai, Z.: A Liouville-type theorem for an integral equation on a half-space $$\mathbb{R}^{n}_{+}$$. J. Math. Anal. Appl. 389, 1365–1373 (2012)

3. Cao, L., Dai, Z.: A Liouville-type theorem for an integral system on a half-space $$\mathbb{R}^{n}_{+}$$. J. Inequal. Appl. 37, 1–9 (2013)

4. Cao, L., Dai, Z.: Symmetry and nonexistence of positive solutions for weighted HLS system of integral equations in a half space. Abstr. Appl. Anal. 2014, 593210 (2014)

5. Chen, W., Li, C., Ou, B.: Classification of solutions for an integral equation. Commun. Pure Appl. Math. 59, 330–343 (2006)

6. Chen, W., Jin, C., Li, C.: Weighted Hardy–Littlewood–oblev inequalities and system of integral equations. Discrete Contin. Dyn. Syst. 2005, 164–172 (2005)

7. Chen, W., Fang, Y., Li, C.: Super poly-harmonic property of solutions for Navier boundary problems on a half space. J. Funct. Anal. 265, 1522–1555 (2013)

8. Ma, L., Chen, D.: A Liouville type theorem for an integral system. Commun. Pure Appl. Anal. 5, 855–859 (2006)

9. Jin, C., Li, C.: Quantitative analysis of some system of integral equations. Calc. Var. Partial Differ. Equ. 59, 330–343 (2006)

10. Jin, C., Li, C.: Symmetry of solutions to some integral equations. Proc. Am. Math. Soc. 134, 1661–1670 (2005)

11. Lei, Y., Lü, Z.: Axisymmetry of locally bounded solutions to an Euler–Lagrange system of the weighted Hardy–Littlewood–Sobolev inequalities. Discrete Contin. Dyn. Syst. 33, 1987–2005 (2013)

12. Li, D., Niu, P., Zhuo, R.: Symmetry and non-existence of positive solutions for PDE system with Navier boundary conditions on a half space. Complex Var. Elliptic Equ. 5, 1436–1450 (2013)

13. Stein, E., Weiss, G.: Fractional integrals in n-dimensional Euclidean space. J. Math. Mech. 7, 503–514 (1958)

14. Zhuo, R., Li, D.: A system of integral equations on half space. J. Math. Anal. Appl. 381, 392–401 (2011)

15. Zhao, W., Yang, J., Zheng, S.: Liouville type theorem for higher-order elliptic system with Navier boundary condition. Nonlinear Differ. Equ. Appl. 22, 311–324 (2015)

16. Zhao, W., Yang, J., Zheng, S.: Liouville type theorem to an integral system in the half-space. Commun. Pure Appl. Anal. 2, 511–525 (2014)

17. Zhuo, R., Li, C.: Classification of anti-symmetric solutions to nonlinear fractional Laplace equations. Calc. Var. Partial Differ. Equ. 61, 17 (2022)

18. Li, C., Zhuo, R.: Classification of anti-symmetric solutions to the fractional Lane–Emden system. Sci. China Math. 61, 723–744 (2022)

19. Dai, W., Qin, G.: Liouville type theorems for fractional and higher order Hénon–Hardy type equations via the method of scaling spheres. Int. Math. Res. Not. 11, 9001–9070 (2023)

20. Dai, W., Qin, G., Zhang, Y.: Liouville type theorem for higher order Hénon equations on a half space. Nonlinear Anal. 183, 284–302 (2019)

21. Cheng, Z., Huang, G.: A Liouville theorem for the subcritical Lane–Emden system. Discrete Contin. Dyn. Syst. 39, 1359–1377 (2019)

22. Leite, E., Montenegro, M.: On positive viscosity solutions of fractional Lane–Emden systems. Topol. Methods Nonlinear Anal. 53, 407–425 (2019)

23. Leite, E., Montenegro, M.: Curves to nonlocal Lane–Emden systems and related maximum principles. Calc. Var. Partial Differ. Equ. 59, 118 (2020)

24. Qin, D., Tang, X., Zhang, J.: Ground states for planar Hamiltonian elliptic systems with critical exponential growth. J. Differ. Equ. 308, 130–159 (2022)

25. Zhang, J., Zhang, W.: Semiclassical states for coupled nonlinear Schrödinger system with competing potentials. J. Geom. Anal. 32, 114 (2022)

26. Li, Q., Nie, J., Zhang, W.: Existence and asymptotics of normalized ground states for a Sobolev critical Kirchhoff equation. J. Geom. Anal. 33, 126 (2023)

27. Li, Q., Zou, W.: Normalized ground states for Sobolev critical nonlinear Schrödinger equation in the L2-supercritical case. Discrete Contin. Dyn. Syst. 44, 205–227 (2024)

28. Papageorgiou, N.S., Zhang, J., Zhang, W.: Solutions with sign information for noncoercive double phase equations. J. Geom. Anal. 34, 14 (2024)

Acknowledgements

The authors thank a lot to the editor and reviewer for valuable suggestions, which helped us improve the quality of the paper.

Funding

This work is supported by the National Natural Science Foundation of China (No. 62262012), the Hainan Provincial Natural Science Foundation of China and the Foundation of Hainan University (No. KYQD22094, No. KYQD23050).

Author information

Authors

Contributions

Weiwei Zhao proposed the idea of this paper and performed all the steps of the proofs. Changhui Hu and Xiaoling Shao wrote the main manuscript text. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Changhui Hu.

Ethics declarations

Not applicable.

Competing interests

The authors declare no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

Zhao, W., Shao, X., Hu, C. et al. Nonexistence of positive solutions for the weighted higher-order elliptic system with Navier boundary condition. Bound Value Probl 2024, 22 (2024). https://doi.org/10.1186/s13661-024-01831-9