Properties of solutions to fractional p-subLaplace equations on the Heisenberg group

Abstract

The aim of this paper is to study properties of solutions to the fractional p-subLaplace equations on the Heisenberg group. Based on the maximum principles and the generalization of the direct method of moving planes, we obtain the symmetry and monotonicity of the solutions on the whole group and the Liouville property of solutions on a half space.

Introduction

Let

$$ (-{{\Delta }_{\mathbb{H}}})_{p}^{s}u(\xi )={{C}_{Q,s,p}}\mathit{PV} \int _{{{ \mathbb{H}}^{n}}} \frac{{{ \vert u(\xi )-u(\eta ) \vert }^{p-2}}(u(\xi )-u(\eta ))}{ \vert \eta ^{-1}\circ \xi \vert _{\mathbb{H}}^{Q+sp}}\,d \eta $$

(1.1)

be the fractional p-subLaplacian on the Heisenberg group ${{\mathbb{H}}^{n}}$, where $0< s <1$, $Q=2n+2$, ${{C}_{Q,s}}$ is a positive constant, and PV is the Cauchy principal value. In this paper we study the properties of cylindrical solutions to the fractional p-subLaplace equation

$$ (-{{\Delta }_{\mathbb{H}}})_{p}^{s}u(\xi )=f \bigl(u(\xi ) \bigr), $$

(1.2)

where $2\leq p< \infty$.

Recall that the fractional Laplacian in ${{\mathbb{R}}^{n}}$ is a nonlocal pseudodifferential operator defined by

$$ {{(-\Delta )}^{\alpha }}u(x)={{C}_{n,\alpha }} \lim _{\varepsilon \to 0} \int _{{{\mathbb{R}}^{n}} \setminus {{B}_{\varepsilon }}(x)}{ \frac{u(x)-u(y)}{{{ \vert x-y \vert }^{n+2\alpha }}}}\,dy, $$

(1.3)

where $0<\alpha <1$, ${{C}_{n,\alpha }}$ is a constant, and u belongs to the Schwartz space. Since the nonlocal property of the operator ${(-\Delta )}^{\alpha }$ brings new difficulties to investigate, Caffarelli and Silvestre in [4] developed the extension method which can reduce the nonlocal problem relating to ${(-\Delta )}^{\alpha }$ to a local one in higher dimensions. This method has been applied to deal with equations involving the fractional Laplacian, and fruitful results have been obtained, see [3] and the references therein. Chen et al. [7] developed a direct method of moving planes to handle the problem involving ${(-\Delta )}^{\alpha }$ for $0<\alpha <1$, and this direct method has been used successfully to study symmetry, monotonicity, and nonexistence for many fractional Laplace equations, see [6, 7] and the references therein. Recently, Chen and Li [6] considered the fractional p-Laplacian

$$ (-\Delta )_{p}^{\alpha }u(x)={{C}_{n,\alpha ,p}} \mathit{PV} \int _{{{\mathbb{R}}^{n}}}{ \frac{{{ \vert u(x)-u(y) \vert }^{p-2}}(u(x)-u(y))}{{{ \vert x-y \vert }^{n+\alpha p }}}}\,dy $$

(1.4)

and obtained the radial symmetry and monotonicity of solutions to the equations involving operator (1.4).

To the elliptic equation

$$ -\Delta u=g(u) $$

(1.5)

in ${{\mathbb{R}}^{n}}$, Li and Ni [19] proved that the positive solutions to (1.5) are radially symmetric with the assumptions that the limit of u is zero at the infinity and ${g}'\le 0$ if u is sufficiently small. Under the same conditions, the authors in [6] extended the result in [19] to the fractional p-Laplace equation

$$ (-\Delta )_{p}^{\alpha }u=g(u), $$

(1.6)

and got the radial symmetry and monotonicity of the solutions. They also pointed out that the fractional p-Laplacian becomes p-Laplacian as $\alpha \rightarrow 1$ and, furthermore, it reduces to −Δ when $p=2$.

There are many interesting results about subLaplace and p-subLaplace equations on the Heisenberg group (see [13, 15, 17, 18] and [10, 11, 20, 21, 25–27]). There have been several different definitions of the fractional power subLaplacian in ${{\mathbb{H}}^{n}}$ (see [12, 14, 22] etc.). The definition of fractional power subLaplacian given by Roncal and Thangavelu in [22] is indeed a generalization of the definition given by Cowling and Haagerup in [9] about the heat semigroup. The fractional power subLaplace equations can also be studied by generalizing the extension method in [4] to ${{\mathbb{H}}^{n}}$, although the fractional power subLaplacian $(-{{\Delta }_{\mathbb{H}}})^{s}$ ($0< s<1$) does not have the concrete integral expression, for example, see [14] and [8] for $s=\frac{1}{2}$. There are also some results of the fractional power subLaplacian which are the extension of [8], see [23, 24]. Note that the expression of fractional power subLaplacian on ${{\mathbb{H}}^{n}}$ (see [22])

$$ (-{{\Delta }_{\mathbb{H}}})^{s}u(\xi )={{C}_{Q,s}}\mathit{PV} \int _{{{ \mathbb{H}}^{n}}}{ \frac{u(\xi )-u(\eta )}{{{ \vert {{\eta }^{-1}}\circ \xi \vert }_{\mathbb{H}}}^{Q+2s}}}\,d \eta $$

(1.7)

is the special form of fractional p-subLaplacian (1.1). By extending the method of moving planes in [5–7] to ${{\mathbb{H}}^{n}}$, in this paper, we study the properties of the solutions to (1.2) on ${{\mathbb{H}}^{n}}$ and $\mathbb{H}_{+}^{n}=\{\xi \in {{\mathbb{H}}^{n}}\mid t>0\}$.

Our main results are the following.

Theorem 1.1

Let$0< s<1$, $2\leq p<\infty$, and$u\in {{L}_{sp}}({{\mathbb{H}}^{n}})\cap C_{\mathrm{loc}}^{1,1}({{\mathbb{H}}^{n}})$be a nonnegative cylindrical solution to (1.2) with

$$ \lim_{{{ \vert \xi \vert }_{\mathbb{H}}}\to \infty } u(\xi )=0, $$

(1.8)

and suppose that${f}'(a)$is nonpositive and locally bounded forasufficiently small. Thenumust be symmetric and monotone with respect totabout some point in${{\mathbb{H}}^{n}}$.

Theorem 1.2

Let$0< s<1$, $2\leq p<\infty$, and$u\in {{L}_{sp}}({{\mathbb{H}}_{+}^{n}})\cap C_{\mathrm{loc}}^{1,1}({{ \mathbb{H}}_{+}^{n}})$be a nonnegative cylindrical solution to the problem

$$ \textstyle\begin{cases} (-{{\Delta }_{\mathbb{H}}})_{p}^{s}u(\xi )=f(u(\xi )),\quad \xi \in \mathbb{H}_{+}^{n}, \\ u(\xi )=0,\quad \xi \notin \mathbb{H}_{+}^{n}, \end{cases} $$

(1.9)

and suppose thatusatisfies (1.8) and is lower semicontinuous on$\bar{\mathbb{H}}_{+}^{n}$. If$f(0)=0$, ${f}'(a)$is nonpositive and locally bounded forasufficiently small, then$u\equiv 0$.

Observe that Theorem 1.1 is the extension of symmetry and monotonicity of solutions to the fractional p-Laplace equation on ${{\mathbb{R}}^{n}}$ in [6] to the Heisenberg group, and Theorem 1.2 is the Liouville property on a half space in ${{\mathbb{H}}^{n}}$. When $f(a)=-a+a^{q}$ ($q>1$), our results still hold.

The authors in [22] assumed that $u\in C_{0}^{\infty }({{\mathbb{H}}^{n}})$ in (1.7). We point out that (1.1) is also well defined for $u\in {{L}_{sp}}({{\mathbb{H}}^{n}})\cap C_{\mathrm{loc}}^{1,1}({{\mathbb{H}}^{n}})$, where

$$ {{L}_{sp}} \bigl({{\mathbb{H}}^{n}} \bigr)= \biggl\{ u:{{ \mathbb{H}}^{n}}\to \mathbb{R}\Bigm| \int _{{{\mathbb{H}}^{n}}}{ \frac{ \vert u(\xi ) \vert ^{p-1}}{1+{{ \vert \xi \vert }_{\mathbb{H}}}^{Q+sp}}\,d \xi < \infty } \biggr\} . $$

The paper is organized as follows. Section 2 collects some well-known results on ${\mathbb{H}}^{n}$, and we show that (1.1) is well defined for $u\in {{L}_{sp}}({{\mathbb{H}}^{n}})\cap C_{\mathrm{loc}}^{1,1}({{\mathbb{H}}^{n}})$. In Sect. 3, we establish three maximum principles. Theorem 1.1 and Theorem 1.2 are proved in Sect. 4.

Preliminaries

The Heisenberg group ${\mathbb{H}}^{n}$ is the Euclidean space ${{\mathbb{R}}^{2n+1}}(n\ge 1)$ endowed with the group law ∘:

$$ \bar{\xi }\circ \xi = \Biggl(x+\bar{x},y+\bar{y},t+\bar{t}+2\sum _{i=1}^{n}{({{x}_{i}} {{{ \bar{y}}}_{i}}-{{y}_{i}} {{{\bar{x}}}_{i}})} \Biggr), $$

(2.1)

where $\xi =({{x}_{1}},\ldots,{{x}_{n}},{{y}_{1}},\ldots,{{y}_{n}},t):=(x,y,t) \in {\mathbb{R}}^{n}\times {\mathbb{R}}^{n}\times {\mathbb{R}}$ and $\bar{\xi }=(\bar{x},\bar{y},\bar{t})$. Denote by $\delta _{\kappa }$ the dilations on ${{\mathbb{H}}^{n}}$, i.e.,

$$ {{\delta }_{\kappa }}(\xi )= \bigl(\kappa x,\kappa y,{{ \kappa }^{2}}t \bigr), \quad \kappa >0, $$

(2.2)

which satisfy ${{\delta }_{\kappa }}(\bar{\xi }\circ \xi )={{\delta }_{\kappa }}( \bar{\xi })\circ {{\delta }_{\kappa }}(\xi )$.

The left invariant vector fields corresponding to ${\mathbb{H}}^{n}$ are

$$\begin{aligned}& {{X}_{i}}=\frac{\partial }{\partial {{x}_{i}}}+2{{y}_{i}} \frac{\partial }{\partial t},\quad i=1,\ldots,n, \\& {{Y}_{i}}=\frac{\partial }{\partial {{y}_{i}}}-2{{x}_{i}} \frac{\partial }{\partial t},\quad i=1,\ldots,n, \\& T=\frac{\partial }{\partial {t}}. \end{aligned}$$

It is easy to check that ${{X}_{i}}$ and ${{Y}_{j}}$ satisfy

$$ [{{X}_{i}},{{Y}_{j}}]=-4T{{\delta }_{ij}}, \quad \quad [{{X}_{i}},{{X}_{j}}]=[{{Y}_{i}},{{Y}_{j}}]=0, \quad i,j=1,\ldots,n. $$

The Heisenberg gradient of a function u is defined by

$$ {{\nabla }_{\mathbb{H}}}u=({{X}_{1}}u, \ldots,{{X}_{n}}u,{{Y}_{1}}u,\ldots,{{Y}_{n}}u), $$

(2.3)

and the subLaplacian ${{\Delta }_{\mathbb{H}}}$ on ${\mathbb{H}}^{n}$ is

$$\begin{aligned} {{\Delta }_{\mathbb{H}}}:={}&\sum_{i=1}^{n} \bigl({{{X}_{i}}^{2}+{{Y}_{i}}^{2}} \bigr) \\ ={}&\sum_{i=1}^{n} \biggl({ \frac{{{\partial }^{2}}}{\partial {{x}_{i}}^{2}}+ \frac{{{\partial }^{2}}}{\partial {{y}_{i}}^{2}}+4{{y}_{i}} \frac{{{\partial }^{2}}}{\partial {{x}_{i}}\partial t}}-4{{x}_{i}} \frac{{{\partial }^{2}}}{\partial {{y}_{i}}\partial t}+4 \bigl({{x}_{i}}^{2}+{{y}_{i}}^{2} \bigr) \frac{{{\partial }^{2}}}{\partial {{t}^{2}}} \biggr). \end{aligned}$$

(2.4)

The family $\{ {{X}_{1}},\ldots,{{X}_{n}},{{Y}_{1}},\ldots, {{Y}_{n}} \}$ satisfies Hörmander’s rank condition (see [16]) which implies that ${{\Delta }_{\mathbb{H}}}$ is hypoelliptic and the maximum principle holds for solutions to the equation involving ${{\Delta }_{\mathbb{H}}}$ (see [2]).

The integer $Q=2n+2$ is called the homogeneous dimension of ${\mathbb{H}}^{n}$. Denote by ${{ \vert \xi \vert }_{\mathbb{H}}}$ the distance from ξ to the zero (see [13])

$$ {{ \vert \xi \vert }_{\mathbb{H}}}={{ \Biggl(\sum _{i=1}^{n}{ \bigl({{x}_{i}}^{2}+{{y}_{i}}^{2}} \bigr)^{2}}}+{{t}^{2}} {{ \Biggr)}^{ \frac{1}{4}}}. $$

(2.5)

Authors in [22] used the norm $\vert (z,w) \vert ={{(\sum_{i=1}^{n}{({{x}_{i}}^{2}+{{y}_{i}}^{2}})^{2}}}+16{{t}^{2}}{{)}^{ \frac{1}{4}}}$ for $(x,y,t):=(z,w)\in {{\mathbb{H}}^{n}}$, which is equivalent to (2.5). The distance between two points of ${\mathbb{H}}^{n}$ is defined by

$$ {{d}_{\mathbb{H}}}(\xi ,\eta )={{ \bigl\vert {{\eta }^{-1}}\circ \xi \bigr\vert }_{\mathbb{H}}}, $$

where $\eta ^{-1}$ denotes the inverse of η with respect to ∘, that is, $\eta ^{-1}=-\eta$. The open ball of radius $R>0$ centered at ξ is the set

$$ {{B}_{\mathbb{H}}}({{\xi }},R)= \bigl\{ \eta \in {\mathbb{H}^{n}} \mid {{d}_{\mathbb{H}}}(\eta ,{{\xi }})< R \bigr\} . $$

It is well known that $\xi \to {{ \vert \xi \vert }_{\mathbb{H}}}$ is homogeneous of degree one with respect to $\delta _{\kappa }$ and

$$ \bigl\vert {{B}_{\mathbb{H}}}({{\xi }},R) \bigr\vert = \bigl\vert {{B}_{ \mathbb{H}}}(0,R) \bigr\vert = \bigl\vert {{B}_{\mathbb{H}}}(0,1) \bigr\vert {{R}^{Q}}, $$

where $\vert \cdot \vert$ denotes the Lebesgue measure.

A function u is called the cylindrical function if

$$ u(x,y,t)=u(r,t), $$

where $(x,y,t)\in {{\mathbb{H}}^{n}}$, $r={{({{ \vert x \vert }^{2}}+{{ \vert y \vert }^{2}})}^{ \frac{1}{2}}}$.

Proposition 2.1

For$u\in {{L}_{sp}}({{\mathbb{H}}^{n}})\cap C_{\mathrm{loc}}^{1,1}({{\mathbb{H}}^{n}})$, the operator in (1.1) is well defined.

Proof

For any $\xi \in {{\mathbb{H}}^{n}}$,

$$\begin{aligned} & \mathit{PV} \int _{\mathbb{H}^{n}} \frac{ \vert u(\xi )-u(\eta ) \vert ^{p-2}(u(\xi )-u(\eta ))}{ \vert \eta ^{-1}\circ \xi \vert _{\mathbb{H}}^{Q+sp}}\,d \eta \\ &\quad =\lim_{\varepsilon \to 0} \biggl[ \int _{B_{\mathbb{H}}(\xi ,1) \setminus B_{\mathbb{H}}(\xi ,\varepsilon )} \bigl(\big\vert \big\langle -(\nabla _{\mathbb{H}}u,Tu),\eta ^{-1}\circ \xi \big\rangle +o\big( \big\vert \eta ^{-1}\circ \xi \big\vert _{\mathbb{H}}^{2}\big) \big\vert ^{p-2} \\ &\quad\quad{}\times \big( \big\langle -(\nabla _{\mathbb{H}}u,Tu ),\eta ^{-1}\circ \xi \big\rangle +o\big( \big\vert \eta ^{-1}\circ \xi \big\vert _{\mathbb{H}}^{2} \big) \big) \bigr) \bigl(\big\vert \eta ^{-1}\circ \xi \big\vert _{\mathbb{H}}^{Q+ sp}\bigr)^{-1} \,d \eta \\ & \quad \quad {} + \int _{\mathbb{H}^{n}\setminus B_{\mathbb{H}}(\xi ,1)} \frac{ \vert u(\xi )-u(\eta ) \vert ^{p-2}(u(\xi )-u(\eta ))}{ \vert \eta ^{-1}\circ \xi \vert _{\mathbb{H}}^{Q+sp}}\,d \eta \biggr] \\ &\quad \leq C\lim_{\varepsilon \to 0} \biggl[ \int _{{{B}_{\mathbb{H}}}(\xi ,1) \setminus {{B}_{\mathbb{H}}}(\xi ,\varepsilon )} \frac{o(1)}{ \vert \eta ^{-1}\circ \xi \vert _{\mathbb{H}}^{Q+sp}}\,d\eta + \int _{{{B}_{ \mathbb{H}}}(\xi ,1)\setminus {{B}_{\mathbb{H}}}(\xi ,\varepsilon )} \frac{o(1)}{ \vert \eta ^{-1}\circ \xi \vert _{\mathbb{H}}^{Q+sp}}\,d\eta \\ & \quad \quad {} + \int _{{{\mathbb{H}}^{n}}\setminus {{B}_{\mathbb{H}}}( \xi ,1)}\frac{{{u}^{p-1}}(\xi )}{ \vert \eta ^{-1}\circ \xi \vert _{\mathbb{H}}^{Q+sp}}\,d\eta + \int _{{{ \mathbb{H}}^{n}}\setminus {{B}_{\mathbb{H}}}(\xi ,1)} \frac{{{u}^{p-1}}(\eta )}{ \vert \eta ^{-1}\circ \xi \vert _{\mathbb{H}}^{Q+sp}}\,d \eta \biggr] \\ &\quad :=C\lim_{\varepsilon \to 0}({{I}_{1}}+{{I}_{2}}+{{I}_{3}}+{{I}_{4}}), \end{aligned}$$

where ε is sufficiently small. Noting that $u\in C_{\mathrm{loc}}^{1,1}({{\mathbb{H}}^{n}})$, $Q+sp-p< Q$, and $Q+sp-2p+2< Q$, we know that $I_{1}$ and $I_{2}$ are finite; $I_{3}$ is clearly convergent when ${{ \vert \xi \vert }_{\mathbb{H}}}\to \infty$; and $I_{4}$ is finite from $u\in {{L}_{sp}}({{\mathbb{H}}^{n}})$. Hence, (1.1) is well defined. □

Maximum principles

In this section, we prove three maximum principles which will be used in the process of moving planes. These maximum principles are on a bounded domain in ${{\mathbb{H}}^{n}}$, on a bounded domain in the left domain of some hyperplane, and on a narrow region.

Lemma 3.1

LetΩbe a bounded domain in${{\mathbb{H}}^{n}}$. Assume$u\in {{L}_{sp}}({{\mathbb{H}}^{n}})\cap C_{\mathrm{loc}}^{1,1}({{\mathbb{H}}^{n}})$is lower semicontinuous onΩ̄and satisfies

$$ \textstyle\begin{cases} (-{{\Delta }_{\mathbb{H}}})_{p}^{s}u(\xi )\ge 0, \quad \xi \in \varOmega , \\ u(\xi )\ge 0, \quad \xi \in \mathbb{H}^{n}\setminus \varOmega , \end{cases} $$

(3.1)

then

$$ u(\xi )\ge 0,\quad \xi \in \varOmega . $$

(3.2)

Furthermore, if$u=0$at some point inΩ, then

$$ u(\xi )=0 \quad \textit{almost everywhere in } {{\mathbb{H}}^{n}}. $$

These conclusions also hold on the unbounded regionΩif we further assume that

$$ \mathop{\underline{\lim }}_{{{ \vert \xi \vert }_{\mathbb{H}}}\to \infty } u(\xi )\ge 0. $$

Proof

Suppose that (3.2) is not true, then by the lower semicontinuity of u on Ω̄ there exists ${{\xi }^{0}}\in \bar{\varOmega }$ such that

$$ u \bigl({{\xi }^{0}} \bigr)=\min_{{\bar{\varOmega }}} u< 0. $$

From (3.1), we know that ${{\xi }^{0}}$ is a point in Ω, and

$$\begin{aligned} (-{{\Delta }_{\mathbb{H}}})_{p}^{s}u \bigl({{\xi }^{0}} \bigr)&={{C}_{Q,s,p}}\mathit{PV} \int _{{{\mathbb{H}}^{n}}}{ \frac{{{ \vert u({{\xi }^{0}})-u(\eta ) \vert }^{p-2}}(u({{\xi }^{0}})-u(\eta ))}{{{ \vert {{\eta }^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}}\,d \eta \\ &\le {{C}_{Q,s,p}} \int _{{{\mathbb{H}}^{n}}\setminus \varOmega }{ \frac{{{ \vert u({{\xi }^{0}})-u(\eta ) \vert }^{p-2}}(u({{\xi }^{0}})-u(\eta ))}{{{ \vert {{\eta }^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}}\,d \eta \\ &< 0, \end{aligned}$$

which contradicts (3.1). This implies (3.2).

If there exists some point ${{\xi }^{0}}\in \varOmega$ such that $u({{\xi }^{0}})=0$, then

$$ 0\le (-{{\Delta }_{\mathbb{H}}})_{p}^{s}u \bigl({{\xi }^{0}} \bigr)={{C}_{Q,s,p}} \int _{{{\mathbb{H}}^{n}}}{ \frac{{{ \vert u(\eta ) \vert }^{p-2}}(-u(\eta ))}{{{ \vert {{\eta }^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}}\,d \eta . $$

Using $u(\xi )\ge 0$, we have $u(\xi )=0$ almost everywhere in ${{\mathbb{H}}^{n}}$.

For an unbounded region Ω, the condition $\underline{\lim }_{{{ \vert \xi \vert }_{\mathbb{H}}}\to \infty } u(\xi )\ge 0$ implies that the negative minimum ${{\xi }^{0}}$ of u cannot be reached at infinity. Then the condition of lower semicontinuity ensures that the proof can go on as above. The proof is ended. □

Let $T_{\lambda }$ be a hyperplane in ${{\mathbb{H}}^{n}}$ defined by

$$ T_{\lambda }= \bigl\{ \xi \in {{\mathbb{H}}^{n}}\mid t=\lambda , \lambda \in \mathbb{R} \bigr\} . $$

Denote by $\tilde{\xi }=(y,x,2\lambda -t)$ the H-reflection of $\xi =(x,y,t)$ about the plane $T_{\lambda }$ and by

$$ \varSigma _{\lambda }= \bigl\{ \xi \in {{\mathbb{H}}^{n}}\mid t< \lambda \bigr\} $$

the region in the left of the plane $T_{\lambda }$. Letting

$$ {{u}_{\lambda }}(\xi )={{u}_{\lambda }} \bigl( \bigl\vert (x,y) \bigr\vert ,t \bigr):=u \bigl( \bigl\vert (x,y) \bigr\vert ,2\lambda -t \bigr) $$

and using the H-refection (see [1]), we have

$$ {{u}_{\lambda }}(\xi )=u(y,x,2\lambda -t)=u \bigl({{\xi }^{\lambda }} \bigr). $$

Set

$$ {{w}_{\lambda }}(\xi )={{u}_{\lambda }}(\xi )-u(\xi ). $$

Lemma 3.2

LetΩbe a bounded domain in${{\varSigma }_{\lambda }}$. Assume that the cylindrical function$u\in {{L}_{sp}}({{\mathbb{H}}^{n}})\cap C_{\mathrm{loc}}^{1,1}({{\mathbb{H}}^{n}})$is lower semicontinuous onΩ̄and satisfies

$$ \textstyle\begin{cases} (-{{\Delta }_{\mathbb{H}}})_{p}^{s}{{u}_{\lambda }}(\xi )-(-{{ \Delta }_{\mathbb{H}}})_{p}^{s}u(\xi )\geq 0, \quad \xi \in \varOmega , \\ {{w}_{\lambda }}(\xi )\ge 0, \quad \xi \in {{\varSigma }_{\lambda }} \setminus \varOmega , \\ w_{\lambda }(\xi ^{\lambda })=-w_{\lambda }(\xi ), \quad \xi \in {{\varSigma }_{ \lambda }}, \end{cases} $$

(3.3)

then

$$ w_{\lambda }(\xi )\ge 0,\quad \xi \in \varOmega . $$

(3.4)

Furthermore, if$w_{\lambda }=0$at some point inΩ, then

$$ w_{\lambda }(\xi )=0 \quad \textit{almost everywhere in } {{ \mathbb{H}}^{n}}. $$

These conclusions also hold for the unbounded regionΩif we further assume that

$$ \mathop{\underline{\lim }}_{{{ \vert \xi \vert }_{\mathbb{H}}}\to \infty } w_{\lambda }(\xi )\ge 0. $$

Proof

Suppose that (3.4) is incorrect. By the lower semicontinuity of $w_{\lambda }$ on Ω̄, there exists ${{\xi }^{0}}\in \bar{\varOmega }$ such that

$$ w_{\lambda } \bigl({{\xi }^{0}} \bigr)=\min_{{\bar{\varOmega }}} w_{\lambda }< 0. $$

For simplicity, we denote

$$ G(a)={{ \vert a \vert }^{p-2}}a, \quad a\geq 0. $$

Note that $G(a)$ is increasing and ${G}'(a)=(p-1){{ \vert a \vert }^{p-2}}\ge 0$. A direct calculation gives

$$\begin{aligned}& (-{{\Delta }_{\mathbb{H}}})_{p}^{s}{{u}_{\lambda }} \bigl({{\xi }^{0}} \bigr)-(-{{ \Delta }_{\mathbb{H}}})_{p}^{s}u \bigl({{\xi }^{0}} \bigr) \\& \quad = {{C}_{Q,s,p}}\mathit{PV} \int _{{{\mathbb{H}}^{n}}}{ \frac{G({{u}_{\lambda }}({{\xi }^{0}})-{{u}_{\lambda }}(\eta ))-G(u({{\xi }^{0}})-u(\eta ))}{{{ \vert {{\eta }^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}}\,d \eta \\& \quad = {{C}_{Q,s,p}}\mathit{PV} \int _{{{\varSigma }_{\lambda }}}{ \frac{G({{u}_{\lambda }}({{\xi }^{0}})-{{u}_{\lambda }}(\eta ))-G(u({{\xi }^{0}})-u(\eta ))}{{{ \vert {{\eta }^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}}\,d \eta \\& \quad\quad{} +{{C}_{Q,s,p}}\mathit{PV} \int _{{{\varSigma }_{\lambda }}}{ \frac{G({{u}_{\lambda }}({{\xi }^{0}})-u(\eta ))-G(u({{\xi }^{0}})-{{u}_{\lambda }}(\eta ))}{{{ \vert {{({{\eta }^{\lambda }})}^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}}\,d \eta \\& \quad = {{C}_{Q,s,p}}\mathit{PV} \int _{{{\varSigma }_{\lambda }}} \biggl( \frac{1}{{{ \vert {{\eta }^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}- \frac{1}{{{ \vert {{({{\eta }^{\lambda }})}^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}} \biggr) \\& \quad\quad{}\times \bigl(G \bigl({{u}_{\lambda }} \bigl({{\xi }^{0}} \bigr)-{{u}_{\lambda }}(\eta ) \bigr)-G \bigl(u \bigl({{ \xi }^{0}} \bigr)-u(\eta ) \bigr) \bigr)\,d\eta \\& \quad\quad{} +{{C}_{Q,s,p}}\mathit{PV} \int _{{{\varSigma }_{\lambda }}} \bigl(G\bigl({{u}_{\lambda }}\bigl({{\xi }^{0}}\bigr)-{{u}_{\lambda }}(\eta )\bigr) -G\bigl(u\bigl({{\xi }^{0}}\bigr)-u_{\lambda }(\eta )\bigr)+ G\bigl({{u}_{\lambda }}\bigl({{\xi }^{0}}\bigr)-{{u}}(\eta )\bigr) \\& \quad\quad{}-G\bigl(u\bigl({{\xi }^{0}}\bigr)-u(\eta )\bigr) \bigr) \bigl( {{ \big\vert {{\bigl({{\eta }^{\lambda }}\bigr)}^{-1}}\circ {{\xi }^{0}} \big\vert }_{\mathbb{H}}}^{Q+ sp}\bigr)^{-1} \,d \eta \\& \quad := {{C}_{Q,s,p}}({{J}_{1}}+{{J}_{2}}). \end{aligned}$$

(3.5)

For $J_{1}$, we have for any ${{\xi }^{0}},\eta \in {{\varSigma }_{\lambda }}$,

$$ \frac{1}{{{ \vert {{\eta }^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}- \frac{1}{{{ \vert {{({{\eta }^{\lambda }})}^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}>0. $$

By the monotonicity of G and the fact that

$$ \bigl({{u}_{\lambda }} \bigl({{\xi }^{0}} \bigr)-{{u}_{\lambda }}( \eta ) \bigr)- \bigl(u \bigl({{\xi }^{0}} \bigr)-u( \eta ) \bigr)={{w}_{\lambda }} \bigl({{\xi }^{0}} \bigr)-{{w}_{\lambda }}( \eta ) $$

is nonpositive but not identity to 0, we deduce that

$$ G \bigl({{u}_{\lambda }} \bigl({{\xi }^{0}} \bigr)-{{u}_{\lambda }}( \eta ) \bigr)-G \bigl(u \bigl({{\xi }^{0}} \bigr)-u( \eta ) \bigr) $$

is also nonpositive but not identity to 0. So we have

$$ J_{1}< 0. $$

(3.6)

For $J_{2}$, by the mean value theorem,

$$\begin{aligned} {{J}_{2}}&= \int _{{{\varSigma }_{\lambda }}}{ \frac{G({{u}_{\lambda }}({{\xi }^{0}})-{{u}_{\lambda }}(\eta ))-G(u({{\xi }^{0}})-{{u}_{\lambda }}(\eta ))+G({{u}_{\lambda }}({{\xi }^{0}})-u(\eta ))-G(u({{\xi }^{0}})-u(\eta ))}{{{ \vert {{({{\eta }^{\lambda }})}^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+ sp}}}\,d \eta \\ &=w_{\lambda } \bigl(\xi ^{0} \bigr) \int _{{{\varSigma }_{\lambda }}}{ \frac{{G}'(g(\eta ))+{G}'(h(\eta ))}{{{ \vert {{({{\eta }^{\lambda }})}^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}}\,d \eta \\ &\le 0. \end{aligned}$$

(3.7)

In fact, if $u_{\lambda }(\eta )\ge u(\eta )$, then we have $w_{\lambda }(\eta )\ge 0$, i.e., (3.4) holds. If $u_{\lambda }(\eta )> u(\eta )$, we know G is strictly increasing, then ${G}'(g(\eta ))\ge 0$ and ${G}'(h(\eta ))\ge 0$. Hence, we have (3.7).

Putting (3.6) and (3.7) into (3.5) implies

$$ (-{{\Delta }_{\mathbb{H}}})_{p}^{s}{{u}_{\lambda }} \bigl({{\xi }^{0}} \bigr)-(-{{ \Delta }_{\mathbb{H}}})_{p}^{s}u \bigl({{\xi }^{0}} \bigr)< 0. $$

This contradicts (3.3) and we obtain (3.4).

If there exists some point ${{\xi }^{0}}\in \varOmega$ such that ${{w}_{\lambda }}({{\xi }^{0}})=0$, then (3.5) holds and ${{J}_{2}}\ge 0$. Hence from the first inequality in (3.3) we have ${{J}_{1}}\ge 0$, and by the monotonicity of G,

$$ G \bigl({{u}_{\lambda }} \bigl({{\xi }^{0}} \bigr)-{{u}_{\lambda }}( \eta ) \bigr)-G \bigl(u \bigl({{\xi }^{0}} \bigr)-u( \eta ) \bigr)\ge 0. $$

We have, for almost all $\eta \in {{\varSigma }_{\lambda }}$,

$$ \bigl({{u}_{\lambda }} \bigl({{\xi }^{0}} \bigr)-{{u}_{\lambda }}( \eta ) \bigr)- \bigl(u \bigl({{\xi }^{0}} \bigr)-u( \eta ) \bigr)={{w}_{\lambda }} \bigl({{\xi }^{0}} \bigr)-{{w}_{\lambda }}( \eta )=-{{w}_{ \lambda }}(\eta )\ge 0. $$

Using (3.4), we have

$$ {{w}_{\lambda }}(\xi )=0\quad \text{almost everywhere in } {{\varSigma }_{ \lambda }}. $$

From the antisymmetry of ${{w}_{\lambda }}$,

$$ {{w}_{\lambda }}(\xi )=0\quad \text{almost everywhere in } {{ \mathbb{H}}^{n}}. $$

 □

Lemma 3.3

LetΩbe a bounded narrow domain in${{\varSigma }_{\lambda }}$and locate in$\{\xi\mid \lambda -l< t<\lambda \}$for small l. Assume that the cylindrical function$u\in {{L}_{sp}}({{\mathbb{H}}^{n}})\cap C_{\mathrm{loc}}^{1,1}({{\mathbb{H}}^{n}})$is lower semicontinuous onΩ̄. If$c(x)$is bounded from below inΩandusatisfies

$$ \textstyle\begin{cases} (-{{\Delta }_{\mathbb{H}}})_{p}^{s}{{u}_{\lambda }}(\xi )-(-{{ \Delta }_{\mathbb{H}}})_{p}^{s}u(\xi )+c(\xi ){{w}_{\lambda }}(\xi ) \ge 0, \quad \xi \in \varOmega , \\ {{w}_{\lambda }}(\xi )\ge 0, \quad \xi \in {{\varSigma }_{\lambda }} \setminus \varOmega , \\ {{w}_{\lambda }}({{\xi }^{\lambda }})=-{{w}_{\lambda }}(\xi ),\quad \xi \in {{\varSigma }_{\lambda }}, \end{cases} $$

(3.8)

then

$$ w_{\lambda }(\xi )\ge 0,\quad \xi \in \varOmega . $$

(3.9)

Furthermore, if$w_{\lambda }=0$at some point inΩ, then

$$ w_{\lambda }(\xi )=0 \quad \textit{almost everywhere in } {{ \mathbb{H}}^{n}}. $$

These conclusions also hold for the unbounded regionΩif we further assume that

$$ \mathop{\underline{\lim }}_{{{ \vert \xi \vert }_{\mathbb{H}}}\to \infty } w_{\lambda }(\xi )\ge 0. $$

Proof

By the proof of Lemma 3.2, we have

$$\begin{aligned} & (-{{\Delta }_{\mathbb{H}}})_{p}^{s}{{u}_{\lambda }} \bigl({{\xi }^{0}} \bigr)-(-{{ \Delta }_{\mathbb{H}}})_{p}^{s}u \bigl({{\xi }^{0}} \bigr) \\ &\quad ={{C}_{Q,s,p}}\mathit{PV} \int _{{{\varSigma }_{\lambda }}} \biggl( \frac{1}{{{ \vert {{\eta }^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}- \frac{1}{{{ \vert {{({{\eta }^{\lambda }})}^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}} \biggr) \\ &\quad\quad{}\times \bigl(G \bigl({{u}_{\lambda }} \bigl({{\xi }^{0}} \bigr)-{{u}_{\lambda }}(\eta ) \bigr)-G \bigl(u \bigl({{ \xi }^{0}} \bigr)-u(\eta ) \bigr) \bigr)\,d\eta \\ &\quad\quad{} +{{C}_{Q,s,p}}\mathit{PV} \int _{{{\varSigma }_{\lambda }}} \bigl(G\bigl({{u}_{\lambda }}\bigl({{\xi }^{0}}\bigr)-{{u}_{\lambda }}(\eta )\bigr) -G\bigl(u\bigl({{\xi }^{0}}\bigr)-u(\eta )\bigr)+ G\bigl({{u}_{\lambda }}\bigl({{\xi }^{0}}\bigr)-{{u}}(\eta )\bigr) \\ &\quad\quad{} -G\bigl(u\bigl({{\xi }^{0}}\bigr)-u_{\lambda }(\eta )\bigr) \bigr) \bigl({{ \big\vert {{\bigl({{\eta }^{\lambda }}\bigr)}^{-1}}\circ {{\xi }^{0}} \big\vert }_{\mathbb{H}}}^{Q+ sp} \bigr)^{-1} \,d \eta \\ &\quad :={{C}_{Q,s,p}}({{I}_{1}}+{{I}_{2}}). \end{aligned}$$

(3.10)

Obviously,

$$ {{I}_{2}}\le 0. $$

(3.11)

Similar to (3.6), we know

$$ I_{1}< 0. $$

Denote

$$ {{\delta }_{{{\xi }^{0}}}}=\operatorname{dist} \bigl({{\xi }^{0}}, \partial {{\varSigma }_{ \lambda }} \bigr)= \bigl\vert \lambda -{{t}^{0}} \bigr\vert . $$

Combining (3.10), (3.11), and $I_{1}<0$, we have

$$ \frac{(-{{\Delta }_{\mathbb{H}}})_{p}^{s}{{u}_{\lambda }}({{\xi }^{0}})-(-{{\Delta }_{\mathbb{H}}})_{p}^{s}u({{\xi }^{0}})}{{{\delta }_{{{\xi }^{0}}}}}< 0. $$

(3.12)

Noting that ${{\xi }^{0}}$ is a negative minimum of ${{w}_{\lambda }}$, we infer $\nabla {{w}_{{{\lambda }_{0}}}}({{\xi }^{0}})=0$, and so

$$ \frac{\partial {{w}_{\lambda }}}{\partial t} \bigl({{\xi }^{0}} \bigr)= \lim _{{{\delta }_{k}}\to 0} \frac{{{w}_{\lambda }}({{\xi }^{0}})}{{{\delta }_{{{\xi }^{0}}}}}=0, $$

i.e.,

$$ \frac{c({{\xi }^{0}}){{w}_{\lambda }}({{\xi }^{0}})}{{{\delta }_{{{\xi }^{0}}}}} \le o(1). $$

(3.13)

Now (3.12) and (3.13) contradict (3.8), and then (3.9) is proved. □

Proof of the main results

Following the idea in [6], we first use Lemma 3.1, Lemma 3.2, and Lemma 3.3 to prove Theorem 1.1.

Proof of Theorem 1.1

First we check that for λ sufficiently negative it holds

$$ {{w}_{\lambda }}(\xi )\ge 0,\quad \xi \in {{\varSigma }_{\lambda }}. $$

(4.1)

Indeed, suppose that (4.1) is violated, then by (1.8) there exists a point $\xi ^{0} \in {{\varSigma }_{\lambda }}$ such that

$$ {{w}_{\lambda }} \bigl({{\xi }^{0}} \bigr)= \min _{{{\varSigma }_{\lambda }}} {{w}_{\lambda }}< 0, $$

i.e., ${{u}_{\lambda }}({{\xi }^{0}})\le {{\varsigma }_{\lambda }}({{\xi }^{0}}) \le u({{\xi }^{0}})$. Note by (1.2) that

$$ (-{{\Delta }_{\mathbb{H}}})_{p}^{s}{{u}_{\lambda }}( \xi )-(-{{\Delta }_{ \mathbb{H}}})_{p}^{s}u(\xi )={f}' \bigl({{\varsigma }_{\lambda }}(\xi ) \bigr){{w}_{ \lambda }}( \xi ), $$

(4.2)

where ${{\varsigma }_{\lambda }}(\xi )$ between ${{u}_{\lambda }}(\xi )$ and $u(\xi )$. For sufficiently negative λ, $u({{\xi }^{0}})$ is small by (1.8), hence so is ${{\varsigma }_{\lambda }}({{\xi }^{0}})$. Due to the condition of ${f}'$, we have ${f}'({{\varsigma }_{\lambda }}({{\xi }^{0}}))\le 0$. From (4.2),

$$ (-{{\Delta }_{\mathbb{H}}})_{p}^{s}{{u}_{\lambda }} \bigl({{\xi }^{0}} \bigr)-(-{{ \Delta }_{\mathbb{H}}})_{p}^{s}u \bigl({{\xi }^{0}} \bigr)\ge 0. $$

(4.3)

On the other hand, it follows by the proof of Lemma 3.2 that

$$ (-{{\Delta }_{\mathbb{H}}})_{p}^{s}{{u}_{\lambda }} \bigl({{\xi }^{0}} \bigr)-(-{{ \Delta }_{\mathbb{H}}})_{p}^{s}u \bigl({{\xi }^{0}} \bigr)< 0. $$

(4.4)

This contradicts (4.3), and hence (4.1) is proved.

The above result provides the starting point of moving planes. Let us move the plane $T_{\lambda }$ to the right as long as (4.1) holds to its limiting position

$$ {{\lambda }_{0}}=\sup \bigl\{ \lambda\mid {{w}_{\mu }}(\xi ) \ge 0, \forall \xi \in {{\varSigma }_{\mu }},\mu \le \lambda \bigr\} . $$

We will show that

$$ {{\lambda }_{0}}=0 $$

(4.5)

i.e.,

$$ {{w}_{{{\lambda }_{0}}}}(\xi )=0,\quad \xi \in {{\varSigma }_{{{\lambda }_{0}}}}. $$

(4.6)

In fact, suppose that (4.6) is false, we have by Lemma 3.2 that

$$ {{w}_{{{\lambda }_{0}}}}(\xi )>0,\quad \xi \in {{\varSigma }_{{{\lambda }_{0}}}}. $$

(4.7)

From the definition of ${{\lambda }_{0}}$, there exist a sequence ${{\lambda }_{k}}\to {{\lambda }_{0}}$ and a point ${{\xi }^{k}}\in {{\varSigma }_{{{\lambda }_{k}}}}$ such that

$$ {{w}_{{{\lambda }_{k}}}} \bigl({{\xi }^{k}} \bigr)= \min _{{{\varSigma }_{{{\lambda }_{k}}}}} {{w}_{{{ \lambda }_{k}}}}< 0, \quad\quad \nabla {{w}_{{{\lambda }_{k}}}} \bigl({{\xi }^{k}} \bigr)=0. $$

(4.8)

Note that

$$ (-{{\Delta }_{\mathbb{H}}})_{p}^{s}{{u}_{{{\lambda }_{k}}}} \bigl({{\xi }^{k}} \bigr)-(-{{ \Delta }_{\mathbb{H}}})_{p}^{s}u \bigl({{\xi }^{k}} \bigr)={f}' \bigl({{\varsigma }_{{{ \lambda }_{k}}}} \bigl({{\xi }^{k}} \bigr) \bigr){{w}_{{{\lambda }_{k}}}} \bigl({{\xi }^{k}} \bigr). $$

(4.9)

If ${{ \vert {{\xi }^{k}} \vert }_{\mathbb{H}}}$ is sufficiently large, then $u({{\xi }^{k}})$ is small and so ${{\varsigma }_{{{\lambda }_{k}}}}({{\xi }^{k}})$ is also small, this implies ${f}'({{\varsigma }_{{{\lambda }_{k}}}}({{\xi }^{k}}))\le 0$ (because ${f}'(a)\le 0$ for the sufficiently small a). It follows

$$ (-{{\Delta }_{\mathbb{H}}})_{p}^{s}{{u}_{{{\lambda }_{k}}}} \bigl({{\xi }^{k}} \bigr)-(-{{ \Delta }_{\mathbb{H}}})_{p}^{s}u \bigl({{\xi }^{k}} \bigr)\ge 0. $$

But this contradicts the fact that ${{\xi }^{k}}$ is a negative minimum of ${{w}_{{{\lambda }_{k}}}}$ (see Lemma 3.2). Hence, $\{ {{\xi }^{k}} \}$ is bounded, i.e., the sequence $\{ {{\xi }^{k}} \}$ is bounded.

It follows that the subsequence of $\{ {{\xi }^{k}} \}$ converges to some point ${{\xi }^{0}}$. Then (4.8) means that, for ${{\xi }^{0}}\in \partial {{\varSigma }_{{{\lambda }_{0}}}}$,

$$ {{w}_{{{\lambda }_{0}}}} \bigl({{\xi }^{0}} \bigr)\le 0, \quad \quad \nabla {{w}_{{{\lambda }_{0}}}} \bigl({{\xi}^{0}} \bigr)=0. $$

Particularly,

$$ \frac{\partial {{w}_{{{\lambda }_{0}}}}}{\partial t} \bigl({{\xi }^{0}} \bigr)= \lim _{{{\delta }_{k}}\to 0} \frac{{{w}_{{{\lambda }_{k}}}}({{\xi }^{k}})}{{{\delta }_{k}}}=0. $$

Applying (4.9), we have

$$ \lim_{{{\delta }_{k}}\to 0} \frac{1}{{{\delta }_{k}}} \bigl((-{{\Delta }_{\mathbb{H}}})_{p}^{s}{{u}_{{{ \lambda }_{k}}}} \bigl({{\xi }^{k}} \bigr)-(-{{\Delta }_{\mathbb{H}}})_{p}^{s}u \bigl({{ \xi }^{k}} \bigr) \bigr)=\lim_{{{\delta }_{k}}\to 0} \frac{1}{{{\delta }_{k}}}{f}' \bigl({{\varsigma }_{{{\lambda }_{k}}}} \bigl({{ \xi }^{k}} \bigr) \bigr){{w}_{{{\lambda }_{k}}}} \bigl({{\xi }^{k}} \bigr)=0, $$

which is a contradiction with Lemma 3.3. Therefore,

$$ {{w}_{{{\lambda }_{0}}}}(\xi )\ge 0,\quad \xi \in {{\varSigma }_{{{\lambda }_{0}}}}. $$

(4.10)

Similarly, we can move the plane from +∞ to the left to get

$$ {{w}_{{{\lambda }_{0}}}}(\xi )\le 0,\quad \xi \in {{\varSigma }_{{{\lambda }_{0}}}}. $$

(4.11)

Then (4.6) follows by combining (4.10) and (4.11). Finally, we see that u must be symmetric and monotone with respect to t about some point. □

Next, we give the proof of Theorem 1.2.

Proof of Theorem 1.2

By condition (1.8) and $f(0)=0$, we claim

$$ u(\xi )>0 \quad \text{or} \quad u(\xi )\equiv 0 \quad \text{for any } \xi \in \mathbb{H}_{+}^{n}. $$

In fact, suppose that the conclusion $u(\xi )>0$ is not correct, we will verify $u(\xi )\equiv 0$. The lower semicontinuity of u on $\mathbb{H}_{+}^{n}$ implies that there exists ${{\xi }^{0}}\in \bar{\mathbb{H}}_{+}^{n}$ such that

$$ u \bigl({{\xi }^{0}} \bigr)=\min_{\bar{\mathbb{H}}_{+}^{n}} u=0, $$

and then

$$\begin{aligned} (-{{\Delta }_{\mathbb{H}}})_{p}^{s}u \bigl({{\xi }^{0}} \bigr)&={{C}_{Q,s,p}}\mathit{PV} \int _{{{\mathbb{H}}^{n}}}{ \frac{{{ \vert u({{\xi }^{0}})-u(\eta ) \vert }^{p-2}}(u({{\xi }^{0}})-u(\eta ))}{{{ \vert {{\eta }^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}}\,d \eta \\ &={{C}_{Q,s,p}}\mathit{PV} \int _{\mathbb{H}_{+}^{n}}{ \frac{{{ \vert u({{\xi }^{0}})-u(\eta ) \vert }^{p-2}}(u({{\xi }^{0}})-u(\eta ))}{{{ \vert {{\eta }^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}}\,d \eta \\ &={{C}_{Q,s,p}}\mathit{PV} \int _{\mathbb{H}_{+}^{n}}{ \frac{-u(\eta ){{ \vert u(\eta ) \vert }^{p-2}}}{{{ \vert {{\eta }^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}}\,d \eta \\ &=f \bigl(u \bigl({{\xi }^{0}} \bigr) \bigr)=0. \end{aligned}$$

Hence $\int _{\mathbb{H}_{+}^{n}}{ \frac{-u(\eta ){{ \vert u(\eta ) \vert }^{p-2}}}{{{ \vert {{\eta }^{-1}}\circ {{\xi }^{0}} \vert }_{\mathbb{H}}}^{Q+sp}}}\,d \eta =0$, and then $u(\xi )\equiv 0$, $\xi \in \mathbb{H}_{+}^{n}$.

In the sequel, we only need to treat the case $u>0$ on $\mathbb{H}_{+}^{n}$. Let us employ the method of moving planes to u along the t direction and denote

$$ T_{\lambda }^{+}= \bigl\{ \xi \in {{\mathbb{H}}_{+}^{n}} \mid t=\lambda ,\lambda \in {{\mathbb{R}}^{+}} \bigr\} $$

and

$$ \varSigma _{\lambda }^{+}= \bigl\{ \xi \in {{\mathbb{H}}_{+}^{n}} \mid 0< t< \lambda \bigr\} . $$

The H-reflection of $\xi =(x,y,t)$ about $T_{\lambda }^{+}$ is ${{\xi }^{\lambda }}=(y,x,2\lambda -t)$, and let

$$ {{w}_{\lambda }}(\xi )={{u}_{\lambda }}(\xi )-u(\xi ). $$

If $\lambda >0$ is sufficiently small, we deduce from Lemma 3.3 with $\varOmega =\varSigma _{\lambda }^{+}$ and ${{\varSigma }_{\lambda }}=\varSigma _{\lambda }^{+}\cup ({{\mathbb{H}}^{n}} \setminus \mathbb{H}_{+}^{n})$ that on the narrow region $\varSigma _{\lambda }^{+}$,

$$ {{w}_{\lambda }}(\xi )\ge 0. $$

(4.12)

This provides the starting point of moving planes. Now we will explain that the plane $T_{\lambda }^{+}$ can be moved to the infinity so that (4.12) holds. Let

$$ {{\lambda }_{0}}=\sup \bigl\{ \lambda >0\mid {{w}_{\mu }}(\xi )\ge 0, \forall \xi \in \varSigma _{\lambda }^{+},\mu \le \lambda \bigr\} , $$

and we will prove

$$ {{\lambda }_{0}}=\infty . $$

(4.13)

In fact, if ${{\lambda }_{0}}<\infty$, then we claim that $T_{\lambda }^{+}$ can be moved further to the right, that is, there exists $\sigma >0$ such that, for any $\lambda \in ({{\lambda }_{0}},{{\lambda }_{0}}+\sigma )$,

$$ {{w}_{\lambda }}(\xi )\ge 0,\quad \xi \in \varSigma _{\lambda }^{+}. $$

(4.14)

This will contradict the definition of ${{\lambda }_{0}}$, and hence (4.13) holds.

At present, we prove (4.14). If ${{ \vert \xi \vert }_{\mathbb{H}}}$ is sufficiently large, then (4.14) is true by using the similar proof to (4.1) in Theorem 1.1. This implies that there exists some ${{R}_{0}}>0$ such that (4.14) holds true on $\mathbb{H}_{+}^{n}\setminus {{B}_{\mathbb{H}}}(0,{{R}_{0}})$. Next we point out that (4.14) is also true on ${{B}_{\mathbb{H}}}(0,{{R}_{0}})$. Noting ${{\lambda }_{0}}<\infty$ and using Lemma 3.2, we find that on $\xi \in \varSigma _{\lambda _{0} }^{+}\cap {{B}_{\mathbb{H}}}(0,{{R}_{0}})$,

$$ {{w}_{\lambda _{0} }}(\xi )>0 $$

(4.15)

or

$$ {{w}_{{{\lambda }_{0}}}}(\xi )\equiv 0. $$

In the case ${{w}_{{{\lambda }_{0}}}}(\xi )\equiv 0$, we observe by the boundary conditions of u that $u(\xi )\equiv 0$. On the other hand, (4.15) implies that there exists small $\delta >0$ such that

$$ {{w}_{{{\lambda }_{0}}}}(\xi )\ge c>0,\quad \xi \in \overline{ \varSigma _{{{\lambda }_{0}}-\delta }^{+}\cap {{B}_{\mathbb{H}}}(0,{{R}_{0}})}. $$

(4.16)

Since ${{w}_{\lambda }}$ relies continuously on λ, there exists $\sigma >\varepsilon >0$ such that

$$ {{w}_{{{\lambda }_{0}}+\varepsilon }}(\xi )\ge 0,\quad \xi \in \overline{\varSigma _{{{\lambda }_{0}}-\delta }^{+}\cap {{B}_{\mathbb{H}}}(0,{{R}_{0}})}. $$

(4.17)

Since $(\varSigma _{{{\lambda }_{0}}+\varepsilon }^{+}\setminus \varSigma _{{{ \lambda }_{0}}-\delta }^{+})\cap {{B}_{\mathbb{H}}}(0,{{R}_{0}})$ is a narrow region, we have by Lemma 3.3

$$ {{w}_{{{\lambda }_{0}}+\varepsilon }}(\xi )\ge 0,\quad \xi \in \overline{\varSigma _{{{\lambda }_{0}}+\varepsilon }^{+}\cap {{B}_{\mathbb{H}}}(0,{{R}_{0}})}, $$

and therefore (4.14) is proved.

Using (4.13) and Lemma 3.2 once again, it follows that, for any $\xi \in \varSigma _{\lambda }^{+}$ (here $0\le \lambda \le \infty$),

$$ {{w}_{\lambda }}(\xi )>0, $$

(4.18)

or

$$ {{w}_{\lambda }}(\xi )\equiv 0. $$

(4.19)

For (4.19), we can use the boundary condition of u to obtain $u(\xi )\equiv 0$. In addition, it follows from (4.18) that $u(\xi )$ is strictly increasing, which contradicts the boundary condition of u and (1.8). □