Existence of positive radial solution for Neumann problem on the Heisenberg group

Abstract

The existence of at least one positive radial solution of the Neumann problem

$$ -\Delta _{\mathbb{H}^{n}} u+R(\xi ) u=a \bigl( \vert \xi \vert _{\mathbb{H}^{n}} \bigr) \vert u \vert ^{p-2} u - b\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u \vert ^{q-2}u, $$

is proved on the Heisenberg group $\mathbb{H}^{n}$, via the variational principle, where $a(|\xi |_{\mathbb{H}^{n}})$, $b(|\xi |_{\mathbb{H}^{n}})$ are nonnegative radial functions and $R(\xi )$ satisfies suitable conditions.

1 Introduction

Semilinear elliptic equations were the first nonlinear generalization of linear elliptic partial differential equations. They are of fundamental importance for the study of engineering, geometry, life sciences, economics, physics and mechanics; see [1, 23–27]. Some examples, like the theory of Bose–Einstein condensates, obey

$$ -\Delta u = f(x, u). $$

The variational approach to these equations has experienced spectacular success in recent years, reaching a high level of complexity and refinement, with a multitude of applications (see [10, 14–16, 22]).

In [3] and [12] some problems which depend on continuous component of time like coherent states in quantum optics are probed. These problems are studied in a space which have a component of time and are known as Heisenberg group. Important topics where the Heisenberg group reveals itself as an essential factor are quantum mechanics, ergodic theory, representation theory of nilpotent Lie group, foundation of abelian harmonic analysis, and the theory of partial differential equations. We are now interested in the last one.

In this paper, the existence of at least one radial solution of a generalized well-known supercritical Neumann problem

$$ \textstyle\begin{cases} -\Delta _{\mathbb{H}^{n}} u+R(\xi ) u=a ( \vert \xi \vert _{\mathbb{H}^{n}}) \vert u \vert ^{p-2} u - b( \vert \xi \vert _{\mathbb{H}^{n}}) \vert u \vert ^{q-2}u & \xi \in \varOmega , \\ u>0 & \xi \in \varOmega , \\ {\frac{\partial u}{\partial n}}=0 & \xi \in \partial \varOmega , \end{cases} $$

(1.1)

is proved where Ω is the Korányi ball in the Heisenberg group. Recently, the existence of a nontrivial weak solution of following singular boundary value problem on the Heisenberg group was proved (see [29]):

$$ \textstyle\begin{cases} -\Delta _{\mathbb{H}^{n}} u= \mu \frac{g(\xi )u}{( \vert z \vert ^{4}+t^{2})^{\frac{1}{2}}} +\lambda f(\xi , t) & \xi \in \varOmega , \\ u=0 & \xi \in \partial \varOmega . \end{cases} $$

In [18] problem (1.1) is studied where $R(\xi )=1$ on the unit ball in $\mathbb{R}^{n}$, $n\geq 2$, see [2, 17, 21, 22] for some applications.

Here we recall some definitions and results on the Heisenberg group (see [4, 7, 30]). Let us recall some features of the Heisenberg group. The Heisenberg group $\mathbb{H}^{n} =(\mathbb{R}^{2n+1},\circ )$ is the space $\mathbb{R}^{2n+1}$ with the noncommutative law of product

$$ (x, y, t) \circ \bigl(x^{\prime } , y^{\prime }, t^{\prime }\bigr) = \bigl(x+x^{ \prime }, y+y^{\prime }, t+t^{\prime }+2 \bigl( \bigl\langle y, x^{\prime } \bigr\rangle - \bigl\langle x, y^{\prime }\bigr\rangle \bigr)\bigr), $$

where $x_{1}, x_{2}, y_{1}, y_{2} \in \mathbb{R}^{n}$, $t_{1},t_{2} \in \mathbb{R}$ and $\langle \; , \; \rangle$ denotes the standard inner product in $\mathbb{R}^{n}$. This operation endows $\mathbb{H}^{n}$ with the structure of a Lie group. The Lie algebra of $\mathbb{H}^{n}$ is generated by the left-invariant vector fields

$$ T=\frac{\partial }{\partial t},\qquad X_{i} = \frac{\partial }{\partial x_{i}}+ 2 {y_{i}} \frac{\partial }{\partial t},\qquad Y_{i} = \frac{\partial }{\partial y_{i}} - 2 {x_{i}} \frac{\partial }{\partial t},\quad i=1,2,3,\ldots ,n. $$

These generators satisfy the noncommutative formula

$$ [X_{i}, Y_{j}]=-4\delta _{ij}T,\qquad [X_{i}, X_{j}]=[Y_{i}, Y_{j}]=[X_{i}, T]=[Y_{i}, T]=0. $$

Let $z=(x,y) \in \mathbb{R}^{2n}$ and $\xi =(z,t) \in \mathbb{H}^{n}$. The parabolic dilation

$$ \delta _{\lambda }\xi = \bigl(\lambda x, \lambda y, \lambda ^{2}t \bigr), $$

satisfies

$$ \delta _{\lambda }(\xi _{0} \circ \xi ) = \delta _{\lambda } \xi \circ \delta _{\lambda }\xi _{0}, $$

and

$$ \vert \xi \vert _{\mathbb{H}^{n}} = \bigl( \vert z \vert ^{4}+t^{2}\bigr)^{\frac{1}{4}} = \bigl(\bigl(x^{2}+y^{2} \bigr)^{2}+t^{2}\bigr)^{\frac{1}{4}}, $$

is a norm with respect to the parabolic dilation which is known as Korányi gauge norm $N(z,t)$. The Heisenberg distance between two points $(z,t)$ and $(z^{\prime }, t^{\prime })$ is given by

$$ \rho \bigl(z,t;z^{\prime },t^{\prime }\bigr)= \bigl\vert \bigl(z^{\prime },t^{\prime }\bigr)^{-1} \circ (z,t) \bigr\vert _{ \mathbb{H}^{n}}. $$

Clearly, the vector fields $X_{i}$, $Y_{i}$, $i=1,2,\ldots ,n$ are homogeneous of degree 1 under the norm $| \cdot |_{\mathbb{H}^{n}}$ and T is homogeneous of degree 2. The Korányi ball of center $\xi _{0}$ and radius r is defined by

$$ B_{\mathbb{H}^{n}}(\xi _{0},r) =\bigl\{ \xi : \bigl\vert \xi ^{-1} \circ \xi _{0} \bigr\vert _{ \mathbb{H}^{n}}\leq r \bigr\} , $$

and it satisfies

$$ \bigl\vert B_{\mathbb{H}^{n}}(\xi _{0},r) \bigr\vert = \bigl\vert B_{\mathbb{H}^{n}}(0,r) \bigr\vert = r^{2n+2} \bigl\vert B_{ \mathbb{H}^{n}}(0,1) \bigr\vert . $$

The Heisenberg gradient and Kohn-Laplacian (Heisenberg-Laplacian) operator on $\mathbb{H}^{n}$ are given by

$$ \nabla _{\mathbb{H}^{n}} = (X_{1}, X_{2}, \ldots ,X_{n}, Y_{1}, Y_{2}, \ldots , Y_{n}) $$

and

$$ \Delta _{\mathbb{H}^{n}} = \sum_{i=1}^{n} {X_{i}^{2}+Y_{i}^{2}}, $$

respectively. Let $\varOmega \subset \mathbb{H}^{n}$, $n\geq 1$, be a smooth bounded open set. Its associated Sobolev space is defined as follows:

$$ H^{1}\bigl(\varOmega , \mathbb{H}^{n}\bigr):=\bigl\{ u\in L^{2}(\varOmega ): X_{i}u, Y_{i}u \in L^{2}(\varOmega ), i=1,2,\ldots ,n\bigr\} , $$

and closure of $C_{0}^{\infty }(\varOmega )$ in $H^{1}(\varOmega , \mathbb{H}^{n})$ under the norm

$$ \Vert u \Vert _{H^{1}(\varOmega , \mathbb{H}^{n})} = \biggl( \int _{\varOmega }\bigl( \vert \nabla _{ \mathbb{H}^{n}}u \vert ^{2}+ \vert u \vert ^{2}\bigr) \,d\xi \biggr)^{\frac{1}{2}}, $$

where $u: \varOmega \subset \mathbb{H}^{n}\rightarrow \mathbb{R}$, is denoted by $H_{0}^{1}(\varOmega , \mathbb{H}^{n})$. The following norm is a norm on $H_{0}^{1}(\varOmega , \mathbb{H}^{n})$:

$$ \Vert u \Vert _{H_{0}^{1}(\varOmega , \mathbb{H}^{n})} = \biggl( \int _{\varOmega } \vert \nabla _{ \mathbb{H}^{n}}u \vert ^{2} \,d\xi \biggr)^{\frac{1}{2}}, $$

which is equivalent to the standard one. The dual space of $H_{0}^{1}(\varOmega , \mathbb{H})$ will be denoted by $H^{-1}(\varOmega , \mathbb{H})$. Here we recall Hardy’s inequality and some results on the Heisenberg group.

Lemma 1.1

([19])

For$n\geq 1$and for any$u\in H_{0}^{1}(\varOmega , \mathbb{H}^{n})$, we have

$$ \int _{\varOmega }\frac{ \vert u \vert ^{2}}{( \vert z \vert ^{4}+ \vert t \vert ^{2})^{\frac{1}{2}}} \,d\xi \leq \biggl( \frac{n+1}{n^{2}}\biggr)^{2} \int _{\varOmega } \vert \nabla _{\mathbb{H}^{n}}u \vert ^{2} \,d\xi . $$

For convenience, we write the above inequality as follows:

$$ \int _{\varOmega }\frac{ \vert u \vert ^{2}}{( \vert z \vert ^{4}+ \vert t \vert ^{2})^{\frac{1}{2}}} \,d\xi \leq \frac{1}{C_{n}} \int _{\varOmega } \vert \nabla _{\mathbb{H}^{n}}u \vert ^{2} \,d \xi , $$

where$C_{n}=(\frac{n^{2}}{n+1})^{2}$.

Lemma 1.2

([13, 29])

Let$\varOmega \subset \mathbb{H}^{n}$be a bounded open set. Then the following compact embedding is satisfied:

$$ H_{0}^{1}\bigl(\varOmega , \mathbb{H}^{n}\bigr) \subset \subset L^{s} (\varOmega ) \quad \textit{for } 1\leq s < \bar{Q}, $$

where$\bar{Q}=\frac{2Q}{Q-2}$is critical exponent of$Q = 2n+2$, which is a homogeneous dimension of$\mathbb{H}^{n}$. We denote the Sobolev embedding constant of the above compact embedding by$C_{s} >0$; i.e.

$$ \Vert u \Vert _{L^{s}(\varOmega )}\leq C_{s} \Vert u \Vert _{H_{0}^{1}(\varOmega , \mathbb{H}^{n})}\quad \textit{for all } u\in H_{0}^{1}( \varOmega , \mathbb{H}), 1\leq s \leq \bar{Q}. $$

Consider two subspaces of the Heisenberg–Sobolev space $H_{0}^{1}(\varOmega , \mathbb{H})$, as follows:

$$ D^{1,2}(\varOmega ) : =\bigl\{ u :\varOmega \rightarrow \mathbb{R} : u, \vert \nabla _{\mathbb{H}^{n}}u \vert \in L^{2} (\varOmega ) \bigr\} , $$

with respect to the norm

$$ \Vert u \Vert _{D^{1,2}(\varOmega )} = \biggl( \int _{\varOmega }\bigl( \vert \nabla _{\mathbb{H}^{n}}u \vert ^{2}+ \vert u \vert ^{2}\bigr) \,d\xi \biggr)^{\frac{1}{2}}, $$

and $D^{1,2}_{0}(\varOmega )$ be the closure of $C_{0}^{\infty }(\varOmega )$ in $\{u \in D^{1,2}(\varOmega ) :0 \leq u , u \text{ is increasing} \}$. According to Poincaré’s inequality in Heisenberg–Sobolev space (see [8]), $D_{0}^{1,2}(\varOmega )$ is equipped with the norm

$$ \Vert u \Vert _{D_{0}^{1,2}(\varOmega )} = \biggl( \int _{\varOmega }\bigl( \vert \nabla _{\mathbb{H}^{n}}u \vert ^{2}\bigr) \,d\xi \biggr)^{\frac{1}{2}}, $$

which is equivalent to the standard one. Additionally, define

$$ D^{2,2}(\varOmega ): =\bigl\{ u :\varOmega \rightarrow \mathbb{R} : u , \vert \nabla _{H}u \vert , \vert \Delta _{H} u \vert \in L^{2} (\varOmega ) \bigr\} , $$

which is equipped with the norm

$$ \Vert u \Vert _{D^{2,2}(\varOmega )} = \biggl( \int _{\varOmega }\bigl( \vert \Delta _{\mathbb{H}^{n}}u \vert ^{2}+ \vert \nabla _{\mathbb{H}^{n}}u \vert ^{2}+ \vert u \vert ^{2}\bigr) \,d\xi \biggr)^{\frac{1}{2}}. $$

The following compact embedding is needed for the main result.

Theorem 1.1

([9])

Let$\varOmega \subset \mathbb{H}^{n}$be a bounded open set. Then the following embeddings are compact:

• (I) If$Q = 4$, then$D^{2,2} (\varOmega )\cap D_{0}^{1,2}(\varOmega )\hookrightarrow L^{s}( \varOmega )$, $1\leq s<\infty$.

• (II) If$Q > 4$, then$D^{2,2} (\varOmega )\cap D_{0}^{1,2}(\varOmega )\hookrightarrow L^{s}( \varOmega )$, $1\leq s < Q^{*}$,

where$Q^{*}=\frac{2Q}{Q-4}$and as above$Q=2n+2$.

Now we recall results we need later. Let V be a real Banach space and $V^{*}$ be its topological dual and pairing between V and $V^{*}$ denoted by $\langle \, , \, \rangle$. A function $\psi : V \rightarrow \mathbb{R}$ is called weakly lower semi-continuous (w.l.s.c.) if

$$ \psi (u) \leq \lim_{n \rightarrow \infty } \inf \psi (u_{n}), $$

for each $u \in V$ and any sequence $\{u_{n}\}$ converging to u in the weak topology $\sigma (V,V^{*})$. Let $\psi : V \rightarrow \mathbb{R} \cup \{+\infty \}$ be a proper convex function. The subdifferential of ψ at u is the set

$$ \partial \psi (u) =\bigl\{ u^{*} \in V^{*} : \psi (v) \geq \psi (u)+ \bigl\langle u^{*} , v-u \bigr\rangle \text{ for all } v \in V \bigr\} . $$

The multivalued mapping $\partial \psi : V \rightarrow 2^{V^{*}}$ (where $2^{V^{*}}$ is the set of all subsets of $V^{*}$) by ψ is Gâteaux differentiable at u, denoted by $D \psi (u)$, if and only if $\partial \psi (u)$ is a singleton. In this case, $\partial \psi (u)=\{D\psi (u)\}$.

Lemma 1.3

([1])

LetXbe a reflexive Banach space and$I: X\rightarrow \mathbb{R}$be a continuous convex functional. ThenIis w.l.s.c..

Theorem 1.2

(Weierstrass, [1])

LetXbe a reflexive Banach space and$I: X\rightarrow \mathbb{R}$be w.l.s.c. and coercive. ThenIhas a global minimum point.

Definition 1.1

LetVbe a real Banach space, $\varphi \in C^{1} (V, \mathbb{R})$and$\psi : V \rightarrow ( -\infty , +\infty ]$be a proper (i.e. $\operatorname{Dom} \psi \ne \emptyset$), convex and lower semi-continuous function, andKbe a convex and weakly closed subset ofV. Define the function$\psi _{K} :V\rightarrow (-\infty , +\infty ]$by

$$ \psi _{K} (u):= \textstyle\begin{cases} \psi (u) & u\in K , \\ +\infty & u\notin K. \end{cases} $$

(1.2)

Suppose the functional$I :V \rightarrow (-\infty , +\infty ]$is defined by

$$ I (w):=\psi _{K} (w)- \varphi (w) , $$

(1.3)

$u \in \operatorname{Dom} ( \psi _{K})$is called a critical point ofI, if$D \varphi (u) \in \partial \psi _{K} (u)$or equivalently,

$$ \psi _{K} (v)-\psi _{K} (u) \geq \bigl\langle D \varphi (u) , v-u \bigr\rangle , \quad \textit{for all } u \in V. $$

Notice that a global minimum point of I is a critical point of I.

Definition 1.2

([11])

We say thatIsatisfies the Palais–Smale compactness condition (PS) if every sequence$\{ u_{n} \}$is such that

• $I[u_{n}] \rightarrow c \in \mathbb{R}$,

• $\langle D \varphi (u_{n}), u_{n}-v \rangle +\psi (v) - \psi (u_{n}) \geq -\varepsilon _{n} \| v-u_{n}\|$, for all$v \in V$,

where$\varepsilon _{n} \rightarrow 0$, then$\{u_{n}\}$possesses a convergent subsequence.

The following Mountain Pass Geometry (MPG) theorem is proved in [28].

Theorem 1.3

Suppose that$I: V \rightarrow (-\infty , + \infty ]$is of the form (1.3) and satisfies the PS condition and the following conditions:

• (i) $I(0)=0$,

• (ii) there exists$e \in V$such that$I(e) \leq 0$,

• (iii) there exists positive constantρsuch that$I(u) > 0$, if$\|u\|= \rho$.

ThenIhas a critical value$c \leq \rho$which is characterized by

$$ c= \operatorname*{\inf\sup}_{g \in \varGamma \ t \in [0,1]} {I\bigl[g(t)\bigr]}, $$

where$\varGamma =\{ g \in C([0,1],V) : g(0)=0 , g(1)=e\}$.

Subsequently, we need the following theorem.

Theorem 1.4

(G. Polya, [20])

Assume that$u_{n}:[a,b] \rightarrow \mathbb{R}$, $n \geq 1$is a monotone increasing sequence of (not necessarily continuous) functions which converge point-wise to a continuous function$u: [a,b] \rightarrow \mathbb{R}$, then the convergence is uniform.

The first eigenvalue plays a very relevant role in nonlinear elliptic problems.

Remark 1.1

(see [1])

According to the variational characterization of the first eigenvalue when$\lambda _{1}=\lambda _{1}(-\Delta _{\mathbb{H}^{n}}+R)>0$ (this happens certainly if$R(\xi ) \geq 0$a.e. inΩ) or whenRsatisfies one of the conditions of Theorem1.5, it is easy to see that the quantity

$$ (u|v)= \int \nabla _{\mathbb{H}^{n}} u \nabla _{\mathbb{H}^{n}} v \,d \xi + \int R(\xi ) u v \,d\xi $$

defines a scalar product on$H_{0}^{1} (\varOmega , \mathbb{H}^{n})$that induces a norm

$$ \Vert u \Vert _{*}= \biggl( \int \vert \nabla _{\mathbb{H}^{n}} u \vert ^{2}+R(\xi ) \vert u \vert ^{2} \,d \xi \biggr)^{\frac{1}{2}}, $$

equivalent to the standard one and similarly on$D_{0}^{1,2} (\varOmega )$.

Now we can state the main result of this paper.

Theorem 1.5

LetΩbe a smooth, bounded open set in the Heisenberg group$\mathbb{H}^{n}$, $2\leq q< p< Q^{*}$and$a , b\in L^{\infty } (0,1)$be increasing positive non-constant radial functions. The problem (1.1) admits at least one radially increasing solution if at least one of the following conditions is met:

• (i) $R(\xi )=\alpha$where$\alpha >0$is a real constant.

• (ii) $R(\xi ) \in L^{\infty } (\varOmega )$and$\lambda _{1}(-\Delta _{\mathbb{H}^{n}} +R(\xi ))\geq 0$, where$\lambda _{1}$is the first eigenvalue.

• (iii) $R(\xi ) \leq c (1+|\omega |_{\mathbb{H}^{n}})^{-2}$, where$c \geq 0$is a small enough constant.

2 Positive radial solution

Here, we recall the following variational principle established in [5].

Definition 2.1

The triple$(\psi , \varphi , K)$satisfies the point-wise invariance condition at a point$u\in V$if there exists a convex Gâteaux differentiable function$G: V\rightarrow \mathbb{R}$and a point$v\in K$such that

$$ D\psi (v) + \operatorname{DG}(v) = D\varphi (u) + \operatorname{DG}(u). $$

Theorem 2.1

LetVbe a reflexive Banach space andKbe a convex and weakly closed convex subset ofV. Let$\psi :V \rightarrow \mathbb{R} \cup \{+\infty \}$be a convex, lower semi-continuous function which is Gâteaux differentiable onK, and let$\varphi \in C^{1}(V, \mathbb{R})$. Assume that the following two assertions hold:

• (i) The functional$I :V \rightarrow \mathbb{R} \cup \{+\infty \}$defined by$I(w)=\psi _{K} (w)- \varphi (w)$has a critical point$u \in V$as in Definition2.1,

• (ii) the triple$(\psi , \varphi , K)$satisfies the point-wise invariance condition at the point u.

Then$u \in K$is a solution of the equation

$$ D\psi (u)= D\varphi (u). $$

To use Theorem 2.1, consider the bounded open set $\varOmega \subset \mathbb{H}^{n}$ and the Banach space $V =D^{1,2}_{0,r} (\varOmega )\cap D^{2,2}(\varOmega ) \cap L^{p}_{a} ( \varOmega )\cap L^{q}_{b}(\varOmega )$, where

$$ L^{p}_{a} (\varOmega )= \biggl\{ u: \int _{\varOmega }a\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u \vert ^{p} \,d\xi < \infty \biggr\} $$

and

$$ L^{q}_{b} (\varOmega )= \biggl\{ u: \int _{\varOmega }b\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u \vert ^{q} \,d\xi < \infty \biggr\} , $$

and also

$$ D^{1,2}_{0,r} (\varOmega )= \bigl\{ u\in D_{0}^{1,2}( \varOmega ): 0\leq u, u \text{ is a radial function}\bigr\} , $$

$D^{2,2}(\varOmega )$ is defined as above and $u: \varOmega \rightarrow \mathbb{R}$ is called radial function if $u(x, y, t)= \phi (\varrho )$ where $\varrho = |(x, y, t)|_{\mathbb{H}^{n}}$. V is equipped with the norm

$$\begin{aligned} \Vert u \Vert _{V} &:= \Vert u \Vert _{*} + \Vert u \Vert _{L^{p}_{a}} + \Vert u \Vert _{L^{q}_{b}} \\ &=\biggl( \int _{\varOmega } \vert \nabla _{\mathbb{H}^{n}}u \vert ^{2}+R(\xi ) \vert u \vert ^{2} \,d\xi \biggr)^{ \frac{1}{2}}+ \biggl( \int _{\varOmega }a\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u \vert ^{p} \,d\xi \biggr)^{ \frac{1}{p}}\\ &\quad {}+ \biggl( \int _{\varOmega }b\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u \vert ^{q} \,d\xi \biggr)^{ \frac{1}{q}} . \end{aligned}$$

We consider the Euler–Lagrange functional corresponding to problem (1.1); i.e.

$$ E(u) = \frac{1}{2} \int _{\varOmega } \vert \nabla _{\mathbb{H}^{n}} u \vert ^{2}+R( \xi ) \vert u \vert ^{2} \,d\xi + \frac{1}{q} \int _{\varOmega }b\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u \vert ^{q}\,d \xi -\frac{1}{p} \int _{\varOmega }a\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u \vert ^{p}\,d\xi , $$

over the convex and weakly closed set

$$ K=\bigl\{ u\in V: u \geq 0, u \text{ is increasing with respect to the radius }\varrho = \vert \omega \vert _{ \mathbb{H}^{n}} \bigr\} . $$

(2.1)

To adapt Theorem 2.1 to our problem, we define $\psi ,\varphi :V\rightarrow \mathbb{R}$ by

$$ \psi (u) = \frac{1}{2} \int _{\varOmega } \vert \nabla _{\mathbb{H}^{n}} u \vert ^{2}+R( \xi ) \vert u \vert ^{2} \,d\xi + \frac{1}{q} \int _{\varOmega }b\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u \vert ^{q}\,d \xi $$

and

$$ \varphi (u) = \frac{1}{p} \int _{\varOmega }a\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u \vert ^{p}\,d \xi . $$

Finally, let us introduce the functional $I : V\rightarrow (-\infty , +\infty ]$ as follows:

$$ I(u) = \psi _{K}(u) -\varphi (u), $$

where $\psi _{K}$ is defined as (1.2). We should be aware that I is indeed the Euler–Lagrange functional corresponding to our problem restricted to K. Theorem 2.1 implies the following corollary.

Corollary 2.1

Let$V=D^{1,2}_{0,r} (\varOmega )\cap D^{2,2}(\varOmega ) \cap L^{p}_{a} ( \varOmega )\cap L^{q}_{b}(\varOmega )$, andKbe the convex closed subset ofVgiven in (2.1). Assume that the functional

$$ I(w):=\psi _{K} (w)- \frac{1}{p} \int _{\varOmega }a\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert w \vert ^{p} \,d\xi , $$

(2.2)

has a critical point$u\in V$and there exists$v \in K$satisfying the linear equation

$$ \textstyle\begin{cases} -\Delta _{H} v+R(\xi ) v + b( \vert \xi \vert _{\mathbb{H}^{n}}) \vert v \vert ^{q-2}v =a( \vert \xi \vert _{\mathbb{H}^{n}}) \vert u \vert ^{p-2}u & \xi \in \varOmega , \\ \frac{\partial v}{\partial n}=0 & \xi \in \partial \varOmega , \end{cases} $$

(2.3)

in the weak sense. Then$u \in K$is a solution of the equation

$$ \textstyle\begin{cases} -\Delta _{H}u+ R(\xi ) u + b( \vert \xi \vert _{\mathbb{H}^{n}}) \vert u \vert ^{q-2}u = a( \vert \xi \vert _{\mathbb{H}^{n}}) \vert u \vert ^{p-2}u & \xi \in \varOmega , \\ {\frac{\partial u}{\partial n}}=0 & \xi \in \partial \varOmega . \end{cases} $$

Notice that satisfying Eq. (2.3) shows that the triple $(\psi _{K}, \varphi , K)$ satisfies the point-wise invariance condition at u, where $G=0$. We need some lemmas and theorems before proving our main result.

Lemma 2.1

Assume$R(\xi )$is given in Theorem1.5. If$h\in L^{2}(\varOmega )$, $f:\mathbb{R}\rightarrow \mathbb{R}$is continuous and there exist$\alpha ,\beta > 0$such that

$$ \bigl\vert f(t) \bigr\vert \leq \alpha +\beta \vert t \vert ^{Q^{*}-1}\quad \textit{and}\quad f(t)t \leq 0 $$

for all$t\in \mathbb{R}$. Then the problem

$$ \textstyle\begin{cases} -\Delta _{\mathbb{H}^{n}} u+R (\xi ) u= f(u) + h(\xi ) & \xi \in \varOmega , \\ {\frac{\partial u}{\partial n}}=0 & \xi \in \partial \varOmega , \end{cases} $$

(2.4)

admits at least one solution.

Proof

First notice that by integration, there exist $\alpha _{1}, \beta _{1} > 0$ such that

$$ \bigl\vert F(t) \bigr\vert \leq \alpha _{1}+ \beta _{1} \vert u \vert ^{Q^{*}} $$

and $F(t)\leq 0$ for all $t\in \mathbb{R}$, where $F(t)=\int _{0}^{t} f(s )\,ds$. Now consider the following energy functional corresponding to problem (2.4) on $D_{0}^{1,2}(\varOmega ) \cap D^{2,2}(\varOmega )$:

$$ J(u) = \frac{1}{2} \Vert u \Vert _{*}^{2} - \int _{\varOmega }F(u)\,dx - \int _{\varOmega }hu\,dx. $$

Theorem 1.1 implies

$$ J(u) \geq \frac{1}{2} \Vert u \Vert _{*}^{2} - C \Vert u \Vert _{*}. $$

So J is coercive and weakly lower semi-continuous on $D_{0}^{1,2}(\varOmega )$. So according to the Weierstrass theorem, J has a global minimum point which means problem (2.4) admits at least one solution. □

Lemma 2.2

For$2\leq q < p< Q^{*}$, there exists$C>0$such that

$$ \Vert u \Vert _{*} \leq \Vert u \Vert _{V} \leq C \Vert u \Vert _{*} $$

for all$u \in K$.

Proof

Theorem 1.1 implies

$$\begin{aligned} \Vert u \Vert _{*} &\leq \Vert u \Vert _{V} = \Vert u \Vert _{*}+ \biggl( \int _{\varOmega }a\bigl( \vert \xi \vert _{ \mathbb{H}^{n}}\bigr) \vert u \vert ^{p} \,d\xi \biggr)^{\frac{1}{p}}+ \biggl( \int _{\varOmega }b\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u \vert ^{q} \,d\xi \biggr)^{\frac{1}{q}} \\ &\leq \Vert u \Vert _{*} +C'_{1} \vert a \vert _{L^{\infty }(\varOmega )} \Vert u \Vert _{D^{1,2}_{0}( \varOmega )}+ C'_{2} \vert b \vert _{L^{\infty }(\varOmega )} \Vert u \Vert _{D^{1,2}_{0}(\varOmega )} \\ &\leq (1+ C_{1} + C_{2}) \Vert u \Vert _{*}. \end{aligned}$$

The last estimate holds, since $\| \cdot \|_{*}$ and $\| \cdot \|_{D_{0}^{1,2}(\varOmega )}$ are equivalent. □

Lemma 2.3

Let$V=D^{1,2}_{0,r}(\varOmega )\cap D^{2,2}(\varOmega ) \cap L_{a}^{p} ( \varOmega ) \cap L_{b}^{q} (\varOmega )$. Suppose the functional$I: V \rightarrow \mathbb{R}$is defined by

$$ I(u):= \psi _{K} (u) - \varphi (u) , $$

whereφand$\psi _{K}$are given in Corollary2.1. ThenIhas a nontrivial critical point.

Proof

We utilize Theorem 1.3 to prove this lemma. First recall that

$$ D\varphi (u)= a\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u \vert ^{p-2}u, $$

and therefore φ is a $C^{1}$-function on the space V. Also notice that ψ is a proper, convex and lower semi-continuous and K is closed in V. We are going to prove this lemma in two steps:

Step 1. We verify that I satisfies in MPG conditions. It is clear that $I(0)=0$. Take $e \in K$. It follows that

$$ I(te) =\frac{t^{2}}{2} \Vert u \Vert _{*}^{2} + \frac{t^{q}}{q} \int _{\varOmega } b\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert e \vert ^{q} \,d\xi -\frac{t^{p}}{p} \int _{\varOmega } a\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert e \vert ^{p} \,d\xi . $$

Now since $p> q\geq 2$, for t sufficiently large $I(te)$ is negative. We now prove condition (iii) of the MPG theorem. Take $u \in \operatorname{Dom}(\psi )$ with $\|u\|_{V}=\rho >0$. Notice that from Lemma 2.2 for $u\in K$ we have

$$ \Vert u \Vert _{V} \leq (1+C_{1}+C_{2}) \Vert u \Vert _{*}. $$

(2.5)

Also

$$\begin{aligned}& \begin{gathered}\varphi (u)=\frac{1}{p} \int _{\varOmega } a\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u \vert ^{p}\,d \xi \leq C_{3} \Vert u \Vert _{V}^{p} = C_{3} \rho ^{p}, \\ I(u) \geq \frac{\rho ^{2}}{2(1+C_{1}+C_{2})^{2}}- C_{3} \rho ^{p}>0, \end{gathered} \end{aligned}$$

(2.6)

provided $\rho >0$ is small enough as $2< p$ and $C_{1}$, $C_{2}$, $C_{3}$ are positive constants. If $u \notin \operatorname{Dom}(\psi )$. Then clearly $I(u)>0$. Therefore, MPG holds for the functional I.

Step 2. We verify the PS compactness condition. Suppose that $\{u_{n}\}$ is a sequence in K such that $I(u_{n}) \rightarrow c\in \mathbb{R}$, $\epsilon _{n} \rightarrow 0$ and

$$ \bigl\langle D\varphi (u_{n}) , u_{n}-v \bigr\rangle + \psi _{K} (v)- \psi _{K}(u_{n}) \geq - \epsilon _{n} \Vert v-u_{n} \Vert _{V}, \quad \text{for all }v \in V. $$

(2.7)

We show that $\{ u_{n}\}$ has a convergent subsequence in V. First notice that $u_{n} \in \operatorname{Dom}(\psi )$, then

$$ I(u_{n})=\psi _{K}(u_{n})-\varphi (u_{n}) \rightarrow c, \quad \mbox{as }n \rightarrow \infty . $$

Thus, for large values of n we have

$$ \psi _{K}(u_{n})- \varphi (u_{n}) \leq 1+c . $$

(2.8)

Now consider the function $g(r)= r^{q} - p(r-1) -1$ on the interval $(1, +\infty )$ and set $r^{*}=(\frac{p}{q})^{\frac{1}{q-1}}$. It is easy to see that for every $r\in (1, r^{*})$ we have $g(r)< 0$. We choose a number r for which we have $r>1$ and $r^{q}-1< p(r-1)$. In (2.7) set $v=ru_{n}$. Then

$$ (1-r) \bigl\langle D\varphi (u_{n}) , u_{n} \bigr\rangle +\bigl(r^{q}-1\bigr)\psi _{K} (u_{n}) \geq -\epsilon _{n}(r-1) \Vert u_{n} \Vert _{V}. $$

(2.9)

Furthermore,

$$ \bigl\langle D\varphi (u_{n}) , u_{n} \bigr\rangle = \int _{\varOmega }a\bigl( \vert \xi \vert _{ \mathbb{H}^{n}}\bigr) u_{n}(\xi )^{p} \,d\xi =p \varphi (u_{n}). $$

(2.10)

Since $r^{q}-1 < p(r-1)$, we can take $\beta >0$ such that $\frac{1}{p(r-1)} < \beta < \frac{1}{r^{q}-1}$. Multiplying (2.9) by β and adding it to (2.8) we obtain

$$ \bigl[ 1-\beta p(1-r) \bigr] \varphi (u_{n})+ \bigl[1-\beta \bigl(r^{q}-1\bigr)\bigr] \psi _{K}(u_{n}) \leq 1+c+\beta C \Vert u_{n} \Vert _{*}. $$

So for a suitable constant $C ^{\prime }>0$

$$ \Vert u_{n} \Vert _{*}^{2} \leq \psi _{K}(u_{n}) \leq C^{\prime }\bigl(1+ \Vert u_{n} \Vert _{*}\bigr). $$

Therefore, $\{u_{n}\}$ is a bounded sequence in Hilbert space $D^{1,2}_{0,r}(\varOmega )\cap D^{2,2}(\varOmega )$. Theorem 1.1 and the standard results in Sobolev space show there exists $\bar{u} \in D^{1,2}_{0,r}(\varOmega )\cap D^{2,2}(\varOmega )$ such that, up to subsequences,

• $u_{n} \rightharpoonup \bar{u}$ in $D^{1,2}_{0,r}(\varOmega )\cap D^{2,2}(\varOmega )$;

• $u_{n} \rightarrow \bar{u}$ in $L^{s}(\varOmega )$, $1 \leq s < Q^{*}$;

• $u_{n}(\xi ) \rightarrow \bar{u}(\xi )$ a.e in Ω;

• there exist $w_{s} \in L^{s} (\varOmega )$ such that $|u_{n}(\xi )| \leq w_{s}(\xi )$ a.e. in Ω and for all $n \in \mathbb{N}$ where $1 \leq s < Q^{*}$.

Now in (2.7) replace v with ū:

$$ - \int _{\varOmega }a\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u_{n} \vert ^{p-1} (\bar{u} - u_{n}) \,d\xi + \psi _{K} (\bar{u}) -\psi _{K} (u_{n}) \geq - \epsilon _{n} \Vert \bar{u} - u_{n} \Vert _{V} . $$

(2.11)

On the one hand, we have

$$ \biggl\vert \int _{\varOmega }a\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u_{n} \vert ^{p-1} (\bar{u} - u_{n}) \,d\xi \biggr\vert \leq \int _{\varOmega }a\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) w_{p-1}^{p-1}( \xi ) \vert \bar{u} - u_{n} \vert \,d\xi . $$

(2.12)

On the other hand, $\{u_{n}\} \subset K$ so $\{u_{n}\}$ is an increasing sequence so $\{|\bar{u}-u_{n}|\}$ is an increasing sequence, too. Now according to Theorem 1.4 the right hand side of the latter inequality goes to zero. Therefore passing into limits in (2.11), one gets

$$ \lim_{n \rightarrow \infty } \sup \psi _{K} (u_{n}) \leq \psi _{K} ( \bar{u}). $$

(2.13)

But the norm is weakly lower semi-continuous and $u_{n} \rightarrow \bar{u}$ in $L_{b}^{q} (\varOmega )$. So $\psi _{K}(u_{n}) \leq \lim_{n \rightarrow \infty } \inf \psi _{K} (u_{n})$. Then

$$ \lim_{n \rightarrow \infty } \psi _{K}(u_{n}) =\psi _{K}(\bar{u}). $$

Then $\|u_{n}\|_{V} \rightarrow \|\bar{u}\|_{V}$, and besides $u_{n}(\xi )\rightarrow \bar{u}(\xi )$ a.e. thus $u_{n} \rightarrow \bar{u}$ strongly in V as desired. Notice that every $u_{n}$ is radial, so u is radial; moreover, $u \in K$. □

Lemma 2.4

Suppose$u \in K$. There exists$v \in K$such that

$$ \textstyle\begin{cases} -\Delta _{\mathbb{H}^{n}}v+R(\xi ) v=a( \vert \xi \vert _{\mathbb{H}^{n}})u^{p-1}-b( \vert \xi \vert _{\mathbb{H}^{n}})v^{q-1} & \textit{in } \varOmega , \\ v=0 & \textit{on } \partial \varOmega , \end{cases} $$

in the weak sense.

Proof

Let $u \in K$ so $0 \leq u \in K \subset D^{2,2}$. According to Theorem 2.4, it is enough to show that $h(\xi )=a(|\xi |_{\mathbb{H}^{n}}) u(\xi )^{p-1}$ belongs to $L^{2}(\varOmega )$ and $f(v)= -b(|\xi |_{\mathbb{H}^{n}})v(\xi )^{q-1}$ satisfies the inequality

$$ \bigl\vert f(t) \bigr\vert \leq \alpha +\beta \vert t \vert ^{Q^{*}-1}\quad \text{and}\quad f(t)t \leq 0 $$

for all $t\in \mathbb{R}$. Clearly since $q< Q^{*}$ both hold. On the other hand, as $u\in L^{Q^{*}}(\varOmega )$ and $\varOmega \subset \mathbb{H}^{n}$ one can see following estimate:

$$ \begin{aligned} \bigl\vert u(\xi ) \bigr\vert &= \bigl\vert \phi (\varrho ) \bigr\vert = \biggl\vert \int _{0}^{\varrho }\phi ^{\prime }(s)\,ds + \phi (0) \biggr\vert \\ &\leq \int _{0}^{\varrho } \bigl\vert \phi ^{\prime }(s) \bigr\vert \,ds+ \bigl\vert \phi (0) \bigr\vert \\ &\leq \biggl( \int _{0}^{\varrho } \bigl\vert \phi ^{\prime }(s) \bigr\vert ^{2} s^{2n}\,ds\biggr)^{\frac{1}{2}} \biggl( \int _{0}^{\varrho }\frac{ds}{s^{2n}} \biggr)^{\frac{1}{2}} + \bigl\vert \phi (0) \bigr\vert \\ &\leq C \biggl(\omega _{2n}^{-1} \int _{\varOmega } \bigl\vert \phi ^{\prime }(\xi ) \bigr\vert ^{2} \,d \xi \biggr)^{\frac{1}{2}} \varrho ^{\frac{1-2n}{2}}+ \bigl\vert \phi (0) \bigr\vert \\ &\leq C' \biggl( \int _{\varOmega }\frac{ \vert \nabla _{\mathbb{H}^{n}}u \vert ^{2}}{ \vert \xi \vert _{\mathbb{H}^{n}}^{2}}\,d \xi \biggr)^{\frac{1}{2}} \biggl( \int _{\varOmega }\xi ^{2} \,d\xi \biggr)^{\frac{1}{2}} \varrho ^{\frac{1-2n}{2}}+ \bigl\vert \phi (0) \bigr\vert \\ &\leq C'' \vert \xi \vert _{\mathbb{H}^{n}}^{2-n} \frac{1}{C_{n}} \int _{0}^{\varrho } \vert \Delta _{\mathbb{H}^{n}}u \vert ^{2} \,d\xi + \bigl\vert \phi (0) \bigr\vert \\ &\leq C_{*} \Vert u \Vert _{D^{2,2}(\varOmega )} \vert \xi \vert _{\mathbb{H}^{n}}^{2-n}, \end{aligned} $$

where $\omega _{2n}$ is the measure of unit ball in $\mathbb{R}^{2n+1} = \mathbb{H}^{n}$, and C, $C'$, $C''$, $C_{*}$ are positive constants, and the fundamental theorem of calculus and Hardy’s inequality (1.1) are applied. Recent inequalities are satisfied since u is a radial function so $|\nabla _{\mathbb{H}^{n}}u| = \frac{\varrho }{r}|\phi ^{\prime }|$ where $\varrho =|\xi |_{\mathbb{H}^{n}} =|(z,t)|_{\mathbb{H}^{n}}$, $r=|z|$ (see [6]). Because of the last inequality and since $p < Q^{*}$, it is easy to see that $h\in L^{2}(\varOmega )$. This means the desired result has been obtained. □

Now we can state the proof of Corollary 2.1, by using Theorem 2.1 and introducing an appropriate space.

Proof

Let $V=D_{0,r}^{1,2} (\varOmega ) \cap D^{2,2}(\varOmega ) \cap L^{p}_{a} ( \varOmega )\cap L^{q}_{b} (\varOmega )$. By introducing the norm

$$ \begin{aligned} \Vert u \Vert _{V} &:= \Vert u \Vert _{*} + \Vert u \Vert _{L^{p}_{a}} + \Vert u \Vert _{L^{q}_{b}} \\ &=\biggl( \int _{\varOmega } \vert \nabla _{\mathbb{H}^{n}}u \vert ^{2}+R(\xi ) \vert u \vert ^{2} \,d\xi \biggr)^{ \frac{1}{2}}+ \biggl( \int _{\varOmega }a\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u \vert ^{p} \,d\xi \biggr)^{ \frac{1}{p}} \\ &\quad {}+ \biggl( \int _{\varOmega }b\bigl( \vert \xi \vert _{\mathbb{H}^{n}}\bigr) \vert u \vert ^{q} \,d\xi \biggr)^{ \frac{1}{q}} \end{aligned} $$

on V, it is converted to a reflexive Banach space. Note that

$$ K =\biggl\{ u \in V : \frac{\partial u}{\partial n}=0 \text{ on } \partial \varOmega \text{ and } u \text{ is a radially increasing function}\biggr\} $$

is a convex closed subset and we define the functional $I =\psi _{K} - \varphi$ which is the Euler–Lagrange functional restricted to K. It follows from the MPG theorem and the PS compactness condition in Lemma 2.3 that the functional I has a critical point u, also it is guaranteed that $I(u)>0$. Lemma 2.4 implies that there exists $v \in \operatorname{Dom} (\psi )$ satisfying the linear equation $D\psi _{K}(v) =D \varphi (u)$. Setting $C(\xi ):=1- a(|\xi |_{\mathbb{H}^{n}}) u(\xi )^{p-1}$ we have $-\Delta _{\mathbb{H}^{n}} u + C(\xi )$$u=0$ in Ω. We show $u>0$ in Ω. Otherwise, by applying the strong maximum principle, we deduce u is identically zero in Ω. On the other hand u is a radially increasing function. This a contradiction and so u must be a nontrivial nonnegative solution of (1.1). So the conditions of Corollary 2.1 hold and the proof is completed. □

Remark 2.1

Corollary2.1implies problem (1.1) has at least one nontrivial radially increasing solution.