Multiple non-negative solutions to a semilinear equation on Heisenberg group with indefinite nonlinearity

Abstract

This paper is concerned with the existence and multiplicity of non-negative solutions to the semilinear equation $-\Delta_{H} u = K(\xi)\vert u\vert ^{2^{\sharp}-2}u + \mu \vert \xi \vert _{H}^{\alpha}u$ in a bounded domain $\Omega\subset\mathbb{H}^{N}$ with Dirichlet boundary conditions. Here $\mathbb{H}^{N}$ is the Heisenberg group and $2^{\sharp}= 2q/(q-2)$ is the critical exponent of the Sobolev embedding on the Heisenberg group. The function $K(\xi)$ may be sign changing on Ω. Using the variational method, we prove that this problem has at least two non-negative solutions provided μ, α, and $K(\xi)$ satisfy some conditions.

1 Introduction

This paper is concerned with the existence and multiplicity of non-negative solutions to the semilinear equation on the Heisenberg group $\mathbb{H}^{N}$ of the form

$$ \textstyle\begin{cases} -\Delta_{H} u = K(\xi)\vert u\vert ^{2^{\sharp}-2}u + \mu \vert \xi \vert _{H}^{\alpha}u &\mbox{in } \Omega, \\ u(\xi) = 0&\mbox{on } \partial\Omega, \end{cases} $$

(1.1)

where $0\in\Omega$ and Ω is a bounded domain with smooth boundary of the Heisenberg group $\mathbb{H}^{N}$. The $\Delta_{H}$ (see the definition below) is the Kohn Laplacian on the Heisenberg group. $2^{\sharp}$ is the critical exponent for the semilinear Dirichlet problem of the Kohn Laplacian, and the exponent $2N/(N-2)$ is critical for the semilinear equation $-\Delta u = \vert u\vert ^{\frac{2N}{N-2}-2}u + h(u)$ in a domain of $\mathbb{R}^{N}$ with the Dirichlet boundary condition. The function $K(\xi)\in L^{\infty}(\Omega)$ and $K(\xi)=K_{+}-K_{-}$ with $K_{+}=\max\{K(\xi),0\}\neq0$ and $K_{-}=\max\{-K(\xi),0\}\neq0$, which is why we use the terms indefinite nonlinearity in the title.

We start with some basic notions (see e.g. [1]). The Heisenberg group $\mathbb{H}^{N}$ is identified with $\mathbb{R}^{2N+1}$ under the following group composition: for all $\xi= (x,y,t)$ and $\xi' = (x',y',t')$,

$$\xi\circ\xi' = \bigl(x+x', y+y', t+t'+2\bigl(x\cdot y' - x'\cdot y\bigr) \bigr), $$

where ‘⋅’ denotes the inner product in $\mathbb{R}^{N}$. For any $\xi\in\mathbb{H}^{N}$, the left translations on $\mathbb{H}^{N}$ is defined by

$$\tau_{\xi}: \mathbb{H}^{N} \to\mathbb{H}^{N},\qquad \tau_{\xi}\bigl(\xi'\bigr) = \xi\circ\xi'. $$

For $\lambda > 0$, a family of dilation on $\mathbb{H}^{N}$ is defined by

$$\delta_{\lambda}: \mathbb{H}^{N} \to\mathbb{H}^{N},\qquad \delta_{\lambda}(x,y,t) = \bigl(\lambda x,\lambda y,\lambda^{2} t \bigr). $$

The homogeneous dimension of $\mathbb{H}^{N}$ is $q = 2N+2$. For $\xi\in \mathbb{H}^{N}$, $\vert \xi \vert _{H}$ is the intrinsic distance of the point ξ to the origin, namely

$$\vert \xi \vert _{H} = \Biggl(\sum_{j=1}^{N} \bigl(x_{j}^{2} + y_{j}^{2} \bigr)^{2} + t^{2} \Biggr)^{\frac{1}{4}}. $$

The Kohn Laplacian $\Delta_{H}$ on $\mathbb{H}^{N}$ is defined as

$$\Delta_{H} = \sum_{j=1}^{N} \bigl(X_{j}^{2} + Y_{j}^{2}\bigr), $$

where

$$X_{j} = \frac{\partial}{\partial x_{j}} + 2y_{j}\frac{\partial}{\partial t},\qquad Y_{j} = \frac{\partial}{\partial y_{j}} - 2x_{j}\frac{\partial}{\partial t}. $$

For every $u\in C_{0}^{\infty}(\Omega)$, the subelliptic gradient is defined as

$$\nabla_{H} u = (X_{1}u, \ldots, X_{N}u, Y_{1}u,\ldots,Y_{N}u). $$

The closure of $C_{0}^{\infty}(\Omega)$ under the norm $\int_{\Omega} \vert \nabla_{H} \cdot \vert ^{2}\,d\xi$ is denoted by $S_{0}^{1,2}(\Omega)$. From [2, 3], we also know that the following Sobolev type inequality holds: there exists $C_{q}> 0$ such that

$$ \vert u\vert _{2q/(q-2)} \leq C_{q} \Vert u \Vert _{S_{0}^{1,2}(\mathbb{H}^{N})} \quad \mbox{for all } u\in S_{0}^{1,2}\bigl( \mathbb{H}^{N}\bigr), $$

(1.2)

where $\vert \cdot \vert _{2q/(q-2)}$ is the norm in $L^{2q/(q-2)}$. The number ${2q/(q-2)}:= 2^{\sharp}$ is the critical Sobolev exponent, since for a bounded domain Ω and $2 < p < {2q/(q-2)}$, the $S_{0}^{1,2}(\Omega)$ is compactly embedded into $L^{p}(\Omega)$, while this inclusion is only continuous if $p = 2q/(q-2)$.

There are several papers studying the existence and nonexistence of solutions of semilinear equations with Kohn Laplacian in the past two decades. For instance, Citti [4] studies the equation

$$ -\triangle_{H} u + au = u^{\frac{q+2}{q-2}} \quad \mbox{in } \Omega,\qquad u = 0 \quad \mbox{on } \partial\Omega, $$

(1.3)

where Ω is a smooth bounded domain in $\mathbb{H}^{N}$. Since (1.3) involves a nonlinearity of critical growth, Citti [4] has proven a representation formula for the Palais-Smale sequence and then proved the existence of one non-negative solution of (1.3) under suitable conditions of a. Some results of Liouville type for semilinear equations on the Heisenberg group have been studied by Birindelli et al. [5, 6]. Uguzzoni [7] has proven a nonexistence theorem for a semilinear Dirichlet problem involving critical nonlinearity on the half space of the Heisenberg group. Yamabe-type equations on the Heisenberg group have been studied in [8–10]. Garofalo et al. [11] have studied some other existence and nonexistence of solutions for the Kohn Laplace semilinear equations. Other existence and nonexistence results for elliptic problems on Heisenberg have been studied in [12–20]. Very recently, Han et al. [21] have proven a class of Hardy-Sobolev type inequalities on H-type group and got the existence of a nontrivial solution for a related equation. A multiplicity result related to noncontractible domain has been studied in [22]. But we do not see any multiplicity results as regards the semilinear equation with critical exponent on the Heisenberg group with general bounded domain.

The purpose of the present paper is to prove that under suitable assumptions on $K(\xi)$ and μ, the problem under consideration has at least two non-negative solutions. Here and subsequently, we say that $u\in S_{0}^{1,2}(\Omega)$ is a solution of (1.1) if and only if for any $\psi\in C_{0}^{\infty}(\Omega)$, we have

$$\int \bigl(\nabla_{H} u\nabla_{H}\psi- \mu \vert \xi \vert _{H}^{\alpha}u\psi \bigr)\,d\xi- \int K(\xi)\vert u \vert ^{2^{\sharp}-2}u\psi \,d\xi= 0. $$

$u\in S_{0}^{1,2}(\Omega)$ is said to be a non-negative solution of (1.1) if u is a solution and $u\geq0$ but $u\not\equiv0$. According to the Sobolev inequality [23], we know that the functional

$$L(u)= \frac{1}{2}\int \bigl(\vert \nabla_{H} u\vert ^{2} - \mu \vert \xi \vert _{H}^{\alpha} \vert u \vert ^{2} \bigr)\,d\xi- \frac{1}{2^{\sharp}}\int K(\xi)\vert u\vert ^{2^{\sharp}}\,d\xi $$

is well defined and $C^{1}$ on $S_{0}^{1,2}(\Omega)$. Note that from Lemma 2.4 (see Section 2) the eigenvalue problem

$$-\Delta_{H} u = \mu \vert \xi \vert _{H}^{\alpha}u,\quad u\in S_{0}^{1,2}(\Omega), $$

has a sequence of eigenvalues $0 < \mu_{1} < \mu_{2} \leq\mu_{3} \leq \cdots\leq\mu_{m} < \cdots$ , $\mu_{m} \to\infty$ as $m\to\infty$, with the first eigenvalue $\mu_{1}$ simple, and all the eigenvalues are of finite multiplicity. Up to a normalization, the first eigenfunction $e_{1}$ corresponding to $\mu_{1}$ is non-negative. The basic assumptions are:

• (A1) $0 < K(0)=\max_{\xi\in\bar{\Omega}} \vert K(\xi)\vert$ and there is $R>0$ such that for $\xi\in B(0,2R)$, $K(\xi)=K(0)+O(\vert \xi \vert _{H}^{\beta})$ with $2 +\alpha<\beta< q$;

• (A2) $\int_{\Omega}K(\xi)e_{1}^{2^{\ast}}\,d\xi< 0$, where $e_{1}>0$ is as mentioned before.

Our main results are

Theorem 1.1

Suppose that (A1) holds. If $\mu\in(0, \mu_{1})$, then (1.1) has at least one non-negative solution.

Theorem 1.2

Suppose that (A1) and (A2) hold.

• (1) If $\mu=\mu_{1}$, then (1.1) has at least one non-negative solution;

• (2) if $0<\alpha< \frac{q}{2} -3$, then there is $\mu_{*}>\mu_{1}$, such that for any $\mu\in(\mu_{1},\mu_{*})$, (1.1) has at least two non-negative solutions.

The proofs of Theorem 1.1 and Theorem 1.2 are based on critical point theory. Our idea originates from [24, 25]. More precisely, we will minimize the functional L over a suitable subset of $S_{0}^{1,2}(\Omega)$ according to the range of μ. However, since the embedding $S_{0}^{1,2}(\Omega)\hookrightarrow L^{2^{\sharp}}(\Omega)$ is not compact, the standard minimization argument cannot be applied directly. We have to estimate the minimum level of the functional L carefully such that it is contained in the range where the Palais-Smale ($(PS)$ for short, see Definition 2.3) condition holds. On getting one non-negative solution, we can modify the argument from [26]. However, in order to get the existence of a second solution, one needs a priori estimate about the property of the first solution. In [25], Drabek et al. overcome this difficulty by the fact that any solutions belong to $L^{\infty}$. While in [24], the author has managed to get two positive solutions by establishing an exact local behavior of positive solutions near singularity. But for the semilinear equation on Heisenberg group, the operator $-\Delta_{H}$ is degenerate. It is not easy to get the boundedness of the solution to semilinear equation with critical exponent. One of our contributions here is to estimate the integrals in a suitable way and do the energy estimates without the boundedness of the solution.

This paper is organized as follows. Section 2 contains some preliminaries. Particular attention is focused on several integral estimates for solutions of (1.1), which will play an important role in the study of multiple solutions of (1.1). The third and fourth sections are devoted to the proofs of Theorem 1.1 and Theorem 1.2, respectively.

2 Preliminaries

Throughout this paper, C, $C_{j}$ ($j=1, 2, \ldots$) will denote various positive constants whose exact value are not important. The dual space of a Banach space E is denoted by $E^{*}$. By $\vert \cdot \vert _{p}$ we denote the norm in $L^{p}(\Omega)$. $S_{0}^{1,2}(\mathbb{H}^{N})$ is the closure of $C_{0}^{\infty}(\mathbb{H}^{N})$ under the norm of $\int_{\mathbb{H}^{N}}\vert \nabla_{H} \cdot \vert ^{2}\,dx$. $B(\xi,r)$ is a ball centered at ξ with radius r. $O(\varepsilon^{m})$ denotes $\vert O(\varepsilon^{m})\vert /{\varepsilon^{m}} \leq C$ and $o(\varepsilon^{m})$ denotes $\vert o(\varepsilon^{m})\vert /{\varepsilon^{m}} \to0$ as $\varepsilon \to0$. All integrals are taken over Ω unless stated otherwise. The following minimization problem will be useful in what follows:

$$S = \inf \biggl\{ \int_{\mathbb{H}^{N}}\vert \nabla_{H} u \vert ^{2}\,d\xi; u\in S_{0}^{1,2}\bigl( \mathbb{H}^{N}\bigr), \int_{\mathbb{H}^{N}}\vert u\vert ^{2^{\sharp}}\,d\xi= 1 \biggr\} . $$

Jerison et al. [23] have proven that S is achieved by

$$\omega(x,y,t) = \frac{C_{0}}{(t^{2} + (1+\vert x\vert ^{2}+\vert y\vert ^{2})^{2})^{\frac{q-2}{4}}} $$

with suitable positive constant $C_{0}$. Moreover, $\omega(x,y,t)$ satisfies

$$ -\Delta_{H} u(\xi) = \bigl\vert u(\xi)\bigr\vert ^{2^{\sharp}-2}u(\xi),\quad \xi\in \mathbb{H}^{N}, u\in S_{0}^{1,2} \bigl(\mathbb{H}^{N}\bigr). $$

(2.1)

All non-negative solutions of (2.1) are of the form

$$\omega_{\lambda, \xi'} = \lambda^{\frac{q-2}{2}}\omega\bigl(\delta_{\lambda}\bigl(\tau^{-1}_{\xi '}\bigr)\bigr), \quad \lambda>0,\xi' \in\mathbb{H}^{N}. $$

Moreover,

$$\int_{\mathbb{H}^{N}}\vert \nabla_{H} \omega_{\lambda, \xi'} \vert ^{2}\,d\xi= \int_{\mathbb{H}^{N}}\vert \omega_{\lambda, \xi'}\vert ^{2^{\sharp}}\,d\xi= S^{\frac{q}{2}}. $$

Define a cut-off function $\phi(\xi)$ and denote $w_{\lambda}(\xi) = \lambda^{\frac{q-2}{2}}\omega(\delta_{\lambda}(\xi))$. Setting $v_{\lambda}(\xi) :=\phi(\xi)w_{\lambda}(\xi)$, one can have from direct computations (see e.g. [4]) that as $\lambda\to +\infty$,

$$ \int \vert v_{\lambda} \vert ^{2^{\sharp}}\,d\xi= S^{\frac{q}{2}} + O\bigl(\lambda^{-q}\bigr) $$

(2.2)

and

$$ \int \vert \nabla_{H} v_{\lambda} \vert ^{2}\,d\xi= S^{\frac{q}{2}} + O\bigl(\lambda^{-(q-2)}\bigr). $$

(2.3)

Using this idea, we can deduce the following lemma, which will play an important role in the proofs of Theorem 1.1 and Theorem 1.2.

Lemma 2.1

Let $v_{\lambda}$ be defined as above. If $2+ \alpha< \beta< q$, then as $\lambda\to+\infty$,

$$\begin{aligned}& \int \vert \xi \vert ^{\beta}_{H} \vert v_{\lambda} \vert ^{2^{\sharp}}\,d\xi= O\bigl(\lambda^{-\beta}\bigr); \\& \int \vert \xi \vert ^{\alpha}_{H} \vert v_{\lambda} \vert ^{2}\,d\xi= O\bigl(\lambda^{-(\alpha+2)}\bigr). \end{aligned}$$

Proof

Keep the definition of $v_{\lambda}$ in mind. We have

$$\begin{aligned} \int_{\Omega} \vert \xi \vert _{H}^{\beta} \vert v_{\lambda} \vert ^{2^{\sharp}}\,d \xi & =\int_{\vert \xi \vert _{H} < 2R}\vert \xi \vert _{H}^{\beta}\bigl(\lambda^{\frac{q-2}{2}}\omega\bigl(\delta _{\lambda}(\xi )\bigr) \bigr)^{2^{\sharp}}\,d\xi \\ & =\lambda^{-\beta}\int_{\vert \eta \vert _{H} < 2\lambda R}\vert \eta \vert _{H}^{\beta}\bigl(\omega(\eta) \bigr)^{2^{\sharp}}\,d\eta \\ & =\lambda^{-\beta} \biggl(\int_{\vert \eta \vert _{H} < 1}\vert \eta \vert _{H}^{\beta}\bigl(\omega(\eta) \bigr)^{2^{\sharp}}\,d\eta+ \int_{1< \vert \eta \vert _{H} < 2\lambda R}\vert \eta \vert _{H}^{\beta}\bigl(\omega(\eta) \bigr)^{2^{\sharp}}\,d\eta \biggr) \\ & \leq\lambda^{-\beta} \biggl(C + \int_{1}^{2\lambda R} \rho^{\beta-q -1}\,d\rho \biggr) \\ & = O\bigl(\bigl(\lambda^{-1}\bigr)^{\beta}\bigr) + O\bigl( \bigl(\lambda^{-1}\bigr)^{q}\bigr)=O\bigl(\bigl( \lambda^{-1}\bigr)^{\beta}\bigr)\quad \mbox{for } \lambda \mbox{ large enough,} \end{aligned} $$

where we have used the assumption of $\beta< q$. Similarly, we have

$$\begin{aligned} \int_{\Omega} \vert \xi \vert _{H}^{\alpha} \vert v_{\lambda} \vert ^{2}\,d\xi&=\int_{\vert \xi \vert _{H} < 2R} \vert \xi \vert _{H}^{\alpha}w^{2}_{\lambda}\bigl(\delta_{\lambda}(\xi)\bigr)\,d\xi \\ &=\lambda^{-2-\alpha}\int_{\vert \eta \vert _{H} < 2\lambda R}\vert \eta \vert _{H}^{\alpha}\omega^{2}(\eta)\,d\eta \\ &=\lambda^{-2-\alpha} \biggl(\int_{\vert \eta \vert _{H} < 1}\vert \eta \vert _{H}^{\alpha}\omega^{2}(\eta)\,d\eta+ \int _{1< \vert \eta \vert _{H} < 2\lambda R}\vert \eta \vert _{H}^{\alpha}\omega^{2}(\eta)\,d\eta \biggr) \\ &\leq\lambda^{-2-\alpha} \biggl(C + \int_{1}^{2\lambda R} \rho^{4-q+\alpha-1}\,d\rho \biggr) \\ &= O\bigl(\bigl(\lambda^{-1}\bigr)^{2+\alpha}\bigr) + O\bigl( \bigl(\lambda^{-1}\bigr)^{q-2}\bigr)\quad \mbox{for } \lambda \mbox{ large enough.} \end{aligned}$$

Therefore $0 < \alpha< q-4$ implies that for λ large enough,

$$\int_{\Omega} \vert \xi \vert _{H}^{\alpha} \vert v_{\lambda} \vert ^{2}\,d\xi= O\bigl( \lambda^{-(2+\alpha)}\bigr). $$

The proof is complete. □

Next, we prove a regularity result for the solutions of (1.1). The idea originates from Brezis-Kato [27]; see also Struwe [28]. The following lemma will play a key role in the process of studying a second non-negative solution of (1.1).

Lemma 2.2

If $u\in S_{0}^{1,2}(\Omega)$ is a solution of (1.1), then $u \in L^{r}(\Omega)$ for each $r\in(1, +\infty)$.

Proof

Since u is a weak solution of (1.1), we test the equation with a test function $\varphi= u\min\{\vert u\vert ^{2s}, m^{2}\}$, where $s\geq0$ and $m>1$. Integrating by parts we obtain

$$\begin{aligned} \int\nabla_{H} u\nabla_{H}\bigl(u\min\bigl\{ \vert u \vert ^{2s}, m^{2}\bigr\} \bigr)\,d\xi={}&\int \vert u \vert ^{2^{\sharp}}\min\bigl\{ \vert u\vert ^{2s}, m^{2}\bigr\} \,d\xi \\ & {}+ \mu\int \vert \xi \vert _{H}^{\alpha}u^{2} \min\bigl\{ \vert u\vert ^{2s}, m^{2}\bigr\} \,d\xi. \end{aligned}$$

For each sufficiently large $M > 0$, we deduce that

$$\begin{aligned} &\int\bigl\vert \nabla_{H}\bigl(u\min\bigl\{ \vert u\vert ^{s}, m\bigr\} \bigr)\bigr\vert \,d\xi \\ &\quad \leq(2s+2)\int \vert u\vert ^{2^{\sharp}}\min\bigl\{ \vert u\vert ^{2s}, m^{2}\bigr\} \,d\xi+ C\int u^{2}\min\bigl\{ \vert u\vert ^{2s}, m^{2}\bigr\} \,d\xi \\ &\quad =(2s+2)\int_{\vert u\vert \leq M}\vert u\vert ^{2^{\sharp}}\min \bigl\{ \vert u\vert ^{2s}, m^{2}\bigr\} \,d\xi+ C\int u^{2}\min\bigl\{ \vert u\vert ^{2s}, m^{2}\bigr\} \,d\xi, \\ &(2s+2)\int_{\vert u\vert > M}\vert u\vert ^{2^{\sharp}}\min\bigl\{ \vert u\vert ^{2s}, m^{2}\bigr\} \,d\xi \\ &\quad \leq(2s+2)\operatorname {meas}(\Omega)M^{2^{\sharp}+2s} + C\int u^{2}\min \bigl\{ \vert u\vert ^{2s}, m^{2}\bigr\} \,d\xi, \\ &(2s+2) \biggl(\int_{\vert u\vert > M}\vert u\vert ^{2^{\sharp}}\,d \xi \biggr)^{\frac{2^{\sharp}-2}{2^{\sharp}}} \biggl(\int \bigl(u\min\bigl\{ \vert u\vert ^{2s}, m^{2}\bigr\} d\bigr)^{2^{\sharp}}\,d\xi \biggr)^{\frac{2}{2^{\sharp}}} \\ &\quad \leq C +\frac{1}{2} \int\bigl\vert \nabla_{H}\bigl(u \min\bigl\{ \vert u\vert ^{s}, m\bigr\} \bigr)\bigr\vert ^{2}\,d\xi + C\int u^{2}\min\bigl\{ \vert u\vert ^{2s}, m^{2}\bigr\} \,d\xi, \end{aligned}$$

which implies that

$$\int\bigl\vert \nabla_{H}\bigl(u\min\bigl\{ \vert u\vert ^{s}, m \bigr\} \bigr)\bigr\vert \,d\xi\leq 4(s+1)\operatorname {meas}(\Omega)M^{2^{\sharp}+2s} + C_{1}\int u^{2}\min\bigl\{ \vert u\vert ^{2s}, m^{2}\bigr\} \,d\xi. $$

Letting $m\to+\infty$, we obtain

$$\int\bigl\vert \nabla_{H}\bigl(u\vert u\vert ^{s} \bigr)\bigr\vert ^{2}\,d\xi\leq4(s+1)\operatorname {meas}(\Omega )M^{2^{\sharp}+2s} + C_{1}\int \vert u\vert ^{2(s+1)}\,d\xi. $$

Now iterate, letting $s_{0} = 0$, $s_{j} + 1 = (s_{j-1} + 1)\frac{q}{q-2}$, if $j\geq1$, to obtain the conclusion. □

We end these preliminaries by the definition of the $(PS)$ conditions and an additional lemma.

Definition 2.3

Let $c \in\mathbb{R}$, E be a Banach space and $I \in C^{1}(E, \mathbb{R})$. We say that I satisfies $(PS)_{c}$ condition, if any sequence $(u_{n})_{n\in\mathbb{N}}$ in E such that $I(u_{n})\to c$ and $I'(u_{n})\to0$ has a convergent subsequence. If this holds for every $c \in \mathbb{R}$, we say that I satisfies the $(PS)$ condition.

Lemma 2.4

Let $\Omega\subset\mathbb{H}^{N}$ be a bounded open domain with smooth boundary. Then $S_{0}^{1,2}(\Omega)$ is continuously and compactly embedded to $L^{2}(\Omega, \vert \xi \vert _{H}^{\alpha}\,d\xi)$.

Proof

Since $\alpha> 0$, we can get the conclusion by a combination of [29], Lemma 3.2, and [30], Lemma 2.6. □

3 Existence of a non-negative solution

In this section, we will prove Theorem 1.1. The $0< \mu< \mu_{1}$ and (A1) will be assumed throughout this section. Define another functional

$$G(u) = \int \vert \nabla_{H} u\vert ^{2}\,d\xi- \mu\int \vert \xi \vert _{H}^{\alpha} \vert u\vert ^{2} \,d\xi -\int K(\xi)\vert u\vert ^{2^{\sharp}}\,d\xi,\quad u\in S_{0}^{1,2}(\Omega), $$

and denote the Nehari set

$$\mathcal{N}_{\mu}= \bigl\{ u\in S_{0}^{1,2}(\Omega) \backslash\{0\} : G(u) = 0 \bigr\} . $$

We have first of all the following.

Lemma 3.1

There is $\rho_{0} > 0$ such that $\Vert u\Vert \geq\rho_{0}$ for all $u\in\mathcal{N}_{\mu}$.

Proof

For any $u\in\mathcal{N}_{\mu}$, the assumption (A1) and the Sobolev inequality imply that

$$\Vert u\Vert ^{2} -\mu\int \vert \xi \vert _{H}^{\alpha} \vert u\vert ^{2}\,d\xi= \int K(\xi)\vert u\vert ^{2^{\sharp}} \,d\xi\leq K(0)\vert u\vert _{2^{\sharp}}^{2^{\sharp}}\leq K(0)S^{-\frac{2^{\sharp}}{2}}\Vert u\Vert ^{2^{\sharp}}. $$

Therefore $(1-{\mu\over \mu_{1}})\Vert u\Vert ^{2}\leq K(0)S^{-\frac{2^{\sharp}}{2}}\Vert u\Vert ^{2^{\sharp}}$. Hence we can choose

$$\rho_{0} = \biggl( \biggl(1-{\mu\over \mu_{1}} \biggr)K(0)^{-1}S^{\frac {2^{\sharp}}{2}} \biggr)^{\frac{1}{2^{\sharp}-2}} $$

such that Lemma 3.1 holds. □

Note that for any $u\in\mathcal{N}_{\mu}$,

$$L(u)={1\over q} \biggl(\Vert u\Vert ^{2}-\mu\int \vert \xi \vert _{H}^{\alpha} \vert u\vert ^{2} \,d\xi \biggr)={1\over q}\int K(\xi)\vert u\vert ^{2^{\sharp}} \,d\xi. $$

We define

$$ d_{1} = \inf_{u\in\mathcal{N}_{\mu}}L(u). $$

(3.1)

From Lemma 3.1, one sees immediately that there is a positive constant $C_{0}$ such that $c_{1} \geq C_{0} > 0$. Next, we have the following lemma.

Lemma 3.2

There is a sequence $(u_{n})_{n\in\mathbb{N}}\subset\mathcal{N}_{\mu}$ such that

$$ L(u_{n})\to c_{1},\qquad L'(u_{n}) \to0\quad \textit{in } \bigl(S_{0}^{1,2}(\Omega)\bigr)^{*}. $$

(3.2)

Proof

Let $(\tilde{u}_{n})_{n\in\mathbb{N}}\subset\mathcal{N}_{\mu}$ be a minimizing sequence of (3.1). By the Ekeland variational principle, we can find a sequence $(u_{n})_{n\in\mathbb{N}}\subset\mathcal{N}_{\mu}$ such that

$$L(u_{n})\to c_{1},\qquad L' \big|_{\mathcal{N}_{\mu}}(u_{n}) \to 0, $$

where $L' |_{\mathcal{N}_{\mu}}$ is the derivative of L restricted to $\mathcal{N}_{\mu}$. The Lagrange multiplier rule implies that there is $a_{n}\in\mathbb{R}$ such that

$$L'(u_{n})-a_{n}G'(u_{n}) \to0\quad \mbox{and}\quad \bigl\langle L'(u_{n}),u_{n} \bigr\rangle = a_{n}\bigl\langle G'(u_{n}),u_{n} \bigr\rangle . $$

Since $u_{n}\in\mathcal{N}_{\mu}$, one deduces that $\langle G'(u_{n}),u_{n}\rangle\neq0$ and then $\langle L'(u_{n}),u_{n}\rangle=0$. Hence $a_{n}=0$. The conclusion follows. □

Lemma 3.3

Let $(u_{n})_{n\in\mathbb{N}}\subset\mathcal{N}_{\mu}$ be as in Lemma  3.2. If $c_{1} < \frac{1}{q}{{S^{\frac{q}{2}}}\over { \vert K\vert _{\infty}^{(q-2)/2}}}$, then $(u_{n})_{n\in\mathbb{N}}$ possesses a convergent subsequence in $S_{0}^{1,2}(\Omega)$.

Proof

The proof can be proceeded by following the same lines as [24], Lemma 3.3; see also Drabek [25]. □

Lemma 3.4

Under the assumptions of Theorem  1.1, we have

$$c_{1} < \frac{1}{q}\frac{S^{\frac{q}{2}}}{\vert K\vert _{\infty}^{\frac{q-2}{2}}}. $$

Proof

It suffices to find some $u\in\mathcal{N}_{\mu}$ such that $L(u)<\frac{1}{q}{{S^{\frac{q}{2}}}\over { \vert K\vert _{\infty}^{(q-2)/2}}}$. Let $v_{\lambda}$ be defined as in Section 2, we have from direct computation that there is a $t_{0}$ with

$$t_{0}= \biggl(\frac{\Vert v_{\lambda} \Vert ^{2}-\mu\int \vert \xi \vert _{H}^{\alpha} \vert v_{\lambda} \vert ^{2}\,d\xi }{\int K(\xi)\vert v_{\lambda} \vert ^{2^{\sharp}}\,d\xi} \biggr)^{\frac{q-2}{4}} $$

such that $t_{0}v_{\lambda}\in\mathcal{N}_{\mu}$. Moreover, we obtain from (2.2), (2.3), and Lemma 2.1

$$\begin{aligned} L(t_{0}v_{\lambda})&=\frac{1}{q}t_{0}^{2} \biggl(\Vert v_{\lambda} \Vert ^{2} - \mu\int \vert \xi \vert _{H}^{\alpha} \vert v_{\lambda} \vert ^{2}\,d\xi \biggr) \\ &=\frac{1}{q} \biggl(\Vert v_{\lambda} \Vert ^{2} - \mu\int \vert \xi \vert _{H}^{\alpha} \vert v_{\lambda} \vert ^{2}\,d\xi \biggr)^{\frac{q}{2}} \biggl(\int K(\xi)\vert v_{\lambda} \vert ^{2^{\sharp}}\,d\xi \biggr)^{\frac {2-q}{2}} \\ &=\frac{1}{q} \bigl(S^{\frac{q}{2}} + O\bigl(\bigl(\lambda^{-1} \bigr)^{q-2}\bigr)- O\bigl(\bigl(\lambda^{-1} \bigr)^{\alpha+ 2}\bigr) \bigr)^{\frac{q}{2}} \bigl(\vert K\vert _{\infty}S^{\frac{q}{2}} + O\bigl(\bigl(\lambda^{-1} \bigr)^{\beta}\bigr) + O\bigl(\bigl(\lambda^{-1} \bigr)^{q}\bigr) \bigr)^{\frac {2-q}{2}} \\ &< \frac{1}{q}\frac{S^{\frac{q}{2}}}{\vert K\vert _{\infty}^{\frac{q-2}{2}}}\quad \mbox{for } \lambda \mbox{ large enough}, \end{aligned} $$

where we have used the fact that $2+\alpha< \beta$ and $2+\alpha< q-2$. □

Proof of Theorem 1.1

Combining Lemmas 3.1-3.4, we have an $w_{1}\in\mathcal{N}_{\mu}$ which leads to $c_{1}$. Since if $(u_{n})_{n\in\mathbb{N}}$ minimize L over $\mathcal{N}_{\mu}$, then so does $(\vert u_{n}\vert )_{n\in\mathbb{N}}$, we can assume that $w_{1}$ is a non-negative critical point of L. Hence $w_{1}$ is a non-negative solution of (1.1). □

4 Existence results for $\mu\geq\mu_{1}$

In this section, we will prove Theorem 1.2. The multiplicity result can be obtained by minimizing L over different subset of $S_{0}^{1,2}(\Omega)$. The idea originates from Drabek et al. [25], where the authors study an indefinite problem in the classical Euclidean space $\mathbb{R}^{N}$, and some refinement from Chen [24], where the author studied an indefinite problem with singular term. The additional assumption (A2) will hold throughout this section. Since we will prove Theorem 1.2 for different μ, we denote $L_{\mu}\equiv L$ from now on. Define the following Nehari type set:

$$ \mathcal{M}_{\mu}= \bigl\{ u\in S_{0}^{1,2}( \Omega) : G(u)\equiv \bigl\langle L_{\mu}'(u),u\bigr\rangle = 0 \bigr\} . $$

(4.1)

We further split $\mathcal{M}_{\mu}$ into three disjoint subsets,

$$\begin{aligned}& \begin{aligned} \mathcal{M}_{\mu}^{+} &= \bigl\{ u\in\mathcal {M}_{\mu}: \bigl\langle G'(u),u\bigr\rangle > 0 \bigr\} \\ &= \biggl\{ u\in\mathcal{M}_{\mu}: \Vert u\Vert ^{2}-\mu \int \vert \xi \vert _{H}^{\alpha} \vert u\vert ^{2}\,d\xi > \bigl(2^{\sharp}-1\bigr)\int K(\xi)\vert u\vert ^{2^{\sharp}}\,d\xi \biggr\} \\ &= \biggl\{ u\in\mathcal{M}_{\mu}: \int K(\xi)\vert u\vert ^{2^{\sharp}}\,d\xi< 0 \biggr\} , \end{aligned} \\& \begin{aligned} \mathcal{M}_{\mu}^{0} &= \bigl\{ u \in\mathcal {M}_{\mu}: \bigl\langle G'(u),u\bigr\rangle = 0 \bigr\} \\ &= \biggl\{ u\in\mathcal{M}_{\mu}: \Vert u\Vert ^{2}-\mu \int \vert \xi \vert _{H}^{\alpha} \vert u\vert ^{2}\,d\xi = \bigl(2^{\sharp}-1\bigr)\int K(\xi)\vert u\vert ^{2^{\sharp}}\,d\xi \biggr\} \\ &= \biggl\{ u\in\mathcal{M}_{\mu}: \int K(\xi)\vert u\vert ^{2^{\sharp}}\,d\xi= 0 \biggr\} , \end{aligned} \end{aligned}$$

and

$$\begin{aligned} \mathcal{M}_{\mu}^{-} &= \bigl\{ u\in\mathcal {M}_{\mu}: \bigl\langle G'(u),u\bigr\rangle < 0 \bigr\} \\ &= \biggl\{ u\in\mathcal{M}_{\mu}: \Vert u\Vert ^{2}-\mu \int \vert \xi \vert _{H}^{\alpha} \vert u\vert ^{2}\,d\xi < \bigl(2^{\sharp}-1\bigr)\int K(\xi)\vert u\vert ^{2^{\sharp}}\,d\xi \biggr\} \\ &= \biggl\{ u\in\mathcal{M}_{\mu}: \int K(\xi)\vert u\vert ^{2^{\sharp}}\,d\xi > 0 \biggr\} . \end{aligned}$$

Remark 4.1

Now some remarks are in order.

• (1) $K_{+}\neq0$ implies that $\mathcal{M}_{\mu}^{-}\neq\emptyset$. Indeed, since

  $$\Vert v_{\lambda} \Vert ^{2}-\mu\int \vert \xi \vert _{H}^{\alpha} \vert v_{\lambda} \vert ^{2}\,d \xi= S^{\frac{q}{2}} + O\bigl(\bigl(\lambda^{-1}\bigr)^{q-2} \bigr)- O\bigl(\bigl(\lambda^{-1}\bigr)^{2+\alpha}\bigr) > 0 $$

  for λ large enough, we know that $t_{0}v_{\lambda}\in\mathcal{M}_{\mu}^{-}$ with

  $$t_{0}= \biggl(\frac{\Vert v_{\lambda} \Vert ^{2} - \mu\int \vert \xi \vert _{H}^{\alpha} \vert v_{\lambda} \vert ^{2}\,d\xi}{\int K(\xi)\vert v_{\lambda} \vert ^{2^{\sharp}}\,d\xi} \biggr)^{\frac{(q-2)}{4}}. $$

• (2) $\mathcal{M}_{\mu}$ and $\mathcal{M}_{\mu}^{0}$ are closed in $S_{0}^{1,2}(\Omega)$.

• (3) For $\mu\in(0,\mu_{1}]$, $\mathcal{M}_{\mu}^{+}=\emptyset$. However, for $\mu> \mu_{1}$, $\mathcal{M}_{\mu}^{+}\neq\emptyset$. Indeed, we obtain from $\int K(\xi)e_{1}^{2^{\sharp}}\,d\xi< 0$ and direct computation

  $$ \biggl(\frac{\Vert e_{1}\Vert ^{2}-\mu\int \vert \xi \vert _{H}^{\alpha} \vert e_{1}\vert ^{2}\,d\xi}{\int K(\xi)e_{1}^{2^{\sharp}}\,d\xi} \biggr)^{\frac{q-2}{4}}e_{1}\in \mathcal{M}_{\mu}^{+}. $$

  (4.2)

In view of Remark 4.1, we will prove Theorem 1.2 in the following outline. For $\mu=\mu_{1}$, we will minimize $L_{\mu}$ on $\mathcal{M}_{\mu_{1}}^{-}$ and prove the minimizer can be achieved and can be chosen to non-negative. For $\mu> \mu_{1}$, we will minimize $L_{\mu}$ on $\mathcal{M}_{\mu}^{+}$ and $\mathcal{M}_{\mu}^{-}$, respectively and show the minimizers exist. Then we will get two non-negative solutions of (1.1). The following lemmas are useful in what follows.

Lemma 4.2

There is $\tau>0$ such that $\Vert \frac{u}{\Vert u\Vert } - e_{1}\Vert \geq\tau$ for all $u\in\mathcal{M}_{\mu}^{-}$ with $\mu>0$.

Proof

Suppose the contrary. There are $\tilde{\mu}_{n}$ and $u_{n}\in\mathcal{M}_{\tilde{\mu}_{n}}^{-}$ such that $v_{n} := {{u_{n}}\over { \Vert u_{n}\Vert }}\to e_{1}$. Using the fact that

$$0\leq \Vert u_{n}\Vert ^{2}-\tilde{\mu}_{n} \int \vert \xi \vert _{H}^{\alpha} \vert u_{n} \vert ^{2}\,d\xi< \bigl(2^{\sharp}-1\bigr)\int K(\xi)\vert u_{n}\vert ^{2^{\sharp}}\,d\xi, $$

and the strong convergence of $v_{n}$ to $e_{1}$, we deduce that

$$0\leq \Vert v_{n}\Vert ^{2}-\tilde{\mu}_{n} \int \vert \xi \vert _{H}^{\alpha} \vert v_{n} \vert ^{2}\,d\xi< \bigl(2^{\sharp}-1\bigr) \biggl(\int K(\xi) \vert v_{n}\vert ^{2^{\sharp}}\,d\xi \biggr)\Vert u_{n}\Vert ^{2^{\sharp}-2}. $$

Hence one obtains

$$0\leq \bigl(2^{\sharp}-1 \bigr)\int K(\xi)\vert v_{n}\vert ^{2^{\sharp}}\,d\xi\to \bigl(2^{\sharp}-1 \bigr)\int K( \xi)e_{1}^{2^{\sharp}}\,d\xi< 0, $$

which is a contradiction. □

Lemma 4.3

For τ given in Lemma  4.2, there is a $\mu_{*1}>\mu_{1}$ such that $\Vert u\Vert ^{2} \geq\mu_{*1}\int \vert \xi \vert _{H}^{\alpha} \vert u\vert ^{2}\,d\xi$ for any u with $\Vert u\Vert =1$ and $\Vert \vert u\vert -e_{1}\Vert \geq\tau$.

Proof

Arguing by a contradiction, we assume that there are $\Vert u_{n}\Vert =1$ with $\Vert u_{n}-e_{1}\Vert \geq\tau$ and $\tilde{\mu}_{n}\to\mu_{1}$ with $\tilde{\mu}_{n} > \mu_{1}$ such that $\Vert u_{n}\Vert ^{2} = \tilde{\mu}_{n}\int \vert \xi \vert _{H}^{\alpha} \vert u_{n}\vert ^{2}\,d\xi$. Going if necessary to a subsequence, still denoted by $(u_{n})_{n\in\mathbb{N}}$, we may assume that $u_{n}\rightharpoonup u_{0}$ in $S_{0}^{1,2}(\Omega)$ and therefore $u_{n}\to u_{0}$ in $L^{2}(\Omega, \vert \xi \vert _{H}^{\alpha}\,d\xi)$ (note that we have from Lemma 2.4 that $\int \vert \xi \vert _{H}^{\alpha} \vert u_{n} -u_{0}\vert ^{2}\,d\xi\to0$ as $n\to\infty$). Combining this with $\tilde{\mu }_{n}\to\mu_{1}$ and $\Vert u\Vert ^{2} -\mu_{1}\int \vert \xi \vert _{H}^{\alpha} \vert u\vert ^{2}\,d\xi\geq0$ for any $u\in S_{0}^{1,2}(\Omega)$, we obtain

$$ \begin{aligned}[b] 0&\leq \Vert u_{0}\Vert ^{2}- \mu_{1}\int \vert \xi \vert _{H}^{\alpha} \vert u_{0}\vert ^{2}\,d\xi \\ &\leq \lim_{n\to\infty} \biggl(\Vert u_{n}\Vert ^{2}-\tilde{\mu}_{n}\int \vert \xi \vert _{H}^{\alpha} \vert u_{n}\vert ^{2}\,d \xi \biggr)=0. \end{aligned} $$

(4.3)

If $u_{0}=0$, then we conclude from

$$\Vert u_{n}\Vert ^{2} = \tilde{\mu}_{n}\int \vert \xi \vert _{H}^{\alpha} \vert u_{n}\vert ^{2}\,d\xi\to \mu_{1}\int \vert \xi \vert _{H}^{\alpha} \vert u_{0}\vert ^{2}\,d\xi $$

that $\Vert u_{n}\Vert ^{2}\to0$, which contradicts $\Vert u_{n}\Vert = 1$. Assume $u_{0}\neq0$, then (4.3) and the variational characterization of $\mu_{1}$ imply $u_{0}=te_{1}$ for some $t\neq0$. From

$$ \begin{aligned}[b] 0&\leq \Vert te_{1} \Vert ^{2} -\mu_{1}\int \vert \xi \vert _{H}^{\alpha} \vert te_{1}\vert ^{2}\,d \xi\leq\lim_{n\to\infty} \biggl(\Vert u_{n}\Vert ^{2}-\tilde{\mu}_{n}\int \vert \xi \vert _{H}^{\alpha} \vert u_{n}\vert ^{2}\,d \xi \biggr) \\ & =\lim_{n\to\infty} \Vert u_{n}\Vert ^{2}- \mu_{1}\int \vert \xi \vert _{H}^{\alpha} \vert te_{1}\vert ^{2}\,d\xi=0, \end{aligned} $$

(4.4)

we have $\lim_{n\to\infty} \Vert u_{n}\Vert ^{2}=\Vert te_{1}\Vert ^{2}$. Hence

$$ \Vert u_{n}-te_{1}\Vert ^{2}= \Vert u_{n}\Vert ^{2}-\Vert te_{1}\Vert ^{2}- 2\langle u_{n},te_{1}\rangle\to0. $$

(4.5)

It follows that $u_{n}\to te_{1}$ and $t=1$. But this is impossible. The proof is complete. □

Lemma 4.4

For any $\mu\in(\mu_{1}, \mu_{*1})$, $\mathcal{M}_{\mu}^{-}$ is closed in $S_{0}^{1,2}(\Omega)$ and open in $\mathcal{M}_{\mu}$.

Proof

The openness in $\mathcal{M}_{\mu}$ is obvious. For the closedness, we argue by a contradiction. Suppose for $u_{n}\in\mathcal{M}_{\mu}^{-}$, $u_{n}\to u_{0}$ strongly in $S_{0}^{1,2}(\Omega)$ with $u_{0}\notin\mathcal{M}_{\mu}^{-}$. Then $u_{0}\in\mathcal{M}_{\mu}^{0}$, or equivalently $\int K(\xi)\vert u_{0}\vert ^{2^{\sharp}}\,d\xi=0$. From $u_{n}\in\mathcal{M}_{\mu}^{-}$, we deduce that as $n\to\infty$,

$$ \begin{aligned}[b] 0&\leq \Vert u_{n} \Vert ^{2}-\mu\int \vert \xi \vert _{H}^{\alpha} \vert u_{n}\vert ^{2}\,d\xi< \bigl(2^{\sharp}-1 \bigr)\int K(\xi)\vert u_{n}\vert ^{2^{\sharp}}\,d\xi \\ &\to\int K(\xi)\vert u_{0}\vert ^{2^{\ast}}\,d\xi=0. \end{aligned} $$

(4.6)

Denote $v_{n}=u_{n}/\Vert u_{n}\Vert$ and divide (4.6) by $\Vert u_{n}\Vert ^{2}$. Using the fact that $u_{n}\in\mathcal{M}_{\mu}^{-}$, $\Vert v_{n}\Vert =1$ and Lemma 4.2, Lemma 4.3, we obtain

$$ 0\leq (\mu_{*1}-\mu )\int \vert \xi \vert _{H}^{\alpha} \vert v_{n}\vert ^{2}\,d \xi \leq \Vert v_{n}\Vert ^{2}-\mu\int \vert \xi \vert _{H}^{\alpha} \vert v_{n}\vert ^{2}\,d \xi\to0. $$

(4.7)

It follows that $v_{n}\to0$ strongly in $L^{2}(\Omega, \vert \xi \vert _{H}^{\alpha}\,d\xi)$. Therefore by (4.7), one gets $\Vert v_{n}\Vert \to0$, which contradicts the fact that $\Vert v_{n}\Vert =1$. □

Lemma 4.5

There is $\mu_{*2} > \mu_{1}$ such that for any $\mu\in(\mu_{1}, \mu_{*2})$, $\mathcal{M}_{\mu}^{+}$ is bounded in $S_{0}^{1,2}(\Omega)$.

Proof

Suppose the contrary, there are $\tilde{\mu}_{n} > \mu_{1}$ and $u_{n}\in\mathcal{M}_{\tilde{\mu}_{n}}^{+}$ such that $\tilde{\mu}_{n} \to\mu_{1}$ and $\Vert u_{n}\Vert \to+\infty$ as $n\to\infty$. Note that $u_{n}\in\mathcal{M}_{\tilde{\mu}_{n}}^{+}$ implies that

$$ \begin{aligned}[b] 0&> \biggl(\Vert u_{n} \Vert ^{2} -\tilde{\mu}_{n}\int \vert \xi \vert _{H}^{\alpha} \vert u_{n}\vert ^{2}\,d \xi \biggr)> \bigl(2^{\sharp}-1 \bigr)\int K(\xi)\vert u_{n} \vert ^{2^{\sharp}}\,d\xi \\ &= \bigl(2^{\sharp}-1 \bigr) \biggl(\Vert u_{n}\Vert ^{2} -\tilde{\mu}_{n}\int \vert \xi \vert _{H}^{\alpha} \vert u_{n}\vert ^{2}\,d \xi \biggr). \end{aligned} $$

(4.8)

Dividing (4.8) by $\Vert u_{n}\Vert ^{2}$ and letting $v_{n}=u_{n}/\Vert u_{n}\Vert$, we obtain from $\tilde{\mu}_{n} \to\mu_{1}$

$$ \int K(\xi)\vert v_{n}\vert ^{2^{\sharp}}\,d\xi \to0. $$

(4.9)

On the other hand, from $\Vert v_{n}\Vert =1$, we may assume that there is a subsequence of $(v_{n})_{n\in\mathbb{N}}$, still denoted by $(v_{n})_{n\in \mathbb{N}}$ such that $v_{n}\rightharpoonup v_{0}$ weakly in $S_{0}^{1,2}(\Omega)$. Then using (4.8) and an argument similar to those in the proof of (4.3) that $v_{0}=te_{1}$ for some $t\neq0$. The same argument as in (4.4) and (4.5) lets us arrive at $v_{n}\to te_{1}$ strongly in $S_{0}^{1,2}(\Omega)$. Thus as $n\to\infty$, we get

$$\int K(\xi)\vert v_{n}\vert ^{2^{\sharp}}\,d\xi\to\int K(\xi) \vert te_{1}\vert ^{2^{\sharp}}\,d\xi< 0, $$

which is a contradiction to (4.9). The proof is complete. □

We are now in a position to prove the existence of one non-negative solution of (1.1) in the case of $\mu=\mu_{1}$.

Proof of (i) of Theorem 1.2

As pointed out in Remark 4.1, when $\mu= \mu_{1}$, $\mathcal{M}_{\mu_{1}}^{+}=\emptyset$. Hence we consider the minimization problem

$$ c_{2}=\inf_{u\in\mathcal{M}_{\mu_{1}}^{-}}L_{\mu_{1}}(u). $$

(4.10)

Note that $((\Vert v_{\lambda} \Vert ^{2} -\mu\int \vert \xi \vert _{H}^{\alpha} \vert v_{\lambda} \vert ^{2}\,d\xi)/\int K(\xi)\vert v_{\lambda} \vert ^{2^{\sharp}} )^{(q-2)/4}v_{\lambda}\in\mathcal {M}_{\mu_{1}}^{-}$, we can see from an argument similar to the proofs of Lemmas 3.2-3.4 that $c_{2}$ is achieved by some $w_{2}$. It then follows that $w_{2}$ is a solution of (1.1) with $\mu=\mu_{1}$. Moreover, $w_{2}$ can be chosen to be non-negative. The proof is complete. □

Next we turn to the case of $\mu> \mu_{1}$. Let $d_{1}=\inf_{u\in\mathcal{M}_{\mu}}L_{\mu}(u)$. From the previous lemma, $L_{\mu}$ is bounded from below on $\mathcal{M}_{\mu}^{+}$ for $\mu\in(\mu_{1},\mu_{*2})$. Since $te_{1}\in\mathcal{M}_{\mu}^{+}$ when $\mu> \mu_{1}$, the infimum of $L_{\mu}$ on $\mathcal{M}_{\mu}^{+}$ must be negative. The characterization of $\mathcal{M}_{\mu}$ (see the beginning of Section 4) implies that $d_{1}=\inf_{u\in\mathcal{M}^{+}_{\mu}}L_{\mu}(u)$. Moreover, we have the following lemma.

Lemma 4.6

For $\mu_{1}<\mu<\min\{\mu_{*1},\mu_{*2}\}$, $d_{1}$ is obtained by some $u_{*}\in\mathcal{M}_{\mu}^{+}$, which define a non-negative solution of (1.1).

Proof

Similar to the previous proof, we know that there is $u_{*}\in\mathcal{M}_{\mu}$ such that $L_{\mu}(u_{*})=d_{1}$. Moreover, $u_{*}$ solves (1.1) and can be chosen to be non-negative. Since $d_{1} < 0$ and $L_{\mu}(u)=0$ for $u\in\mathcal{M}_{\mu}^{0}$ and $L_{\mu}(u)>0$ for $u\in\mathcal{M}_{\mu}^{-}$, we can conclude that $u_{*}\in\mathcal{M}_{\mu}^{+}$. □

Let

$$d_{2}=\inf_{u\in\mathcal{M}_{\mu}^{-}}L_{\mu}(u). $$

Lemma 4.7

For $\mu_{1}<\mu<\min\{\mu_{*1},\mu_{*2}\}$, there is a sequence $(u_{n})_{n\in\mathbb{N}}\subset\mathcal{M}_{\mu}^{-}$ such that $L_{\mu}(u_{n})\to d_{2}$, $L_{\mu}'(u_{n})\to0$, and if the $d_{2} < d_{1}+{1\over q}{{S^{\frac{q}{2}}}\over { \vert K\vert _{\infty}^{(q-2)/2}}}$, then $(u_{n})_{n\in \mathbb{N}}$ possesses a convergent subsequence in $S_{0}^{1,2}(\Omega)$.

Proof

The idea of the proof is the same as [24], Lemma 4.7; see also [25]. We only outline the proof here. Similar to the proof in Lemma 3.2, there is a sequence $(u_{n})_{n\in\mathbb{N}}\subset\mathcal{M}_{\mu}^{-}$ such that

$$L_{\mu}(u_{n})\to d_{2},\qquad L_{\mu}'(u_{n}) \to0\quad \mbox{in } \bigl(S_{0}^{1,2}(\Omega)\bigr)^{*}. $$

We first claim that $(u_{n})_{n\in\mathbb{N}}$ is bounded in $S_{0}^{1,2}(\Omega)$. Indeed if $\Vert u_{n}\Vert \to\infty$, we denote $v_{n}=u_{n}/\Vert u_{n}\Vert$, then $\Vert v_{n}\Vert =1$. From $u_{n}\in\mathcal{M}_{\mu}^{-}$, we have

$$ \begin{aligned}[b] 0&\leq\int K(\xi)\vert u_{n}\vert ^{2^{\sharp}}\,d\xi \\ &=\Vert u_{n}\Vert ^{2}-\mu\int \vert \xi \vert _{H}^{\alpha} \vert u_{n}\vert ^{2}\,d \xi< \bigl(2^{\sharp}-1 \bigr)\int K(\xi)\vert u_{n}\vert ^{2^{\sharp}}\,d\xi. \end{aligned} $$

(4.11)

Dividing (4.11) by $\Vert u_{n}\Vert ^{2}$, we get

$$\begin{aligned} 0&\leq(\mu_{*1}-\mu)\int \vert \xi \vert _{H}^{\alpha} \vert v_{n}\vert ^{2}\,d \xi \leq \Vert v_{n}\Vert ^{2}-\mu\int \vert \xi \vert _{H}^{\alpha} \vert v_{n}\vert ^{2}\,d \xi \\ &= \Vert u_{n}\Vert ^{2^{\sharp}-2}\int K(\xi)\vert v_{n}\vert ^{2^{\sharp}}\,d\xi\to0. \end{aligned} $$

Therefore, $v_{n} \to 0$ strongly in $L^{2}(\Omega, \vert \xi \vert _{H}^{\alpha}\,d\xi)$ and hence $\Vert v_{n}\Vert ^{2}\to0$, which contradicts $\Vert v_{n}\Vert =1$. Thus $(u_{n})_{n\in\mathbb{N}}$ is bounded in $S_{0}^{1,2}(\Omega)$.

Going if necessary to a subsequence, we may assume that $u_{n}$ converges to u weakly in $S_{0}^{1,2}(\Omega)$ and almost everywhere in Ω. Moreover, $\nabla_{H} u_{n}\to\nabla_{H} u$ a.e. in Ω. Combining these with $L_{\mu}'(u_{n})\to0$ we have $L_{\mu}'(u)=0$. In particular, we have $u\in\mathcal{M}_{\mu}$. Hence

$$L_{\mu}(u)=\frac{1}{ q} \biggl(\Vert u\Vert ^{2}- \mu\int \vert \xi \vert _{H}^{\alpha} \vert u\vert ^{2}\,d\xi \biggr) \geq d_{1}. $$

If $u_{n}\to u$ strongly in $S_{0}^{1,2}(\Omega)$, then we complete the proof. If $u_{n}$ does not converge strongly to u in $S_{0}^{1,2}(\Omega )$, then we denote $\tilde{u}_{n} = u_{n} - u$. From $L'_{\mu}(u_{n}) = 0$, we can deduce that, for n large enough,

$$\int \vert \nabla_{H} \tilde{u}_{n}\vert ^{2} \,d\xi- \int K(\xi)\vert \tilde {u}_{n}\vert ^{2^{\sharp}}\,d\xi= o(1). $$

Suppose that $\int \vert \tilde{u}_{n}\vert ^{2^{\sharp}}\,d\xi\not\to0$ as $n\to \infty$, we may deduce from the Sobolev inequality (1.2) that

$$L_{\mu}(\tilde{u}_{n}) \geq\frac{1}{q} {{S^{\frac{q}{2}}}\over {\vert K\vert _{\infty}^{(q-2)/2}}}. $$

Therefore we obtain from the Brezis-Lieb lemma again for n large enough

$$\begin{aligned} d_{2} + o(1) &= L_{\mu}(u_{n}) \geq\frac {1}{q} \biggl(\Vert u\Vert ^{2}-\mu\int \vert \xi \vert _{H}^{\alpha} \vert u\vert ^{2}\,d\xi \biggr) + L_{\mu}(\tilde {u}_{n}) +o(1) \\ &\geq d_{1}+ {1\over q}{{S^{q/2}}\over { \vert K\vert _{\infty}^{(q-2)/2}}}, \end{aligned} $$

which is a contradiction. Thus we can conclude that $u_{n}\to u$ strongly in $S_{0}^{1,2}(\Omega)$. □

Lemma 4.8

There is $\mu_{*}>\mu_{1}$ such that for any $\mu\in(\mu_{1}, \mu_{*})$, the $d_{2} < d_{1}+{1\over q}\frac{S^{\frac{q}{2}}}{\vert K\vert _{\infty}^{(q-2)/2}}$.

In order to prove Lemma 4.8, we need some further lemmas, which play a key role in the proof of Lemma 4.8. It is Lemma  4.9 and Lemma  4.10 that we need to address the regularity for the solution of (1.1).

Lemma 4.9

Let w be a non-negative solution of (1.1). If $0 < \alpha< \frac{q}{2} - 3$, then for λ large enough,

$$\int w_{1}^{2^{\sharp}-1}v_{\lambda}\,d\xi= o\bigl( \lambda^{-\frac{q-2}{2}}\bigr)\quad \textit{and}\quad \int w_{1}(v_{\lambda})^{2^{\sharp}-1} \,d\xi= o\bigl(\bigl(\lambda^{-1}\bigr)^{2+\alpha}\bigr). $$

Proof

Since $w_{1}\in L^{r}(\Omega)$ for any $r\in(1,\infty)$, we obtain from the Hölder inequality

$$\int w_{1}^{2^{\sharp}-1}v_{\lambda}\,d\xi\leq \biggl(\int v_{\lambda}^{\gamma}\,d\xi \biggr)^{\frac{1}{\gamma}} \biggl(\int w_{1}^{\frac{(2^{\sharp}-1)\gamma}{\gamma-1}}\,d\xi \biggr)^{\frac{\gamma-1}{\gamma}}, $$

where $\gamma > 1$ and $q > (q-2)\gamma$. Note that

$$\begin{aligned} \int v_{\lambda}^{\gamma}\,d\xi&= \int _{\vert \xi \vert _{H} < 2R} \bigl(w_{\lambda}(\xi) \bigr)^{\gamma}d \xi= \lambda^{\frac{(q-2)\gamma}{2}}\int_{\vert \xi \vert _{H} < 2R} \bigl(w\bigl( \delta_{\lambda}(\xi)\bigr) \bigr)^{\gamma}\,d\xi \\ &=\lambda^{\frac{(q-2)\gamma}{2}-q}\int_{\vert \eta \vert _{H} < 2\lambda R} \bigl(w(\eta) \bigr)^{\gamma}d\eta \\ &=\lambda^{\frac{(q-2)\gamma}{2}-q} \biggl( C + \int_{1}^{2\lambda R} \rho^{-1+q-(q-2)\gamma}\,d\rho \biggr). \end{aligned} $$

From the choice of γ, we have

$$\int v_{\lambda}^{\gamma}\,d\xi= C\cdot\lambda^{\frac{q-2}{2}\gamma- q} + C \cdot\lambda^{-\frac{q-2}{2}\gamma}. $$

Therefore as λ is sufficiently large, one deduces that

$$\int w_{1}^{2^{\sharp}-1}v_{\lambda}\,d\xi= o\bigl( \lambda^{-\frac{q-2}{2}}\bigr). $$

Similarly, we can use the regularity of $w_{1}$ to prove that as λ is large enough, there is β with $1 < \beta< \frac{2q}{q+2+2(2+\alpha)}$ such that

$$\int w_{1}(v_{\lambda})^{2^{\sharp}-1}\,d\xi= C\cdot \bigl( \lambda^{-1} \bigr)^{\frac{q}{\beta} - \frac{q+2}{2}}, $$

where we have used the assumption $0 < \alpha< \frac{q}{2} - 3$. Therefore as λ is sufficiently large, one has

$$\int w_{1}(v_{\lambda})^{2^{\sharp}-1}\,d\xi= o\bigl(\bigl( \lambda^{-1}\bigr)^{2+\alpha}\bigr). $$

The proof is complete. □

Lemma 4.10

Let w be a non-negative solution of (1.1). Then there are $s_{0}>0$ and $\tilde{\mu}>\mu_{1}$ such that $w+s_{0}v_{\lambda}\in \mathcal{M}^{-}_{\mu}$ for all $0 < \mu< \tilde{\mu}$.

Proof

For any $s > 0$, since $G(w)=0$ and w satisfies (1.1), we have

$$ \begin{aligned}[b] G(w+sv_{\lambda})={}& G(sv_{\lambda})+2s\int K(\xi )w^{2^{\sharp}-1}v_{\lambda}\,d\xi \\ &{}+\int K(\xi) \bigl(w^{2^{\sharp}}+(sv_{\lambda})^{2^{\sharp}}- \vert w+sv_{\lambda} \vert ^{2^{\sharp}} \bigr)\,d\xi. \end{aligned} $$

(4.12)

Using the elementary inequality

$$ \vert a+b\vert ^{p}\geq \vert a\vert ^{p} + \vert b\vert ^{p}-M\bigl(\vert a\vert ^{p-1}\vert b\vert +\vert a\vert \vert b\vert ^{p-1}\bigr),\quad \forall p>1, a, b \in\mathbb{R,} $$

(4.13)

and the fact that $K(\xi)$ is bounded in Ω, we obtain

$$\begin{aligned}[b] &\biggl\vert \int K(\xi) \bigl(w^{2^{\sharp}}+(sv_{\lambda})^{2^{\sharp}}- \vert w+sv_{\lambda} \vert ^{2^{\sharp}} \bigr)\,d\xi \biggr\vert \\ &\quad \leq Cs\int w^{2^{\sharp}-1} v_{\lambda}\,d\xi +Cs^{2^{\sharp}-1} \int w v_{\lambda}^{2^{\sharp}-1}\,d\xi. \end{aligned} $$

Therefore for any finite s, we obtain from (2.2) and Lemma 4.9

$$\begin{aligned}[b] G(w+sv_{\lambda})&=G(sv_{\lambda})+o \bigl( \bigl(\lambda^{-1}\bigr)^{(2+\alpha)} \bigr) \\ &=s^{2}\int \vert \nabla_{H} v_{\lambda} \vert ^{2}\,d\xi- s^{2^{\sharp}}\int K(\xi)v_{\lambda}^{2^{\sharp}} \,d\xi-o \bigl(\lambda ^{-(2+\alpha)} \bigr) \end{aligned} $$

for λ large enough. Thus there is $s_{0}>0$ such that $G(w+s_{0}v_{\lambda})=0$, which implies that $w+s_{0}v_{\lambda}\in\mathcal{M}_{\mu}$.

Next, to see $w+s_{0}v_{\lambda}\in\mathcal{M}_{\mu}^{-}$, it suffices to prove that

$$ \int K(\xi)\vert w+s_{0}v_{\lambda} \vert ^{2^{\sharp}}\,d\xi> 0\quad \mbox{for } \lambda\mbox{ large enough}. $$

(4.14)

Indeed, using inequality (4.13) and Lemma 4.9, we obtain

$$\begin{aligned} &\int K(\xi)\vert w+s_{0}v_{\lambda} \vert ^{2^{\sharp}}\,d \xi \\ &\quad =\int K(\xi)w^{2^{\sharp}}\,d\xi+s_{0}^{2^{\sharp}}\int K( \xi)v_{\lambda}^{2^{\sharp}}\,d\xi+o \bigl( \bigl(\lambda^{-1} \bigr)^{2+\alpha}\bigr) \\ &\quad =\int \bigl(\vert \nabla_{H} w\vert ^{2}-\mu \vert \xi \vert ^{\alpha}_{H} w^{2} \bigr)\,d\xi+ s_{0}^{2^{\sharp}}\int K(\xi)v^{2^{\sharp}}_{\lambda}\,d\xi+ o \bigl( \bigl(\lambda^{-1}\bigr)^{2+\alpha}\bigr) \\ &\quad \geq \biggl(1-{\frac{\mu}{\mu_{1}}} \biggr)\int \vert \nabla_{H} w\vert ^{2}\,d\xi+s_{0}^{2^{\sharp}} \int K(\xi)v^{2^{\sharp}}_{\lambda}\,d\xi+o \bigl( \bigl( \lambda^{-1}\bigr)^{2+\alpha}\bigr) \end{aligned} $$

for λ large enough. It follows from $G(w+ s_{0}v_{\lambda})=0$ that there is $\tilde{\mu}>\mu_{1}$ such that

$$\int K(\xi)\vert w+ s_{0}v_{\lambda} \vert ^{2^{\sharp}}\,d \xi> 0,\quad 0 < \mu < \tilde{\mu}. $$

The proof is complete. □

Proof of Lemma 4.8

Using the fact that $L_{\mu}(u_{*})=d_{1}$, $u_{*}$ satisfies (1.1) and (4.13), Lemma 4.9, we obtain from a direct computation for λ large enough

$$L_{\mu}(u_{*}+ sv_{\lambda})\leq L_{\mu}(u_{*})+L_{\mu}(sv_{\lambda})+o \bigl( \bigl(\lambda^{-1}\bigr)^{2+\alpha}\bigr) . $$

In view of Lemma 4.10, it suffices to prove that

$$\sup_{s>0}L_{\mu}(u_{*}+sv_{\lambda}) < d_{1}+\frac{1}{q} \frac{S^{\frac {q}{2}}}{\vert K\vert _{\infty}^{\frac{q-2}{2}}}. $$

Note that

$$\begin{aligned} &\sup_{s>0}L_{\mu}(sv_{\lambda}) \\ &\quad =\frac{1}{q} \biggl(\Vert v_{\lambda} \Vert ^{2} - \mu\int \vert \xi \vert _{H}^{\alpha} \vert v_{\lambda} \vert ^{2}\,d\xi \biggr)^{\frac{q}{2}} \biggl(\int K(\xi)\vert v_{\lambda} \vert ^{2^{\sharp}}\,d\xi \biggr)^{\frac {2-q}{2}} \\ &\quad =\frac{1}{q} \bigl(S^{\frac{q}{2}} + O\bigl(\bigl( \lambda^{-1}\bigr)^{q-2}\bigr)- O\bigl(\bigl( \lambda^{-1}\bigr)^{\alpha+ 2}\bigr) \bigr)^{\frac{q}{2}} \bigl( \vert K\vert _{\infty}S^{\frac{q}{2}} + O\bigl(\bigl( \lambda^{-1}\bigr)^{\beta}\bigr) + O\bigl(\bigl( \lambda^{-1}\bigr)^{q}\bigr) \bigr)^{\frac {2-q}{2}} \\ &\quad = \frac{1}{q}{\frac{S^{\frac{q}{2}}}{\vert K\vert _{\infty}^{\frac{q-2}{2}}}} -O \bigl(\lambda^{-(2+\alpha)} \bigr)+ o \bigl( \lambda^{-\frac{q-2}{2}} \bigr) \end{aligned}$$

for λ large enough. Denote $\mu_{*}=\min\{\mu_{*1}, \mu_{*2}, \tilde{\mu}\}$. Then one has

$$\begin{aligned} \sup_{t>0}L_{\mu}(u_{*}+tv_{\lambda})={}& d_{1}+\frac{1}{q}{\frac{S^{\frac{q}{2}}}{\vert K\vert _{\infty}^{\frac{q-2}{2}}}} -O \bigl( \lambda^{-(2+\alpha)} \bigr) \\ &{}+o \bigl(\lambda^{-(2+\alpha)} \bigr) +O \bigl( \lambda^{-\frac{q-2}{2}} \bigr) \\ < {}& d_{1} + \frac{1}{q}{\frac{S^{\frac{q}{2}}}{\vert K\vert _{\infty}^{\frac{q-2}{2}}}}. \end{aligned}$$

 □

Proof of (ii) of Theorem 1.2

The proof is a combination of Lemma 4.6, Lemma 4.7, Lemma 4.8, and the fact that if $(u_{n})_{n\in\mathbb{N}}$ is a minimizing sequence of $d_{2}$, then so is $(\vert u_{n}\vert )_{n\in\mathbb{N}}$. The proof is complete. □