On a Schrödinger–Poisson system with singularity and critical nonlinearities

Abstract

In this paper, we study the Schrödinger–Poisson system with singularity and critical growth terms. By means of variational methods with an appropriate truncation argument, the existence and multiplicity of positive solutions are obtained.

1 Introduction and main result

In this article, we study the existence and multiplicity of positive solutions to the following Schrödinger–Poisson system:

$$ \textstyle\begin{cases} -\Delta u-\phi u= u^{5}+\frac{\lambda }{u^{\gamma }} , &\text{in } \varOmega ,\\ -\Delta \phi =u^{2}, &\text{in } \varOmega ,\\ u=\phi =0, &\text{on } \partial \varOmega ,\end{cases} $$

(1.1)

where $\varOmega \subset \mathbb{R}^{3}$ is a smooth bounded domain, $0<\gamma <1$, $\lambda >0$ is a real parameter. It is well known that system (1.1) is related to the following system:

$$ \textstyle\begin{cases} -\Delta u+Vu+\phi u= f(x,u) , &\text{in } \mathbb{R}^{3},\\ -\Delta \phi =u^{2}, &\text{in } \mathbb{R}^{3}, \end{cases} $$

(1.2)

which was firstly introduced by Benci and Fortunato in [1]. It described the quantum mechanics models and semiconductor theory. We can learn more details about physical background from [2, 3] and the references therein. System (1.2) has been extensively studied, focusing on the existence of positive solutions, multiplicity of solutions, ground state solutions, sign-changing solutions, radial solutions, by using the variational methods and critical point theory under various assumptions of potential V and nonlocal term f, see for example [4–17] and the references therein.

In addition, existence and multiplicity of the Schrödinger–Poisson problem in a bounded domain has been paid attention to by many authors, we can see [18–24]. More precisely, Fan [21] considered the following system:

$$ \textstyle\begin{cases} -\Delta u+l(x)\phi u= f_{\lambda } \vert u \vert ^{q-1}u+g(x)u^{5}, &\text{in } \varOmega ,\\ -\Delta \phi =l(x)u^{2}, &\text{in } \varOmega ,\\ u=\phi =0, &\text{on } \partial \varOmega ,\end{cases} $$

where $\varOmega \subset \mathbb{R}^{3}$ is a bounded domain with smooth boundary, $1< q<2$, and the functions $l(x)$, $f_{\lambda }$, and $g(x)$ satisfy some assumptions, the author proved multiple positive solutions with the help of Nehari manifold and Ljusternik–Schnirelmann category theory.

Zhang in [22] considered the system involving singularity on bounded domain as follows:

$$ \textstyle\begin{cases} -\Delta u+\eta \phi u= \lambda u^{-\gamma }, &\text{in } \varOmega ,\\ -\Delta \phi =u^{2}, &\text{in } \varOmega ,\\ u>0, &\text{in } \varOmega ,\\ u=\phi =0, &\text{on } \partial \varOmega . \end{cases} $$

(1.3)

For $\eta =1$ and $\lambda >0$, the author obtained the existence and uniqueness of positive solution of system (1.3) by using variational method; for $\eta =-1$ and $\lambda >0$ small enough, the author also considered the existence and multiplicity of positive solutions via Nehari manifold. For the case that replaced with concave-convex nonlinearities and critical growth terms of system (1.3), the authors in [23] got two positive solutions by using the variational method and the concentration-compactness principle when λ is small enough.

Recently, Zheng [24] studied the following Schrödinger–Poisson system:

$$ \textstyle\begin{cases} -\Delta u-\phi u= u^{5}+\lambda u^{q-1} , &\text{in } \varOmega,\\ -\Delta \phi =u^{2}, &\text{in } \varOmega ,\\ u=\phi =0, &\text{on } \partial \varOmega ,\end{cases} $$

where $2< q<6$, $\lambda >0$ is a parameter, the authors obtained one positive ground state solution with the mountain pass theorem and the concentration compactness principle.

As far as we know, there have been no works concerning the existence for system (1.1) up to now. Motivated by the above papers, we study the Schrödinger–Poisson system involving critical and weak singular nonlinearities. Compared with the above mentioned papers, our system has a special point, which makes it difficult to estimate the critical value level. In order to overcome the difficulty, we shall give a special estimate so that two positive solutions of the system can be found by applying the variational method.

Now, our main result is as follows:

Theorem 1.1

Assume that$\gamma \in (0,1)$, then there exists$\lambda _{\ast }>0$such that, for any$\lambda \in (0,\lambda _{\ast })$, system (1.1) has at least two pairs of different positive solutions.

2 Preliminaries

In this section, we give the variational setting for system (1.1) and use the following notations:

$H_{0}^{1}(\varOmega )$ is the usual Sobolev space with the norm $\|u\|=(\int _{\varOmega }|\nabla u|^{2}\,dx)^{1/2}$, and the norm in $L^{p}(\varOmega )$ is represented by $|u|_{p}=(\int _{\varOmega }|u|^{p}\,dx)^{1/p}$. We denote by $B_{r}$ (respectively, $\partial B_{r}$) the closed ball (respectively, the sphere) of center zero and radius r. $u^{+}_{n}(x)=\max \{u_{n},0\}$, $u^{-}_{n}(x)=\max \{-u_{n},0\}$. $C_{1},C_{2},C_{3},\ldots$ denote various positive constants, which may vary from line to line. S is the best Sobolev constant, namely

$$ S=\inf_{u\in H_{0}^{1}\{\varOmega \}\backslash \{0\}} \frac{\int _{\varOmega } \vert \nabla u \vert ^{2}\,dx}{(\int _{\varOmega } \vert u \vert ^{6}\,dx)^{\frac{1}{3}}}. $$

(2.1)

By using the Lax–Milgram theorem, for each $u\in H_{0}^{1}(\varOmega )$, there exists a unique solution $\phi _{u}$ which satisfies the second equation of system (1.1). We substitute $\phi _{u}$ to the first equation of system (1.1), we can rewrite system (1.1) as follows:

$$ \textstyle\begin{cases} -\Delta u-\phi _{u} u= u^{5}+\lambda u^{-\gamma }, &\text{in } \varOmega ,\\ u=0, &\text{on } \partial \varOmega . \end{cases} $$

(2.2)

Now we define the energy functional $I_{\lambda }$ on $u\in H_{0}^{1}({\varOmega })$ by

$$ I_{\lambda }(u)=\frac{1}{2} \Vert u \Vert ^{2}- \frac{1}{4} \int _{\varOmega }\phi _{u}\bigl(u^{+} \bigr)^{2}\,dx-\frac{1}{6} \int _{\varOmega }\bigl(u^{+}\bigr)^{6} \,dx- \frac{\lambda }{1-\gamma } \int _{\varOmega }\bigl(u^{+}\bigr)^{1-\gamma } \,dx. $$

If a function $u\in H_{0}^{1}({\varOmega })$ satisfies

$$ \int _{\varOmega }(\nabla u,\nabla v)\,dx - \int _{\varOmega }\phi _{u}\bigl(u^{+} \bigr)v \,dx - \int _{\varOmega }\bigl(u^{+}\bigr)^{5}v \,dx-\lambda \int _{\varOmega }\bigl(u^{+}\bigr)^{- \gamma }v \,dx=0 $$

for $v\in H_{0}^{1}({\varOmega })$, then we say u is a weak solution of (2.2) and $(u, \phi _{u})$ is a pair solution of system (1.1).

Because of the singular nonlinearity $u^{-\gamma }$, the functional $I_{\lambda }$ on $H_{0}^{1}(\varOmega )$ is not differentiable. Therefore, we cannot apply directly the usual critical point theory to solve this problem. However, we can find two positive solutions by an approximation approach. That is, for $\alpha >0$, we consider the following perturbation problem:

$$ \textstyle\begin{cases} -\Delta u-\phi _{u} u= (u^{+})^{5}+ \frac{\lambda }{(u^{+}+\alpha )^{\gamma }} , &\text{in } \varOmega,\\ u=0, &\text{on } \partial \varOmega . \end{cases} $$

(2.3)

The solution of problem (2.3) corresponds to critical point of the $C^{1}$-functional on $H_{0}^{1}({\varOmega })$ by

$$\begin{aligned} I_{\lambda ,\alpha }(u) =&\frac{1}{2} \Vert u \Vert ^{2}-\frac{1}{4} \int _{ \varOmega }\phi _{u}\bigl(u^{+} \bigr)^{2}\,dx-\frac{1}{6} \int _{\varOmega }\bigl(u^{+}\bigr)^{6}\,dx \\ &{}- \frac{\lambda }{1-\gamma } \int _{\varOmega }\bigl(u^{+}+\alpha \bigr)^{1-\gamma }- \alpha ^{1-\gamma }\,dx. \end{aligned}$$

(2.4)

Moreover, if a function $u\in H_{0}^{1}({\varOmega })$, and for $v \in H_{0}^{1}({\varOmega })$, then $(u, \phi _{u})$ is a pair solution of problem (2.3) satisfying

$$ \int _{\varOmega }(\nabla u,\nabla v)\,dx - \int _{\varOmega }\phi _{u}\bigl(u^{+} \bigr)v \,dx - \int _{\varOmega }\bigl(u^{+}\bigr)^{5}v \,dx-\lambda \int _{\varOmega } \frac{v}{(u^{+}+\alpha )^{\gamma }}\,dx=0. $$

3 Existence of positive solution for problem (2.3)

Before proving Theorem 1.1, we recall the following lemma (see [1, 22]).

Lemma 3.1

For every$u \in H_{0}^{1}(\varOmega )$, there exists a unique solution$\phi _{u} \in H_{0}^{1}(\varOmega )$of

$$ \textstyle\begin{cases} -\Delta \phi =u^{2}, & \textit{in } \varOmega ,\\ \phi =0, & \textit{on } \partial \varOmega ,\end{cases} $$

and

• (1) $\Vert \phi _{u} \Vert ^{2}=\int _{\varOmega } \phi _{u}u^{2}\,dx$;

• (2) $\phi _{u}\geq 0$; moreover, $\phi _{u}>0$when$u\neq 0$;

• (3) For each$t\neq 0$, $\phi _{tu}=t^{2}\phi _{u}$;

• (4) $\int _{\varOmega } \phi _{u}u^{2}\,dx=\int _{\varOmega } \vert \nabla \phi _{u} \vert ^{2}\,dx \leq S^{-1}|u|_{4}^{4}|\varOmega |^{2/3}\leq S^{-1}|u|_{12/5}^{4} \leq S^{-3}\|u\|^{4}|\varOmega |$;

• (5) Assume that$u_{n}\rightharpoonup u$in$H_{0}^{1}(\varOmega )$, then$\phi _{u_{n}}\rightarrow \phi _{u}$in$H_{0}^{1}(\varOmega )$and

  $$ \int _{\varOmega }\phi _{u_{n}}u_{n}v\,dx \rightarrow \int _{\varOmega }\phi _{u}uv\,dx, \quad \forall v\in H_{0}^{1}(\varOmega ); $$

• (6) Set$\mathcal{F}(u)=\int _{\varOmega }\phi _{u}u^{2}\,dx$, then$\mathcal{F}(u): H_{0}^{1}(\varOmega )\rightarrow H_{0}^{1}(\varOmega )$is$C^{1}$and

  $$ \bigl\langle \mathcal{F}'(u),v\bigr\rangle =4 \int _{\varOmega }\phi _{u}uv\,dx, \quad \forall v\in H_{0}^{1}(\varOmega ). $$

Lemma 3.2

There exist$\varLambda _{0}$, $\rho >0$such that, for every$\lambda \in (0,\varLambda _{0})$, we have

$$ I_{\lambda ,\alpha }(u)\geq \kappa \quad \textit{for } u \in \overline{\partial B_{\rho }} \quad \textit{and}\quad I_{\lambda ,\alpha }(u)< 0\quad \textit{for } u\in \overline{B_{\rho }}. $$

(3.1)

Proof of Lemma 3.2

According to Hölder’s inequality and (2.1), we have

$$ \int _{\varOmega } \bigl(u^{+} \bigr)^{1-\gamma } \,dx \leq \int _{ \varOmega } \vert u \vert ^{1-\gamma }\,dx \leq \vert u \vert _{6}^{1-\gamma } \vert \varOmega \vert ^{ \frac{5+\gamma }{6}} \leq \vert \varOmega \vert ^{\frac{5+\gamma }{6}} S^{- \frac{1-\gamma }{2}} \Vert u \Vert ^{1-\gamma }. $$

(3.2)

Note the subadditivity of $t^{1-\gamma }$, namely

$$ \bigl(u^{+}+\alpha \bigr)^{1-\gamma }-\alpha ^{1-\gamma }\leq \bigl(u^{+}\bigr)^{1- \gamma }, \quad \forall u \in H_{0}^{1}(\varOmega ). $$

(3.3)

It follows from (2.1), (3.2), and (3.3) that

$$\begin{aligned} I_{\lambda ,\alpha }(u) =& \frac{1}{2} \Vert u \Vert ^{2}-\frac{1}{4} \int _{\varOmega }\phi _{u}\bigl(u^{+} \bigr)^{2}\,dx-\frac{1}{6} \int _{\varOmega }\bigl(u^{+}\bigr)^{6} \,dx-\frac{\lambda }{1-\gamma } \int _{\varOmega }\bigl(u^{+}+ \alpha \bigr)^{1-\gamma }-\alpha ^{1-\gamma }\,dx \\ \geq & \frac{1}{2} \Vert u \Vert ^{2}- \frac{1}{4} \int _{\varOmega } \phi _{u}\bigl(u^{+} \bigr)^{2}\,dx-\frac{1}{6} \int _{\varOmega }\bigl(u^{+}\bigr)^{6} \,dx- \frac{\lambda }{1-\gamma } \int _{\varOmega }\bigl(u^{+}\bigr)^{1-\gamma } \\ \geq & \frac{1}{2} \Vert u \Vert ^{2}- \frac{ \vert \varOmega \vert }{4}S^{-3} \Vert u \Vert ^{4}- \frac{1}{6} S^{-3} \Vert u \Vert ^{6}- \frac{\lambda }{1-\gamma } \vert \varOmega \vert ^{ \frac{5+\gamma }{6}}S^{-\frac{1-\gamma }{2}} \Vert u \Vert ^{1-\gamma } \\ \geq & \Vert u \Vert ^{1-\gamma } \biggl(\frac{1}{2} \Vert u \Vert ^{1+ \gamma }-\frac{ \vert \varOmega \vert }{4}S^{-3} \Vert u \Vert ^{3+\gamma }-\frac{1}{6} S^{-3} \Vert u \Vert ^{5+\gamma }-\frac{\lambda }{1-\gamma } \vert \varOmega \vert ^{ \frac{5+\gamma }{6}} S^{-\frac{1-\gamma }{2}} \biggr). \end{aligned}$$

Set $g(t)=\frac{1}{2}t^{1+\gamma }-\frac{|\varOmega |}{4}S^{-3}t^{3+\gamma }- \frac{1}{6} S^{-3}t^{5+\gamma }$ for $t>0$, then there exists a positive constant

$$ \rho = \biggl[ \frac{-3(3+\gamma ) \vert \varOmega \vert +\sqrt{9(3+\gamma )^{2} \vert \varOmega \vert ^{2}+48(5+6\gamma +\gamma ^{2})S^{3}}}{4(5+\gamma )} \biggr]^{\frac{1}{2}}>0 $$

such that $\max_{t>0}g(t)=g(\rho )>0$. Letting $\varLambda _{0}= \frac{(1-\gamma )S^{\frac{1-\gamma }{2}}}{2|\varOmega |^{\frac{5+\gamma }{6}}}g( \rho )$, it follows that there exists a constant $\kappa >0$ such that $I_{\lambda ,\alpha }(u)|_{S_{\rho }}\geq \kappa$ for every $\lambda \in (0,\varLambda _{0})$.

Especially, we define a function $f(x)=x^{1-\gamma }$, $x\in \varOmega$, by using the Lagrange mean value theorem, there exists $\xi >0$ such that

$$ \bigl(u^{+}+\alpha \bigr)^{1-\gamma }-\alpha ^{1-\gamma }=f^{\prime }(\xi )u^{+}, $$

here $\xi \in (\alpha , u^{+}+\alpha )$. For every $u\in \overline{B_{\rho }}$, $u^{+}\neq 0$, we have

$$\begin{aligned} \lim_{t\rightarrow 0^{+}} \frac{I_{\lambda ,\alpha }(tu)}{t} =& - \frac{\lambda }{1-\gamma }\lim_{t\rightarrow 0^{+}}\frac{1}{t} \int _{ \varOmega }\bigl(tu^{+}+\alpha \bigr)^{1-\gamma }-\alpha ^{1-\gamma }\,dx \\ =& -\lambda \int _{\varOmega } \xi ^{-\gamma }tu^{+}\,dx \\ < &0. \end{aligned}$$

For t small enough, we have $I_{\lambda ,\alpha }(tu)<0$. Hence, there exists u small enough such that $I_{\lambda ,\alpha }(u)<0$. Therefore, we deduce that

$$ d=:\inf_{u\in \overline{B_{\rho }}}I_{\lambda ,\alpha }(u)< \inf _{u \in \overline{\partial B_{\rho }}}I_{\lambda ,\alpha }(u). $$

The proof is complete. □

Lemma 3.3

Let$0<\alpha <1$, if$\{u_{n}\}\subset H_{0}^{1}(\varOmega )$is a$(PS)_{c}$sequence for$I_{\lambda ,\alpha }$with$c<\frac{1}{3}S^{\frac{2}{{2}}}-D\lambda ^{\frac{2}{1+\gamma }}$, where$D=\frac{1+\gamma }{4(1-\gamma )} (\frac{3+\gamma }{2}|\varOmega |^{ \frac{5+\gamma }{6}}S^{-\frac{1-\gamma }{2}} )^{ \frac{2}{1+\gamma }}$, then there exists$u_{0} \in H_{0}^{1}(\varOmega )$such that$u_{n}\rightarrow u_{0}$in$H_{0}^{1}(\varOmega )$and$\int _{\varOmega }u_{n}^{6}\,dx\rightarrow \int _{\varOmega }u_{0}^{6}\,dx$.

Proof of Lemma 3.3

Let $\{u_{n}\}\subset H_{0}^{1}(\varOmega )$ be such that

$$ I_{\lambda ,\alpha }(u_{n})\rightarrow c, \qquad I_{\lambda ,\alpha }^{\prime }(u_{n}) \rightarrow 0. $$

(3.4)

Now, we claim that $\{u_{n}\}$ is bounded in $H_{0}^{1}(\varOmega )$. Otherwise, we assume that $\|u_{n}\|\rightarrow \infty$, as $n\rightarrow \infty$. It follows from (3.2), (3.3), and (3.4) that

$$\begin{aligned} 1+c+o(1) \Vert u_{n} \Vert =& I_{\lambda ,\alpha } (u_{n} )-\frac{1}{4} \bigl\langle I_{\lambda ,\alpha }^{ \prime } (u_{n} ), u_{n} \bigr\rangle \\ \geq & \frac{1}{4} \Vert u_{n} \Vert ^{2}+ \frac{1}{12} \int _{\varOmega }\bigl(u^{+}\bigr)^{6} \,dx-\frac{\lambda }{1-\gamma } \int _{\varOmega } \bigl[ \bigl(u_{n}^{+}+ \alpha \bigr)^{1-\gamma }- \alpha ^{1-\gamma } \bigr]\,dx \\ \geq & \frac{1}{4} \Vert u_{n} \Vert ^{2}- \frac{\lambda }{1-\gamma } \int _{\varOmega } \bigl(u_{n}^{+} \bigr)^{1- \gamma }\,dx \\ \geq & \frac{1}{4} \Vert u_{n} \Vert ^{2}- \frac{\lambda }{1-\gamma } \vert \varOmega \vert ^{\frac{5+\gamma }{6}} S^{- \frac{1-\gamma }{2}} \bigl\Vert u_{n}^{+} \bigr\Vert ^{1-\gamma }. \end{aligned}$$

Since $0<\gamma <1$, the last inequality above is impossible, which implies that $\{u_{n}\}$ is bounded in $H_{0}^{1}(\varOmega )$. So there exists $\tau \in L^{1}(\varOmega )$ for all n such that $|u_{n}(x)|\leq \tau (x)$ a.e. in Ω. And there exists a subsequence, still denoted by $\{u_{n}\}$. We assume that there exists $u_{0}\in H_{0}^{1}(\varOmega )$ such that

$$ \textstyle\begin{cases} u_{n}\rightharpoonup u_{0}, \quad \text{weakly in } H_{0}^{1}( \varOmega ), \\ u_{n}\rightarrow u_{0}, \quad \text{strongly in } L^{p}(\varOmega )\ (1 \leq p< 6), \\ u_{n}(x)\rightarrow u_{0}(x), \quad \text{a.e. in } \varOmega . \end{cases} $$

(3.5)

Note the given condition $\alpha >0$, we can easily get $\frac{|u_{0}|}{(u_{0}^{+}+\alpha )^{\gamma }}\leq \frac{|u_{0}|}{\alpha ^{\gamma }}$. Then, by the dominated convergence theorem and (3.5), we have

$$ \lim_{n \rightarrow \infty } \int _{\varOmega } \bigl(u_{n}^{+}+\alpha \bigr)^{-\gamma } u_{0}\,dx= \int _{\varOmega } \bigl(u_{0}^{+}+\alpha \bigr)^{-\gamma } u_{0}\,dx. $$

(3.6)

Moreover, we have $|\frac{u_{n}}{(u_{n}^{+}+\alpha )^{\gamma }}|\leq \frac{\tau }{\alpha ^{\gamma }}$, by the dominated convergence theorem, we also have

$$ \lim_{n \rightarrow \infty } \int _{\varOmega } \bigl(u_{n}^{+}+\alpha \bigr)^{-\gamma } u_{n}\,dx= \int _{\varOmega } \bigl(u_{0}^{+}+\alpha \bigr)^{-\gamma } u_{0}\,dx. $$

(3.7)

Now, set $w_{n}=u_{n}-u_{0}$, then $\|w_{n}\|\rightarrow 0$ as $n\rightarrow \infty$. Otherwise, there exists a subsequence, still denoted by $w_{n}$, such that

$$ \lim_{n\rightarrow \infty } \Vert w_{n} \Vert =l>0. $$

Note that $\lim_{n\rightarrow \infty } \langle I_{\lambda ,\alpha }^{ \prime } (u_{n} ), u_{0} \rangle =0$ and (3.6), we deduce

$$ \Vert u_{0} \Vert ^{2}- \int _{\varOmega }\phi _{u_{0}}\bigl(u_{0}^{+} \bigr)^{2}\,dx- \int _{ \mathbb{R}^{3}}\bigl(u_{0}^{+} \bigr)^{6}\,dx-\lambda \int _{\varOmega }\bigl(u_{0}^{+}+ \alpha \bigr)^{-\gamma }u_{0}\,dx=0. $$

(3.8)

Using the Brézis–Lieb lemma [25], we have

$$ \textstyle\begin{cases} \Vert u_{n} \Vert ^{2}= \Vert w_{n} \Vert ^{2}+ \Vert u_{0} \Vert ^{2}+o(1), \\ \int _{\varOmega }(u_{n}^{+})^{6}\,dx=\int _{\varOmega }(w_{n}^{+})^{6}\,dx+ \int _{\varOmega }(u_{0}^{+})^{6}\,dx+o(1). \end{cases} $$

(3.9)

It follows from (3.4), (3.7), and (3.9) that

$$\begin{aligned} o(1) =& \Vert w_{n} \Vert ^{2}+ \Vert u_{0} \Vert ^{2}- \int _{ \varOmega }\phi _{u_{0}}\bigl(u_{0}^{+} \bigr)^{2}\,dx \\ &{} - \int _{\varOmega }\bigl(w_{n}^{+} \bigr)^{6}\,dx- \int _{\varOmega }\bigl(u_{0}^{+} \bigr)^{6}\,dx- \lambda \int _{\varOmega }\bigl(u_{0}^{+}+\alpha \bigr)^{-\gamma }u_{0}\,dx. \end{aligned}$$

(3.10)

Therefore, (3.8) and (3.10) lead to

$$ \Vert w_{n} \Vert ^{2}- \int _{\varOmega }\bigl(w_{n}^{+} \bigr)^{6}\,dx=o(1). $$

(3.11)

Since also $\int _{\varOmega }(w_{n}^{+})^{6}\,dx\leq \int _{\varOmega }|w_{n}|^{6}\,dx$, then, according to (2.1), (3.11) implies that

$$ l^{2}\geq S^{\frac{3}{2}}. $$

From (3.2) and using the Young inequality, we have

$$\begin{aligned} I_{\lambda ,\alpha }(u_{0}) =& \frac{1}{2} \Vert u_{0} \Vert ^{2}- \frac{1}{4} \int _{\varOmega }\phi _{u_{0}}\bigl(u_{0}^{+} \bigr)^{2}\,dx-\frac{1}{6} \int _{\varOmega } u_{0}^{6}\,dx \\ &{}-\frac{\lambda }{1-\gamma } \int _{\varOmega } \bigl[ \bigl(u_{0}^{+}+ \alpha \bigr)^{1-\gamma }-\alpha ^{1- \gamma } \bigr]\,dx \\ \geq & \frac{1}{2} \Vert u_{0} \Vert ^{2}-\frac{1}{4} \int _{ \varOmega }\phi _{u_{0}}\bigl(u_{0}^{+} \bigr)^{2}\,dx-\frac{1}{6} \int _{\varOmega } u_{0}^{6}\,dx- \frac{\lambda }{1-\gamma } \int _{\varOmega }\bigl(u_{0}^{+} \bigr)^{1-\gamma }\,dx \\ =& \frac{1}{4} \Vert u_{0} \Vert ^{2}+ \frac{1}{12} \int _{\varOmega } u_{0}^{6}\,dx- \lambda \biggl(\frac{1}{1-\gamma }-\frac{1}{4} \biggr) \int _{\varOmega }\bigl(u_{0}^{+} \bigr)^{1- \gamma }\,dx \\ \geq & \frac{1}{4} \Vert u_{0} \Vert ^{2}-\lambda \biggl( \frac{1}{1-\gamma }-\frac{1}{4} \biggr) \vert \varOmega \vert ^{\frac{5+\gamma }{6}}S^{- \frac{1-\gamma }{2}} \Vert u_{0} \Vert ^{1-\gamma } \\ \geq & -D\lambda ^{\frac{2}{1+\gamma }}, \end{aligned}$$

where $D=\frac{1+\gamma }{4(1-\gamma )} (\frac{3+\gamma }{2}|\varOmega |^{ \frac{5+\gamma }{6}}S^{-\frac{1-\gamma }{2}} )^{ \frac{2}{1+\gamma }}$. Combining (3.10) with (3.11), we also have

$$\begin{aligned} I_{\lambda ,\alpha }(u_{0}) =& I_{\lambda ,\alpha }(u_{n})- \frac{1}{2} \Vert w_{n} \Vert ^{2}+ \frac{1}{6} \int _{\varOmega } \vert w_{n} \vert ^{6}\,dx+o(1) \\ =& c-\frac{1}{3} \Vert w_{n} \Vert ^{2}+o(1) \\ < & c-\frac{1}{3}S^{\frac{3}{{2}}} \\ =& \frac{1}{3}S^{\frac{2}{{2}}}-D\lambda ^{ \frac{2}{1+\gamma }}- \frac{1}{3}S^{\frac{3}{2}} \\ =& -D\lambda ^{\frac{2}{1+\gamma }}. \end{aligned}$$

It is obvious that the above two inequalities are impossibility. Thus, we get $l=0$, which yields $u_{n}\rightarrow u_{0}$ in $H_{0}^{1}(\varOmega )$. By (3.11), we get

$$ 0\leq \int _{\varOmega }u_{n}^{6}\,dx- \int _{\varOmega }u_{0}^{6}\,dx= \int _{ \varOmega }w_{n}^{6}\,dx+o(1)= \Vert w_{n} \Vert ^{2}\rightarrow 0, $$

which implies that $\int _{\varOmega }u_{n}^{6}\,dx\rightarrow \int _{\varOmega }u_{0}^{6}\,dx$ as $n\rightarrow \infty$. The proof is complete. □

Note that $0<\alpha <1$, we can get

$$\begin{aligned} I_{\lambda ,\alpha }(u) =& \frac{1}{2} \Vert u \Vert ^{2}- \frac{1}{4} \int _{\varOmega }\phi _{u}\bigl(u^{+} \bigr)^{2}\,dx-\frac{1}{6} \int _{ \varOmega }\bigl(u^{+}\bigr)^{6} \,dx-\frac{\lambda }{1-\gamma } \int _{\varOmega }\bigl(u^{+}+ \alpha \bigr)^{1-\gamma }-\alpha ^{1-\gamma }\,dx \\ \leq & \frac{1}{2} \Vert u \Vert ^{2}- \frac{1}{6} \int _{\varOmega }\bigl(u^{+}\bigr)^{6} \,dx- \frac{\lambda }{1-\gamma } \int _{\varOmega }\bigl(u^{+}+\alpha \bigr)^{1-\gamma }\,dx+ \frac{\lambda }{1-\gamma } \int _{\varOmega }\alpha ^{1-\gamma }\,dx \\ \leq & \frac{1}{2} \Vert u \Vert ^{2}- \frac{1}{6} \int _{\varOmega }\bigl(u^{+}\bigr)^{6} \,dx- \frac{\lambda }{1-\gamma } \int _{\varOmega }\bigl(u^{+}\bigr)^{1-\gamma } \,dx+ \frac{\lambda }{1-\gamma } \vert \varOmega \vert . \end{aligned}$$

(3.12)

Now, we define a new functional $J_{\lambda }(u):H_{0}^{1}(\varOmega )\rightarrow \mathbb{R}$ as follows:

$$ J_{\lambda }(u)= \frac{1}{2} \Vert u \Vert ^{2}-\frac{1}{6} \int _{ \varOmega }\bigl(u^{+}\bigr)^{6} \,dx-\frac{\lambda }{1-\gamma } \int _{\varOmega }\bigl(u^{+}\bigr)^{1- \gamma } \,dx. $$

(3.13)

Consequently, we consider the following problem:

$$ \textstyle\begin{cases} -\Delta u= u^{5}+\frac{\lambda }{u^{\gamma }} , &\text{in } \varOmega ,\\ u=0, &\text{on } \varOmega . \end{cases} $$

(3.14)

And we find that the weak solutions of problem (3.14) correspond to the critical points of the functional $J_{\lambda }$.

Remark 3.4

There exists ρ, $\varLambda _{0}>0$ (given by Lemma 3.2 such that problem (3.14) has a positive solution $v_{0}\in \overline{B_{\rho }}$ with $J_{\lambda }(v_{0})<0$ and $J_{\lambda }|_{\overline{\partial B_{\rho }}}>0$ for every $\lambda \in (0,\varLambda _{0})$. In fact, from (3.13), we have

$$\begin{aligned} J_{\lambda }(u) =& \frac{1}{2} \Vert u \Vert ^{2}- \frac{1}{6} \int _{\varOmega }\bigl(u^{+}\bigr)^{6} \,dx-\frac{\lambda }{1-\gamma } \int _{\varOmega }\bigl(u^{+}\bigr)^{1-\gamma } \\ \geq & \frac{1}{2} \Vert u \Vert ^{2}- \frac{1}{6} S^{-3} \Vert u \Vert ^{6}- \frac{\lambda }{1-\gamma } \vert \varOmega \vert ^{\frac{5+\gamma }{6}}S^{- \frac{1-\gamma }{2}} \Vert u \Vert ^{1-\gamma } \\ \geq & \Vert u \Vert ^{1-\gamma } \biggl(\frac{1}{2} \Vert u \Vert ^{1+ \gamma }-\frac{1}{6} S^{-3} \Vert u \Vert ^{5+\gamma }- \frac{\lambda }{1-\gamma } \vert \varOmega \vert ^{\frac{5+\gamma }{6}} S^{- \frac{1-\gamma }{2}} \biggr). \end{aligned}$$

By Lemma 3.2, when $\|u\|=\rho$, we have

$$ \frac{1}{2}\rho ^{1+\gamma }-\frac{ \vert \varOmega \vert }{4}S^{-3} \rho ^{3+ \gamma }-\frac{1}{6}S^{-3}\rho ^{5+\gamma }- \frac{\lambda }{1-\gamma } \vert \varOmega \vert ^{\frac{5+\gamma }{6}}S^{- \frac{1-\gamma }{2}}>0 $$

for every $\lambda \in (0,\varLambda _{0})$. Then we deduce that $J_{\lambda }|_{\overline{\partial B_{\rho }}}>0$ for $\lambda \in (0,\varLambda _{0})$. Similar to Lemma 3.2, we get $v_{0}\in \overline{B_{\rho }}$ and $J_{\lambda }(v_{0})<0$ for every $\lambda \in (0,\varLambda _{0})$. Moreover, there exist two constants $m,M>0$ such that $m< v_{0}(x)< M$.

As usual, we consider the following function:

$$ U_{\varepsilon }(x)= \frac{(3\varepsilon ^{2})^{\frac{1}{2}}}{(\varepsilon ^{2}+ \vert x \vert ^{2})^{\frac{1}{2}}}, $$

where ε is a positive constant. Moreover, we know that $U_{\varepsilon }$ is a positive solution of problem $-\Delta u=|u|^{4}u$ in $\mathbb{R}^{3}$ and $\int _{\varOmega }|\nabla U_{\varepsilon }|^{2}\,dx=\int _{\varOmega }| U_{\varepsilon }|^{6}+S^{\frac{3}{2}}$. Let ζ be a smooth cut-off function $\zeta \in C_{0}^{\infty }(\varOmega )$ such that $0\leq \zeta (x)\leq 1$ in Ω. $\zeta (x)=1$ near $x=0$ and it is radially symmetric. Set $v_{\varepsilon }(x)=\zeta (x) U(x)$. Then we have the following.

Lemma 3.5

Assume$0<\gamma <1$, there holds

$$ \sup_{t\geq 0}I_{\lambda ,\alpha } (v_{0}+tv_{\varepsilon })< \frac{1}{3}S^{\frac{3}{2}}-D \lambda ^{\frac{2}{1+\gamma }}. $$

(3.15)

Proof of Lemma 3.5

From [26], one has

$$ \int _{\varOmega } \vert \nabla v_{\varepsilon } \vert ^{2}\,dx= S^{\frac{3}{2}}+O( \varepsilon ), \qquad \int _{\varOmega } \bigl\vert v_{\varepsilon }(x) \bigr\vert ^{6}\,dx= S^{\frac{3}{2}}+O\bigl(\varepsilon ^{3} \bigr). $$

It is well known that the following inequality

$$ (a+b)^{6}\geq a^{6}+b^{6}+6a^{5}b+{6}ab^{5} $$

holds true for each $a, b\geq 0$. With no loss of generality, for $a\geq m$ and $b\geq 0$, we can get that

$$ (a+b)^{1-\gamma }-a^{1-\gamma }\geq 0. $$

Since $v_{0}$ is a positive solution of problem (3.14), then there holds

$$\begin{aligned}& J_{\lambda }(v_{0}+tv_{\varepsilon }) \\& \quad = \frac{1}{2} \Vert v_{0}+tv_{\varepsilon } \Vert ^{2}-\frac{1}{6} \int _{\varOmega }(v_{0}+tv_{\varepsilon })^{6} \,dx- \frac{\lambda }{1-\gamma } \int _{\varOmega }(v_{0}+tv_{\varepsilon })^{1- \gamma } \,dx \\& \quad = I_{\lambda }(v_{0})+\frac{t^{2}}{2} \Vert v_{\varepsilon } \Vert ^{2}+t \int _{\varOmega } \bigl[(\nabla v_{0},\nabla v_{\varepsilon }) -v_{0}^{5}v_{\varepsilon }-\lambda v_{0}^{-\gamma }v_{\varepsilon } \bigr]\,dx - \frac{1}{6} \int _{\varOmega } \bigl[ \vert v_{0}+tv_{\varepsilon } \vert ^{6} \\& \qquad {} -v_{0}^{6}-6v_{0}^{5}tv_{\varepsilon } \bigr]\,dx- \frac{\lambda }{1-\gamma } \int _{\varOmega } \bigl[ \vert v_{0}+tv_{\varepsilon } \vert ^{1- \gamma }-v_{0}^{1-\gamma }-(1-\gamma )v_{0}^{-\gamma }tu_{\varepsilon } \bigr]\,dx \\& \quad \leq I_{\lambda }(v_{0})+\frac{t^{2}}{2} \Vert v_{\varepsilon } \Vert ^{2}+t \int _{\varOmega } \bigl[(\nabla v_{0},\nabla v_{\varepsilon }) -v_{0}^{5}v_{\varepsilon }-\lambda v_{0}^{-\gamma }v_{\varepsilon } \bigr]\,dx - \frac{1}{6} \int _{\varOmega } \bigl[ \vert v_{0}+tv_{\varepsilon } \vert ^{6} \\& \qquad {} -v_{0}^{6}-6v_{0}^{5}tv_{\varepsilon } \bigr]\,dx+\lambda \int _{\varOmega }v_{0}^{-\gamma }tv_{\varepsilon } \,dx \\& \quad \leq \frac{t^{2}}{2} \Vert v_{\varepsilon } \Vert ^{2}- \frac{t^{6}}{6} \int _{\varOmega } \vert v_{\varepsilon } \vert ^{6}\,dx-t^{5} \int _{ \varOmega }v_{0} \vert v_{\varepsilon } \vert ^{5}\,dx+\lambda \int _{\varOmega }v_{0}^{- \gamma }tv_{\varepsilon } \,dx \\& \quad \leq \frac{t^{2}}{2} \Vert v_{\varepsilon } \Vert ^{2}- \frac{t^{6}}{6} \int _{\varOmega } \vert v_{\varepsilon } \vert ^{6}\,dx-mt^{5} \int _{ \varOmega } \vert v_{\varepsilon } \vert ^{5}\,dx+M^{-\gamma } \lambda t \int _{\varOmega }v_{\varepsilon }\,dx. \end{aligned}$$

Let

$$ h(t)= \frac{t^{2}}{2} \Vert v_{\varepsilon } \Vert ^{2}- \frac{t^{6}}{6} \int _{\varOmega } \vert v_{\varepsilon } \vert ^{6}\,dx-mt^{5} \int _{ \varOmega } \vert v_{\varepsilon } \vert ^{5}\,dx+M^{-\gamma } \lambda t \int _{\varOmega }v_{\varepsilon }\,dx. $$

Similar to paper [27], we can find that there exist $t_{\varepsilon }$ and positive constants $t_{1}$, $t_{2}$ (independent of ε, λ) such that $\sup_{t\geq 0} h(t)=h(t_{\varepsilon })$ and

$$ 0< t_{1}\leq t_{\varepsilon }\leq t_{2}< \infty . $$

Indeed, since $\lim_{t\rightarrow 0} h(t)=0$, $\lim_{t\rightarrow +\infty } h(t)=-\infty$, there exists $t_{\varepsilon }$ such that

$$ h(t_{\varepsilon })=\sup_{t\geq 0}h(t),\quad \text{and} \quad h'(t)|_{t=t_{\varepsilon }}=0. $$

Note that $\int _{\varOmega }|v_{\varepsilon }(x)|^{5}\,dx=C_{1}\varepsilon ^{ \frac{1}{2}}$ and $\int _{\varOmega }|v_{\varepsilon }(x)|\,dx=C_{2}\varepsilon ^{\frac{1}{2}}$, one has

$$\begin{aligned} \sup_{t\geq 0}J_{\lambda }(v_{0}+tv_{\varepsilon }) \leq & \sup_{t\geq 0}h(t)=h(t_{\varepsilon }) \\ \leq & \sup_{t\geq 0} \biggl\{ \frac{t^{2}}{2} \Vert u_{\varepsilon } \Vert ^{2}-\frac{t^{6}}{6} \int _{\varOmega } \vert u_{\varepsilon } \vert ^{6}\,dx \biggr\} -mt_{1}^{5} \int _{\varOmega } \vert v_{\varepsilon } \vert ^{5}\,dx \\ & {} +M^{-\gamma } \lambda t_{2} \int _{\varOmega }v_{\varepsilon }\,dx \\ \leq & \sup_{t\geq 0} \biggl\{ \frac{t^{2}}{2} \Vert u_{\varepsilon } \Vert ^{2}-\frac{t^{6}}{6} \int _{\varOmega } \vert u_{\varepsilon } \vert ^{6}\,dx \biggr\} -C_{3}\varepsilon ^{\frac{1}{2}}+ \lambda C_{4}\varepsilon ^{ \frac{1}{2}} \\ \leq & \sup_{t\geq 0} \biggl\{ \frac{t^{2}}{2}S^{ \frac{3}{2}}- \frac{t^{6}}{6}S^{\frac{3}{2}} \biggr\} +C_{5} \varepsilon -C_{3}\varepsilon ^{\frac{1}{2}}+ \lambda C_{4} \varepsilon ^{\frac{1}{2}} \\ \leq & \frac{1}{3}S^{\frac{3}{2}}+C_{5}\varepsilon -C_{3} \varepsilon ^{\frac{1}{2}}+\lambda C_{4} \varepsilon ^{\frac{1}{2}}. \end{aligned}$$

From (3.12), we get the following estimate:

$$\begin{aligned} \sup_{t\geq 0}I_{\lambda ,\alpha }(v_{0}+tv_{\varepsilon }) \leq & \sup_{t\geq 0}J_{\lambda }(v_{0}+tv_{\varepsilon })+ \frac{\lambda }{1-\gamma } \vert \varOmega \vert \\ \leq & \frac{1}{3}S^{\frac{3}{2}}+C_{5}\varepsilon -C_{3} \varepsilon ^{\frac{1}{2}}+\lambda C_{4} \varepsilon ^{\frac{1}{2}}+C_{6} \lambda \\ \leq & \frac{1}{3}S^{\frac{3}{2}}+C_{5}\varepsilon +C_{7} \lambda -C_{3}\varepsilon ^{\frac{1}{2}}. \end{aligned}$$

Let $\varepsilon =\lambda ^{\frac{2}{1+\gamma }}$, and for $\frac{2}{1+\gamma }>1$, there holds

$$\begin{aligned} C_{5}\varepsilon +C_{7}\lambda -C_{3} \varepsilon ^{ \frac{1}{2}} =&C_{5}\lambda ^{\frac{2}{1+\gamma }}+C_{7} \lambda -C_{3} \lambda ^{\frac{1}{1+\gamma }} \\ \leq & \lambda ^{\frac{2}{1+\gamma }}\bigl( C_{8}\lambda ^{ \frac{\gamma -1}{1+\gamma }}-C_{3}\lambda ^{\frac{-1}{1+\gamma }}\bigr) \\ =& \lambda ^{\frac{2}{1+\gamma }} \biggl( \frac{C_{8}}{\lambda ^{\frac{1-\gamma }{1+\gamma }}} - \frac{C_{3}}{\lambda ^{\frac{1}{1+\gamma }}} \biggr). \end{aligned}$$

As $0<\gamma <1$, we have $\frac{1-\gamma }{1+\gamma }<\frac{1}{1+\gamma }$. Moreover, we get that $\lambda ^{\frac{1-\gamma }{1+\gamma }}>\lambda ^{\frac{1}{1+\gamma }}$ for every $\lambda \in (0,1)$. Consequently, there exists $\varLambda _{1}>0$ such that $\lambda \leq \varLambda _{1}$, then it is shown that

$$ \frac{C_{8}}{\lambda ^{\frac{1-\gamma }{1+\gamma }}} - \frac{C_{3}}{\lambda ^{\frac{1}{1+\gamma }}}\leq -D. $$

Thereby, from the above inequality, we conclude that

$$ \sup_{t\geq 0}I_{\lambda ,\alpha }(v_{0}+tv_{\varepsilon }) \leq \frac{1}{3}S^{\frac{3}{2}}-D\lambda ^{\frac{2}{1+\gamma }}. $$

Hence, (3.15) holds true for $\lambda <\min \{\varLambda _{0},\varLambda _{1} \}$. The proof is complete. □

Theorem 3.6

Assume$0<\alpha <1$, $0<\gamma <1$, there exists$\lambda _{*}>0$such that$0<\lambda <\lambda _{*}$, problem (2.3) has at least a positive solution$v_{\alpha }\in H_{0}^{1}(\varOmega )$satisfying$I_{\lambda ,\alpha }(v_{\alpha })>0$.

Proof of Theorem 3.6

Let $\lambda _{*}=\min \{\varLambda _{0},\varLambda _{1}\}$, then Lemmas 3.3 and 3.5 hold for $0<\lambda <\lambda _{*}$. As a matter of fact, according to Remark 3.4, we have $I_{\lambda ,\alpha }(0)=0$, $I_{\lambda ,\alpha }(v_{0})<0$ and $I_{\lambda ,\alpha }|_{\overline{B_{\rho }}}>0$. By Lemma 3.5, we can choose $T_{0}>0$ large enough so that $I_{\lambda ,\alpha }(v_{0}+T_{0}v_{\varepsilon })<0$. Consequently, $I_{\lambda ,\alpha }$ satisfies the geometry of the mountain pass lemma [28]. Applying the mountain pass lemma, there exists a sequence $\{v_{n}\}\subset H_{0}^{1}$ such that

$$ I_{\lambda ,\alpha }(v_{n})\rightarrow c>0 \quad \text{and}\quad I_{ \lambda ,\alpha }^{\prime }(v_{n}) \rightarrow 0, \quad \text{as } n \rightarrow \infty , $$

(3.16)

where

$$ c =\inf_{\gamma \in \varGamma } \max_{t\in [0,1]} I_{\lambda ,\alpha }\bigl( \gamma (t)\bigr) $$

and

$$ \varGamma =\bigl\{ \gamma \in C\bigl([0,1],H_{0}^{1} \bigr):\gamma (0)=u_{0},\gamma (1)=v_{0}+T_{0}v_{\varepsilon } \bigr\} . $$

Moreover, by Lemmas 3.2 and 3.5, we get

$$ 0< \kappa < c\leq \max_{t\in [0,1]}I_{\lambda ,\alpha }(v_{0}+T_{0}v_{\varepsilon }) \leq \sup_{t\geq 0}I_{\lambda ,\alpha }(v_{0}+T_{0}v_{\varepsilon })< \frac{1}{3}S^{\frac{3}{2}}-D\lambda ^{ \frac{2}{1+\gamma }}. $$

(3.17)

According to Lemma 3.3, we know that $\{v_{n}\}\subset H_{0}^{1}(\varOmega )$ has a convergent subsequence, still denoted by $\{v_{n}\}$, we may assume that $v_{n}\rightarrow v_{\alpha }$ in $H_{0}^{1}(\varOmega )$ as $n\rightarrow \infty$. Hence, from (3.16) and (3.17) we have

$$ I_{\lambda ,\alpha }(v_{\alpha })=\lim _{n\rightarrow \infty }I_{ \lambda ,\alpha }(v_{n})=c>\kappa >0, $$

(3.18)

which implies $v_{\alpha }\not \equiv 0$. Furthermore, from the continuity of $I_{\lambda ,\alpha }^{\prime }$, we find that $v_{\alpha }$ is a solution of problem (2.3), namely

$$ \int _{\varOmega }(\nabla v_{\alpha },\nabla \varphi )\,dx - \int _{\varOmega } \phi _{u}\bigl( v_{\alpha }^{+} \bigr)\varphi \,dx - \int _{\varOmega }\bigl( v_{\alpha }^{+} \bigr)^{5} \varphi \,dx-\lambda \int _{\varOmega } \frac{\varphi }{( v_{\alpha }^{+}+\alpha )^{\gamma }}\,dx=0 $$

for all $\varphi \in H_{0}^{1}(\varOmega )$. Taking the test function $\varphi = v_{\alpha }^{-}$, we have

$$ - \bigl\Vert v_{\alpha }^{-} \bigr\Vert ^{2}=\lambda \int _{\varOmega } \frac{v_{\alpha }^{-}}{( v_{\alpha }^{+}+\alpha )^{\gamma }}\,dx\geq 0, $$

we infer that $v_{\alpha }^{-}=0$. Then we deduce that $v_{\alpha }\geq 0$ and $v_{\alpha }\not \equiv 0$. Hence, by the strong maximum principle, we obtain $v_{\alpha }>0$ in Ω and $v_{\alpha }$ is a positive solution of problem (2.3). The proof is complete. □

Theorem 3.7

Assume$0<\alpha <1$, $0<\gamma <1$, there exists$\lambda _{*}>0$such that$0<\lambda <\lambda _{*}$, problem (2.3) has at least a positive solution$v_{\alpha }\in H_{0}^{1}(\varOmega )$satisfying$I_{\lambda ,\alpha }(v_{\alpha })>0$.

Proof of Theorem 3.7

From Lemma 3.2, by applying Ekeland’s variational principle in $\overline{B_{\rho }}$, there exists a minimizing sequence $\{u_{n}\}\subset \overline{B_{\rho }}$ such that

$$ I_{\lambda ,\alpha }(u_{n})\leq \inf_{u\in \overline{B_{\rho }}}I_{ \lambda ,\alpha }(u)+ \frac{1}{n},\qquad I_{\lambda ,\alpha }(v)\geq I_{ \lambda ,\alpha }(u_{n})- \frac{1}{n} \Vert v-u_{n} \Vert , \quad v\in \overline{B_{\rho }}. $$

Therefore,

$$ I'_{\lambda ,\alpha }(u_{n})\to 0 \quad \text{and}\quad I_{\lambda , \alpha }(u_{n})\to d. $$

Since $\{u_{n}\}$ is bounded and $\overline{B_{\rho }}$ is a closed convex set, there exist $u_{\alpha }\in \overline{B_{\rho }}\subset H_{0}^{1}(\varOmega )$ and a subsequence still denoted by $\{u_{n}\}$ such that $u_{n}\rightharpoonup u_{\lambda }$ in $H_{0}^{1}(\varOmega )$ as $n\to \infty$.

Note that $I_{\lambda ,\alpha }(|u_{n}|)=I_{\lambda ,\alpha }(u_{n})$, by Lemma 3.3, we can obtain $u_{n}\to u_{\alpha }$ in $H_{0}^{1}(\varOmega )$ and $d=\lim_{n\to \infty }I_{\lambda ,\alpha }(u_{n})=I_{\lambda , \alpha }(u_{\alpha })<0$, which suggests that $u_{\lambda }\geq 0$ and $u_{\alpha }\not \equiv 0$. Similar to Theorem 3.6, we obtain $u_{\alpha }>0$ in Ω, then $u_{\alpha }$ is a solution of problem (2.3) with $I_{\lambda ,\alpha }(u_{\alpha })<0$. The proof is complete. □

4 Existence of positive solutions for system (1.1)

Proof of Theorem 1.1

Now, we need to prove that system (1.1) has two positive solutions. Let $\{v_{\alpha }\}$ be a family of positive solutions of problem (2.3), one has

$$ \Vert v_{\alpha } \Vert - \int _{\varOmega }\phi _{v_{\alpha }}v_{\alpha }^{2} \,dx - \int _{ \varOmega }v_{\alpha }^{6}\,dx-\lambda \int _{\varOmega }(v_{\alpha }+\alpha )^{- \gamma }v_{\alpha } \,dx=0. $$

(4.1)

Hence, it follows from (2.1), (3.2), and (4.1) that

$$\begin{aligned} \frac{1}{3}S^{\frac{3}{2}}-D\lambda ^{ \frac{2}{1+\gamma }} >& I_{\lambda ,\alpha } (v_{ \alpha } )-\frac{1}{4} \bigl\langle I_{\lambda ,\alpha }^{ \prime } (v_{\alpha } ), v_{\alpha } \bigr\rangle \\ =& \frac{1}{4} \Vert v_{\alpha } \Vert ^{2}+ \frac{1}{12} \int _{\varOmega }v_{\alpha }^{6}\,dx+ \frac{\lambda }{4} \int _{ \varOmega }\frac{v_{\alpha }}{v_{\alpha }+\alpha }\,dx \\ & {} -\frac{\lambda }{1-\gamma } \int _{\varOmega } \bigl[ (v_{\alpha }+\alpha )^{1-\gamma }-\alpha ^{1-\gamma } \bigr]\,dx \\ \geq & \frac{1}{4} \Vert v_{\alpha } \Vert ^{2}+ \frac{1}{12} \int _{\varOmega }v_{\alpha }^{6}\,dx- \frac{\lambda }{1-\gamma } \int _{\varOmega } \bigl(v_{\alpha }^{+} \bigr)^{1-\gamma }\,dx \\ \geq & \frac{1}{4} \Vert v_{\alpha } \Vert ^{2}+ \frac{1}{12} \int _{\varOmega }v_{\alpha }^{6}\,dx- \frac{\lambda }{1-\gamma } \vert \varOmega \vert ^{\frac{5+\gamma }{6}} S^{- \frac{1-\gamma }{2}} \Vert v_{\alpha } \Vert ^{1-\gamma }. \end{aligned}$$

Obviously, $\{v_{\alpha }\}$ is bounded in $H_{0}^{1}(\varOmega )$ for $0<\gamma <1$. Going if necessary to a subsequence, also denoted by $\{v_{\alpha }\}$, there exists $\{v_{*}\} \in H_{0}^{1}(\varOmega )$ such that

$$ \textstyle\begin{cases} v_{\alpha }\rightharpoonup v_{*},\quad \text{weakly in } H_{0}^{1}(\varOmega ), \\ v_{\alpha }\rightarrow v_{*}, \quad \text{strongly in } L^{p}(\varOmega )\ (1\leq p< 6), \\ v_{\alpha }(x)\rightarrow v_{*}(x), \quad \text{a.e. in } \varOmega . \end{cases} $$

(4.2)

Next, we prove that $(v_{*},\phi _{u_{*}})$ is a pair solution of system (1.1). Notice that $\{v_{\alpha }\}$ satisfies problem (2.3), with an easy computation, we get that

$$ -\Delta v_{\alpha }\geq v_{\alpha }^{5}+ \frac{\lambda }{(v_{\alpha }+\alpha )^{\gamma }}\geq \min \biggl\{ 1, \frac{\lambda }{2^{\gamma }}\biggr\} , $$

it follows that $-\Delta v_{\alpha }\geq \min \{1,\frac{\lambda }{2^{\gamma }}\}$. We denote by e the positive solution of

$$ \textstyle\begin{cases} -\Delta u=1 , &\text{in } \varOmega ,\\ u=0, &\text{on } \partial \varOmega . \end{cases} $$

Hence, we get that $e>0$ by using the strong maximum principle. For every $\varOmega _{0}\subset \subset \varOmega$, there exists $e_{0}>0$ such that $e|_{\varOmega _{0}}\geq e$; therefore, by comparison principle, we get

$$ v_{\alpha }\geq \min \biggl\{ 1,\frac{\lambda }{2^{\gamma }}\biggr\} e. $$

In particular, from $e|_{\varOmega _{0}}\geq e>0$, we deduce that

$$ v_{\alpha }|_{\varOmega _{0}}\geq \min \biggl\{ 1,\frac{\lambda }{2^{\gamma }} \biggr\} e_{0}>0. $$

Now, we shall prove that $v_{\alpha }\rightarrow v_{*}$ as $\alpha \rightarrow 0$. It is similar to [29], for ang $\varphi \in H_{0}^{1}(\varOmega )$, we have

$$ \int _{\varOmega }(\nabla v_{*},\nabla \varphi )\,dx- \int _{\varOmega }\phi _{v_{*}}v_{*} \varphi \,dx - \int _{\varOmega }v_{*}^{5}\varphi \,dx- \lambda \int _{\varOmega }v_{*}^{- \gamma }\varphi \,dx=0. $$

(4.3)

Then, take a test function $\varphi =v_{*}$ in (4.3), there holds

$$ \Vert v_{*} \Vert ^{2}- \int _{\varOmega }\phi _{v_{*}}v_{*}^{2} \,dx - \int _{\varOmega }v_{*}^{6}\,dx- \lambda \int _{\varOmega }v_{*}^{1-\gamma }\,dx=0. $$

(4.4)

Without loss of generality, set $w_{\alpha }=v_{\alpha }-v_{*}$, then $\|w_{\alpha }\|\rightarrow 0$ as $\alpha \rightarrow 0$. Otherwise, there exists a subsequence (still denoted by $w_{\alpha }$) such that $\lim_{\alpha \rightarrow 0}\|w_{\alpha }\|=l>0$. Notice the given condition $\alpha >0$, we obtain $0\leq \frac{v_{\alpha }}{(v_{\alpha }+\alpha )^{\gamma }}\leq v_{\alpha }^{1- \gamma }$, by the Hölder inequality and subadditivity, from (4.2), we have

$$\begin{aligned} \int _{\varOmega }\frac{v_{\alpha }}{(v_{\alpha }+\alpha )^{\gamma }}\,dx \leq & \int _{\varOmega }{v_{\alpha }}^{1-\gamma }\leq \int _{\varOmega } \vert w_{\alpha } \vert ^{1-\gamma }\,dx+ \int _{\varOmega }{v_{*}}^{1-\gamma }\,dx \\ \leq & \Vert w_{\alpha } \Vert _{2}^{1-\gamma } \vert \varOmega \vert ^{ \frac{1+\gamma }{2}}+ \int _{\varOmega }{v_{*}}^{1-\gamma }\,dx \\ \leq & \int _{\varOmega }{v_{*}}^{1-\gamma }\,dx+o(1). \end{aligned}$$

Similarly,

$$ \int _{\varOmega }{v_{*}}^{1-\gamma }\,dx\leq \int _{\varOmega }\frac{v_{\alpha }}{(v_{\alpha }+\alpha )^{\gamma }}\,dx+o(1). $$

Hence, one has

$$ \lim_{\alpha \rightarrow 0} \int _{\varOmega }\frac{v_{\alpha }}{(v_{\alpha }+\alpha )^{\gamma }}\,dx= \int _{\varOmega }{v_{*}}^{1- \gamma }\,dx. $$

Using the Brézis–Lieb lemma and by $\langle I'_{\alpha }(v_{\alpha }), v_{\alpha }\rangle =0$, there holds

$$ \Vert w_{\alpha } \Vert ^{2}+ \Vert v_{*} \Vert ^{2}- \int _{\varOmega }\phi _{v_{*}}(v_{*})^{2} \,dx- \int _{\varOmega }w_{\alpha }^{6}\,dx- \int _{\varOmega }v_{*}^{6}\,dx-\lambda \int _{\varOmega }v_{*}^{1-\gamma }\,dx=o(1). $$

(4.5)

It follows from (4.4) and (4.5) that

$$ \Vert w_{\alpha } \Vert ^{2}- \int _{\varOmega }w_{\alpha }^{6}\,dx=o(1). $$

(4.6)

Then (2.1) and (4.6) imply that

$$ l^{2}\geq S^{\frac{3}{2}}. $$

(4.7)

From (3.2), (4.4) and using the Young inequality, we have

$$\begin{aligned} I_{\lambda }(v_{*}) =& \frac{1}{2} \Vert v_{*} \Vert ^{2}- \frac{1}{4} \int _{\varOmega }\phi _{v_{*}}v_{0}^{2} \,dx-\frac{1}{6} \int _{ \varOmega } v_{*}^{6}\,dx- \frac{\lambda }{1-\gamma } \int _{\varOmega }v_{*}^{1- \gamma }\,dx \\ =& \frac{1}{4} \Vert v_{*} \Vert ^{2}+ \frac{1}{12} \int _{\varOmega } v_{*}^{6}\,dx- \lambda \biggl(\frac{1}{1-\gamma }-\frac{1}{4} \biggr) \int _{\varOmega }v_{*}^{1- \gamma }\,dx \\ \geq & \frac{1}{4} \Vert v_{*} \Vert ^{2}-\lambda \biggl( \frac{1}{1-\gamma }-\frac{1}{4} \biggr) \vert \varOmega \vert ^{\frac{5+\gamma }{6}}S^{- \frac{1-\gamma }{2}} \Vert v_{*} \Vert ^{1-\gamma } \\ \geq & -D\lambda ^{\frac{2}{1+\gamma }}, \end{aligned}$$

where $D=\frac{1+\gamma }{4(1-\gamma )} (\frac{3+\gamma }{2}|\varOmega |^{ \frac{5+\gamma }{6}}S^{-\frac{1-\gamma }{2}} )^{ \frac{2}{1+\gamma }}$. Moreover, from (3.15), (4.7), and the Brézis–Lieb lemma, one has

$$\begin{aligned} I_{\lambda }(v_{*}) \leq & I_{\lambda ,\alpha }(v_{\alpha })- \frac{1}{2} \Vert w_{\alpha } \Vert ^{2}+ \frac{1}{6} \int _{\varOmega } \vert w_{\alpha } \vert ^{6}\,dx \\ < & \frac{1}{3}S^{\frac{2}{{2}}}-D\lambda ^{ \frac{2}{1+\gamma }}- \frac{1}{3} \Vert w_{n} \Vert ^{2} \\ < & \frac{1}{3}S^{\frac{2}{{2}}}-D\lambda ^{ \frac{2}{1+\gamma }}- \frac{1}{3}S^{\frac{3}{2}} \\ =& -D\lambda ^{\frac{2}{1+\gamma }}. \end{aligned}$$

It is obvious that the above inequalities are impossibility. Thus, we get $l=0$, which yields $v_{\alpha }\rightarrow v_{*}$ in $H_{0}^{1}(\varOmega )$ as $\alpha \rightarrow 0$.

In addition, we claim that $I_{\lambda ,\alpha }$ is uniformly bounded. In fact, define a function $f(t)=-(u+t)^{1-\gamma }+t^{1-\gamma }$, we easily get $f^{\prime }(t)<0$ for $t>0$. Obviously, $f(t)$ is decreasing for $0< t<1$. It follows that

$$ I_{\lambda ,1}(u)< I_{\lambda ,\alpha }(u)< I_{\lambda ,0}(u) $$

for $u\in H_{0}^{1}(\varOmega )$. So the claim is true. Therefore, by (3.18), we have $I_{\alpha }(v_{*})=\lim_{\alpha \rightarrow 0}I_{\lambda ,\alpha }v_{\alpha }=c>0$.

Similarly, by Theorem 3.7, there exists $u_{*}\in H_{0}^{1}(\varOmega )$ such that $u_{\alpha }\rightarrow u_{*}$ and $I_{\alpha }(u_{*})=\lim_{\alpha \rightarrow 0}I_{\lambda ,\alpha }(u_{\alpha })=d<0$.

Therefore, $u_{*}$, $v_{*}$ are two different positive solutions of problem (2.2). And $(u_{*}, \phi _{u_{*}})$, $(v_{*}, \phi _{v_{*}})$ are two pairs of different positive solutions of system (1.1). This completes the proof of Theorem 1.1. □