In this section, we are going to establish a series of lemmas to prove Theorem 1.1, in which we tacitly assume that the conditions in Theorem 1.1 are satisfied. We first set up the variational framework for problem (1.1).
Let \(J(u):E\rightarrow\mathbb{R} \) be the energy functional associated with problem (1.1) defined by (1.16), and its Gateaux derivative is given by (1.18).
Since the functional J is not bounded from below on E, a good candidate of an appropriate subset to study J is the so-called Nehari manifold for problem (1.1):
$$ \mathcal{N}= \bigl\{ u\in E\setminus\{0\} : J'(u)u=0 \bigr\} = \bigl\{ u\in E\setminus\{0\} : \|u\|_{E}^{2}+F(u)= \|u\|_{m,a}^{m}+\lambda\|u\|_{q,b}^{q} \bigr\} . $$
(2.1)
Notice that, if \(u\in\mathcal{N}\), then
$$\begin{aligned} J(u) =&\biggl(\frac{1}{2}-\frac{1}{m}\biggr)\|u \|_{E}^{2}+\biggl(\frac{1}{4}-\frac {1}{m} \biggr)F(u)+\biggl(\frac{1}{m}-\frac{1}{q}\biggr)\lambda\|u \|_{q,b}^{q} \\ =&\biggl(\frac{1}{2}-\frac{1}{q}\biggr)\|u\|_{E}^{2}+ \biggl(\frac{1}{4}-\frac {1}{q}\biggr)F(u)+\biggl(\frac{1}{q}- \frac{1}{m}\biggr)\|u\|_{m,a}^{m} \\ =&\frac{1}{2}\|u\|_{E}^{2}+\biggl( \frac{1}{4}-\frac{1}{m}\biggr)\|u\|_{m,a}^{m}+ \biggl(\frac {1}{4}-\frac{1}{q}\biggr)\lambda\|u\|_{q,b}^{q}. \end{aligned}$$
(2.2)
Lemma 2.1
The Nehari manifold
\(\mathcal{N}\neq\emptyset\).
Proof
Let \(u\in E\), \(u\not\equiv0\) in \(\mathbb{R}^{3}\). We consider the function
$$ h(t)=J'(tu)tu=t^{2}\|u \|_{E}^{2}+t^{4}F(u)-t^{m}\|u \|_{m,a}^{m}-\lambda t^{q} \|u\|_{q,b}^{q}, \quad t>0. $$
(2.3)
Since \(4< q< m\), it follows that \(h(t)>0\) for small \(t>0\) and \(h(t)\to-\infty\) as \(t\to\infty\). Then there exists \(t_{1}>0\) such that \(h(t_{1})=0\). Obviously, \(t_{1}u\not\equiv0\). Thus, we conclude that \(t_{1}u\in\mathcal{N}\) and \(\mathcal{N}\neq\emptyset\). □
Lemma 2.2
The functional
J
is coercive and bounded from below on
\(\mathcal{N}\). Moreover,
$$ d=\inf_{u\in\mathcal{N}}J(u)>0. $$
(2.4)
Proof
Let \(u\in\mathcal{N}\). Then, from (1.4) and (2.1), it follows that
$$ \|u\|_{E}^{2}\le\|u\|_{E}^{2}+F(u)= \|u\|_{m,a}^{m}+\lambda\|u\|_{q,b}^{q}\le c_{1}\bigl(\|u\|_{E}^{m}+\|u\|_{E}^{q} \bigr) $$
(2.5)
and
$$ 1\le c_{1}\bigl(\|u\|_{E}^{m-2}+\|u \|_{E}^{q-2}\bigr) $$
(2.6)
with some \(c_{1}>1\). If \(\|u\|_{E}\le1\), inequality (2.6) implies \(1\le2c_{1}\|u\|_{E}^{q-2}\). Hence,
$$ \|u\|_{E}\ge (2c_{1})^{\frac{1}{2-q}}. $$
(2.7)
If \(\|u\|_{E}\ge1\), inequality (2.6) gives \(1\le 2c_{1}\|u\|_{E}^{m-2}\) and
$$ \|u\|_{E}\ge\max\bigl\{ 1, (2c_{1})^{\frac{1}{2-m}} \bigr\} . $$
(2.8)
Hence, if \(u\in\mathcal{N}\), it follows from (2.7) and (2.8) that there exists \(c_{2}>0\) such that
$$ \|u\|_{E}\ge c_{2},\quad \forall u\in \mathcal{N}. $$
(2.9)
Moreover, it follows from (2.2) that
$$\begin{aligned} J(u)&=\biggl(\frac{1}{2}-\frac{1}{q}\biggr)\|u \|_{E}^{2}+\biggl(\frac{1}{4}-\frac {1}{q} \biggr)F(u)+\biggl(\frac{1}{q}-\frac{1}{m}\biggr)\|u \|_{m,a}^{m} \\ &\ge\biggl(\frac{1}{2}-\frac{1}{q} \biggr)\|u\|_{E}^{2}\ge \biggl(\frac{1}{2}- \frac{1}{q}\biggr)c_{2}^{2}\equiv c_{3}. \end{aligned}$$
This shows that the functional J is coercive and bounded from below on \(\mathcal{N}\) and \(d\ge c_{3}>0\). Then the proof of Lemma 2.2 is completed. □
Let \(\{u_{n}\}\) be a minimizing sequence for d in \(\mathcal{N}\), that is, \(J(u_{n})\to d\) as \(n\to\infty\) and
$$ \|u_{n}\|_{E}^{2}+F(u_{n})= \|u_{n}\|_{m,a}^{m}+\lambda\|u_{n} \|_{q,b}^{q}, \quad \forall n\in\mathbb{N}. $$
(2.10)
Furthermore, it follows from (2.2) that
$$ J(u_{n})\ge \biggl(\frac{1}{2}-\frac{1}{q} \biggr)\|u_{n}\|_{E}^{2},\quad \forall n\in \mathbb{N}. $$
(2.11)
This shows that \(\{u_{n}\}\) is bounded in E and, from (2.10), \(\{u_{n}\}\) is bounded in \(L^{m}(\mathbb{R}^{3},a)\) and \(L^{q}(\mathbb{R}^{3},b)\). Therefore, up to a subsequence, there exists \(u\in E\) such that as \(n\to\infty\),
$$\begin{aligned} \textstyle\begin{cases} u_{n}\rightharpoonup u \quad \mbox{weakly in } E, L^{m}(\mathbb{R}^{3},a), L^{q}(\mathbb{R}^{3},b); \\ u_{n}\to u\quad \mbox{strongly in } L^{p}_{\mathrm{loc}}(\mathbb{R}^{3}), 1\le p< 6, \\ u_{n}(x)\to u(x)\quad \mbox{a.e. in } \mathbb{R}^{3};\qquad \|u_{n}\|_{E}, \|u_{n}\|_{p} \le M_{0},\quad \forall n\ge1, 2\le p\le6, \end{cases}\displaystyle \end{aligned}$$
(2.12)
with some \(M_{0}>0\). Since \(J(u_{n})=J(|u_{n}|)\), we assume \(u_{n}(x)\ge0\) a.e. in \(\mathbb{R}^{3}\) for every \(n\ge1\) and thus \(u(x)\ge0\) a.e. in \(\mathbb{R}^{3}\). By the weak lower semi-continuity of the norm, we get
$$ \|u\|_{m,a}^{m}\le\beta:=\liminf _{n\to\infty}\|u_{n}\|_{m,a}^{m}. $$
(2.13)
By extracting a further subsequence, if necessary, we assume
$$ \beta=\lim_{n\to\infty}\|u_{n} \|_{m,a}^{m}. $$
(2.14)
Denote
$$ \theta=\|u\|_{m,a}^{m}= \int_{\mathbb{R}^{3}}a(x)|u|^{m}\,dx,\qquad \sigma=\|u \|_{q,b}^{q}= \int_{\mathbb{R}^{3}}b(x)|u|^{q}\,dx. $$
(2.15)
By weak convergence, it is obvious that \(\theta\in[0, \beta]\). First, we have the following.
Lemma 2.3
There results
\(\beta>0\).
Proof
Obviously, \(\beta\ge0\). If \(\beta=0\), we have \(\lim_{n\to\infty}\|u_{n}\|_{m,a}^{m}=0\). Let \(t\in(0,1)\) be such that \(q=2t+(1-t)m\). Then, by the Hölder inequality and (H5), we derive
$$ \|u_{n}\|_{q,b}^{q}\le b_{1}\|u_{n}\|_{2}^{2t} \|u_{n}\|_{m,a}^{m(1-t)},\quad \forall n\in\mathbb{N} $$
(2.16)
with the constant \(b_{1}=a_{0}^{t-1}a_{1}\). Since \(\{u_{n}\}\) is bounded in E, so it is bounded in \(L^{p}(\mathbb{R}^{3})\) (\(2\le p\le6\)), and it follows from (2.16) that \(\lim_{n\to\infty}\|u_{n}\|_{q,b}^{q}=0\). Then (2.10) implies that \(u_{n}\to0\) in E and \(F(u_{n})\to0\) as \(n\to\infty\) and so \(d=0\). By Lemma 2.2, it is impossible. The proof of Lemma 2.3 is completed. □
Lemma 2.4
If
\(\beta=\theta\), then
\(u\in\mathcal{N}\)
and
\(J(u)=d\).
Proof
If \(\beta=\theta\), then \(\lim_{n\to\infty}\|u_{n}\|_{m,a}^{m}=\|u\|_{m,a}^{m}\). Since \(u_{n}\rightharpoonup u\) in \(L^{m}(\mathbb{R}^{3})\), it follows from the Brezis-Lieb lemma that \(u_{n}\to u\) in \(L^{m}(\mathbb{R}^{3},a)\). Similar to (2.16), we have
$$ \|u_{n}-u\|_{q,b}^{q}\le b_{1}\|u_{n}-u\|_{2}^{2t} \|u_{n}-u\|_{m,a}^{m(1-t)},\quad \forall n\ge1, $$
(2.17)
and \(u_{n}\to u\) in \(L^{q}(\mathbb{R}^{3},b)\). Hence, by the weak lower semi-continuity of the norm, we obtain
$$\begin{aligned} J(u)&=\frac{1}{2}\|u\|_{E}^{2}+\frac{1}{4}F(u)- \frac{1}{m}\|u\| _{m,a}^{m}-\frac{\lambda}{q}\|u \|_{q,b}^{q} \\ &\le \frac{1}{2}\liminf_{n\to\infty}\|u_{n} \|_{E}^{2}+\frac{1}{4}\liminf_{n\to\infty}F(u_{n}) -\frac{1}{m}\lim_{n\to\infty}\|u_{n} \|_{m,a}^{m}-\frac{\lambda }{q}\lim_{n\to\infty} \|u_{n}\|_{q,b}^{q} \\ &\le\liminf_{n\to\infty} \biggl(\frac{1}{2} \|u_{n}|_{E}^{2}+\frac{1}{4}F(u_{n}) -\frac{1}{m}\|u_{n}\|_{m,a}^{m}- \frac{\lambda}{q}\|u_{n}\|_{q,b}^{q} \biggr) \\ &= \liminf_{n\to\infty}J(u_{n})=d. \end{aligned}$$
(2.18)
Furthermore, we have from (2.10) that
$$ \|u\|_{E}^{2}+F(u)\le\|u\|_{m,a}^{m}+ \lambda\|u\|_{q,b}^{q}. $$
(2.19)
If the equality in (2.19) holds, then \(u\in\mathcal{N}\) and the lemma is proved.
We now assume that the equality in (2.19) fails to hold. Let
$$ h(t)=J'(tu) (tu)=t^{2}\|u \|_{E}^{2}+t^{4}F(u)- t^{m}\|u \|_{m,a}^{m}-\lambda t^{q}\|u\|_{q,b}^{q}, \quad t\ge0. $$
(2.20)
Clearly, \(h(t)>0\) for small \(t>0\) and \(h(1)<0\). Then there exists \(t\in(0,1)\) such that \(h(t)=0\), and then \(tu\in\mathcal{N}\) and
$$\begin{aligned} d&\le J(tu)= p_{1}t^{2}\|u\|_{E}^{2}+p_{2}t^{4}F(u)+p_{3}t^{m} \|u\|_{m,a}^{m} \\ &< p_{1}\|u\|_{E}^{2}+p_{2}F(u)+p_{3} \|u\|_{m,a}^{m}\le\liminf_{n\to \infty}J(u_{n})=d. \end{aligned}$$
(2.21)
Here and in the sequel,
$$ p_{1}=2^{-1}-q^{-1}>0,\qquad p_{2}=4^{-1}-q^{-1}>0,\qquad p_{3}=q^{-1}-m^{-1}>0. $$
(2.22)
Relation (2.21) is a contradiction, and thus the proof of Lemma 2.4 is completed. □
We now turn to study the value of θ. First we introduce the Sobolev space
$$ Y= \bigl\{ u\in W^{1,2}\bigl(\mathbb{R}^{3} \bigr) : \|\nabla u\|_{2}^{2}+\alpha\|u\|_{2}^{2}< \infty \bigr\} $$
(2.23)
endowed with the norm
$$ \|u\|_{Y}:=\bigl(\|\nabla u\|_{2}^{2}+ \alpha\|u\|_{2}^{2}\bigr)^{1/2}= \biggl( \int_{\mathbb{R}^{3}}\bigl(|\nabla u|^{2}+\alpha|u|^{2} \bigr)\,dx \biggr)^{1/2}, $$
(2.24)
where α is the positive number defined in (H2). Consider the Schrödinger-Poisson system
$$ \textstyle\begin{cases} -\Delta u+\alpha u+K(x)\phi u=a(x)|u|^{m-2}u+\lambda b(x)|u|^{q-2}u, & \mbox{in } \mathbb {R}^{3}, \\ -\Delta\phi=K(x)u^{2},\qquad \lim_{|x|\to\infty}\phi(x)=0, &\mbox{in } \mathbb {R}^{3}. \end{cases} $$
(2.25)
The functional associated with problem (2.25) is
$$ J_{\alpha}(u)=\frac{1}{2}\|u\|_{Y}^{2}+ \frac{1}{4}F(u)-\frac{1}{m}\|u\| _{m,a}^{m}- \frac{\lambda}{q}\|u\|_{q,b}^{q}, $$
(2.26)
and the associated Nehari manifold is
$$ \mathcal{N}_{\alpha}= \bigl\{ u\in Y\setminus\{0\} : J'_{\alpha}(u)u=0 \bigr\} = \bigl\{ u\in Y\setminus\{0\} : \|u \|_{Y}^{p}+F(u)=\|u\|_{m,a}^{m}+\lambda\|u \|_{q,b}^{q} \bigr\} . $$
(2.27)
Notice that \(u\in\mathcal{N}_{\alpha}\),
$$\begin{aligned} J_{\alpha}(u)&=\biggl(\frac{1}{2}-\frac{1}{m}\biggr)\|u \|_{Y}^{2}+\biggl(\frac {1}{4}-\frac{1}{m} \biggr)F(u)+\biggl(\frac{1}{m}-\frac{1}{q}\biggr)\lambda\|u \|_{q,b}^{q} \\ &=\biggl(\frac{1}{2}-\frac{1}{q}\biggr)\|u\|_{Y}^{2}+ \biggl(\frac{1}{4}-\frac {1}{q}\biggr)F(u)+\biggl(\frac{1}{q}- \frac{1}{m}\biggr)\|u\|_{m,a}^{m} \\ &=\frac{1}{2}\|u\|_{Y}^{2}+\biggl(\frac{1}{4}- \frac{1}{m}\biggr)\|u\|_{m,a}^{m}+\biggl( \frac {1}{4}-\frac{1}{q}\biggr)\lambda\|u\|_{q,b}^{q}. \end{aligned}$$
(2.28)
Finally, we define
$$ d_{\alpha}=\inf_{u\in\mathcal{N}_{\alpha}}J_{\alpha}(u). $$
(2.29)
Lemma 2.5
There results
\(d< d_{\alpha}\), that is, \(\inf_{u\in \mathcal{N}}J(u)<\inf_{u\in \mathcal{N}_{\alpha}}J_{\alpha}(u)\).
Proof
Similar to the proofs of Lemmas 2.1 and 2.2, we can obtain that \(\mathcal{N}_{\alpha}\neq\emptyset\) and \(d_{\alpha}>0\). Arguing in Lemma 3.1.9 in [20], we see that there exists the nonnegative function \(u_{0}\in\mathcal{N}_{\alpha}\) such that \(J_{\alpha}(u_{0})=d_{\alpha}\). On the other hand, from (H1)-(H2), we infer
$$ \int_{\mathbb{R}^{3}}V(x)|u_{0}|^{2}\,dx< \alpha \int_{\mathbb{R}^{3}}|u_{0}|^{2}\,dx, $$
(2.30)
which implies that
$$ \int_{\mathbb{R}^{3}}\bigl(|\nabla u_{0}|^{2}+V(x)|u_{0}|^{2} \bigr)\,dx< \int_{\mathbb{R}^{3}}\bigl(|\nabla u_{0}|^{2}+ \alpha|u_{0}|^{2}\bigr)\,dx, \quad \mbox{or}\quad \|u_{0}\|_{E}^{2}< \|u_{0} \|_{Y}^{2}. $$
(2.31)
Then we have
$$\begin{aligned} J'(u_{0})u_{0}&=\|u_{0} \|_{E}^{2}+F(u_{0})-\|u_{0} \|_{m,a}^{m}-\lambda\|u_{0}\|_{q,b}^{q} \\ &< \|u_{0}\|_{Y}^{2}+F(u_{0})- \|u_{0}\|_{m,a}^{m}-\lambda\|u_{0} \|_{q,b}^{q}=J'_{\alpha }(u_{0})u_{0}=0. \end{aligned}$$
(2.32)
As the argument of Lemma 2.4, there exists \(t\in(0, 1)\) such that \(tu_{0}\in\mathcal{N}\), so that
$$\begin{aligned} d&\le J(tu_{0})=p_{1}t^{2}\|u_{0} \|_{E}^{2}+ p_{2}t^{4}F(u_{0})+p_{3}t^{m} \|u_{0}\|_{m,a}^{m} < p_{1} \|u_{0}\|_{E}^{2}+ p_{2}F(u_{0})+p_{3} \|u_{0}\|_{m,a}^{m} \\ &< p_{1}\|u_{0}\|_{Y}^{2}+ p_{2}F(u_{0})+p_{3}\|u_{0} \|_{m,a}^{m} =J_{\alpha}(u_{0})=d_{\alpha}, \end{aligned}$$
(2.33)
where \(p_{i}\) is given by (2.22). Then (2.33) finishes the proof of Lemma 2.5. □
Lemma 2.6
There results
\(\theta>0\).
Proof
If \(\theta=0\), then \(u=0\), which implies in particular that \(u_{n}\to 0\) in \(L^{p}_{\mathrm{loc}}(\mathbb{R}^{3})\) (\(1\le p<6\)). We first prove the following claim:
$$ \lim_{n\to\infty} \int_{\mathbb{R}^{3}}\bigl(\alpha-V(x)\bigr)|u_{n}|^{2} \,dx=0. $$
(2.34)
By (H2), for any \(\varepsilon>0\), there exists \(R_{\varepsilon}>0\) such that \(0<\alpha-V(x)\le\varepsilon, \forall|x|\ge R_{\varepsilon}\). Then
$$\begin{aligned} \int_{\mathbb{R}^{3}}\bigl(\alpha-V(x)\bigr)|u_{n}|^{2} \,dx&= \int_{|x|\le R_{\varepsilon}}\bigl(\alpha-V(x)\bigr)|u_{n}|^{2} \,dx+ \int_{|x|\ge R_{\varepsilon}}\bigl(\alpha-V(x)\bigr)|u_{n}|^{2} \,dx \\ &\le\varepsilon \int_{|x|\ge R_{\varepsilon}}|u_{n}|^{2}\,dx+C_{1} \int_{|x|\le R_{\varepsilon}}|u_{n}|^{2}\,dx \\ &\le M_{1}\varepsilon+C_{1} \int_{|x|\le R_{\varepsilon}}|u_{n}|^{2}\,dx, \end{aligned}$$
(2.35)
where
$$ C_{1}=\max_{x\in\mathbb{R}^{3}} \bigl\vert \alpha-V(x) \bigr\vert ,\qquad M_{1}=\sup_{n\ge 1} \|u_{n}\|_{2}^{2}< \infty. $$
(2.36)
Noticing that
$$ \lim_{n\to\infty} \int_{|x|\le R_{\varepsilon}}|u_{n}|^{2}\,dx=0, $$
we have from (2.35) that
$$ \limsup_{n\to\infty} \int_{\mathbb{R}^{3}}\bigl(\alpha-V(x)\bigr)|u_{n}|^{2} \,dx\le M_{1}\varepsilon. $$
(2.37)
Since ε is arbitrarily small, (2.37) implies that (2.34), and then
$$ \|u_{n}\|_{Y}^{2}= \|u_{n}\|_{E}^{2}+\epsilon_{n}, $$
(2.38)
where \(\epsilon_{n}\to0\) as \(n\to\infty\).
On the other hand, the fact \(u_{n}\in \mathcal{N}\) shows that
$$ \|u_{n}\|_{m,a}^{m}+\lambda \|u_{n}\|_{q,b}^{q}=\|u_{n} \|_{E}^{2}+F(u_{n})\le \|u_{n} \|_{Y}^{2}+F(u_{n}). $$
(2.39)
Denote
$$ \gamma_{n}(t)=J'_{\alpha}(tu_{n})tu_{n}=t^{2} \|u_{n}\|_{Y}^{2}+t^{4}F(u_{n})- t^{m}\|u_{n}\|_{m,a}^{m}-\lambda t^{q}\|u_{n}\|_{q,b}^{q},\quad t>0. $$
(2.40)
Since \(4< q< m\) and (2.39), it follows that \(\gamma_{n}(1)\ge0\) and \(\gamma_{n}(t)\to-\infty\) as \(t\to\infty\). Then there exists \(t_{n}\ge1\) such that \(\gamma_{n}(t_{n})=0\), and then \(t_{n}u_{n}\in \mathcal{N}_{\alpha}\), that is,
$$ t_{n}^{2}\|u_{n} \|_{Y}^{2}+t_{n}^{4}F(u_{n})=t_{n}^{m} \|u_{n}\|_{m,a}^{m}+\lambda t_{n}^{q} \|u_{n}\|_{q,b}^{q}. $$
(2.41)
Using the facts that \(\|u_{n}\|_{Y}\) and \(\|u_{n}\|_{q,b}\) are bounded and \(\lim_{n\to\infty}\|u_{n}\|_{m,a}^{m}=\beta\), we deduce from (2.41) that the sequence \(\{t_{n}\}\) is bounded. If necessary, up to a subsequence, we can assume \(t_{n}\to t_{0}\ge1\). Then it follows from (2.38), (2.39) and (2.41) that
$$ \bigl(t_{n}^{2}-t_{n}^{q} \bigr)\|u_{n}\|_{E}^{2}+\epsilon_{n}t_{n}^{2}+ \bigl(t_{n}^{4}-t_{n}^{q} \bigr)F(u_{n})= \bigl(t_{n}^{m}-t_{n}^{q} \bigr)\|u_{n}\|_{m,a}^{m}. $$
(2.42)
Since \(t_{n}^{4}\le t_{n}^{q}\) and \(F(u_{n})\ge0\), we have
$$ \bigl(t_{n}^{m}-t_{n}^{q} \bigr)\|u_{n}\|_{m,a}^{m}-\epsilon_{n}t_{n}^{2} \le \bigl(t_{n}^{2}-t_{n}^{q}\bigr) \|u_{n}\|_{E}^{2},\quad \forall n\ge1. $$
(2.43)
Letting \(n\to\infty\) in (2.43) yields
$$ \bigl(t_{0}^{m}-t_{0}^{q} \bigr)\beta\le\bigl(t_{0}^{2}-t_{0}^{q} \bigr)\nu, $$
(2.44)
where \(\nu=\limsup_{n\to\infty}\|u_{n}\|_{E}^{2}>0\) and \(\beta=\lim_{n\to\infty}\|u_{n}\|_{m,a}^{m}\). Since \(t_{0}\ge1\) and \(4< q< m\), we have \(t_{0}=1\). That is, \(t_{n}\to1\) as \(n\to\infty\). Therefore, it follows from (2.28) and (2.38) that
$$\begin{aligned} d_{\alpha}&\le J_{\alpha}(t_{n}u_{n}) =p_{1}t_{n}^{2}\|u\|_{Y}^{2}+p_{2}t_{n}^{4}F(u_{n}) +p_{3}t_{n}^{m}\|u_{n} \|_{m,a}^{m} \\ &=t_{n}^{2} \bigl[p_{1}\|u_{n} \|_{E}^{2}+p_{2}F(u_{n})+p_{3} \|u_{n}\|_{m,a}^{m} \bigr]+\gamma _{n}=t_{n}^{2}J(u_{n})+ \gamma_{n}, \end{aligned}$$
(2.45)
where \(\gamma_{n}=p_{1}\varepsilon_{n}t_{n}^{2}+p_{2}(t_{n}^{4}-t_{n}^{2})F(u_{n}) +p_{3}(t_{n}^{m}-t_{n}^{2})\|u_{n}\|_{m,a}^{m}\). Since \(t_{n}\to1\), we get \(\gamma_{n}\to0\). Moreover, the facts \(J(u_{n})\to d\) and \(t_{n}\to1\) in (2.45) imply that \(d_{\alpha}\le d\). This contradicts the result in Lemma 2.5. Therefore, we have \(\theta>0\) and complete the proof of Lemma 2.6. □
In the following, we consider the case \(\theta\in(0, \beta)\). As in [20, 21], we let \(\theta\in(0, \beta)\) and take \(\{u_{n}\}\) as a minimizing sequence for d on \(\mathcal{N}\), which satisfies (2.12) and
$$ \begin{aligned} & \int_{B_{r_{n}}}a(x)|u_{n}|^{m}\,dx= \theta+o_{n}(1),\qquad \int_{\Omega_{n}}a(x)|u_{n}|^{m} \,dx=o_{n}(1),\\ &\int_{B_{r_{n}}}b(x)|u|^{q}\,dx=\sigma+o_{n}(1),\qquad \int_{\Omega_{n}}b(x)|u|^{q}\,dx=o_{n}(1), \\ & \int_{B^{c}_{r_{n}}}|u_{n}|^{2}\,dx=o_{n}(1), \qquad \int_{\Omega_{n}}|\nabla u_{n}|^{2} \,dx=o_{n}(1), \\ & \int_{B^{c}_{r_{n}}} \bigl\vert u_{n}(x) \bigr\vert ^{6}\,dx=o_{n}(1),\qquad \|u_{n} \|_{E}, \|u_{n}\|_{p}\le M_{0},\quad \forall n\ge1, 2\le p\le6, \end{aligned} $$
(2.46)
where and in the sequel, \(B_{r}=\{x\in\mathbb{R}^{3}:|x|< r\}\), \(B_{r}^{c}=\{x\in\mathbb{R}^{3}:|x|\ge r\}\), \(\Omega_{n}=\{x\in \mathbb{R}^{3}: r_{n}\le|x|< r_{n+1}\}\) with \(r_{n}\uparrow\infty\) and \(o_{n}(1)\) is a quantity which goes to zero as \(n\to\infty\).
Since \(J(u_{n})=J(|u_{n}|)\) in E, we assume \(u_{n}\ge0\) in \(\mathbb{R}^{3}\). Furthermore, we also consider, for every \(n\in\mathbb{N}\), a function \(\varphi_{n}\in C_{0}^{\infty}(\mathbb{R}^{3})\) such that
-
(1)
\(0\le\varphi_{n}(x)\le1\), \(\forall x\in \mathbb{R}^{3}\),
-
(2)
\(\varphi_{n}(x)=1\) if \(|x|\le r_{n}\), \(\varphi_{n}(x)=0\) if \(|x|\ge r_{n+1}\),
-
(3)
\(|\nabla\varphi_{n}(x)|\le C_{0}\), \(\forall x\in\mathbb {R}^{3}\) and \(\forall n\ge1\),
where \(C_{0}\) is some positive number independent of n. Furthermore, we set
$$ u'_{n}(x)=\varphi_{n}(x)u_{n}(x), \qquad u''_{n}(x)=\bigl(1- \varphi_{n}(x)\bigr)u_{n}(x),\quad x\in\mathbb{R}^{3}. $$
(2.47)
Then \(u'_{n}, u''_{n}\ge0\) and \(u_{n}=u'_{n}+u''_{n}\) in \(\mathbb{R}^{3}\) for every \(n\ge1\).
Lemma 2.7
The following properties for
\(u_{n}\), \(u'_{n}\), \(u''_{n}\)
hold:
- (P1):
-
\(u'_{n}\rightharpoonup u\)
weakly in
\(E=W^{1,2}(\mathbb{R}^{3})\), \(u'_{n}\to u\)
strongly in
\(L^{m}(\mathbb{R}^{3},a)\cap L^{q}(\mathbb{R}^{3},b)\).
- (P2):
-
\(\int_{\mathbb{R}^{3}}a(x)|u_{n}|^{m}\,dx=\int_{\mathbb {R}^{3}}a(x)|u'_{n}|^{m}\,dx+\int_{\mathbb{R}^{3}}a(x)|u''_{n}|^{m}\,dx+o_{n}(1)\).
- (P3):
-
\(\int_{\mathbb{R}^{3}}b(x)|u_{n}|^{q}\,dx=\int_{\mathbb {R}^{3}}b(x)|u'_{n}|^{q}\,dx+\int_{\mathbb{R}^{3}}b(x)|u''_{n}|^{q}\,dx+o_{n}(1)\).
- (P4):
-
\(\|u_{n}\|_{E}^{2}\ge\|u'_{n}\|_{E}^{2}+\|u''_{n}\|_{E}^{2}+o_{n}(1)\).
- (P5):
-
\(F(u_{n})\ge F(u'_{n})+F(u''_{n})\), \(\forall n\ge1\).
- (P6):
-
\(F(u_{n})\le F(u'_{n})+F(u''_{n})+\sum_{j=1}^{5}A_{n}^{j}\), \(\forall n\ge 1\), in which
$$\begin{aligned}& A_{n}^{1}\equiv \int_{\Omega_{n}}K(x)\phi_{u_{n}}(x)u_{n}^{2}(x) \,dx=o_{n}(1), \\& A_{n}^{2}\equiv \int_{\Omega_{n}}K(x)\phi_{u'_{n}}(x)u_{n}^{2}(x) \,dx=o_{n}(1), \\& A_{n}^{3}\equiv \int_{\Omega_{n}}K(x)\phi_{u''_{n}}(x)u_{n}^{2}(x) \,dx=o_{n}(1), \\ & A_{n}^{4}\equiv \int_{B^{c}_{r_{n+1}}}K(x)\phi_{u'_{n}}(x)u_{n}^{2}(x) \,dx=o_{n}(1), \\ & A_{n}^{5}\equiv \int_{B_{r_{n}}}K(x)\phi_{u''_{n}}(x)u_{n}^{2}(x) \,dx=o_{n}(1),\quad n=1,2,\ldots, \end{aligned}$$
(2.48)
where
\(\phi_{u}\)
is defined by (1.10).
Proof
The proof of properties (P1)-(P4) is similar to that in [20, 21] and is omitted. Here, we prove (P5) and (P6). Note that \(u'_{n}(x)=u_{n}(x)\), \(u''_{n}(x)=0\) if \(x\in B_{r_{n}}\) and \(u''_{n}(x)=u_{n}(x)\), \(u'_{n}(x)=0\) if \(x\in B_{r_{n+1}}^{c}\). So,
$$ \bigl\vert u'_{n}(x) \bigr\vert ^{2}+ \bigl\vert u''_{n}(x) \bigr\vert ^{2}= \bigl\vert u_{n}(x) \bigr\vert ^{2},\quad x\in B_{r_{n}}\cup B_{r_{n+1}}^{c}. $$
(2.49)
Moreover, if \(x\in\Omega_{n}\), we have \(0\le\varphi_{n}(x)\le1\) and
$$ \bigl\vert u'_{n}(x) \bigr\vert ^{2}+ \bigl\vert u''_{n}(x) \bigr\vert ^{2}= \bigl\vert u_{n}(x) \bigr\vert ^{2}\bigl( \bigl\vert \varphi_{n}(x) \bigr\vert ^{2}+ \bigl\vert 1-\varphi _{n}(x) \bigr\vert ^{2}\bigr)\le \bigl\vert u_{n}(x) \bigr\vert ^{2}. $$
(2.50)
Thus, one sees that
$$ \bigl\vert u_{n}(x) \bigr\vert ^{2}\ge \bigl\vert u'_{n}(x) \bigr\vert ^{2}+ \bigl\vert u''_{n}(x) \bigr\vert ^{2},\quad \forall x\in \mathbb{R}^{3}. $$
(2.51)
Clearly,
$$\begin{aligned} \phi_{u_{n}}(x) =&\frac{1}{4\pi} \int_{\mathbb {R}^{3}}\frac{K(y)u_{n}^{2}(y)}{|x-y|}\,dy \\ =& \frac{1}{4\pi} \int_{B_{r_{n}}}\frac{K(y)(u'_{n}(y))^{2}}{|x-y|}\,dy+\frac {1}{4\pi} \int_{B^{c}_{r_{n+1}}}\frac{K(y)(u''_{n}(y))^{2}}{|x-y|}\,dy \\ &{}+\frac{1}{4\pi} \int_{\Omega_{n}}\frac{K(y)(u_{n}(y))^{2}}{|x-y|}\,dy \\ =&\frac{1}{4\pi} \int_{\Omega_{n}}\frac{K(y)}{|x-y|} \bigl[\bigl(u_{n}(y) \bigr)^{2}-\bigl(u'_{n}(y)\bigr)^{2}- \bigl(u''_{n}(y)\bigr)^{2} \bigr] \,dy +\phi_{u'_{n}}(x)+\phi_{u''_{n}}(x) \\ \ge&\phi_{u'_{n}}(x)+\phi_{u''_{n}}(x),\quad \forall x\in \mathbb {R}^{3}. \end{aligned}$$
(2.52)
So, we have
$$\begin{aligned} F(u_{n}) =& \int_{\mathbb {R}^{3}}K(x)\phi_{u_{n}}(x)u_{n}^{2}(x) \,dx \\ \ge& \int_{\mathbb {R}^{3}}K(x)\phi_{u'_{n}}(x)u_{n}^{2}(x) \,dx+ \int_{\mathbb {R}^{3}}K(x)\phi _{u''_{n}}(x)u_{n}^{2}(x) \,dx \\ =&F\bigl(u'_{n}\bigr)+F\bigl(u''_{n} \bigr)+ \int_{\Omega_{n}}K(x)\phi_{u'_{n}} \bigl[u_{n}^{2}- \bigl(u'_{n}\bigr)^{2} \bigr]\,dx+ \int_{\Omega_{n}}K(x)\phi_{u''_{n}} \bigl[u_{n}^{2}- \bigl(u''_{n}\bigr)^{2} \bigr] \,dx \\ &{} + \int_{B_{r_{n}}}K(x)\phi_{u''_{n}}\bigl(u'_{n} \bigr)^{2}\,dx + \int_{B^{c}_{r_{n+1}}}K(x)\phi_{u'_{n}}\bigl(u''_{n} \bigr)^{2}\,dx \\ \ge& F\bigl(u'_{n}\bigr)+F\bigl(u''_{n} \bigr),\quad \forall n\ge1, \end{aligned}$$
(2.53)
which proves (P5).
On the other hand, it follows from (2.52) that
$$ \phi_{u_{n}}(x)\le\phi_{u'_{n}}(x)+ \phi_{u''_{n}}(x)+\frac{1}{4\pi} \int_{\Omega_{n}}\frac{K(y)u^{2}_{n}(y)}{|x-y|}\,dy. $$
(2.54)
Noticing that
$$\begin{aligned}& \frac{1}{4\pi} \int_{\mathbb {R}^{3}} \biggl( \int_{\Omega_{n}}\frac{K(y)u_{n}^{2}(y)}{|x-y|} \,dy \biggr)K(x)u_{n}^{2}(x) \,dx \\& \quad =\frac{1}{4\pi} \int_{\Omega_{n}} \biggl( \int _{R^{3}}\frac{K(x)u_{n}^{2}(x)}{|x-y|}\,dx \biggr)K(y)u_{n}^{2}(y) \,dy \\& \quad = \int_{\Omega_{n}}K(x)\phi_{u_{n}}(x)u_{n}^{2}(x) \,dx, \end{aligned}$$
(2.55)
we have from (2.54) and (2.55) that
$$\begin{aligned} \int_{\mathbb {R}^{3}}K(x)\phi_{u_{n}}(x)u_{n}^{2}(x) \,dx \le& \int_{\mathbb {R}^{3}}K(x)u_{n}^{2}(x) \bigl( \phi_{u'_{n}}(x)+\phi_{u''_{n}}(x)\bigr)\,dx+ \int_{\Omega_{n}}K(x)\phi_{u_{n}}(x)u_{n}^{2}(x) \,dx \\ \le& F\bigl(u'_{n}\bigr)+F\bigl(u''_{n} \bigr)+\sum_{j=1}^{5}A_{n}^{j}. \end{aligned}$$
(2.56)
If \(K\in L^{\infty}(\mathbb {R}^{3})\) and \(K\ge0\), we have from (2.12) that
$$\begin{aligned} A_{n}^{1}&= \int_{\Omega_{n}}K\phi_{u_{n}}u_{n}^{2}\,dx \le d_{0}\|\phi_{u_{n}}\|_{L^{6}(\Omega_{n})}\|u_{n} \|^{2}_{L^{\frac{12}{5}}(\Omega _{n})} \\ &\le c_{2}\|u_{n} \|_{E}^{2}\|u_{n}\|^{2}_{L^{\frac{12}{5}}(\Omega_{n})} \le c_{2}M_{0}^{2}\|u_{n} \|^{2}_{L^{\frac{12}{5}}(\Omega_{n})}. \end{aligned}$$
(2.57)
From (2.46), we have \(\|u_{n}\|^{2}_{L^{\frac{12}{5}}(\Omega_{n})}=o_{n}(1)\) and so \(A_{n}^{1}=o_{n}(1)\).
If \(K\in L^{2}(\mathbb {R}^{3})\) and \(K\ge0\), we derive from the Hölder inequality and the Sobolev inequality that
$$\begin{aligned} A_{n}^{1}&\le\|\phi_{u_{n}} \|_{L^{6}(\Omega_{n})}\|K\|_{L^{2}(\Omega_{n})}\|u_{n}\| _{L^{6}(\Omega_{n})}^{2} \\ &\le c_{1}\|u_{n}\|_{E}^{2} \|u_{n}\|_{L^{6}(\Omega_{n})}^{2}\|K\|_{L^{2}(\Omega_{n})}\le c_{1}M_{0}^{4}\|K\|_{L^{2}(\Omega_{n})}. \end{aligned}$$
(2.58)
Similarly, the assumption \(K\in L^{2}(\mathbb {R}^{3})\) implies that \(\|K\|_{L^{2}(\Omega_{n})}=o_{n}(1)\) and \(A_{n}^{1}=o_{n}(1)\).
Furthermore, since \(u_{n}(x)\ge u'_{n}(x), u''_{n}(x)\), we have \(\phi_{u_{n}}(x)\ge\phi_{u'_{n}}(x), \phi_{u''_{n}}(x)\), and then \(0\le A_{n}^{2}, A_{n}^{3}\le A_{n}^{1}=o_{n}(1)\). Noticing that
$$ A_{n}^{4}= \int_{B^{c}_{r_{n+1}}}K(x)\phi_{u'_{n}}(x)u_{n}^{2}(x) \,dx\le \int_{B^{c}_{r_{n}}}K(x)\phi_{u_{n}}(x)u_{n}^{2}(x) \,dx, $$
(2.59)
we argue as in the proof of (2.57) and (2.58) and obtain \(A_{n}^{4}=o_{n}(1)\). Similarly,
$$\begin{aligned} \begin{aligned}[b] A_{n}^{5}&= \int_{B_{r_{n}}}K(x)\phi_{u''_{n}}(x)u_{n}^{2}(x) \,dx=\frac{1}{4\pi} \int_{B_{r_{n}}} \int_{\mathbb {R}^{3}}\frac{K(y)(u''_{n}(y))^{2}}{|x-y|}K(x)u_{n}^{2}(x) \,dy\,dx \\ &\le \frac{1}{4\pi} \int_{B_{r_{n}}} \int_{B_{r_{n}}^{c}}\frac {K(y)u^{2}_{n}(y)}{|x-y|}K(x)u_{n}^{2}(x) \,dy\,dx= \int_{B^{c}_{r_{n}}}K(y)\phi _{u_{n}}(y)u_{n}^{2}(y) \,dy, \end{aligned} \end{aligned}$$
(2.60)
we get from (2.59) and \(A_{n}^{4}=o_{n}(1)\) that \(A_{n}^{5}=o_{n}(1)\) if \(K\in L^{2}(\mathbb {R}^{3})\cup L^{\infty}(\mathbb {R}^{3})\) and finish the proof of (P6). Then the proof of Lemma 2.7 is completed. □