Ground state solutions of Nehari–Pohožaev type for a kind of nonlinear problem with general nonlinearity and nonlocal convolution term

Abstract

In this paper, we consider the following nonlinear problem with general nonlinearity and nonlocal convolution term:

$$ \textstyle\begin{cases} -\Delta u+V(x)u+(I_{\alpha }\ast \vert u \vert ^{q}) \vert u \vert ^{q-2}u=f(u), \quad x\in {\mathbb{R}}^{3}, \\ u\in H^{1}(\mathbb{R}^{3}), \quad \end{cases} $$

where $a\in (0,3)$, $q\in [1+\frac{\alpha }{3},3+\alpha )$, $I_{\alpha }:\mathbb{R}^{3}\rightarrow \mathbb{R}$ is the Riesz potential, $V\in \mathcal{C}(\mathbb{R}^{3},[0,\infty ))$, $f\in \mathcal{C}(\mathbb{R},\mathbb{R})$ and $F(t)=\int _{0}^{t}f(s)\,ds$ satisfies $\lim_{|t|\to \infty }F(t)/|t|^{\sigma }=\infty$ with $\sigma =\min \{2,\frac{2\beta +2}{\beta }\}$ where $\beta =\frac{ \alpha +2}{2(q-1)}$. By using new analytic techniques and new inequalities, we prove the above system admits a ground state solution under mild assumptions on V and f.

1 Introduction

In this paper we are concerned with the existence of ground state solutions for the nonlinear problem:

$$ \textstyle\begin{cases} -\Delta u+V(x)u+(I_{\alpha }\ast \vert u \vert ^{q}) \vert u \vert ^{q-2}u=f(u), & x\in {\mathbb{R} }^{3}, \\ u\in H^{1}(\mathbb{R}^{3}), \end{cases} $$

(1.1)

where $\alpha \in (0,3)$, $q\in [1+\frac{\alpha }{3},3+\alpha )$, $V\in \mathcal{C}(\mathbb{R}^{3},[0,\infty ))$, $f\in \mathcal{C}( \mathbb{R},\mathbb{R})$ and $I_{\alpha }:\mathbb{R}^{3}\rightarrow \mathbb{R}$ is the Riesz potential of order $\alpha \in (0,3)$, given by

$$ I_{\alpha }(x)=\frac{A_{\alpha }}{ \vert x \vert ^{N-\alpha }} \quad \mbox{with } A_{\alpha }= \frac{\varGamma (\frac{N-\alpha }{2})}{\varGamma (\frac{\alpha }{2})\pi ^{N/2}2^{\alpha }}, x\in \mathbb{R}^{3}\setminus \{0\}. $$

When $q=2$ and $\alpha =2$, the convolution term of (1.1) can be considered as analogous to the internal potential ϕ in Schrödinger–Poisson system like the following equation:

$$ \textstyle\begin{cases} -\Delta u +V(x)u +\phi u=f(u), & x\in \mathbb{R}^{3}, \\ -\Delta \phi =u^{2}, & x\in \mathbb{R}^{3}. \end{cases} $$

(1.2)

System (1.2) was first introduced in [1] as a model describing solitary waves for the nonlinear stationary Schrödinger equation interacting with the electrostatic field. We note that system (1.2) is also called a Schrödinger–Maxwell equation system, for more details on the physical aspects of this problem, we refer to [1] and the references therein.

In recent years, the existence, multiplicity and concentration of nontrivial solutions of (1.2) have been the subject of extensive mathematical studies, for example, [2,3,4,5,6,7,8]. Problem (1.2) with $V(x)\equiv 1$ or being radially symmetric, has been widely studied under various conditions on f; see for example [9,10,11,12,13,14,15,16,17]. When $V=1$ and $f(u)=|u|^{p-2}u$, by introducing a new manifold that is defined by a condition which is a combination of the Nehari equation and the Pohožaev equality, Ruiz [16] showed that (1.2) admits a positive radial solution if $3< p<6$, but does not have a nontrivial solution for $2< p\leq 3$. Under the same assumptions, based on Ruiz’s approach in [16], Azzollini and Pomponio [3] obtained the existence of ground state solutions for (1.2) by using a concentration-compactness argument. When $f(u)=|u|^{p-2}u$ and V satisfies the following assumptions:

(V1):

$V\in \mathcal{C}(\mathbb{R}^{3},[0,\infty ))$ and $V_{\infty }:=\lim_{|y|\rightarrow \infty }V(y)\geq V(x)$ for $x\in \mathbb{R}^{3}$;

(V2′):

$V(x)$ is weakly differentiable, and satisfies $\nabla V(x) \cdot x \in L^{\infty }(\mathbb{R}^{3})\cup L^{3/2}( \mathbb{R}^{3})$, and

$$ 2V(x)+\nabla V(x) \cdot x \geq 0 \quad \mbox{a.e. } x\in \mathbb{R}^{3}, $$

Zhao and Zhao [8] established the existence of ground state solutions for (1.2) by using the Jeanjean’s monotonicity trick [18]. In a recent paper [19], Tang and Chen introduced some new tricks to generalize and improve the results in [3, 8, 16] to the more general case where V satisfies (V1) and (V2′) and f satisfies the following assumptions:

• (F1) $f\in \mathcal{C}(\mathbb{R},\mathbb{R})$, and there exist constants $C>0$ and $q\in (2,6)$ such that

  $$ \bigl\vert f(t) \bigr\vert \leq C\bigl(1+ \vert t \vert ^{q-1} \bigr), \quad \forall t\in \mathbb{R}; $$

• (F2) $f(t)=o(t)$ as $t\rightarrow 0$;

• (F3) $\lim_{|t|\rightarrow \infty }\frac{F(t)}{|t|^{3}}=\infty$, where $F(t)=\int _{0}^{t}f(s)\,ds$;

• (F4) $[2f(t)t-3F(t)]/t^{3}$ is nondecreasing on $(-\infty ,0) \cup (0,+\infty )$.

There is also other work about ground state solutions for (1.2); we refer to [20, 21]. Motivated by the above work and [22,23,24,25,26,27,28,29,30], in the present paper, we shall extend the results concerning the existence of ground state solutions for (1.2) in [23] to (1.1). Compared with (1.2), it is more difficult to deal with (1.1) for the reason that $q \in [1+\frac{ \alpha }{3},3+\alpha )$. Because of the changing of q, the competing effect of $(I_{\alpha }\ast |u|^{q})|u|^{q-2}u$ and $f(u)$ is also changing.

For any $\varepsilon >0$, it follows from (F1) and (F2) that there exists $C_{\varepsilon } >0$ such that

$$ \bigl\vert f(t) \bigr\vert \leq \varepsilon \vert t \vert +C_{\varepsilon } \vert t \vert ^{p-1}, \quad \forall t \in \mathbb{R}. $$

(1.3)

Under assumptions (V1), (F1), (F2) and (1.3), the functional

$$ \varPhi (u)=\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl[ \vert \nabla u \vert ^{2}+V(x)u^{2} \bigr]\,dx+ \frac{1}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}\,dx- \int _{\mathbb{R}^{3}}F(u)\,dx $$

(1.4)

is well defined in $H^{1}(\mathbb{R}^{3})$ and $\varPhi \in \mathcal{C} ^{1}(H^{1}(\mathbb{R}^{3}),\mathbb{R})$, where $F(t)=\int _{0}^{t}f(s) \,ds$. Moreover, for any $u,\upsilon \in H^{1}(\mathbb{R}^{3})$,

$$\begin{aligned} \bigl\langle \varPhi ^{\prime }(u),v\bigr\rangle =& \int _{\mathbb{R}^{3}}\bigl[\nabla u\cdot \nabla v+V(x)uv\bigr]\,dx \\ &{}+ \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q-2}uv\,dx- \int _{\mathbb{R}^{3}}f(u)v\,dx. \end{aligned}$$

(1.5)

Hence, the solution of (1.1) are critical points of $\varPhi (u)$. A solution is called a ground state solution if its energy is minimal among all nontrivial solutions.

In this paper, let $\beta =\frac{\alpha +2}{2(q-1)}$, and in addition to (V1), (F1) and (F2), we also need to introduce the following assumptions:

(V2):

$V\in \mathcal{C}^{1}(\mathbb{R}^{3})$, the set $\{x\in \mathbb{R}^{3}:|\nabla V(x)\cdot x|\geq \epsilon \}$ has finite Lebesgue measure for every $\epsilon >0$, and the function $t\mapsto t^{2}[(2\beta -3)V(tx)-\nabla V(tx)\cdot (tx)]$ is increasing on $(0,+\infty )$ for every $x\in \mathbb{R}^{3}$;

(V3):

$V\in \mathcal{C}^{1}(\mathbb{R}^{3})$, $\nabla V(x) \cdot x \in L^{\infty }(\mathbb{R}^{3})$ and there exists $\varrho >0$ such that

$$ 2V(x)+\nabla V(x)\cdot x\geq \varrho , \quad \forall x\in \mathbb{R}^{3}; $$

(V3′):

$V\in \mathcal{C}^{1}(\mathbb{R}^{3})$, $\nabla V(x) \cdot x \in L^{\infty }(\mathbb{R}^{3})$, $2V(x)+\nabla V(x)\cdot x \geq \varrho$, $\forall \ x\in \mathbb{R}^{3}$ and there exists $\mu > \frac{2\beta +2}{\beta }$ such that

$$ f(t)t-\mu F(t)\geq 0,\quad \forall t\in \mathbb{R} ; $$

(F5):

$\lim_{|t|\rightarrow \infty } \frac{F(t)}{|t|^{\frac{2 \beta +2}{\beta }}}=\infty$;

(F6):

$[\beta f(t)t-3F(t)]/t|t|^{\frac{\beta +2}{\beta }}$ is nondecreasing on both $(-\infty ,0)$ and $(0,+\infty )$.

To state our results, we define the Nehari–Pohožaev manifold as follows:

$$ \mathcal{M}:=\bigl\{ {u\in H^{1}\bigl( \mathbb{R}^{3}\bigr)\setminus \{0\}}:J(u):= \beta \bigl\langle \varPhi '(u),u\bigr\rangle -\mathcal{P}(u)=0\bigr\} , $$

(1.6)

where

$$\begin{aligned} \mathcal{P}(u) = &\frac{1}{2} \Vert \nabla u \Vert _{2}^{2}+\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl[3V(x)+\nabla V(x)\cdot x\bigr]u^{2} \,dx +\frac{3+ \alpha }{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}\,dx \\ &{}-3 \int _{\mathbb{R}^{3}}F(u)\,dx \end{aligned}$$

(1.7)

which is associated with the Pohožaev identity $\mathcal{P}(u)=0$ of (1.1) that can be obtained by the same argument as in [13, 31]. And

$$\begin{aligned} J(u) = &\frac{2\beta -1}{2} \Vert \nabla u \Vert _{2}^{2}+\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl[(2\beta -3)V(x)-\nabla V(x)\cdot x \bigr]u^{2}\,dx \\ &{}+\frac{2\beta -1}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}\,dx - \int _{\mathbb{R}^{3}}\bigl[\beta f(u)u-3F(u)\bigr]\,dx. \end{aligned}$$

(1.8)

Throughout the paper we use the following notations:

• $H^{1}(\mathbb{R}^{3})$ denotes the usual Sobolev space equipped with the inner product and norm

  $$ (u,v)= \int _{\mathbb{R}^{3}}(\nabla u\cdot \nabla v+uv)\,dx, \quad \Vert u \Vert =(u,u)^{\frac{1}{2}},\quad \forall u,v\in H^{1}\bigl( \mathbb{R}^{3}\bigr). $$

• $L^{s}(\mathbb{R}^{3})(1\leq s <\infty )$ denotes the Lebesgue space with the norm $\|u\|_{s}=(\int _{\mathbb{R}^{3}}|u|^{s}\,dx)^{1/s}$;

• for any $u\in H^{1}(\mathbb{R}^{3}) \setminus \{0\}$, $u_{t}(x):=u(tx)$ for $t > 0$;

• for any $x\in \mathbb{R}^{3}$ and $r > 0$, $B_{r}(x):=\{y\in \mathbb{R}^{3}:|y-x| < r\}$;

• $C_{1},C_{2},\ldots$ denotes positive constants possibly different in different places.

Next,we state the main results of this paper.

Theorem 1.1

Assume that V and f satisfy (V1), (V2), (F1), (F2), (F5) and (F6) hold. Then problem (1.1) has a ground state solution $u_{0}\in H^{1}(\mathbb{R}^{3})$ such that $\varPhi (u_{0})= \inf_{\mathcal{M}}\varPhi =\inf_{u\in H^{1}(\mathbb{R}^{3})\setminus \{0\}}\max_{t > 0} \varPhi (t^{\beta }u_{t}) > 0$.

Theorem 1.2

Assume that V and f satisfy (V1), (V3), (F1), (F2), (F5) and (F6) hold. Then problem (1.1) has a positive ground state solution.

Theorem 1.3

Assume that V and f satisfy (V1), (V3′), (F1), (F2), (F5) and (F6) hold. Then problem (1.1) has a positive ground state solution.

Inspired by [19, 32], we shall prove Theorem 1.1 following this scheme:

• Step (i) we verify $\mathcal{M}\neq \emptyset$ and establish the minimax characterization of $m=\inf_{\mathcal{M}} \varPhi >0$;

• Step (ii) we prove that m can be obtained;

• Step (iii) we show that the minimizer of Φ on $\mathcal{M}$ is a critical point.

Although we mainly follow the procedure of [19, 32], we have to face many new difficulties due to the mutual competing effect between $f(u)$ and $(I_{\alpha }\ast |u|^{q})|u|^{q-2}u$. More precisely, in Step (i), we first establish a key inequality in Lemma 2.2 by using some properties about nonlinearity term, and also we use Nehari–Pohožaev manifold to remove the influence of the term $(I_{\alpha }\ast |u|^{q})|u|^{q-2}u$ which needs a kind of computational technique; it is worth mentioning that [23, Proposition 2.7] gives us an excellent numerical property as regards $V(x)$ which is convenient for us to eliminate the potential term $V(x)$ in some inequalities. Then we construct a saddle point structure with respect to the fibre $\{t^{\beta }u_{t}:t>0\} \subset H^{1}(\mathbb{R}^{3})$ for $u\in H^{1}(\mathbb{R}^{3}) \setminus \{0\}$; see Lemma 2.6, finally based on these constructions we obtain the minimax characterization of m; see Lemma 2.8. In Step (ii), we first choose a minimizing sequence $\{u_{n}\}$ of Φ on $\mathcal{M}$, and show that $\{u_{n}\}$ is bounded in $H^{1}(\mathbb{R}^{3})$, then with the help of the key inequality established in Lemma 2.2 and a concentration-compactness argument, we prove that there exist $\hat{u} \in H^{1}(\mathbb{R}^{3})$ and $\hat{t} >0$ such that $u_{n}\rightharpoonup \hat{u}$ in $H^{1}(\mathbb{R}^{3})$ up to translations and extraction of a subsequence, and $\hat{t}^{\beta }\hat{u}_{\hat{t}} \in \mathcal{M}$ is a minimizer of $\inf_{\mathcal{M}} \varPhi$; see Lemma 2.13 and Lemma 2.14. Step (iii) is similar to [19, Lemma 2.10].

Motivated by [8, 19], we use the Jeanjean’s monotonicity trick [18] to prove Theorems 1.2 and 1.3, which can helps us to construct a bounded (PS) sequence. The difficulty in the proof is to overcome the lack of compactness, and a more careful analysis is needed to consider the relationship between the mountain pass level for Φ and the least energy of the functional associated “limit problem” of (1.1) which is used to recover the compactness; see Lemma 3.4. By using Theorem 1.1 and applying the global compactness lemma and (V1) and (V3) (or (V3′)), we can prove Theorems 1.2 and 1.3; see Lemma 3.5 and Lemma 3.6.

2 Proof of Theorem 1.1

First, by a simple calculation, we establish some key inequalities.

Lemma 2.1

Assume that (F1) and (F6) hold, Then

$$\begin{aligned} \frac{1}{t^{3}}F\bigl(t^{\beta }\tau \bigr)-F(\tau )+ \frac{1-t^{2\beta -1}}{2 \beta -1}\bigl[\beta f(\tau )\tau -3F(\tau )\bigr] \geq 0, \quad \forall t\geq 0, \tau \in \mathbb{R}. \end{aligned}$$

(2.1)

Proof

It is evident that (2.1) holds for $\tau =0$. For $\tau \neq 0$, let

$$ g(t)=\frac{1}{t^{3}}F\bigl(t^{\beta }\tau \bigr)-F(\tau )+\frac{1-t^{2\beta -1}}{2 \beta -1}\bigl[\beta f(\tau )\tau -3F(\tau )\bigr] \geq 0. $$

(2.2)

Then from (F4), one has

$$\begin{aligned} {g}'(t) =&-\frac{3}{t^{4}}F \bigl(t^{\beta }\tau \bigr)+\frac{\beta }{t^{4}} {f\bigl(t^{\beta }\tau \bigr)}t^{\beta }\tau -t^{2\beta -2}\bigl[\beta f(\tau )\tau -3F(\tau ) \bigr] \\ =&t^{2\beta -2} \biggl[\frac{\beta f(t^{\beta }\tau )t^{\beta }\tau -3F(t ^{\beta }\tau )}{t^{2\beta +2}}-\beta f(\tau )\tau +3F(\tau ) \biggr] \\ & \textstyle\begin{cases} \geq 0, & t\geq 1, \\ \leq 0, & 0< t < 1. \end{cases}\displaystyle \end{aligned}$$

(2.3)

It follows that $g(t)\geq g(1)=0$ for $t \geq 0$. This, together with (2.2) implies (2.1) holds. □

Define

$$\begin{aligned} \kappa (x,t) = &2V(x)-(2\beta -1)t^{2\beta -3}V \bigl(t^{-1}x\bigr)+(2\beta -3)t ^{2\beta -1}V(x) \\ &{}+\bigl[1-t^{2\beta -1}\nabla V(x)\cdot x\bigr], \quad \forall x\in \mathbb{R}^{3}, \forall t>0. \end{aligned}$$

(2.4)

It is easy to check that (V2) implies

$$\begin{aligned} \kappa (x,t)>0, \quad \forall x\in \mathbb{R}^{3}, \forall t\in (0,1)\cup (1,\infty ). \end{aligned}$$

(2.5)

Lemma 2.2

Assume that (V1), (V4), (F1), (F2) and (F6) hold, Then

$$\begin{aligned} \begin{aligned}[b] &\varPhi (u) \geq \varPhi \bigl(t^{\beta }u_{t}\bigr)+ \frac{1-t^{2\beta -1}}{2\beta -1}J(u)+ \frac{1}{2(2\beta -1)} \int _{\mathbb{R}^{3}}\kappa (x,t)u^{2}, \\ &\quad \forall u\in H^{1}\bigl(\mathbb{R}^{3}\bigr),\forall t>0. \end{aligned} \end{aligned}$$

(2.6)

Proof

Note that

$$ \begin{aligned}[b] \varPhi \bigl(t^{\beta }u_{t} \bigr)= {}&\frac{t^{2\beta -1}}{2} \Vert \nabla u \Vert _{2}^{2}+ \frac{t ^{2\beta -3}}{2} \int _{\mathbb{R}^{3}}V\bigl(t^{-1}x\bigr)u^{2}\,dx\\ &{}+ \frac{t^{2 \beta -1}}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}\,dx -\frac{1}{t^{3}} \int _{\mathbb{R}^{3}}F\bigl(t^{\beta }u\bigr)\,dx. \end{aligned} $$

(2.7)

Thus, by (1.4), (1.8), (2.1), (2.5) and (2.7), one has

$$\begin{aligned} &\varPhi (u)-\varPhi \bigl(t^{\beta }u_{t}\bigr) \\ &\quad =\frac{1-t^{2\beta -1}}{2} \Vert \nabla u \Vert _{2}^{2}+ \frac{1}{2} \int _{\mathbb{R}^{3}}\bigl[V(x)-t^{2\beta -3}V\bigl(t^{-1}x \bigr)\bigr]u^{2}(x)\,dx \\ &\qquad {}+\frac{1-t^{2\beta -1}}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}\,dx + \int _{\mathbb{R}^{3}}\bigl[t^{-3}F\bigl(t^{\beta }u \bigr)-F(u)\bigr]\,dx \\ &\quad =\frac{1-t^{2\beta -1}}{2\beta -1} \biggl\{ \frac{2\beta -1}{2} \Vert \nabla u \Vert _{2}^{2}+\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl[(2\beta -3)V(x)- \nabla V(x)\cdot x \bigr]u^{2}\,dx \biggr\} \\ &\qquad {}+\frac{1-t^{2\beta -1}}{2\beta -1} \biggl\{ \frac{2\beta -1}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}- \int _{\mathbb{R}^{3}}\bigl[\beta f(u)u-3F (u)\bigr] \biggr\} \\ &\qquad {}+ \int _{\mathbb{R}^{3}} \biggl\{ \frac{1}{t^{3}}F\bigl(t^{\beta }u \bigr)-F(u)+\frac{1-t ^{2\beta -1}}{2\beta -1}\bigl[\beta f(u)u-3F(u)\bigr] \biggr\} \,dx \\ &\qquad {}+\frac{1}{2(2\beta -1)} \int _{\mathbb{R}^{3}}\kappa (x,t)u^{2}\,dx \\ &\quad \geq \frac{1-t^{2\beta -1}}{2\beta -1}J(u)+\frac{1}{2(2\beta -1)} \int _{\mathbb{R}^{3}}\kappa (x,t)u^{2}\,dx. \end{aligned}$$

(2.8)

This shows that (2.6) holds. □

Remark that (2.6) with $t\rightarrow 0$ implies

$$\begin{aligned} \varPhi (u)\geq \frac{1}{2\beta -1}J(u)+\frac{1}{2(2\beta -1)} \int _{\mathbb{R}^{3}}\bigl[2V(x)+\nabla V(x)\cdot x\bigr]u^{2} \,dx,\quad \forall u \in H^{1}\bigl(\mathbb{R}^{3}\bigr). \end{aligned}$$

(2.9)

To overcome the lack of compactness of Sobolev space embedding in $\mathbb{R}^{3}$, we define the following energy functional:

$$\begin{aligned} \begin{aligned}[b] \varPhi ^{\infty }(u)={} & \frac{1}{2} \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u \vert ^{2}+V _{\infty }u^{2}\bigr)\,dx+\frac{1}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha } \ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}\,dx \\ &{}- \int _{\mathbb{R}^{3}}F(u)\,dx. \end{aligned} \end{aligned}$$

(2.10)

Corresponding to (1.6) and (1.8), we define

$$\begin{aligned} \mathcal{M}^{\infty }:=\bigl\{ u\in H^{1}\bigl( \mathbb{R}^{3}\bigr)\setminus \{0\}:J ^{\infty }(u)=0\bigr\} \end{aligned}$$

(2.11)

and

$$\begin{aligned} \begin{aligned}[b] J^{\infty }(u):= {}& \frac{2\beta -1}{2} \Vert \nabla u \Vert _{2}^{2}+ \frac{2 \beta -3}{2}V_{\infty } \Vert u \Vert _{2}^{2}+ \frac{2\beta -1}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}\,dx \\ &{}- \int _{\mathbb{R}^{3}}\bigl[\beta f(u)u-3F(u)\bigr]\,dx. \end{aligned} \end{aligned}$$

(2.12)

From Lemma 2.2, we have the following two corollaries.

Corollary 2.3

Assume that (F1), (F2) and (F6) hold. Then

$$ \begin{aligned}[b] &\varPhi ^{\infty }(u) \geq \varPhi ^{\infty }\bigl(t^{\beta }u_{t}\bigr) + \frac{1-t ^{2\beta -1}}{2\beta -1}J^{\infty }(u) \\ &\hphantom{\varPhi ^{\infty }(u) \geq}{}+\frac{(2\beta -1)(1-t^{2\beta -3})-(2\beta -3)(1-t^{2\beta -1})}{2(2 \beta -1)}V_{\infty } \Vert u \Vert _{2}^{2}, \\ &\quad \forall u\in H^{1}\bigl(\mathbb{R}^{3}\bigr),\forall t\geq 0. \end{aligned} $$

(2.13)

Corollary 2.4

Assume that (V1), (V2), (F1) and (F6) hold. Then for $u\in \mathcal{M}$

$$ \varPhi (u)=\max_{t>0} \varPhi \bigl(t^{\beta }u_{t}\bigr) . $$

(2.14)

From [23, Proposition 2.7], we can obtain the following lemma.

Lemma 2.5

Assume that (V1) and (V2) hold. Then there exist two constants $\rho _{1}$, $\rho _{2} > 0$ such that

$$ 2V(x)+\nabla V(x)\cdot x\geq \rho _{1} $$

(2.15)

and

$$ (2\beta -3)V(x)-\nabla V(x)\cdot x\geq \rho _{2}. $$

(2.16)

Lemma 2.6

Assume that (V1), (V2), (F1), (F2) and (F6) hold. Then, for any $u\in H^{1}(\mathbb{R}^{3})\setminus \{0\}$, there exists a unique $t_{u}>0$ such that $t_{u}^{\beta }u_{t_{u}}\in \mathcal{M}$.

Proof

Let $u\in H^{1}(\mathbb{R}^{3})\setminus \{0\}$ be fixed and define a function $\zeta (t):=\varPhi (t^{\beta }u_{t})$ on $(0,\infty )$. Clearly, by (1.8) and (2.5), we have

$$ \begin{aligned}[b] \zeta '(t)=0 \quad \Leftrightarrow \quad &\frac{2\beta -1}{2}t^{2\beta -2} \Vert \nabla u \Vert _{2}^{2} \\ &\qquad {}+\frac{t^{2\beta -4}}{2} \int _{\mathbb{R}^{3}}\bigl[(2\beta -3)V\bigl(t^{-1}x\bigr)- \nabla V\bigl(t^{-1}x\bigr)\cdot \bigl(t^{-1}x\bigr) \bigr]u^{2}\,dx \\ &\qquad {} +\frac{2\beta -1}{2q}t^{2\beta -2} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}\,dx \\ &\qquad {}- \int _{\mathbb{R}^{3}}\bigl[\beta f\bigl(t^{\beta }u\bigr)ut ^{\beta -4}-3F\bigl(t^{\beta }u\bigr)t^{-4}\bigr]\,dx \\ &\quad =0 \quad \Leftrightarrow \quad J\bigl(t^{\beta }u_{t} \bigr)=0\quad \Leftrightarrow\quad t^{\beta }u _{t} \in \mathcal{M}. \end{aligned} $$

(2.17)

By (V1), (F1) and (F5), we have $\lim_{t\rightarrow 0^{+}}\zeta '(t)=0$, $\zeta '(t)>0$ for $t>0$ small and $\zeta '(t)<0$ for t large. Therefore $\max_{t\in [0,+\infty )}\zeta (t)$ is obtained at $t_{u}>0$ so that $\zeta '(t_{u})=0$ and $t_{u}^{\beta }u_{t_{u}}\in \mathcal{M}$.

Next we claim that $t_{u}$ is unique for any $u\in H^{1}(\mathbb{R} ^{3})\setminus \{0\}$. In fact, for any given $u\in H^{1}(\mathbb{R} ^{3})\setminus \{0\}$, let $t_{1},t_{2}>0$ such that $t_{1}^{\beta }u _{t_{1}},t_{2}^{\beta }u_{t_{2}}\in \mathcal{M}$. Then $J(t_{1}^{ \beta }u_{t_{1}})=J(t_{2}^{\beta }u_{t_{2}})=0$. Jointly with (2.9), we have

$$\begin{aligned}& \begin{aligned}[b] \varPhi \bigl(t_{1}^{\beta }u_{t_{1}} \bigr)\geq{}& \varPhi \bigl(t_{2}^{\beta }u_{t_{2}}\bigr)\\ &{}+ \frac{1}{2 \beta -1} \frac{t_{2}^{2\beta -1}-t_{1}^{2\beta -1}}{t_{2}^{2\beta -1}}J\bigl(t_{1} ^{\beta }u_{t_{1}} \bigr)+\frac{1}{2(2\beta -1)} \int _{\mathbb{R}^{3}}\kappa \biggl(x,\frac{t_{2}}{t_{1}} \biggr)u^{2}\,dx \\ \geq{}& \varPhi \bigl(t_{2}^{\beta }u_{t_{2}}\bigr)+ \frac{1}{2(2\beta -1)} \int _{\mathbb{R}^{3}}\kappa \biggl(x,\frac{t_{2}}{t_{1}} \biggr)u^{2}\,dx, \end{aligned} \end{aligned}$$

(2.18)

$$\begin{aligned}& \begin{aligned}[b] \varPhi \bigl(t_{2}^{\beta }u_{t_{2}} \bigr) &\geq \varPhi \bigl(t_{1}^{\beta }u_{t_{1}}\bigr)+ \frac{1}{2 \beta -1} \frac{t_{1}^{2\beta -1}-t_{2}^{2\beta -1}}{t_{1}^{2\beta -1}}J\bigl(t_{2} ^{\beta }u_{t_{2}} \bigr)\\ &{}+\frac{1}{2(2\beta -1)} \int _{\mathbb{R}^{3}}\kappa \biggl(x,\frac{t_{1}}{t_{2}} \biggr)u^{2}\,dx \\ &\geq \varPhi \bigl(t_{1}^{\beta }u_{t_{1}}\bigr)+ \frac{1}{2(2\beta -1)} \int _{\mathbb{R}^{3}}\kappa \biggl(x,\frac{t_{1}}{t_{2}} \biggr)u^{2}\,dx. \end{aligned} \end{aligned}$$

(2.19)

Combining (2.18) and (2.19), we have $t_{1}=t_{2}$. Therefore, $t_{u}>0$ is unique for any $u\in H^{1}(\mathbb{R}^{3})\setminus \{0 \}$. □

Corollary 2.7

Assume that (F1), (F5) and (F6) hold. Then, for any $u\in H^{1}(\mathbb{R}^{3})\setminus \{0\}$, there exists a unique $t_{u}>0$ such that $t_{u}^{\beta }u_{t_{u}}\in \mathcal{M}^{\infty }$.

Combining Corollary 2.4 with Lemma 2.6, we have the following lemma.

Lemma 2.8

Assume that (V1), (V2), (F1), (F2), (F5) and (F6) hold. Then

$$ \inf_{u\in \mathcal{M}}\varPhi (u) =m= \inf_{u\in H^{1}(\mathbb{R}^{3})\setminus \{0\}}\max _{t> 0}\varPhi \bigl(t ^{\beta }u_{t}\bigr). $$

Lemma 2.9

Assume that (F1) and (F2) hold. If $u_{n}\rightharpoonup \bar{u}$ in $H^{1}(\mathbb{R}^{3})$, then along a subsequence of $\{u_{n}\}$

$$ \lim_{n\rightarrow \infty } \sup_{\varphi \in H^{1}(\mathbb{R}^{3}), \Vert \varphi \Vert \leq 1} \biggl\vert \int _{\mathbb{R}^{3}}\bigl[f(u_{n})-f(u_{n}- \bar{u})-f(\bar{u})\bigr]\varphi \,dx \biggr\vert =0. $$

(2.20)

From [33, Proposition 4.3], we can obtain the following Lemma.

Lemma 2.10

Let $\alpha \in (0,3)$ and $q \in [1,3+\alpha )$ hold. If $u_{n} \rightharpoonup \bar{u}$ in $H^{1}(\mathbb{R}^{3})$, then along a subsequence of $\{u_{n}\}$

$$ \lim_{n\rightarrow \infty } \int _{\mathbb{R}^{3}} \bigl\vert \bigl(I_{\alpha } \ast \vert u_{n} \vert ^{q}\bigr) \vert u_{n} \vert ^{q}-\bigl(I_{\alpha }\ast \vert u_{n}-u \vert ^{q}\bigr) \vert u_{n}-u \vert ^{q}-\bigl(I _{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q} \bigr\vert =0. $$

(2.21)

Lemma 2.11

Assume that (V1), (V2), (F1), (F5) and (F6) hold. Then

• (i) there exists $\rho > 0$ such that $\|u\| \geq \rho$, $\forall u\in \mathcal{M}$;

• (ii) $m=\inf_{\mathcal{M}} \varPhi >0$.

Proof

(i) Since $J(u)=0$, $\forall u\in \mathcal{M}$, by (F1), (F2), (1.8), (2.16) and due to the Sobolev embedding theorem, one has

$$ \begin{aligned}[b] \frac{\min \{2\beta -1,\rho _{2}\}}{2} \Vert u \Vert ^{2} \leq {}& \frac{2\beta -1}{2} \Vert \nabla u \Vert _{2}^{2}+\frac{1}{2} \int _{\mathbb{R} ^{3}}\bigl[(2\beta -3)V(x)-\nabla V(x) \cdot x \bigr]u^{2}\,dx \\ &{}+\frac{2\beta -1}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}\,dx \\ = {}& \int _{\mathbb{R}^{3}}\bigl[\beta f(u)u-3F(u)\bigr]\,dx \\ \leq {}& \frac{\min \{2\beta -1,\rho _{2}\}}{4} \Vert u \Vert ^{2}+ {C_{1}} \Vert u \Vert ^{p}, \end{aligned} $$

(2.22)

which implies

$$ \Vert u \Vert \geq \rho :=\biggl(\frac{\min \{2\beta -1,\rho _{2}\}}{4 {C_{1}}} \biggr)^{\frac{1}{p-2}},\quad \forall u\in \mathcal{M}. $$

(2.23)

(ii) Let $\{u_{n}\} \subset \mathcal{M}$ be such that $\varPhi (u_{n}) \rightarrow m$. There are two possible cases:

• (1) $\inf_{n\in \mathbb{N}}\|u_{n}\|_{2}>0$ and

• (2) $\inf_{n\in \mathbb{N}}\|u_{n}\| _{2}=0$.

Case (1) $\inf_{n\in \mathbb{N}}\|u_{n}\|_{2}:=\varrho _{1}>0$. In this case, by (2.9) and (2.15), one has

$$ m+o(1)=\varPhi (u_{n})=\varPhi (u_{n})- \frac{1}{2\beta -1}J(u_{n})\geq \frac{ {\rho _{1}}}{2(2\beta -1)}\varrho _{1}^{2}. $$

(2.24)

Case (2) $\inf_{n\in \mathbb{N}}\|u_{n}\|_{2}:=0$, by (2.23), passing to a subsequence, we have

$$ \Vert u_{n} \Vert _{2}\rightarrow 0, \qquad \Vert \nabla u_{n} \Vert _{2}\geq \frac{1}{2}\rho . $$

(2.25)

Note that (F1) implies that, for any $\varepsilon >0$, there exists $C_{\varepsilon }>0$ such that

$$ \bigl\vert F(t) \bigr\vert \leq C_{\varepsilon } \vert t \vert ^{2}+\varepsilon \vert t \vert ^{6}, \quad \forall t\in \mathbb{R}. $$

(2.26)

By (2.26) and the Sobolev embedding inequality, we have

$$ \int _{\mathbb{R}^{3}}F(u)\,dx\leq C_{2} \Vert u \Vert _{2}^{2}+\frac{1}{2}S^{3} \Vert u \Vert _{6}^{6}\leq C_{2} \Vert u \Vert _{2}^{2}+\frac{1}{4} \Vert \nabla u \Vert _{2}^{6}. $$

(2.27)

Let $t_{n}=\|\nabla u_{n}\|_{2}^{-\frac{2}{2\beta -1}}$, then (2.25) implies that $\{t_{n}\}$ is bounded. Since $J(u_{n})=0$, it follows from (2.6), (2.7), (2.25) and (2.27) that

$$\begin{aligned} m+o(1) =&\varPhi (u_{n})\geq \varPhi \bigl(t_{n}^{\beta }(u_{n})_{t_{n}}\bigr) \\ =&\frac{t_{n}^{2\beta -1}}{2} \Vert \nabla u_{n} \Vert _{2}^{2}+\frac{t_{n} ^{2\beta -3}}{2} \int _{\mathbb{R}^{3}}V\bigl(t^{-1}x\bigr)u_{n}^{2} \,dx \\ &{}+\frac{t _{n}^{2\beta -1}}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u_{n} \vert ^{q}\bigr) \vert u _{n} \vert ^{q}\,dx \\ &{}-t_{n}^{3} \int _{\mathbb{R}^{3}}F\bigl(t_{n}^{\beta }u_{n} \bigr)\,dx \\ \geq &\frac{t_{n}^{2\beta -1}}{2} \Vert \nabla u \Vert _{2}^{2}- \frac{t_{n} ^{6\beta -3}}{4}C_{2} \Vert \nabla u \Vert _{2}^{6} \\ =&\frac{t_{n}^{2\beta -1}}{4} \Vert \nabla u \Vert _{2}^{2} \bigl[2-\bigl(t_{n}^{2 \beta -1} \Vert \nabla u \Vert _{2}^{2}\bigr)^{2} \bigr]=\frac{1}{4}+o(1). \end{aligned}$$

(2.28)

Case (1) and Case (2) show that $m=\inf_{\mathcal{M}}\varPhi >0$. □

Lemma 2.12

Assume that (V1), (F1) and (F2) hold. If $u_{n} \rightharpoonup \bar{u}$ in $H^{1}(\mathbb{R}^{3})$, then along a subsequence

$$ \begin{gathered} \varPhi (u_{n}) =\varPhi ( \bar{u})+\varPhi (u_{n}-\bar{u})+o(1), \quad J(u_{n}) =J( \bar{u})+J(u_{n}-\bar{u})+o(1), \quad \\ \varPhi '(u_{n}) =\varPhi '(\bar{u})+ \varPhi '(u_{n}-\bar{u})+o(1), \\ \bigl\langle \varPhi '(u_{n}),u_{n}\bigr\rangle =\bigl\langle \varPhi '(\bar{u}),\bar{u} \bigr\rangle +\bigl\langle \varPhi '(u_{n}-\bar{u}),(u_{n}- \bar{u})\bigr\rangle +o(1). \end{gathered} $$

(2.29)

Lemma 2.13

Assume that (V1), (V2), (F1), (F2), (F5) and (F6) hold. Then $m^{\infty }:=\inf_{\mathcal{M}^{\infty }}\varPhi ^{\infty } \geq m$.

Proof

In view of Lemma 2.2 and Corollary 2.4, we have $\mathcal{M}^{\infty }\neq \emptyset$. Arguing indirectly, we assume that $m>m^{\infty }$. Let $\varepsilon :=m-m^{\infty }$, then there exists $u_{\varepsilon }^{\infty }$ such that

$$ \begin{aligned} u_{\varepsilon }^{\infty }\in \mathcal{M}^{\infty } \quad \mbox{and} \quad m^{\infty }+ \frac{\varepsilon }{2}>\varPhi ^{\infty }\bigl(u_{\varepsilon } ^{\infty } \bigr) \end{aligned} . $$

(2.30)

In view of Corollary 2.7, there exists $t_{\varepsilon }>0$ such that $t_{\varepsilon }^{\beta }(u_{\varepsilon }^{\infty })_{t_{\varepsilon }}\in \mathcal{M}$. Thus, it follows from (V1), (1.4), (2.10), (2.13) and (2.30) that

$$ \begin{aligned} m^{\infty }+\frac{\varepsilon }{2}> \varPhi ^{\infty }\bigl(u_{\varepsilon } ^{\infty }\bigr)\geq \varPhi ^{\infty }\bigl(t_{\varepsilon }^{\beta }\bigl(u_{\varepsilon }^{\infty } \bigr)_{t_{\varepsilon }}\bigr)\geq \varPhi \bigl(t_{\varepsilon }^{\beta } \bigl(u _{\varepsilon }^{\infty }\bigr)_{t_{\varepsilon }}\bigr)\geq m. \end{aligned} $$

(2.31)

This contradiction shows that $m^{\infty }>m$. □

Lemma 2.14

Assume that (V1), (V2), (F1), (F2), (F5) and (F6) hold. Then m is obtained.

Proof

In view of Lemma 2.6 and Lemma 2.11, we have $\mathcal{M}\neq \emptyset$ and $m>0$. Let $\{u_{n}\}\subset \mathcal{M}$ be such that $\varPhi (u_{n})\rightarrow m$. Since $J(u_{n})=0$, if follows from (2.9) and (2.15) that

$$ \begin{aligned}[b] m+o(1) &=\varPhi (u_{n})=\varPhi (u_{n})-\frac{1}{2(2\beta -1)}J(u_{n}) \\ & \geq \frac{1}{2(2\beta -1)} \int _{\mathbb{R}^{3}}\bigl[2V(x)+\nabla V(x) \cdot x\bigr]u_{n}^{2} \,dx \\ &\geq \frac{\rho _{1}}{2(2\beta -1)} \Vert u_{n} \Vert _{2} ^{2}. \end{aligned} $$

(2.32)

This shows that ${\|u_{n}\|_{2}}$ is bounded. Next, we prove that ${\|\nabla u_{n}\|_{2}}$ is also bounded. Arguing by contradiction, suppose that $\|\nabla u_{n}\|_{2}\rightarrow \infty$. By (1.3) and the Sobolev embedding inequality, for $u\in H^{1}(\mathbb{R}^{3})$ one has

$$ \begin{aligned} \int _{\mathbb{R}^{3}}F(u)\,dx\leq C_{3} \Vert u \Vert _{2}^{2}+ \frac{1}{2(8m)^{2}}S^{3} \Vert u \Vert _{6}^{6}\leq C_{3} \Vert u \Vert _{2}^{2}+ \frac{1}{4(8m)^{2}} \Vert \nabla u \Vert _{2}^{6}. \end{aligned} $$

(2.33)

Let $t_{n}=(8m/\|\nabla u_{n}\|_{2}^{2})^{\frac{1}{2\beta -1}}$. Since $J(u_{n})=0$, it follows from (1.4), (2.6) and (2.32) that

$$\begin{aligned} m+o(1) =&\varPhi (u_{n})\geq \varPhi \bigl(t_{n}^{\beta }(u_{n})_{t_{n}}\bigr) \\ =&\frac{t_{n}^{2\beta -1}}{2} \Vert \nabla u_{n} \Vert _{2}^{2}+\frac{t_{n} ^{2\beta -3}}{2} \int _{\mathbb{R}^{3}}V\bigl(t_{n}^{-1}x \bigr)u_{n}^{2}\,dx \\ &{}+\frac{t_{n}^{2\beta -1}}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha } \ast \vert u_{n} \vert ^{q}\bigr) \vert u_{n} \vert ^{q} \,dx-t_{n}^{-3} \int _{\mathbb{R}^{3}}F\bigl(t_{n} ^{\beta }u_{n} \bigr) \\ \geq& \frac{t_{n}^{2\beta -1}}{2} \Vert \nabla u_{n} \Vert _{2}^{2}-\frac{C _{3} t_{n}^{2\beta }}{t_{n}^{3}} \Vert u_{n} \Vert _{2}^{2}-\frac{1}{4(8m)^{2}}t _{n}^{6\beta -3} \Vert \nabla u_{n} \Vert _{2}^{6} \\ =&\frac{t_{n}^{2\beta -1}}{2} \Vert \nabla u_{n} \Vert _{2}^{2} \biggl[1- \frac{1}{2}\biggl( \frac{t_{n}^{2\beta -1} \Vert \nabla u_{n} \Vert _{2}^{2}}{8m}\biggr)^{2} \biggr]+o(1) \\ =&2m+o(1). \end{aligned}$$

(2.34)

This contradiction shows that $\{\|\nabla u_{n}\|_{2}^{2}\}$ is also bounded, and so $\{u_{n}\}$ is bounded in $H^{1}(\mathbb{R}^{3})$. Passing to a subsequence, we have $u_{n}\rightharpoonup \bar{u}$ in $H^{1}(\mathbb{R}^{3})$. Then $u_{n}\rightarrow \bar{u}$ in $L_{\mathrm{loc}}^{s}(\mathbb{R}^{3})$ for $2 \leq s <6$ and $u_{n}\rightarrow \bar{u}$ a.e. in $\mathbb{R}^{3}$. There are two possible cases: (i) $\bar{u}=0$ and (ii) $\bar{u}\neq 0$.

Case (i) $\bar{u}=0$. i.e. $u_{n}\rightharpoonup 0$ in $H^{1}( \mathbb{R}^{3})$. Then $u_{n}\rightarrow 0$ in $L_{\mathrm{loc}}^{s}( \mathbb{R}^{3})$ for $2 \leq s <2^{*}$ and $u_{n}\rightarrow 0$ a.e. in $\mathbb{R}^{3}$. Using (V1) and (V2), it is easy to show that

$$ \begin{aligned} \lim_{n\rightarrow \infty } \int _{\mathbb{R}^{3}}\bigl[V_{\infty }-V(x)\bigr]u _{n}^{2}\,dx = \lim_{n\rightarrow \infty } \int _{\mathbb{R}^{3}} \nabla V(x) \cdot x u_{n}^{2} \,dx =0 \end{aligned} $$

(2.35)

From (1.4), (1.8), (2.10), (2.12) and (2.35), one can get

$$ \begin{aligned} \varPhi ^{\infty }(u_{n}) \rightarrow m , \qquad J^{\infty }(u_{n})\rightarrow 0. \end{aligned} $$

(2.36)

Note that (F1) and (F2) imply that, for any $\varepsilon >0$, there exists $C_{\varepsilon }>0$ such that

$$ \begin{aligned} \bigl\vert F(t) \bigr\vert \leq \varepsilon \vert t \vert ^{2}+C_{\varepsilon } \vert t \vert ^{p}, \quad \forall t\in \mathbb{R}. \end{aligned} $$

(2.37)

By (1.3), (1.8), (2.10), (2.37) and Lemma 2.11(i), one has

$$ \begin{aligned}[b] \frac{\min \{\rho _{2},3\}}{2}\rho ^{2} \leq {}&\frac{2\beta -1}{2} \Vert \nabla u_{n} \Vert _{2}^{2}+ \int _{\mathbb{R}^{3}} \biggl[\frac{2\beta -3}{2}V(x)- \frac{1}{2} \nabla V(x) \cdot x \biggr]u_{n}^{2}\,dx \\ &{}+\frac{2\beta -1}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u_{n} \vert ^{q}\bigr) \vert u _{n} \vert ^{q}\,dx \\ ={}& \int _{\mathbb{R}^{3}}\bigl[\beta f(u_{n})u_{n}-3F(u) \bigr]\,dx \\ \leq {}&C_{4}\bigl(\varepsilon \vert u_{n} \vert _{2}^{2}+C_{\varepsilon } \vert u_{n} \vert _{p} ^{p}\bigr). \end{aligned} $$

(2.38)

Using (2.38) and Lion’s concentration compactness principle [34, Lemma 1.21], we can prove that there exists $\delta >0$ and $y_{n}\in \mathbb{R}^{3}$, such that $\int _{B_{1}(y_{n})}|u_{n}|^{2}\,dx> \delta$. Let $\hat{u}_{n}(x)=u_{n}(x+y_{n})$. Then we have $\|\hat{u}_{n}\|=\|u_{n}\|$ and

$$ \begin{aligned} J^{\infty }(\hat{u}_{n})=o(1), \qquad \varPhi ^{\infty }(\hat{u}_{n})\rightarrow m, \qquad \int _{B_{1}(0)} \vert \hat{u}_{n} \vert ^{2}\,dx >\delta . \end{aligned} $$

(2.39)

Therefore, there exists $\hat{u} \in H^{1}(\mathbb{R}^{3})\setminus \{0\}$ such that, passing to a subsequence,

$$ \textstyle\begin{cases} \hat{u}_{n} \rightharpoonup \hat{u}, & \mbox{in } H^{1}(\mathbb{R}^{3}); \\ \hat{u}_{n} \rightarrow \hat{u}, & \mbox{in } L_{loc}^{s}(\mathbb{R}^{3}), \forall \ s\in [1,6); \\ \hat{u}_{n} \rightarrow \hat{u}, &\mbox{a.e. on } \mathbb{R}^{3}. \end{cases} $$

(2.40)

Let $w_{n}=\hat{u}_{n}-\hat{u}$. then (2.40) and Lemma 2.12 yield

$$ \varPhi ^{\infty }(\hat{u}_{n})=\varPhi ^{\infty }(\hat{u})+\varPhi ^{\infty }(w_{n})+o(1), \qquad J^{\infty }(\hat{u}_{n})=J^{\infty }(\hat{u})+J^{\infty }(w_{n})+o(1). $$

(2.41)

We define the functional $\varPsi _{\infty }:H^{1}(\mathbb{R}^{3})\rightarrow \mathbb{R}$ for $u\in H^{1}(\mathbb{R}^{3})$ by

$$ \begin{aligned}[b] \varPsi ^{\infty }(u)&= \varPhi ^{\infty }(u)-\frac{1}{2\beta -1}J^{\infty }(u)\\ &=\frac{V _{\infty }}{2\beta -1} \Vert u \Vert _{2}^{2} +\frac{1}{2(2\beta -1)} \int _{\mathbb{R}^{3}}\bigl[2\beta f(u)u-(4\beta +4)F(u)\bigr]\,dx. \end{aligned} $$

(2.42)

From (2.5), (2.7), (2.39), (2.41) and (2.42), one has

$$ \varPsi ^{\infty }(w_{n})=m-\varPsi ^{\infty }(\hat{u})+o(1), \qquad J^{\infty }(w_{n})=-J^{\infty }( \hat{u})+o(1). $$

(2.43)

If there exists a subsequence $\{w_{n_{i}}\}$ of $w_{n}$ such that $w_{n_{i}}=0$, then we have

$$ \varPhi ^{\infty }(\hat{u}) =m, \qquad J^{\infty }( \hat{u})=0. $$

(2.44)

Next, we assume that $w_{n}\neq 0$, we claim that $J^{\infty }(w_{n}) \leq 0$. Otherwise, if $J^{\infty }(\hat{u})>0$, then (2.43) implies $J^{\infty }(w_{n})< 0$ for large n. In view of Corollary 2.7, there exists $t_{n}>0$ such that $t_{n}^{\beta }(w_{n})_{t_{n}}\in \mathcal{M}^{\infty }$ for large n. From (2.5), (2.12), (2.13), (2.43) and Lemma 2.13, we obtain

$$\begin{aligned} m-\varPsi ^{\infty }(\hat{u})+o(1) =&\varPsi ^{\infty }(w_{n}) \\ =&\varPhi ^{\infty }(w_{n})-\frac{1}{2\beta -1}J^{\infty }(w_{n}) \\ \geq& \varPhi ^{\infty }\bigl(t_{n}^{\beta }(w_{n})_{t_{n}} \bigr)-\frac{t^{2\beta -1}}{2\beta -1}J^{\infty }(w_{n}) \\ &{} + \biggl[\frac{(2\beta -1)(1-t^{2\beta -3})-(2\beta -3)(1-t^{2\beta -3})V _{\infty }}{2(2\beta -1)} \biggr] \Vert \nabla w_{n} \Vert _{2}^{2} \\ \geq& m^{\infty }-\frac{t^{2\beta -1}}{2\beta -1}J^{\infty }(w_{n}) \\ & + \biggl[\frac{(2\beta -1)(1-t^{2\beta -3})-(2\beta -3)(1-t^{2\beta -3})V _{\infty }}{2(2\beta -1)} \biggr] \Vert \nabla w_{n} \Vert _{2}^{2} \\ \geq &m, \end{aligned}$$

(2.45)

which is a contradiction due to $\varPsi ^{\infty }(\hat{u})> 0$. This shows that $J^{\infty }(\hat{u})\leq 0$. In view of Lemma 2.6, there exists $t_{\infty }>0$ such that $t_{\infty }^{\beta }\hat{u}_{t_{\infty }} \in \mathcal{M}^{\infty }$. By (2.5), (2.7), (2.36), (2.39), (2.42), Fatou’s lemma and Lemma 2.13, we have

$$\begin{aligned} m =&\lim_{n\rightarrow \infty } \biggl[\varPhi ^{\infty }( \hat{u}_{n})-\frac{1}{2 \beta -1}J^{\infty }(\hat{u}_{n}) \biggr] \\ \geq& \varPsi ^{\infty }(\hat{u}_{n})\geq \varPsi ^{\infty }(\hat{u}) = \varPhi ^{\infty }(\hat{u})-\frac{1}{2\beta -1}J^{\infty }( \hat{u}) \\ \geq &\varPhi ^{\infty }\bigl(t_{\infty }^{\beta } \hat{u}_{t_{\infty }}\bigr)-\frac{t ^{2\beta -1}}{2\beta -1}J^{\infty }(\hat{u}) \\ &{} + \biggl[\frac{(2\beta -1)(1-t^{2\beta -3})-(2\beta -3)(1-t^{2\beta -3})V _{\infty }}{2(2\beta -1)} \biggr] \Vert \hat{u} \Vert _{2}^{2} \\ \geq& m^{\infty }-\frac{t^{2\beta -1}}{2\beta -1}J^{\infty }(\hat{u})+ \biggl[ \frac{(2\beta -1)(1-t^{2\beta -3})-(2\beta -3)(1-t^{2\beta -3})V _{\infty }}{2(2\beta -1)} \biggr] \Vert \hat{u} \Vert _{2}^{2} \\ \geq& m, \end{aligned}$$

(2.46)

which implies (2.44) holds also. In view of Lemma 2.6, there exists $\hat{t}>0$ such that $\hat{t}^{\beta }\hat{u}_{\hat{t}} \in \mathcal{M}$, moreover, it follows from (V1), (1.3), (2.7), (2.44) and Corollary 2.4 that

$$ m \leq \varPhi \bigl(\hat{t}^{\beta }\hat{u}_{\hat{t}} \bigr) \leq \varPhi ^{\infty }\bigl( \hat{t}^{\beta } \hat{u}_{\hat{t}}\bigr)\leq \varPhi ^{\infty }(\hat{u})=m. $$

(2.47)

This shows that m is obtained at $\hat{t}^{\beta }\hat{u}_{\hat{t}} \in \mathcal{M}$.

Case (ii) $\bar{u}\neq 0$. In this case, analogous to the proof of (2.44), by using Φ and J instead of $\varPhi ^{\infty }$ and $J^{\infty }$, we can deduce that $\varPhi (\bar{u})=m$ and $J(\bar{u})=0$. □

In the same way as [19] or [35], we can obtain the following lemma.

Lemma 2.15

Assume that (V1), (V2), (F1), (F2), (F5) and (F6) hold. If $\bar{u}\in \mathcal{M}$ and $\varPhi (\bar{u})=m$, then ū is a critical point of Φ.

Proof of Theorem 1.1

In view of Lemma 2.14 and 2.15, there exists $\bar{u}\in \mathcal{M}$ such that

$$ \begin{aligned} \varPhi (\bar{u})=m=\inf _{u\in H^{1}(\mathbb{R}^{3})\setminus \{0\}} \max_{t>0}\varPhi \bigl(t^{\beta }u_{t} \bigr), \qquad \varPhi '(\bar{u})=0. \end{aligned} $$

(2.48)

This shows that ū is a ground state solution of (1.1) such that $\varPhi (\bar{u})=m=\inf_{\mathcal{M}}\varPhi$. □

3 Proofs of Theorem 1.2 and 1.3

Since we are looking for positive solutions to (1.1), without loss of generality, we suppose that $f(t)=0$ for $t<0$ in this section.

To use the Jeanjean’s monotonicity trick [18, Theorem 1.1], for $\lambda \in [1/2,1]$ we introduce two families of $\mathcal{C}^{1}$-functions on $H^{1}(\mathbb{R}^{3})$ defined by

$$ \begin{aligned} \varPhi _{\lambda }(u)= \frac{1}{2} \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u \vert ^{2}+V(x)u ^{2}\bigr)\,dx+\frac{1}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}\,dx- \lambda \int _{\mathbb{R}^{3}} F(u)\,dx \end{aligned} $$

(3.1)

and

$$ \begin{aligned} \varPhi _{\lambda }^{\infty }(u)= \frac{1}{2} \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u \vert ^{2}+V_{\infty }u^{2} \bigr)\,dx+\frac{1}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{ \alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}\,dx- \lambda \int _{\mathbb{R}^{3}} F(u)\,dx. \end{aligned} $$

(3.2)

In the same way as [13, 31], we can obtain the following lemma.

Lemma 3.1

Assume that (V1), (V3) (or(V3′)), (F1), (F2), (F5) and (F6) hold. Let u be a critical point of $\varPhi _{\lambda }$ in $H^{1}(\mathbb{R}^{3})$, then we have the following Pohožaev type identity:

$$ \begin{aligned}[b] P_{\lambda }(u):= {}& \frac{1}{2} \Vert \nabla u \Vert _{2}^{2}+ \frac{1}{2} \int _{\mathbb{R}^{3}}\bigl[3V(x)+\nabla V(x)\cdot x\bigr]u^{2} \,dx+\frac{3+\alpha }{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}\,dx \\ &{}- 3\lambda \int _{\mathbb{R}^{3}} F(u)\,dx. \end{aligned} $$

(3.3)

We set $J_{\lambda }(u):= \beta \langle \varPhi _{\lambda }^{\prime }(u),u \rangle -P_{\lambda }(u)$, then, for $\lambda \in [1/2,1]$,

$$ \begin{aligned}[b] J_{\lambda }(u) ={} & \frac{2\beta -1}{2} \Vert \nabla u \Vert _{2}^{2} + \frac{1}{2} \int _{\mathbb{R}^{3}}\bigl[(2\beta -3)V(x)-\nabla V(x)\cdot x\bigr]u ^{2}\,dx \\ &{}+\frac{2\beta -1}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}\,dx \\ &{}-\lambda \int _{\mathbb{R}^{3}}\bigl[\beta f(u)u-3F(u)\bigr]\,dx. \end{aligned} $$

(3.4)

Correspondingly, for $\lambda \in [1/2,1]$ we also let

$$ \begin{aligned}[b] J_{\lambda }^{\infty }(u) ={} &\frac{2\beta -1}{2} \Vert \nabla u \Vert _{2}^{2} + \frac{(2 \beta -3)V_{\infty }}{2} \Vert u \Vert _{2}^{2}+ \frac{2\beta -1}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u \vert ^{q}\bigr) \vert u \vert ^{q}\,dx \\ &{}-\lambda \int _{\mathbb{R}^{3}}\bigl[\beta f(u)u-3F(u)\bigr]\,dx. \end{aligned} $$

(3.5)

Set

$$ \begin{aligned} \mathcal{M}_{\lambda }^{\infty } := \bigl\{ u\in H^{1}\bigl(\mathbb{R}^{3}\bigr) \setminus \{0 \} :J_{\lambda }^{\infty }(u)=0 \bigr\} , \qquad m_{\lambda }^{\infty }:= \inf_{\mathcal{M}_{\lambda }^{\infty }} \varPhi _{\lambda }^{\infty }. \end{aligned} $$

(3.6)

By Corollary 2.3, we have the following lemma.

Lemma 3.2

Assume that (F1), (F5) and (F6) hold. Then

$$ \begin{aligned}[b] &\varPhi _{\lambda }^{\infty }(u) \geq \varPhi _{\lambda }^{\infty }\bigl(t^{ \beta }u_{t} \bigr)+\frac{1-t^{2\beta -1}}{2\beta -1}J_{\lambda }^{\infty }(u) \\ &\hphantom{\varPhi _{\lambda }^{\infty }(u) \geq}{}+\frac{(2\beta -1)(1-t^{2\beta -3})-(2\beta -3)(1-t^{2\beta -1})}{2(2 \beta -1)}V_{\infty } \Vert u \Vert _{2}^{2}, \\ &\quad \forall v\in H^{1}\bigl(\mathbb{R} ^{3}\bigr), t> 0. \end{aligned} $$

(3.7)

Since $f(t)=0$ for $t<0$, from Theorem 1.1, the strong maximum principle and standard arguments, we can deduce that $\varPhi _{1}^{\infty }$ has a minimizer $u_{1}^{\infty }>0$ on $\mathcal{M}_{1}^{\infty }$, i.e.

$$ \begin{aligned} u_{1}^{\infty }\in \mathcal{M}_{1}^{\infty }, \qquad \bigl(\varPhi _{1}^{\infty }\bigr)'\bigl(u_{1}^{\infty } \bigr)=0 \quad \mbox{and} \quad m_{1}^{\infty } = \varPhi _{1}^{\infty }\bigl(u_{1}^{\infty }\bigr). \end{aligned} $$

(3.8)

Lemma 3.3

Under the assumptions of Theorem 1.2 or Theorem 1.3, we have

• (i) There exists $T>0$ independent of λ such that $\varPhi _{\lambda }(T^{\beta }(u_{1}^{\infty })_{T})<0$ for all $\lambda \in [1/2,1]$;

• (ii) there exists a positive constant $\kappa _{0}$ independent of λ such that, for all $\lambda \in [1/2,1]$,

  $$ c_{\lambda }=\inf_{\gamma \in \tau }\max _{t\in [0,1]} \varPhi _{\lambda }\bigl( \gamma (t)\bigr) \geq \kappa _{0} > \max \bigl\{ \varPhi _{\lambda }(0),\varPhi _{\lambda }\bigl(T^{\beta }\bigl(u_{1}^{\infty } \bigr)_{T}\bigr)\bigr\} , $$

  (3.9)

  where $\tau =\{\gamma \in \mathcal{C}([0,1],H^{1}(\mathbb{R}^{3})): \gamma (0)=0,\gamma (1)=T^{\beta }(u_{1}^{\infty })_{T}\}$;

• (iii) $c_{\lambda }$ and $m_{\lambda }^{\infty }$ are non-increasing on $\lambda \in [1/2,1]$.

The proof of Lemma 3.3 is standard, so we omit it.

Lemma 3.4

Under the assumptions of Theorem 1.2 or Theorem 1.3, there exists $\bar{\lambda }\in [1/2,1)$ such that $c_{\lambda }< m_{\lambda }^{ \infty }$ for $\lambda \in (\bar{\lambda },1]$.

Proof

It is easy to see that $\varPhi _{\lambda }(t^{\beta }(u_{1}^{\infty })_{t})$ is continuous on $t\in (0,\infty )$. Hence for any $\lambda \in [1/2,1)$, we can choose $t_{\lambda }\in (0,T)$ such that $\varPhi _{\lambda }(t_{\lambda }^{ \beta }((u_{1}^{\infty })_{t_{\lambda }}))$. Let $\beta _{0}= \inf_{\lambda \in [1/2,1]}t_{\lambda }$. If $\beta _{0}=0$, then there exists a sequence $\{\lambda _{n}\}\subset [1/2,1]$ such that $\lambda _{n}\rightarrow \lambda _{0} \in [1/2,1]$ and $t_{\lambda _{n}} \rightarrow 0$, and so by (3.1) and Lemma 3.3(iii), one has

$$ \begin{aligned} 0< c_{1}\leq c_{\lambda _{n}} \leq \varPhi _{\lambda _{n}}\bigl(t_{\lambda _{n}} ^{\beta } \bigl(u_{1}^{\infty }\bigr)_{t_{\lambda _{n}}}\bigr)=o(1). \end{aligned} $$

(3.10)

This contradiction shows $\beta _{0}>0$. Thus $0<\beta _{0}\leq t_{ \lambda }<T$ for all $\lambda \in [1/2,1]$. Let

$$ \begin{aligned} \bar{\lambda } :=\max \biggl\{ \frac{1}{2},1-\frac{\beta _{0}^{2\beta -3}T^{3}\min_{\beta _{0}\leq s\leq T}\int _{\mathbb{R}^{3}}[V _{\infty }-V(sx)] \vert u_{1}^{\infty } \vert ^{2}\,dx}{2\int _{\mathbb{R}^{3}}F(T ^{\beta }u_{1}^{\infty })\,dx} \biggr\} . \end{aligned} $$

(3.11)

Then $1/2 \leq \bar{\lambda } <1$. From (3.1), (3.2), (3.7), (3.11) and Lemma 3.3(iii), we derive

$$\begin{aligned} m_{\lambda }^{\infty } \geq &m_{1}^{\infty }= \varPhi _{1}^{\infty }\bigl(u _{1}^{\infty }\bigr) \geq \varPhi _{1}^{\infty }\bigl(t_{\lambda }^{\beta } \bigl(u_{1} ^{\infty }\bigr)_{t_{\lambda }}\bigr) \\ =&\varPhi _{\lambda }\bigl(t_{\lambda }^{\beta } \bigl(u_{1}^{\infty }\bigr)_{t_{\lambda }}\bigr)- \frac{(1-\lambda )}{t_{\lambda }^{3}} \int _{\mathbb{R}^{3}}F\bigl(t_{ \lambda }^{\beta } u_{1}^{\infty }\bigr)\,dx + \frac{t_{\lambda }^{2\beta -3}}{2} \int _{\mathbb{R}^{3}}\bigl[V_{\infty }-V(t _{\lambda }x)\bigr] \bigl\vert u_{1}^{\infty } \bigr\vert ^{2}\,dx \\ \geq &c_{\lambda }-\frac{(1-\lambda )}{T^{3}} \int _{\mathbb{R}^{3}}F\bigl(T ^{\beta }u_{1}^{\infty } \bigr)\,dx+\frac{\beta _{0}^{2\beta -3}}{2} \min_{\beta _{0}\leq s \leq T} \int _{\mathbb{R}^{3}}\bigl[V_{\infty }-V(sx)\bigr] \bigl\vert u_{1}^{\infty } \bigr\vert ^{2}\,dx \\ >&c_{\lambda } , \quad \forall \lambda \in (\bar{\lambda },1]. \end{aligned}$$

(3.12)

 □

Lemma 3.5

Under the assumptions of Theorem 1.2 or Theorem 1.3, for almost every $\lambda \in (\bar{\lambda },1]$, there exists $u_{\lambda }\in H^{1}( \mathbb{R}^{3})\setminus \{0\}$ such that.

$$ \varPhi _{\lambda }'(u_{\lambda })= 0, \qquad \varPhi _{\lambda }(u_{\lambda })=c_{\lambda }. $$

(3.13)

Proof

In view of the Jeanjean’s monotonicity trick [36, Theorem 1.1] and Lemma 3.3, for almost every $\lambda \in [1/2,1]$, there exists a bounded sequence $\{u_{n}(\lambda )\}\subset H^{1}(\mathbb{R}^{3})$, for simplicity, we denote it by $\{u_{n}\}$ instead of $\{u_{n}(\lambda ) \}$ such that

$$ \begin{aligned} \varPhi _{\lambda }(u_{n}) \rightarrow c_{\lambda }, \qquad \bigl\Vert \varPhi _{\lambda }'(u_{n}) \bigr\Vert \rightarrow 0. \end{aligned} $$

(3.14)

Using Lemma 2.12, we can deduce that there exists $u_{\lambda }\in H ^{1}(\mathbb{R}^{3})$, an integer $l\in \mathbb{N} \cup \{0\}$, a sequence $\{y_{n}^{k}\} \subset \mathbb{R}^{3}$ and $w^{k}\in H^{1}( \mathbb{R}^{3})$ for $1\leq k \leq l$ such that $u_{n}\rightharpoonup u_{\lambda }$ in $H^{1}(\mathbb{R}^{3})$, $\varPhi _{\lambda }^{\prime }(u_{ \lambda })=0$, $(\varPhi _{\lambda }^{\infty })^{\prime }(w^{k})=0$ and $\varPhi _{\lambda }^{\infty }(w^{k})\geq m_{\lambda }^{\infty }$ for $1 \leq k \leq l$,

$$ \begin{aligned} \Biggl\Vert u_{n}-u_{\lambda }- \sum_{k=1}^{l}w^{k}\bigl(\cdot + y_{n}^{k}\bigr) \Biggr\Vert \rightarrow 0 \quad \mbox{and} \quad \varPhi _{\lambda }(u_{n})\rightarrow \varPhi _{\lambda }(u_{\lambda })+\sum_{k=1}^{l} \varPhi _{\lambda }^{\infty }\bigl(w^{i}\bigr). \end{aligned} $$

(3.15)

Since $\varPhi _{\lambda }^{\prime }(u_{\lambda })=0$, $J_{\lambda }(u_{\lambda })=0$. It follows from (3.1) and (3.4) that

$$ \begin{aligned}[b] \varPhi _{\lambda }(u_{\lambda }) ={}& \varPhi _{\lambda }(u_{\lambda })-\frac{1}{2 \beta -1}J_{\lambda }(u_{\lambda }) \\ ={}&\frac{1}{2(2\beta -1)} \int _{\mathbb{R}^{3}}\bigl[2V(x)+\nabla V(x) \cdot x\bigr]u_{\lambda }^{2} \,dx \\ &{}+\frac{\lambda }{2\beta -1} \int _{\mathbb{R}^{3}}\bigl[\beta f(u_{\lambda })u_{\lambda }-(2 \beta +2)F(u_{\lambda })\bigr]\,dx \geq 0. \end{aligned} $$

(3.16)

If $l\neq 0$, then

$$ \begin{aligned} c_{\lambda } =\lim _{n\rightarrow \infty }\varPhi _{\lambda }(u_{n})= \varPhi _{\lambda }(u_{\lambda })+\sum_{i=1}^{l} \varPhi _{\lambda }^{\infty }\bigl(w ^{i}\bigr)\geq m_{\lambda }^{\infty }, \quad \forall \lambda \in (\bar{\lambda },1], \end{aligned} $$

(3.17)

which is a contradiction by Lemma 3.4. Thus $l=0$, and (3.15) implies that $u_{n}\rightarrow u_{\lambda }$ in $H^{1}(\mathbb{R}^{3})$ and $\varPhi _{\lambda }(u_{\lambda })=c_{\lambda }$ for almost every $\lambda \in (\bar{\lambda },1]$. □

Lemma 3.6

Under the assumptions of Theorem 1.2 or Theorem 1.3, there exists $\bar{u}>0$ such that

$$ \varPhi '(\bar{u})= 0, \qquad \varPhi ( \bar{u})=c_{1}>0. $$

(3.18)

Proof

Under the assumptions of Theorem 1.2 or 1.3, in view of Lemma 3.5, there exists two sequences of $\{\lambda _{n}\} \subset (\bar{\lambda },1]$ and $\{u_{\lambda _{n}}\} \subset H^{1}(\mathbb{R}^{3})$, denoted $\{u_{n}\}$, such that

$$ \begin{aligned} \lambda _{n}\rightarrow 1, \qquad \varPhi _{\lambda _{n}}'(u_{n})=0, \qquad \varPhi _{\lambda _{n}}(u_{n})=c_{\lambda _{n}}. \end{aligned} $$

(3.19)

By (3.1), (3.4), (3.19) and Lemma 3.3(iii), one has

$$ \begin{aligned}[b] c_{1/2} \geq c_{\lambda _{n}} ={}&\varPhi _{\lambda _{n}}(u_{n})- \frac{1}{2 \beta -1}J_{\lambda _{n}}(u_{n}) \\ ={}&\frac{1}{2(2\beta -1)} \int _{\mathbb{R}^{3}}\bigl[2V(x)+\nabla V(x) \cdot x\bigr]u_{n}^{2} \,dx \\ &{}+\frac{\lambda _{n}}{2\beta -1} \int _{\mathbb{R}^{3}}\bigl[\beta f(u_{n})u _{n}-(2 \beta +2)F(u_{n})\bigr]\,dx. \end{aligned} $$

(3.20)

To prove the boundedness of $\{\|u_{n}\|\}$, we distinguish two cases: (1) (V3) holds; (2) (V3′) holds.

Case (1) (V3) holds. In this case, by (F6) and (3.20), one has

$$ c_{1/2} \geq \frac{\rho _{1}}{2(2\beta -1)} \Vert u_{n} \Vert _{2}^{2}, $$

(3.21)

which implies that $\{\|u_{n}\|_{2}\}$ is bounded. Next, we prove that $\{\|\nabla u_{n}\|_{2}\}$ is also bounded. Arguing by contradiction, suppose that $\|\nabla u_{n}\|_{2} \rightarrow \infty$. By (V1), (V3), (3.21) and Lemma 3.3(iii), one has

$$ c_{\lambda _{n}} + \int _{\mathbb{R}^{3}}\bigl[(2\beta -2)V_{\infty }-(2 \beta -2)V(x)- \bigl\vert \nabla V(x)\cdot x \bigr\vert \bigr]u_{n}^{2} \leq M_{0}, $$

(3.22)

for some constant $M_{0}$. Let $t_{n}=\min \{1,2(M_{0}/\|\nabla u_{n} \|_{2}^{2})^{1/{2\beta -1}}\}$, then $t_{n}\rightarrow 0$. Thus, it follows from (3.1), (3.2), (3.4), (3.5) and (3.22) that

$$\begin{aligned} \varPhi _{\lambda _{n}}^{\infty }\bigl(t_{n}^{\beta }(u_{n})_{t_{n}} \bigr) \leq& \varPhi _{\lambda _{n}}^{\infty }(u_{n}) - \frac{1-t^{2\beta -1}}{2\beta -1}J_{\lambda _{n}}^{\infty }(u_{n}) \\ =& \varPhi _{\lambda _{n}}(u_{n}) +\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl[V_{\infty }-V(x)\bigr]u_{n}^{2} \,dx \\ &{}- \frac{1-t^{2\beta -1}}{2\beta -1} \biggl\{ J_{\lambda _{n}}(u_{n})+ \frac{1}{2} \int _{\mathbb{R}^{3}}\bigl[(2\beta -3)V_{\infty } \\ &{}-(2\beta -3)V(x)- \bigl\vert \nabla V(x)\cdot x \bigr\vert \bigr]u_{n}^{2} \,dx \biggr\} \\ \leq& c_{\lambda _{n}} + \int _{\mathbb{R}^{3}}\bigl[(2\beta -2)V_{\infty }-(2 \beta -2)V(x)- \bigl\vert \nabla V(x)\cdot x \bigr\vert \bigr]u_{n}^{2} \leq M_{0}. \end{aligned}$$

(3.23)

Analogous to the proof of (2.34), we can deduce a contradiction by using (3.23). Hence, $\{u_{n}\}$ is bounded in $H^{1}(\mathbb{R} ^{3})$ under the assumptions of Theorem 1.2.

Case (2) (V3′) holds. In this case, (V3′) and (3.20) imply

$$ \begin{aligned}[b] c_{1/2} &\geq \frac{\lambda _{n}}{2\beta -1} \int _{\mathbb{R}^{3}}\bigl[ \beta f(u_{n})u_{n}-(2 \beta +2)F(u_{n})\bigr]\,dx \\ &\geq C_{5}\biggl(\mu -\frac{2\beta +2}{\beta }\biggr) \int _{\mathbb{R}^{3}}F(u_{n})\,dx. \end{aligned} $$

(3.24)

Then it follows from (V1), (3.1), and (3.24) that

$$ \begin{aligned}[b] \frac{\gamma _{0}}{2} \Vert u_{n} \Vert ^{2} & \leq \frac{1}{2} \int _{\mathbb{R} ^{3}} \bigl[ \vert \nabla u_{n} \vert ^{2}+V(x)u_{n}^{2} \bigr]\,dx+\frac{1}{2q} \int _{\mathbb{R}^{3}}\bigl(I_{\alpha }\ast \vert u_{n} \vert ^{q}\bigr) \vert u_{n} \vert ^{q}\,dx \\ &=\lambda \int _{\mathbb{R}^{3}} F(u_{n})\,dx \leq C_{6}, \end{aligned} $$

(3.25)

where $\gamma _{0}$ is a positive constant. Hence, $\{u_{n}\}$ is bounded in $H^{1}(\mathbb{R}^{3})$ under the assumptions of Theorem 1.3. Similar to the proof of Lemma 3.5, there exists $\bar{u}\in H^{1}(\mathbb{R} ^{3})\setminus \{0\}$ such that (3.18) holds. Moreover, by the strong maximum principle and a standard argument, we can conclude that $\bar{u}>0$. □

Proofs of Theorem 1.2 and Theorem 1.3 . Let

$$ \begin{aligned} K:= \bigl\{ u\in H^{1}\bigl( \mathbb{R}^{3}\bigr)\setminus \{0\}: \varPhi '(u)=0\bigr\} , \qquad \hat{m}:=\inf_{u\in K}\varPhi (u). \end{aligned} $$

(3.26)

Then Lemma 3.6 shows that $K \neq \emptyset$ and $\hat{m} \leq c_{1}$. For any $u\in K$, (1.8), (3.4) and Lemma 3.1 imply $J(u)=J_{1}(u)=\beta \langle \varPhi '(u),u\rangle -\mathcal{P}(u)=0$. As in (3.16), we have $\varPhi (u)=\varPhi _{1}(u)\geq 0$ for any $u\in K$, and so $\hat{m}\geq 0$. Let $\{u_{n}\} \subset K$ such that $\varPhi '(u_{n})=0$ and $\varPhi (u_{n})\rightarrow \hat{m}$. In view of Lemma 3.4, $\hat{m}\leq c_{1}\leq m_{1}^{\infty }$. Similar to the proof of Lemma 3.6, we can deduce that there exists $\hat{u}>0$ such that $\varPhi '(\hat{u})=0$ and $\varPhi (\hat{u})=\hat{m}$. This shows that $\hat{u}\in H^{1}(\mathbb{R}^{3})$ is a positive ground state solution of (1.1).