Positive bound state solutions for the nonlinear Schrödinger–Poisson systems with potentials

Abstract

In this paper, we study the following Schrödinger–Poisson system in $\mathbb{R} ^{3}$

$$ \textstyle\begin{cases} (-\Delta )^{\sigma }u+A(y)u+B(y)\phi (y)u=b(y) \vert u \vert ^{p-1}u, & y \in \mathbb{R} ^{3}, \\ (-\Delta )^{\sigma }\phi =B(y)u^{2}, & y\in \mathbb{R} ^{3}, \end{cases} $$

with $\frac{3}{4}<\sigma <1$, $p\in (3,\frac{3+2\sigma }{3-2 \sigma })$. Then, under some suitable assumptions on the coefficients not requiring any symmetry property, we prove the existence of a bound state solution of the above problem.

1 Introduction

This paper concerns the non-autonomous Schrödinger–Poisson system

$$ \textstyle\begin{cases} (-\Delta )^{\sigma }u+A(y)u+B(y)\phi (y)u=b(y) \vert u \vert ^{p-1}u, & y \in \mathbb{R} ^{3}, \\ (-\Delta )^{\sigma }\phi =B(y)u^{2}, & y\in \mathbb{R} ^{3}, \end{cases} $$

(1.1)

where $\frac{3}{4}<\sigma <1$, $p\in (3,\frac{3+2\sigma }{3-2\sigma })$, $A(y)$, $B(y)$, and $b(y)$ are positive functions. Here $B(y):\mathbb{R} ^{3} \rightarrow R$ denotes the nonnegative measurable function which represents a nonconstant charge corrector to the density $u^{2}$ and $A(y)$ and $b(y)$ are called the potentials of system (1.1). Moreover, the fractional Laplacian $(-\Delta )^{\sigma }$ in $\mathbb{R} ^{N}$ is defined by

$$ (-\Delta )^{\sigma }u=C_{N,\sigma }\mathit{P.V.}\int _{\mathbb{R} ^{N}}\frac{u(x)-u(y)}{ \vert x-y \vert ^{N+2 \sigma }}\,dy, $$

where P.V. stands for the Cauchy principal value, $C_{N,\sigma }$ is a normalization constant.

This kind of system also arises in many fields of physics. Indeed, one considers the following system:

$$ \textstyle\begin{cases} i\frac{\partial \psi }{\partial t}+(-\Delta )^{\sigma }\psi +(A(y)-h) \psi +B(y)\phi (y)\psi =b(y) \vert \psi \vert ^{p-1}\psi , & y\in \mathbb{R} ^{3}, t\in \mathbb{R}, \\ (-\Delta )^{\sigma }\phi =B(y)\psi ^{2}, & y\in \mathbb{R} ^{3}, \end{cases} $$

(1.2)

where i is the imaginary unit, $(-\Delta )^{\sigma }$ is the fractional operator. From the physical as well as the mathematical point of view, a central issue is the existence and dynamic of standing waves of (1.2). By standing waves, we want to look for the form $\psi =e^{-iht}u$ of the solution of (1.2), where $y\in \mathbb{R} ^{3}$, $t>0$. It is clear that ψ solves (1.2) if and only if u solves (1.1). The fractional Schrödinger equation in (1.2) is an important model in the study of fractional quantum mechanics. In Refs. [12, 13], Laskin introduced this equation by expanding the Feynman path integral from the Brownian-like to the Lévy-like quantum mechanical paths.

Many researches have been devoted to the study of (1.1) when $\sigma =1$, i.e.,

$$ \textstyle\begin{cases} -\Delta u+A(y)u+B(y)\phi (y)u=b(y) \vert u \vert ^{p-1}u, & y\in \mathbb{R} ^{3}, \\ -\Delta \phi =B(y)u^{2}, & y\in \mathbb{R} ^{3}, \end{cases} $$

(1.3)

which mainly concerns either the autonomous or the non-autonomous case. But it is well known that, dealing with system (1.3), one has to face different kinds of difficulties, which are related to potentials and the unboundedness of the space $\mathbb{R} ^{3}$. So many studies were devoted to the autonomous or the non-autonomous case in which the coefficients are supposed to be radial. In [19], the existence of multiple solutions of (1.3) have been found in a radial setting under some suitable assumptions on $A(y)$, $B(y)$, $b(y)$. In [16], the author considered the case that $B=1$, $A(y)$, $b(y)$ are radial and satisfy some decay conditions and proved the existence of nontrivial positive classical mountain-pass solution of (1.3). Moreover, some more general case, replacing $b(y)|u|^{p-1}u$ by $f(x,u)$, was considered in [25, 29]. More recently, many contributions to (1.3) have also been given in which no symmetry assumptions are given on the coefficients appearing in (1.3). Cerami and Molle [6] obtained the existence of bound state, finite energy solution of (1.3) under suitable assumptions on the decay rate of the coefficients A, B, b. In [17], Mercuri and Tyler proved the existence of mountain-pass solutions and least energy solutions to the nonlinear Schrödinger–Poisson system (1.3) with $A(y)=b(y)=1$ and $p\in (2,5)$ under different assumptions on $B: \mathbb{R} ^{3}\rightarrow \mathbb{R} _{+}$ at infinity. Furthermore, they also studied the singularly perturbed problem and found necessary conditions for concentration at points to occur for solutions to the singularly perturbed problem in various functional settings. For more results on the existence of positive or sign-changing solutions, ground and bound states, one can refer to [1, 2, 5, 9, 18, 20, 24] and the references therein.

Since fractional Schrödinger equation is coupled with a fractional Poisson term $\phi (y)u$, the existence of multiple nonlocal terms causes some mathematical difficulties and makes the study of system (1.1) very interesting. In recent years, several scholars paid their attention to the existence of positive, ground state, semiclassical, and other solutions to fractional Schrödinger–Poisson system or similar problems. For the information, one can refer to [21–23, 27–29] and the references therein. However, it is worth to point out that in most of the papers mentioned above, the study involves positive ground state solutions to (1.1). In the present paper we consider a situation that has to be studied in a different way. We will find the positive solution that differs from positive ground state solution. Here a solution u of (1.1) is nontrivial if $u\neq 0$. A solution of (1.1) is a nontrivial bound state solution if u is a nontrivial solution. A solution u with $u>0$ is called a positive solution. A solution is called a nontrivial ground state solution if its energy (see (2.3)) is minimal among all the nontrivial solutions of (1.1).

In order to state our main result, we give the conditions imposed on $A(y)$, $B(y)$, and $b(y)$ as follows:

($A_{1}$):

$A(y)=A_{\infty }+W(y)$, where $A_{\infty }\in \mathbb{R} ^{+} \backslash \{0\}$ and $W(y)\in L^{3/2\sigma }(\mathbb{R} ^{3})$ is a nonnegative function such that

$$ \lim_{ \vert y \vert \rightarrow \infty } \vert y \vert ^{-3\sigma -3}W(y)=0. $$

($A_{2}$):

$0\not \equiv B(y)\in L^{2}(\mathbb{R} ^{3})$ is a nonnegative function such that, for some $\epsilon >2\sigma$ and $\hat{c},c, \bar{R}>0$,

$$ B(y)\leq \frac{\hat{c}}{(1+ \vert y \vert )^{\epsilon }}\leq \frac{c}{ \vert y \vert ^{\epsilon }}, \quad \vert y \vert >\bar{R}. $$

($A_{3}$):

$b(y)=b_{\infty }-\beta (y)$, where $b_{\infty }\in \mathbb{R} ^{+}\backslash \{0\}$ and $0\leq \beta (y)< b_{\infty }$ and

$$ \lim_{ \vert y \vert \rightarrow \infty } \vert y \vert ^{-3\sigma -3}\beta (y)=0. $$

Now we state our main result as follows.

Theorem 1.1

Suppose that conditions$(A_{1})$, $(A_{2})$, $(A_{3})$hold and$p\in (3,\frac{3+2 \sigma }{3-2\sigma })$. Then (1.1) admits a bound state solution$(u,\phi )\in H^{\sigma }(\mathbb{R} ^{3})\times D^{\sigma }(\mathbb{R} ^{3})$, whose components are positive functions.

Remark 1.2

It should be pointed out that in this paper, we just consider (1.1) with $\frac{3}{4}<\sigma <1$. But it would be interesting if one can find an analogous result to Theorem 1.1 to (1.1) for all $0<\sigma <1$. However, in the radial setting, Bellazzini et al. [4] studied (1.1) with $A(y)=0$, $B(y)=b(y)=1$ by discussing the existence of the optimizers of the Gagliardo–Nirenberg type inequalities.

To the best of our knowledge, this is the first result on the existence of bound state solution of (1.1) with competing coefficients. It is worth mentioning that the conditions imposed on our potentials decay algebraically at infinity, which is a contrast to the fact that the potentials decay exponentially at infinity in [6].

Here we give the following notations which can be used in this paper.

• (i) $H^{\sigma }(\mathbb{R} ^{3})$ is the usual Sobolev space endowed with the standard scalar product and norm

  $$ (u,v)= \int _{\mathbb{R} ^{3}}\bigl[(-\Delta )^{\frac{\sigma }{2}}u(-\Delta )^{\frac{ \sigma }{2}}v+uv\bigr], \qquad \Vert u \Vert ^{2}_{H^{\sigma }}= \int _{\mathbb{R} ^{3}} \bigl( \bigl\vert (-\Delta )^{\frac{\sigma }{2}}u \bigr\vert ^{2}+u^{2} \bigr). $$

• (ii) $D^{\sigma }(\mathbb{R} ^{3})$ is the completion of $C_{0}^{\infty }(\mathbb{R} ^{3})$ with the norm defined by $\Vert u\Vert ^{2}_{D^{\sigma }}= \int _{\mathbb{R} ^{3}}|(-\Delta )^{\frac{\sigma }{2}}u|^{2}$.

• (iii) $\|u\|_{q}$ is the norm of the Lebesgue space $L^{q}(\mathbb{R} ^{3})$.

• (iv) Denote by $C>0$ various positive constants which may vary from one line to another and which are not important for the analysis of the problem.

This paper is organized as follows. In Sect. 2, we give some preliminary results which contain some known results and some useful estimates. And then the proof of Theorem 1.1 is given in Sect. 3.

2 Preliminaries

In this part we mainly give some basic knowledge which will be used later. We first show that the second equation of (1.1) can be solved. For $u\in H^{\sigma }(\mathbb{R} ^{3})$, the linear functional $J_{u}$ is defined in $D^{\sigma }(\mathbb{R} ^{3})$ by

$$ J_{u}(v)= \int _{\mathbb{R} ^{3}}B(y)u^{2}v. $$

Applying condition $(A_{2})$ and Hölder’s inequality, we find that

$$ \bigl\vert J_{u}(v) \bigr\vert \leq C \Vert u \Vert _{\frac{12}{3+2\sigma }}^{2} \Vert v \Vert _{D^{\sigma }}. $$

By the Lax–Milgram theorem, we know that there exists unique $\phi _{u}\in D^{\sigma }(\mathbb{R} ^{3})$ such that

$$ \int _{\mathbb{R} ^{3}}(-\Delta )^{\frac{\sigma }{2}}\phi _{u}(- \Delta )^{\frac{ \sigma }{2}}v = \int _{\mathbb{R} ^{3}}B(y)u^{2}v, \quad \forall v\in D^{\sigma }\bigl(\mathbb{R} ^{3}\bigr). $$

So, $\phi _{u}$ is a weak solution of $(-\Delta )^{\sigma }\phi =B(y)u ^{2}$, and there holds

$$ \phi _{u}(y)=C_{\sigma } \int _{\mathbb{R} ^{3}}\frac{B(x)u^{2}(x)}{ \vert y-x \vert ^{3-2\sigma }}\,dx, $$

(2.1)

where $C_{\sigma }=\pi ^{-\frac{3}{2}}2^{-2\sigma }\frac{\varGamma (\frac{3-2 \sigma }{2})}{\varGamma (\sigma )}$.

Thus, substituting $\phi _{u}$ into the first equation of (1.1), then (1.1) is reduced to

$$ (-\Delta )^{\sigma }u+A(y)u+B(y)\phi _{u}(y)u=b(y) \vert u \vert ^{p-1}u, \quad y\in \mathbb{R} ^{3}. $$

(2.2)

Moreover, it is well known that solutions of (1.1) correspond to the critical points of the energy functional

$$ I(u)=\frac{1}{2} \int _{\mathbb{R} ^{3}} \bigl( \bigl\vert (-\Delta )^{\frac{\sigma }{2}}u \bigr\vert ^{2}+A(y)u^{2} \bigr)+\frac{1}{4} \int _{\mathbb{R} ^{3}}B(y)\phi _{u}u^{2} - \frac{1}{p+1} \int _{\mathbb{R} ^{3}}b(y) \vert u \vert ^{p+1}. $$

(2.3)

Without loss of generality, in what follows, we assume that $A_{\infty }=b_{\infty }=1$, and let us now define

$$ \varPhi (u)=\phi _{u}. $$

Then in the following, we summarize some properties of Φ, useful to studying our problem, and which can be verified by using the same argument as the case of Poisson equations in $D^{1,2}(\mathbb{R} ^{3})$ (see [8, 19]).

Lemma 2.1

• (1) Φis continuous;

• (2) Φmaps bounded sets into bounded sets;

• (3) $\varPhi (tu)=t^{2}\varPhi (u)$.

Lemma 2.2

Suppose that$u_{n}\rightharpoonup u$in$H^{\sigma }(\mathbb{R} ^{3})$, then

• (1) $\varPhi (u_{n})\rightarrow \varPhi (u)$in$D^{\sigma }(\mathbb{R} ^{3})$;

• (2) $\int _{\mathbb{R} ^{3}}B(y)\phi _{u_{n}}u_{n}^{2}\,dy\rightarrow \int _{\mathbb{R} ^{3}}B(y)\phi _{u}u^{2}\,dy$;

• (3) $\int _{\mathbb{R} ^{3}}B(y)\phi _{u_{n}}\varphi \,dy\rightarrow \int _{\mathbb{R} ^{3}}B(y)\phi _{u}\varphi \,dy$, $\forall \varphi \in H^{\sigma }(\mathbb{R} ^{3})$.

It is not difficult to show that the functional I is bounded neither from below, nor from above. So it is convenient to consider I restricted to a natural constraint, the Nehari manifold, which contains all the critical points of I.

Set

$$ \mathcal{N}= \bigl\{ u\in H^{\sigma }\bigl(\mathbb{R} ^{3}\bigr) \backslash \{0\}: \bigl\langle I'(u), u\bigr\rangle =0 \bigr\} . $$

So, for all $u\in \mathcal{N}$, we are led to

$$\begin{aligned} I|_{\mathcal{N}}(u) =& \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \int _{\mathbb{R} ^{3}} \bigl( \bigl\vert (-\Delta )^{\frac{\sigma }{2}}u \bigr\vert ^{2}+A(y)u^{2} \bigr) + \biggl( \frac{1}{4}-\frac{1}{p+1} \biggr) \int _{\mathbb{R} ^{3}}B(y)\phi _{u}u^{2} \\ =& \frac{1}{4} \int _{\mathbb{R} ^{3}} \bigl( \bigl\vert (-\Delta )^{\frac{\sigma }{2}}u \bigr\vert ^{2}+A(y)u^{2} \bigr) + \biggl( \frac{1}{4}-\frac{1}{p+1} \biggr) \int _{\mathbb{R} ^{3}}b(y) \vert u \vert ^{p+1}, \end{aligned}$$

(2.4)

which tells that I is bounded from below on $\mathcal{N}$.

Then we have the following.

Lemma 2.3

• (i) $\mathcal{N}$is a$C^{1}$regular manifold diffeomorphic to sphere of$H^{\sigma }(\mathbb{R} ^{3})$;

• (ii) Iis bounded from below on$\mathcal{N}$by a positive constant;

• (iii) uis a free critical point ofIif and only ifuis a critical point ofIconstrained on$\mathcal{N}$.

Proof

(i) Let $u\in H^{\sigma }(\mathbb{R} ^{3})\backslash \{0\}$ with $\Vert u\Vert _{H ^{\sigma }}=1$. Then we claim that there exists a unique $t\in \mathbb{R} ^{+}\backslash \{0\}$ such that $tu\in \mathcal{N}$. In fact, considering that t satisfies

$$\begin{aligned} 0 =&\bigl\langle I'(tu), tu\bigr\rangle \\ =&t^{2} \int _{\mathbb{R} ^{3}} \bigl( \bigl\vert (-\Delta )^{\frac{\sigma }{2}}u \bigr\vert ^{2}+A(y)u^{2} \bigr) +t^{4} \int _{\mathbb{R} ^{3}}B(y)\phi _{u}u^{2}-t^{p+1} \int _{\mathbb{R} ^{3}}b(y) \vert u \vert ^{p+1}, \end{aligned}$$

(2.5)

we have

$$\begin{aligned}& t^{2} \biggl(1+ \int _{\mathbb{R} ^{3}}W(y)u^{2}(y)+t^{2} \int _{\mathbb{R} ^{3}}B(y)\phi _{u}u^{2}-t^{p-1} \int _{\mathbb{R} ^{3}}b(y) \vert u \vert ^{p+1} \biggr) \\& \quad =:t^{2}\bigl(1+d_{1}+t^{2}d_{2}-t^{p-1}d_{3} \bigr)=0 \end{aligned}$$

(2.6)

with $d_{1},d_{2},d_{3}>0$. So, from $p>3$, the equation $1+d_{1}+t ^{2}d_{2}-t^{p-1}d_{3}=0$ has a unique solution $t:=t_{u}>0$ and then $t_{u}u\in \mathcal{N}$, which is called the projection of u on $\mathcal{N}$, satisfies

$$ I(t_{u}u)=\max_{t>0}I(tu). $$

(2.7)

(ii) Now suppose that $u\in \mathcal{N}$. Then

$$\begin{aligned} 0 =& \int _{\mathbb{R} ^{3}} \bigl( \bigl\vert (-\Delta )^{\frac{\sigma }{2}}u \bigr\vert ^{2}+\bigl(1+W(y)\bigr)u ^{2} \bigr) + \int _{\mathbb{R} ^{3}}B(y)\phi _{u}u^{2}- \int _{\mathbb{R} ^{3}}b(y) \vert u \vert ^{p+1} \\ \geq& \Vert u \Vert ^{2}_{H^{\sigma }}-C_{0} \Vert u \Vert ^{p+1}_{H^{\sigma }}, \end{aligned}$$

(2.8)

which yields

$$ \Vert u \Vert _{H^{\sigma }}\geq C_{1}>0, \quad \forall u\in \mathcal{N}. $$

(2.9)

Thus, using (2.4) and (2.9), we find

$$\begin{aligned} I(u) =& \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \int _{\mathbb{R} ^{3}} \bigl( \bigl\vert (-\Delta )^{\frac{\sigma }{2}}u \bigr\vert ^{2}+A(y)u^{2} \bigr) + \biggl( \frac{1}{4}-\frac{1}{p+1} \biggr) \int _{\mathbb{R} ^{3}}B(y)\phi _{u}u^{2} \\ \geq& \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \Vert u \Vert ^{2}_{H^{\sigma }}>C_{2}>0. \end{aligned}$$

(2.10)

(iii) First, it is obvious that if $u\neq 0$ is a critical point of I, $I'(u)=0$ and then $u\in \mathcal{N}$. On the other hand, writing $G(u)=\langle I'(u), u\rangle$, then from (2.9), for $u\in \mathcal{N}$, we get

$$\begin{aligned} \bigl\langle G'(u), u\bigr\rangle =&2 \int _{\mathbb{R} ^{3}} \bigl( \bigl\vert (-\Delta )^{\frac{\sigma }{2}}u \bigr\vert ^{2}+A(y)u^{2} \bigr) +4 \int _{\mathbb{R} ^{3}}B(y)\phi _{u}u^{2}-(p+1) \int _{\mathbb{R} ^{3}}b(y) \vert u \vert ^{p+1} \\ =&(1-p) \int _{\mathbb{R} ^{3}} \bigl( \bigl\vert (-\Delta )^{\frac{\sigma }{2}}u \bigr\vert ^{2}+A(y)u^{2} \bigr)+(3-p) \int _{\mathbb{R} ^{3}}B(y)\phi _{u}u^{2} \\ \leq& (1-p) \Vert u \Vert ^{2}_{H^{\sigma }}< 0. \end{aligned}$$

(2.11)

Letting u be a critical point of I constrained on $\mathcal{N}$, then there is $\lambda \in \mathbb{R}$ such that

$$ I'(u)=\lambda G'(u). $$

Hence

$$ 0=G(u)=\bigl\langle I'(u), u\bigr\rangle =\lambda \bigl\langle G'(u), u\bigr\rangle , $$

which, by (2.11), implies that $\lambda =0$ and then $I'(u)=0$. □

Now, we introduce the following problem:

$$ \textstyle\begin{cases} (-\Delta )^{\sigma }u+u= \vert u \vert ^{p-1}u, & y\in \mathbb{R} ^{3}, \\ u\in H^{\sigma }(\mathbb{R} ^{3}). \end{cases} $$

(2.12)

Concerning problem (2.12), we have the following proposition.

Proposition 2.4

(see [10, 11])

(2.12) has a ground state, positive solution$U\in H^{\sigma }(\mathbb{R} ^{3})$, which is radially symmetric about the origin, unique up to translations, and satisfies

$$ \frac{C_{1}}{1+ \vert y \vert ^{3+2\sigma }}\leq U(y)\leq \frac{C_{2}}{1+ \vert y \vert ^{3+2 \sigma }} $$

and

$$ \vert \partial _{y_{j}}U \vert \leq \frac{C}{1+ \vert y \vert ^{3+2\sigma }}, \quad j=1,2,3. $$

Moreover, the linearized operator$L_{0}:=(-\Delta )^{\sigma }+1-p|U|^{p-1}$is non-degenerate, i.e., its kernel is given by

$$ \operatorname{ker}L_{0}=\operatorname{span}\biggl\{ \frac{\partial U}{\partial y_{1}},\frac{\partial U}{ \partial y_{2}},\frac{\partial U}{\partial y_{3}}\biggr\} . $$

Throughout this paper, we write by $I_{\infty }: H^{\sigma }(\mathbb{R} ^{3}) \rightarrow \mathbb{R}$ the functional of (2.12), that is,

$$ I_{\infty }(u)=\frac{1}{2} \int _{\mathbb{R} ^{3}} \bigl( \bigl\vert (-\Delta )^{\frac{\sigma }{2}}u \bigr\vert ^{2}+u^{2} \bigr) - \frac{1}{p+1} \int _{\mathbb{R} ^{3}} \vert u \vert ^{p+1}, $$

and by $\mathcal{N}_{\infty }$ the corresponding Nehari manifold

$$ \mathcal{N}_{\infty }= \bigl\{ u\in H^{\sigma }\bigl(\mathbb{R} ^{3}\bigr)\setminus \{0\}: \Vert u \Vert _{H^{\sigma }}^{2}= \Vert u \Vert _{p+1}^{p+1} \bigr\} . $$

Furthermore, for any $u\in H^{\sigma }(\mathbb{R} ^{3})\setminus \{0\}$, there exists a unique $h_{u}>0$ such that $h_{u}u\in \mathcal{N}_{\infty }$, called the projection of u on $\mathcal{N}_{\infty }$, and

$$ I_{\infty }(h_{u}u)=\max _{h>0}I_{\infty }(h u). $$

(2.13)

On the other hand, we find that $\forall u\in \mathcal{N}_{\infty }$,

$$ I_{\infty }(u)= \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \Vert u \Vert ^{2}_{H^{\sigma }}= \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \Vert u \Vert _{p+1}^{p+1}, $$

(2.14)

and in what follows, we denote

$$ m_{\infty }:=\inf_{u\in \mathcal{N}_{\infty }}I_{\infty }(u) = \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \Vert U \Vert ^{2}_{H^{\sigma }}= \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \Vert U \Vert _{p+1}^{p+1}. $$

Remark 2.5

It is worth noticing that any sign-changing solution $u_{0}$ of (2.12) satisfies $I_{\infty }(u_{0})\geq 2m_{\infty }$. In fact, suppose that $u_{0}=u_{0}^{+}-u_{0}^{-}$ and $\langle I'_{ \infty }(u_{0}), u_{0}\rangle =\Vert u_{0}\Vert ^{2}_{H^{\sigma }}-\|u_{0}\| _{p+1}^{p+1}$. Then we have

$$ 0= \bigl\Vert u^{+}_{0} \bigr\Vert ^{2}_{H^{\sigma }}- \bigl\Vert u^{+}_{0} \bigr\Vert _{p+1}^{p+1}=\bigl\langle I'_{ \infty }(u_{0}), u^{+}_{0}\bigr\rangle =\bigl\langle I'_{\infty } \bigl(u^{+}_{0}\bigr), u ^{+}_{0} \bigr\rangle , $$

which implies $u_{0}^{+}\in \mathcal{N}_{\infty }$ and so $I_{\infty }(u_{0}^{+})\geq m_{\infty }$. Similarly, $I_{\infty }(u_{0}^{-}) \geq m_{\infty }$. Hence, $I_{\infty }(u_{0})\geq 2m_{\infty }$.

Next, we deal with the behavior of the Palais–Smale sequences of I. This study will be important for our research of the critical point of I.

Lemma 2.6

Let$u\in H^{\sigma }(\mathbb{R} ^{3})$, $t_{u}u$, $h_{u}u$be the projections of it on$\mathcal{N}$and$\mathcal{N}_{\infty }$respectively. Then

$$ h_{u}\leq t_{u}. $$

Proof

Since $t_{u}u\in \mathcal{N}$ and $h_{u}u\in \mathcal{N}_{\infty }$, we have

$$ t_{u}^{2} \Vert u \Vert _{H^{\sigma }}^{2}= -t_{u}^{2} \int _{\mathbb{R} ^{3}}W(y)u^{2}-t_{u}^{4} \int _{\mathbb{R} ^{3}}B(y)\phi _{u}u^{2}+t_{u}^{p+1} \int _{\mathbb{R} ^{3}}b(y) \vert u \vert ^{p+1} $$

and

$$ h_{u}^{2} \Vert u \Vert _{H^{\sigma }}^{2}=h_{u}^{p+1} \int _{\mathbb{R} ^{3}} \vert u \vert ^{p+1}. $$

So from $(A_{3})$, we find

$$ h_{u}^{p-1}=\frac{ \Vert u \Vert _{H^{\sigma }}^{2}}{ \Vert u \Vert _{p+1}^{p+1}}= \frac{t _{u}^{p-1} \int _{\mathbb{R} ^{3}}b(y) \vert u \vert ^{p+1}-t_{u}^{2} \int _{\mathbb{R} ^{3}}B(y)\phi _{u}u^{2}- \int _{\mathbb{R} ^{3}}W(y)u^{2}}{ \Vert u \Vert _{p+1}^{p+1}} \leq t_{u}^{p-1}, $$

and then our result follows. □

Lemma 2.7

Let$\{u_{n}\}$be a (PS) sequence ofIconstrained on$\mathcal{N}$, that is, $u_{n}\in \mathcal{N}$and (i) $I(u_{n})$is bounded, (ii) $\nabla I|_{\mathcal{N}}(u_{n})\rightarrow 0$in$H^{\sigma }(\mathbb{R} ^{3})$. Then there exist a solution$u^{*}$of (1.1), a number$k\in \mathbb{N}\cup \{0\}$, kfunctions$u^{1},\ldots ,u^{k}$of$H^{\sigma }(\mathbb{R} ^{3})$, and sequences of points$\{y_{n}^{j}\}$, $0\leq j\leq k$, such that

• (1) $|y_{n}^{j}|\rightarrow +\infty$, $|y_{n}^{i}-y_{n}^{j}|\rightarrow +\infty$, if$i\neq j$, $n\rightarrow +\infty$;

• (2) $u_{n}-\sum_{j=1}^{k}u^{j}(\cdot -y_{n}^{j})\rightarrow u^{*}$in$H^{\sigma }(\mathbb{R} ^{3})$;

• (3) $I(u_{n})\rightarrow I(u^{*})+\sum_{j=1}^{k}I_{\infty }(u^{j})$;

• (4) $u^{j}$are nontrivial weak solutions of (2.12). Here, we must emphasize that in the case$k=0$, the above holds without$u^{j}$.

Proof

First, since $I(u_{n})$ is bounded, using (2.10), one has

$$ I(u_{n})\geq \biggl(\frac{1}{2}-\frac{1}{p+1} \biggr) \Vert u_{n} \Vert ^{2}_{H^{ \sigma }}, $$

which tells that $\{u_{n}\}$ is bounded in $H^{\sigma }(\mathbb{R} ^{3})$. Now we claim that

$$ \nabla I(u_{n})\rightarrow 0\quad \text{in } H^{\sigma }\bigl(\mathbb{R} ^{3}\bigr). $$

(2.15)

In fact, from the assumption, we find

$$ o(1)=\nabla I|_{\mathcal{N}}(u_{n})=\nabla I(u_{n})-\lambda _{n}\nabla G(u_{n}), $$

(2.16)

where $\lambda _{n}\in \mathbb{R}$ and G can be seen in (2.11). So, by (2.16), we get

$$ o(1)=\bigl\langle \nabla I(u_{n}),u_{n} \bigr\rangle -\lambda _{n}\bigl\langle \nabla G(u_{n}),u_{n} \bigr\rangle . $$

(2.17)

Being $\langle \nabla I(u_{n}),u_{n}\rangle =0$ and $\langle \nabla G(u _{n}),u_{n}\rangle <0$ from (2.11), it follows from (2.17) that $\lambda _{n}\rightarrow 0$ as $n\rightarrow +\infty$. So, by the boundedness of $\nabla G(u_{n})$, we have $\lambda _{n}\nabla G(u_{n})=o(1)$ and then the claim holds by applying (2.16).

On the other hand, since $u_{n}$ is bounded in $H^{\sigma }(\mathbb{R} ^{3})$, there is $u^{*}\in H^{\sigma }(\mathbb{R} ^{3})$ such that, up to a subsequence, $u_{n}\rightharpoonup u^{*}$ in $H^{\sigma }(\mathbb{R} ^{3})$ and in $L^{p+1}(\mathbb{R} ^{3})$, and $u_{n}\rightarrow u^{*}$ a.e. in $\mathbb{R} ^{3}$. So, applying Lemma 2.2 and (2.15), we get that $u^{*}$ is a weak solution of (1.1).

If $u_{n}\rightarrow u^{*}$ in $H^{\sigma }(\mathbb{R} ^{3})$, we are done. Otherwise, we assume $z_{n}^{1}(y)=u_{n}(y)-u^{*}(y)$ and proceed as done in [8], our desired results follow. □

3 Proof of the main result

To prove our main theorem, we first give some important results.

Proposition 3.1

We have$\inf_{\mathcal{N}}I=m_{\infty }$and the infimum is not achieved.

Proof

First we write $m:= \inf_{\mathcal{N}}I$ and by Lemma 2.3, $m>0$. Now let us show that $m\geq m_{\infty }$. For all $u\in \mathcal{N}_{\infty }$, by the assumptions on $B(y)$, $W(y)$, $\beta (y)$, and (2.7), we find

$$\begin{aligned} I_{\infty }(u) =&\frac{1}{2} \int _{\mathbb{R} ^{3}} \bigl( \bigl\vert (-\Delta )^{\frac{\sigma }{2}}u \bigr\vert ^{2}+u^{2} \bigr) - \frac{1}{p+1} \int _{\mathbb{R} ^{3}} \vert u \vert ^{p+1} \\ \leq &\frac{1}{2} \int _{\mathbb{R} ^{3}} \bigl( \bigl\vert (-\Delta )^{\frac{\sigma }{2}}u \bigr\vert ^{2}+A(y)u^{2} \bigr)+\frac{1}{4} \int _{\mathbb{R} ^{3}}B(y)\phi _{u}u^{2} - \frac{1}{p+1} \int _{\mathbb{R} ^{3}}b(y) \vert u \vert ^{p+1} \\ =&I(u)\leq I(t_{u}u), \end{aligned}$$

from which, considering that $\mathcal{N}_{\infty }$ and $\mathcal{N}$ are diffeomorphic to sphere of $H^{\sigma }(\mathbb{R} ^{3})$, we find

$$ m_{\infty }=\inf_{u\in \mathcal{N}_{\infty }}I_{\infty }(u)\leq \inf _{u\in \mathcal{N}_{\infty }}I(t_{u}u) =\inf_{v\in \mathcal{N}}I(v)=m. $$

Next, we will prove the opposite side $m\leq m_{\infty }$. To do this, take $u_{n}=t_{n}U_{n}$, where $U_{n}=U(y-z_{n})$, $t_{n}=t_{U_{n}}$, and $\{z_{n}\}$ is a sequence of points in $\mathbb{R} ^{3}$ such that $|z_{n}|\rightarrow \infty$ as $n\rightarrow \infty$. Now we claim that

$$ \lim_{n\rightarrow \infty }I(u_{n})=m_{\infty }. $$

(3.1)

In fact, since $U_{n}$ is bounded and weakly converges to zero in $H^{\sigma }(\mathbb{R} ^{3})$ and from Lemma 2.2, we find

$$ \lim_{n\rightarrow \infty } \int _{\mathbb{R} ^{3}}B(y)\phi _{U_{n}}U_{n}^{2}=0. $$

(3.2)

Using condition $(A_{1})$, we can get that

$$ \lim_{n\rightarrow \infty } \int _{\mathbb{R} ^{3}}W(y)U_{n}^{2}=0. $$

(3.3)

Thus, by (2.4), in order to prove (3.1), we just need to show that $t_{n}\rightarrow 1$ as $n\rightarrow \infty$. To this end, being $t_{n}U_{n}\in \mathcal{N}$, we obtain that

$$ \Vert U \Vert _{H^{\sigma }}^{2}= \Vert U_{n} \Vert _{H^{\sigma }}^{2}=- \int _{\mathbb{R} ^{3}}W(y)U_{n}^{2}(y)-t_{n}^{2} \int _{\mathbb{R} ^{3}}B(y)\phi _{U_{n}}U_{n}^{2} +t_{n}^{p-1} \int _{\mathbb{R} ^{3}}b(y) \vert U_{n} \vert ^{p+1}. $$

(3.4)

Noting that

$$\begin{aligned} \lim_{n\rightarrow \infty } \int _{\mathbb{R} ^{3}}b(y) \vert U_{n} \vert ^{p+1}=\lim_{n\rightarrow \infty } \int _{\mathbb{R} ^{3}} \bigl(1-\beta (y) \bigr) \vert U_{n} \vert ^{p+1} = \int _{\mathbb{R} ^{3}} \vert U \vert ^{p+1}, \end{aligned}$$

and

$$ \Vert U \Vert _{H^{\sigma }}^{2}= \Vert U \Vert _{p+1}^{p+1}. $$

Thus, from (3.2)–(3.4), we find that $\lim_{n\rightarrow \infty }t_{n}=1$ and then $m\leq \lim_{n\rightarrow \infty }I(u_{n})=m_{\infty }$.

Finally, to finish our proof, we assume by contradiction that there exists $u_{*}\in \mathcal{N}$ such that $I(u_{*})=m=m_{\infty }$. Letting $h_{u_{*}}>0$ such that $h_{u_{*}}u_{*}\in \mathcal{N}_{ \infty }$, then using Lemma 2.6, one has

$$\begin{aligned} m_{\infty } \leq &I_{\infty }(h_{u_{*}}u_{*}) \\ =& \biggl(\frac{1}{2}- \frac{1}{p+1} \biggr) \Vert h_{u_{*}}u_{*} \Vert ^{2}_{H^{\sigma }} \\ \leq & \biggl(\frac{1}{2}-\frac{1}{p+1} \biggr)h_{u_{*}}^{2} \int _{\mathbb{R} ^{3}} \bigl( \bigl\vert (-\Delta )^{\frac{\sigma }{2}}u_{*} \bigr\vert ^{2}+A(y) {u_{*}}^{2} \bigr) \\ &{} + \biggl(\frac{1}{4}-\frac{1}{p+1} \biggr)h_{u_{*}}^{4} \int _{\mathbb{R} ^{3}}B(y)\phi _{u_{*}}(y)u_{*}^{2} \\ \leq & \biggl(\frac{1}{2}-\frac{1}{p+1} \biggr) \int _{\mathbb{R} ^{3}} \bigl( \bigl\vert (-\Delta )^{\frac{\sigma }{2}}u_{*} \bigr\vert ^{2}+A(y) {u_{*}}^{2} \bigr) + \biggl(\frac{1}{4}-\frac{1}{p+1} \biggr) \int _{\mathbb{R} ^{3}}B(y)\phi _{u_{*}}(y)u_{*}^{2} \\ =&I(u_{*})=m=m_{\infty }, \end{aligned}$$

which implies $h_{u_{*}}=1$ and

$$ \int _{\mathbb{R} ^{3}}B(y)\phi _{u_{*}}(y)u_{*}^{2}=0. $$

(3.5)

Thus, $u_{*}\in \mathcal{N}_{\infty }$ and $I_{\infty }(u_{*})=m_{ \infty }$. But it follows from Proposition 2.4 that up to translations, U is unique and $m_{\infty }=I_{\infty }(U)$. So, by the uniqueness of the family achieving $m_{\infty }$, we infer that

$$ u_{*}=U(y-z_{0}), \quad \forall y\in \mathbb{R} ^{3} $$

for some $z_{0}\in \mathbb{R} ^{3}$. This contradicts (3.5) and our result has been proved. □

Proposition 3.2

The functionalIconstrained on$\mathcal{N}$satisfies a$(PS)_{d}$sequence for all$d\in (m_{\infty },2m_{\infty })$. Moreover, if$\{u_{n}\}$is a$(PS)_{m_{\infty }}$sequence, then, up to a subsequence, we have

$$ u_{n}=U(y-z_{n})+o(1) $$

with$z_{n}\in \mathbb{R} ^{3}$, $|z_{n}|\rightarrow +\infty$.

Proof

Let $\{u_{n}\}$ be a $(PS)_{d}$ sequence of I constrained on $\mathcal{N}$. Then it follows from (3) of Lemma 2.7 that

$$ d=\lim_{n\rightarrow \infty }I(u_{n})= I \bigl(u^{*}\bigr)+\sum_{j=1}^{k}I_{ \infty } \bigl(u^{j}\bigr), $$

(3.6)

where $u_{n}\rightharpoonup u^{*}$ and $I_{\infty }(u^{j})\geq m_{ \infty }$. Since $m_{\infty }< d<2m_{\infty }$, from (3.6), we can infer that $k<2$. Now if $k=1$, there are the following two possibilities:

• (i) $u^{*}\neq 0$, being $I(u^{*})>m_{\infty }$, we see

  $$ 2m_{\infty }>d=\lim_{n\rightarrow \infty }I(u_{n})= I \bigl(u^{*}\bigr)+I_{ \infty }\bigl(u^{1} \bigr)>2m_{\infty }, $$

  this is a contradiction.

• (ii) $u^{*}=0$, then $I(u^{*})=0$ and

  $$ d=\lim_{n\rightarrow \infty }I(u_{n})=I_{\infty } \bigl(u^{1}\bigr)\in (m_{ \infty },2m_{\infty }), $$

  this is impossible since either $I_{\infty }(u^{1})=m_{\infty }$ or $I_{\infty }(u^{1})\geq 2m_{\infty }$ if $u^{1}$ is changing sign.

Therefore, from the above, we can deduce $k=0$. □

From Proposition 3.1, we know that (1.1) can not be solved by minimization. So we will prove the existence of a higher level solution by the barycenter technique, which has been successfully used in the case of scalar filed equation (see [3]). Let us now recall the definition of barycenter of a function $u\in H^{\sigma }(\mathbb{R} ^{3})\setminus \{0\}$, which was also introduced in [6, 7]. Set

$$ \alpha _{u}(y)=\frac{1}{ \vert B(0,1) \vert } \int _{B(y,1)} \bigl\vert u(x) \bigr\vert \,dx, $$

and then $\alpha (u)$ is bounded and continuous. So the function

$$ \hat{\alpha }(y)=\biggl[\alpha _{u}(y)-\frac{1}{2}\max _{y\in \mathbb{R} ^{3}} \alpha _{u}(y)\biggr]^{+} $$

is well defined, continuous and has compact support. Thus, we can define $\gamma :H^{\sigma }(\mathbb{R} ^{3})\setminus \{0\}\rightarrow \mathbb{R} ^{3}$ as

$$ \gamma (u)=\frac{1}{ \Vert \hat{\alpha } \Vert _{1}} \int _{\mathbb{R} ^{3}}\hat{\alpha }(y)y\,dy. $$

Then $\gamma (u)$ is well defined and the following properties hold:

• 1. γ is continuous in $H^{\sigma }(\mathbb{R} ^{3})\setminus \{0\}$.

• 2. If u is a radial function, $\gamma (u)=0$.

• 3. For all $t\neq 0$ and for all $u\in H^{\sigma }(\mathbb{R} ^{3})\setminus \{0\}$, $\gamma (tu)=\gamma (u)$.

• 4. Given $z\in \mathbb{R} ^{3}$ and taking $u_{z}(y)=u(y-z)$, then $\gamma (u_{z})=\gamma (u)+z$.

Now we define

$$ a_{0}=\inf \bigl\{ I(u): u\in \mathcal{N},\gamma (u)=0\bigr\} . $$

(3.7)

Then we are led to the following lemma.

Lemma 3.3

$$ a_{0}>m_{\infty }. $$

Proof

First, it is obvious to see that $a_{0}\geq m_{\infty }$. Next we argue by contradiction, suppose that $a_{0}=m_{\infty }$. Then there exists $\{u_{n}\}$ such that $u_{n}\in \mathcal{N}$, $\gamma (u_{n})=0$ and $I(u_{n})\rightarrow m_{\infty }=m$. Moreover, by Ekeland’s variational principle (see [15] or [26]), there is another sequence $\tilde{u}_{n}\in \mathcal{N}$ such that $I( \tilde{u}_{n})\rightarrow m_{\infty }$, $\nabla I|_{\mathcal{N}}( \tilde{u}_{n})\rightarrow 0$ and $\|\tilde{u}_{n}-u_{n}\|_{H^{\sigma }}\rightarrow 0$. Thus, by the properties of $\gamma (u)$, we have $\gamma (\tilde{u}_{n})=o(1)$.

On the other hand, by Proposition 3.2, $\tilde{u}_{n}(y)=U(y-z _{n})+o(1)$, where $\{z_{n}\}\subset \mathbb{R} ^{3}$ and $|z_{n}|\rightarrow +\infty$. So we get

$$ o(1)=\gamma (\tilde{u}_{n})=\gamma \bigl(U(y-z_{n}) \bigr)+o(1)=z_{n}+o(1), $$

which implies a contradiction. □

Now we define a set

$$ \mathcal{S}=\bigl\{ x\in \mathbb{R} ^{3}: \vert x-e_{1} \vert =2\bigr\} , \quad \text{where } e_{1}=(1,0,0) $$

and a function

$$ \bar{\varphi }_{\rho }[x,\tau ](y)=(1-\tau )U(y-\rho e_{1})+\tau U(y- \rho x),\quad y\in \mathbb{R} ^{3}, \rho >0. $$

Furthermore, we denote by $\varphi _{\rho }[x,\tau ]$ the projection of $\bar{\varphi }_{\rho }[x,\tau ]$ on $\mathcal{N}$ and by $\varphi _{\infty ,\rho }[x,\tau ]$ the projection of $\bar{\varphi } _{\rho }[x,\tau ]$ on $\mathcal{N}_{\infty }$. Thus, from the definitions of $\varphi _{\rho }[x,\tau ]$ and $\varphi _{\infty , \rho }[x,\tau ]$, there exist positive numbers $t_{\rho ,x,\tau }:=t _{\bar{\varphi }_{\rho }[x,\tau ]}$ and $h_{\rho ,x,\tau }:=h_{\bar{ \varphi }_{\rho }[x,\tau ]}$ such that

$$ \varphi _{\rho }[x,\tau ]=t_{\rho ,x,\tau }\bar{ \varphi }_{\rho }[x, \tau ],\qquad \varphi _{\infty ,\rho }[x,\tau ]=h_{\rho ,x,\tau }\bar{\varphi }_{ \rho }[x,\tau ]. $$

(3.8)

Then we have the following.

Proposition 3.4

• (i) $\gamma (\varphi _{\rho }[x,1])=\rho x$for all$\rho >0$and$x\in \mathcal{S}$.

• (ii) For every$\rho >0$, there exists$(\bar{x},\bar{\tau }) \in \mathcal{S}\times (0,1)$such that$\gamma (\varphi _{\rho }[ \bar{x},\bar{\tau }])=0$.

Proof

(i) Note that $\bar{\varphi }_{\rho }[x,1](y)=U(y-\rho x)$. Then, by the properties of $\gamma (u)$, we find

$$ \gamma \bigl(\varphi _{\rho }[x,1]\bigr)=\gamma \bigl(\bar{\varphi } _{\rho }[x,1]\bigr)=\gamma \bigl(U(y-\rho x)\bigr)=\gamma \bigl(U(y) \bigr)+\rho x=\rho x. $$

(ii) For all $\rho >0$, define the map $\mathcal{F}_{\rho }: \mathcal{S}\times [0,1]\rightarrow \mathbb{R} ^{3}$ by $\mathcal{F}_{\rho }(x, \tau )=(1-\tau )\rho e_{1}+\tau \rho x$. Hence, using (i) and the invariance of topological degree by homotopy, we can deduce that

$$ 0\neq d\bigl(\mathcal{F}_{\rho },\mathcal{S}\times [0,1),0\bigr)=d \bigl(\gamma \circ \varphi _{\rho },\mathcal{S}\times [0,1),0\bigr) $$

and then $\gamma \circ \varphi _{\rho }[x,\tau ]=0$ has a solution $(\bar{x},\bar{\tau })\in \mathcal{S}\times [0,1)$. □

Proposition 3.5

There exists$\rho _{a}\in \mathbb{R} ^{+}\backslash \{0\}$such that, for all$\rho >\rho _{a}$,

$$ \mathfrak{B}:=\max \bigl\{ I\bigl(\varphi _{\rho }[x,1]\bigr),x\in \mathcal{S}\bigr\} < a_{0}. $$

Proof

Being $\varphi _{\rho }[x,1](y)=t_{\rho ,x,1}\bar{\varphi }_{\rho }[x,1](y)$ and $\bar{\varphi }_{\rho }[x,1](y)=U(y-\rho x)$, with the same argument as the proof of (3.1), we can prove our result. □

Now we introduce a lemma, which can be found in [14].

Lemma 3.6

For any constant$0<\kappa <N-2\sigma$, there is a constant$C>0$such that

$$ \int _{\mathbb{R} ^{N}}\frac{1}{ \vert x \vert ^{N-2\sigma }}\frac{1}{(1+ \vert y-x \vert )^{2\sigma + \kappa }}\,dx\leq \frac{C}{(1+ \vert y \vert )^{\kappa }}. $$

From the above lemma, we have the following.

Lemma 3.7

There exists$C>0$such that

$$ \int _{\mathbb{R} ^{3}}B(y)\phi _{U(\cdot -\rho \xi )}(y)U^{2}(y- \rho \xi )\,dy \leq C\rho ^{-(12+8\sigma )} $$

for all$\xi \in \mathbb{R} ^{3}$with$|\xi |\geq 1$and$\rho >0$.

Proof

Without loss of generality, we can assume $|\xi |=1$ and fix $\xi =e_{1}$. Letting q such that $((\frac{1}{2}-q)\rho )^{\epsilon }=((\frac{1}{2}+q)\rho )^{6+4\sigma }$ and $\rho >\frac{\bar{R}}{ \frac{1}{2}-q}$, then using condition $(A_{2})$, Proposition 2.4, and Lemma 3.6, we have

$$\begin{aligned} \phi _{U(\cdot -\rho e_{1})}(y) =& \int _{\mathbb{R} ^{3}}\frac{1}{ \vert y-x \vert ^{3-2\sigma }}B(x)U^{2}(x-\rho e_{1})\,dx \\ =& \biggl( \int _{\{x_{1}< (\frac{1}{2}-q)\rho \}}+ \int _{\{x_{1}>(\frac{1}{2}-q)\rho \}} \biggr) \frac{1}{ \vert y-x \vert ^{3-2\sigma }}B(x)U^{2}(x- \rho e_{1})\,dx \\ \leq &\frac{C}{ \vert (\frac{1}{2}+q)\rho \vert ^{2(3+2\sigma )}} \int _{\{x_{1}< (\frac{1}{2}-q)\rho \}}\frac{B(x)}{ \vert y-x \vert ^{3-2\sigma }} \\ & {} +\frac{C}{ \vert (\frac{1}{2}-q)\rho \vert ^{\epsilon }} \int _{\{x_{1}>(\frac{1}{2}-q)\rho \}}\frac{U^{2}(x-\rho e_{1})}{ \vert y-x \vert ^{3-2 \sigma }} \\ \leq &\frac{C}{ \vert (\frac{1}{2}+q)\rho \vert ^{2(3+2\sigma )}} \biggl( \int _{\{x_{1}< (\frac{1}{2}-q)\rho \}}\frac{B(x)}{ \vert y-x \vert ^{3-2\sigma }} + \int _{\{x_{1}>(\frac{1}{2}-q)\rho \}}\frac{U^{2}(x- \rho e_{1})}{ \vert y-x \vert ^{3-2\sigma }} \biggr) \\ \leq &\frac{C}{ \vert (\frac{1}{2}+q)\rho \vert ^{2(3+2\sigma )}}, \end{aligned}$$

where we used that

$$\begin{aligned} \int _{\{x_{1}< (\frac{1}{2}-q)\rho \}}\frac{B(x)}{ \vert y-x \vert ^{3-2\sigma }} \leq & \biggl( \int _{\{ \vert x \vert < \bar{R}\}}+ \int _{\{ \vert x \vert >\bar{R}\}} \biggr)\frac{B(x)}{ \vert y-x \vert ^{3-2\sigma }} \\ \leq &C \biggl( \int _{\{ \vert x-y \vert < \frac{\bar{R}}{2}\}\cap \{ \vert x \vert < \bar{R}\}} + \int _{\{ \vert x-y \vert \geq \frac{\bar{R}}{2}\}\cap \{ \vert x \vert < \bar{R}\}} \biggr)\frac{1}{ \vert y-x \vert ^{3-2 \sigma }} \\ & {} +C \int _{\mathbb{R} ^{3}}\frac{1}{(1+ \vert x \vert )^{\epsilon } \vert x-y \vert ^{3-2\sigma }} \\ \leq & C+\frac{C}{(1+ \vert y \vert )^{\epsilon -2\sigma }}\leq C, \end{aligned}$$

and similarly,

$$ \int _{\{x_{1}>(\frac{1}{2}-q)\rho \}}\frac{U^{2}(x-\rho e_{1})}{ \vert y-x \vert ^{3-2 \sigma }}\leq C. $$

As a result,

$$\begin{aligned} \int _{\mathbb{R} ^{3}}B(y)\phi _{U(\cdot -\rho \xi )}(y)U^{2}(y- \rho \xi )\,dy \leq & \frac{C}{ \vert (\frac{1}{2}+q)\rho \vert ^{2(3+2\sigma )}} \int _{\mathbb{R} ^{3}}B(y)U^{2}(y-\rho \xi )\,dy \\ \leq &\frac{C}{ \vert (\frac{1}{2}+q)\rho \vert ^{2(3+2\sigma )}} \biggl( \int _{\{y_{1}< (\frac{1}{2}-q)\rho \}} B(y)U^{2}(y-\rho \xi )\,dy \\ &{} + \int _{\{y_{1}>(\frac{1}{2}-q)\rho \}}B(y)U^{2}(y-\rho \xi )\,dy \biggr) \\ \leq &\frac{C}{ \vert (\frac{1}{2}+q)\rho \vert ^{2(6+4\sigma )}}=\frac{C}{(( \frac{1}{2}+q)\rho )^{12+8\sigma }}. \end{aligned}$$

 □

Lemma 3.8

Let$t_{\rho ,x,\tau }$and$h_{\rho ,x,\tau }$be given in (3.8). There exists a constant$C>0$such that

$$ t_{\rho ,x,\tau }< C, \quad \forall \rho >0, \forall (x,\tau ) \in \mathcal{S}\times (0,1). $$

(3.9)

Furthermore, $t_{\rho ,x,\tau }=h_{\rho ,x,\tau }+o(\rho ^{-2\sigma })$.

Proof

First, it follows from (2.5) that

$$\begin{aligned} t^{p-1}_{\rho ,x,\tau } =& \frac{ \Vert \bar{\varphi }_{\rho }[x,\tau ] \Vert ^{2}_{H^{\sigma }}}{ \Vert \bar{ \varphi }_{\rho }[x,\tau ] \Vert _{p+1}^{p+1} - \int _{\mathbb{R} ^{3}}\beta \vert \bar{\varphi }_{\rho }[x,\tau ] \vert ^{p+1}} \\ &{} + \frac{t^{2}_{\rho ,x,\tau } \int _{\mathbb{R} ^{3}}B(y)\phi _{\bar{\varphi }_{\rho }[x,\tau ]}\bar{\varphi }^{2}_{\rho }[x,\tau ] + \int _{\mathbb{R} ^{3}}W(y)\bar{\varphi }^{2}_{\rho }[x,\tau ]}{ \Vert \bar{\varphi }_{\rho }[x,\tau ] \Vert _{p+1}^{p+1}- \int _{\mathbb{R} ^{3}}\beta \vert \bar{\varphi }_{\rho }[x,\tau ] \vert ^{p+1}}. \end{aligned}$$

(3.10)

Note that

$$ \bigl\Vert \bar{\varphi }_{\rho }[x,\tau ] \bigr\Vert _{H^{\sigma }}\leq \bigl\Vert U(y-\rho e _{1}) \bigr\Vert _{H^{\sigma }}+ \bigl\Vert U(y-\rho x) \bigr\Vert _{H^{\sigma }}= 2 \Vert U \Vert _{H^{ \sigma }} $$

(3.11)

and

$$ \bigl\Vert \bar{\varphi }_{\rho }[x,\tau ] \bigr\Vert _{p+1}^{p+1}\geq \bigl((1-\tau )^{p+1}+ \tau ^{p+1} \bigr) \int _{\mathbb{R} ^{3}}U^{p+1}>\frac{1}{2^{p}} \int _{\mathbb{R} ^{3}}U^{p+1}>0. $$

(3.12)

So (3.9) comes directly from (3.10)–(3.12) and since $t_{\rho ,x,\tau }< C$, from Lemma 3.7, one has

$$ t^{2}_{\rho ,x,\tau } \int _{\mathbb{R} ^{3}}B(y)\phi _{\bar{\varphi }_{\rho }[x,\tau ]}\bar{\varphi }^{2}_{\rho }[x,\tau ]=o\bigl(\rho ^{-2\sigma -3}\bigr). $$

(3.13)

On the other hand, by assumptions $(A_{1})$, $(A_{3})$ and the decay property of U, we are led to

$$ \int _{\mathbb{R} ^{3}}W(y)\bar{\varphi }^{2}_{\rho }[x, \tau ] \leq C \int _{\mathbb{R} ^{3}}W(y) \bigl(U^{2}(y-\rho e_{1})+U^{2}(y-\rho x) \bigr)=o\bigl( \rho ^{-2\sigma -3}\bigr), $$

(3.14)

where we used that

$$\begin{aligned} \int _{\mathbb{R} ^{3}}W(y)U^{2}(y-\rho e_{1}) =& \biggl( \int _{\{y_{1}< \frac{1}{2}\rho \}}+ \int _{\{y_{1}>\frac{1}{2}\rho \}} \biggr)W(y)U^{2}(y-\rho e_{1}) \\ \leq &C \int _{\{y_{1}< \frac{1}{2}\rho \}}\frac{W(y)}{ \vert y-\rho e_{1} \vert ^{2(3+2 \sigma )}} +C \int _{\{y_{1}>\frac{1}{2}\rho \}}\frac{U^{2}(y-\rho e_{1})}{ \vert y \vert ^{3+3 \sigma }} \\ \leq &\frac{C}{\rho ^{6+4\sigma }}+\frac{C}{\rho ^{3+3\sigma }}=o\bigl( \rho ^{-2\sigma -3}\bigr), \end{aligned}$$

and similarly,

$$ \int _{\mathbb{R} ^{3}}W(y)U^{2}(y-\rho x)=o\bigl(\rho ^{-2\sigma -3}\bigr). $$

With the same argument as above, we can infer that

$$ \int _{\mathbb{R} ^{3}}\beta \bigl\vert \bar{\varphi }_{\rho }[x,\tau ] \bigr\vert ^{p+1}=o\bigl( \rho ^{-2\sigma -3}\bigr). $$

(3.15)

Combining (3.11) and (3.12), we have

$$ 0< C_{1}\leq \bigl\Vert \bar{\varphi }_{\rho }[x,\tau ] \bigr\Vert _{p+1}\leq 2 \Vert U \Vert _{p+1} $$

and

$$ 0< \frac{1}{4} \Vert U \Vert _{H^{\sigma }}\leq \bigl\Vert \bar{\varphi }_{\rho }[x, \tau ] \bigr\Vert _{H^{\sigma }}\leq 2 \Vert U \Vert _{H^{\sigma }}. $$

Therefore,

$$ \frac{ \Vert \bar{\varphi }_{\rho }[x,\tau ] \Vert ^{2}_{H^{\sigma }}}{ \Vert \bar{ \varphi }_{\rho }[x,\tau ] \Vert _{p+1}^{p+1} - \int _{\mathbb{R} ^{3}}\beta \vert \bar{\varphi }_{\rho }[x,\tau ] \vert ^{p+1}}= \frac{ \Vert \bar{\varphi }_{\rho }[x,\tau ] \Vert ^{2}_{H^{\sigma }}}{ \Vert \bar{ \varphi }_{\rho }[x,\tau ] \Vert _{p+1}^{p+1}} +o\bigl(\rho ^{-2\sigma -3}\bigr). $$

(3.16)

Inserting (3.14), (3.15), and (3.16) into (3.10), we deduce

$$ t^{p-1}_{\rho ,x,\tau }= \frac{ \Vert \bar{\varphi }_{\rho }[x,\tau ] \Vert ^{2}_{H^{\sigma }}}{ \Vert \bar{ \varphi }_{\rho }[x,\tau ] \Vert _{p+1}^{p+1}} +o\bigl(\rho ^{-2\sigma -3}\bigr)=h ^{p-1}_{\rho ,x,\tau }+o\bigl(\rho ^{-2\sigma -3}\bigr). $$

 □

Proposition 3.9

There is a constant$\rho _{\infty }>0$such that, for all$\rho >\rho _{\infty }$,

$$ \mathcal{A}:=\max \bigl\{ I\bigl(\varphi _{\rho }[x,\tau ]\bigr):(x,\tau )\in \mathcal{S}\times [0,1]\bigr\} < 2m_{\infty }. $$

Proof

First, by using (2.4), we can obtain that

$$\begin{aligned} I\bigl(\varphi _{\rho }[x,\tau ]\bigr) =& \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \bigl\Vert t_{\rho ,x,\tau }\bar{\varphi }_{\rho }[x,\tau ] \bigr\Vert ^{2}_{H^{\sigma }} + \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr)t^{2}_{\rho ,x,\tau } \int _{\mathbb{R} ^{3}}W(y)\bar{\varphi }^{2}_{\rho }[x, \tau ] \\ &{} + \biggl(\frac{1}{4}-\frac{1}{p+1} \biggr)t^{4}_{\rho ,x,\tau } \int _{\mathbb{R} ^{3}}B(y)\phi _{\bar{\varphi }_{\rho }[x,\tau ]}\bar{ \varphi }^{2}_{\rho }[x,\tau ]. \end{aligned}$$

Then from Lemma 3.8, (3.13), and (3.14), for any $(x,\tau )\in \mathcal{S}\times [0,1]$, we get that

$$\begin{aligned} I\bigl(\varphi _{\rho }[x,\tau ]\bigr) =& \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \bigl\Vert t_{\rho ,x,\tau }\bar{\varphi }_{\rho }[x,\tau ] \bigr\Vert ^{2}_{H^{\sigma }} +o\bigl(\rho ^{-2\sigma -3}\bigr) \\ =& \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \bigl\Vert h_{\rho ,x,\tau }\bar{\varphi }_{\rho }[x,\tau ] \bigr\Vert ^{2}_{H^{\sigma }} \\ &{}+ \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \bigl(t^{2}_{\rho ,x,\tau }-h^{2}_{\rho ,x,\tau } \bigr) \bigl\Vert \bar{ \varphi }_{\rho }[x,\tau ] \bigr\Vert ^{2}_{H^{\sigma }} \\ & {} +o\bigl(\rho ^{-2\sigma -3}\bigr) \\ =& \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \bigl\Vert h_{\rho ,x,\tau }\bar{\varphi }_{\rho }[x,\tau ] \bigr\Vert ^{2}_{H^{\sigma }}+o\bigl(\rho ^{-2\sigma -3}\bigr) \\ =&I_{\infty }\bigl(\varphi _{\infty ,\rho }[x,\tau ]\bigr)+o\bigl(\rho ^{-2\sigma -3}\bigr). \end{aligned}$$

Next, we will estimate $I_{\infty }(\varphi _{\infty ,\rho }[x,\tau ])$. Observe that

$$\begin{aligned} I_{\infty }\bigl(\varphi _{\infty ,\rho }[x, \tau ]\bigr) =& \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \bigl\Vert h_{\rho ,x,\tau }\bar{\varphi }_{\rho }[x,\tau ] \bigr\Vert ^{2}_{H^{\sigma }} \\ =& \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \biggl( \frac{ \Vert \bar{\varphi }_{\rho }[x,\tau ] \Vert ^{2} _{H^{\sigma }}}{ \Vert \bar{\varphi }_{\rho }[x,\tau ] \Vert ^{2}_{p+1}} \biggr)^{ \frac{p+1}{p-1}}. \end{aligned}$$

(3.17)

By direct computation, we have

$$\begin{aligned} \bigl\Vert \bar{\varphi }_{\rho }[x,\tau ] \bigr\Vert ^{2}_{H^{\sigma }} =& \bigl\Vert (1-\tau )U(y- \rho e_{1})+\tau U(y-\rho x) \bigr\Vert ^{2}_{H^{\sigma }} \\ =&\bigl[\tau ^{2}+(1-\tau )^{2}\bigr] \Vert U \Vert ^{2}_{H^{\sigma }}+2\tau (1-\tau ) \int _{\mathbb{R} ^{3}}U^{p}(y-\rho e_{1})U(y- \rho x), \end{aligned}$$

(3.18)

and there exists $C_{1}>0$ such that

$$\begin{aligned}& \int _{\mathbb{R} ^{3}}U^{p}(y-\rho e_{1})U(y- \rho x) \\& \quad = \biggl( \int _{\{|y|< \frac{\rho |x-e_{1}|}{2}\}}+ \int _{\{|y|\geq \frac{\rho |x-e_{1}|}{2}\}} \biggr) U^{p}(y+\rho x- \rho e_{1})U(y) \\& \quad \leq \frac{C}{\rho ^{p(3+2\sigma )}}+ \frac{C}{\rho ^{3+2\sigma }}=C_{1}\rho ^{-2\sigma -3}+o\bigl(\rho ^{-2\sigma -3}\bigr). \end{aligned}$$

(3.19)

So,

$$ \bigl\Vert \bar{\varphi }_{\rho }[x,\tau ] \bigr\Vert ^{2}_{H^{\sigma }}=\bigl[\tau ^{2}+(1- \tau )^{2}\bigr] \Vert U \Vert ^{2}_{H^{\sigma }}+2 \tau (1-\tau )C_{1}\rho ^{-2\sigma -3}+o\bigl(\rho ^{-2\sigma -3}\bigr). $$

(3.20)

On the other hand, since for all $a,b\in \mathbb{R} ^{+}$ and $p\geq 1$, one has

$$ (a+b)^{p+1}\geq a^{p+1}+b^{p+1}+p \bigl(a^{p}b+ab^{p}\bigr). $$

Hence using (3.19), we find

$$\begin{aligned} \bigl\Vert \bar{\varphi }_{\rho }[x,\tau ] \bigr\Vert ^{p+1}_{p+1} =& \int _{\mathbb{R} ^{3}} \bigl\vert (1-\tau )U(y-\rho e_{1})+\tau U(y-\rho x) \bigr\vert ^{p+1} \\ \geq& \bigl[\tau ^{p+1}+(1-\tau )^{p+1}\bigr] \Vert U \Vert _{p+1}^{p+1}+p\bigl[\tau (1-\tau )^{p}+(1- \tau )\tau ^{p}\bigr]C_{1}\rho ^{-2\sigma -3} \\ &{}+o\bigl(\rho ^{-2\sigma -3}\bigr), \end{aligned}$$

(3.21)

which and (3.20) imply that, for all $\tau \in [0,1]$ and $x\in \mathcal{S}$,

$$\begin{aligned}& \frac{ \Vert \bar{\varphi }_{\rho }[x,\tau ] \Vert ^{2}_{H^{\sigma }}}{ \Vert \bar{ \varphi }_{\rho }[x,\tau ] \Vert ^{2}_{p+1}} \\& \quad \leq \frac{[\tau ^{2}+(1-\tau )^{2}] \Vert U \Vert ^{2}_{H^{\sigma }}+2\tau (1- \tau )C_{1}\rho ^{-2\sigma -3}+o(\rho ^{-2\sigma -3})}{ \{[\tau ^{p+1}+(1- \tau )^{p+1}] \Vert U \Vert _{p+1}^{p+1}+p[\tau (1-\tau )^{p}+(1-\tau )\tau ^{p}]C _{1}\rho ^{-2\sigma -3}+o(\rho ^{-2\sigma -3}) \}^{\frac{2}{p+1}}} \\& \quad =\frac{[\tau ^{2}+(1-\tau )^{2}] \Vert U \Vert ^{2}_{H^{\sigma }}}{[\tau ^{p+1}+(1- \tau )^{p+1}]^{\frac{2}{p+1}} \Vert U \Vert _{p+1}^{2}} +\kappa (\tau ) \rho ^{-2\sigma -3}+o\bigl(\rho ^{-2\sigma -3}\bigr), \end{aligned}$$

where

$$\begin{aligned} \kappa (\tau )=\frac{2\tau (1-\tau )C_{1}}{[\tau ^{p+1}+(1-\tau )^{p+1}]^{ \frac{2}{p+1}} \Vert U \Vert _{p+1}^{2}} \biggl\{ 1-\frac{p}{p+1} \frac{\tau ^{2}+(1- \tau )^{2}}{\tau ^{p+1}+(1-\tau )^{p+1}}\bigl[\tau ^{p-1}+(1-\tau )^{p-1}\bigr] \biggr\} . \end{aligned}$$

Noting that $\kappa (\frac{1}{2})<0$, from (3.17), we can infer that, for all $x\in \mathcal{S}$ and $\tau \in \delta (\frac{1}{2})$,

$$\begin{aligned} I_{\infty }\bigl(\varphi _{\infty ,\rho }[x, \tau ]\bigr) \leq& \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \biggl[2^{\frac{p-1}{p+1}}\frac{ \Vert U \Vert ^{2}_{H^{\sigma }}}{ \Vert U \Vert ^{p+1}_{p+1}} \biggr]^{\frac{p+1}{p-1}}-\bar{C} \rho ^{-2\sigma -3}+o\bigl(\rho ^{-2\sigma -3}\bigr) \\ =&2 \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \Vert U \Vert ^{p+1}_{p+1}-\bar{C}\rho ^{-2\sigma -3}+o\bigl( \rho ^{-2\sigma -3}\bigr) \\ =&2m_{\infty }-\bar{C}\rho ^{-2\sigma -3}+o\bigl(\rho ^{-2\sigma -3}\bigr), \end{aligned}$$

(3.22)

where $\delta (\frac{1}{2})$ is a neighborhood of $\frac{1}{2}$.

Furthermore, applying the same argument, we can prove that

$$\begin{aligned}& \lim_{\rho \rightarrow +\infty }\max \biggl\{ I_{\infty }\bigl( \varphi _{\infty ,\rho }[x,\tau ]\bigr):x\in \mathcal{S},\tau \in [0,1] \setminus \delta \biggl(\frac{1}{2}\biggr)\biggr\} \\& \quad =\max \biggl\{ \biggl( \frac{1}{2}- \frac{1}{p+1} \biggr) \biggl[\frac{\tau ^{2}+(1-\tau )^{2}}{\tau ^{p+1}+(1- \tau )^{p+1}} \biggr] ^{\frac{p+1}{p-1}}m_{\infty }, \tau \in [0,1]\setminus \delta \biggl(\frac{1}{2}\biggr) \biggr\} < 2m_{\infty }. \end{aligned}$$

This, combining with (3.22), completes our result. □

Now, we will prove the existence of a bound state solution of (1.1).

Proof of Theorem 1.1

Fix $\rho > \max \{\rho _{a},\rho _{\infty }\}$, where $\rho _{a}$, $\rho _{\infty }$ are given in Propositions 3.5 and 3.9 respectively. It follows from Proposition 3.1 that $m=m_{\infty }$ and m is not achieved. Thus we can not get our result by using minimization. However, we can prove that (2.2) has a bound state solution, whose energy can be higher than $m_{\infty }$. For any $c\in \mathbb{R}$, we let $I^{c}:=\{u\in \mathcal{N}: I(u)\leq c\}$. By Propositions 3.1, 3.5, 3.9, and Lemma 3.3, we have

$$ m_{\infty }< \mathfrak{B}< a_{0}< \mathcal{A}< 2m_{\infty }. $$

We end the proof by showing that there exists a number $c^{*}\in [a _{0},\mathcal{A}]$ which is a critical level of $I|_{\mathcal{N}}$. We use the contradiction argument. Assume that this is not the case. Then the Palais–Smale condition holds in $(m_{\infty }, 2m_{\infty })$ by Lemma 3.2. We can apply usual deformation arguments(see [26]) and assert the existence of a number $\vartheta >0$ and a continuous function $\eta : I^{\mathcal{A}}\rightarrow I^{a_{0}- \vartheta }$ such that $a_{0}-\vartheta >\mathfrak{B}$ and $\eta (u)=u$ for all $u\in I^{a_{0}-\vartheta }$. Thus we see

$$ 0\notin (\gamma \circ \eta \circ \varphi _{\rho }) \bigl(\mathcal{S}\times [0,1]\bigr). $$

(3.23)

On the other hand, since $\varphi _{\rho }(\mathcal{S}\times \{1\}) \subseteq I^{\mathfrak{B}}$, applying the invariance of topological degree by homotopy as in Proposition 3.4,

$$ 0\neq d\bigl(\mathcal{F}_{\rho },\mathcal{S}\times [0,1),0\bigr)=d \bigl(\gamma \circ \eta \circ \varphi _{\rho },\mathcal{S}\times [0,1 ),0\bigr). $$

Therefore there exists $(\bar{x},\bar{\tau })\in \mathcal{S}\times [0,1)$ such that

$$ \gamma \circ \eta \circ \varphi _{\rho }(\bar{x},\bar{\tau })=0, $$

which contradicts (3.23).

Finally, to complete the proof, we only show that the solution of (2.2) corresponding to the critical level existing in the interval $(m_{\infty }, 2m_{\infty })$ is a constant sign solution. To this end, applying the same argument as Remark 2.5, if u is a solution of (2.2) with $u^{+}\neq 0$ and $u^{-}\neq 0$, then $I(u)\geq 2m_{\infty }$. This concludes that it is positive. □