Ground state solutions for a class of generalized quasilinear Schrödinger–Poisson systems

Liejun Shen

doi:10.1186/s13661-018-0957-3

Research
Open Access

Ground state solutions for a class of generalized quasilinear Schrödinger–Poisson systems

Liejun Shen^{1, 2}Email author View ORCID ID profile

Boundary Value Problems20182018:44

https://doi.org/10.1186/s13661-018-0957-3

Received: 11 September 2017
Accepted: 13 March 2018
Published: 2 April 2018

Abstract

This paper is concerned with the existence of ground state solutions for a class of generalized quasilinear Schrödinger–Poisson systems in $\mathbb {R}^{3}$ which have appeared in plasma physics, as well as in the description of high-power ultrashort lasers in matter. By employing a change of variables, the generalized quasilinear systems are reduced to a semilinear one, whose associated functionals are well defined in the usual Sobolev space and satisfy the mountain-pass geometric. Finally, we use Ekeland’s variational principle and the mountain-pass theorem to obtain the ground state solutions for the given problem.

Keywords

Ground state
Generalized quasilinear
Variational principle
Mountain-pass theorem

MSC

35J20
35J60
35J92

1 Introduction and main results

The aim of this paper is to establish the existence of ground state solutions to the following generalized quasilinear Schrödinger–Poisson system:

$$ \textstyle\begin{cases} -\operatorname{div}(g^{2}(u)\nabla u)+g(u)g^{\prime}(u)|\nabla u|^{2}+a(x)u+\phi G(u)g(u)=k(x,u), & x\in \mathbb {R}^{3}, \\ -\Delta\phi=G^{2}(u), & x\in \mathbb {R}^{3}, \end{cases} $$

(1.1)

where $g:\mathbb {R}\to \mathbb {R}^{+}=[0,\infty)$ is an even differential function, $g^{\prime}(s)\geq0$ for all $s\geq0$, and $G(t)=\int_{0}^{t}g(s)\,ds$, $k:\mathbb {R}^{3}\times \mathbb {R}\to \mathbb {R}$ is continuous, $a(x):\mathbb {R}^{3}\to \mathbb {R}^{+}$ is continuous.

If $\phi=0$ in (1.1), solutions of this type are related to the existence of solitary wave solutions for quasilinear Schrödinger equations of the form

$$ i\partial_{t}z=-\Delta z+W(x)z-k(x,z)-\omega\Delta l \bigl(|z|^{2}\bigr)l^{\prime }\bigl(|z|^{2}\bigr)z,\quad x\in \mathbb {R}^{N}, $$

(1.2)

where ω is a real constant, $N\geq3$, $z:\mathbb {R}\times \mathbb {R}^{N}\to \mathbb{C}$, $W:\mathbb {R}^{N}\to \mathbb {R}$ is a given potential, $l:\mathbb {R}\to \mathbb {R}$ and $k:\mathbb {R}^{N}\times \mathbb {R}\to \mathbb {R}$ are suitable functions.

The semilinear case corresponding to $\omega=0$ has been studied extensively by many scholars in recent years (see [1–5]). Quasilinear equations of the form (1.2) have been derived as models of several physical phenomena corresponding to various types of $l(s)$. For instance, the case $l(s)=s$ models the time evolution of the condensate wave function in a superfluid film [6, 7], and it is called the superfluid film equation in fluid mechanics by Kurihara [6]. In the case $l(s)=(1+s)^{1/2}$, problem (1.2) models the self-channeling of a high-power ultra short laser in matter, the propagation of a high-irradiance laser in a plasma creates an optical index depending nonlinearly on the light intensity and this leads to an interesting new nonlinear wave equation (see [8–11]). Equation (1.2) also appears in plasma physics and fluid mechanics [12–15], in dissipative quantum mechanics [16] and in condensed matter theory [17].

Recently, Deng–Peng–Yan [18] introduced a class of generalized quasilinear critical Schrödinger equations,

$$ -\operatorname{div}\bigl(g^{2}(u)\nabla u \bigr)+g(u)g^{\prime}(u)|\nabla u|^{2}+a(x)u =k(x,u),\quad x\in \mathbb {R}^{N} , $$

(1.3)

to study the existence of positive soliton solutions. The reason we call Eq. (1.3) a generalized quasilinear Schrödinger equation is that if we take

$$g^{2}(u)=1+\frac{ [l^{\prime}(u^{2}) ]^{2}}{2}, $$

then the following quasilinear equation:

$$-\Delta u+a(x)u-\Delta l\bigl(u^{2}\bigr)l^{\prime} \bigl(u^{2}\bigr)u=k(x,u),\quad x\in \mathbb {R}^{N}, $$

turns into it (see [18, 19]). Equation (1.3) also arises in biological models and propagation of laser beams when $g(u)$ is a positive constant. If we set $g^{2}(u) =1 +2u^{2}$, i.e. $l(s) =s$, we get the superfluid film equation in plasma physics:

$$-\Delta u+V(x)u-\Delta\bigl(u^{2}\bigr)u=k(x,u),\quad x\in \mathbb {R}^{N}. $$

If we set $g^{2}(u) =1 + \frac{u^{2}}{2(1+u^{2})}$, i.e. $l(s)=(1+s)^{1/2}$, we get the equation

$$-\Delta u+V(x)u- \bigl[\Delta\bigl(1+u^{2}\bigr)^{\frac{1}{2}} \bigr] \frac {1}{2(1+u^{2})^{\frac{1}{2}}}u=k(x,u), \quad x\in \mathbb {R}^{N}, $$

which models the self-channeling of a high-power ultrashort laser in matter. For the related and important results on quasilinear Schrödinger equations, we refer the reader to [19–26] and the references therein.

We call problem (1.1) the generalized quasilinear Schrödinger–Poisson system because of the coupling of the Poisson equation with (1.3). Indeed, if we choose $g(t)=1$ for all $t\in \mathbb {R}$, then (1.1) transforms to the following classical Schrödinger–Poisson system:

$$\textstyle\begin{cases} -\Delta u+a(x)u+\phi u=k(x,u), & x\in \mathbb {R}^{3}, \\ -\Delta\phi=u^{2}, & x\in \mathbb {R}^{3}, \end{cases} $$

proposed by Benci–Fortunato [27, 28] to represent solitary waves for nonlinear Schrödinger type equations and look for the existence of standing waves interacting with an unknown electrostatic field. We refer the reader to [29–34] for some related and important results. In view of this, it is also reasonable to consider the generalized quasilinear Schrödinger–Poisson system.

According to Ruiz [35], for any $u\in H^{1}(\mathbb {R}^{3})$ we can define

$$\phi_{u}(x)=\frac{1}{4\pi} \int_{\mathbb {R}^{3}}\frac{u^{2}(y)}{|x-y|}\, dy, $$

which is a weak solution to $-\Delta\phi=u^{2}$ in $\mathbb {R}^{3}$. Therefore the weak solution of $-\Delta\phi=G^{2}(u)$ can be represented as

$$\phi_{G(u)}(x)=\frac{1}{4\pi} \int_{\mathbb {R}^{3}}\frac{G^{2}(u(y))}{|x-y|}\,dy $$

and then (1.1) can be reduced to a single equation:

$$ -\operatorname{div}\bigl(g^{2}(u)\nabla u \bigr)+g(u)g^{\prime}(u)|\nabla u|^{2}+a(x)u+\phi _{G(u)} G(u)g(u)=k(x,u),\quad x\in \mathbb {R}^{3}. $$

(1.4)

In this paper, we establish the existence of ground state solutions for problem (1.1). To this end, we assume $k(x,t)=b(x)|G(u)|^{p-2}G(u)g(u)-c(x)|G(u)|^{q-2}G(u)g(u)$. Hence the problem (1.4) can be rewritten in the following form:

$$\begin{aligned}& -\operatorname{div}\bigl(g^{2}(u)\nabla u\bigr)+g(u)g^{\prime}(u) \vert \nabla u \vert ^{2}+a(x)u+\phi _{G(u)} G(u)g(u) \\& \quad =b(x) \bigl\vert G(u) \bigr\vert ^{p-2}G(u)g(u)-c(x) \bigl\vert G(u) \bigr\vert ^{q-2}G(u)g(u), \end{aligned}$$

(1.5)

whose corresponding variational functional is given by

$$\begin{aligned} I(u) =&\frac{1}{2} \int_{\mathbb {R}^{3}}g^{2}(u) \vert \nabla u \vert ^{2}\,dx+\frac{1}{2} \int _{\mathbb {R}^{3}}a(x)u^{2}\,dx +\frac{1}{4} \int_{\mathbb {R}^{3}}\phi_{G(u)}G^{2}(u)\,dx \\ &{}-\frac{1}{p} \int_{\mathbb {R}^{3}}b(x) \bigl\vert G(u) \bigr\vert ^{p} \,dx+\frac{1}{q} \int_{\mathbb {R}^{3}}c(x) \bigl\vert G(u) \bigr\vert ^{q} \,dx. \end{aligned}$$

Unfortunately, the above functional I may be not well defined in $H^{1}(\mathbb {R}^{3})$. To overcome this difficulty, we make a change of variable constructed by Shen–Wang [19],

$$v=G(u)= \int^{u}_{0}g(\tau)\,d\tau. $$

Then we get

$$\begin{aligned} J(v) =&\frac{1}{2} \int_{\mathbb {R}^{3}} \vert \nabla v \vert ^{2}\,dx+ \frac {1}{2} \int_{\mathbb {R}^{3}}a(x) \bigl\vert G^{-1}(v) \bigr\vert ^{2}\,dx + \frac{1}{4} \int_{\mathbb {R}^{3}}\phi_{v}v^{2}\,dx \\ &{}-\frac{1}{p} \int_{\mathbb {R}^{3}}b(x) \vert v \vert ^{p}\,dx+ \frac{1}{q} \int_{\mathbb {R}^{3}}c(x) \vert v \vert ^{q} \,dx. \end{aligned}$$

(1.6)

Since g is a nondecreasing positive function, we get $|G^{-1}(v)|\leq |v|/g(0)$. It is clear that J is well defined in $H^{1}(\mathbb {R}^{3})$ and $J\in C^{1}$ if assumption (H₁) holds.

If u is a nontrivial solution of (1.5), then it should satisfy

$$\begin{aligned}& \int_{\mathbb {R}^{3}} \bigl[g^{2}(u)\nabla u\nabla \varphi+g(u)g^{\prime }(u) \vert \nabla u \vert ^{2} \varphi+a(x)u\varphi +\phi_{G(u)}G(u)g(u)\varphi \\& \quad {}-b(x) \bigl\vert G(u) \bigr\vert ^{p-2}G(u)g(u)\varphi+c(x) \bigl\vert G(u) \bigr\vert ^{q-2}G(u)g(u)\varphi \bigr]\,dx=0, \end{aligned}$$

for any $\varphi\in C^{\infty}_{0}(\mathbb {R}^{3})$. Let $\varphi=\psi/g(u)$, we know that the above formula is equivalent to

$$\begin{aligned} \bigl\langle J^{\prime}(v),\psi\bigr\rangle =& \int_{\mathbb {R}^{3}} \biggl[ \nabla v \nabla\psi+a(x)\frac{G^{-1}(v)}{g(G^{-1}(v))} \psi+\phi_{v} v\psi -b(x)|v|^{p-2}v\varphi+c(x)|v|^{q-2}v \varphi \,dx \biggr] \\ =&0,\quad \forall\psi\in C^{\infty}_{0}\bigl(\mathbb {R}^{3} \bigr). \end{aligned}$$

Therefore, in order to find the nontrivial solutions of (1.5), it suffices to study the existence of the nontrivial solutions of the following equations:

$$ -\Delta v+a(x)\frac{G^{-1}(v)}{g(G^{-1}(v))}+\phi_{v} v=b(x)|v|^{p-2}v-c(x)|v|^{q-2}v. $$

(1.7)

It is easy to verify that the problem (1.5) is equivalent to problem (1.7) and the nontrivial critical points of $J(v)$ are the nontrivial solutions of problem (1.7). Inspired by all the work described above, particularly, by the results in [18, 25], we intend to show the existence of ground state solutions of problem (1.7). To this end, we first give some assumptions on g, a, b and c.

(g):: $g\in C^{1}(\mathbb {R})$ is an even positive function and $g^{\prime}(t)\geq0$ for all $t\geq0$ and $g(0) =1$;

(H₁):: $a(x)$, $b(x)$ and $c(x)$ are continuous and nonnegative and bounded;

(H₂):: $a(x)\leq\lim_{|x|\to\infty}a(x)\triangleq a_{\infty}$, $b(x)\geq\lim_{|x|\to\infty}b(x)\triangleq b_{\infty}$ and $c(x)\leq\lim_{|x|\to\infty}c(x)\triangleq c_{\infty}$ and one of these inequalities is strict on a set of positive measure.

Our main result is as follows.

Theorem 1.1

Suppose (g) and (H₁)–(H₂) hold. Problem (1.1) admits at least a ground state solution if $2< q<4<p<6$.

To prove our main theorem, we need to introduce the limiting equation at infinity related to problem (1.7)

$$ -\Delta v+a_{\infty}\frac{G^{-1}(v)}{g(G^{-1}(v))}+\phi_{v} v=b_{\infty}|v|^{p-2}v-c_{\infty}|v|^{q-2}v, $$

(1.8)

which plays a vital role.

The outline of this paper is as follows. In Sect. 2, we introduce and provide several lemmas. In Sect. 3, we prove the limiting equation (1.8) has a ground state solution. The proof of Theorem 1.1 is completed in Sect. 4.

Notations

Throughout this paper we shall denote by C and $C_{i}$ ($i=1, 2,\ldots $) various positive constants whose exact value may change from line to line but are not essential to the analysis of the problem. $L^{p}(\mathbb {R}^{3})$ ($1\leq p\leq+\infty$) is the usual Lebesgue space with the standard norm $|u|_{p}$. We use “→” and “⇀” to denote the strong and weak convergence in the related function space, respectively. For any $\rho>0$ and any $x\in \mathbb {R}^{3}$, $B_{\rho}(x)$ denotes the ball of radius ρ centered at x, that is, $B_{\rho}(x):=\{y\in \mathbb {R}^{3}:|y-x|<\rho \}$.

2 Variational settings and preliminaries

In this section, we will give some lemmas which are useful for the main results. To solve problem (1.1), we firstly introduce some function spaces. Throughout the paper, we consider the Hilbert space $H^{1}(\mathbb {R}^{3})$ with the inner product and the norm as follows:

$$\begin{aligned}& (u,v)= \int_{\mathbb {R}^{3}}\nabla u\nabla v+a(x)uv\,dx\quad \text{and} \\& \|u\|= \biggl( \int_{\mathbb {R}^{3}}|\nabla u|^{2}+ a(x)u^{2}\,dx \biggr)^{\frac{1}{2}}, \quad \forall u,v\in H^{1}\bigl(\mathbb {R}^{3} \bigr) \end{aligned}$$

which are equivalent to the usual inner product and the norm in $H^{1}(\mathbb {R}^{3})$ because of the assumption (H₁).

Lemma 2.1

Assume (g), the functions $g(s)$ and $G(s)$ have the following properties:

(1)

both G and $G^{-1}$ are odd and for all $s\geq0$, $t\geq0$, we have
$$G(t)\leq g(t)t,\qquad s/g\bigl(G^{-1}(s)\bigr)\leq G^{-1}(s) \leq s; $$
(2)

for all $s\geq0$, $G^{-1}(s)/s$ is non-increasing and
$$\lim_{s\to0}\frac{G^{-1}(s)}{s}=\frac{1}{g(0)}=1\quad \textit{and}\quad \lim_{s\to\infty}\frac{G^{-1}(s)}{s}= \textstyle\begin{cases} \frac{1}{g(\infty)}, & \textit{if } g \textit{ is bounded}, \\ 0, & \textit{if } g \textit{ is unbounded}. \end{cases} $$

Proof

The proof is standard, see [18, 25] for example. □

Denote

$$ f(x,s)\triangleq b(x)|s|^{p-2}s-c(x)|s|^{q-2}s+a(x)s-a(x) \frac {G^{-1}(s)}{g(G^{-1}(s))} $$

(2.1)

and

$$ F(x,s)\triangleq\frac{1}{p}b(x)|s|^{p}- \frac{1}{q}c(x)|s|^{q}+\frac {1}{2}a(x)s^{2}- \frac{1}{2}a(x) \bigl\vert G^{-1}(s) \bigr\vert ^{2}. $$

(2.2)

Lemma 2.2

The functions $f(x,s)$ and $F(x,s)$ satisfy the following properties under the assumptions (g) and (H₁)–(H₂).

(1)

$f(x,s)=o(s)$ and $F(x,s)=o(s^{2})$ as $s\to0^{+}$ uniformly in $x\in \mathbb {R}^{3}$;
(2)

$f(x,s)=o(s^{5})$ and $F(x,s)=o(s^{6})$ as $s\to+\infty$ uniformly in $x\in \mathbb {R}^{3}$;
(3)

$\frac{1}{4}f(x,s)s-F(x,s)+\frac{1}{4}a(x)s^{2}\geq\frac {1}{4}a(x)|G^{-1}(s)|^{2}$ uniformly in $x\in \mathbb {R}^{3}$;
(4)

$\lim_{|x|\to\infty}f(x,s)=f_{\infty}(s)$ exists and
$$f_{\infty}(s)\triangleq b_{\infty}|s|^{p-2}s-c_{\infty}|s|^{q-2}s+a_{\infty}s-a_{\infty}\frac{G^{-1}(s)}{g(G^{-1}(s))}. $$
Furthermore, we have
$$ 3f_{\infty}(s)s-f^{\prime}_{\infty}(s)s^{2}-2a_{\infty}s^{2}\leq0\quad \textit {for any } s\in \mathbb {R}. $$

(2.3)

Proof

Points (1)–(2) are obvious, see [18] for example. Recalling that $2< q<4<p<6$ and using Lemma 2.1(1), we have

$$ \begin{aligned}[b] \frac{1}{4}f(x,s)s-F(x,s)+ \frac{1}{4}a(x)s^{2}&=\frac{p-4}{4p}b(x) \vert s \vert ^{p} +\frac{4-q}{4q}c(x) \vert s \vert ^{q} + \frac{1}{4}a(x) \bigl\vert G^{-1}(s) \bigr\vert ^{2} \\ &\quad {}+\frac{1}{4}a(x) \biggl[ \bigl\vert G^{-1}(s) \bigr\vert ^{2}-\frac {G^{-1}(s)s}{g(G^{-1}(s))} \biggr], \end{aligned} $$

(2.4)

which yields the point (3). The first part of point (4) follows from our assumption (H₂). Using $2< q<4<p<6$ and Lemma 2.1(1) again, we obtain

$$\begin{aligned}& 3f_{\infty}(s)s-f^{\prime}_{\infty}(s)s^{2}-2a_{\infty}s^{2} \\& \quad = (4-p)b_{\infty}|s|^{p}+(q-4)c_{\infty}|s|^{q} \\& \qquad {}+a_{\infty}\frac{g(G^{-1}(s))s^{2}-G^{-1}(s)g^{\prime }(G^{-1}(s))s^{2}-3G^{-1}(s)g^{2}(G^{-1}(s))s}{g^{3}(G^{-1}(s))} \\& \quad \leq a_{\infty}\frac{g(G^{-1}(s))s^{2}-G^{-1}(s)g^{\prime }(G^{-1}(s))s^{2}-3G^{-1}(s)g^{2}(G^{-1}(s))s}{g^{3}(G^{-1}(s))} \\& \quad \leq a_{\infty}\frac{-2g(G^{-1}(s))s^{2}-G^{-1}(s)g^{\prime }(G^{-1}(s))s^{2}}{g^{3}(G^{-1}(s))}\leq0, \end{aligned}$$

which gives the second part of point (4). The proof is complete. □

We collect some properties of the functions $\phi_{v}$.

Lemma 2.3

(see [29, Lemmas 2.1–2.2])

For any $v\in H^{1}(\mathbb {R}^{3})$, we have:

(1)

There exists a constant $C>0$ such that $\int_{\mathbb {R}^{3}}\phi _{v}v^{2}\,dx\leq C|v|_{12/5}^{4}$;
(2)

$\phi_{v}(x)\geq0$, $\phi_{tv}(x)=t^{2}\phi_{v}(x)$ and $\phi _{v(\cdot+y)}=\phi_{v}(\cdot+y)$;
(3)

If $v_{n}\rightharpoonup v$ in $H^{1}(\mathbb {R}^{3})$ and $v_{n}\to v$ a.e. in $\mathbb {R}^{3}$, we have
$$ \lim_{n\to\infty} \biggl[ \int_{\mathbb {R}^{3}}\phi_{v_{n}}v_{n}^{2}\,dx- \int_{\mathbb {R}^{3}}\phi_{v_{n}-v}(v_{n}-v)^{2} \,dx- \int_{\mathbb {R}^{3}}\phi_{v}v^{2}\,dx \biggr]=0 $$

(2.5)
and
$$ \lim_{n\to\infty} \biggl[ \int_{\mathbb {R}^{3}}\phi_{v_{n}}v_{n}\varphi \,dx- \int _{\mathbb {R}^{3}}\phi_{v}v\varphi \,dx \biggr]=0 \quad \textit{for any } \varphi\in C^{\infty}_{0}\bigl(\mathbb {R}^{3} \bigr). $$

(2.6)

We now introduce some definitions. Let $(X,\|\cdot\|)$ be a Banach space with its dual space $(X^{-1},\|\cdot\|_{*})$, and Φ be its functional on X. The $(\mathit{Ce})$ sequence (or $(\mathit{PS})$ sequence) at level $c\in \mathbb {R}$ ($(\mathit{Ce})_{c}$ sequence ($(\mathit{PS})_{c}$ sequence) in short) corresponding to Φ assumes that $\Phi(x_{n})\to c$ and $(1+\|x_{n}\| )\|\Phi^{\prime}(x_{n})\|_{*}\to0$ ($\Phi^{\prime}(x_{n})\to0$) as $n\to\infty$, where $\{x_{n}\}\subset X$. If for any $(\mathit{Ce})_{c}$ sequence $\{x_{n}\}$ in X, there exists a subsequence $\{x_{n_{k}}\}$ such that $x_{n_{k}}\to x_{0}$ in X for some $x_{0}\in X$, then we say that the functional Φ satisfies the so called $(\mathit{Ce})_{c}$ condition.

Lemma 2.4

The functional $J(v)$ satisfies the mountain-pass geometry, that is,

(i)

there exist $\eta,\rho>0$ such that $J(v)\geq\eta>0$ when $\| v\|=\rho$;
(ii)

there exists $e\in H^{1}(\mathbb {R}^{3})$ with $\|e\|>\rho$ such that $J(e)<0$.

Proof

(i) From Lemma 2.2(1)–(2), for any $\varepsilon >0$, there exists $C_{\varepsilon }>0$ such that

$$\begin{aligned} J(v) =& \frac{1}{2}\|v\|^{2}+\frac{1}{4} \int_{\mathbb {R}^{3}}\phi_{v}v^{2}\,dx- \int _{\mathbb {R}^{3}}F(x,v)\,dx \\ \geq& \frac{1}{2}\|v\|^{2}-\varepsilon \|v\|^{2}-C_{\varepsilon }\|v\|^{6}. \end{aligned}$$

It follows that

$$J(v) \geq C\|v\|^{2}-C\|v\|^{6} $$

if we choose sufficiently small $\varepsilon >0$ and $\rho>0$, which implies the result (i).

(ii) Choosing $v_{0}\in H^{1}(\mathbb {R}^{3})\setminus \{0\}$ and using Lemma 2.1(1), one has

$$\begin{aligned} J(tv_{0}) \leq&\frac{t^{2}}{2}\|v_{0}\|^{2}+ \frac{t^{4}}{4} \int_{\mathbb {R}^{3}}\phi _{v}v^{2}\,dx- \frac{t^{p}}{p} \int_{\mathbb {R}^{3}}b(x)|v_{0}|^{p}\,dx \\ &{}+ \frac{t^{q}}{q} \int _{\mathbb {R}^{3}}c(x)|v_{0}|^{q}\,dx \to-\infty \end{aligned}$$

as $t\to+\infty$. Hence letting $e=t_{0}v_{0}\in H^{1}(\mathbb {R}^{3})\setminus \{ 0\}$ with $t_{0}$ sufficiently large, we have $\|e\|>\rho$ and $J(e)<0$. □

By Lemma 2.4 and the variant mountain-pass theorem [36, Theorem 1], a $(\mathit{Ce})_{c}$ sequence of the functional $J(v)$ at the level

$$ c:=\inf_{\gamma\in\Gamma}\max_{t\in[0,1]}J\bigl( \gamma(t)\bigr)>0 $$

(2.7)

can be constructed, where the set of paths is defined as

$$\Gamma:= \bigl\{ \gamma\in C \bigl([0,1],H^{1}\bigl(\mathbb {R}^{3} \bigr) \bigr):\gamma (0)=0, J\bigl(\gamma(1)\bigr)< 0 \bigr\} . $$

In other words, there exists a sequence $\{v_{n}\}\subset H^{1}(\mathbb {R}^{3})$ such that

$$ J(v_{n})\to c, \bigl(1+ \Vert v_{n} \Vert \bigr) \bigl\Vert J^{\prime}(v_{n}) \bigr\Vert _{*}\to0 \quad \text{as } n\to \infty. $$

(2.8)

Lemma 2.5

Any sequence $\{v_{n}\}\subset H^{1}(\mathbb {R}^{3})$ verifying (2.8) is bounded.

Proof

Since $\{v_{n}\}\subset H^{1}(\mathbb {R}^{3})$ is a $(\mathit{Ce})_{c}$ sequence, we have

$$\begin{aligned} c+1 \geq& J(v_{n})-\frac{1}{4}\bigl\langle J^{\prime }(v_{n}),v_{n}\bigr\rangle \\ =&\frac{1}{4} \int_{\mathbb {R}^{3}}|\nabla v_{n}|^{2}\,dx+ \int_{\mathbb {R}^{3}} \biggl[\frac{1}{4}f(x,v_{n})v_{n}-F(x,v_{n})+ \frac{1}{4}a( x)v_{n}^{2} \biggr]\,dx \\ \stackrel{\text{(2.4)}}{\geq}&\frac{1}{4} \int_{\mathbb {R}^{3}}|\nabla v_{n}|^{2}\,dx+ \frac{1}{4} \int_{\mathbb {R}^{3}}a( x) \bigl\vert G^{-1}(v_{n}) \bigr\vert ^{2} \end{aligned}$$

(2.9)

and by Lemma 2.1(1),

$$\begin{aligned} \begin{aligned}[b] \int_{\mathbb {R}^{3}}a(x)v_{n}^{2}\,dx &= \int _{|G^{-1}(v_{n})|>1}a(x)v_{n}^{2}\,dx+ \int_{|G^{-1}(v_{n})|\leq1}a(x)v_{n}^{2}\,dx \\ & \leq C \int_{\mathbb {R}^{3}}v_{n}^{6}\,dx+g^{2}(1) \int_{|G^{-1}(v_{n})|\leq 1}a(x)\frac{v_{n}^{2}}{g^{2}(G^{-1}(v_{n}))}\,dx \\ &\leq C \int_{\mathbb {R}^{3}}v_{n}^{6}\,dx+g^{2}(1) \int_{\mathbb {R}^{3}}a(x) \bigl\vert G^{-1}(v_{n}) \bigr\vert ^{2}\,dx. \end{aligned} \end{aligned}$$

(2.10)

Combining (2.9)–(2.10) and the Sobolev inequality, the sequence $\{v_{n}\}$ is bounded. □

Let $\{v_{n}\}$ be a $(\mathit{Ce})$ sequence of J. Going to a subsequence if necessary, we may assume that $v_{n}\rightharpoonup v$ in $H^{1}(\mathbb {R}^{3})$, $v_{n}\to v$ in $L^{r}_{\mathrm{loc}}(\mathbb {R}^{3})$ with $r\in[1,6)$ and $v_{n}\to v$ a.e. in $\mathbb {R}^{3}$. Set $w_{n}=v_{n}-v$, by Lemma 2.2 we have the following lemma.

Lemma 2.6

(see [37, Lemma 1.3])

If $v_{n}\rightharpoonup v$ in $H^{1}(\mathbb {R}^{3})$ and $v_{n}\to v$ a.e. in $\mathbb {R}^{3}$, then

$$\lim_{n\to\infty} \biggl[ \int_{\mathbb {R}^{3}}F(x,v_{n})\,dx- \int_{\mathbb {R}^{3}}F(x,v)\,dx- \int_{\mathbb {R}^{3}}F(x,w_{n})\,dx \biggr]=0. $$

As a consequence of Lemma 2.6, we have the following lemma.

Lemma 2.7

Let $\{v_{n}\}$ be a $(\mathit{Ce})$ sequence of J at the level c, and set $w_{n}=v_{n}-v$, then $\{w_{n}\}$ is a $(\mathit{PS})$ sequence of J at the level $c-J(v)$.

Proof

We claim first that $J^{\prime}(v)=0$. In fact, it is enough to show that $\langle J^{\prime}(v),\varphi\rangle=0$ for any $\varphi\in C^{\infty}_{0}(\mathbb {R}^{3})$. By Lemma 2.2(1)–(2), it is easy to verify

$$\lim_{n\to\infty} \int_{\mathbb {R}^{3}}f(x,w_{n})\varphi \,dx=\lim _{n\to\infty } \biggl[ \int_{\mathbb {R}^{3}}f(x,v_{n})\varphi \,dx- \int_{\mathbb {R}^{3}}f(x,v)\varphi \,dx \biggr]=0. $$

Using the above formula and (2.6), we have

$$ 0=\lim_{n\to\infty}\bigl\langle J^{\prime}(v_{n}), \varphi\bigr\rangle =\bigl\langle J^{\prime}(v),\varphi\bigr\rangle $$

(2.11)

which yields the claim. By (2.11), we derive

$$\begin{aligned}& \lim_{n\to\infty}J(w_{n})=\lim_{n\to\infty} \bigl[J(v_{n})-J(v) \bigr]=c-J(v), \\& \lim_{n\to\infty}\bigl\langle J^{\prime}(w_{n}), \varphi\bigr\rangle =\lim_{n\to\infty} \bigl[\bigl\langle J^{\prime}(v_{n}),\varphi\bigr\rangle -\bigl\langle J^{\prime}(v),\varphi\bigr\rangle \bigr]=0, \end{aligned}$$

which show that $\{w_{n}\}$ is a $(\mathit{PS})$ sequence of J at the level $c-J(v)$. □

The proofs of the following lemmas can be found in the corresponding references.

Lemma 2.8

(see [38, 39])

Let $\{\rho_{n}\}$ be a sequence of nonnegative functions satisfying $|\rho_{n}|_{1}=\lambda $ and $\lambda >0$ is fixed, then there exists a subsequence, still denoted by $\{\rho_{n}\}$, satisfying one of the following two possibilities:

(i)

(Vanishing) for any fixed $R > 0$, we have
$$\lim_{n\to\infty}\sup_{y\in \mathbb {R}^{N}} \int_{B_{R}(y)}\rho_{n}(x)\,dx=0. $$
(ii)

(Nonvanishing) there exist $\beta>0$, $\overline{R}\in (0,+\infty)$ and $\{y_{n}\}\subset \mathbb {R}^{N}$ such that
$$\lim_{n\to\infty} \int_{B_{\overline{R}}(y_{n})}\rho_{n}(x)\,dx\geq \beta>0. $$

Lemma 2.9

(see [38, 39])

Assume that $\{u_{n}\}$ is bounded in $H^{1}(\mathbb {R}^{3})$ and satisfies

$$\lim_{n\to\infty}\sup_{y\in \mathbb {R}^{3}} \int_{B_{R}(y)}|u_{n}|^{2}\,dx=0, $$

for some $R>0$. Then $u_{n}\to0$ in $L^{r}(\mathbb {R}^{3})$ for every $2< r<6$.

3 The existence of ground state solution for limit equation at infinity

In this section, by employing Ekeland’s variational principle [40], we prove the existence of ground state solution for problem (1.8) which is the limit equation of problem (1.7) at infinity. To establish the ground sate solution of problem (1.8), we set

$$m_{\infty}=\inf_{u\in\mathcal{N}_{\infty}}J_{\infty}(u) \quad \text{and}\quad \mathcal{N}_{\infty}= \bigl\{ u\in H^{1}\bigl( \mathbb {R}^{3}\bigr)\setminus \{0\}:\bigl\langle J_{\infty}^{\prime}(u),u \bigr\rangle =0 \bigr\} . $$

Since $a_{\infty}$ is positive, the norm and inner product in this section are not distinguished by the norm and inner product used in the previous section. To show that the Nehair manifold $\mathcal{N}_{\infty}$ is nonempty and $m_{\infty}$ is well defined, we prove the following lemma.

Lemma 3.1

Assume (g) and (H₁)–(H₂), then we have the following properties:

(a)

For any $v\in H^{1}(\mathbb {R}^{3})\setminus \{0\}$, there exists a unique $t_{v}>0$ such that $t_{v}v\in\mathcal{N}_{\infty}$ and $J_{\infty}(t_{v}v)=\max_{t\geq0}J_{\infty}(tv)$. In particular, if $v\in\mathcal {N}_{\infty}$ we have $J_{\infty}(v)=\max_{t\geq0}J_{\infty}(tv)$;
(b)

There exists $\alpha> 0$ such that $\|u\|\geq\alpha$ for all $u\in\mathcal{N}_{\infty}$;
(c)

$J_{\infty}$ is bounded from below on $\mathcal{N}_{\infty}$ by a positive constant;
(d)

$J_{\infty}$ is coercive on $\mathcal{N}_{\infty}$, i.e. $J_{\infty}(v)\to\infty$ as $\|v\|\to\infty$ when $v\in\mathcal {N}_{\infty}$.

Proof

(a) Given $v\in H^{1}(\mathbb {R}^{3})\setminus \{0\}$, by [41, Lemma 2.4] and Lemma 2.2(1)–(2) we derive that

$$\xi(t)=c_{1}t^{2}+c_{2}t^{4}- \int_{\mathbb {R}^{3}}F_{\infty}(tv)\,dx $$

has a unique positive critical point which corresponds to its maximum, where the fact $F_{\infty}(tv)/t^{2}\to\infty$ as $t\to\infty$ is used. Let $\xi(t)=J_{\infty}(tv)$, then we can conclude the result (a).

(b) If $v\in\mathcal{N}_{\infty}$, using Lemma 2.2(1)–(2) one has

$$\begin{aligned} \|v\|^{2} &\leq\|v\|^{2}+ \int_{\mathbb {R}^{3}}\phi_{v}v^{2}\,dx= \int_{\mathbb {R}^{3}}f_{\infty}(v)v\,dx\leq \varepsilon \int_{\mathbb {R}^{3}}v^{2}\,dx+C_{\varepsilon }\int_{\mathbb {R}^{3}}v^{6}\,dx \\ & \leq C\varepsilon \|v\|^{2}+CC_{\varepsilon }\|v\|^{6}= \frac{1}{2}\|v\|^{2}+C\|v\|^{6} \end{aligned}$$

if we choose $C\varepsilon =1/2$ and the result (b) follows.

(c) If $v\in\mathcal{N}_{\infty}$, then

$$J_{\infty}(v)=J_{\infty}(v)-\frac{1}{4}\bigl\langle J_{\infty}^{\prime }(v),v\bigr\rangle \geq \frac{1}{4} \int_{\mathbb {R}^{3}}|\nabla v|^{2}\,dx+\frac{1}{4} \int_{\mathbb {R}^{3}}a_{\infty}\bigl\vert G^{-1}(v) \bigr\vert ^{2}\,dx, $$

which together with the result (b) and (2.10) gives the result (c).

(d) Combining the above formula and (2.10), the result (d) is obvious. □

Lemma 3.2

Assume (g), then $m_{\infty}=\inf_{u\in\mathcal{N}_{\infty}}J_{\infty}(u)$ can be attained.

Proof

If it is possible to verify that a minimizing sequence of $m_{\infty}$ is radially symmetric, the minimizer may be easily obtained. In particular, the minimizer is a critical point of $J_{\infty}$ restricted on $\mathcal{N}_{\infty}$. Finally, we proceed as follows.

Step 1: Any minimizing sequence of $m_{\infty}$ can be radially symmetric.

Let $\{v_{n}\}\subset H^{1}(\mathbb {R}^{3})$ be a minimizing sequence of $m_{\infty}$, that is, $\langle J^{\prime}_{\infty}(v_{n}),v_{n}\rangle=0$ and $J_{\infty}(v_{n})\to m_{\infty}$ as $n\to\infty$. According to the Schwarz symmetrization $v_{n}^{*}$ of $v_{n}$, we know that $v_{n}^{*}$ is continuous and nonnegative and satisfies

$$\begin{aligned}& \int_{\mathbb {R}^{3}} \bigl\vert \nabla v_{n}^{*} \bigr\vert ^{2}\,dx\leq \int_{\mathbb {R}^{3}} \vert \nabla v_{n} \vert ^{2}\,dx \quad \text{and} \\& \int_{\mathbb {R}^{3}}h\bigl(v_{n}^{*}\bigr)\,dx= \int_{\mathbb {R}^{3}}h(v_{n})\,dx\quad \text{for any } h(v_{n})\in L^{1}\bigl(\mathbb {R}^{3}\bigr), \end{aligned}$$

which give $\langle J^{\prime}_{\infty}(v_{n}^{*}),v_{n}^{*}\rangle\leq \langle J^{\prime}_{\infty}(v_{n}),v_{n}\rangle=0$. It is obvious that $\langle J^{\prime}_{\infty}(tv_{n}^{*}),tv_{n}^{*}\rangle>0$ for sufficiently small $t>0$. Hence there is $t_{0}\in(0,1]$ satisfying $\langle J^{\prime}_{\infty}(t_{0}v_{n}^{*}),t_{0}v_{n}^{*}\rangle=0$ and then

$$\begin{aligned} m_{\infty} \leq&J_{\infty}\bigl(t_{0}v_{n}^{*} \bigr)=J_{\infty}\bigl(t_{0}v_{n}^{*}\bigr)- \frac {1}{4}\bigl\langle J^{\prime}_{\infty}\bigl(t_{0}v_{n}^{*}\bigr),t_{0}v_{n}^{*} \bigr\rangle \\ =&\frac{t_{0}^{2}}{4} \int_{\mathbb {R}^{3}} \bigl\vert \nabla v_{n}^{*} \bigr\vert ^{2}\,dx+\frac{1}{4} \int _{\mathbb {R}^{3}}a_{\infty}\bigl\vert G^{-1} \bigl(t_{0}v_{n}^{*}\bigr) \bigr\vert ^{2}\,dx+ \frac{p-4}{4p}t_{0}^{p} \int_{\mathbb {R}^{3}}b_{\infty}\bigl\vert v_{n}^{*} \bigr\vert ^{p}\,dx \\ &{}+\frac{4-q}{4q}t_{0}^{q} \int_{\mathbb {R}^{3}}c_{\infty}\bigl\vert v_{n}^{*} \bigr\vert ^{q}\,dx \\ \leq&\frac{1}{4} \int_{\mathbb {R}^{3}} \bigl\vert \nabla v_{n}^{*} \bigr\vert ^{2}\,dx+\frac{1}{4} \int _{\mathbb {R}^{3}}a_{\infty}\bigl\vert G^{-1} \bigl(v_{n}^{*}\bigr) \bigr\vert ^{2}\,dx+\frac{p-4}{4p} \int_{\mathbb {R}^{3}}b_{\infty}\bigl\vert v_{n}^{*} \bigr\vert ^{p}\,dx \\ &{}+\frac{4-q}{4q} \int_{\mathbb {R}^{3}}c_{\infty}\bigl\vert v_{n}^{*} \bigr\vert ^{q}\,dx \\ \leq&\frac{1}{4} \int_{\mathbb {R}^{3}} \vert \nabla v_{n} \vert ^{2}\,dx+\frac{1}{4} \int_{\mathbb {R}^{3}}a_{\infty}\bigl\vert G^{-1}(v_{n}) \bigr\vert ^{2}\,dx+\frac{p-4}{4p} \int_{\mathbb {R}^{3}}b_{\infty} \vert v_{n} \vert ^{p}\,dx \\ &{}+\frac{4-q}{4q} \int_{\mathbb {R}^{3}}c_{\infty} \vert v_{n} \vert ^{q}\,dx \\ =&J_{\infty}(v_{n})-\frac{1}{4}\bigl\langle J^{\prime}_{\infty}(v_{n}),v_{n}\bigr\rangle =J_{\infty}(v_{n})\to m_{\infty} \end{aligned}$$

which yields $t_{0}=1$. Therefore we conclude that $\langle J^{\prime }_{\infty}(v_{n}^{*}),v_{n}^{*}\rangle=0$ and $J_{\infty}(v_{n}^{*})\to m_{\infty}$ as $n\to\infty$. So the proof of Step 1 is complete.

Step 2: Any minimizing sequence of $m_{\infty}$ contains a strongly convergent subsequence.

Let $\{v_{n}\}\subset H^{1}(\mathbb {R}^{3})$ be a minimizing sequence of $m_{\infty}$, then similar to Lemma 2.5 we know that $\{v_{n}\}$ is bounded in $H^{1}(\mathbb {R}^{3})$. From the Step 1, we know that $\{v_{n}\}$ is radially symmetric. Up to a subsequence, there exists $v\in H^{1}(\mathbb {R}^{3})$ such that $v_{n}\rightharpoonup v$ in $H^{1}(\mathbb {R}^{3})$, $v_{n}\to v$ in $L^{r}(\mathbb {R}^{3})$ with $r\in(2,6)$ and $v_{n}\to v$ a.e. in $\mathbb {R}^{3}$. By using Fatou’s lemma, one has

$$\begin{aligned}& \int_{\mathbb {R}^{3}}|\nabla v|^{2}\,dx+ \int_{\mathbb {R}^{3}}a_{\infty}\bigl|G^{-1}(v)\bigr|^{2} \,dx+ \int _{\mathbb {R}^{3}}\phi_{v}v^{2}\,dx \\& \quad \leq\liminf_{n\to\infty} \biggl( \int_{\mathbb {R}^{3}}|\nabla v_{n}|^{2}\,dx+ \int_{\mathbb {R}^{3}}a_{\infty}\bigl\vert G^{-1}(v_{n}) \bigr\vert ^{2}\,dx+ \int_{\mathbb {R}^{3}}\phi _{v_{n}}{v_{n}}^{2} \,dx \biggr) \\& \quad = \liminf_{n\to\infty} \biggl( \int_{\mathbb {R}^{3}}b_{\infty}| v_{n}|^{p}\,dx- \int_{\mathbb {R}^{3}}c_{\infty}|v_{n}|^{q}\,dx \biggr) \\& \quad = \int_{\mathbb {R}^{3}}b_{\infty}|v|^{p}\,dx- \int_{\mathbb {R}^{3}}c_{\infty}|v|^{q}\,dx, \end{aligned}$$

which gives $\langle J^{\prime}_{\infty}(v),v\rangle\leq0$. It is easy to check that $\langle J^{\prime}_{\infty}(tv),tv\rangle>0$ for sufficiently small $t>0$. Hence there exists $\overline{t}\in(0,1]$ such that $\langle J^{\prime}_{\infty}(\overline{t}v),\overline {t}v\rangle=0$ and then

$$\begin{aligned} m_{\infty} \leq&J_{\infty}(\overline{t}v)=J_{\infty}(\overline {t}v)-\frac{1}{4}\bigl\langle J^{\prime}_{\infty}( \overline{t}v),\overline {t}v\bigr\rangle \leq J_{\infty}(v)- \frac{1}{4}\bigl\langle J^{\prime}_{\infty}(v),v\bigr\rangle \\ \leq& \liminf_{n\to\infty} \biggl[J_{\infty}(v_{n})- \frac {1}{4}\bigl\langle J^{\prime}_{\infty}(v_{n}),v_{n} \bigr\rangle \biggr]=m_{\infty}, \end{aligned}$$

which indicates that $\overline{t}=1$. Thus we have $\langle J^{\prime }_{\infty}(v),v\rangle=0$ and $J_{\infty}(v)=m_{\infty}$. □

Proposition 3.3

Assume (g), any minimizer of $m_{\infty}$ is a critical point of $J_{\infty}$ in $H^{1}(\mathbb {R}^{3})$.

Proof

If v is a minimizer of $m_{\infty}$, according to Lemma 3.2 we know that v is a critical point of ${J_{\infty }}|_{\mathcal{N}_{\infty}}$, that is, $v\in\mathcal{N}_{\infty}$ and ${J_{\infty}^{\prime}}|_{\mathcal{N}_{\infty}}(v)=0$. Hence there is a Lagrange multiplier $\lambda\in \mathbb {R}$ such that $J_{\infty}^{\prime }(v)=\lambda\Psi_{\infty}^{\prime}(v)$, where $\Psi_{\infty}(v)=\langle J_{\infty}^{\prime}(v),v\rangle$. To end the proof, it is enough to show that $\lambda=0$.

In fact, using $0=\langle J_{\infty}^{\prime}(v),v\rangle=\lambda \langle\Psi_{\infty}^{\prime}(v),v\rangle$ and $v\in H^{1}(\mathbb {R}^{3})\setminus \{0\}$ we have

$$\begin{aligned} \bigl\langle \Psi_{\infty}^{\prime}(v),v\bigr\rangle =&2 \int_{\mathbb {R}^{3}}|\nabla v|^{2}+a_{\infty}v^{2}\,dx+4 \int_{\mathbb {R}^{3}}\phi_{v}v^{2}\,dx - \int_{\mathbb {R}^{3}} \bigl[f_{\infty}^{\prime}(v)v^{2}+f_{\infty}(v)v \bigr]\,dx \\ =&-2 \int_{\mathbb {R}^{3}}|\nabla v|^{2}\,dx+ \int_{\mathbb {R}^{3}} \bigl[3f_{\infty}(v)v-f_{\infty}^{\prime}(v)v^{2}-2a_{\infty}v^{2} \bigr]\,dx \\ \stackrel{\text{(2.3)}}{\leq}&-2 \int_{\mathbb {R}^{3}}|\nabla v|^{2}\,dx< 0, \end{aligned}$$

which implies that $\lambda=0$. Hence the proof is complete. □

Theorem 3.4

Assume (g), the system (1.8) has a ground state solution.

Proof

In view of Lemma 3.1, we know that the Nehair manifold $\mathcal{N}_{\infty}$ is nonempty and $m_{\infty}$ is well defined. It follows from Lemma 3.2 that $m_{\infty}$ is attained by some $v\in\mathcal{N}_{\infty}$ and v is a critical point of ${J_{\infty}}|{\mathcal{N}_{\infty}}$. By Proposition 3.4, we have $J_{\infty}^{\prime}(v)=0$ in the whole space $H^{1}(\mathbb {R}^{3})$. Consequently, we have shown that $J(v)=m_{\infty}>0$ and $J_{\infty}^{\prime}(v)=0$, which show that v is a ground state solution of problem (1.8). The proof is complete. □

4 Proof of Theorem 1.1

In this section, we prove Theorem 1.1 by applying the variant mountain-pass theorem [36]. In the following, we will verify that the mountain-pass value c given by (2.7) is satisfied.

Lemma 4.1

Suppose that (g) and (H₁)–(H₂) hold, then $0< c< m_{\infty}$.

Proof

By Theorem 3.4, we know that there exists $v\in H^{1}(\mathbb {R}^{3})$ such that $J_{\infty}^{\prime}(v)=0$ and $J_{\infty}(v)=m_{\infty}>0$. Since $J(0)=0$ and $\lim_{t\to\infty}J(tv)=-\infty$, there exists $\tilde{t}>0$ such that

$$J(\tilde{t}v)=\max_{t\geq0}J(tv). $$

Choosing a sufficient large $t_{0}>0$ to satisfy $J(t_{0}v)<0$, then $\gamma_{0}(t)=tt_{0}v\in\Gamma$ and hence

$$c\leq\max_{t\in[0,1]}J\bigl(\gamma_{0}(t)\bigr)\leq\max _{t\geq 0}J(tv)=J(\tilde{t}v)< J_{\infty}(\tilde{t}v) \leq J_{\infty}(v)=m_{\infty}, $$

where (H₂) yields the strict inequality. The proof is complete. □

Proposition 4.2

Assume that (g) and (H₁)–(H₂) hold, then $J(v)$ satisfies the $(\mathit{Ce})_{c}$ condition if $c\in(0,m_{\infty})$.

Proof

Let $\{v_{n}\}$ be a $(\mathit{Ce})_{c}$ sequence of $J(v)$. Similar to Lemma 2.5, we find that $\{v_{n}\}$ is bounded in $H^{1}(\mathbb {R}^{3})$ and there exists a subsequence, still denoted by $\{v_{n}\}$, such that

$$\textstyle\begin{cases} v_{n}\rightharpoonup v, &\text{in } H^{1}(\mathbb {R}^{3}), \\ v_{n}\to v, &\text{in } L^{r}_{\mathrm{loc}}(\mathbb {R}^{3}) \text{ for } 1\leq r< 6, \\ v_{n}\to v, &\text{a.e. in } \mathbb {R}^{3}, \end{cases} $$

and $J^{\prime}(v)=0$ with $J(v)\geq0$ by (2.4). Denote $w_{n}=v_{n}-v$, then it follows from Lemma 2.7 that $\{w_{n}\}$ is a $(\mathit{PS})$ sequence of $J(v)$ at the level $c-J(v)$. To prove that $\|w_{n}\|\to0$ as $n\to\infty$, we divide our proof into the following two steps.

Step 1: The nonvanishing case for $\rho_{n}=|w_{n}|_{2}^{2}$ in Lemma 2.8 can never occur.

Arguing indirectly, by Lemma 2.8 we know that there exist $\beta>0$, $\overline{R}\in(0,+\infty)$ and $\{y_{n}\}\subset \mathbb {R}^{N}$ such that

$$ \liminf_{n\to\infty} \int_{B_{\overline{R}}(y_{n})}|w_{n}|^{2}\,dx\geq \beta>0. $$

(4.1)

Without loss of generality, we choose $|y_{n}|\to\infty$ as $n\to \infty$. Otherwise, $\{w_{n}\}$ is tight, and thus $|w_{n}|_{2}\to0$ as $n\to\infty$, which yields a contradiction to (4.1). Denote $\overline{w}_{n}(x)=w_{n}(x+y_{n})$. Since $\limsup_{n\to\infty}\| \overline{w}_{n}\|=\limsup_{n\to\infty}\|w_{n}\|\leq C<+\infty $, we may assume that there exists $w_{0}\in H^{1}(\mathbb {R}^{3})$ such that $\overline{w}_{n}\rightharpoonup w_{0}$ in $H^{1}(\mathbb {R}^{3})$, $\overline{w}_{n}\to w_{0}$ in $L^{r}_{\mathrm{loc}}(\mathbb {R}^{3})$ with $r\in[1,6)$ and $\overline{w}_{n}\to w_{0}$ a.e. in $\mathbb {R}^{3}$. We then claim that $\{ \overline{w}_{n}\}$ is a $(\mathit{PS})$ sequence of $J_{\infty}(v)$ at the level $c-J(v)$. In fact, in view of $\lim_{n\to\infty}a(x+y_{n})=a_{\infty}$, we have

$$\int_{\mathbb {R}^{3}}a(x) \bigl\vert G^{-1}(w_{n}) \bigr\vert ^{2}\,dx= \int_{\mathbb {R}^{3}}a(x+y_{n}) \bigl\vert G^{-1}( \overline{w}_{n}) \bigr\vert ^{2}\,dx = \int_{\mathbb {R}^{3}}a_{\infty}\bigl\vert G^{-1}( \overline{w}_{n}) \bigr\vert ^{2}\,dx+o(1) $$

and by $\lim_{n\to\infty}b(x+y_{n})=b_{\infty}$ and $\lim_{n\to\infty }c(x+y_{n})=c_{\infty}$,

$$\begin{aligned}& \int_{\mathbb {R}^{3}}b(x)|w_{n}|^{p}\,dx= \int_{\mathbb {R}^{3}}b_{\infty}|\overline{w}_{n}|^{p} \,dx+o(1), \\& \int_{\mathbb {R}^{3}}c(x)|w_{n}|^{q}\,dx= \int_{\mathbb {R}^{3}}c_{\infty}|\overline{w}_{n}|^{q} \,dx+o(1), \end{aligned}$$

which give

$$\begin{aligned} J(w_{n}) &=\frac{1}{2} \int_{\mathbb {R}^{3}}|\nabla w_{n}|^{2}+a(x)w_{n}^{2} \,dx+\frac {1}{4} \int_{\mathbb {R}^{3}}\phi_{w_{n}}w_{n}^{2}\,dx- \int_{\mathbb {R}^{3}}F(x,w_{n})\,dx \\ & =\frac{1}{2} \int_{\mathbb {R}^{3}}|\nabla\overline{w}_{n}|^{2}+a_{\infty}\overline{w}_{n}^{2}\,dx+\frac{1}{4} \int_{\mathbb {R}^{3}}\phi_{\overline{w}_{n}}\overline{w}_{n}^{2} \,dx- \int_{\mathbb {R}^{3}}F_{\infty}(\overline{w}_{n}) \,dx+o(1) \\ &=J_{\infty}(\overline{w}_{n})+o(1). \end{aligned}$$

On the other hand, since $\lim_{n\to\infty}a(x+y_{n})=a_{\infty}$, for any $\phi\in C^{\infty}_{0}(R^{N})$ we have

$$\begin{aligned} &\biggl\vert \int_{\mathbb {R}^{3}} \bigl[a(x+y_{n})-a_{\infty}\bigr] \frac {G^{-1}(\overline{w}_{n})}{g(G^{-1}(\overline{w}_{n}))}\phi \,dx \biggr\vert \\ &\quad \leq \int_{\mathbb {R}^{3}} \bigl\vert \bigl[a(x+y_{n})-a_{\infty}\bigr]\overline{w}_{n}\phi \bigr\vert \,dx \\ &\quad \leq C \biggl( \int_{\mathbb {R}^{3}} \bigl\vert a(x+y_{n})-a_{\infty}\bigr\vert ^{2}|\phi|^{2} \biggr)^{\frac{1}{2}}\to0. \end{aligned}$$

Denote $\phi_{n}(x)=\phi(x-y_{n})$, we can deduce that

$$\begin{aligned} &\int_{\mathbb {R}^{3}}a(x)\frac{G^{-1}(w_{n})}{g(G^{-1}(w_{n}))}\phi _{n}\,dx \\ &\quad = \int_{\mathbb {R}^{3}}a(x+y_{n})\frac{G^{-1}(\overline{w}_{n})}{g(G^{-1}(\overline{w}_{n}))}\phi \,dx \\ &\quad = \int_{\mathbb {R}^{3}}a_{\infty}\frac{G^{-1}(\overline{w}_{n})}{g(G^{-1}(\overline{w}_{n}))}\phi \,dx+ \int_{\mathbb {R}^{3}} \bigl[a(x+y_{n})-a_{\infty}\bigr] \frac{G^{-1}(\overline{w}_{n})}{g(G^{-1}(\overline{w}_{n}))}\phi \,dx \\ &\quad = \int_{\mathbb {R}^{3}}a_{\infty}\frac{G^{-1}(\overline{w}_{n})}{g(G^{-1}(\overline{w}_{n}))}\phi \,dx+o(1). \end{aligned}$$

(4.2)

Similarly, we derive

$$ \begin{aligned} & \int_{\mathbb {R}^{3}}b(x)|w_{n}|^{p-2}w_{n} \phi_{n}\,dx= \int_{\mathbb {R}^{3}}b_{\infty}|\overline{w}_{n}|^{p-2} \overline{w}_{n}\phi \,dx +o(1), \\ & \int_{\mathbb {R}^{3}}c(x)|w_{n}|^{q-2}v_{n} \phi_{n}\,dx= \int_{\mathbb {R}^{3}}c_{\infty}|\overline{w}_{n}|^{q-2} \overline{w}_{n}\phi \,dx +o(1). \end{aligned} $$

(4.3)

Combining (4.2) and (4.3), we have

$$\bigl\langle J_{\infty}^{\prime}(\overline{w}_{n}),\phi \bigr\rangle = \bigl\langle J^{\prime}(w_{n}),\phi_{n} \bigr\rangle +o(1)=o(1). $$

Hence the claim is true. Furthermore, we can conclude that $J_{\infty}^{\prime}(w_{0})=0$. We now use (4.1) to show that $w_{0}\not \equiv0$. In the contrary case $\overline{w}_{n}\to0$ in $L^{2}_{\mathrm{loc}}(\mathbb {R}^{3})$, we have

$$\begin{aligned} 0 =&\lim_{n\to\infty} \int_{B_{\overline{R}}(0)}|\overline{w}_{n}|^{2}\,dx= \lim _{n\to\infty} \int_{B_{\overline{R}}(y_{n})}|w_{n}|^{2}\,dx \\ \geq&\liminf _{n\to\infty} \int_{B_{\overline {R}}(y_{n})}|w_{n}|^{2}\,dx\geq\beta>0, \end{aligned}$$

which yields a contradiction. Thus $w_{0}\not\equiv0$.

Denote $z_{n}=\overline{w}_{n}-w_{0}$, by the Brézis–Lieb lemma [42] we easily get

$$J_{\infty}(\overline{w}_{n})=J_{\infty}(z_{n})+J_{\infty}(w_{0})+o(1) $$

and

$$J_{\infty}^{\prime}(\overline{w}_{n})=J_{\infty}^{\prime}(z_{n})+J_{\infty}^{\prime}(w_{0})+o(1)=J_{\infty}^{\prime}(z_{n})+o(1). $$

Hence we have

$$\liminf_{n\to\infty}J_{\infty}(z_{n}) = \liminf _{n\to\infty} \biggl[J_{\infty}(z_{n})- \frac{1}{4}\bigl\langle J_{\infty}^{\prime}(z_{n}),z_{n} \bigr\rangle \biggr] \stackrel{\text{(2.4)}}{\geq} 0 $$

and then

$$\begin{aligned} c & =J(v_{n})+o(1)=J(v)+J(w_{n})+o(1)=J(v)+J_{\infty}( \overline{w}_{n})+o(1) \\ &=J(v)+J_{\infty}(z_{n})+J_{\infty}(w_{0})+o(1), \end{aligned}$$

which implies that

$$c\geq J_{\infty}(w_{0})\geq m_{\infty}, $$

a contradiction. So the proof of Step 1 is complete.

Step 2: $\|w_{n}\|\to0$ as $n\to\infty$.

In fact, as a consequence of Step 1 and Lemma 2.8, for any fixed $R>0$, we have

$$\lim_{n\to\infty}\sup_{y\in \mathbb {R}^{3}} \int_{B_{R}(y)}w_{n}^{2}\,dx=0. $$

By Lemma 2.9, we have

$$\lim_{n\to\infty} \int_{\mathbb {R}^{3}}|w_{n}|^{r}\,dx=0 \quad \text{for all } 2< r< 6. $$

Since $2< q< p<6$, by (H₁) we have

$$\lim_{n\to\infty} \int_{\mathbb {R}^{3}}b(x)|w_{n}|^{p}\,dx=\lim _{n\to\infty} \int _{\mathbb {R}^{3}}c(x)|w_{n}|^{q}\,dx=0. $$

Therefore

$$\lim_{n\to\infty} \biggl( \int_{\mathbb {R}^{3}}|\nabla w_{n}|^{2}\,dx + \int_{\mathbb {R}^{3}}a(x) \bigl\vert G^{-1}(w_{n}) \bigr\vert ^{2}\,dx \biggr)=\lim_{n\to\infty}\bigl\langle J^{\prime}(w_{n}),w_{n}\bigr\rangle =0, $$

which together with (2.10) yields $\|w_{n}\|\to0$ as $n\to \infty$.

Summing the above two steps, we obtain $v_{n}\to v$ in $H^{1}(\mathbb {R}^{3})$ as $n\to\infty$. □

Complement of proof of Theorem 1.1

By Lemma 2.4 and the variant mountain-pass theorem [36], a sequence verifying (2.8) can be obtained. Using Lemma 4.1 and Proposition 4.2, there exists $v\in H^{1}(\mathbb {R}^{3})$ such that $J^{\prime}(v)=0$ and $J(v)=c>0$, which imply that v is a nontrivial solution to (1.1). To show the existence of ground state solution, we set

$$m=\inf_{u\in\mathcal{N}}J(u) \quad \text{and}\quad \mathcal{N}= \bigl\{ u \in H^{1}\bigl(\mathbb {R}^{3}\bigr)\setminus \{0\}:\bigl\langle J^{\prime}(u),u\bigr\rangle =0 \bigr\} . $$

Obviously, we have $c\geq m$. On the other hand, for any $v\in\mathcal {N}$, choosing a sufficient large $t_{0}>0$ to satisfy $J(t_{0}v)<0$, then $\gamma_{0}(t)=tt_{0}v\in\Gamma$ and similar to the proof of Lemma 3.1 $(a)$ we obtain

$$c\leq\max_{t\in[0,1]}J\bigl(\gamma_{0}(t)\bigr)\leq\max _{t\geq0}J(tv)=J(v), $$

which indicates that $m\geq c$. Hence $J(v)=m>0$. The proof is complete. □

5 Conclusion

In this paper, we consider the existence of ground state solutions for a class of generalized quasilinear Schrödinger–Poisson systems in $\mathbb {R}^{3}$. By employing a change of variables constructed by Shen–Wang [19] and used in [18, 25] and the references therein, the generalized quasilinear systems are reduced to a semilinear one, whose associated functionals are well defined in the usual Sobolev space and satisfy the mountain-pass geometric. To obtain a ground state solution, the nonlinearities in [18, 25] are assumed to satisfy the monotone condition which is unnecessary in this paper and we believe that it is a partial extension which can reduce the restrictions on the nonlinearity.

Declarations

Acknowledgements

The author would like to thank the handling editors and anonymous referee for the help in the processing of the paper.

Funding

The author was supported by NSFC (Grant Nos. 11371158, 11771165), the program for Changjiang Scholars and Innovative Research Team in University (No. IRT13066).

Authors’ contributions

The author read and approved the final manuscript.

Competing interests

The author declares that he has no interests.

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)

Hubei Key Laboratory of Mathematical Sciences, Central China Normal University, Wuhan, P.R. China

(2)

School of Mathematics and Statistics, Central China Normal University, Wuhan, P.R. China

References

Berestycki, H., Lions, P.L.: Nonlinear scalar field equations I, existence of a ground state. Arch. Ration. Mech. Anal. 84, 313–346 (1983) MathSciNetMATHGoogle Scholar
Berestycki, H., Lions, P.L.: Nonlinear scalar field equations II, existence of infinitely many solutions. Arch. Ration. Mech. Anal. 82, 347–375 (1983) MathSciNetMATHGoogle Scholar
Rabinowitz, P.: On a class of nonlinear Schrödinger equations. Z. Angew. Math. Phys. 43, 270–291 (1992) MathSciNetView ArticleMATHGoogle Scholar
Bahrouni, A., Ounaies, H., Radulescu, V.D.: Infinitely many solutions for a class of sublinear Schrödinger equations with indefinite potentials. Proc. R. Soc. Edinb., Sect. A 145, 445–465 (2015) View ArticleMATHGoogle Scholar
Chaieb, M., Dhifli, A., Zermani, S.: Existence and asymptotic behavior of positive solutions of a semilinear elliptic system in a bounded domain. Opusc. Math. 36, 315–336 (2016) MathSciNetView ArticleMATHGoogle Scholar
Kurihara, S.: Large-amplitude quasi-solitons in superfluid films. J. Phys. Soc. Jpn. 50, 3262–3267 (1981) View ArticleGoogle Scholar
Laedke, E., Spatschek, K., Stenflo, L.: Evolution theorem for a class of perturbed envelope soliton solutions. J. Math. Phys. 24, 2764–2769 (1983) MathSciNetView ArticleMATHGoogle Scholar
Borovskii, A., Galkin, A.: Dynamical modulation of an ultrashort high-intensity laser pulse in matter. J. Exp. Theor. Phys. 77, 562–573 (1983) Google Scholar
Chen, X., Sudan, R.: Necessary and sufficient conditions for self-focusing of short ultraintense laser pulse in underdense plasma. Phys. Rev. Lett. 70, 2082–2085 (1993) View ArticleGoogle Scholar
Ritchie, B.: Relativistic self-focusing and channel formation in laser-plasma interactions. Phys. Rev. E 50, 687–689 (1994) View ArticleGoogle Scholar
De Bouard, A., Hayashi, N., Saut, J.: Global existence of small solutions to a relativistic nonlinear Schrödinger equation. Commun. Math. Phys. 189, 73–105 (1997) View ArticleMATHGoogle Scholar
Takeno, S., Homma, S.: Classical planar Heisenberg ferromagnet, complex scalar field and nonlinear excitations. Prog. Theor. Phys. 65, 172–189 (1981) View ArticleGoogle Scholar
Quispel, G., Capel, H.: Equation of motion for the Heisenberg spin chain. Physica A 110, 41–80 (1982) MathSciNetView ArticleGoogle Scholar
Kosevich, A., Ivanov, B., Kovalev, A.: Magnetic solitons in superfluid films. Phys. Rep. 194, 117–238 (1990) View ArticleGoogle Scholar
Poppenberg, M., Schmitt, K., Wang, Z.: On the existence of soliton solutions to quasilinear Schrödinger equations. Calc. Var. Partial Differ. Equ. 14, 329–344 (2002) View ArticleMATHGoogle Scholar
Hasse, R.: A general method for the solution of nonlinear soliton and kink Schrödinger equations. Z. Phys. B 37, 83–87 (1980) MathSciNetView ArticleGoogle Scholar
Makhankov, V.G., Fedyanin, V.K.: Nonlinear effects in quasi-one-dimensional models of condensed matter theory. Phys. Rep. 104, 1–86 (1984) MathSciNetView ArticleGoogle Scholar
Deng, Y., Peng, S., Yan, S.: Positive soliton solutions for generalized quasilinear Schrödinger equations with critical growth. J. Differ. Equ. 258, 115–147 (2015) View ArticleMATHGoogle Scholar
Shen, Y., Wang, Y.: Soliton solutions for generalized quasilinear Schrödinger equations. Nonlinear Anal. 80, 194–201 (2013) MathSciNetView ArticleMATHGoogle Scholar
Ding, Y., Lin, F.: Solutions of perturbed Schrödinger equations with critical nonlinearity. Calc. Var. Partial Differ. Equ. 30, 231–249 (2007) View ArticleMATHGoogle Scholar
Ding, Y., Liu, X.: Semiclassical solutions of Schrödinger equations with magnetic fields and critical nonlinearities. Manuscr. Math. 140, 51–82 (2013) View ArticleMATHGoogle Scholar
He, X., Qian, A., Zou, W.: Existence and concentration of positive solutions for quasilinear Schrödinger equations with critical growth. Nonlinearity 26, 3137–3168 (2013) MathSciNetView ArticleMATHGoogle Scholar
He, Y., Li, G.: Concentration soliton solutions for quasilinear Schrödinger equations involving critical Sobolev exponents. Discrete Contin. Dyn. Syst. 36, 731–762 (2016) MathSciNetView ArticleMATHGoogle Scholar
Li, F., Zhu, X., Liang, Z.: Multiple solutions to a class of generalized quasilinear Schrödinger equations with a Kirchhoff-type perturbation. J. Math. Anal. Appl. 443, 11–38 (2016) MathSciNetView ArticleMATHGoogle Scholar
Zhu, X., Li, F., Liang, Z.: Existence of ground state solutions to a generalized quasilinear Schrödinger–Maxwell system. J. Math. Phys. 57, 101505 (2016) MathSciNetView ArticleMATHGoogle Scholar
Li, Z., Zhang, Y.: Solutions for a class of quasilinear Schrödinger equations with critical Sobolev exponents. J. Math. Phys. 58, 021501 (2017) MathSciNetView ArticleMATHGoogle Scholar
Benci, V., Fortunato, D.: An eigenvalue problem for the Schrödinger–Maxwell equations. Topol. Methods Nonlinear Anal. 11, 283–293 (1998) MathSciNetView ArticleMATHGoogle Scholar
Benci, V., Fortunato, D.: Solitary waves of the nonlinear Klein–Gordon coupled with Maxwell equations. Rev. Math. Phys. 14, 409–420 (2002) MathSciNetView ArticleMATHGoogle Scholar
Zhao, L., Zhao, F.: On the existence of solutions for the Schrödinger–Poisson equations. J. Math. Anal. Appl. 346, 155–169 (2008) MathSciNetView ArticleMATHGoogle Scholar
Jiang, Y., Zhou, H.: Schrödinger–Poisson system with steep potential well. J. Differ. Equ. 251, 582–608 (2011) View ArticleMATHGoogle Scholar
Huang, L., Rocha, E., Chen, J.: Two positive solutions of a class of Schrödinger–Poisson system with indefinite nonlinearity. J. Differ. Equ. 255, 2463–2483 (2013) View ArticleMATHGoogle Scholar
Sun, J., Ma, S.: Ground state solutions for some Schrödinger–Poisson systems with periodic potentials. J. Differ. Equ. 260, 2119–2149 (2016) View ArticleMATHGoogle Scholar
Cortázar, C., Elgueta, M., García-Melián, J.: Analysis of an elliptic system with infinitely many solutions. Adv. Nonlinear Anal. 6, 1–12 (2017) MathSciNetView ArticleMATHGoogle Scholar
Filippucci, R., Vinti, F.: Coercive elliptic systems with gradient terms. Adv. Nonlinear Anal. 6, 165–182 (2017) MathSciNetMATHGoogle Scholar
Ruiz, D.: On the Schrodinger–Poisson–Slater system: behavior of minimizers, radial and nonradial cases. Arch. Ration. Mech. Anal. 198, 349–368 (2010) MathSciNetView ArticleMATHGoogle Scholar
Costa, D., Miyagaki, O.: Nontrival solutions for pertubations of the p-Laplacian on unbounded domains. J. Math. Anal. Appl. 193, 737–755 (1995) MathSciNetView ArticleMATHGoogle Scholar
Chabrowski, J.: Weak Convergence Methods for Semilinear Elliptic Equations. World Scientific, Singapore (1999) View ArticleMATHGoogle Scholar
Lions, P.L.: The concentration-compactness principle in the calculus of variation. The locally compact case. Part I. Ann. Inst. Henri Poincaré, Anal. Non Linéaire 1, 109–145 (1984) MathSciNetView ArticleMATHGoogle Scholar
Lions, P.L.: The concentration-compactness principle in the calculus of variation. The locally compact case. Part II. Ann. Inst. Henri Poincaré, Anal. Non Linéaire 1, 223–283 (1984) MathSciNetView ArticleMATHGoogle Scholar
Ekeland, I.: Nonconvex minimization problems. Bull. Am. Math. Soc. 1, 443–473 (1979) MathSciNetView ArticleMATHGoogle Scholar
Guo, Z.: Ground states for Kirchhoff equations without compactcondition. J. Differ. Equ. 259, 2884–2902 (2015) View ArticleMATHGoogle Scholar
Brézis, H., Lieb, E.: A relation between pointwise convergence of functions and convergence of functionals. Proc. Am. Math. Soc. 88, 486–490 (1983) MathSciNetView ArticleMATHGoogle Scholar