# Ground state and nodal solutions for critical Kirchhoff–Schrödinger–Poisson systems with an asymptotically 3-linear growth nonlinearity

## Abstract

In this paper, we consider the existence of a least energy nodal solution and a ground state solution, energy doubling property and asymptotic behavior of solutions of the following critical problem:

$$\textstyle\begin{cases} -(a+ b\int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2}\,dx)\Delta u+V(x)u+\lambda \phi u= \vert u \vert ^{4}u+ k f(u),&x\in \mathbb{R}^{3}, \\ -\Delta \phi =u^{2},&x\in \mathbb{R}^{3}. \end{cases}$$

By nodal Nehari manifold method, for each $$b>0$$, we obtain a least energy nodal solution $$u_{b}$$ and a ground-state solution $$v_{b}$$ to this problem when $$k\gg1$$, where the nonlinear function $$f\in C(\mathbb{R},\mathbb{R})$$. We also give an analysis on the behavior of $$u_{b}$$ as the parameter $$b\to 0$$.

## Introduction and main results

Our goal of this paper is to consider the existence of nodal solution and ground state solution of the following Kirchhoff–Schrödinger–Poisson system:

$$\textstyle\begin{cases} -(a+ b\int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2}\,dx)\Delta u +V(x)u+\lambda \phi u=f(x,u),&x\in \mathbb{R}^{3}, \\ -\Delta \phi =u^{2},&x\in \mathbb{R}^{3}, \end{cases}$$

where $$V(x)$$ is a smooth function and $$b>0, \lambda >0$$. When $$a=1$$, $$b=0$$, Kirchhoff–Schrödinger–Poisson equation reduces to the undermentioned Schrödinger–Poisson system

$$\textstyle\begin{cases} -\Delta u+V(x)u+\lambda \phi u=f(x,u),&x\in \mathbb{R}^{3}, \\ -\Delta \phi =u^{2},&x\in \mathbb{R}^{3}. \end{cases}$$
(1.1)

System (1.1) is derived from the time-varying Schrödinger equation, which describes the interaction of quantum (non-relativistic) particles with the electromagnetic field generated by motion. On the other hand, recently a great attention has been given to the so-called Kirchhoff equations

$$-\biggl(a+b \int _{\varOmega } \vert \nabla u \vert ^{2}\,dx\biggr) \Delta u=f(x,u),$$
(1.2)

where $$\varOmega \subset \mathbb{R}^{N}$$ is a bounded domain or $$\varOmega =\mathbb{R}^{N}$$, $$a>0$$, $$b>0$$ and u satisfies some boundary conditions. Problem (1.2) is related to the stationary analogue of the Kirchhoff–Schrödinger type equation

$$u_{tt}-\biggl(a+b \int _{\varOmega } \vert \nabla u \vert ^{2}\,dx\biggr) \Delta u=f(x,u),$$
(1.3)

which was introduced by Kirchhoff  as a generalization of the well-known D’Alembert wave equation

$$\rho \frac{\partial ^{2}u}{\partial t^{2}}- \biggl(\frac{p_{0}}{h}+ \frac{E}{2L} \int ^{L}_{0} \biggl\vert \frac{\partial u}{\partial x} \biggr\vert ^{2}\,dx \biggr)=f(x,u),$$
(1.4)

for free vibration of elastic strings. The Kirchhoff’s model takes into account the length variation of the string produced by the transverse vibration, so the nonlocal term appears. For more mathematical and physical background on Schrödinger–Poisson systems or Kirchhoff-type problems, we refer the readers to [1, 2, 13] and the references therein.

The appearance of nonlocal term not only makes it playing an important role in many physical applications, but also brings some difficulties and challenges in mathematical analysis. This fact makes the study of Kirchhoff–Schrödinger–Poisson system or similar problems particularly interesting. A lot of interesting results on the existence of nonlocal problems were obtained recently in, for example, [4, 5, 79, 11, 1317, 21, 25, 2729] and the cited references. We especially refer to the paper  for the existence of ground state positive solutions of Kirchhoff–Schrödinger-type equations with singular exponential nonlinearities in $$\mathbb{R}^{N}$$.

In the past few years, many researchers began to search for nodal solutions to Kirchhoff–Schrödinger-type equations or similar problems and got some interesting results. Zhong and Tang  considered the following subcritical Schrödinger–Poisson system:

$$\textstyle\begin{cases} -\Delta u+V(x)u+ k\phi u= \vert u \vert ^{2}u+\lambda f(u),&x\in \mathbb{R}^{3}, \\ -\Delta \phi =u^{2},&x\in \mathbb{R}^{3}, \end{cases}$$
(1.5)

where the nonlinearity $$f(u)$$ satisfies 3-linear growth condition at infinity and linear growth at zero. With the help of the nodal Nehari manifold, they studied the existence and asymptotic behavior of least energy nodal solution to system (1.5).

Wang  studied the existence of a least energy sign-changing solution for the following Kirchhoff-type equation:

$$\textstyle\begin{cases} -(a+ b\int _{\varOmega } \vert \nabla u \vert ^{2}\,dx)\Delta u= \vert u \vert ^{4}u+\lambda f(x,u), & x\in \varOmega , \\ u=0, & x\in \partial \varOmega , \end{cases}$$
(1.6)

where $$\varOmega \subset \mathbb{R}^{3}$$ is a bounded domain, $$\lambda , a, b>0$$ are fixed parameters. $$f(x,\cdot )$$ is continuously differentiable for a.e. $$x\in \varOmega$$. By using the constraint variational method and the degree theory, he got the existence of a least energy nodal solution to the Kirchhoff-type equation.

Wang, Zhang, and Guan  studied the following Schrödinger–Poisson system with critical growth:

$$\textstyle\begin{cases} -\Delta u+V(x)u+\lambda \phi u= \vert u \vert ^{4}u+\mu f(u),&x\in \mathbb{R}^{3}, \\ -\Delta \phi =u^{2},&x\in \mathbb{R}^{3}, \end{cases}$$

where $$\mu , \lambda >0$$, $$f\in C^{1}(\mathbb{R},\mathbb{R})$$. They got the existence and asymptotic behavior of a least energy sign-changing solution to the above system.

Motivated by the above references, in this paper, we study the existence of both ground state and least energy nodal solution for the following critical Kirchhoff–Schrödinger–Poisson system with asymptotically 3-linear growth nonlinearity:

$$\textstyle\begin{cases} -(a+ b\int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2}\,dx)\Delta u+V(x)u+\lambda \phi u= \vert u \vert ^{4}u+ k f(u),&x\in \mathbb{R}^{3}, \\ -\Delta \phi =u^{2},&x\in \mathbb{R}^{3}, \end{cases}$$
(1.7)

where a, b, k, λ are positive real numbers. Similar to , we suppose that $$V\in C(\mathbb{R}^{3},\mathbb{R}^{+})$$ and satisfies that $$E\hookrightarrow \hookrightarrow L^{p}(\mathbb{R}^{3})$$ (compact embedding) for $$2< p<6$$, and $$E\hookrightarrow L^{6}(\mathbb{R}^{3})$$ is continuous, where E is a Hilbert space defined by

\begin{aligned} E= \textstyle\begin{cases} H^{1}_{r}(\mathbb{R}^{3})=\{u\in H^{1}(\mathbb{R}^{3}):u(x)=u( \vert x \vert )\},& \text{if }V(x) \text{ is a constant}, \\ \{u\in D^{1,2}(\mathbb{R}^{3}):\int _{\mathbb{R}^{3}}V(x)u^{2} \,\mathrm{d}x< \infty \},&\text{if }V(x)\text{ is not a constant} \end{cases}\displaystyle \end{aligned}

with the inner product defined by

$$\langle u,v\rangle = \int _{\mathbb{R}^{3}}\bigl(a \nabla u\cdot \nabla v+ V(x)uv\bigr)\,dx,\quad \forall u, v\in E$$

and the norm $$\|\cdot \|$$:

$$\Vert u \Vert ^{2}= \int _{\mathbb{R}^{3}}\bigl(a \vert \nabla u \vert ^{2}+ V(x)u^{2}\bigr)\,dx.$$

As for the function f, we assume $$f\in C(\mathbb{R},\mathbb{R})$$ and satisfies the following hypotheses:

$$(f_{1})$$:

$$f(t)\cdot t>0$$ for $$t\ne 0$$;

$$(f_{2})$$:

$$\lim_{t\rightarrow \infty }\frac{f(t)}{t^{3}}=1$$ and $$\frac{f(t)}{t^{3}}<1$$ for all $$t\in \mathbb{R}\setminus \{0\}$$;

$$(f_{3})$$:

$$\frac{f(t)}{|t|^{3}}$$ is an increasing function in $$(-\infty ,0)$$ and $$(0,+\infty )$$.

### Remark 1.1

We note that under conditions ($$f_{1}$$)–($$f_{3}$$), it is easy to see

$$\lim_{t\rightarrow 0}\frac{f(t)}{t}=0.$$
(1.8)

The function $$f(t)=\frac{t^{5}}{1+t^{2}}$$ is an example satisfying all conditions ($$f_{1}$$)–($$f_{3}$$).

It is well known that the equation $$-\Delta \phi =u^{2}$$ can be solved as

\begin{aligned} \phi _{u}(x)=\frac{1}{4\pi } \int _{\mathbb{R}^{3}} \frac{u^{2}(y)}{ \vert x-y \vert }\,dy. \end{aligned}
(1.9)

So system (1.7) is merely a single equation on u:

\begin{aligned} -\biggl(a+ b \int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2}\,dx\biggr) \Delta u+V(x)u+\lambda \phi _{u} u= \vert u \vert ^{4}u+ k f(u), \quad x\in \mathbb{R}^{3}. \end{aligned}
(1.10)

Based on the results above, the energy functional associated with system (1.7) and so with (1.10) is defined by

\begin{aligned} J^{b}_{ k}(u)&=\frac{1}{2} \int _{\mathbb{R}^{3}}\bigl( a \vert \nabla u \vert ^{2}+V(x)u^{2}\bigr)\,dx+ \frac{b}{4}\biggl( \int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2}\,dx \biggr)^{2} \\ &\quad {}+\frac{\lambda }{4} \int _{\mathbb{R}^{3}}\phi _{u} u^{2}\,dx- k \int _{ \mathbb{R}^{3}}F(u)\,dx-\frac{1}{6} \int _{\mathbb{R}^{3}} \vert u \vert ^{6}\,dx \end{aligned}

for any $$u\in E$$. Moreover, under our conditions, $$J^{b }_{ k}(u)$$ belongs to $$C^{1}(E,\mathbb{R})$$, and the Fréchet derivative of $$J^{b}_{ k}$$ is

\begin{aligned} \bigl\langle \bigl(J^{b}_{ k}\bigr)'(u),v \bigr\rangle & = \int _{\mathbb{R}^{3}}\bigl({ a} \nabla u\cdot \nabla v+V(x)uv\bigr)\,dx+b \biggl( \int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2}\,dx\biggr) \biggl( \int _{\mathbb{R}^{N}}\nabla u\cdot \nabla v\,dx\biggr) \\ &\quad {}+\lambda \int _{\mathbb{R}^{3}}\phi _{u}uv\,dx - k \int _{\mathbb{R}^{3}}f(u)v\,dx- \int _{\mathbb{R}^{3}} \vert u \vert ^{4}uv\,dx \end{aligned}

for any $$u,v\in E$$.

As it is well known, if $$u\in E$$ is a solution of system (1.7) and $$u^{\pm }\neq 0$$, then u is a nodal solution of system (1.7), where

$$u^{+}=\max \bigl\{ u(x),0\bigr\} ,\qquad u^{-}=\min \bigl\{ u(x),0\bigr\} .$$

Note that, since system (1.7) involved pure critical nonlinearity $$|u|^{4}u$$, it will prevent us from using the standard arguments as in [3, 12, 19, 22]. Hence, we need to show some techniques to overcome the lack of compactness in $$E\hookrightarrow L^{6}(\mathbb{R}^{3})$$.

The main results can be stated as follows.

### Theorem 1.1

Suppose that$$(f_{1})$$$$(f_{3})$$are satisfied. Then there exists$$k^{\star }>0$$such that, for all$$k\geq k^{\star }$$, system (1.7) has a least energy nodal solution$$u_{b}$$, which has precisely two nodal domains.

### Remark 1.2

The least energy nodal solution $$u_{b}$$ is a solution of (1.7) satisfying

$$J^{b}_{ k}(u_{b })=\inf_{u\in \mathcal{M}_{k}^{b }}J^{ b }_{ k}(u),$$

where $$\mathcal{M}_{k}^{b }$$ is defined by (2.1) in the next section. We recall that the nodal of a continuous function $$u:\mathbb{R}^{3}\to \mathbb{R}$$ is the surface $$u^{-1}(0)$$. Every connected component of $$\mathbb{R}^{3}\setminus u^{-1}(0)$$ is called a nodal domain.

### Theorem 1.2

Suppose that$$(f_{1})$$$$(f_{3})$$are satisfied. Then there exists$$k^{\star \star }>0$$such that, for all$$k\geq k^{\star \star }$$, the$$c^{\ast }>0$$is achieved and

$$J^{b }_{ k}({ u_{b }})>2c^{\ast },$$

where$$c^{\ast }=\inf_{u\in \mathcal{N}_{ k}^{b }}J^{b }_{ k}(u)$$, $$\mathcal{N}_{ k}^{b}=\{u\in H\backslash \{0\}|\langle (J^{ b }_{ k})'(u),u \rangle =0 \}$$, and$$u_{b }$$is the least energy nodal solution obtained in Theorem 1.1. In particular, $$c^{\ast }>0$$is achieved either by a positive or a negative function$$v_{b }$$which is a ground state solution of system (1.7).

### Theorem 1.3

Suppose that$$(f_{1})$$$$(f_{3})$$are satisfied. Then there exists$$k^{\star \star \star }>0$$such that, for all$$k\geq k^{\star \star \star }$$, for any least energy nodal solution sequence$$\{u_{b _{n}}\}$$with$$b _{n}\rightarrow 0$$as$$n\rightarrow \infty$$, there exists a subsequence, still denoted by$${ \{u_{b _{n}}\}}$$, such that$$u_{b _{n}}$$converges to$$u_{0}$$weakly inEas$$n\rightarrow \infty$$, where$$u_{0}$$is a least energy nodal solution of the following problem:

\begin{aligned} \textstyle\begin{cases} -a\Delta u+V(x)u+\lambda\phi u=|u|^{4}u+ k f(u), ~\ &x\in\mathbb{R}^{3},\\ -\Delta\phi=u^{2},&x\in\mathbb{R}^{3}. \end{cases}\displaystyle \end{aligned}
(1.11)

Comparing with the literature works, the above three results can be regarded as a generalization of those in [12, 19, 20]. As for Kirchhoff–Schrödinger–Poisson equation, to the best of our knowledge, few results involved the existence and asymptotic behavior of ground state nodal solutions in case of critical growth. It is worth noting that the Brower degree method used in [20, 23] is strictly dependent on the nonlinearity $$f\in C^{1}(\mathbb{R},\mathbb{R})$$, so we have to find new ways to solve our model where we only allow $$f\in C(\mathbb{R},\mathbb{R})$$. On the other hand, in our modeling, both of the nonlocal terms $$\int _{\mathbb{R}^{3}}|\nabla u|^{2}\,dx$$ and ϕu appear, we need to overcome the difficulties caused by the nonlocal terms under a uniform variational framework. It is also due to the lack of compactness embedded in full space that we cannot use the method in . Thankfully, after appropriate modifications, the deformation lemma used in  can be applied to get the existence of a least energy nodal solution of the Kirchhoff–Schrödinger–Poisson system.

## Some technical lemmas

To fix some notations, the letter C, $$C_{i}$$ will be repeatedly used to denote various positive constants whose exact values are irrelevant. $$|\cdot|_{p}$$ denote the norm in $$L^{p}(\mathbb{R}^{3})$$ for $$p>1$$.

We first list some properties of $$\phi _{u}$$ for our use, one can find the details in [14, 26].

### Proposition 2.1

For any$$u\in E$$, we have

1. (i)

there exists$$C>0$$such that

$$\int _{\mathbb{R}^{3}}\phi _{u}u^{2}\,dx\leq C \Vert u \Vert ^{4}\quad \forall u \in E;$$
2. (ii)

$$\phi _{u}\geq 0$$, $$\forall u\in E$$;

3. (iii)

$$\phi _{tu}=t^{2}\phi _{u}$$, $$\forall t>0$$and$$u\in E$$;

4. (iv)

if$$u_{n}\rightharpoonup u$$inE, then$$\phi _{u_{n}}\rightharpoonup \phi _{u}$$in$$D^{1,2}(\mathbb{R}^{3})$$and

\begin{aligned} \int _{\mathbb{R}^{3}}\phi _{u_{n}}u_{n}^{2} \,dx\rightarrow \int _{ \mathbb{R}^{3}}\phi _{u}u^{2}\,dx. \end{aligned}

For fixed $$u\in E$$ with $$u^{\pm }\neq 0$$, the function $$\psi _{u}:[0,\infty )\times [0,\infty )\rightarrow \mathbb{R}$$ and the mapping $$W_{u}:[0,\infty )\times [0,\infty )\rightarrow \mathbb{R}^{2}$$ are well defined by

\begin{aligned}& \psi _{u}(s,t)=J^{b }_{ k}\bigl(su^{+}+tu^{-} \bigr), \\& W_{u}(s,t)=\bigl(\bigl\langle \bigl(J^{b }_{ k} \bigr)'\bigl(su^{+}+tu^{-}\bigr),su^{+} \bigr\rangle , \bigl\langle \bigl(J^{b }_{ k} \bigr)'\bigl(su^{+}+tu^{-}\bigr),tu^{-} \bigr\rangle \bigr), \end{aligned}

and

\begin{aligned} \mathcal{M}_{k}^{b }=\bigl\{ u\in E,u^{\pm }\neq 0 \text{ and } \bigl\langle \bigl(J^{b }_{ k} \bigr)'(u),u^{+}\bigr\rangle =\bigl\langle \bigl(J^{b }_{ k}\bigr)'(u),u^{-} \bigr\rangle =0\bigr\} . \end{aligned}
(2.1)

### Lemma 2.1

Assume that$$(f_{1})$$$$(f_{3})$$are satisfied, if$$u\in E$$with$$u^{\pm }\neq 0$$, then$$\psi _{u}$$has the following properties:

1. (i)

The pair$$(s,t)$$is a critical point of$$\psi _{u}$$with$$s,t>0 \Leftrightarrow s u^{+}+t u^{-}\in \mathcal{M}_{k}^{b }$$;

2. (ii)

The function$$\psi _{u}$$has a unique critical point$$(s_{u}, t_{u})$$on$$(0,\infty )\times (0,\infty )$$, which is also the unique maximum point of$$\psi _{u}$$on$$[0,\infty )\times [0,\infty )$$; Furthermore, if$$\langle (J^{b }_{ k})'(u),u^{\pm }\rangle \leq 0$$, then$$0< s_{u}$$, $$t_{u}\leq 1$$.

### Proof

(i) By the definition of $$\psi _{u}$$, we have that

\begin{aligned} \nabla \psi _{u}(s,t)&=\biggl(\frac{\partial \psi _{u}}{\partial s}, \frac{\partial \psi _{u}}{\partial t} \biggr) \\ &=\biggl(\frac{1}{s}\bigl\langle \bigl(J^{b }_{ k} \bigr)'\bigl(s u^{+}+t u^{-}\bigr),s u^{+} \bigr\rangle ,\frac{1}{t}\bigl\langle \bigl(J^{b }_{ k}\bigr)'\bigl(s u^{+}+t u^{-}\bigr),{ t u^{-}} \bigr\rangle \biggr). \end{aligned}

From the definition, item (i) is obvious.

(ii) It is easy to see

\begin{aligned} &\bigl\langle \bigl(J^{b }_{ k} \bigr)'\bigl(su^{+}+tu^{-}\bigr),su^{+} \bigr\rangle \\ &\quad =s^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+b s^{4}\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\biggr)^{2}+bs^{2}t^{2}\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\biggr) \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx\biggr) \\ &\qquad{}+s^{4}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx+s^{2}t^{2} \lambda \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx -s^{6} \int _{ \mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx \\ &\qquad{}- k \int _{\mathbb{R}^{3}}f\bigl(su^{+}\bigr)su^{+}\,dx \end{aligned}
(2.2)

and

\begin{aligned} &\bigl\langle \bigl(J^{b }_{ k} \bigr)'\bigl(su^{+}+tu^{-}\bigr),tu^{-} \bigr\rangle \\ &\quad =t^{2} \bigl\Vert u^{-} \bigr\Vert ^{2}+bt^{4} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \biggr)^{2} \\ &\qquad{}+bs^{2}t^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \int _{ \mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx +t^{4}\lambda \int _{\mathbb{R}^{3}} \phi _{u^{-}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx \\ &\qquad{}+s^{2}t^{2}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx -t^{6} \int _{\mathbb{R}^{3}} \bigl\vert u^{-} \bigr\vert ^{6}\,dx - k \int _{\mathbb{R}^{3}}f\bigl(tu^{-}\bigr)tu^{-}\,dx. \end{aligned}
(2.3)

From $$(f_{1})$$ and $$(f_{2})$$, for any $$\varepsilon >0$$, there is $$C_{\varepsilon }>0$$ satisfying

\begin{aligned} \bigl\vert f(t) \bigr\vert \leq \varepsilon \vert t \vert +C_{\varepsilon } \vert t \vert ^{4} \end{aligned}
(2.4)

for all $$t\in \mathbb{R}$$. From the Sobolev embedding theorem it follows that

\begin{aligned} &\bigl\langle \bigl(J^{b }_{ k}\bigr)' \bigl(su^{+}+tu^{-}\bigr),su^{+}\bigr\rangle \\ &\quad \geq s^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}-s^{6} \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx- k \varepsilon s^{2} \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx- k C_{ \varepsilon } s^{q} \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{5}\,dx \\ &\quad \geq s^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}-C_{1}s^{6} \bigl\Vert u^{+} \bigr\Vert ^{6}- k\varepsilon C_{2}s^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}- k C_{\varepsilon } C_{3}s^{q} \bigl\Vert u^{+} \bigr\Vert ^{5} \\ &\quad \geq (1- k\varepsilon C_{4})s^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}-C_{4}s^{6} \bigl\Vert u^{+} \bigr\Vert ^{6}- k C_{4}s^{5} \bigl\Vert u^{+} \bigr\Vert ^{5}. \end{aligned}

By choosing $$\varepsilon >0$$ such that $$(1- k\varepsilon C_{4})>0$$, we can infer that

\begin{aligned} \bigl\langle \bigl(J^{b }_{ k}\bigr)' \bigl(su^{+}+tu^{-}\bigr),su^{+}\bigr\rangle >0 \end{aligned}

for $$0< s\ll 1$$ and all $$t\geq 0$$. Similarly, there holds

\begin{aligned} \bigl\langle \bigl(J^{b}_{ k}\bigr)'\bigl(su^{+}+tu^{-}\bigr),tu^{-}\bigr\rangle >0 \end{aligned}

for $$0< t\ll 1$$ and all $$s\geq 0$$. Hence, there exists $$\delta _{1}>0$$ such that

\begin{aligned} \bigl\langle \bigl(J^{b }_{ k} \bigr)'\bigl(\delta _{1}u^{+}+tu^{-} \bigr), \delta _{1}u^{+} \bigr\rangle >0,\qquad \bigl\langle \bigl(J^{b }_{ k}\bigr)'\bigl(su^{+}+ \delta _{1}u^{-}\bigr), \delta _{1}u^{-} \bigr\rangle >0 \end{aligned}
(2.5)

for all $$s\geq 0$$, $$t\geq 0$$. It is worth noting that assumption $$(f_{1})$$ implies

\begin{aligned} F(t)\geq 0, \quad t\in \mathbb{R}. \end{aligned}
(2.6)

Thus, choosing $$s=\delta _{2}'>\delta _{1}$$, it follows that, for $$t\in [\delta _{1},\delta _{2}']$$ and $$\delta _{2}'\gg 1$$,

\begin{aligned} &\bigl\langle \bigl(J^{b }_{ k}\bigr)'\bigl( \delta _{2}'u^{+}+tu^{-}\bigr),\delta _{2}'u^{+} \bigr\rangle \\ &\quad \leq \bigl({\delta _{2}'}\bigr)^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+b\bigl({\delta _{2}'} \bigr)^{4} \bigl\Vert u^{+} \bigr\Vert ^{4}+b \bigl({ \delta _{2}'}\bigr)^{4} \bigl\Vert u^{+} \bigr\Vert ^{2} \bigl\Vert u^{-} \bigr\Vert ^{2} \\ &\qquad{}+\bigl({\delta _{2}'}\bigr)^{4}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx+ \bigl({\delta _{2}'} \bigr)^{4}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx-\bigl({ \delta _{2}'} \bigr)^{6} \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx\\ &\quad \leq 0. \end{aligned}

Analogously, one can show that

\begin{aligned} &\bigl\langle \bigl(J^{b }_{ k}\bigr)' \bigl(su^{+}+tu^{-}\bigr),tu^{-}\bigr\rangle \\ &\quad \leq t^{2} \bigl\Vert u^{-} \bigr\Vert ^{2}+bt^{4} \bigl\Vert u^{-} \bigr\Vert ^{4}+bs^{2}t^{2} \bigl\Vert u^{+} \bigr\Vert ^{2} \bigl\Vert u^{-} \bigr\Vert ^{2} \\ &\qquad{}+t^{4}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx+s^{2}t^{2} \lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx-t^{6} \int _{ \mathbb{R}^{3}} \bigl\vert u^{-} \bigr\vert ^{6}\,dx. \end{aligned}

Choosing $$\delta _{2}>\delta _{2}'\gg 1$$, we deduce

\begin{aligned} \bigl\langle \bigl(J^{b }_{ k} \bigr)'\bigl(\delta _{2}u^{+}+tu^{-} \bigr),\delta _{2}u^{+} \bigr\rangle < 0,\qquad \bigl\langle \bigl(J^{b }_{ k}\bigr)'\bigl(su^{+}+ \delta _{2}u^{-}\bigr), \delta _{2}u^{-} \bigr\rangle < 0 \end{aligned}
(2.7)

for all $$s,t\in [\delta _{1},\delta _{2}]$$.

From (2.5) and (2.7), the assumptions of Miranda’s theorem (see Lemma 2.4 in ) are satisfied. Thus there is $$(s_{u}, t_{u})\in (0,\infty )\times (0,\infty )$$ satisfying $$W_{u}(s_{u},t_{u})=(0,0)$$. So $$s_{u}u^{+}+t_{u}u^{-}\in \mathcal{M}_{k}^{b }$$.

Now we turn to proving that the pair $$(s_{u}, t_{u})$$ is unique. We first suppose that $$u\in \mathcal{M}_{k}^{b }$$, thus

\begin{aligned} & \bigl\Vert u^{+} \bigr\Vert ^{2}+b\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\biggr)^{2}+b \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\cdot \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx +\lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx \\ &\quad{}+\lambda \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx = \int _{ \mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx+ k \int _{\mathbb{R}^{3}}f\bigl(u^{+}\bigr)u^{+}\,dx \end{aligned}
(2.8)

and

\begin{aligned} & \bigl\Vert u^{-} \bigr\Vert ^{2}+b\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx\biggr)^{2} +b \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\cdot \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx +\lambda \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx \\ &\quad{}+\lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx= \int _{ \mathbb{R}^{3}} \bigl\vert u^{-} \bigr\vert ^{6}\,dx+ k \int _{\mathbb{R}^{3}}f\bigl(u^{-}\bigr)u^{-}\,dx. \end{aligned}
(2.9)

We will show that the pair $$(s_{u},t_{u})=(1,1)$$ is the unique one such that $$s_{u}u^{+}+t_{u}u^{-}\in \mathcal{M}_{k}^{b }$$. Let $$(s_{0},t_{0})$$ be a pair of numbers such that $$s_{0}u^{+}+t_{0}u^{-}\in \mathcal{M}_{k}^{b }$$ with $$0< s_{0}\leq t_{0}$$. We have

\begin{aligned} & s_{0}^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+bs_{0}^{4}\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\biggr)^{2}+bs_{0}^{2}t_{0}^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\cdot \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &\qquad{}+s_{0}^{4}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx+s_{0}^{2}t_{0}^{2} \lambda \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx \\ &\quad =s_{0}^{6} \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx+ k \int _{\mathbb{R}^{3}}f\bigl(s_{0}u^{+} \bigr)s_{0}u^{+}\,dx \end{aligned}
(2.10)

and

\begin{aligned} & t_{0}^{2} \bigl\Vert u^{-} \bigr\Vert ^{2} +bt_{0}^{4}\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx\biggr)^{2}+bs_{0}^{2}t_{0}^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\cdot \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &\qquad{}+t_{0}^{4}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx+s_{0}^{2}t_{0}^{2} \lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx \\ &\quad =t_{0}^{6} \int _{\mathbb{R}^{3}} \bigl\vert u^{-} \bigr\vert ^{6}\,dx+ k \int _{\mathbb{R}^{3}}f\bigl(t_{0}u^{-} \bigr)t_{0}u^{-}\,dx. \end{aligned}
(2.11)

By comparing (2.9) and (2.11), we deduce

\begin{aligned} &\frac{ \Vert u^{-} \Vert ^{2}}{t_{0}^{2}}+b \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \biggr)^{2}+b \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \cdot \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &\qquad{}+\lambda \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx+\lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx \\ &\quad \geq t_{0}^{2} \int _{\mathbb{R}^{3}} \bigl\vert u^{-} \bigr\vert ^{6}\,dx+ k \int _{ \mathbb{R}^{3}} \biggl[\frac{f(t_{0}u^{-})}{(t_{0}u^{-})^{3}} \biggr] \bigl(u^{-}\bigr)^{4}\,dx. \end{aligned}
(2.12)

Combining (2.9) with (2.12), one has that

$$\biggl(\frac{1}{t_{0}^{2}}-1 \biggr) \bigl\Vert u^{-} \bigr\Vert ^{2} \geq \bigl(t_{0}^{2}-1\bigr) \int _{\mathbb{R}^{3}} \bigl\vert u^{-} \bigr\vert ^{6}\,dx+ k \int _{\mathbb{R}^{3}} \biggl[ \frac{f(t_{0}u^{-})}{(t_{0}u^{-})^{3}} -\frac{f(u^{-})}{(u^{-})^{3}} \biggr]\bigl(u^{-}\bigr)^{4}\,dx.$$

By using assumption $$(f_{3})$$, we get $$t_{0}\le 1$$. Analogously, from (2.8), (2.10), and $$0< s_{0}\leq t_{0}$$,

$$\biggl(\frac{1}{s_{0}^{2}}-1 \biggr) \bigl\Vert u^{+} \bigr\Vert ^{2} \leq \bigl(s_{0}^{2}-1\bigr) \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx+ k \int _{\mathbb{R}^{3}} \biggl[ \frac{f(s_{0}u^{+})}{(s_{0}u^{+})^{3}} -\frac{f(u^{+})}{(u^{+})^{3}} \biggr]\bigl(u^{+}\bigr)^{4}\,dx.$$

By using assumption $$(f_{3})$$, we get $$s_{0}\geq 1$$. Consequently, $$s_{0}=t_{0}=1$$.

In the case $$u\notin \mathcal{M}_{k}^{b }$$, we suppose that there are $$(s_{1},t_{1})$$, $$(s_{2},t_{2})$$ such that

$$u_{1}=s_{1}u^{+}+t_{1}u^{-} \in \mathcal{M}_{k}^{b },\qquad u_{2}=s_{2}u^{+}+t_{2}u^{-} \in \mathcal{M}_{k}^{b }.$$

Thus,

$$u_{2}=\biggl(\frac{s_{2}}{s_{1}}\biggr)s_{1}u^{+}+ \biggl(\frac{t_{2}}{t_{1}}\biggr)t_{1}u^{-} =\biggl( \frac{s_{2}}{s_{1}}\biggr)u_{1}^{+}+\biggl(\frac{t_{2}}{t_{1}} \biggr)u_{1}^{-}\in \mathcal{M}_{k}^{b }.$$

According to $$u_{1}\in \mathcal{M}_{k}^{b }$$ and the fact of the previous case, one has that

$$\frac{s_{2}}{s_{1}}=\frac{t_{2}}{t_{1}}=1.$$

Thus $$s_{1}=s_{2}$$, $$t_{1}=t_{2}$$. Therefore $$(s_{u}, t_{u})$$ is the unique critical point of $$\psi _{u}$$ in $$(0,\infty )\times (0,\infty )$$.

In the following, we show that the critical point $$(s_{u},t_{u})$$ of $$\psi _{u}$$ is its unique maximum point on $$[0,+\infty )\times [0,+\infty )$$. By definition

\begin{aligned} \psi _{u}(s,t)&=\frac{s^{2}}{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+\frac{bs^{4}}{4} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+ \frac{s^{4}}{4}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx- \frac{s^{6}}{6} \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx \\ &\quad{}- \int _{\mathbb{R}^{3}}F\bigl(su^{+}\bigr)\,dx+\frac{t^{2}}{2} \bigl\Vert u^{-} \bigr\Vert + \frac{bt^{4}}{4}\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx\biggr)^{2}+ \frac{t^{4}}{4}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx \\ &\quad{}-\frac{t^{6}}{6} \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx- \int _{ \mathbb{R}^{3}}F\bigl(tu^{-}\bigr)\,dx+\frac{s^{2}t^{2}}{4} \lambda \int _{ \mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx \\ &\quad{}+\frac{s^{2}t^{2}}{4}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx+ \frac{bs^{2}t^{2}}{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\cdot \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx. \end{aligned}

Now (2.6) implies that

$$\lim_{|(s,t)|\rightarrow \infty }\psi _{u}(s,t)=-\infty .$$

By contradiction, we suppose that the boundary point $$(0, t_{0})$$ is a maximum point of $$\psi _{u}$$ with $$t_{0}\geq 0$$. By direct computation, it follows that

\begin{aligned} (\psi _{u})'_{s}(s,t_{0})&=s \bigl\Vert u^{+} \bigr\Vert ^{2}+bs^{3}\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\biggr)^{2}+ast_{0}^{2}\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\biggr) \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx\biggr) \\ &\quad{}+s^{3}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx+ \frac{st_{0}^{2}}{2}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx \\ &\quad{}+\frac{st_{0}^{2}}{2}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx-s^{5} \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx- \int _{\mathbb{R}^{3}}f\bigl(su^{+}\bigr)u^{+}\,dx\\ & >0 \end{aligned}

when $$s\ll 1$$. It follows that $$\psi _{u}$$ is an increasing function with respect to s when $$s\ll 1$$, which is a contradiction. Analogously, $$\psi _{u}$$ cannot achieve its global maximum on the boundary point $$(s,0)$$ with $$s\geq 0$$.

In the remainder of our proof, we will prove that $$0< s_{u}$$, $$t_{u}\leq 1$$ when $$\langle (J^{b }_{ k})'(u),u^{\pm }\rangle \leq 0$$. Suppose $$s_{u}\geq t_{u}>0$$. One has

\begin{aligned} &s_{u}^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+bs_{u}^{4} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+bs_{u}^{4} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\cdot \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &\qquad{}+s_{u}^{4}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx+ s_{u}^{4} \lambda \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx \\ &\quad \geq s_{u}^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+bs_{u}^{4} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+bs_{u}^{2}t_{u}^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\cdot \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &\qquad{}+s_{u}^{4}\lambda \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx+ s_{u}^{2}t_{u}^{2} \lambda \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx \\ &\quad =s_{u}^{6} \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx+ k \int _{\mathbb{R}^{3}}f\bigl(s_{u}u^{+} \bigr)s_{u}u^{+}\,dx. \end{aligned}
(2.13)

In view of $$\langle (J^{b }_{ k})'(u),u^{+}\rangle \leq 0$$, one has that

\begin{aligned} & \bigl\Vert u^{+} \bigr\Vert ^{2}+b \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+b \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\cdot \int _{ \mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx+\lambda \int _{\mathbb{R}^{3}} \phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx \\ &\quad{}+\lambda \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx\leq \int _{ \mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx+ k \int _{\mathbb{R}^{3}}f\bigl(u^{+}\bigr)u^{+}\,dx. \end{aligned}
(2.14)

By comparing (2.13) and (2.14), it follows that

$$\biggl(\frac{1}{s_{u}^{2}}-1\biggr) \bigl\Vert u^{+} \bigr\Vert ^{2}\geq \bigl(s_{u}^{2}-1\bigr) \int _{ \mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx+ k \int _{\mathbb{R}^{3}} \biggl[ \frac{f(s_{u}u^{+})}{(s_{u}u^{+})^{3}} -\frac{f(u^{+})}{(u^{+})^{3}} \biggr]\bigl(u^{+}\bigr)^{4}\,dx.$$

It implies $$s_{u}\leq 1$$. Therefore $$0< s_{u}$$, $$t_{u}\leq 1$$. □

### Lemma 2.2

If$$u\in \mathcal{M}^{b }_{k}$$, then$$tu\notin \mathcal{M}^{b }_{k}$$for every$$t>0$$, $$t\neq 1$$. More precisely,

\begin{aligned} &\bigl\langle \bigl(J^{b }_{ k}\bigr)'(tu),tu^{\pm } \bigr\rangle >0\quad \textit{for } t\in (0,1), \\ &\bigl\langle \bigl(J^{b }_{ k}\bigr)'(tu),tu^{\pm } \bigr\rangle < 0\quad \textit{for } t>1. \end{aligned}

### Proof

From (2.2) and $$u\in \mathcal{M}^{b }_{k}$$, we have that

\begin{aligned} \bigl\langle \bigl(J^{b }_{ k}\bigr)'(tu),tu^{+} \bigr\rangle &=t^{2}\bigl(1-t^{2}\bigr) \bigl\Vert u^{+} \bigr\Vert ^{2}+t^{4}\bigl(1-t^{2} \bigr) \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx \\ &\quad{}+ k t^{4} \int _{\mathbb{R}^{3}} \biggl(f\bigl(u^{+}\bigr)- \frac{f(tu^{+})}{t^{3}} \biggr) u^{+}\,dx. \end{aligned}

According to $$(f_{3})$$, when $$0< t<1$$,

\begin{aligned} \bigl\langle \bigl(J^{b }_{ k}\bigr)'(tu),tu^{+} \bigr\rangle >0, \end{aligned}

while in the case $$t>1$$,

\begin{aligned} \bigl\langle \bigl(J^{b }_{ k}\bigr)'(tu),tu^{+} \bigr\rangle < 0. \end{aligned}

Similarly, it is easy to get

\begin{aligned} &\bigl\langle \bigl(J^{b }_{ k}\bigr)'(tu),tu^{-} \bigr\rangle >0\quad \text{for } t \in (0,1),\qquad \bigl\langle \bigl(J^{b }_{ k}\bigr)'(tu),tu^{-} \bigr\rangle < 0\quad \text{for } t>1. \end{aligned}

The proof is complete. □

### Lemma 2.3

Let$$c_{b }^{ k}=\inf_{u\in \mathcal{M}_{k}^{b }}J^{ b }_{ k}(u)$$, then we have that

\begin{aligned} \lim_{ k\rightarrow \infty } c_{b }^{ k}=0. \end{aligned}

### Proof

For any $$u\in \mathcal{M}_{k}^{b }$$, we can deduce

\begin{aligned} & \bigl\Vert u^{\pm } \bigr\Vert ^{2}+b \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{\pm } \bigr\vert ^{2}\,dx \biggr)^{2}+b \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx\cdot \int _{ \mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx +\lambda \int _{\mathbb{R}^{3}} \phi _{u^{\pm }} \bigl\vert u^{\pm } \bigr\vert ^{2}\,dx \\ &\quad{}+\lambda \int _{\mathbb{R}^{3}}\phi _{u^{\mp }} \bigl\vert u^{\pm } \bigr\vert ^{2}\,dx= k \int _{\mathbb{R}^{3}}f\bigl(u^{\pm }\bigr)u^{\pm }\,dx+ \int _{\mathbb{R}^{3}} \bigl\vert u^{ \pm } \bigr\vert ^{6}\,dx. \end{aligned}

Hence, in view of (2.4), it follows that

\begin{aligned} \bigl\Vert u^{\pm } \bigr\Vert ^{2}&\leq k \int _{\mathbb{R}^{3}}f\bigl(u^{\pm }\bigr)u^{\pm }\,dx+ \int _{ \mathbb{R}^{3}} \bigl\vert u^{\pm } \bigr\vert ^{6}\,dx \\ &\leq k\varepsilon C_{1} \bigl\Vert u^{\pm } \bigr\Vert ^{2}+ k C_{2} \bigl\Vert u^{\pm } \bigr\Vert ^{5}+C_{3} \bigl\Vert u^{\pm } \bigr\Vert ^{6}. \end{aligned}

Therefore, we have that

\begin{aligned} (1- k\varepsilon C_{1}) \bigl\Vert u^{\pm } \bigr\Vert ^{2}\leq k C_{2} \bigl\Vert u^{\pm } \bigr\Vert ^{5}+C_{3} \bigl\Vert u^{\pm } \bigr\Vert ^{6}. \end{aligned}

We now choose ε small enough such that $$(1- k\varepsilon C_{1})>0$$, so there is $$\rho >0$$ such that

$$\bigl\Vert u^{\pm } \bigr\Vert \geq \rho$$
(2.15)

for all $$u\in \mathcal{M}_{k}^{b }$$. For any $$u\in \mathcal{M}_{k}^{b }$$, in view of the definition of $$\mathcal{M}_{k}^{b }$$, $$\langle (J^{b }_{ k})'(u),u\rangle =0$$. From assumption $$(f_{3})$$, we have

\begin{aligned} f(t)t-4F(t)\geq 0, \end{aligned}
(2.16)

and $$f(t)t-4F(t)$$ is increasing in $$(0,+\infty )$$ and decreasing in $$(-\infty ,0)$$. Hence, one gets

\begin{aligned} J^{b }_{ k}(u)&=J^{b }_{ k}(u)- \frac{1}{4}\bigl\langle \bigl(J^{ b }_{ k} \bigr)'(u),u\bigr\rangle \\ &=\frac{1}{4} \Vert u \Vert ^{2}+\frac{1}{12} \int _{\mathbb{R}^{3}} \vert u \vert ^{6}\,dx+ \frac{ k}{4} \int _{\mathbb{R}^{3}}\bigl[f(u)u-4F(u)\bigr]\,dx \\ &\geq \frac{1}{4} \Vert u \Vert ^{2} \end{aligned}

for any $$u\in \mathcal{M}_{k}^{b }$$.

From the above discussion, we can see that $$c_{b }^{ k}=\inf_{u\in \mathcal{M}_{k}^{b }}J^{b }_{ k}(u)$$ is well defined.

Let $$u\in E$$ with $$u^{\pm }\neq 0$$ be fixed. According to Lemma 2.1, for each $$k>0$$, there exist $$s_{ k}, t_{ k}>0$$ such that $$s_{ k}u^{+}+t_{ k}u^{-}\in \mathcal{M}_{k}^{b }$$. Hence, by (2.6), the Sobolev embedding theorem and Proposition 2.1, we have

\begin{aligned} 0&\leq c^{ k}=\inf_{u\in \mathcal{M}_{k}^{b }}J^{b }_{ k}(u) \leq J^{b }_{ k}\bigl(s_{ k}u^{+}+t_{ k}u^{-} \bigr) \\ &\leq \frac{1}{2} \bigl\Vert s_{ k}u^{+}+t_{ k}u^{-} \bigr\Vert ^{2}+\frac{b}{4}\biggl( \int _{ \mathbb{R}^{3}} \bigl\vert \nabla \bigl(s_{ k}u^{+}+t_{ k}u^{-} \bigr) \bigr\vert ^{2}\,dx\biggr)^{2}\\ &\quad {}+ \frac{\lambda }{4} \int _{\mathbb{R}^{3}}\phi _{s_{ k}u^{+}+t_{ k}u^{-}} \bigl\vert s_{ k}u^{+}+t_{ k}u^{-} \bigr\vert ^{2}\,dx \\ &\leq { \frac{s_{ k}^{2}}{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+\frac{t_{ k}^{2}}{2} \bigl\Vert u^{-} \bigr\Vert ^{2}+Cs_{ k}^{4} \bigl\Vert u^{+} \bigr\Vert ^{4}+Ct_{ k}^{4} \bigl\Vert u^{-} \bigr\Vert ^{4} } \end{aligned}

for some constants $$C>0$$. We now define

\begin{aligned} \varPhi _{u}=\bigl\{ (s_{ k},t_{ k})\in [0,\infty ) \times [0,\infty ):W_{u}(s_{ k},t_{ k})=(0,0), k>0 \bigr\} . \end{aligned}

Hence we have that

\begin{aligned} s_{ k}^{6}& \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx+t_{ k}^{6} \int _{ \mathbb{R}^{3}} \bigl\vert u^{-} \bigr\vert ^{6}\,dx \\ &\leq s_{ k}^{6} \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx+t_{ k}^{6} \int _{ \mathbb{R}^{3}} \bigl\vert u^{-} \bigr\vert ^{6}\,dx+ k \int _{\mathbb{R}^{3}}f\bigl(s_{ k}u^{+} \bigr)s_{ k}u^{+}\,dx+ k \int _{\mathbb{R}^{3}}f\bigl(t_{ k}u^{-} \bigr)t_{ k}u^{-}\,dx \\ &= \bigl\Vert s_{ k}u^{+}+t_{ k}u^{-} \bigr\Vert ^{2}+b\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla \bigl(s_{ k}u^{+}+t_{ k}u^{-} \bigr) \bigr\vert ^{2}\,dx\biggr)^{2}+\lambda \int _{\mathbb{R}^{3}}\phi _{s_{ k}u^{+}+t_{ k}u^{-}} \bigl\vert s_{ k}u^{+}+t_{ k}u^{-} \bigr\vert ^{2}\,dx \\ &\leq s_{ k}^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+t_{ k}^{2} \bigl\Vert u^{-} \bigr\Vert ^{2}+Cs_{ k}^{4} \bigl\Vert u^{+} \bigr\Vert ^{4}+Ct_{ k}^{4} \bigl\Vert u^{-} \bigr\Vert ^{4} . \end{aligned}

It follows that $$\varPhi _{u}$$ is a bounded set. We suppose that $$k_{n}\rightarrow \infty$$ as $$n\rightarrow \infty$$. For $$(s_{ k_{n}}, t_{ k_{n}})\in \varPhi _{u}$$, there exist $$s_{0}$$ and $$t_{0}$$ such that

$$(s_{ k_{n}}, t_{ k_{n}})\rightarrow (s_{0}, t_{0})$$

as $$n\rightarrow \infty$$ (in the subsequence sense). We suppose that $$s_{0} > 0$$ or $$t_{0}> 0$$. Thanks to $$s_{ k_{n}}u^{+}+t_{ k_{n}}u^{-}\in \mathcal{M}_{b }^{ k_{n}}$$, we get

\begin{aligned} & \bigl\Vert s_{ k_{n}}u^{+}+t_{ k_{n}}u^{-} \bigr\Vert ^{2}+b\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla \bigl(s_{ k_{n}}u^{+}+t_{ k_{n}}u^{-} \bigr) \bigr\vert ^{2}\,dx\biggr)^{2} \\ &\qquad{}+\lambda \int _{\mathbb{R}^{3}}\phi _{s_{ k_{n}}u^{+}+t_{ k_{n}}u^{-}} \bigl\vert s_{ k_{n}}u^{+}+t_{ k_{n}}u^{-} \bigr\vert ^{2}\,dx \\ &\quad = \int _{\mathbb{R}^{3}} \bigl\vert s_{ k_{n}}u^{+}+t_{ k_{n}}u^{-} \bigr\vert ^{6}\,dx+ k_{n} \int _{\mathbb{R}^{3}}f\bigl(s_{ k_{n}}u^{+} +t_{ k_{n}}u^{-}\bigr) \bigl(s_{ k_{n}}u^{+}+t_{ k_{n}}u^{-} \bigr)\,dx. \end{aligned}
(2.17)

According to $$s_{ k_{n}}u^{+}\rightarrow s_{0}u^{+}$$ and $$t_{ k_{n}}u^{-}\rightarrow t_{0}u^{-}$$ in E, $$\int _{\mathbb{R}^{3}}|\nabla (s_{ k_{n}}u^{+}+t_{ k_{n}}u^{-})|^{2}\,dx \leq \|s_{ k_{n}}u^{+}+t_{ k_{n}}u^{-}\|^{2}$$, (2.4) and (2.6), so as $$n\rightarrow \infty$$, there holds

\begin{aligned} \int _{\mathbb{R}^{3}}f\bigl(s_{ k_{n}}u^{+}+t_{ k_{n}}u^{-} \bigr) \bigl(s_{ k_{n}}u^{+}+t_{ k_{n}}u^{-} \bigr)\,dx \rightarrow \int _{\mathbb{R}^{3}}f\bigl(s_{0}u^{+}+t_{0}u^{-} \bigr) \bigl(s_{0}u^{+}+t_{0}u^{-} \bigr)\,dx>0. \end{aligned}

Because $$k_{n}\rightarrow \infty$$ as $$n\rightarrow \infty$$ and $$\{s_{ k_{n}}u^{+}+t_{ k_{n}}u^{-}\}$$ is bounded in E, following the Sobolev embedding theorem, we have a contradiction with equality (2.17). Thus, $$s_{0}=t_{0}=0$$, and so $$\lim_{ k\rightarrow \infty } c_{b }^{ k}=0$$. □

### Lemma 2.4

There exists$$k^{\star }>0$$such that, for all$$k\geq k^{\star }$$, the infimum$$c_{b }^{ k}$$is achieved.

### Proof

In view of the definition of $$c_{b }^{ k}$$, we deduce that there exists a sequence $$\{u_{n}\}\subset \mathcal{M}^{ k}_{b }$$ satisfying

$$\lim_{n\rightarrow \infty }J^{b }_{ k}(u_{n})=c_{b }^{ k}.$$

Following from (2.8) and (2.9), $$\{u_{n}\}$$ is bounded in E. So in the subsequence sense, there exists $$u_{b }=u^{+}_{b }+u^{-}_{b }\in E$$ such that $$u_{n}\rightharpoonup u_{b }$$. Since the embedding $$E\hookrightarrow L^{p}(\mathbb{R}^{3})$$ is compact for $$p\in (2, 6)$$, we deduce

\begin{aligned} &u_{n}\rightarrow u_{b } \quad \text{in } L^{p} \bigl(\mathbb{R}^{3}\bigr), \forall p\in (2, 6), \\ &u_{n}(x)\rightarrow u_{b }(x) \quad \text{a.e. } x\in \mathbb{R}^{3}. \end{aligned}

Then we have

\begin{aligned} &u^{\pm }_{n}\rightharpoonup u_{b}^{\pm }\quad \text{in } E, \\ &u^{\pm }_{n}\rightarrow u_{b}^{\pm } \quad \text{in } L^{p}\bigl(\mathbb{R}^{3}\bigr), \\ &u^{\pm }_{n}(x)\rightarrow u_{b}^{\pm }(x) \quad \text{a.e. } x\in \mathbb{R}^{3}. \end{aligned}

Denote $$\beta :=\frac{(S)^{\frac{3}{2}}}{3}$$, where

$$S:=\inf_{u\in E\backslash \{0\}} \frac{ \Vert u \Vert ^{2}}{(\int _{\mathbb{R}^{3}} \vert u \vert ^{6}\,dx)^{\frac{1}{3}}}.$$

The Sobolev embedding theorem insures that $$\beta >0$$. Lemma 2.3 implies that there exists $$k^{\star }>0$$ such that $$c_{b }^{ k}<\beta$$ for all $$k\geq k^{\star }$$. Fix $$k\geq k^{\star }$$, in view of Lemma 2.1, we have

\begin{aligned} J^{b }_{ k}\bigl(su^{+}_{n}+tu ^{-}_{n}\bigr)\leq J^{b }_{ k}(u_{n}) \end{aligned}

for all $$s, t\in [0,+\infty )$$. Because $$u^{\pm }_{n}\rightharpoonup u_{b}^{\pm }$$ in E, E is a Hilbert space, we can deduce

\begin{aligned} \bigl\Vert u^{\pm }_{n} \bigr\Vert ^{2}- \bigl\Vert u^{\pm }_{n}-u_{b}^{\pm } \bigr\Vert ^{2}=2\bigl(u^{\pm }_{n},u_{b}^{\pm }\bigr)- \bigl\Vert u_{b}^{\pm } \bigr\Vert ^{2}, \end{aligned}

where we can assume that the sequence $$\{\|u^{\pm }_{n}\|\}$$ is convergent, so we have

$$\lim_{n\to \infty } \bigl\Vert u^{\pm }_{n} \bigr\Vert ^{2}=\lim_{n\to \infty } \bigl\Vert u^{\pm }_{n}-u_{b}^{\pm } \bigr\Vert ^{2}+ \bigl\Vert u_{b}^{\pm } \bigr\Vert ^{2}.$$

Obviously, we can let $$n\rightarrow \infty$$ in both sides of the above equation. On the other hand, by (2.4) we have

\begin{aligned} \int _{\mathbb{R}^{3}}F\bigl(su_{n}^{\pm }\bigr)\,dx \rightarrow \int _{\mathbb{R}^{3}}F\bigl(su_{b}^{ \pm }\bigr)\,dx. \end{aligned}

Thus, we get

\begin{aligned} &\liminf_{n\rightarrow \infty }J^{b }_{ k} \bigl(su^{+}_{n}+tu^{-}_{n}\bigr)\\ &\quad \geq \frac{s^{2}}{2}\lim_{n\rightarrow \infty }\bigl( \bigl\Vert u^{+}_{n}-u_{b}^{+} \bigr\Vert ^{2}+ \bigl\Vert u_{b}^{+} \bigr\Vert ^{2}\bigr) +\frac{t^{2}}{2}\lim _{n\rightarrow \infty }\bigl( \bigl\Vert u^{-}_{n}-u_{b}^{-} \bigr\Vert ^{2}+ \bigl\Vert u_{b}^{-} \bigr\Vert ^{2}\bigr) \\ &\qquad{}+\frac{bs^{2}}{4} \biggl[\liminf_{n\rightarrow \infty }\biggl( \int _{ \mathbb{R}^{3}} \bigl\vert \nabla u^{+}_{n} \bigr\vert ^{2}\,dx\biggr) \biggr]^{2}+\frac{bt^{2}}{4} \biggl[\liminf_{n\rightarrow \infty }\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-}_{n} \bigr\vert ^{2}\,dx\biggr) \biggr]^{2} \\ &\qquad{}+\frac{bs^{2}t^{2}}{4} \liminf_{n\rightarrow \infty }\biggl( \int _{ \mathbb{R}^{3}} \bigl\vert \nabla u^{+}_{n} \bigr\vert ^{2}\,dx\biggr)\cdot \liminf_{n\rightarrow \infty }\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-}_{n} \bigr\vert ^{2}\,dx\biggr) \\ &\qquad{}+\frac{\lambda s^{4}}{4}\liminf_{n\rightarrow \infty } \int _{ \mathbb{R}^{3}}\phi _{u_{n}^{+}} \bigl\vert u_{n}^{+} \bigr\vert ^{2}\,dx + \frac{\lambda t^{4}}{4}\liminf_{n\rightarrow \infty } \int _{ \mathbb{R}^{3}}\phi _{u_{n}^{-}} \bigl\vert u_{n}^{-} \bigr\vert ^{2}\,dx \\ &\qquad{}-\frac{s^{6}}{6}\lim_{n\rightarrow \infty }\bigl( \bigl\vert u^{+}_{n}-u_{b}^{+} \bigr\vert _{6}^{6}+ \bigl\vert u_{b}^{+} \bigr\vert _{6}^{6}\bigr)- \frac{t^{6}}{6}\lim_{n\rightarrow \infty }\bigl( \bigl\vert u^{-}_{n}-u_{b}^{-} \bigr\vert _{6}^{6}+ \bigl\vert u_{b}^{-} \bigr\vert _{6}^{6}\bigr) \\ &\qquad{}- k \int _{\mathbb{R}^{3}}F\bigl(su_{b}^{+}\bigr)\,dx- k \int _{\mathbb{R}^{3}}F\bigl(tu_{b}^{-}\bigr)\,dx \\ &\qquad{}+\frac{\lambda s^{2}t^{2}}{4}\liminf_{n\rightarrow \infty } \int _{ \mathbb{R}^{3}}\phi _{u_{n}^{+}} \bigl\vert u_{n}^{-} \bigr\vert ^{2}\,dx + \frac{\lambda s^{2}t^{2}}{4}\liminf_{n\rightarrow \infty } \int _{ \mathbb{R}^{3}}\phi _{u_{n}^{-}} \bigl\vert u_{n}^{+} \bigr\vert ^{2}\,dx. \end{aligned}

By using Fatou’s lemma, there holds

\begin{aligned} \liminf_{n\rightarrow \infty }J^{b }_{ k} \bigl(su^{+}_{n}+tu^{-}_{n}\bigr)& \geq J^{b }_{ k}\bigl(su_{b}^{+}+tu_{b}^{-}\bigr)+ \frac{s^{2}}{2}\lim_{n \rightarrow \infty } \bigl\Vert u^{+}_{n}-u_{b}^{+} \bigr\Vert ^{2} +\frac{t^{2}}{2}\lim_{n \rightarrow \infty } \bigl\Vert u^{-}_{n}-u_{b}^{-} \bigr\Vert ^{2} \\ &\quad{}-\frac{s^{6}}{6}\lim_{n\rightarrow \infty } \bigl\vert u^{+}_{n}-u_{b}^{+} \bigr\vert _{6}^{6}- \frac{t^{6}}{6}\lim_{n\rightarrow \infty } \bigl\vert u^{-}_{n}-u_{b}^{-} \bigr\vert _{6}^{6} \\ &=J^{b }_{ k}\bigl(su_{b}^{+}+tu_{b}^{-}\bigr)+ \frac{s^{2}}{2}A_{1} - \frac{s^{6}}{6}B_{1}+ \frac{t^{2}}{2}A_{2} -\frac{t^{6}}{6}B_{2}, \end{aligned}

where

\begin{aligned} &A_{1}=\lim_{n\rightarrow \infty } \bigl\Vert u^{+}_{n}-u_{b}^{+} \bigr\Vert ^{2},\qquad A_{2}= \lim_{n\rightarrow \infty } \bigl\Vert u^{-}_{n}-u_{b}^{-} \bigr\Vert ^{2}, \\ &B_{1}=\lim_{n\rightarrow \infty } \bigl\vert u^{+}_{n}-u_{b}^{+} \bigr\vert _{6}^{6},\qquad B_{2}= \lim _{n\rightarrow \infty } \bigl\vert u^{-}_{n}-u_{b}^{-} \bigr\vert _{6}^{6}. \end{aligned}

From the above fact, one has that

\begin{aligned} &J^{b }_{ k}\bigl(su_{b}^{+}+tu_{b}^{-} \bigr)+\frac{s^{2}}{2}A_{1} - \frac{s^{6}}{6}B_{1}+ \frac{t^{2}}{2}A_{2} -\frac{t^{6}}{6}B_{2}\leq c_{ b }^{ k} \end{aligned}
(2.18)

for all $$s \geq 0$$, $$t \geq 0$$.

Claim 1. $$u_{b}^{\pm }\neq 0$$. In fact, by contradiction, if $$u_{b}^{+}=0$$, we divide it into two cases.

Case 1:$$B_{1}=0$$. In this case, if $$A_{1}=0$$, in view of the fact (2.15), we obtain $$\|u_{b}^{+}\|>0$$, which is absurd. If $$A_{1}>0$$, we let $$t=0$$ in (2.18) that $$\frac{s^{2}}{2}A_{1}\leq c_{b }^{ k}$$ for all $$s\geq 0$$, which is false.

Case 2:$$B_{1}>0$$. In this case, by the definition of S, we deduce

\begin{aligned} \beta =\frac{(S)^{\frac{3}{2}}}{3}\leq \frac{1}{3} \biggl( \frac{A_{1}}{(B_{1})^{\frac{1}{3}}} \biggr)^{\frac{3}{2}}. \end{aligned}

On the other hand,

\begin{aligned} \frac{1}{3} \biggl(\frac{A_{1}}{(B_{1})^{\frac{1}{3}}} \biggr)^{ \frac{3}{2}}=\max _{s\geq 0} \biggl\{ \frac{s^{2}}{2}A_{1}- \frac{s^{6}}{6}B_{1} \biggr\} . \end{aligned}

Thanks to $$c_{b }^{ k}<\beta$$, by substituting $$t=0$$ into (2.18), we have that

\begin{aligned} \beta \leq \max_{s\geq 0}\biggl\{ \frac{s^{2}}{2}A_{1}- \frac{s^{6}}{6}B_{1} \biggr\} \le c_{b }^{ k}< \beta , \end{aligned}

which is a contradiction. Thus $$u_{b}^{+}\neq 0$$. Similarly, we also get $$u_{b}^{-}\neq 0$$. Therefore $$u_{b}^{\pm }\neq 0$$ as claimed.

Claim 2. $$B_{1}=B_{2}=0$$. We only prove $$B_{1}=0$$. By contradiction, we suppose that $$B_{1}>0$$. We have two cases.

Case 1:$$B_{2}>0$$. Let $$s_{a}$$ and $$t_{b}$$ be the numbers such that

\begin{aligned}& \frac{s_{a}^{2}}{2}A_{1}-\frac{s_{a}^{6}}{6}B_{1} =\max _{s\geq 0}\biggl\{ \frac{s^{2}}{2}A_{1}- \frac{s^{6}}{6}B_{1}\biggr\} , \\& \frac{t_{b}^{2}}{2}A_{2}-\frac{t_{b}^{6}}{6}B_{2} =\max _{t\geq 0}\biggl\{ \frac{t^{2}}{2}A_{2}- \frac{t^{6}}{6}B_{2}\biggr\} . \end{aligned}

Since $$\psi _{u}$$ is continuous, we have $$(s_{u},t_{u})\in [0, s_{a}]\times [0, t_{b}]$$ satisfying

\begin{aligned} \psi _{u}(s_{u},t_{u})=\max _{(s,t)\in [0, s_{a}]\times [0, t_{b}]} \psi _{u}(s,t). \end{aligned}

Note that if $$0< t\ll 1$$, we deduce

\begin{aligned} \psi _{u}(s,0)=J^{b}_{ k}\bigl(su_{b}^{+} \bigr)< J^{b }_{ k}\bigl(su_{b}^{+}\bigr)+J^{ b }_{ k} \bigl(tu_{b}^{-}\bigr)\leq J^{b }_{ k} \bigl(su_{b}^{+}+tu_{b}^{-}\bigr)=\psi _{u}(s,t) \end{aligned}

for all $$s\in [0, s_{a}]$$. Thus there is $$t_{0}\in [0, t_{b}]$$ such that

$$\psi _{u}(s,0)\leq \psi _{u}(s,t_{0})$$

for all $$s\in [0,s_{a}]$$. It follows that any point of the form $$(s, 0)$$ with $$0\leq s\leq s_{a}$$ is not the maximizer of $$\psi _{u}$$. Thus, $$(s_{u},t_{u})\notin [0, s_{a}]\times \{0\}$$. Similarly, it shows that $$(s_{u},t_{u})\notin \{0\}\times [0,t_{b}]$$. By direct computation, we get

\begin{aligned}& \frac{s^{2}}{2}A_{1} -\frac{s^{6}}{6}B_{1}>0, \end{aligned}
(2.19)
\begin{aligned}& \frac{t^{2}}{2}A_{2} -\frac{t^{6}}{6}B_{2}>0 \end{aligned}
(2.20)

for all $$s\in (0,s_{a}]$$, $$t\in (0,t_{b}]$$. Hence there hold

\begin{aligned}& \beta \leq \frac{s_{a}^{2}}{2}A_{1}-\frac{s_{a}^{6}}{6}B_{1} + \frac{t^{2}}{2}A_{2}-\frac{t^{6}}{6}B_{2}, \\& \beta \leq \frac{t_{b}^{2}}{2}A_{2}-\frac{t_{b}^{6}}{6}B_{2} + \frac{s^{2}}{2}A_{1}-\frac{s^{6}}{6}B_{1} \end{aligned}

for all $$s\in [0,s_{a}]$$, $$t\in [0,t_{b}]$$. In view of (2.18), it follows that

\begin{aligned} \psi _{u}(s,t_{b})\leq 0,\qquad \psi _{u}(s_{a},t) \leq 0 \end{aligned}

for all $$s\in [0,s_{a}]$$, $$t\in [0,t_{b}]$$. That is, $$(s_{u},t_{u})\notin \{s_{a}\}\times [0,t_{b}]$$ and $$(s_{u},t_{u})\notin \times [0,s_{a}]\times \{t_{b}\}$$. Hence, we can deduce that $$(s_{u},t_{u})\in (0, s_{a})\times (0, t_{b})$$. By Lemma 2.1, it follows that $$(s_{u},t_{u})$$ is a critical point of $$\psi _{u}$$. Thus, $$s_{u}u^{+}+t_{u}u^{-}\in \mathcal{M}_{k}^{b }$$. By (2.18), (2.19), and (2.20), we deduce

\begin{aligned} c_{b }^{ k}&\geq J^{b }_{ k} \bigl(s_{u}u_{b}^{+}+t_{u}u_{b}^{-}\bigr)+ \frac{s_{u}^{2}}{2}A_{1}-\frac{s_{u}^{6}}{6}B_{1} + \frac{t_{u}^{2}}{2}A_{2}-\frac{t^{6}_{u}}{6}B_{2} \\ &>J^{b }_{ k}\bigl(s_{u}u_{b}^{+}+t_{u}u_{b}^{-} \bigr) \\ &\geq c_{b }^{ k}. \end{aligned}

It is impossible. The proof of Case 1 is completed.

Case 2:$$B_{2}= 0$$. From the definition of $$J^{b }_{ k}$$, it is easy to show that there exists $$t_{0}\in [0,\infty )$$ such that $$J^{b }_{ k}(su_{b}^{+}+tu_{b}^{-})\leq 0$$ for all $$(s,t)\in [0,s_{a}]\times [t_{0},\infty )$$. Thus, there is $$(s_{u},t_{u})\in [0,s_{a}]\times [0,\infty )$$ satisfying

\begin{aligned} \psi _{u}(s_{u},t_{u})=\max _{(s,t)\in [0,s_{a}]\times [0,\infty )} \psi _{u}(s,t). \end{aligned}

We need to prove that $$(s_{u},t_{u})\in (0,s_{a})\times (0,\infty )$$. Similarly, it is noticed that $$\psi _{u}(s,0)<\psi _{u}(s,t)$$ for $$s\in [0,s_{a}]$$ and $$0< t\ll 1$$, that is, $$(s_{u},t_{u})\notin [0,s_{a}]\times \{0\}$$. Also, for s small enough, we get $$\psi _{u}(0,t)<\psi _{u}(s,t)$$ for $$t\in [0,\infty )$$, that is, $$(s_{u},t_{u})\notin \{0\}\times [0,\infty )$$. We note that

\begin{aligned} \beta &\leq \frac{s_{a}^{2}}{2}A_{1}-\frac{s_{a}^{6}}{6}B_{1}+ \frac{t^{2}}{2}A_{2} \end{aligned}

for all $$t\in [0,\infty )$$. Thus also from (2.20) and $$B_{2}=0$$, we have $$\psi _{u}(s_{a},t)\leq 0$$ for all $$t\in [0,\infty )$$. Hence, $$(s_{u},t_{u})\notin \{s_{a}\}\times [0,\infty )$$. That is, $$(s_{u},t_{u})$$ is an inner maximizer of $$\psi _{u}$$ in $$[0,s_{a})\times [0,\infty )$$. So $$s_{u}u^{+}+t_{u}u^{-}\in \mathcal{M}_{k}^{b }$$. Hence, by using (2.19), we obtain

\begin{aligned} c_{b }^{ k}&\geq J^{b }_{ k} \bigl(s_{u}u_{b}^{+}+t_{u}u_{b}^{-}\bigr)+ \frac{s_{u}^{2}}{2}A_{1}-\frac{s_{u}^{6}}{6}B_{1} + \frac{t_{u}^{2}}{2}A_{2}-\frac{t^{6}_{u}}{6}B_{2} \\ &>J^{b }_{ k}\bigl(s_{u}u_{b}^{+}+t_{u}u_{b}^{-} \bigr) \\ &\geq c_{b }^{ k}, \end{aligned}

which is a contradiction. It is similar for $$B_{2}=0$$. From the above discussion, we know that Claim 2 is true.

Claim 3.$$c^{ k}_{b }$$ is achieved. Since $$u_{b}^{\pm }\neq 0$$, by Lemma 2.1, there are $$s_{u}, t_{u}>0$$ such that $$\widetilde{u}:=s_{u}u_{b}^{+}+t_{u}u_{b}^{-}\in \mathcal{M}^{ k}_{b }$$. On the other hand, $$u_{n}\rightharpoonup u_{b}$$ in E, then $$\int _{\mathbb{R}^{3}}(V(x)u_{n}^{2})\,dx\rightarrow \int _{\mathbb{R}^{3}}(V(x)u_{b}^{2})\,dx$$ and $$\liminf_{n\rightarrow \infty }\|u_{n}\|\ge \|u_{b}\|$$, so we get

\begin{aligned} \liminf_{n\rightarrow \infty } \int _{\mathbb{R}^{3}} \vert \nabla u_{n} \vert ^{2}\,dx \geq \int _{\mathbb{R}^{3}} \vert \nabla u_{b} \vert ^{2}\,dx. \end{aligned}

On the other hand, by (2.4), we deduce

\begin{aligned} \lim_{n\rightarrow \infty } \int _{\mathbb{R}^{3}}f\bigl(u_{n}^{\pm } \bigr)u_{n}^{ \pm }\,dx= \int _{\mathbb{R}^{3}}f(u_{b})^{\pm }u_{b}^{\pm }\,dx. \end{aligned}

Thanks to Proposition 2.1, we get

\begin{aligned} \bigl\langle \bigl(J^{b }_{ k}\bigr)'(u_{b}),u_{b}^{\pm } \bigr\rangle & \leq \liminf_{n\rightarrow \infty } \bigl\Vert u^{\pm }_{n} \bigr\Vert ^{2}+b \biggl[ \liminf_{n\rightarrow \infty }\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{\pm }_{n} \bigr\vert ^{2}\,dx\biggr) \biggr]^{2}\\ &\quad {}+b\liminf _{n\rightarrow \infty }\biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+}_{n} \bigr\vert ^{2}\,dx\biggr)\cdot \liminf_{n\rightarrow \infty }\biggl( \int _{ \mathbb{R}^{3}} \bigl\vert \nabla u^{-}_{n} \bigr\vert ^{2}\,dx\biggr) \\ &\quad{}+\liminf_{n\rightarrow \infty } \int _{\mathbb{R}^{3}}\phi _{u_{n}} \bigl\vert u_{n}^{ \pm } \bigr\vert ^{2}\,dx -\lim _{n\rightarrow \infty } \int _{\mathbb{R}^{3}}f\bigl(u_{n}^{ \pm } \bigr)u_{n}^{\pm }\,dx -\lim_{n\rightarrow \infty } \int _{\mathbb{R}^{3}} \bigl\vert u_{n}^{ \pm } \bigr\vert ^{6} \\ & \leq \lim_{n\rightarrow \infty }\bigl\langle \bigl(J^{b }_{ k} \bigr)'(u_{n}),u_{n}^{ \pm }\bigr\rangle =0. \end{aligned}

Therefore from Lemma 2.2 we have that $$0< s_{u}$$, $$t_{u}\leq 1$$. Since $$u_{n}\in \mathcal{M}^{ k}_{b }$$, $$B_{1}=B_{2}=0$$ and $$\|u\|$$ is lower semicontinuous, it follows that

\begin{aligned} c_{b }^{ k}&\leq J^{b }_{ k}( \widetilde{u}))-\frac{1}{4} \bigl\langle \bigl(J^{b }_{ k} \bigr)'(\widetilde{u}),\widetilde{u}\bigr\rangle \\ &=\frac{1}{4} \Vert \widetilde{u} \Vert ^{2}+ \frac{1}{12} \vert \widetilde{u} \vert _{6}^{6}+ \frac{ k}{4} \int _{\mathbb{R}^{3}}\bigl[f(\widetilde{u})\widetilde{u}-4F( \widetilde{u})\bigr]\,dx \\ &=\frac{1}{4}\bigl( \bigl\Vert s_{u_{b}}u_{b}^{+} \bigr\Vert ^{2}+ \bigl\Vert t_{u_{b}}u_{b}^{-} \bigr\Vert ^{2}\bigr)+\frac{1}{12}\bigl( \bigl\vert s_{u_{b}}u_{b}^{+} \bigr\vert _{6}^{6}+ \bigl\vert t_{u_{b}}u_{b}^{-} \bigr\vert _{6}^{6}\bigr) \\ &\quad{}+\frac{ k}{4} \int _{\mathbb{R}^{3}}\bigl[f\bigl(s_{u_{b}}u_{b}^{+}\bigr) \bigl(s_{u_{b}}u_{b}^{+}\bigr)-4F\bigl(s_{u_{b}}u_{b}^{+} \bigr)\bigr]\,dx\\ &\quad {}+ \frac{ k}{4} \int _{\mathbb{R}^{3}}\bigl[f\bigl(x,t_{u_{b}}u_{b}^{-}\bigr) \bigl(t_{u_{b}}u_{b}^{-}\bigr)-4F\bigl(x,t_{u_{b}}u_{b}^{-} \bigr)\bigr]\,dx. \end{aligned}

By using $$0< s_{u_{b}}$$, $$t_{u_{b}}\leq 1$$, $$f(t)t-4F(t)$$ is increasing in $$(0,+\infty )$$ and decreasing in $$(-\infty ,0)$$, we have

\begin{aligned} c_{b }^{ k}&\leq \frac{1}{4} \Vert u_{b } \Vert ^{2}+\frac{1}{12} \vert u_{b } \vert _{6}^{6}+\frac{ k}{4} \int _{\mathbb{R}^{3}}\bigl[f(u_{b })u_{b }-4F(u_{b }) \bigr]\,dx \\ &\leq \liminf_{n\rightarrow \infty }\biggl[J^{b }_{ k}(u_{n})- \frac{1}{4}\bigl\langle \bigl(J^{b }_{ k} \bigr)'(u_{n}),u_{n}\bigr\rangle \biggr] \\ &= \liminf_{n\rightarrow \infty }J^{b }_{ k}(u_{n}) \\ &= c_{b }^{ k}. \end{aligned}

Therefore the infimum $$c^{ k}_{b }$$ is achieved by $$u_{b }=u_{b}^{+}+u_{b}^{-}\in \mathcal{M}^{ k}_{b }$$. □

## The proof of the main results

In this section, we prove the main results. Firstly, we prove Theorem 1.1. In fact, thanks to Lemma 2.4, we should prove that the minimizer $$u_{b }$$ for $$c^{ k}_{b }$$ is indeed a nodal solution of system (1.7), but $$\mathcal{M}_{k}^{b }$$ is not a smooth manifold, we will apply a new method to complete our certification.

### Proof

Since $$u_{b }\in \mathcal{M}^{ k}_{b }$$ and $$J^{b }_{ k}(u_{b }^{+}+u_{b }^{-}) =c^{ k}_{b }$$, we have $$\langle (J^{b }_{ k})'(u_{b }),u_{b }^{+}\rangle = \langle (J^{b }_{ k})'(u_{b }),u_{b }^{-}\rangle =0$$. By Lemma 2.1, for $$(s,t)\in (\mathbb{R}_{+}\times \mathbb{R}_{+})\backslash (1,1)$$, we have

\begin{aligned} J^{b }_{ k}\bigl(su_{b }^{+}+tu_{b }^{-} \bigr)< J^{b }_{ k}\bigl(u_{ b }^{+}+u_{b }^{-} \bigr) =c^{ k}_{b }. \end{aligned}
(3.1)

If $$(J^{b }_{ k})'(u_{b })\neq 0$$, then there exist $$\delta >0$$ and $$\theta >0$$ such that

$$\bigl\Vert \bigl(J^{b }_{ k}\bigr)'(v) \bigr\Vert \geq \theta \quad \text{for all } \Vert v-u_{ b } \Vert \leq 3\delta .$$

We know by result (2.15), if $$u\in \mathcal{M}_{k}^{b }$$, there exists $$L>0$$ such that $$\|u_{b}^{\pm }\|>L$$, and we can assume $$6\delta < L$$. Let $$Q:=(\frac{1}{2},\frac{3}{2})\times (\frac{1}{2},\frac{3}{2})$$ and $$g(s,t)=su_{b }^{+}+tu_{b }^{-}$$, $$(s,t)\in Q$$. In view of (3.1), it is easy to see that

\begin{aligned} \overline{c}^{ k}_{b }:=\max _{\partial Q}I\circ g< c^{ k}_{ b }. \end{aligned}
(3.2)

Let $$\varepsilon :=\min \{(c^{ k}_{b }-\overline{c}^{ k}_{b })/4, \theta \delta /8\}$$ and $$S_{\delta }:=B(u_{b },\delta )$$, according to Lemma 2.3 of , there exists a deformation $$\eta \in C([0,1]\times E,E)$$ satisfying

1. (a)

$$\eta (t,v)=v$$ if $$t=0$$, or $$v\notin (J^{b }_{ k})^{-1}([c^{ k}_{b }-2\varepsilon ,c^{ k}_{ b }+2\varepsilon ])\cap S_{2\delta }$$;

2. (b)

$$\eta (1,(J^{b }_{ k})^{c^{ k}_{b }+\varepsilon }\cap S_{ \delta })\subset (J^{b }_{ k})^{c^{ k}_{b }-\varepsilon }$$;

3. (c)

$$J^{b }_{ k}(\eta (1,v))\leq J^{b }_{ k}(v)$$ for all $$v\in E$$;

4. (d)

$$J^{b }_{ k}(\eta (\cdot ,v))$$ is nonincreasing for every $$v\in E$$.

We remind that, for a functional $$\varPhi :E\to \mathbb{R}$$, the level set $$\varPhi ^{\mu}$$ is defined by $$\varPhi ^{\mu}=\{u\in E:\varPhi (u)\le \mu\}$$. Firstly, we need to prove that

\begin{aligned} \max_{(s,t)\in \bar{Q}}J^{b }_{ k}\bigl( \eta \bigl(1,g(s,t)\bigr)\bigr)< c^{ k}_{ b }. \end{aligned}
(3.3)

In fact, it follows from Lemma 2.1 that $$J^{b }_{ k}(g(s,t))\leq c^{ k}_{b }< c^{ k}_{b }+ \varepsilon$$. That is,

$$g(s,t)\in \bigl(J^{b }_{ k}\bigr)^{c^{ k}_{b }+\varepsilon }.$$

On the other hand, from (a) and (d), we get

\begin{aligned} J^{b }_{ k}\bigl(\eta (1,v)\bigr)\leq J^{b }_{ k}\bigl(\eta (0,v)\bigr)=J^{ b }_{ k}(v), \quad \forall v\in E. \end{aligned}
(3.4)

For $$(s,t)\in Q$$, when $$s\neq 1$$ or $$t\neq 1$$, according to (3.1) and (3.4),

\begin{aligned} J^{b }_{ k}\bigl(\eta \bigl(1,g(s,t)\bigr)\bigr)\leq J^{b}_{ k}\bigl(g(s,t)\bigr)< c^{ k}_{ b }. \end{aligned}

If $$s= 1$$ and $$t= 1$$, that is, $$g(1,1)=u_{b }$$, so that it holds $$g(1,1)\in (J^{b }_{ k})^{c^{ k}_{b }+\varepsilon }\cap S_{ \delta }$$, then by (b)

\begin{aligned} J^{b }_{ k}\bigl(\eta \bigl(1,g(1,1)\bigr)\bigr)\leq c^{ k}_{b }-\varepsilon < c^{ k}_{b }. \end{aligned}

Thus (3.3) holds. In the following, we prove that $$\eta (1,g(Q))\cap \mathcal{M}^{ k}_{b }\neq \varnothing$$, which contradicts the definition of $$c^{ k}_{b }$$. Let $$\varphi (s,t):=\eta (1,g(s,t))$$ and

\begin{aligned} \varPsi (s,t):=\biggl(\frac{1}{s}\bigl\langle \bigl(J^{b }_{ k} \bigr)'\bigl(\varphi (s,t)\bigr),\bigl( \varphi (s,t) \bigr)^{{+}}\bigr\rangle , \frac{1}{t}\bigl\langle \bigl(J^{b }_{ k}\bigr)'\bigl( \varphi (s,t) \bigr),\bigl(\varphi (s,t)\bigr)^{{-}}\bigr\rangle \biggr). \end{aligned}

The claim holds if there exists $$(s_{0},t_{0})\in Q$$ such that $$\varPsi (s_{0},t_{0})=(0,0)$$. Since

\begin{aligned} \bigl\Vert g(s,t)-u_{b } \bigr\Vert ^{2} &= \bigl\Vert (s-1)u_{b }^{+}+(t-1)u_{b }^{-} \bigr\Vert ^{2} \\ &\geq \vert s-1 \vert ^{2} \bigl\Vert u_{b }^{+} \bigr\Vert ^{2} \\ &> \vert s-1 \vert ^{2}(6\delta )^{2}, \end{aligned}

and $$|s-1|^{2}(6\delta )^{2}>4\delta ^{2}\Leftrightarrow s<2/3$$ or $$s>4/3$$, using the item (a) above and the range of s, for $$s=\frac{1}{2}$$ and for every $$t\in [\frac{1}{2},\frac{3}{2}]$$, we have $$g(\frac{1}{2},t)\notin S_{2\delta }$$. So from (a) we have $$\varphi (\frac{1}{2},t)=g(\frac{1}{2},t)$$. Thus

\begin{aligned} \varPsi \biggl(\frac{1}{2},t\biggr)=\large \biggl(2\biggl\langle \bigl(J^{b }_{ k}\bigr)'\biggl( \frac{1}{2}u_{b }^{+}+t u_{b }^{-} \biggr),\frac{1}{2}u_{b }^{+}\biggr\rangle , \frac{1}{t}\biggl\langle \bigl( J^{b }_{ k} \bigr)'\biggl(\frac{1}{2}u_{b }^{+}+t u_{b }^{-}\biggr),tu^{-}\biggr\rangle \large \biggr). \end{aligned}

By Lemma 2.2, we know that

\begin{aligned} &\biggl\langle \bigl(J^{b }_{ k}\bigr)'\biggl( \frac{1}{2}u_{b }^{+}+t u_{b }^{-} \biggr), \frac{1}{2}u_{b }^{+}\biggr\rangle \\ &\quad =\biggl\langle \bigl(J^{b }_{ k}\bigr)'\biggl( \frac{1}{2}u_{b }^{+}\biggr),\frac{1}{2}u_{b }^{+} \biggr\rangle + \frac{t^{2}b}{4} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u_{b }^{-} \bigr\vert ^{2}\,dx \cdot \int _{\mathbb{R}^{3}} \bigl\vert \nabla u_{b }^{+} \bigr\vert ^{2}\,dx +\frac{t^{2}b }{4} \int _{\mathbb{R}^{3}}\phi _{u_{b }^{-}} \bigl\vert u_{b }^{+} \bigr\vert ^{2}\,dx \\ &\quad \geq \biggl\langle \bigl(J^{b }_{ k}\bigr)'\biggl( \frac{1}{2}u_{b }\biggr),\frac{1}{2}u_{b }^{+} \biggr\rangle >0, \end{aligned}

from which we obtain

\begin{aligned} \biggl\langle \bigl(J^{b }_{ k} \bigr)'\biggl(\frac{1}{2}u_{b}^{+}+t u_{b}^{-} \biggr),\frac{1}{2}u_{b}^{+} \biggr\rangle >0\quad \text{for every } t\in \biggl[\frac{1}{2},\frac{3}{2}\biggr]. \end{aligned}
(3.5)

Similarly, for $$s=\frac{3}{2}$$ and for every $$t\in [\frac{1}{2},\frac{3}{2}]$$, we have $$\varphi (\frac{3}{2},t)=g(\frac{3}{2},t)$$, so that

\begin{aligned} &\biggl\langle \bigl(J^{b }_{ k}\bigr)'\biggl( \frac{3}{2}u_{b}^{+}+t u_{b}^{-}\biggr),\frac{3}{2}u_{b}^{+} \biggr\rangle \\ &\quad =\biggl\langle \bigl(J^{b }_{ k} \bigr)'\biggl(\frac{3}{2}u_{b}^{+}\biggr), \frac{3}{2}u_{b}^{+} \biggr\rangle +\frac{9t^{2}}{4}b \int _{\mathbb{R}^{3}} \bigl\vert \nabla u_{b}^{-} \bigr\vert ^{2}\,dx \cdot \int _{\mathbb{R}^{3}} \bigl\vert \nabla u_{b}^{+} \bigr\vert ^{2}\,dx +\frac{9t^{2}}{4}b \int _{\mathbb{R}^{3}}\phi _{u_{b}^{-}} \bigl\vert u_{b}^{+} \bigr\vert ^{2}\,dx \\ &\quad \leq \biggl\langle \bigl(J^{b }_{ k} \bigr)'\biggl(\frac{3}{2}u_{b}\biggr),\frac{3}{2}u_{b}^{+} \biggr\rangle < 0, \end{aligned}

so that

\begin{aligned} \biggl\langle \bigl(J^{b }_{ k} \bigr)'\biggl(\frac{3}{2}u_{b}^{+}+t u_{b}^{-} \biggr),\frac{3}{2}u_{b}^{+} \biggr\rangle < 0\quad \text{for every } t\in \biggl[\frac{1}{2},\frac{3}{2}\biggr]. \end{aligned}
(3.6)

Similarly, we have

\begin{aligned} \biggl\langle \bigl(J^{b }_{ k} \bigr)'\biggl(s u_{b}^{+}+\frac{1}{2} u_{b}^{-} \biggr),\frac{1}{2}u_{b}^{-} \biggr\rangle >0\quad \text{for every } s\in \biggl[\frac{1}{2},\frac{3}{2}\biggr], \end{aligned}
(3.7)
\begin{aligned} \biggl\langle \bigl(J^{b }_{ k}\bigr)'\biggl(s u_{b}^{+}+\frac{3}{2} u_{b}^{-}\biggr),\frac{3}{2}u_{b}^{-} \biggr\rangle < 0\quad \text{for every } s\in \biggl[\frac{1}{2}, \frac{3}{2}\biggr]. \end{aligned}
(3.8)

Since Ψ is continuous on Q, according to (3.5)–(3.7), by Miranda’s theorem (Lemma 2.4 ), we have $$\varPsi (s_{0},t_{0})=0$$ for some $$(s_{0},t_{0})\in Q$$, so $$\eta (1,g(s_{0},t_{0}))=\varphi (s_{0},t_{0})\in \mathcal{M}^{ k}_{ b }$$. By (3.3), we have a contradiction. From the above discussion, we conclude that $$u_{b }$$ is a nodal solution for system (1.7).

Finally, we prove that $$u_{b }$$ has exactly two nodal domains. To this end, we first write $$u_{b }$$ as

$$u_{b }=u_{1}+u_{2}+u_{3}$$

with $$u_{1}\geq 0$$, $$u_{2}\leq 0$$. Set $$\varOmega _{i}=\{x\in \mathbb{R}^{3}:u_{i}(x)\ne 0\}$$. We further assume $$\varOmega _{i}\cap \varOmega _{j}=\emptyset$$ for $$i\neq j$$, $$i,j=1,2,3$$. Since $$u_{b }$$ is a nodal solution, we suppose the nodal domains $$\varOmega _{1}\ne \emptyset$$, $$\varOmega _{2}\ne \emptyset$$. By contradiction, we suppose $$u_{b }$$ possesses more than two nodal domains, then we have $$u_{3}\ne 0$$ and so $$\varOmega _{3}\ne \emptyset$$. Setting $$v:=u_{1}+u_{2}$$, we easily see that $$v^{\pm }\neq 0$$. So, there exists a positive pair $$(s_{v},t_{v})$$ such that

$$s_{v}u_{1}+t_{v}u_{2}\in \mathcal{M}^{ k}_{b }.$$

Thus,

$$J^{b }_{ k}(s_{v}u_{1}+t_{v}u_{2}) \geq c^{ k}_{b }.$$

Moreover, using the fact that $$\langle (J^{b }_{ k})'(u_{b }),u_{i} \rangle =0$$, from the definition, we get $$\langle (J^{b }_{ k})'(v),v^{\pm } \rangle \le 0$$. So, thanks to Lemma 2.1, we have that

$$(s_{v},t_{v})\in (0,1]\times (0,1].$$

By direct calculation,

\begin{aligned}& 0=\bigl\langle \bigl(J^{b }_{ k} \bigr)'(u_{b }),u_{3} \bigr\rangle \\& \hphantom{0}= \Vert u_{3} \Vert ^{2} +b \int _{\mathbb{R}^{3}} \vert \nabla u_{1} \vert ^{2}\,dx\cdot \int _{\mathbb{R}^{N}} \vert \nabla u_{3} \vert ^{2}\,dx \\& \hphantom{0}\quad{} +b \int _{\mathbb{R}^{3}} \vert \nabla u_{2} \vert ^{2}\,dx\cdot \int _{ \mathbb{R}^{N}} \vert \nabla u_{3} \vert ^{2}\,dx +b \biggl( \int _{\mathbb{R}^{N}} \vert \nabla u_{3} \vert ^{2}\,dx \biggr)^{2} \\& \hphantom{0}\quad{}+\lambda \int _{\mathbb{R}^{3}}\phi _{u_{1}} \vert u_{3} \vert ^{2}\,dx+ \frac{\lambda }{4} \int _{\mathbb{R}^{3}}\phi _{u_{2}} \vert u_{3} \vert ^{2}\,dx + \lambda \int _{\mathbb{R}^{3}}\phi _{u_{3}} \vert u_{3} \vert ^{2}\,dx \\& \hphantom{0}\quad{}- \lambda \int _{\mathbb{R}^{3}} \vert u_{3} \vert ^{6}\,dx- \frac{ k}{4} \int _{ \mathbb{R}^{3}}f(u_{3})u_{3}\,dx \\& \hphantom{0}=\bigl\langle \bigl(J^{b }_{ k}\bigr)'(u_{3}),u_{3} \bigr\rangle +b \int _{ \mathbb{R}^{3}}\bigl( \vert \nabla u_{1} \vert ^{2}+ \vert \nabla u_{2} \vert ^{2}\bigr)\,dx \cdot \int _{ \mathbb{R}^{N}} \vert \nabla u_{3} \vert ^{2}\,dx \\& \hphantom{0}\quad{}+ \lambda \int _{\mathbb{R}^{3}}\phi _{u_{1}} \vert u_{3} \vert ^{2}\,dx+ \lambda \int _{\mathbb{R}^{3}}\phi _{u_{2}} \vert u_{3} \vert ^{2}\,dx \\& \hphantom{0}< 4J^{b }_{ k}(u_{3})+b \int _{\mathbb{R}^{3}} \vert \nabla u_{1} \vert ^{2}\,dx \cdot \int _{\mathbb{R}^{N}} \vert \nabla u_{3} \vert ^{2}\,dx+b \int _{\mathbb{R}^{3}} \vert \nabla u_{2} \vert ^{2}\,dx\cdot \int _{\mathbb{R}^{N}} \vert \nabla u_{3} \vert ^{2}\,dx \\& \hphantom{0}\quad{}+ \lambda \int _{\mathbb{R}^{3}}\phi _{u_{1}} \vert u_{3} \vert ^{2}\,dx+ \lambda \int _{\mathbb{R}^{3}}\phi _{u_{2}} \vert u_{3} \vert ^{2}\,dx, \end{aligned}
(3.9)
\begin{aligned}& \begin{aligned}[b] -\bigl\langle \bigl(J^{b }_{ k} \bigr)'(u_{3}),u_{3}\bigr\rangle &=b \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u_{1} \vert ^{2}+ \vert \nabla u_{2} \vert ^{2}\bigr)\,dx \cdot \int _{\mathbb{R}^{N}} \vert \nabla u_{3} \vert ^{2}\,dx\\ &\quad {}+ \lambda \int _{\mathbb{R}^{3}}\phi _{u_{1}} \vert u_{3} \vert ^{2}\,dx+ \lambda \int _{\mathbb{R}^{3}}\phi _{u_{2}} \vert u_{3} \vert ^{2}\,dx. \end{aligned} \end{aligned}
(3.10)

Then, by using (2.16), we get

\begin{aligned} c^{ k}_{b } &\leq J^{b }_{ k}(s_{v}u_{1}+t_{v}u_{2})= J^{b }_{ k}(s_{v}u_{1}+t_{v}u_{2}) -\frac{1}{4}\bigl\langle \bigl(J^{b }_{ k} \bigr)'(s_{v}u_{1}+t_{v}u_{2}), s_{v}u_{1}+t_{v}u_{2}\bigr\rangle \\ &=\frac{1}{4}\bigl( \Vert s_{v}u_{1} \Vert ^{2}+ \Vert t_{v}u_{2} \Vert ^{2} \bigr) +\frac{ k}{4} \int _{\mathbb{R}^{3}}\bigl[f(s_{v}u_{1}) (s_{v}u_{1}) -4F(s_{v}u_{1})\bigr]\,dx \\ &\quad{}+\frac{ k}{4} \int _{\mathbb{R}^{3}}\bigl[f(t_{v}u_{2}) (t_{v}u_{2}) -4F(t_{v}u_{2})\bigr]\,dx+ \frac{s^{6}_{v}}{12} \int _{\mathbb{R}^{3}} \vert u_{1} \vert ^{6}\,dx+ \frac{t^{6}_{v}}{12} \int _{\mathbb{R}^{3}} \vert u_{2} \vert ^{6}\,dx \\ &\leq \frac{1}{4}\bigl( \Vert u_{1} \Vert ^{2}+ \Vert u_{2} \Vert ^{2}\bigr) +\frac{ k}{4} \int _{ \mathbb{R}^{3}}\bigl[f(u_{1})u_{1} -4F(u_{1})\bigr]\,dx \\ &\quad{}+\frac{ k}{4} \int _{\mathbb{R}^{3}}\bigl[f(u_{2})u_{2} -4F(u_{2})\bigr]\,dx + \frac{1}{12} \int _{\mathbb{R}^{3}} \vert u_{1} \vert ^{6}\,dx+ \frac{1}{12} \int _{ \mathbb{R}^{3}} \vert u_{2} \vert ^{6}\,dx \\ &=J^{b }_{ k}(u_{1}+u_{2}) - \frac{1}{4}\bigl\langle \bigl(J^{b }_{ k} \bigr)'(u_{1}+u_{2}),(u_{1}+u_{2}) \bigr\rangle . \end{aligned}

Similar to the computation of (3.10), from $$\langle (J^{b}_{k })'(u_{b }),u_{b }\rangle =0$$, there holds

\begin{aligned} &-\bigl\langle \bigl(J^{b}_{k }\bigr)' (u_{1}+u_{2}),u_{1}+u_{2}\bigr\rangle \\ &\quad = \bigl\langle \bigl(J^{b}_{k }\bigr)'(u_{3}),u_{3} \bigr\rangle +2b \int _{\mathbb{R}^{3}}\bigl( \vert \nabla u_{1} \vert ^{2}+ \vert \nabla u_{2} \vert ^{2}\bigr)\,dx \cdot \int _{\mathbb{R}^{3}} \vert \nabla u_{3} \vert ^{2}\,dx \\ &\qquad{}+\lambda \int _{\mathbb{R}^{3}}(\phi _{u_{1}}+\phi _{u_{2}}) \vert u_{3} \vert ^{2}\,dx+ \lambda \int _{\mathbb{R}^{3}}\phi _{u_{3}}\bigl( \vert u_{1} \vert ^{2}+ \vert u_{2} \vert ^{2}\bigr)\,dx. \end{aligned}
(3.11)

By using (3.9), (3.10), and (3.11), we get

\begin{aligned} c^{ k}_{b }&\le J^{b }_{ k}(u_{1}+u_{2}) -\frac{1}{4} \bigl\langle \bigl(J^{b }_{ k} \bigr)'(u_{1}+u_{2}),(u_{1}+u_{2}) \bigr\rangle \\ &=J^{b }_{ k}(u_{1}+u_{2}) + \frac{1}{4}\bigl\langle \bigl(J^{b }_{ k} \bigr)'(u_{ b }),u_{3}\bigr\rangle + \frac{b}{4} \int _{\mathbb{R}^{3}} \vert \nabla u_{1} \vert ^{2}\,dx \cdot \int _{\mathbb{R}^{N}} \vert \nabla u_{3} \vert ^{2}\,dx \\ &\quad{}+\frac{b}{4} \int _{\mathbb{R}^{3}} \vert \nabla u_{2} \vert ^{2}\,dx\cdot \int _{ \mathbb{R}^{N}} \vert \nabla u_{3} \vert ^{2}\,dx +\frac{\lambda }{4} \int _{ \mathbb{R}^{3}}\phi _{u_{3}} \vert u_{1} \vert ^{2}\,dx+\frac{\lambda }{4} \int _{ \mathbb{R}^{3}}\phi _{u_{3}} \vert u_{2} \vert ^{2}\,dx \\ &< J^{b }_{ k}(u_{1})+J^{b }_{ k}(u_{2})+J^{b }_{ k}(u_{3}) +\frac{b}{4} \int _{\mathbb{R}^{N}}\bigl( \vert \nabla u_{2} \vert ^{2}+ \vert \nabla u_{3} \vert ^{2}\bigr)\,dx \cdot \int _{\mathbb{R}^{N}} \vert \nabla u_{1} \vert ^{2}\,dx \\ &\quad{}+\frac{b}{4} \int _{\mathbb{R}^{N}}\bigl( \vert \nabla u_{1} \vert ^{2}+ \vert \nabla u_{3} \vert ^{2}\bigr)\,dx \cdot \int _{\mathbb{R}^{N}} \vert \nabla u_{2} \vert ^{2}\,dx \\ &\quad {}+\frac{b}{4} \int _{ \mathbb{R}^{N}} \bigl( \vert \nabla u_{1} \vert ^{2}+ \vert \nabla u_{2} \vert ^{2}\bigr)\,dx \cdot \int _{ \mathbb{R}^{N}} \vert \nabla u_{3} \vert ^{2}\,dx \\ &\quad{}+\frac{\lambda }{4} \int _{\mathbb{R}^{3}}\phi _{u_{3}}\bigl( \vert u_{1} \vert ^{2}+ \vert u_{2} \vert ^{2}\bigr)\,dx+ \frac{\lambda }{4} \int _{\mathbb{R}^{3}}\phi _{u_{1}+u_{2}} \vert u_{3} \vert ^{2}\,dx \\ &=J^{b }_{ k}(u_{b })=c^{ k}_{b }. \end{aligned}

So we get $$u_{3}=0$$ and $$u_{b }$$ has exactly two nodal domains. □

### The proof of Theorem 1.2

To prove Theorem 1.2, we should first prove that there exists a ground state solution of (1.7) for k large enough, and then to prove that the energy of sign-changing solution $$u_{b }$$ is strictly larger than twice of that of the ground state solution.

### Proof

Similar to the proof of Lemma 2.4, we claim that there exists $$k_{1}^{\star }>0$$ such that, for all $$k\geq k_{1}^{\star }$$, and $$\forall b >0$$, there exists $$v_{b }\in \mathcal{N}^{ k}_{b }$$ such that $$J^{b }_{ k}(v_{b }) =c^{\ast }>0$$. We give a brief proof of this claim.

We first list some results for the Nehari manifold $$\mathcal{N}^{ k}_{b }$$. One can prove them by following the ideas as those in Lemma 2.4.

1. (i)

If $$v\in \mathcal{N}^{ k}_{b }$$, then $$J^{b }_{ k}(tv)\le J^{b }_{ k}(v)$$ for all $$t\ge 0$$;

2. (ii)

There exists $$\rho >0$$ such that $$\|v\|\ge \rho$$ for all $$v \in \mathcal{N}^{ k}_{b }$$;

3. (iii)

There exists $$M>0$$ such that $$\|v\|\le M$$ for all $$v \in \mathcal{N}^{ k}_{b }$$.

According to the definition of $$c^{\ast }$$, there is a sequence $$\{v_{n}\}\subset \mathcal{N}^{ k}_{b }$$ such that $$\lim_{n\rightarrow \infty }J^{b }_{ k}(v_{n})=c^{\ast }$$. By property (iii), $$\{v_{n}\}$$ is bounded in E. In the subsequence sense, there exists $$v_{b }\in E$$ such that $$v_{n}\rightharpoonup v_{b }$$.

Denote $$\beta :=\frac{(S)^{\frac{3}{2}}}{3}$$, where $$S:=\inf_{u\in E\backslash \{0\}} \frac{\|u\|^{2}}{(\int _{\mathbb{R}^{3}}|u|^{6}\,dx)^{\frac{1}{3}}}$$. Similar to the proof of Lemma 2.3, there is $$k^{\star }>0$$ such that $$c^{\ast }<\beta$$ for all $$k\geq k^{\star }$$. Therefore, $$\liminf_{n\rightarrow \infty }J^{b }_{ k}(tv_{n})\geq J^{ b }_{ k}(tv_{b })+\frac{t^{2}}{2}A-\frac{t^{6}}{6}B$$, where $$A=\lim_{n\rightarrow \infty }\|v_{n}-v_{b }\|^{2}$$, $$B=\lim_{n \rightarrow \infty }|v_{n}-v_{b }|_{6}^{6}$$. From the above fact and property (i), we have

\begin{aligned} J^{b }_{ k}(tv_{b })+\frac{t^{2}}{2}A- \frac{t^{6}}{6}B\le c^{*} \end{aligned}
(3.12)

for all $$t \geq 0$$.

Firstly, we prove that $$v_{b }\neq 0$$. By contradiction, we suppose $$v_{b }=0$$.

Case 1:$$B=0$$. If $$A=0$$, that is, $$v_{n}\rightarrow v_{b }$$ in E, then $$v_{b }\in \mathcal{N}^{ k}_{b }$$, and so we have $$\|v_{b }\|>\rho$$ by property (ii), which contradicts our supposition. If $$A>0$$, $$\frac{t^{2}}{2}A\leq c^{\ast }$$ for all $$t\geq 0$$, which is a contradiction.

Case 2:$$B>0$$. According to the definition of S, we have that $$\beta =\frac{(S)^{\frac{3}{2}}}{3}\leq \frac{1}{3}( \frac{A}{(B)^{\frac{1}{3}}})^{\frac{3}{2}}$$. It is easy to see that

$$\frac{1}{3}\biggl(\frac{A}{(B)^{\frac{1}{3}}}\biggr)^{\frac{3}{2}}= \frac{\tilde{t}^{2}}{2}A-\frac{\tilde{t}^{6}}{6}B:=\max_{t\geq 0}\biggl\{ \frac{t^{2}}{2}A-\frac{t^{6}}{6}B\biggr\} ,$$

so we have that

$$\beta \leq \max_{t\geq 0}\biggl\{ \frac{t^{2}}{2}A- \frac{t^{6}}{6}B\biggr\} \le c_{ b }^{ k}< \beta ,$$

Secondly, we claim that $$B=0$$. By contradiction, we suppose that $$B>0$$. Firstly, we can maximize $$\psi _{v_{b }}(t)=J^{b }_{ k}(tv_{b })$$ in $$[0,\infty )$$. Indeed, there exists $$t_{0}\in [0,\infty )$$ such that $$J^{b }_{ k}(tv_{b })\leq 0$$ for all $$t\in [t_{0},\infty )$$. Let $$t_{v}$$ be an inner maximizer of $$\psi _{v}$$ in $$[0,\infty )$$. $$J^{b }_{ k}(\tilde{t}v_{b })+\beta \le J^{b }_{ k}( \tilde{t}v_{b })+\frac{\tilde{t}^{2}}{2}A- \frac{\tilde{t}^{6}}{6}B\le c^{*}<\beta$$ implies that $$J^{b }_{ k}(\tilde{t}v_{b })<0$$. So $$t_{v}\le \tilde{t}$$ and $$\frac{t_{v}^{2}}{2}A-\frac{t_{v}^{6}}{6}B>0$$. Thus from $$t_{v}v_{b }\in \mathcal{N}_{b }^{ k}$$ we get a contradiction by

$$c^{*}\le J^{b }_{ k}(t_{v}v_{b })< J^{b }_{ k}(t_{v}v_{ b})+ \frac{t_{v}^{2}}{2}A-\frac{t_{v}^{6}}{6}B\le c^{*}.$$

Lastly, we prove that $$c^{\ast }$$ is achieved by $$v_{b}$$. From the above arguments, we have $$v_{b}\neq 0$$ and $$\widetilde{v}:=t_{v}v_{b }\in \mathcal{ N}^{ k}_{b }$$. Furthermore, because $$v_{n}\rightharpoonup v_{b }$$ in E and $$v_{n}\in \mathcal{N}^{ k}_{b }$$, we have that $$\langle (J^{b }_{ k})'(v_{b }),v_{b }\rangle \le 0$$. Similar to Lemma 2.1, we have $$0< t_{v}\le 1$$. Also as in the proof of Lemma 2.4, we have

\begin{aligned}& \begin{aligned} c^{*}&\le J^{b }_{ k}(\tilde{v})- \frac{1}{4} \bigl\langle \bigl(J^{b }_{ k} \bigr)'( \tilde{v}),\tilde{v}\bigr\rangle \\ &=\frac{1}{4} \Vert t_{v}v_{b } \Vert ^{2}+\frac{1}{12} \vert t_{v}v_{b } \vert _{6}^{6}+ \frac{k}{4} \int _{\mathbb{R}^{3}}\bigl[f(t_{v}v_{b })t_{v}v_{b }-4F(t_{v}v_{b }) \bigr]\,dx \\ & \le \frac{1}{4} \Vert v_{b } \Vert ^{2}+ \frac{1}{12} \vert v_{b } \vert _{6}^{6}+ \frac{k}{4} \int _{\mathbb{R}^{3}}\bigl[f(v_{b })v_{b }-4F(v_{b }) \bigr]\,dx, \end{aligned} \\& \lim \inf_{n\to \infty }\biggl[J^{b }_{ k}(v_{n})- \frac{1}{4} \bigl\langle \bigl(J^{ b }_{ k} \bigr)'(v_{n}),v_{n}\bigr\rangle \biggr]=c^{*}. \end{aligned}

Therefore, $$t_{v}=1$$, and $$c^{\ast }$$ is achieved by $$v_{b }\in \mathcal{N}^{ k}_{b }$$.

By standard arguments, the critical points of the functional $$J^{b }_{ k}$$ on $$\mathcal{N}^{ k}_{b }$$ are critical points of $$J^{b }_{ k}$$ in E, and we obtain $$(J^{b }_{ k})'(v_{b }) =0$$, so $$v_{b }$$ is a positive or negative solution. That is, $$v_{b }$$ is a ground state solution of system (1.7). For all $$k\geq k^{\star }$$, and $$\forall b >0$$, problem (1.7) has a least energy nodal solution $$u_{b }$$. Let

$$k^{\star \star }=\max \bigl\{ k^{\star }, k^{\star }_{1} \bigr\} .$$

Suppose that $$u_{b }=u^{+}+u^{-}$$. As in the proof of Lemma 2.1, there exist $$s_{u^{+}},t_{u^{-}}\in (0,1)$$ such that

$$s_{u^{+}}u^{+}\in \mathcal{N}^{ k}_{b }, \quad t_{u^{-}}u^{-} \in \mathcal{N}^{ k}_{b }.$$

Hence, by Lemma 2.1, we deduce

\begin{aligned} 2c^{\ast }\leq J^{b }_{ k}\bigl(s_{u^{+}}u^{+} \bigr)+J^{b }_{ k}\bigl(t_{u^{-}}u^{-}\bigr) \leq J^{b }_{ k}\bigl(s_{u^{+}}u^{+}+t_{u^{-}}u^{-} \bigr)< J^{b }_{ k}\bigl(u^{+}+u^{-} \bigr)=c^{ k}_{b }. \end{aligned}

□

### Proof of Theorem 1.3

At the end of the section, we give an analysis for the behavior of $$u_{b }$$ as $$b \rightarrow 0$$. We regard $$b >0$$ as a parameter in equation (1.7).

### Proof

For any $$b >0$$, let $$u_{b }\in E$$ be the least energy nodal solution of system (1.7) obtained in Theorem 1.1. We will complete our proof with the following three assertions. We recall that $$u_{b _{n}}$$ is a least energy nodal solution of system (1.7) with $$b =b _{n}\to 0$$ as $$n\to \infty$$.

Claim (a). As n is large enough, $$\{u_{b _{n}}\}$$ is bounded in E.

Choose a test function $$\phi \in C^{\infty }_{c}(\mathbb{R}^{3})$$ with $$\phi ^{\pm }\neq 0$$. From (2.7), for any $$b \in [0,1]$$, there exists a pair of positive numbers $$( k_{1}, k_{2})$$ such that

\begin{aligned} \bigl\langle \bigl(J^{b }_{ k}\bigr)'\bigl( k_{1}\phi ^{+}+ k_{2}\phi ^{-}\bigr), k_{1} \phi ^{+}\bigr\rangle < 0, \end{aligned}

and

\begin{aligned} \bigl\langle \bigl(J^{b }_{ k}\bigr)'\bigl( k_{1}\phi ^{+}+ k_{2}\phi ^{-}\bigr), k_{2} \phi ^{-}\bigr\rangle < 0. \end{aligned}

Thus, according to Lemma 2.1(ii), for any $$b \in [0,1]$$, there is a unique pair $$s_{\phi }(b ), t_{\phi }(b )\in (0,1]\times (0,1]$$ such that

\begin{aligned} \overline{\phi }:=s_{\phi }(b ) k_{1}\phi ^{+}+t_{\phi }(b ) k_{2} \phi ^{-}\in \mathcal{M}^{ k}_{b }. \end{aligned}
(3.13)

Hence, for any $$b \in [0,1]$$, by using (2.4), we get

\begin{aligned} J^{b }_{ k}(u_{b })&\leq J^{b }_{ k}( \overline{\phi })=J^{ b }_{ k}(\overline{\phi })- \frac{1}{4}\bigl\langle \bigl(J^{b }_{ k} \bigr)'( \overline{\phi }),\overline{\phi }\bigr\rangle \\ &=\frac{1}{4} \Vert \overline{\phi } \Vert ^{2}+ \frac{ k}{4} \int _{\mathbb{R}^{3}}\bigl[f( \overline{\phi })\overline{\phi }-4F( \overline{\phi })\bigr]\,dx+ \frac{1}{12} \int _{\mathbb{R}^{3}} \vert \overline{\phi } \vert ^{6}\,dx \\ &\leq \frac{1}{4} \Vert \overline{\phi } \Vert ^{2}+ \frac{ k}{4} \int _{ \mathbb{R}^{3}}\bigl(C_{1}\overline{\phi }^{2}+C_{2}\overline{\phi }^{q}\bigr)\,dx+ \frac{1}{12} \int _{\mathbb{R}^{3}} \vert \overline{\phi } \vert ^{6}\,dx \\ &\leq \frac{1}{4}\bigl( k_{1}^{2} \bigl\Vert \phi ^{+} \bigr\Vert ^{2}+ k_{2}^{2} \bigl\Vert \phi ^{-} \bigr\Vert ^{2}\bigr)+\frac{ k}{4} \int _{\mathbb{R}^{3}}\bigl(C_{1}{ k_{1}}^{2} \bigl\vert \phi ^{+} \bigr\vert ^{2}+C_{1}{ k_{2}}^{2} \bigl\vert \phi ^{-} \bigr\vert ^{2}\bigr)\,dx \\ &\quad{}+ \frac{ k}{4} \int _{\mathbb{R}^{3}}\bigl(C_{2}{ k_{1}}^{5} \bigl\vert \phi ^{+} \bigr\vert ^{5}+C_{2}{ k_{2}}^{5} \bigl\vert \phi ^{-} \bigr\vert ^{5}\bigr)\,dx +\frac{{ k_{1}}^{6}}{12} \int _{ \mathbb{R}^{3}} \bigl\vert \phi ^{+} \bigr\vert ^{6}\,dx+\frac{{ k_{2}}^{6}}{12} \int _{ \mathbb{R}^{3}} \bigl\vert \phi ^{-} \bigr\vert ^{6}\,dx \\ &:=C^{\ast }, \end{aligned}

where $$C^{\ast }>0$$ is a constant independent of b. So, as n is large enough, it follows that

\begin{aligned} C^{\ast }+1 \geq J^{ k}_{b _{n}}(u_{b _{n}})=J^{ k}_{ b _{n}}(u_{b _{n}})- \frac{1}{4}\bigl\langle \bigl(J^{ k}_{b _{n}} \bigr)'(u_{ b _{n}}),u_{b _{n}}\bigr\rangle \geq \frac{1}{4} \Vert u_{b _{n}} \Vert ^{2}. \end{aligned}

Therefore, we can deduce Claim (a) from the above inequality.

Claim (b). System (1.11) possesses a nodal solution $$u_{0}$$.

Since $$\{u_{b _{n}}\}$$ is bounded in E, in the subsequence sense, there exists $$u_{0}\in E$$ such that

\begin{aligned} &u_{b _{n}}\rightharpoonup u_{0} \quad \text{in } E, \\ &u_{b _{n}}\rightarrow u_{0}\quad \text{in } L^{p} \bigl(\mathbb{R}^{3}\bigr) \quad \text{for } p\in (2,6), \\ &u_{b _{n}}\rightarrow u_{0}\quad \text{a.e. in } \mathbb{R}^{3}. \end{aligned}
(3.14)

Thanks to $$\{u_{b _{n}}\}$$ being a least energy nodal solution of system (1.7) with $$b =b_{n}$$, we have that

\begin{aligned} & \int _{\mathbb{R}^{3}}\bigl(a\nabla u_{b _{n}}\cdot \nabla v +V(x)u_{ b _{n}}v\bigr)\,dx+b_{n}\biggl( \int _{\mathbb{R}^{3}} \vert \nabla u_{b _{n}} \vert ^{2}\,dx\biggr) \biggl( \int _{\mathbb{R}^{3}}\nabla u_{b _{n}}\cdot \nabla v\,dx\biggr) \\ &\quad{}+\lambda \int _{\mathbb{R}^{3}}\phi _{u_{b_{n}}}u_{ b_{n}}v\,dx - k \int _{\mathbb{R}^{3}}f(u_{b _{n}})v \,dx- \int _{\mathbb{R}^{3}} \vert u_{b _{n}} \vert ^{4}u_{b _{n}}v \,dx=0 \end{aligned}
(3.15)

for any $$v\in C^{\infty }_{c}(\mathbb{R}^{3})$$. Combining (3.14), (3.15) with Claim (a), we have that

\begin{aligned} & \int _{\mathbb{R}^{3}}\bigl(a\nabla u_{0}\cdot \nabla v +V(x)u_{0}v\bigr)\,dx+\lambda \int _{\mathbb{R}^{3}} \phi_{u_{0}}u_{0}v \,dx \\ &\quad{}- k \int _{\mathbb{R}^{3}}f(u_{0})v \,dx- \int _{\mathbb{R}^{3}} \vert u_{0} \vert ^{4}u_{0}v \,dx=0 \end{aligned}

for any $$v\in C^{\infty }_{c}(\mathbb{R}^{3})$$. It implies that $$u_{0}$$ is a weak solution of the Kirchhoff equation (1.11). We next deduce that $$u_{0}^{\pm }\neq 0$$. Since $$u_{b _{n}}\in \mathcal{M}^{ k}_{b _{n}}$$, we have

\begin{aligned} & \bigl\Vert u_{b _{n}}^{\pm } \bigr\Vert ^{2}+b_{n} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u_{ b _{n}}^{\pm } \bigr\vert ^{2}\,dx\biggr)^{2}+b_{n} \int _{\mathbb{R}^{3}}\nabla u_{ b_{n}}^{\pm } \vert ^{2}\,dx\cdot \int _{\mathbb{R}^{3}} \bigl\vert \nabla u_{ b _{n}}^{\mp } \bigr\vert ^{2}\,dx) \\ &\qquad{}+\lambda \int _{\mathbb{R}^{3}}\phi _{u_{b _{n}}^{\pm }} \bigl\vert u_{ b _{n}}^{\pm } \bigr\vert ^{2}\,dx+\lambda \int _{\mathbb{R}^{3}}\phi _{u_{ b_{n}}^{\mp }} \bigl\vert u_{b _{n}}^{\pm } \bigr\vert ^{2}\,dx\\ &\quad = \int _{\mathbb{R}^{3}} \bigl\vert u_{ b _{n}}^{\pm } \bigr\vert ^{6}\,dx+ k \int _{\mathbb{R}^{3}}f\bigl(u_{b _{n}}^{ \pm } \bigr)u_{b _{n}}^{\pm }\,dx. \end{aligned}

Hence, by using Claim (a) and the continuous embedding $$E\hookrightarrow L^{6}(\mathbb{R}^{3})$$, we have $$u_{b _{n}}$$ is bounded in $$L^{6}(\mathbb{R}^{3})$$, thus there exists $$k^{\star }_{2}>0$$ such that, for all $$k\geq k^{\star }_{2}$$, we have that

\begin{aligned} \rho \leq \bigl\Vert u_{b _{n}}^{\pm } \bigr\Vert ^{2}\leq \int _{\mathbb{R}^{3}} \bigl\vert u_{ b _{n}}^{\pm } \bigr\vert ^{6}\,dx+ k \int _{\mathbb{R}^{3}}f\bigl(u_{b _{n}}^{ \pm } \bigr)u_{b _{n}}^{\pm }\,dx\leq 2 k \int _{\mathbb{R}^{3}}f\bigl(u_{ b _{n}}^{\pm } \bigr)u_{b _{n}}^{\pm }\,dx. \end{aligned}

By using (2.4), we have that

\begin{aligned} 0< \int _{\mathbb{R}^{3}}f\bigl(u_{0}^{\pm } \bigr)u_{0}^{\pm }\,dx. \end{aligned}

Since $$u_{0}$$ is a solution of system (1.11), we have that

\begin{aligned} \bigl\Vert u_{0}^{\pm } \bigr\Vert ^{2}\geq k \int _{\mathbb{R}^{3}}f\bigl(x,u_{0}^{\pm } \bigr)u_{0}^{ \pm }\,dx+ \int _{\mathbb{R}^{3}} \bigl\vert u_{0}^{\pm } \bigr\vert ^{6}\,dx\geq k \int _{ \mathbb{R}^{3}}f\bigl(u_{0}^{\pm } \bigr)u_{0}^{\pm }\,dx>0. \end{aligned}

It implies $$u_{0}^{\pm }\neq 0$$.

Claim (c). Problem (1.11) possesses a least energy nodal solution $$v_{0}$$.

Similar to the proof of Theorem 1.1, there is $$k^{\star }_{3}>0$$ such that, for all $$k\geq k^{\star }_{3}$$, problem (1.11) possesses a least energy nodal solution $$v_{0}$$, where $$J^{0}_{k}(v_{0})=c^{0}_{k}$$ and $$(J^{0}_{k})'(v_{0})=0$$. Let

$$k^{\star \star \star }=\max \bigl\{ k^{\star }, k^{\star }_{2}, k^{\star }_{3} \bigr\} .$$

According to Lemma 2.1, there exists a positive pair $$(s_{b _{n}}, t_{b _{n}})\in (0,\infty )\times (0,\infty )$$ such that $$s_{b_{n}}v^{+}_{0}+t_{b _{n}}v^{-}_{0}\in \mathcal{M}^{ k}_{ b _{n}}$$. That is,

\begin{aligned} &s_{b_{n}}^{2} \bigl\Vert v_{0}^{+} \bigr\Vert ^{2}+\lambda s_{b_{n}}^{4} \int _{\mathbb{R}^{3}}\phi _{v_{0}^{+}} \bigl\vert v_{0}^{+} \bigr\vert ^{2}\,dx+\lambda s_{ b _{n}}^{2}t_{b _{n}}^{2} \int _{\mathbb{R}^{3}}\phi _{v_{0}^{-}} \bigl\vert v_{0}^{+} \bigr\vert ^{2}\,dx +b_{n}s_{b _{n}}^{4} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla v_{0}^{+} \bigr\vert ^{2}\,dx\biggr)^{2} \\ &\qquad{}+b_{n}s_{b _{n}}^{2}t_{b _{n}}^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla v_{0}^{+} \bigr\vert ^{2}\,dx\cdot \int _{\mathbb{R}^{3}} \bigl\vert \nabla v_{0}^{-} \bigr\vert ^{2}\,dx \\ &\quad =s_{ b _{n}}^{6} \int _{\mathbb{R}^{3}} \bigl\vert v_{0}^{+} \bigr\vert ^{6}\,dx+ k \int _{ \mathbb{R}^{3}}f\bigl(s_{b _{n}}v_{0}^{+} \bigr)s_{b _{n}}v_{0}^{+}\,dx \end{aligned}
(3.16)

and

\begin{aligned} &t_{b _{n}}^{2} \bigl\Vert v_{0}^{-} \bigr\Vert ^{2}+\lambda t_{b _{n}}^{4} \int _{\mathbb{R}^{3}}\phi _{v_{0}^{-}} \bigl\vert v_{0}^{-} \bigr\vert ^{2}\,dx+\lambda s_{ b _{n}}^{2}t_{b _{n}}^{2} \int _{\mathbb{R}^{3}}\phi _{v_{0}^{+}} \bigl\vert v_{0}^{-} \bigr\vert ^{2} \,dx+b_{n}t_{b _{n}}^{4} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla v_{0}^{-} \bigr\vert ^{2}\,dx\biggr)^{2} \\ &\qquad{}+b_{n}s_{b _{n}}^{2}t_{b _{n}}^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla v_{0}^{+} \bigr\vert ^{2}\,dx\cdot \int _{\mathbb{R}^{3}} \bigl\vert \nabla v_{0}^{-} \bigr\vert ^{2}\,dx \\ &\quad =t_{ b _{n}}^{6} \int _{\mathbb{R}^{3}} \bigl\vert v_{0}^{-} \bigr\vert ^{6}\,dx+ k \int _{ \mathbb{R}^{3}}f\bigl(t_{b _{n}}v_{0}^{-} \bigr)t_{b _{n}}v_{0}^{-}\,dx. \end{aligned}
(3.17)

By recalling Claim (a), up to a subsequence, we can deduce $$s_{b _{n}}\rightarrow s_{0}$$ and $$t_{b _{n}}\rightarrow t_{0}$$, then it follows from (3.16) and (3.17) that

\begin{aligned} &s_{0}^{2}\|v_{0}^{+}\|^{2}+\lambda s_{0}^{4}\int_{\mathbb{R}^{3}}\phi_{v_{0}^{+}}|v_{0}^{+}|^{2}dx+\lambda s_{0}^{2}t_{0}^{2}\int_{\mathbb{R}^{3}}\phi_{v_{0}^{-}}|v_{0}^{+}|^{2}\\ &=s_{0}^{6}\int_{\mathbb{R}^{3}}|v_{0}^{+}|^{6}dx+ k\int_{\mathbb{R}^{3}}f(s_{0}v_{0}^{+})s_{0}v_{0}^{+}dx \end{aligned}
(3.18)

and

\begin{aligned} &t_{0}^{2}\|v_{0}^{-}\|^{2}+\lambda t_{0}^{4}\int_{\mathbb{R}^{3}}\phi_{v_{0}^{-}}|v_{0}^{-}|^{2}+\lambda s_{0}^{2}t_{0}^{2}\int_{\mathbb{R}^{3}}\phi_{v_{0}^{+}}|v_{0}^{-}|^{2}\\ &=t_{0}^{6}\int_{\mathbb{R}^{3}}|v_{0}^{-}|^{6}dx+ k\int_{\mathbb{R}^{3}}f(t_{0}v_{0}^{-})t_{0}v_{0}^{-}dx. \end{aligned}
(3.19)

Thanks to $$v_{0}$$ being a weak solution of problem (1.11), we get

\begin{aligned} \begin{aligned}[b] &\bigl\Vert v_{0}^{+} \bigr\Vert ^{2}+\lambda \int _{\mathbb{R}^{3}}\phi_{v_{0}^{+}}|v_{0}^{+}|^{2}\,dx +\lambda \int_{\mathbb{R}^{3}}\phi_{v_{0}^{-}}|v_{0}^{+}|^{2}\,dx \\ &\quad = \int _{\mathbb{R}^{3}} \bigl\vert v_{0}^{+} \bigr\vert ^{6}\,dx+ k \int _{\mathbb{R}^{3}}f\bigl(v_{0}^{+} \bigr)v_{0}^{+}\,dx \end{aligned} \end{aligned}
(3.20)

and

\begin{aligned} \begin{aligned}[b] &\bigl\Vert v_{0}^{-} \bigr\Vert ^{2}+\lambda \int _{\mathbb{R}^{3}}\phi_{v_{0}^{-}}|v_{0}^{-}|^{2}\,dx +\lambda \int_{\mathbb{R}^{3}}\phi_{v_{0}^{+}}|v_{0}^{-}|^{2}\,dx \\ &\quad = \int _{\mathbb{R}^{3}} \bigl\vert v_{0}^{-} \bigr\vert ^{6}\,dx+ k \int _{\mathbb{R}^{3}}f\bigl(v_{0}^{-} \bigr)v_{0}^{-}\,dx. \end{aligned} \end{aligned}
(3.21)

By comparing formulas (3.18)–(3.21), it is obvious that $$(s_{0}, t_{0}) = (1, 1)$$. Similar to the proof of Lemma 2.1, we have

\begin{aligned} J^{0}_{k}(v_{0})\leq J^{0}_{k}(u_{0})= \lim_{n\rightarrow \infty }J^{ b _{n}}_{k}(u_{b _{n}}) \leq \lim_{n\rightarrow \infty }J^{ b _{n}}_{k}\bigl(s_{b _{n}}v^{+}_{0}+t_{b _{n}}v^{-}_{0} \bigr) =J^{0}_{k}\bigl(v^{+}_{0}+v^{-}_{0} \bigr)=J^{0}_{k}(v_{0}). \end{aligned}

The above inequality implies that $$u_{0}$$ is a least energy nodal solution of problem (1.11). So far, we have proved Theorem 1.3. □

## References

1. 1.

Benci, V., Fortunato, D.: Solitary waves of nonlinear Klein–Gordon equation coupled with Maxwell equations. Rev. Math. Phys. 14, 409–420 (2002)

2. 2.

Carrier, G.F.: On the non-linear vibration problem of the elastic string. Q. Appl. Math. 3, 157–165 (1945)

3. 3.

Chen, S., Tang, X.: Radial ground state sign-changing solutions for a class of asymptotically cubic or super-cubic Schrödinger–Poisson type problems. Rev. R. Acad. Cienc. Exactas Fís. Nat., Ser. A Mat. (2018). https://doi.org/10.1007/s13398-018-0493-0

4. 4.

Deng, Y., Peng, S., Shuai, W.: Existence and asymptotic behavior of nodal solutions for the Kirchhoff-type problems in $$\mathbb{R}^{3}$$. J. Funct. Anal. 269 3500–3527 (2015)

5. 5.

Jiang, Y., Zhou, H.: Schrödinger–Poisson system with steep potential well. J. Differ. Equ. 251, 582–608 (2011)

6. 6.

Kirchhoff, G.: Mechanik. Teubner, Leipzig (1883)

7. 7.

Li, F., Li, Y., Shi, J.: Existence of positive solutions to Schrödinger–Poisson type systems with critical exponent. Commun. Contemp. Math. 16, 1450036, 28pp (2014)

8. 8.

Li, F., Song, Z., Zhang, Q.: Existence and uniqueness results for Kirchhoff-Schrödinger-Poisson system with general singularity. Appl. Anal. 96, 2906–2916 (2017)

9. 9.

Li, G., Peng, S., Wang, C.: Multi-bump solutions for the nonlinear Schrödinger-Poisson system. J. Math. Phys. 52, 053505, 19pp (2011)

10. 10.

Liu, Y., Liu, C.: The ground state solutions for Kirchhoff–Schrödinger type equations with singular exponential nonlinearities in $$\mathbb{R}^{N}$$. Preprint

11. 11.

Liu, Z., Wang, Z., Zhang, J.: Infinitely many sign-changing solutions for the nonlinear Schrödinger-Poisson system. Ann. Mat. Pura. Appl. 4, 775–794 (2016)

12. 12.

Murcia, E., Siciliano, G.: Least energy radial sign-changing solution for the Schrödinger–Poisson system in $$\mathbb{R}^{3}$$ under an asymptotically cubic nonlinearity. J. Math. Anal. Appl. 474, 544–571 (2019)

13. 13.

Oplinger, D.: Frequency response of a nonlinear stretched string. J. Acoust. Soc. Am. 32, 1529–1538 (1960)

14. 14.

Ruiz, D.: The Schrödinger–Poisson equation under the effect of a nonlinear local term. J. Funct. Anal. 237, 655–674 (2006)

15. 15.

Ruiz, D.: On the Schrödinger–Poisson–Slater system: behavior of minimizers, radial and nonradial cases. Arch. Ration. Mech. Anal. 198, 349–368 (2010)

16. 16.

Sun, J., Wu, T., Feng, Z.: Multiplicity of positive solutions for a nonlinear Schrödinger-Poisson system. J. Differ. Equ. 260, 586–627 (2016)

17. 17.

Tang, X., Cheng, B.: Ground state sign-changing solutions for Kirchhoff type problems in bounded domains. J. Differ. Equ. 261, 2384–2402 (2016)

18. 18.

Wang, D.: Least energy sign-changing solutions of Kirchhoff-type equation with critical growth. J. Math. Phys. 61, 011501 (2020). https://doi.org/10.1063/1.5074163

19. 19.

Wang, D., Li, T., Hao, X.: Least-energy sign-changing solutions for Kirchhoff–Schrödinger–Poisson systems in $$\mathbb{R}^{3}$$. Bound. Value Probl. 2019, 75 (2019). https://doi.org/10.1186/s13661-019-1183-3

20. 20.

Wang, D., Zhang, H., Guan, W.: Existence of least-energy sign-changing solutions for Schrödinger–Poisson system with critical growth. J. Math. Anal. Appl. 479, 2284–2301 (2019)

21. 21.

Wang, J., Tian, L., Xu, J., Zhang, F.: Existence and concentration of positive solutions for semilinear Schrödinger–Poisson systems in $$\mathbb{R}^{3}$$. Calc. Var. Partial Differ. Equ. 48, 243–273 (2013)

22. 22.

Wang, Z., Zhou, H.: Sign-changing solutions for the nonlinear Schrödinger–Poisson system in $$\mathbb{R}^{3}$$. Calc. Var. Partial Differ. Equ. 52, 927–943 (2015)

23. 23.

Weth, T.: Energy bounds for entire nodal solutions of autonomous superlinear equations. Calc. Var. Partial Differ. Equ. 27, 421–437 (2006)

24. 24.

Willem, M.: Minimax Theorems. Birkhäuser, Bosten (1996)

25. 25.

Zhang, J., do Ó, J.M., Squassina, M.: Schrödinger–Poisson systems with a general critical nonlinearity. Commun. Contemp. Math. 19, 1650028, 16pp (2017)

26. 26.

Zhao, L., Zhao, F.: Positive solutions for Schrödinger–Poisson equations with a critical exponent. Nonlinear Anal. 70, 2150–2164 (2009)

27. 27.

Zhao, G., Zhu, X., Li, Y.: Existence of infinitely many solutions to a class of Kirchhoff-Schrödinger-Poisson system. Appl. Math. Comput. 256, 572–581 (2015)

28. 28.

Zhong, X., Tang, C.: Ground state sign-changing solutions for a Schrödinger–Poisson system with a 3-linear growth nonlinearity. J. Math. Anal. Appl. 455, 1956–1974 (2017)

29. 29.

Zhong, X., Tang, C.: Ground state sign-changing solutions for a Schrödinger-Poisson system with a critical nonlinearity in $$\mathbb{R}^{3}$$. Nonlinear Anal.: Real World Applications 39, 166–184 (2018)

### Acknowledgements

The authors would like to thank the editor and referees for their valuable comments.

Not applicable.

## Funding

Partially supported by the NSF of China (11790271), Guangdong Basic and Applied basic Research Foundation (2020A1515011019), Innovation and Development Project of Guangzhou University.

## Author information

Authors

### Contributions

The authors equally contributed in preparing this manuscript. All authors read and approved the final manuscript.

### Corresponding author

Correspondence to Chungen Liu.

## Ethics declarations

### Competing interests

The authors declare that there is no conflict of interest regarding the publication of this paper.

## Rights and permissions 