Least-energy sign-changing solutions for Kirchhoff–Schrödinger–Poisson systems in $\mathbb{R}^{3}$

Abstract

In this paper, we study the following Kirchhoff–Schrödinger–Poisson systems:

$$\textstyle\begin{cases} -(a+b\int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2}\,dx)\Delta u+V(x)u+\phi u=f(u), &x \in \mathbb{R}^{3}, \\ -\Delta \phi =u^{2}, &x\in \mathbb{R}^{3}, \end{cases} $$

where a, b are positive constants, $V\in \mathcal{C}(\mathbb{R} ^{3},\mathbb{R}^{+})$. By using constraint variational method and the quantitative deformation lemma, we obtain a least-energy sign-changing (or nodal) solution $u_{b}$ to this problem, and study the energy property of $u_{b}$. Moreover, we investigate the asymptotic behavior of $u_{b}$ as the parameter ${b\searrow 0}$.

1 Introduction

In this paper, we discuss the existence and asymptotic behavior of sign-changing solutions for the Kirchhoff–Schrödinger–Poisson systems

$$ \textstyle\begin{cases} -(a+b\int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2}\,dx)\Delta u+V(x)u+\phi u=f(u),&x \in \mathbb{R}^{3}, \\ -\Delta \phi =u^{2}, &x\in \mathbb{R}^{3}, \end{cases} $$

(1.1)

where $a,b>0$, $V\in \mathcal{C}(\mathbb{R}^{3},\mathbb{R}^{+})$ such that $H\subset H^{1}(\mathbb{R}^{3})$ and the embedding

$$ H\hookrightarrow L^{q}\bigl(\mathbb{R}^{3}\bigr),\quad 2< q< 6, $$

is compact, denoting by $H^{1}_{r}(\mathbb{R}^{3})$ the set of radially symmetric functions in the Sobolev space $H^{1}(\mathbb{R}^{3})$, we define

$$ H:= \textstyle\begin{cases} H^{1}_{r}(\mathbb{R}^{3})=\{u\in H^{1}(\mathbb{R}^{3}): u(x)=u( \vert x \vert )\}, & \text{if }V(x) \text{ is a constant}, \\ \{u\in D^{1,2}(\mathbb{R}^{3}): \int _{\mathbb{R}^{3}}(a \vert \nabla u \vert ^{2}+V(x)u ^{2})\,dx< +\infty \},& \text{if } V(x) \text{ is not a constant}. \end{cases} $$

with the norm

$$ \| u\| ^{2}= \int _{\mathbb{R}^{3}}\bigl(a \vert \nabla u \vert ^{2}+V(x)u ^{2}\bigr)\,dx. $$

As for f, we assume that $f\in C^{1}(\mathbb{R},\mathbb{R})$ and satisfy the following assumptions:

$(f_{1})$ :

$f(s)=0(|s|)$ as $s \rightarrow 0$;

$(f_{2})$ :

$\lim_{s\rightarrow \infty }\frac{f(s)}{s^{6}}=0$;

$(f_{3})$ :

$\lim_{s\rightarrow \infty }\frac{F(s)}{s^{4}}=+ \infty$, where $F(s)=\int ^{s}_{0}f(t)\,dt$;

$(f_{4})$ :

$\frac{f(s)}{|s|^{3}}$ is an increasing function of $s\in \mathbb{R}\backslash \{{0}\}$.

It is noticed that, to avoid involving too much details for checking the compactness, assumptions on V were first introduced in [60].

The nonlocal operator $(a+b\int _{\mathbb{R}^{3}}|\nabla u|^{2}\,dx) \Delta$ comes from the Kirchhof–Dirichlet problem

$$ \textstyle\begin{cases} -(a+b\int _{\varOmega } \vert \nabla u \vert ^{2}\,dx)\Delta u=f(u), &x\in \varOmega , \\ u=0, &x\in \partial \varOmega , \end{cases} $$

(1.2)

where $\varOmega \subset \mathbb{R}^{N}$ is a bounded domain or $\varOmega =\mathbb{R}^{N}$, $a>0$, $b>0$ and u satisfies some boundary conditions. Problem (1.2) is related to the following stationary analog of the equation of Kirchhoff type:

$$ u_{tt}-\biggl(a+b \int _{\varOmega } \vert \nabla u \vert ^{2}\,dx\biggr) \Delta u=f(x,u), $$

(1.3)

which was introduced by Kirchhoff [22] as a generalization of the well-known D’Alembert wave equation

$$ \rho \frac{\partial ^{2}u}{\partial t^{2}}-\biggl(\frac{p_{0}}{h}+ \frac{E}{2L} \int ^{L}_{0} \biggl\vert \frac{\partial u}{\partial x} \biggr\vert ^{2}\,dx\biggr)\frac{ \partial ^{2}u}{\partial x^{2}}=f(x,u), $$

(1.4)

for free vibration of elastic strings.

Kirchhoff’s model takes into account the changes in length of the string produced by transverse vibrations, so the nonlocal term appears. For more mathematical and physical background of Kirchhoff-type problems, we refer the reader to [8, 40, 50].

After the pioneer work of Lions [28], a lots of interesting results to problem (1.2) or similar problems were obtained in last decades; see for example [14,15,16,17,18,19, 24, 26, 32, 34, 36, 37, 41, 43, 45, 51, 52, 57, 59, 64, 65]. For the sake of space, many interesting results we do not cite here.

Especially, many authors pay their attention to find sign-changing solutions to problem (1.2) or similar problems and indeed some interesting results were obtained. For example, Zhang et al. [65] used the method of invariant sets of descent flow to obtain the existence of sign-changing solution of problem (1.2). It is noticed that, combining constraint variational methods and the quantitative deformation lemma, Shuai [45] studied the existence and asymptotic behavior of least-energy sign-changing solution to problem (1.2). Soon afterwards, under some more weak assumptions on f (especially, a Nehari type monotonicity condition been removed), Tang and Cheng [51] improved and generalized some results obtained in [45]. For more results on sign-changing solutions for Kirchhoff-type equations, we refer the reader to [14, 15, 17, 32, 34, 36, 43, 52] and the references therein.

When $a=1$, $b=0$, system (1.1) reduces to the Schrödinger–Poisson system

$$ \textstyle\begin{cases} -\Delta u+V(x)+\phi (x)u=f(u),&x\in \mathbb{R}^{3}, \\ -\Delta \phi =u^{2},&x\in \mathbb{R}^{3}. \end{cases} $$

(1.5)

System (1.5) comes from the time-dependent Schrödinger–Poisson equation, which describes quantum (nonrelativistic) particles interacting with the electromagnetic field generated by the motion. For more details of the mathematical and physical background of the system (1.5), we refer the reader to [6, 7] and the references therein. In the past several decades, there has been increasing attention toward systems (1.5) or similar problems, and the existence of positive solutions, multiple solutions, bound state solutions, multi-bump solutions, semiclassical state solutions has been investigated; see for example [3,4,5,6, 9, 25, 29, 33, 35, 42, 44, 48, 49, 54, 55, 58, 67].

For sign-changing solutions, Alves and Souto [1] proved that system (1.5) possesses a least-energy sign-changing solution in which $\mathbb{R}^{3}$ be replaced by bounded domains with smooth boundary. Soon afterwards, Alves, Souto and Soares [2] improved and generalized results obtained in [1] to on whole space $\mathbb{R}^{3}$. Via a constraint variational method combining the Brouwer degree theory, Wang and Zhou [60] investigated the existence of least-energy sign-changing solutions for the system (1.5) when $f(u)=|u|^{p-1}u$, $p\in (3,5)$. By using the constraint variation methods and the quantitative deformation lemma, Shuai and Wang [46] studied the existence and the asymptotic behavior of least-energy sign-changing solution for system (1.5). Latter, under some more weak assumptions on f, Chen and Tang [11] improve and generalize some results obtained in [46]. For the other work on a sign-changing solution of system (1.5) or similar problems, we refer the reader to [5, 20, 21, 27, 30, 56, 68] and the references therein. It is noticed that there are some interesting results, for example [10, 13, 53, 61], considered sign-changing solutions for other nonlocal problems.

For $u\in H$, let $\phi _{u}$ be unique solution of $-\triangle \phi =u ^{2}$ in $D^{1,2}(\mathbb{R}^{3})$, then

$$ \phi _{u}(x)=\frac{1}{4\pi } \int _{\mathbb{R}^{3}} \frac{u^{2}(y)}{ \vert x-y \vert }\,dy. $$

(1.6)

Using the expression of (1.6), we see that the system (1.1) is merely a single equation on u:

$$\begin{aligned} -\biggl(a+b \int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2}\,dx\biggr) \Delta u+V(x)u+\phi _{u}(x) u=f(u). \end{aligned}$$

(1.7)

So, the energy functional associated with system (1.1) is defined by

$$ \begin{aligned}[b] I_{b}(u):={}& \frac{1}{2} \int _{\mathbb{R}^{3}}\bigl(a \vert \nabla u \vert ^{2}+V(x)u^{2} \bigr)\,dx+ \frac{b}{4}\biggl( \int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2}\,dx \biggr)^{2}+\frac{1}{4} \int _{\mathbb{R}^{3}}\phi _{u}(x)u^{2}\,dx \\ &{}- \int _{\mathbb{R}^{3}}F(u)\,dx. \end{aligned} $$

(1.8)

Moreover, under our conditions, $I_{b}\in C^{1}(H,\mathbb{R})$, and we have

$$\begin{aligned} \begin{aligned}[b] \bigl\langle I_{b}'(u), \psi \bigr\rangle = {}& \int _{\mathbb{R}^{3}}\bigl(a\nabla u \nabla \psi +V(x)u\psi \bigr)\,dx+b \int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2}\,dx \int _{\mathbb{R}^{3}}\nabla u\nabla \psi \,dx \\ &{}+ \int _{\mathbb{R}^{3}}\phi _{u}u\psi \,dx- \int _{\mathbb{R}^{3}}f(u) \psi \,dx, \end{aligned} \end{aligned}$$

(1.9)

for any $u, \psi \in H$.

The weak solutions of system (1.1) are critical points of $I_{b}$. Moreover, we call u a sign-changing solution to (1.1) if u is a solution of (1.1) with $u^{\pm } \neq 0$, where

$$ u^{+}(x)=\max \bigl\{ u(x),0\bigr\} , \qquad u^{-}(x)=\min \bigl\{ u(x),0\bigr\} . $$

For system (1.1) contains both nonlocal operator and nonlocal nonlinear term, the study of system (1.1) become technically complicated. In recent years, there were some scholars paying attention to system (1.1) or similar problems; see [12, 23, 31, 38, 63, 66] and the references therein. However, to the best of our knowledge, few papers considered sign-changing solutions to system (1.1) or similar problems. Via gluing the function methods, Deng and Yang [12] studied the sign-changing solutions for system (1.1) with $f(u)=|u|^{p-2}u$, $p\in (4,6)$. But they did not study the energy property and asymptotic behavior of this solution.

Inspired by the work mentioned above, in this paper, we seek the least-energy sign-changing solutions to system (1.1). As in [1, 11, 17, 45, 46, 59], we first try to seek a minimizer of the energy functional $I_{b}$ over the following constraint:

$$\mathcal{M}_{b}=\bigl\{ u\in H:u^{\pm }\neq 0,~\bigl\langle I_{b}'(u),u^{+} \bigr\rangle =\bigl\langle I_{b}'(u),u^{-}\bigr\rangle =0\bigr\} , $$

and then will prove that the minimizer is a sign-changing solution of system (1.1).

The following are the main results of this paper.

Theorem 1.1

If the assumptions $(f_{1})$–$(f_{4})$ hold, then the problem (1.1) has a least-energy sign-changing solution $u_{b}$, which has precisely two nodal domains.

Theorem 1.2

Under the assumptions of Theorem 1.1,

$$ I_{b}(u_{b})>2c_{b} $$

where $c_{b}:=\inf_{u\in \mathcal{N}_{b}}I_{b}(u)$, $\mathcal{N}_{b}:=\{u\in H \backslash \{{0}\}: \langle I_{b}'(u),u \rangle =0\}$ and $u_{b}$ is the least-energy sign-changing solution in H obtained in Theorem 1.1. In particular, $c_{b}$ is achieved either by a positive or a negative function.

Theorem 1.3

If the assumptions of Theorem 1.1 hold, then, for any sequence $\{b_{n}\}$ with $b_{n}\rightarrow 0$ as $n\rightarrow \infty$, there exists a subsequence, still denoted by $\{b_{n}\}$, such that $u_{b_{n}}\rightarrow u_{0}$ strongly in H as $n\rightarrow \infty$, where $u_{0}$ is a least-energy sign-changing solution in H of the problem

$$ \textstyle\begin{cases} -a\triangle u+V(x)u+\phi u=f(u),&x\in \mathbb{R}^{3}, \\ -\Delta \phi =u^{2}, &x\in \mathbb{R}^{3}, \end{cases} $$

(1.10)

which changes sign only once.

2 Some technical lemmas

In this section, we prove some technical lemmas related to the existence of sign-changing solutions of system (1.1).

Lemma 2.1

Assume that $(f_{1})$–$(f_{4})$ hold, if $u\in H$ with $u^{\pm } \neq 0$, then:

• (i) There exists a unique pair $(s_{u},t_{u})$ of positive numbers such that $s_{u}u^{+}+t_{u}u^{-}\in \mathcal{M}_{b}$.

• (ii) The vector $(s_{u},t_{u})$ is the unique maximum point of the function φ: $\mathbb{R}_{+}\times \mathbb{R}_{+}\rightarrow \mathbb{R}$ defined as $\varphi (s,t)=I_{b}(su^{+}+tu^{-})$.

Proof

(i) Having fixed $u\in H$ with $u^{\pm }\neq 0$, let

$$\begin{aligned} &\begin{aligned}[b] g(s,t) ={} & \bigl\langle I_{b}' \bigl(s u^{+}+t u^{-}\bigr),s u^{+}\bigr\rangle \\ = {}& \int _{\mathbb{R}^{3}}\bigl[a\nabla \bigl(s u^{+}+t u^{-}\bigr)\nabla \bigl(su^{+}\bigr)+V(x) \bigl(su ^{+}+tu^{-}\bigr)s u^{+}\bigr]\,dx \\ &{} +b \int _{\mathbb{R}^{3}} \bigl\vert \nabla \bigl(s u^{+}+t u^{-}\bigr) \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}}\nabla \bigl(s u^{+}+t u^{-} \bigr)\nabla \bigl(s u^{+}\bigr)\,dx \\ &{} + \int _{\mathbb{R}^{3}}\phi _{su^{+}+t u^{-}}\bigl(s u^{+}+t u^{-}\bigr) \bigl(s u ^{+}\bigr)\,dx- \int _{\mathbb{R}^{3}}f\bigl(s u^{+}+t u^{-}\bigr)s u^{+}\,dx \\ ={} & s ^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+b \int _{\mathbb{R}^{3}}\bigl(s ^{2} \bigl\vert \nabla u^{+} \bigr\vert ^{2}+t ^{2} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\bigr)\,dx \int _{\mathbb{R}^{3}}s ^{2} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \\ &{} + \int _{\mathbb{R}^{3}}s ^{4}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx+ \int _{\mathbb{R}^{3}}s ^{2}t^{2}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx- \int _{\mathbb{R}^{3}}f\bigl(s u^{+}\bigr)tu^{+}\,dx \\ ={} & s^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+bs ^{4} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u ^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+bs ^{2}t^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u ^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &{} +s ^{4} \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx+s ^{2}t^{2} \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx- \int _{\mathbb{R}^{3}}f\bigl(s u^{+}\bigr)su^{+}\,dx, \end{aligned} \end{aligned}$$

(2.1)

$$\begin{aligned} &\begin{aligned}[b] h(s ,t)= {}& \bigl\langle I_{b}' \bigl(s u^{+}+tu^{-}\bigr),t u^{-}\bigr\rangle \\ ={} & t ^{2} \bigl\Vert u^{-} \bigr\Vert ^{2}+b \int _{\mathbb{R}^{3}}\bigl(s^{2} \bigl\vert \nabla u^{+} \bigr\vert ^{2}+t ^{2} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\bigr)\,dx \int _{\mathbb{R}^{3}}t^{2} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &{} + \int _{\mathbb{R}^{3}}t^{4}\phi _{u^{-}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx+ \int _{\mathbb{R}^{3}}s ^{2}t^{2}\phi _{u^{+}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx- \int _{\mathbb{R}^{3}}f\bigl(t u^{-}\bigr)t u^{-}\,dx \\ ={} & t^{2} \bigl\Vert u^{-} \bigr\Vert ^{2}+bt^{4} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \biggr)^{2}+bt^{2}s ^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &{} +t^{4} \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx+t^{2}s ^{2} \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx- \int _{\mathbb{R}^{3}}f\bigl(tu ^{-}\bigr)t u^{-} \,dx. \end{aligned} \end{aligned}$$

(2.2)

We will show that there exists $r\in (0,R)$ such that

$$\begin{aligned} g(r,t)>0, \qquad h(s,r)>0, \quad \forall s,t\in [r,R], \end{aligned}$$

(2.3)

and

$$\begin{aligned} g(R,t)< 0, \qquad h(s,R)< 0, \quad \forall s,t\in [r,R], \end{aligned}$$

(2.4)

where $R>0$ is a constant.

By assumption $(f_{1})$ and $(f_{2})$, for any $\varepsilon >0$, there exists a positive constant $C_{\varepsilon }$ such that

$$\begin{aligned} f(s)s\leq \varepsilon s^{2}+C_{\varepsilon } \vert s \vert ^{6},\quad \text{for all }s \in \mathbb{R}. \end{aligned}$$

(2.5)

Then we have

$$\begin{aligned} g(s,t)\geq{} & s ^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+bs ^{4} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+bs ^{2}t^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &{} +s ^{4} \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx+s ^{2}t^{2} \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx-\varepsilon \int _{\mathbb{R}^{3}} \bigl\vert s u^{+} \bigr\vert ^{2}\,dx \\ &{}-C_{\varepsilon } \int _{\mathbb{R}^{3}} \bigl\vert s u^{+} \bigr\vert ^{6}\,dx \\ \geq{} &s ^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+s ^{4} \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u ^{+} \bigr\vert ^{2}\,dx+s ^{2}t^{2} \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx -C_{1}\varepsilon s^{2} \bigl\Vert u^{+} \bigr\Vert ^{2} \\ &{}- C_{2}C_{\varepsilon }s^{6} \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx \\ \geq {}&(1-C_{1}\varepsilon )s ^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+s ^{4} \int _{\mathbb{R} ^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx+s ^{2}t^{2} \int _{\mathbb{R}^{3}} \phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx \\ &{}- C_{3}s^{6} \int _{\mathbb{R}^{3}} \bigl\vert u^{+} \bigr\vert ^{6}\,dx, \end{aligned}$$

(2.6)

where $C_{1}$, $C_{2}$, $C_{3}$ are positive constants.

On the other hand, since $u^{+}\neq 0$, there exists a constant $\delta >0$ such that $\operatorname{meas}\{x\in \mathbb{R}^{3},u^{+}>\delta \}>0$. In addition, by $(f_{3})$ and $(f_{4})$, we deduce that, for any $L>0$, there exists $T>0$ such that $\frac{f(\omega )}{\omega ^{3}}>L$ for all $\omega >T$. Therefore, for $s>\frac{T}{\delta }$, we have

$$\begin{aligned} \int _{\mathbb{R}^{3}}f\bigl(s u^{+}\bigr)s u^{+}\,dx \geq \int _{\{u^{+}(x)>\delta \}}\frac{f(s u^{+})}{(s u^{+})^{3}}\bigl(s u^{+} \bigr)^{4}\,dx \geq Ls^{4} \int _{\{u^{+}(x)>\delta \}}\bigl(u^{+}\bigr)^{4}\,dx. \end{aligned}$$

(2.7)

Choose L sufficiently large so that

$$\begin{aligned} &L\int _{\{u^{+}(x)>\delta \}}\bigl(u^{+}\bigr)^{4}\,dx\\ &\quad >2 \biggl(\frac{3b}{2} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+\frac{b}{2} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \biggr)^{2} + \int _{\mathbb{R} ^{3}}\phi _{u} \bigl\vert u^{+} \bigr\vert ^{2}\,dx \biggr). \end{aligned}$$

Suppose $t\leq s$, we have

$$\begin{aligned} g(s,t)\leq {}& s^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+bs^{4} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+bs^{2}t^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &{} +s^{4} \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx+s^{2}t^{2} \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx -Ls^{4} \int _{\{u^{+}(x)>\delta \}}\bigl(u^{+}\bigr)^{4}\,dx \\ \leq {}&s^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+\frac{b}{2}s^{4} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+\frac{b}{2}t^{4} \biggl( \int _{\mathbb{R} ^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \biggr)^{2} +s^{4} \int _{\mathbb{R}^{3}} \phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx \\ &{}+s^{2}t^{2} \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx +bs^{4} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}-Ls^{4} \int _{\{u^{+}(x)>\delta \}}\bigl(u^{+}\bigr)^{4}\,dx \\ \leq {}&s^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+\frac{3b}{2}s^{4} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+\frac{b}{2}s^{4} \biggl( \int _{\mathbb{R} ^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \biggr)^{2} +s^{4} \int _{\mathbb{R}^{3}}\phi _{u} \bigl\vert u^{+} \bigr\vert ^{2}\,dx \\ &{}-Ls^{4} \int _{\{u^{+}(x)>\delta \}}\bigl(u^{+}\bigr)^{4}\,dx \\ = {}&s^{4} \biggl(\frac{3b}{2} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr) ^{2}+\frac{b}{2} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \biggr) ^{2} \\ &{}+ \int _{\mathbb{R}^{3}}\phi _{u} \bigl\vert u^{+} \bigr\vert ^{2}\,dx-L \int _{\{u^{+}(x)>\delta \}}\bigl(u^{+}\bigr)^{4}\,dx \biggr) \\ &{}+s^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}. \end{aligned}$$

(2.8)

Similarly, we derive that

$$\begin{aligned} h(s,t) \geq {}&(1-C_{4}\varepsilon )t^{2} \bigl\Vert u^{-} \bigr\Vert ^{2}+t ^{4} \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx+s ^{2}t^{2} \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx \\ &{}- C_{6}t^{5} \int _{\mathbb{R}^{3}} \bigl\vert u^{-} \bigr\vert ^{6}\,dx \end{aligned}$$

(2.9)

and

$$\begin{aligned} h(s,t)\leq{} &t^{4} \biggl(\frac{3b}{2} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \biggr)^{2}+\frac{b}{2} \biggl( \int _{\mathbb{R} ^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}\\ &{} + \int _{\mathbb{R}^{3}}\phi _{u} \bigl\vert u ^{-} \bigr\vert ^{2}\,dx-L \int _{\{u^{-}(x)>\delta \}}\bigl(u^{-}\bigr)^{4}\,dx \biggr) +t^{2} \bigl\Vert u^{-} \bigr\Vert ^{2}, \end{aligned}$$

(2.10)

if $s\leq t$.

Hence, in view of (2.6), (2.8), (2.9), (2.10) and Miranda’s theorem [39], there exists some point $(s_{u},t_{u})$ such that $g(s_{u},t_{u})=h(s_{u},t_{u})=0$. That is, $s_{u}u^{+}+t_{u}u^{-}\in \mathcal{M}_{b}$.

We now prove that the pair $(s_{u},t_{u})$ is unique and consider two situations.

Case 1. $u\in \mathcal{M}_{b}$.

If $u\in \mathcal{M}_{b}$, we have

$$\begin{aligned} \bigl\langle I_{b}'(u),u^{+}\bigr\rangle = \bigl\langle I_{b}'(u),u^{-}\bigr\rangle =0. \end{aligned}$$

That is,

$$\begin{aligned} \begin{aligned}[b] & \bigl\Vert u^{+} \bigr\Vert ^{2}+b \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr) ^{2}+b \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &\quad {} + \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx + \int _{\mathbb{R} ^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx = \int _{\mathbb{R}^{3}}f\bigl(u^{+}\bigr)u^{+}\,dx, \end{aligned} \end{aligned}$$

(2.11)

$$\begin{aligned} \begin{aligned}[b] & \bigl\Vert u^{-} \bigr\Vert ^{2}+b \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \biggr) ^{2}+b \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &\quad {} + \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx + \int _{\mathbb{R} ^{3}}\phi _{u^{+}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx = \int _{\mathbb{R}^{3}}f\bigl(u^{-}\bigr)u^{-}\,dx. \end{aligned} \end{aligned}$$

(2.12)

We prove that $(s_{u},t_{u})=(1,1)$ is the only pair of numbers so that $s_{u}u^{+}+t_{u}u^{-}\in \mathcal{M}_{b}$.

Suppose that $(\tilde{s}_{u},\tilde{t}_{u})$ is another pair of numbers so that $\tilde{s}_{u}u^{+}+\tilde{t}_{u}u^{-}\in \mathcal{M} _{b}$. According to the definition of $\mathcal{M}_{b}$, it is easy to obtain

$$\begin{aligned} & \tilde{s}_{u}^{2} \bigl\Vert u^{+} \bigr\Vert ^{2} +b\tilde{s}_{u}^{4} \biggl( \int _{ \mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+b\tilde{s}_{u}^{2} \tilde{t}_{u}^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &\quad {} +\tilde{s}_{u}^{4} \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx + \tilde{s}_{u}^{2} \tilde{t}_{u}^{2} \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u ^{+} \bigr\vert ^{2}\,dx = \int _{\mathbb{R}^{3}}f\bigl(\tilde{s}_{u}u^{+}\bigr) \tilde{s}_{u}u ^{+}\,dx \end{aligned}$$

(2.13)

and

$$\begin{aligned} & \tilde{t}_{u}^{2} \bigl\Vert u^{-} \bigr\Vert ^{2} +b\tilde{t}_{u}^{4} \biggl( \int _{ \mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \biggr)^{2}+b\tilde{t}_{u}^{2} \tilde{s}_{u}^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &\quad {} +\tilde{t}_{u}^{4} \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx + \tilde{t}_{u}^{2} \tilde{s}_{u}^{2} \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u ^{-} \bigr\vert ^{2}\,dx = \int _{\mathbb{R}^{3}}f\bigl(\tilde{t}_{u}u^{-}\bigr) \tilde{t}_{u}u ^{-}\,dx. \end{aligned}$$

(2.14)

Without loss of generality, we can suppose that $0<\tilde{s}_{u} \leq \tilde{t}_{u}$. Thus, from (2.13), we get

$$\begin{aligned} & \tilde{s}_{u}^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+b\tilde{s}_{u}^{4} \biggl( \int _{ \mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+b\tilde{s}_{u}^{4} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &\qquad {} +\tilde{s}_{u}^{4} \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx + \tilde{s}_{u}^{4} \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx \\ & \quad \leq \tilde{s}_{u}^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+b\tilde{s}_{u}^{4} \biggl( \int _{ \mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+b\tilde{s}_{u}^{2} \tilde{t}_{u}^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &\qquad {} +\tilde{s}_{u}^{4} \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx + \tilde{s}_{u}^{2} \tilde{t}_{u}^{2} \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u ^{+} \bigr\vert ^{2}\,dx \\ &\quad = \int _{\mathbb{R}^{3}}f\bigl(\tilde{s}_{u}u^{+}\bigr) \tilde{s}_{u}u^{+}\,dx. \end{aligned}$$

So,

$$\begin{aligned} & \frac{1}{\tilde{s}_{u}^{2}} \bigl\Vert u^{+} \bigr\Vert ^{2}+b \biggl( \int _{\mathbb{R} ^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+b \int _{\mathbb{R}^{3}} \bigl\vert \nabla u ^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx \\ &\quad {} + \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx+ \int _{\mathbb{R} ^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx \leq \int _{\mathbb{R}^{3}}\frac{f( \tilde{s}_{u}u^{+})}{\tilde{s}_{u}^{3}}u^{+}\,dx. \end{aligned}$$

(2.15)

Combining (2.15) with (2.11), we get

$$\begin{aligned} \bigl(\tilde{s}_{u}^{-2}-1\bigr) \bigl\Vert u^{+} \bigr\Vert ^{2} \leq & \int _{\mathbb{R}^{3}} \biggl(\frac{f(x, \tilde{s}_{u}u^{+})}{(\tilde{s}_{u}u^{+})^{3}}-\frac{f(x,u^{+})}{(u ^{+})^{3}} \biggr) \bigl(u^{+}\bigr)^{4}\,dx. \end{aligned}$$

(2.16)

If $\tilde{s_{u}}<1$, the left side of the above inequality is positive, which is absurd because the right side is negative by condition $(f_{4})$.

Therefore, we obtain $1\leq \tilde{s_{u}}\leq \tilde{t_{u}}$.

Similarly, by (2.12), (2.14) and $0<\tilde{s}_{u} \leq \tilde{t}_{u}$, one has

$$\begin{aligned} \bigl(\tilde{t}_{u}^{-2}-1\bigr) \bigl\Vert u^{-} \bigr\Vert ^{2} \geq & \int _{\mathbb{R}^{3}} \biggl(\frac{f(x, \tilde{t}_{u}u^{-})}{(\tilde{t}_{u}u^{-})^{3}}-\frac{f(x,u^{-})}{(u ^{-})^{3}} \biggr) \bigl(u^{-}\bigr)^{4}\,dx. \end{aligned}$$

(2.17)

Thanks to $(f_{4})$, we must have $\tilde{t_{u}}\leq 1$.

So, $\tilde{s_{u}}=\tilde{t_{u}}=1$.

Case 2. $u\notin \mathcal{M}_{b}$.

If $u\notin \mathcal{M}_{b}$, then there exists a pair of positive numbers $(s_{u},t_{u})$ such that $s_{u}u^{+}+t_{u}u^{-} \in \mathcal{M}_{b}$. Assume that there exists another pair of positive numbers $(s_{u}',t_{u}')$ such that $s_{u}'u^{+}+t_{u}'u^{-}\in \mathcal{M}_{b}$. Define $v:=s_{u}u^{+}+t_{u}u^{-}$ and $v':=s_{u}'u ^{+}+t_{u}'u^{-}$, we get

$$\begin{aligned} \frac{s_{u}'}{s_{u}}v^{+}+\frac{t_{u}'}{t_{u}}v^{-}=s_{u}'u^{+}+t_{u}'u ^{-}=v'\in \mathcal{M}_{b}. \end{aligned}$$

Thanks to $v\in \mathcal{M}_{b}$, we find that $s_{u}=s_{u}'$ and $t_{u}=t_{u}'$.

(ii) From (i), we know that $(s_{u},t_{u})$ is the unique critical point of φ in $\mathbb{R}_{+}\times \mathbb{R}_{+}$. By the hypothesis $(f_{3})$, we conclude that $\varphi (s,t)\rightarrow -\infty$ uniformly as $|(s,t)|\rightarrow \infty$, so it is sufficient to show that a maximum point cannot be achieved on the boundary of $(\mathbb{R}_{+},\mathbb{R}_{+})$. If we may suppose that $(0,\bar{t})$ is a maximum point of φ, it is easy to deduce that

$$\begin{aligned} \varphi '_{s}(s,\bar{t})= {}& \bigl(I_{b} \bigl(su^{+}+\bar{t}u^{-}\bigr)\bigr)'_{s} \\ ={} &s \int _{\mathbb{R}^{3}}a \bigl\vert \nabla u^{+} \bigr\vert ^{2}+V(x) \bigl\vert u^{+} \bigr\vert ^{2}\,dx \\ &{} +s^{3} \biggl( \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+s \bar{t}^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R} ^{3}} \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx+s^{3} \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u ^{+} \bigr\vert ^{2}\,dx \\ &{} +\frac{1}{2}s\bar{t}^{2} \int _{\mathbb{R}^{3}}\phi _{u^{+}} \bigl\vert u^{-} \bigr\vert ^{2}\,dx + \frac{1}{2}\bar{t}^{2}s \int _{\mathbb{R}^{3}}\phi _{u^{-}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx \\ &{} - \int _{\mathbb{R}^{3}}f\bigl(su^{+}\bigr)u^{+}\,dx \\ >{}&0, \end{aligned}$$

for s small enough.

That is, $\varphi (s,\bar{t})$ is an increasing function with respect to s if s is small enough.

From the above discussion, we know that the pair $(0,\bar{t})$ is not a maximum point of φ in $\mathbb{R}_{+}\times \mathbb{R}_{+}$. □

Next, we consider the minimization problem

$$\begin{aligned} m_{b}:=\inf \bigl\{ I_{b}(u):u\in \mathcal{M}_{b}\bigr\} . \end{aligned}$$

(2.18)

Lemma 2.2

Assume that $(f_{1})$–$(f_{4})$ hold, then $m_{b}>0$ is achieved.

Proof

Firstly, we prove $m_{b}>0$.

For every $u\in \mathcal{M}_{b}$, we have $\langle I_{b}'(u),u\rangle =0$. So, according to (2.5) and the Sobolev embedding, we have

$$\begin{aligned} \Vert u \Vert ^{2} &\leq \int _{\mathbb{R}^{3}}\bigl(a \vert \nabla u \vert ^{2}+V(x)u^{2} \bigr)\,dx+b \biggl( \int _{\mathbb{R}^{3}} \vert \nabla u \vert ^{2}\,dx \biggr)^{2}+ \int _{\mathbb{R}^{3}}\phi _{u} \vert u \vert ^{2} \,dx= \int _{\mathbb{R}^{3}}f(u)u\,dx \\ & \leq \varepsilon \int _{\mathbb{R}^{3}} \vert u \vert ^{2} \,dx+C_{\varepsilon } \int _{\mathbb{R}^{3}} \vert u \vert ^{6}\,dx \\ & \leq \varepsilon C_{1} \Vert u \Vert ^{2}+C_{2} \Vert u \Vert ^{6}. \end{aligned}$$

(2.19)

Selecting $\varepsilon =\frac{1}{2C_{1}}$, it is easy to see that there exists a constant $\alpha >0$ such that $\|u\|^{2}\geq \alpha$.

On the other hand, we obtain, by the condition $(f_{5})$,

$$ H(t):=f(t)t-4F(t)\geq 0,\quad t\in \mathbb{R}, $$

(2.20)

and $H(t)$ is increasing when $t>0$ and decreasing when $t<0$.

Then we have

$$\begin{aligned} I_{b}(u)=I_{b}(u)-\frac{1}{4}\bigl\langle I_{b}'(u),u\bigr\rangle \geq \frac{1}{4} \Vert u \Vert ^{2}\geq \frac{1}{4}\alpha . \end{aligned}$$

That is, $m_{b}\geq \frac{1}{4}\alpha >0$.

In the following, we prove that $m_{b}$ is achieved.

Let $\{u_{n}\}\subset \mathcal{M}_{b}$ be so that $I_{b}(u _{n})\rightarrow m_{b}$. Then $\{u_{n}\}$ is bounded in H. And there exists $u_{b}\in H$ such that $u_{n}^{\pm }$ converges to $u_{b}^{ \pm }$ weakly in H. Since $u_{n}\in \mathcal{M}_{b}$, we can get $\langle I_{b}'(u_{n}),u_{n}^{\pm }\rangle =0$, i.e.,

$$\begin{aligned} \bigl\Vert u_{n}^{\pm } \bigr\Vert ^{2}+b \int _{\mathbb{R}^{3}} \vert \nabla u_{n} \vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u_{n}^{\pm } \bigr\vert ^{2}\,dx+ \int _{\mathbb{R}^{3}} \phi _{u_{n}} \bigl\vert u_{n}^{\pm } \bigr\vert ^{2}\,dx= \int _{\mathbb{R}^{3}}f\bigl(u_{n} ^{\pm } \bigr)u_{n}^{\pm }\,dx. \end{aligned}$$

(2.21)

Analogous to the discussion in (2.19), there exists $\beta >0$ such that $\|u_{n}^{\pm }\|^{2}\geq \beta$ for all $n\in \mathbb{N}$.

Thanks to $(f_{1})$ and $(f_{2})$, for any $\delta >0$, there is a positive constants $C_{\delta }$ such that

$$\begin{aligned} f(s)s\leq \delta s^{2}+\delta \vert s \vert ^{6}+C_{\delta } \vert s \vert ^{p}, \quad \text{for all }s\in \mathbb{R}. \end{aligned}$$

So, by $u_{n}\in \mathcal{M}_{b}$, we have

$$\begin{aligned} \beta \leq \bigl\| u_{n}^{\pm }\bigr\| ^{2}< \int _{\mathbb{R}^{3}}f\bigl(u_{n} ^{\pm } \bigr)u_{n}^{\pm }\,dx \leq \delta \int _{\mathbb{R}^{3}} \bigl\vert u_{n}^{ \pm } \bigr\vert ^{2}\,dx+C_{\delta } \int _{\mathbb{R}^{3}} \bigl\vert u_{n}^{\pm } \bigr\vert ^{p}\,dx + \delta \int _{\mathbb{R}^{3}} \bigl\vert u_{n}^{\pm } \bigr\vert ^{6}\,dx. \end{aligned}$$

In view of the boundedness of $\{u_{n}\}$, there exists $C_{1}>0$ that satisfies

$$\begin{aligned} \beta \leq \delta C_{1}+C_{\delta } \int _{\mathbb{R}^{3}} \bigl\vert u_{n}^{ \pm } \bigr\vert ^{p}\,dx. \end{aligned}$$

Choosing $\delta =\frac{\beta }{2C_{1}}$, from the above equality, we can obtain

$$\begin{aligned} \int _{\mathbb{R}^{3}} \bigl\vert u_{n}^{\pm } \bigr\vert ^{p}\,dx\geq \frac{\beta }{2C_{2}}>0. \end{aligned}$$

So, according to the compactness embedding $H\hookrightarrow L^{q}( \mathbb{R}^{3})$ for $2< q<2^{*}$, we have

$$\begin{aligned} \int _{\mathbb{R}^{3}} \bigl\vert u_{b}^{\pm } \bigr\vert ^{p}\,dx\geq \frac{\beta }{2C_{2}}. \end{aligned}$$

(2.22)

That is, $u_{b}^{\pm }\neq 0$.

By Lemma 2.1, there exists $(s_{u_{b}},t_{u_{b}})\in (0, \infty )\times (0,\infty )$ such that

$$\begin{aligned} \bar{u_{b}}:=s_{u_{b}}u_{b}^{+}+t_{u_{b}}u_{b}^{-} \in \mathcal{M}_{b}. \end{aligned}$$

We assert that

$$\begin{aligned} 0< s_{u_{b}},t_{u_{b}}\leq 1. \end{aligned}$$

In fact, by $(f_{1})$, $(f_{2})$ and the compactness lemma of Strauss [47] we see that

$$\begin{aligned} \begin{aligned} &\lim _{n\rightarrow \infty } \int _{\mathbb{R}^{3}}f\bigl(u_{n}^{ \pm } \bigr)u_{n}^{\pm }\,dx = \int _{\mathbb{R}^{3}}f\bigl(u_{b}^{\pm } \bigr)u_{b}^{ \pm }\,dx, \\ &\lim _{n\rightarrow \infty } \int _{\mathbb{R}^{3}}F\bigl(u_{n}^{ \pm }\bigr)\,dx = \int _{\mathbb{R}^{3}}F\bigl(u_{b}^{\pm }\bigr)\,dx. \end{aligned} \end{aligned}$$

(2.23)

Since the embedding $H\hookrightarrow D^{1,2}$ is continuous and we have weak semicontinuity of the norm, we have

$$\begin{aligned} & \bigl\Vert u_{b}^{\pm } \bigr\Vert ^{2}+b \int _{\mathbb{R}^{3}} \vert \nabla u_{b} \vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u_{b}^{\pm } \bigr\vert ^{2}\,dx \\ & \quad \leq \lim _{n\rightarrow \infty }\inf \biggl( \bigl\Vert u_{n}^{\pm } \bigr\Vert ^{2}+b \int _{\mathbb{R}^{3}} \vert \nabla u_{n} \vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u_{n}^{\pm } \bigr\vert ^{2}\,dx \biggr). \end{aligned}$$

(2.24)

By (1.6) and the Hardy–Littlewood–Sobolev inequality, we have

$$\begin{aligned} \lim _{n\rightarrow \infty }\inf \int _{\mathbb{R}^{3}}\phi _{u _{n}} \bigl\vert u_{n}^{\pm } \bigr\vert ^{2}\,dx = \int _{\mathbb{R}^{3}}\phi _{u_{b}} \bigl\vert u_{b} ^{\pm } \bigr\vert ^{2}\,dx. \end{aligned}$$

(2.25)

Therefore, thanks to $\{u_{n}\}\subset \mathcal{M}_{b}$, (2.23), (2.24) and (2.25), we obtain

$$\begin{aligned} \bigl\Vert u_{b}^{\pm } \bigr\Vert ^{2}+b \int _{\mathbb{R}^{3}} \vert \nabla u_{b} \vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u_{b}^{\pm } \bigr\vert ^{2}\,dx+ \int _{\mathbb{R}^{3}} \phi _{u_{b}} \bigl\vert u_{b}^{\pm } \bigr\vert ^{2}\,dx\leq \int _{\mathbb{R}^{3}}f\bigl(u_{b}^{ \pm } \bigr)u_{b}^{\pm }\,dx. \end{aligned}$$

That is,

$$ \bigl\langle I'_{b}(u_{b}),u_{b}^{\pm } \bigr\rangle \leq \liminf_{n\rightarrow \infty } \bigl\langle I'_{b}(u_{n}),u_{n}^{\pm } \bigr\rangle =0. $$

(2.26)

Suppose that $s_{u_{b}}\geq t_{u_{b}}>0$, thanks to $s_{u_{b}}u_{b} ^{+}+t_{u_{b}}u_{b}^{-}\in \mathcal{M}_{b}$, we have

$$\begin{aligned} & s_{u_{b}}^{2} \bigl\Vert u_{b}^{+} \bigr\Vert ^{2}+b s_{u_{b}}^{4} \biggl( \int _{\mathbb{R} ^{3}} \bigl\vert \nabla u_{b}^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+b s_{u_{b}}^{4} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u_{b}^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u_{b}^{-} \bigr\vert ^{2}\,dx \\ &\qquad {} +s_{u_{b}}^{4} \int _{\mathbb{R}^{3}}\phi _{u_{b}^{+}} \bigl\vert u_{b}^{+} \bigr\vert ^{2}\,dx+s _{u_{b}}^{4} \int _{\mathbb{R}^{3}}\phi _{u_{b}^{-}} \bigl\vert u_{b}^{+} \bigr\vert ^{2}\,dx \\ &\quad \geq s_{u_{b}}^{2} \bigl\Vert u^{+} \bigr\Vert ^{2}+b s_{u_{b}}^{4} \biggl( \int _{\mathbb{R} ^{3}} \bigl\vert \nabla u_{b}^{+} \bigr\vert ^{2}\,dx \biggr)^{2}+b s_{u_{b}}^{2}t_{u}^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla u_{b}^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla u_{b}^{-} \bigr\vert ^{2}\,dx \\ &\qquad {}+s_{u_{b}}^{4} \int _{\mathbb{R}^{3}}\phi _{u_{b}^{+}} \bigl\vert u_{b}^{+} \bigr\vert ^{2}\,dx+s _{u_{b}}^{2}t_{u_{b}}^{2} \int _{\mathbb{R}^{3}}\phi _{u_{b}^{-}} \bigl\vert u_{b} ^{+} \bigr\vert ^{2}\,dx \\ &\quad = \int _{\mathbb{R}^{3}}f\bigl(s_{u_{b}}u_{b}^{+} \bigr)s_{u_{b}}u_{b}^{+}\,dx. \end{aligned}$$

(2.27)

Combining (2.26) and (2.27), we have

$$\begin{aligned} \biggl(\frac{1}{s_{u_{b}}^{2}}-1 \biggr) \bigl\Vert u_{b}^{+} \bigr\Vert ^{2}\geq \int _{\mathbb{R}^{3}} \biggl(\frac{f(s_{u_{b}}u_{b}^{+})}{(s_{u_{b}}u _{b}^{+})^{3}}-\frac{f(u_{b}^{+})}{(u_{b}^{+})^{3}} \biggr) \bigl(u_{b}^{+}\bigr)^{4}\,dx. \end{aligned}$$

If $s_{u_{b}}>1$, the left-hand side of this inequality is negative. But from $(f_{4})$, the right-hand side of this inequality is positive. So, we have $s_{u_{b}}\leq 1$.

From the above discussions and (2.20), we get

$$\begin{aligned} m_{b}\leq {}& I_{b}(\bar{u_{b}})=I_{b}( \bar{u_{b}})-\frac{1}{4}\bigl\langle I_{b}'( \bar{u_{b}}),\bar{u_{b}}\bigr\rangle \\ ={} & \frac{1}{4} \bigl\Vert s_{u_{b}}u_{b}^{+} \bigr\Vert ^{2}+\frac{1}{4} \bigl\Vert t_{u_{b}}u_{b} ^{-} \bigr\Vert ^{2} +\frac{1}{4} \int _{\mathbb{R}^{3}}\bigl(f\bigl(s_{u_{b}}u_{b}^{+} \bigr)s _{u_{b}}u_{b}^{+}-4F\bigl(s_{u_{b}}u_{b}^{+} \bigr)\bigr)\,dx \\ &{} +\frac{1}{4} \int _{\mathbb{R}^{3}}\bigl(f\bigl(t_{u_{b}}u_{b}^{-} \bigr)t_{u_{b}}u _{b}^{-}-4F\bigl(t_{u_{b}}u_{b}^{-} \bigr)\bigr)\,dx \\ \leq {}& \frac{1}{4} \Vert u_{b} \Vert ^{2}+ \frac{1}{4} \int _{\mathbb{R}^{3}}\bigl(f(u _{b})u_{b}-4F(u_{b}) \bigr)\,dx \\ \leq {}& \lim _{n\rightarrow \infty }\inf \biggl(I_{b}(u_{n})- \frac{1}{4}\bigl\langle I_{b}'(u_{n}),u_{n} \bigr\rangle \biggr)=m_{b}. \end{aligned}$$

It follows from the above fact that $s_{u_{b}}=t_{u_{b}}=1$. Then $\bar{u_{b}}=u_{b}$ and $I_{b}(u_{b})=m_{b}$. The proof if finished. □

3 Proof of main results

Proof of Theorem 1.1

We just prove that the minimizer $u_{b}$ for (2.18) is indeed a sign-changing solution of system (1.1), i.e., $I_{b}'(u_{b})=0$.

Since $u_{b}\in \mathcal{M}_{b}$, we have $I_{b}'(u_{b})u_{b}^{+}=0=I _{b}'(u_{b})u_{b}^{-}$. By (ii) of Lemma 2.1, for $(s,t)\in (\mathbb{R}_{+}\times \mathbb{R}_{+})$ and $(s,t)\neq (1,1)$, we obtain

$$\begin{aligned} I_{b}\bigl(su_{b}^{+}+tu_{b}^{-} \bigr)< I_{b}\bigl(u_{b}^{+}+u_{b}^{-} \bigr)=m_{b}. \end{aligned}$$

(3.1)

If $I_{b}'(u_{b})\neq 0$, then exist $\delta >0$ and $\lambda >0$ such that $\|I_{b}'(v)\|\geq \lambda$ for all $\|v-u_{b}\|\leq 3\delta$.

Choose $\sigma \in (0,\min \{1/2,\delta /\sqrt{2}\|u\|\})$. Let $\varOmega =(1-\sigma , 1+\sigma )\times (1-\sigma ,1+\sigma )$ and $\eta (s,t):=su_{b}^{+}+tu_{b}^{-}$, $(s,t)\in \varOmega$. From (ii) of Lemma 2.1, one has

$$\begin{aligned} \bar{m}_{b}:=\max \limits _{\partial \varOmega }I_{b}\circ \eta < m_{b}. \end{aligned}$$

(3.2)

For $\varepsilon :=\min \{(m_{b}-\bar{m}_{b})/2, \lambda \delta /8\}$ and $S_{\delta }:=B(u_{b},\delta )$. By Lemma 2.3 of [62], there exists a deformation ξ such that:

• (a) $\xi (1,u)=u$ if $u\notin I_{b}^{-1}([m_{b}-2\varepsilon ,m _{b}+2\varepsilon ])\cap S_{2\delta }$;

• (b) $\xi (1,I_{b}^{m_{b}+\varepsilon }\cap s)\subset I_{b}^{m _{b}-\varepsilon }$;

• (c) $I_{b}(\xi (1,u))\leq I_{b}(u)$ for all $u\in H$.

Firstly, we need to prove that

$$ \max_{(s,t)\in \bar{\varOmega }}I_{b}\bigl(\xi \bigl(1, \eta (s,t)\bigr)\bigr)< m_{b}. $$

(3.3)

By Lemma 2.1, we know $I_{b}(\eta (s,t))\leq m_{b}< m_{b}+ \varepsilon$, which shows that

$$ \eta (s,t)\in I_{b}^{m_{b}+\varepsilon }. $$

At the same time, we have

$$ \bigl\Vert \eta (s,t)-u_{b} \bigr\Vert ^{2} \leq 2\bigl((s-1)^{2} \bigl\Vert u_{b}^{+} \bigr\Vert ^{2}+(t-1)^{2} \bigl\Vert u_{b}^{-} \bigr\Vert ^{2}\bigr) \leq 2\sigma \Vert u_{b} \Vert ^{2}\leq \delta ^{2}, $$

that is, $\eta (s,t)\in \mathcal{S}_{\delta }$, $\forall (s,t)\in \bar{ \varOmega }$.

Therefore, according to $(b)$, we have $I_{b}(\xi (1,\eta (s,t)))< m- \varepsilon$. Hence, (3.3) holds.

In the following, we show that $\xi (1,\eta (D))\cap \mathcal{M}_{b} \neq \emptyset$, which contradicts the definition of $m_{b}$.

Let us set $\psi (s,t):=\xi (1,\eta (s,t))$ and

$$\begin{aligned} \varPsi _{0}(s,t):= {}& \bigl(\bigl\langle I_{b}' \bigl(\eta (s,t)\bigr),u_{b}^{+} \bigr\rangle ,\bigl\langle I_{b}'\bigl(\eta (s,t)\bigr),u_{b}^{-} \bigr\rangle \bigr) \\ ={} & \bigl(\bigl\langle I_{b}'\bigl(su_{b}^{+}+tu_{b}^{-} \bigr),u_{b}^{+} \bigr\rangle ,\bigl\langle I_{b}'\bigl(su_{b}^{+}+tu_{b}^{-} \bigr),u_{b}^{-}\bigr\rangle \bigr) \\ := {}& \bigl(\varphi _{1}(s,t),\varphi _{2}(s,t)\bigr), \\ \varPsi _{1}(s,t):= {}& \biggl(\frac{1}{s}\bigl\langle I_{b}'\bigl( \psi (s,t)\bigr),\psi ^{+}(s,t) \bigr\rangle ,\frac{1}{t}\bigl\langle I_{b}'\bigl( \psi (s,t)\bigr), \psi ^{-}(s,t)\bigr\rangle \biggr). \end{aligned}$$

By direct calculation, we have

$$\begin{aligned}& \begin{aligned}[b] \frac{\partial \varphi _{1}(s,t)}{\partial s}\bigg|_{(1,1)} ={}& \bigl\Vert u_{b}^{+} \bigr\Vert ^{2}+3b \biggl( \int _{\mathbb{R}^{N}} \bigl\vert \nabla u_{b}^{+} \bigr\vert ^{2}\,dx \biggr) ^{2}+b \int _{\mathbb{R}^{N}} \bigl\vert \nabla u_{b}^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R} ^{N}} \bigl\vert \nabla u_{b}^{-} \bigr\vert ^{2}\,dx \\ &{} +3 \int _{\mathbb{R}^{3}}\phi _{u_{b}^{+}} \bigl\vert u_{b}^{+} \bigr\vert ^{2}\,dx + \int _{\mathbb{R}^{3}}\phi _{u_{b}^{-}} \bigl\vert u_{b}^{+} \bigr\vert ^{2}\,dx- \int _{\mathbb{R}^{3}}f'\bigl(u_{b}^{+} \bigr) \bigl(u_{b}^{+}\bigr)^{2}\,dx, \end{aligned} \end{aligned}$$

(3.4)

$$\begin{aligned}& \frac{\partial \varphi _{1}(s,t)}{\partial t}\bigg|_{(1,1)}=2b \int _{\mathbb{R}^{N}} \bigl\vert \nabla u_{b}^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{N}} \bigl\vert \nabla u_{b}^{-} \bigr\vert ^{2}\,dx+2 \int _{\mathbb{R}^{3}}\phi _{u_{b}^{-}} \bigl\vert u_{b} ^{+} \bigr\vert ^{2}\,dx, \end{aligned}$$

(3.5)

$$\begin{aligned}& \frac{\partial \varphi _{2}(s,t)}{\partial s}\bigg|_{(1,1)}=2b \int _{\mathbb{R}^{N}} \bigl\vert \nabla u_{b}^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{N}} \bigl\vert \nabla u_{b}^{-} \bigr\vert ^{2}\,dx+2 \int _{\mathbb{R}^{3}}\phi _{u_{b}^{+}} \bigl\vert u_{b} ^{-} \bigr\vert ^{2}\,dx, \end{aligned}$$

(3.6)

$$\begin{aligned}& \begin{aligned}[b] \frac{\partial \varphi _{2}(s,t)}{\partial t}\bigg|_{(1,1)} ={}& \bigl\Vert u_{b}^{-} \bigr\Vert ^{2}+3b \biggl( \int _{\mathbb{R}^{N}} \bigl\vert \nabla u_{b}^{-} \bigr\vert ^{2}\,dx \biggr) ^{2}+b \int _{\mathbb{R}^{N}} \bigl\vert \nabla u_{b}^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R} ^{N}} \bigl\vert \nabla u_{b}^{-} \bigr\vert ^{2}\,dx \\ &{} +3 \int _{\mathbb{R}^{3}}\phi _{u_{b}^{-}} \bigl\vert u_{b}^{-} \bigr\vert ^{2}\,dx + \int _{\mathbb{R}^{3}}\phi _{u_{b}^{+}} \bigl\vert u_{b}^{-} \bigr\vert ^{2}\,dx- \int _{\mathbb{R}^{3}}f'\bigl(u_{b}^{-} \bigr) \bigl(u_{b}^{-}\bigr)^{2}\,dx. \end{aligned} \end{aligned}$$

(3.7)

Let

$$M=\left[\begin{array}{cc}\frac{\partial {\phi }_1(s,t)}{\partial s}{|}_{(1,1)}& \frac{\partial {\phi }_2(s,t)}{\partial s}{|}_{(1,1)}\\ \frac{\partial {\phi }_1(s,t)}{\partial t}{|}_{(1,1)}& \frac{\partial {\phi }_2(s,t)}{\partial t}{|}_{(1,1)}\end{array}\right].$$

By condition $(f_{5})$, for $s\neq 0$, we have

$$ f'(s)s^{2}-3f(s)s>0. $$

Then

$$\begin{aligned}& \frac{\partial \varphi _{1}(s,t)}{\partial s}\bigg|_{(1,1)}< -2 \bigl\Vert u_{b}^{+} \bigr\Vert ^{2}-2b \int _{\mathbb{R}^{N}} \bigl\vert \nabla u_{b}^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{N}} \bigl\vert \nabla u_{b}^{-} \bigr\vert ^{2}\,dx-2 \int _{\mathbb{R}^{3}} \phi _{u_{b}^{-}} \bigl\vert u_{b}^{+} \bigr\vert ^{2}\,dx, \\& \frac{\partial \varphi _{2}(s,t)}{\partial t}\bigg|_{(1,1)}< -2 \bigl\Vert u_{b}^{-} \bigr\Vert ^{2}-2b \int _{\mathbb{R}^{N}} \bigl\vert \nabla u_{b}^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{N}} \bigl\vert \nabla u_{b}^{-} \bigr\vert ^{2}\,dx-2 \int _{\mathbb{R}^{3}} \phi _{u_{b}^{+}} \bigl\vert u_{b}^{-} \bigr\vert ^{2}\,dx. \end{aligned}$$

Therefore, we have

$$ \det M>0. $$

Since $\varPsi _{0}(s,t)$ is a $C^{1}$ function and $(1,1)$ is the unique isolated zero point of $\varPsi _{0}$, by using degree theory, we deduce that deg$(\varPsi _{0},D,0)=1$. So, combining (3.2) with (a), we know that $g=h$ on ∂D. Consequently, we get deg$(\varPsi _{1},D,0)=1$. Hence, $\varPsi _{1}(s_{0},t_{0})=0$ for some $(s_{0},t_{0})\in D$, such that

$$\begin{aligned} \xi \bigl(1,\eta (s_{0},t_{0})\bigr)=\psi (s_{0},t_{0})\in \mathcal{M}_{b}, \end{aligned}$$

which is a contradiction according to (3.3).

From the above discussion, we conclude that $u_{b}$ is a sign-changing solution for problem (1.1).

Finally, we prove that $u_{b}$ has exactly two nodal domains. By contradiction, we suppose that $u_{b}$ has at least three nodal domains $\varOmega _{1}$, $\varOmega _{2}$, $\varOmega _{3}$. Without loss generality, we can suppose that $u_{b}>0$ a.e. in $\varOmega _{1}$ and $u_{b}<0$ a.e. in $\varOmega _{2}$. Define

$$\begin{aligned} u_{b_{i}}:=\chi _{\varOmega _{i}}u_{b}, \quad i=1,2,3, \end{aligned}$$

where

$$ \chi _{\varOmega _{i}}= \textstyle\begin{cases} 1, &x\in \varOmega _{i}, \\ 0, &x\in \mathbb{R}^{N}\backslash \varOmega _{i}, \end{cases} $$

and $u_{b_{i}}\neq 0$ and $\langle I'(u_{b}),u_{b_{i}}\rangle =0$ for $i=1,2,3$.

Let $v:=u_{b_{1}}+u_{b_{2}}$, then $v^{+}=u_{b_{1}}$ and $v^{-}=u_{b _{2}}$, i.e., $v^{\pm }\neq 0$. Then there exists a unique pair $(s_{v},t_{v})$ of positive numbers such that

$$s_{v}u_{b_{1}}+t_{v}u_{b_{2}}\in \mathcal{M}_{b}. $$

Hence, we have

$$\begin{aligned} I_{b}(s_{v}u_{b_{1}}+t_{v}u_{b_{2}}) \geq m_{b}. \end{aligned}$$

(3.8)

Thanks to $\langle I_{b}'(u_{b}),u_{b_{i}} \rangle =0$, we obtain $\langle I_{b}'(v),v^{\pm } \rangle <0$.

Similar to the proof of Lemma 2.2, we have

$$(s_{v},t_{v})\in (0,1]\times (0,1]. $$

So, by (2.20), we have

$$\begin{aligned} 0 ={}&\frac{1}{4}\bigl\langle I_{b}'(u_{b}),u_{b_{3}} \bigr\rangle \\ ={}&\frac{1}{4} \Vert u_{b_{3}} \Vert ^{2}+ \frac{b}{4} \biggl( \int _{\mathbb{R} ^{3}} \vert \nabla u_{b_{3}} \vert ^{2}\,dx \biggr)^{2}+\frac{b}{4} \int _{\mathbb{R} ^{3}} \vert \nabla u_{b_{1}} \vert ^{2}\,dx \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{3}} \vert ^{2}\,dx \\ &{}+\frac{b}{4} \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{2}} \vert ^{2}\,dx \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{3}} \vert ^{2}\,dx+\frac{1}{4} \int _{\mathbb{R}^{3}}\phi _{u_{b}}u_{b_{3}}^{2} \,dx-\frac{1}{4} \int _{\mathbb{R}^{3}}f(u_{b_{3}})u_{b_{3}}\,dx \\ \leq{}& \frac{1}{4} \Vert u_{b_{3}} \Vert ^{2}+ \frac{b}{4} \biggl( \int _{\mathbb{R} ^{3}} \vert \nabla u_{b_{3}} \vert ^{2}\,dx \biggr)^{2}+\frac{b}{4} \int _{\mathbb{R} ^{3}} \vert \nabla u_{b_{1}} \vert ^{2}\,dx \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{3}} \vert ^{2}\,dx \\ &{}+\frac{b}{4} \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{2}} \vert ^{2}\,dx \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{3}} \vert ^{2}\,dx+\frac{1}{4} \int _{\mathbb{R}^{3}}\phi _{u_{b}}u_{b_{3}}^{2} \,dx- \int _{\mathbb{R}^{3}}F(u _{b_{3}})\,dx \\ < {}&I_{b}(u_{b_{3}})+\frac{b}{4} \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{1}} \vert ^{2}\,dx \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{3}} \vert ^{2}\,dx+\frac{b}{4} \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{2}} \vert ^{2}\,dx \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{3}} \vert ^{2}\,dx \\ &{}+\frac{1}{4} \int _{\mathbb{R}^{3}}\phi _{u_{b_{1}}}u_{b_{3}}^{2} \,dx+ \frac{1}{4} \int _{\mathbb{R}^{3}}\phi _{u_{b_{2}}}u_{b_{3}}^{2} \,dx. \end{aligned}$$

Consequently, from the above inequality, we obtain

$$\begin{aligned} m_{b}\leq{} & I_{b}(s_{v}u_{b_{1}}+t_{v}u_{b_{2}}) \\ ={} &I_{b}(s_{v}u_{b_{1}}+t_{v}u_{b_{2}})- \frac{1}{4}\bigl\langle I_{b}'(s _{v}u_{b_{1}}+t_{v}u_{b_{2}}),s_{v}u_{b_{1}}+t_{v}u_{b_{2}} \bigr\rangle \\ = {}&\frac{s_{v}^{2} \Vert u_{b_{1}} \Vert ^{2}+t_{v}^{2} \Vert u_{b_{2}} \Vert ^{2}}{4}+ \int _{\mathbb{R}^{3}} \biggl(\frac{1}{4}f(s_{v}u_{b_{1}})s_{v}u_{b_{1}}-F(s _{v}u_{b_{1}}) \biggr)\,dx \\ &{} + \int _{\mathbb{R}^{3}} \biggl(\frac{1}{4}f(t_{v}u_{b_{2}})t_{v}u _{b_{2}}-F(t_{v}u_{b_{2}}) \biggr)\,dx \\ \leq {}& \frac{ \Vert u_{b_{1}} \Vert ^{2}+ \Vert u_{b_{2}} \Vert ^{2}}{4}\\ &{}+ \int _{\mathbb{R}^{3}} \biggl(\frac{1}{4}f(u_{b_{1}})u_{b_{1}}-F(u_{b _{1}}) \biggr)\,dx + \int _{\mathbb{R}^{3}} \biggl(\frac{1}{4}f(u_{b_{2}})u _{b_{2}}-F(u_{b_{2}}) \biggr)\,dx \\ = {}&I_{b}(u_{b_{1}}+u_{b_{2}})-\frac{1}{4}\bigl\langle I_{b}'(u_{b_{1}}+u _{b_{2}}),u_{b_{1}}+u_{b_{2}} \bigr\rangle \\ < {} &I_{b}(u_{b_{1}})+I_{b}(u_{b_{2}})+I_{b}(u_{b_{3}})+ \frac{1}{4} \int _{\mathbb{R}^{3}}\phi _{u_{b_{2}}}u_{b_{1}}^{2} \,dx +\frac{1}{4} \int _{\mathbb{R}^{3}}\phi _{u_{b_{3}}}u_{b_{1}}^{2} \,dx \\ &{} +\frac{1}{4} \int _{\mathbb{R}^{3}}\phi _{u_{b_{1}}}u_{b_{2}}^{2} \,dx+ \frac{1}{4} \int _{\mathbb{R}^{3}}\phi _{u_{b_{3}}}u_{b_{2}}^{2} \,dx + \frac{1}{4} \int _{\mathbb{R}^{3}}\phi _{u_{b_{1}}}u_{b_{3}}^{2} \,dx+ \frac{1}{4} \int _{\mathbb{R}^{3}}\phi _{u_{b_{2}}}u_{b_{3}}^{2} \,dx \\ &{}+\frac{b}{2} \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{1}} \vert ^{2}\,dx \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{3}} \vert ^{2}\,dx+\frac{b}{2} \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{2}} \vert ^{2}\,dx \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{3}} \vert ^{2}\,dx \\ &{}+\frac{b}{2} \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{1}} \vert ^{2}\,dx \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{2}} \vert ^{2}\,dx \\ ={} &I_{b}(u_{b}) \\ = {}&m_{\lambda }, \end{aligned}$$

which is impossible. Thus, $u_{b}$ has exactly two nodal domains. □

Proof of Theorem 1.2

Similar to the proof of Lemma 2.2, for each $b>0$, there exists $v_{b}\in \mathcal{N}_{b}$ so that $I_{b}(v_{b})=c_{b}>0$. By standard arguments, it is easy to see that the critical points of $I_{b}$ on $\mathcal{N}_{b}$ are critical points of $I_{b}$ in H, that is, $I_{b}'(v_{b})=0$. Therefore, $v_{0}$ is a ground-state solution of (1.1).

According to Theorem 1.1, problem (1.1) has a sign-changing solution $u_{b}$ which changes sign only once. Let $u_{b}=u^{+}_{b}+u^{-}_{b}$, as in the proof of Lemma 2.1, there exist unique $s_{u_{b}^{+}}>0$ and $t_{u_{b}^{-}}>0$ such that

$$\begin{aligned} s_{u_{b}^{+}}u_{b}^{+}\in \mathcal{N}_{b}, \qquad t_{u_{b}^{-}}u_{b}^{-}\in \mathcal{N}_{b}. \end{aligned}$$

Thanks to $\langle I_{b}'(u_{b}^{+}),u_{b}^{+}\rangle <0$, $\langle I _{b}'(u_{b}^{-}),u_{b}^{-}\rangle <0$, and similar to proof in Lemma 2.2, we obtain $s_{u_{b}^{+}}\in (0,1)$ and $t_{u_{b} ^{-}}\in (0,1)$.

Thus, by (ii) of Lemma 2.1, one has

$$\begin{aligned} 2c_{b}\leq I_{b}\bigl(s_{u_{b}^{+}}u_{b}^{+} \bigr)+I_{b}\bigl(t_{u_{b}^{-}}u_{b}^{-}\bigr) \leq I_{b}\bigl(s_{u_{b}^{+}}u_{b}^{+}+t_{u_{b}^{-}}u_{b}^{-} \bigr)< I_{b}\bigl(u _{b}^{+}+u_{b}^{-} \bigr)=m_{b}. \end{aligned}$$

It follows that $c_{b}>0$, which cannot be achieved by a sign-changing function. □

Lastly, we shall analyze the asymptotic behavior of $u_{b}$ as $b\rightarrow 0$. In the following, we regard $b>0$ as a parameter in problem (1.1).

Proof of Theorem 1.3

For any $b>0$, let $u_{b}\in H$ be the least-energy sign-changing solution of (1.1) obtained in Theorem 1.1. We shall proceed through three steps to complete the proof.

Step 1. If $b_{n}\rightarrow 0$ as $n\rightarrow \infty$, then $\{u_{b_{n}}\}$ is bounded in H.

Choose a nonzero function $\eta \in C_{c}^{\infty }(\mathbb{R}^{3})$ with $\eta ^{\pm }\neq 0$. In view of $(f_{3})$, for any $b\in [0,1]$, there is a pair $(\lambda _{1},\lambda _{2})\in (\mathbb{R}_{+}\times \mathbb{R}_{+})$ independent of b, such that

$$\begin{aligned} \bigl\langle I_{b}'\bigl(\lambda _{1}\eta ^{+}+\lambda _{2}\eta ^{-}\bigr),\lambda _{1} \eta ^{+}\bigr\rangle < 0 \end{aligned}$$

and

$$\begin{aligned} \bigl\langle I_{b}'\bigl(\lambda _{1}\eta ^{+}+\lambda _{2}\eta ^{-}\bigr),\lambda _{2} \eta ^{-}\bigr\rangle < 0. \end{aligned}$$

Hence, according to Lemma 2.1 and similar to the proof in Lemma 2.2, for any $b\in [0,1]$, there exists a unique pair $(s_{\eta }(b),t_{\eta }(b))\in (0,1]\times (0,1]$ so that

$$\begin{aligned} \bar{\eta }:=s_{\eta }(b)\lambda _{1}\eta ^{+}+t_{\eta }(b)\lambda _{2} \eta ^{-}\in \mathcal{M}_{b}. \end{aligned}$$

(3.9)

Thus, for any $b\in [0,1]$, we have

$$\begin{aligned} I_{b}(u_{b}) \leq {}& I_{b}(\bar{\eta })=I_{b}(\bar{\eta })-\frac{1}{4} \bigl\langle I_{b}'(\bar{\eta }),\bar{\eta }\bigr\rangle \\ = {}& \frac{1}{4} \Vert \bar{\eta } \Vert ^{2}+ \frac{1}{4} \int _{\mathbb{R}^{3}}\bigl(f(\bar{ \eta })\bar{\eta }-4F(\bar{\eta }) \bigr)\,dx \\ \leq{} &\frac{1}{4} \Vert \bar{\eta } \Vert ^{2}+ \frac{1}{4} \int _{\mathbb{R}^{3}}\bigl(C _{1}\bar{\eta }^{2}+C_{2} \bar{\eta }^{6}\bigr)\,dx \\ \leq{} & \frac{1}{4} \bigl\Vert \lambda _{1}\eta ^{+} \bigr\Vert ^{2}+\frac{1}{4} \bigl\Vert \lambda _{2}\eta ^{-} \bigr\Vert ^{2} + \frac{1}{4} \int _{\mathbb{R}^{3}}\bigl(C_{1}\lambda _{1} ^{2} \bigl\vert \eta ^{+} \bigr\vert ^{2}+C_{1} \lambda _{2}^{2} \bigl\vert \eta ^{-} \bigr\vert ^{2}\bigr)\,dx \\ &{}+\frac{1}{4} \int _{\mathbb{R}^{3}}\bigl(C_{2}\lambda _{1}^{6} \bigl\vert \eta ^{+} \bigr\vert ^{6}+C _{2} \lambda _{2}^{6} \bigl\vert \eta ^{-} \bigr\vert ^{6}\bigr)\,dx \\ :={} &C^{\ast }, \end{aligned}$$

(3.10)

where $C^{\ast }$ does not depend on b. So, letting $n\rightarrow \infty$, it follows that

$$\begin{aligned} C^{\ast }+1\geq I_{b_{n}}(u_{b_{n}})=I_{b_{n}}(u_{b_{n}})- \frac{1}{4} \bigl\langle I_{b_{n}}'(u_{b_{n}}),u_{b_{n}} \bigr\rangle \geq \frac{1}{4} \Vert u _{b_{n}} \Vert ^{2}, \end{aligned}$$

(3.11)

which implies that $\{u_{b_{n}}\}$ is bounded in H.

Step 2. Problem (1.10) possesses one sign-changing solution $u_{0}$.

Since $\{u_{b_{n}}\}$ is bounded in H according to Claim 1, going if necessary to a subsequence, there exists $u_{0}\in H$ such that

$$\begin{aligned} &u_{b_{n}}\rightharpoonup u_{0}\quad \text{in }H, \\ &u_{b_{n}}\rightarrow u_{0}\quad \text{in }L^{q}\bigl( \mathbb{R}^{3}\bigr) \text{ for }q\in (2,6), \\ &u_{b_{n}}\rightarrow u_{0}\quad \text{a.e. in } \mathbb{R}^{3}. \end{aligned}$$

(3.12)

We assert that $u_{0}$ is a weak solution of (1.10). In fact, because $u_{b_{n}}$ is the sign-changing solution of (1.1) with $b=b_{n}$, then, by (3.12), we have

$$\begin{aligned} \Vert u_{b_{n}}-u_{0} \Vert ^{2}={} & \bigl\langle I_{b_{n}}'(u_{b_{n}})-I_{0}'(u_{0}),u _{b_{n}}-u_{0}\bigr\rangle \\ &{}-b_{n} \int _{\mathbb{R}^{3}} \vert \nabla u_{b_{n}} \vert ^{2}\,dx \int _{\mathbb{R}^{3}}\nabla u_{b_{n}}(\nabla u_{b_{n}} - \nabla u_{0})\,dx \\ &{} - \int _{\mathbb{R}^{3}}\phi _{u_{b_{n}}}u_{b_{n}}(u_{b_{n}}-u_{0}) \,dx + \int _{\mathbb{R}^{3}}f(u_{b_{n}}) (u_{b_{n}}-u_{0}) \,dx \\ &{} - \int _{\mathbb{R}^{3}}f(u_{0}) (u_{b_{n}}-u_{0}) \,dx\rightarrow 0 ,\quad \mbox{as } n\rightarrow \infty . \end{aligned}$$

So, $u_{0}\neq 0$ and $u_{0}$ changes sign only once.

Step 3. Problem (1.10) possesses a least-energy sign-changing solution $v_{0}$. Furthermore, there exists a unique pair $(s_{b_{n}}, t_{b_{n}})\in [0,\infty )\times [0,\infty )$ such that $s_{b_{n}}v^{+}_{0}+t_{b_{n}}v^{-}_{0}\in \mathcal{M}_{b_{n}}$ and $(s_{b_{n}}, t_{b_{n}})\rightarrow (1,1)$ as $n\rightarrow \infty$.

With a similar argument to the proof of Theorem 1.1, we see that (1.10) possesses a least-energy sign-changing solution $v_{0}$ (for the existence of $v_{0}$, we also refer to [46]), where $I^{\lambda }_{0}(v_{0})=c^{\lambda }_{0}$ and $(I^{\lambda } _{0})'(v_{0})=0$.

Hence, by Lemma 2.1, it is easy to see that there uniquely exists the pair $(s_{b_{n}}, t_{b_{n}})\in (0,\infty )\times (0, \infty )$ such that $s_{b_{n}}v^{+}_{0}+t_{b_{n}}v^{-}_{0}\in \mathcal{M}_{b_{n}}$. Then we have

$$\begin{aligned}& \begin{aligned}[b] &(s_{b_{n}})^{2} \bigl\Vert v_{0}^{+} \bigr\Vert ^{2} +b_{n}(s_{b_{n}})^{4} \biggl( \int _{ \mathbb{R}^{3}} \bigl\vert \nabla v_{0}^{+} \bigr\vert ^{2}\,dx \biggr)^{2} +b_{n}(s_{b_{n}}t _{b_{n}})^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla v_{0}^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla v_{0}^{-} \bigr\vert ^{2}\,dx \\ &\qquad {}+(s_{b_{n}})^{4} \int _{\mathbb{R}^{3}}\phi _{v_{0}^{+}} \bigl\vert v_{0}^{+} \bigr\vert ^{2}\,dx +(s_{b_{n}}t_{b_{n}})^{2} \int _{\mathbb{R}^{3}}\phi _{v_{0}^{-}} \bigl\vert v_{0} ^{+} \bigr\vert ^{2}\,dx \\ &\quad = \int _{\mathbb{R}^{3}}f\bigl(s_{b_{n}}v_{0}^{+} \bigr)s_{b_{n}}v_{0} ^{+}\,dx, \end{aligned} \end{aligned}$$

(3.13)

$$\begin{aligned}& \begin{aligned}[b] &(t_{b_{n}})^{2} \bigl\Vert v_{0}^{-} \bigr\Vert ^{2} +b_{n}(t_{b_{n}})^{4} \biggl( \int _{ \mathbb{R}^{3}} \bigl\vert \nabla v_{0}^{-} \bigr\vert ^{2}\,dx \biggr)^{2} +b_{n}(t_{b_{n}}s _{b_{n}})^{2} \int _{\mathbb{R}^{3}} \bigl\vert \nabla v_{0}^{+} \bigr\vert ^{2}\,dx \int _{\mathbb{R}^{3}} \bigl\vert \nabla v_{0}^{-} \bigr\vert ^{2}\,dx \\ &\qquad {}+(t_{b_{n}})^{4} \int _{\mathbb{R}^{3}}\phi _{v_{0}^{-}} \bigl\vert v_{0}^{-} \bigr\vert ^{2}\,dx +(t_{b_{n}}s_{b_{n}})^{2} \int _{\mathbb{R}^{3}}\phi _{v_{0}^{+}} \bigl\vert v_{0} ^{-} \bigr\vert ^{2}\,dx \\ &\quad = \int _{\mathbb{R}^{3}}f\bigl(t_{b_{n}}v_{0}^{-} \bigr)s_{b_{n}}v_{0} ^{-}\,dx. \end{aligned} \end{aligned}$$

(3.14)

According to $(f_{3})$, $(f_{4})$ and $b_{n}\rightarrow 0$ as $n\rightarrow \infty$, $\{s_{b_{n}}\}$ and $\{t_{b_{n}}\}$ are bounded. Up to a subsequence, suppose that $s_{b_{n}}\rightarrow s_{0}$ and $t_{b_{n}}\rightarrow t_{0}$, then it follows from (3.13) and (3.14) that

$$\begin{aligned} s_{0}^{2} \bigl\Vert v_{0}^{+} \bigr\Vert ^{2}+s_{0}^{4} \int _{\mathbb{R}^{3}}\phi _{v_{0} ^{+}} \bigl\vert v_{0}^{+} \bigr\vert ^{2}\,dx +s_{0}^{2}t_{0}^{2} \int _{\mathbb{R}^{3}} \phi _{v_{0}^{-}} \bigl\vert v_{0}^{+} \bigr\vert ^{2}\,dx+ = \int _{\varOmega }f\bigl(x,s_{0}v_{0}^{+} \bigr)s _{0}v_{0}^{+}\,dx \end{aligned}$$

(3.15)

and

$$\begin{aligned} t_{0}^{2} \bigl\Vert v_{0}^{-} \bigr\Vert ^{2}+t_{0}^{4} \int _{\mathbb{R}^{3}}\phi _{v_{0} ^{-}} \bigl\vert v_{0}^{-} \bigr\vert ^{2}\,dx +s_{0}^{2}t_{0}^{2} \int _{\mathbb{R}^{3}} \phi _{v_{0}^{+}} \bigl\vert v_{0}^{-} \bigr\vert ^{2}\,dx = \int _{\varOmega }f\bigl(x,t_{0}v_{0}^{-} \bigr)t _{0}v_{0}^{-}\,dx. \end{aligned}$$

(3.16)

Thanks to $v_{0}$ being a sign-changing solution of problem (1.10), we get

$$\begin{aligned} \bigl\Vert v_{0}^{+} \bigr\Vert ^{2}+ \int _{\mathbb{R}^{3}}\phi _{v_{0}^{+}} \bigl\vert v_{0}^{+} \bigr\vert ^{2}\,dx + \int _{\mathbb{R}^{3}}\phi _{v_{0}^{-}} \bigl\vert v_{0}^{+} \bigr\vert ^{2}\,dx = \int _{\varOmega }f\bigl(x,v_{0}^{+} \bigr)v_{0}^{+}\,dx \end{aligned}$$

(3.17)

and

$$\begin{aligned} \bigl\Vert v_{0}^{-} \bigr\Vert ^{2}+ \int _{\mathbb{R}^{3}}\phi _{v_{0}^{-}} \bigl\vert v_{0}^{-} \bigr\vert ^{2}\,dx + \int _{\mathbb{R}^{3}}\phi _{v_{0}^{+}} \bigl\vert v_{0}^{-} \bigr\vert ^{2}\,dx = \int _{\varOmega }f\bigl(x,v_{0}^{-} \bigr)v_{0}^{-}\,dx. \end{aligned}$$

(3.18)

Then, by (3.15)–(3.18), it is easy to see that $(s_{0}, t_{0}) = (1, 1)$.

Now, we prove that $u_{0}$ is a least-energy sign-changing solution of (1.10) in H which changes sign only once. According to Lemma 2.1, we derive that

$$\begin{aligned} I_{0}(v_{0})&\leq I_{0}(u_{0})=\lim _{n\rightarrow \infty }I _{b_{n}}(u_{b_{n}})= \lim _{n\rightarrow \infty }I_{b_{n}}\bigl(u_{b _{n}}^{+}+u_{b_{n}}^{-} \bigr) \\ &\leq \lim _{n\rightarrow \infty }I_{b_{n}}\bigl(s_{b_{n}}v_{0} ^{+}+t_{b_{n}}v_{0}^{-} \bigr)=I_{0}\bigl(v_{0}^{+}+v_{0}^{-} \bigr)=I_{0}(v_{0}). \end{aligned}$$

(3.19)

The proof is thus complete. □