As mentioned in the introduction, in order to overcome the lack of compactness, let us consider \(H = H^{2}_{\mathrm{rad}}(\mathbb{R}^{N})\) which is a Hilbert space when endowed with the following inner product:
$$\langle u,v \rangle= \int \bigl(\Delta u\Delta v + V(x)uv \bigr), $$
which gives rise to the following norm:
$$\u\ = \biggl(\int \bigl(\Delta u^{2} + V(x)u^{2} \bigr) \biggr)^{\frac{1}{2}}. $$
By (V_{1}), it follows easily that \(\\cdot\\) is equivalent to the usual norm in \(H^{2}(\mathbb{R}^{N})\).
Before introducing the energy functional associated to (1.2), let us remember some results proved by Ebihara and Schonbek in [16] that will be used along this text.
Lemma 2.1
(Corollary 2 in [16])
The following embeddings are compact:
$$\begin{aligned}& H^{2}_{\mathrm{rad}}\bigl(\mathbb{R}^{N}\bigr) \hookrightarrow L^{q}\bigl(\mathbb{R}^{N}\bigr), \quad \textit{for all }2< q<2_{*}\textit{ if }N \geq5, \\& H^{2}_{\mathrm{rad}}\bigl(\mathbb{R}^{N}\bigr) \hookrightarrow L^{q}\bigl(\mathbb{R}^{N}\bigr),\quad \textit{for all }2<q\textit{ if }N = 1,2,3,4. \end{aligned}$$
Lemma 2.2
(Theorem 2.1 in [16])
\(H^{2}_{\mathrm{rad}}(\mathbb{R}^{N})\)
is not compactly embedded into
\(L^{2}(\mathbb{R}^{N})\).
Let us consider the restriction to H of the energy functional whose EulerLagrange equation is (1.2), \(I:H \to\mathbb{R}\), given by
$$I(u) = \frac{1}{2} \int \bigl(\Delta u^{2} + V(x)u^{2} \bigr)  \int F(x,u) = \frac{1}{2}\u\^{2}  \int F(x,u). $$
Note that by (f_{3}) and Sobolev embeddings, I is well defined.
Since for each \(u \in H\), \(v \mapsto\int f(x,u)v\) is a continuous linear functional in H, it is well defined \(A(u) \in H\) such that
$$\bigl\langle A(u),v \bigr\rangle = \int f(x,u)v, \quad \mbox{for all }v\in H. $$
The following result states some interesting properties of the operator \(A: H \to H\) defined above.
Lemma 2.3
A
is a compact, locally Lipschitz operator, such that
\(A(u) = \nabla \Psi\)
where
\(\Psi:H \to\mathbb{R}\)
is given by
\(\Psi(u) = \int F(x,u)\). Moreover, for each
\(\epsilon> 0\), there exists
\(A(\epsilon) > 0\)
such that
$$\bigl\vert \bigl\langle A(u),v \bigr\rangle \bigr\vert \leq \bigl(\epsilon\u \ + A(\epsilon)\u\ ^{p+1} \bigr)\v\, \quad \textit{for all }u,v \in H, $$
where
p
is given in (f_{3}).
Proof
Let us prove the estimate. Note that from (f_{2}) and (f_{3}) it follows that for each \(\epsilon< 0\), there exists \(A(\epsilon) > 0\) such that
$$\bigl\vert f(x,s)\bigr\vert \leq\epsilons + A(\epsilon)s^{p+1}, \quad \mbox{for a.e. }x \in\mathbb{R}^{N}\mbox{ and for all }s \in \mathbb{R}. $$
Using Hölder with the conjugated exponents \(\frac{2_{*}}{p+1}\) and \(\frac{2_{*}}{2_{*} (p+1)}\), we have
$$\begin{aligned} \bigl\vert \bigl\langle A(u),v \bigr\rangle \bigr\vert \leq& \int \bigl( \epsilonu + A(\epsilon)u^{p+1} \bigr)v \\ \leq& \epsilon\u\\v\ + A(\epsilon)\bigl\Vert u^{p+1}\bigr\ \v \Vert . \end{aligned}$$
Just by definition and Lebesgue dominated convergence theorem, it follows that \(A = \nabla\Psi\), where \(\Psi(u) = \int F(u)\). On the other hand, (f_{2}) implies that A is a locally Lipschitz operator.
What is left to show is that A is a compact operator. Although this follows by straightforward calculations, we describe all the details, since this was the reason why we had to consider the space \(H^{2}_{\mathrm{rad}}(\mathbb{R}^{N})\) rather than \(H^{2}(\mathbb{R}^{N})\).
Let \((u_{n}) \subset H^{2}_{\mathrm{rad}}(\mathbb{R}^{N})\) be a bounded sequence. Along a subsequence, we have
$$ \begin{aligned} &u_{n} \rightharpoonup u \quad \mbox{in }H^{2}_{\mathrm{rad}}\bigl(\mathbb{R}^{N}\bigr), \\ &u_{n} \to u \quad \mbox{in }L^{q}\bigl( \mathbb{R}^{N}\bigr), \mbox{for }2 < q < 2N/(N4). \end{aligned} $$
(2.1)
For each \(v \in C^{\infty}_{0}(\mathbb{R}^{N})\), it follows by (f_{3}) and the Hölder inequality with \(2+\epsilon\) and \((2+\epsilon)'\), for small enough \(\epsilon> 0\), that
$$\begin{aligned}& \bigl\vert \bigl\langle A(u_{n})  A(u), v\bigr\rangle \bigr\vert \\& \quad \leq \int\bigl\vert f(x,u_{n})  f(x,u)\bigr\vert v \\& \quad \leq \int_{\operatorname{supp} (v)} c_{1}u_{n}  uv + \int_{\operatorname{supp} (v)} c_{2}\bigl(u_{n}^{p} + u^{p}\bigr)u_{n}  uv \\& \quad \leq C(v)\u_{n}  u\_{2+\epsilon} + C(v) \biggl(\int _{\operatorname{supp} (v)}u_{n}^{p}u_{n}  u + \int_{\operatorname{supp} (v)} u^{p}u_{n}  u \biggr). \end{aligned}$$
By taking \(2 < r < \min\{2,2_{*}/p\}\), using the Hölder inequality with r and \(r'\) and by (2.1), we obtain
$$\begin{aligned} \bigl\vert \bigl\langle A(u_{n})  A(u), v\bigr\rangle \bigr\vert \leq& C(v)o_{n}(1) + C(v)\u_{n}\ _{p,r}^{p} \u_{n}  u\_{r'} + C(v)\u\_{p,r}^{p} \u_{n}  u\_{r'} \\ = & o_{n}(1). \end{aligned}$$
Therefore, for all \(v \in C^{\infty}_{0}(\mathbb{R}^{N})\) it follows that
$$ \lim_{n \to\infty} \bigl\langle A(u_{n})  A(u), v \bigr\rangle = 0. $$
(2.2)
Since \(C^{\infty}_{0}(\mathbb{R}^{N})\) is dense in \(H^{2}_{\mathrm{rad}}(\mathbb {R}^{N})\), it follows that (2.2) holds for all \(v \in H^{2}_{\mathrm{rad}}(\mathbb{R}^{N})\), and this implies that
$$\lim_{n \to\infty} A(u_{n}) = A(u). $$
□
It is straightforward to prove that critical points of I correspond to fixed points of A.
Let us consider the following Cauchy problem in the Hilbert space H:
$$ \left \{ \begin{array}{l} \frac{\partial}{\partial t}\varphi(t,u) = \nabla I(\varphi (t,u)) = A(\varphi(t,u))  \varphi(t,u), \\ \varphi(0,u) = u, \end{array} \right . $$
(2.3)
where \(\varphi:\mathcal{G} \to H\) and \(\mathcal{G} = \{(t,u) \in \mathbb{R} \times H; u \in H\mbox{ and }t\in[0,T(u))\}\) and \([0,T(u))\) is the maximal interval of the existence of the trajectory \(t \mapsto\varphi(t,u)\). Note that since A is a Lipschitz continuous operator, the flow φ is well defined.
The following is a key point in our approach.
Proposition 2.4
If for some
\(u\in H\), \(\{I(\varphi(t,u)); 0 \leq t < T(u)\}\)
is bounded from below, then:

(i)
\(T(u)=\infty\),

(ii)
there exists
\(t_{n}\rightarrow\infty\)
such that
\(\{\varphi(t_{n},u), n\in\mathbb{N} \}\)
is bounded in
H
and the
ωlimit set of u,
$$\omega(u) = \bigcap_{0\leq t<\infty} \overline{\bigcup _{t \leq s<\infty}\varphi(s,u)} $$
is a nonempty set formed by critical points of
I.
Proof
(i) Note that
$$\begin{aligned} \bigl\Vert \varphi(t,u)  \varphi(s,u)\bigr\Vert \leq& \int _{s}^{t}\bigl\Vert \nabla I\bigl(\varphi (\tau,u) \bigr)\bigr\Vert \, d \tau \\ \leq& \sqrt{ts} \biggl(\int_{s}^{t} \bigl\Vert \nabla I\bigl(\varphi(\tau,u)\bigr)\bigr\Vert ^{2}\, d \tau \biggr)^{\frac{1}{2}} \\ = & \sqrt{ts} \bigl(I\bigl(\varphi(s,u)\bigr)  I\bigl(\varphi(t,u)\bigr) \bigr)^{\frac{1}{2}}, \end{aligned}$$
where we have used Hölder and (2.3). Suppose the assertion of the item is false. Then by the last estimate, the trajectory \(\{\varphi (t,u); t \in[0,T(u))\}\) would be bounded, which implies that \(T(u)=\infty\), which gives rise to a contradiction.
(ii) First of all, let us note that there exists a sequence \(t_{n} \rightarrow\infty\) such that \(\\nabla I(\varphi (t_{n},u))\ \rightarrow0\), as \(n\rightarrow\infty\). This follows just by noting that
$$\begin{aligned} \begin{aligned} \int_{0}^{\infty}\bigl\Vert \nabla I\bigl(\varphi( \tau,u)\bigr)\bigr\Vert ^{2}\, d\tau& = \lim_{t\to\infty} \int_{0}^{t} \bigl\Vert \nabla I\bigl(\varphi( \tau,u)\bigr)\bigr\Vert ^{2}\, d\tau \\ & = \lim_{t\to\infty}\bigl\vert I\bigl(\varphi(t,u)\bigr)  I(u) \bigr\vert < \infty. \end{aligned} \end{aligned}$$
We claim that \(\{\varphi(t_{n},u)\}_{n\in\mathbb{N}}\) is uniformly bounded in H with respect to \(n\in\mathbb{N}\).
Suppose, contrary to our claim, that \(\\varphi(t_{n},u)\ \rightarrow \infty\) as \(n \rightarrow\infty\). Let us define
$$w_{n} = \frac{\varphi(t_{n},u)}{\\varphi(t_{n},u)\}. $$
Since \((w_{n})\) is a bounded sequence in H, it follows that there exists \(w\in H\) such that \(w_{n} \rightharpoonup w\) in H, up to a subsequence. Then Lemma 2.1 implies that \(w_{n}\rightarrow w\) in \(L^{q}(\mathbb{R}^{N})\), \(2 < q < 2_{*}\), and also \(w_{n} \to w\) a.e. in \(\mathbb{R}^{N}\).
In order to prove that \(w = 0\), let us consider \(\Gamma= \{x\in \mathbb{R}^{N} ; w(x)\neq0\}\), and prove that Γ has zero Lebesgue measure. For all \(x \in\Gamma\), we have \(\lim_{n\to\infty } \varphi(t_{n},u)(x) = \infty\). By (f_{4}), for each \(M>0\), there exists \(r > 0\) such that
$$F(x,s) \geq Ms^{2}, \quad \mbox{for all }s \geq r\mbox{ and for a.e. }x\in \mathbb{R}^{N}. $$
Since \(\{I(\varphi(t_{n},u))\}_{n\to\infty}\) is bounded from below,
$$\begin{aligned} \frac{1}{2} + o_{n}(1) \geq& \int\frac{F(x,\varphi(t_{n},u))}{\ \varphi(t_{n},u)\^{2}} \\ \geq& \int_{\{\varphi(t_{n},u)>r\}\cap\Gamma} \frac{F(x,\varphi (t_{n},u))}{\varphi(t_{n},u)^{2}}w_{n}^{2} \\ \geq& M\int_{\{\varphi(t_{n},u)>r\}\cap\Gamma} w_{n}^{2}. \end{aligned}$$
Observing that \(\varphi(t_{n},u)(x) \to+\infty\) as \(n \to\infty\) for all \(x \in\Gamma\) and by Fatou’s lemma, it follows that
$$\frac{1}{2} \geq M \int_{\Gamma}w^{2}, $$
which is a contradiction since \(\int_{\Gamma}w^{2} > 0\) and M is arbitrary. Hence, \(\Gamma = 0\).
Note that the function \(t\mapsto I(t\varphi(t_{n},u))\) is smooth in \((0,1)\). Let \(s_{n}\in[0,1]\) such that
$$I\bigl(s_{n}\varphi(t_{n},u)\bigr) = \max _{t\in[0,1]}I\bigl(t\varphi(t_{n},u)\bigr). $$
For each \(R > 0\), and for all n large enough,
$$\begin{aligned} I\bigl(s_{n} \varphi(t_{n},u)\bigr) \geq& I \biggl( \frac{R}{\\varphi(t_{n},u)\} \varphi(t_{n},u) \biggr) \\ = & \frac{R^{2}}{2} \int F(x,R w_{n}) = \frac{R^{2}}{2} + o_{n}(1). \end{aligned}$$
Hence \(\lim_{n\to\infty}I(s_{n} \varphi(t_{n},u)) = +\infty\), which implies that \(s_{n} \in(0,1)\). Then, for n large enough \(I'(s_{n}\varphi (t_{n},u))s_{n}\varphi(t_{n},u) = 0\). By (f_{4}), for all \(t\in[0,1]\) we have
$$\begin{aligned} 2I\bigl(t\varphi(t_{n},u)\bigr) \leq& 2I\bigl(s_{n} \varphi(t_{n},u)\bigr)  I'\bigl(s_{n}\varphi (t_{n},u)\bigr)s_{n}\varphi(t_{n},u) \\ = & \int \bigl(f\bigl(x,s_{n}\varphi(t_{n},u) \bigr)s_{n}\varphi(t_{n},u)  2F\bigl(x,s_{n} \varphi(t_{n},u)\bigr) \bigr) \\ \leq& \int \bigl(f\bigl(x,\varphi(t_{n},u)\bigr) \varphi(t_{n},u)  2F\bigl(x,\varphi (t_{n},u)\bigr) \bigr) \\ = & 2I\bigl(\varphi(t_{n},u)\bigr) + o_{n}(1) \leq C_{1}. \end{aligned}$$
For a given \(R_{0} > 0\), for n large enough, it follows that \(\frac {R_{0}}{\\varphi(t_{n},u)\} < 1\). Then
$$ 2I(R_{0}w_{n}) = 2I \biggl(\frac{R_{0}}{\\varphi(t_{n},u)\}\varphi (t_{n},u) \biggr) \leq C_{1}. $$
(2.4)
On the other hand, for all \(R_{0}>0\),
$$ 2I(R_{0}w_{n}) = R_{0}^{2}  2\int F(x,R_{0}w_{n}) = R_{0}^{2} + o_{n}(1). $$
(2.5)
Since (2.4) contradicts (2.5), we find that \(\{\varphi (t_{n},u)\}_{n\in\mathbb{N}}\) is uniformly bounded in H with respect to \(n\in\mathbb{N}\) and the claim is proved.
In order to show that \(\omega(u) \neq\emptyset\), let us consider the bounded sequence \(\{\varphi(t_{n},u)\}\). Since A is compact, there exists \(u_{0} \in H\) such that \(A(\varphi(t_{n},u)) \rightarrow u_{0}\) along a subsequence. Hence
$$\begin{aligned} 0 = & \lim_{n\to\infty}\bigl\Vert \nabla I\bigl( \varphi(t_{n},u)\bigr)\bigr\Vert \\ = & \lim_{n\to\infty} \bigl( \varphi(t_{n},u)  A\bigl( \varphi (t_{n},u)\bigr) \bigr) \\ = & \lim_{n\to\infty} \bigl(\varphi(t_{n},u)  u_{0} \bigr). \end{aligned}$$
Then \(\lim_{n\to\infty}\varphi(t_{n},u) = u_{0}\) and \(u_{0} \in\omega(u)\).
Concerning the proof that every point in \(\omega(u)\) is a critical point of I, note that \(I(\varphi(t,u))\rightarrow d\) as \(t\rightarrow+\infty\). Then, if \(v\in\omega(u)\), there exists \(t_{n}\rightarrow\infty\) such that \(\varphi(t_{n},u) \rightarrow v\) in H as \(n\rightarrow\infty\). Hence
$$I\bigl(\varphi(t,v)\bigr) = \lim_{n\to\infty}I\bigl(\varphi\bigl(t, \varphi(t_{n},u)\bigr)\bigr) = \lim_{n\to\infty}I\bigl( \varphi(t+t_{n},u)\bigr) = d, $$
for all \(t \geq0\). Then, for all \(t \geq0\),
$$0 = \frac{\partial}{\partial t}I\bigl(\varphi(t,v)\bigr) = \bigl\Vert \nabla I\bigl( \varphi (t,v)\bigr)\bigr\Vert ^{2}, $$
which implies that \(\nabla I(v) = 0\) and v is a critical point of I. □
Definition 2.5
We call \(D \subset H\) a positive invariant set if \(\varphi(t,u) \in D\) for all \(t \in[0,T(u))\) and \(u \in D\). If D is a positive invariant set we define its absorption domain by
$$\mathcal{A}(D)= \bigl\{ u\in H; \exists t_{0}\in\bigl[0,T(u)\bigr) \mbox{ such that }\varphi(t_{0},u)\in D\bigr\} . $$
Let us define the following set:
$$\mathcal{A}_{0} = \bigl\{ u\in H; T(u)=\infty \mbox{ and } \varphi (t,u)\rightarrow0 \mbox{ as } t\rightarrow\infty\bigr\} . $$
Lemma 2.6
\(\mathcal{A}_{0}\)
is an open subset of
H
and there exists
\(r > 0\)
such that
\(B_{r}(0) \subset\mathcal{A}_{0}\).
Proof
Let \(0 < \epsilon< 1\) and \(A(\epsilon)\) given in Lemma 2.3. Let us consider \(\alpha_{0} = (\frac{1 \epsilon }{2A(\epsilon)} )^{\frac{1}{p}}\). For each \(u\in\overline {B_{\alpha_{0}}(0)}\), by Lemma 2.3 we have
$$\begin{aligned} \Psi(u) = & \int_{0}^{1} \frac{\partial}{\partial t} \Psi(tu)\, dt \\ = & \int_{0}^{1}\bigl\langle A(tu) , u \bigr\rangle \, dt \\ \leq& \int_{0}^{1} \bigl( \epsilon\tu\ + A( \epsilon)\tu\ ^{p+1} \bigr)\u\\, dt \\ \leq& \int_{0}^{1} t\u\^{2} \bigl( \epsilon+ A(\epsilon)\u\ ^{p} \bigr) \, dt \\ = & \frac{\u\^{2}}{2} \bigl( \epsilon+ A(\epsilon)\u\^{p} \bigr) \\ \leq& \frac{\u\^{2}}{2} \bigl( \epsilon+ A(\epsilon)\alpha _{0}^{p} \bigr) \\ = & \frac{\epsilon+ 1}{4}\u\^{2}. \end{aligned}$$
Then
$$I(u) = \frac{1}{2}\u\^{2}  \Psi(u) \geq \biggl( \frac{1}{2}  \frac {\epsilon+ 1}{4} \biggr)\u\^{2} \geq0, $$
since \(\epsilon\in(0,1)\).
Moreover, for all \(u \in\partial B_{\alpha_{0}}(0)\),
$$ I(u) \geq \biggl(\frac{1}{2}  \frac{\epsilon+ 1}{4} \biggr)\alpha _{0}^{2}=: \beta_{0} > 0. $$
(2.6)
By continuity, there exists \(r\in(0,\alpha_{0})\) such that
$$I(u) < \beta_{0},\quad \mbox{for all }u\in B_{r}(0). $$
Since the energy does not grow along any trajectory, if \(u\in B_{r}(0)\), then \(I(\varphi(t,u)) < \beta_{0}\), for all \(t\in[0,T(u))\). Then, by (2.6), \(\varphi(t,u) \in B_{\alpha_{0}}(0)\) for all \(t\in [0,T(u))\). Hence \(I(\varphi(t,u)) \geq0\) for all \(t\in[0,T(u))\), which implies that \(T(u) = \infty\). Moreover, \(\omega(u) \subset B_{\alpha_{0}}(0)\) is a compact nonempty set formed of critical points of I. On the other hand, if \(v\in B_{\alpha_{0}}(0)\) is a critical point of I, then
$$\v\^{2} = \bigl\langle A(v),v\bigr\rangle \leq\v\^{2} \bigl(\epsilon+ A(\epsilon)\ v\^{p}\bigr) \leq\v\^{2} \bigl(\epsilon+ A(\epsilon)\alpha_{0}^{p}\bigr) = \frac {\epsilon+ 1}{2}\v\^{2}, $$
which only is possible in the case where \(v=0\). Therefore, \(\omega(u) = \{0\}\) for all \(u\in B_{r}(0)\) and then \(B_{r}(0) \subset\mathcal {A}_{0}\), which implies that \(\mathcal{A}_{0} = \mathcal{A}(B_{r}(0))\). Since \(B_{r}(0)\) is an open set, then so is \(\mathcal{A}(B_{r}(0))\). □
The following is a key result in our argument and in particular implies that \(\partial\mathcal{A}_{0}\) is a great place to look for nontrivial critical points of I.
Proposition 2.7
\(\partial\mathcal{A}_{0}\)
is a closed positively invariant set of
H
and
\(\inf_{u \in\partial\mathcal{A}_{0}}I(u) \geq0\). In particular, for all
\(u \in\partial\mathcal{A}_{0}\), \(\omega(u)\)
is a nonempty set consisting in nontrivial critical points of
I.
The proof of the positively invariance can be found in [19] while the other results are straightforward to see.
Although \(\partial\mathcal{A}_{0}\) is a great set to look for nontrivial critical points of I, once found, nothing can be said about its signal. Let us introduce the concept of dual cone and state the dual cone decomposition theorem, which is given by Moreau in [13].
Definition 2.8
Given a cone \(\mathcal{K}\) in a Hilbert space H, its dual cone is defined by
$$\mathcal{K}^{*} = \bigl\{ v \in H; \langle u,v \rangle\leq0, \forall u \in \mathcal{K}\bigr\} . $$
Theorem 2.9
Let
\(\mathcal{K} \subset H\)
a closed convex cone. Then for all
\(x\in H\), there exist
\(y\in\mathcal{K}\)
and
\(z\in\mathcal{K}^{*}\)
such that
$$x = y+z \quad \textit{and}\quad \langle y,z\rangle= 0. $$
Let us define the following cones and afterwards prove some invariance properties of them.
Let
$$ \begin{aligned} &\mathcal{K} = \bigl\{ u\in H; u \geq0 \mbox{ a.e. in } \mathbb {R}^{N}\bigr\} , \\ &\mathcal{K} = \bigl\{ u\in H; u \leq0 \mbox{ a.e. in }\mathbb {R}^{N}\bigr\} , \end{aligned} $$
(2.7)
which are closed convex cones.
Let us denote by P and Q the orthogonal projections of H in \(\mathcal{K}\) and \(\mathcal{K}\), respectively. Denoting by \(P^{*}= \mathrm{Id}  P\) and \(Q^{*} = \mathrm{Id}  Q\), note that
$$\bigl\langle Pu, P^{*}u \bigr\rangle = 0, \quad \mbox{for all }u \in H $$
and
$$P^{*}u \in\mathcal{K}^{*}, $$
where \(K^{*}\) is the dual cone associated to \(\mathcal{K}\). We have analogous results involving Q, \(\mathcal{K}\), and \((\mathcal{K})^{*}\).
In order to prove the invariance of \(\mathcal{K}\) and \(\mathcal{K}\), as we will see, it will be necessary to prove that \(\mathcal{K}^{*} \subset\mathcal{K}\) and \((\mathcal{K})^{*} \subset\mathcal{K}\). In the classical argument developed by Weth in [14], the maximum principle to the operator \(\Delta^{2}\) under certain boundary conditions and in certain domains is absolutely useful. Since this is not an option for us, let us prove some result that in some sense will substitute the lack of this result.
Lemma 2.10
For each radially symmetric
\(h \in C^{\infty}_{0}(\mathbb{R}^{N})\), \(h \geq 0\), and
\(h \not\equiv0\), there exists a positive continuous radial solution
\(v \in H^{2}_{\mathrm{rad}}(\mathbb{R}^{N})\)
of the linear problem
$$ \Delta^{2} v + V(x)v = h(x)\quad \textit{in }\mathbb{R}^{N}. $$
(2.8)
Proof
The existence of a solution \(v \in H^{2}_{\mathrm{rad}}(\mathbb{R}^{N})\) follows straightforwardly just by applying the Riesz theorem. Regularity is a simple matter just by calling Proposition 2.5 in [21].
To the positiveness we apply some arguments of Chabrowski and Yang in [15] which we describe below.
Since v is smooth, \(\Omega= \{x \in\mathbb{R}^{N}; v(x) < 0\}\) is an open subset of \(\mathbb{R}^{N}\). Supposing by contradiction that there exists \(x_{0} \in\Omega\), let \(R > 0\) such that
$$B_{2R}(x_{0}) \subset\Omega. $$
Assuming for simplicity that \(R = 1\), let us denote
$$\bar{v}(x) = \left \{ \begin{array}{l@{\quad}l} v(x), &\mbox{if }xx_{0} \leq1, \\ 0, &\mbox{if }xx_{0} > 1, \end{array} \right . $$
and \(\rho\in C^{\infty}_{0}(\mathbb{R}^{N})\) with \(\operatorname{supp} (\rho) \subset B_{1}(0)\) and \(\int\rho= 1\). Let us define
$$\tilde{v}(x) = \int r^{N}\rho \biggl(\frac{xy}{r} \biggr)\bar {v}(y)\, dy = \int_{B_{1}(x_{0})} r^{N}\rho \biggl( \frac{xy}{r} \biggr)v(y)\, dy, $$
where \(0 < r \leq1\).
Note that \(\operatorname{supp}(\tilde{v}) \subset B_{r+1}(x_{0})\). Then, for \(x \in B_{r+1}(x_{0})\), \(\tilde{v}\) satisfies
$$\begin{aligned} \Delta^{2} \tilde{v}(x) = & \int_{B_{1}(x_{0})} r^{N} \Delta^{2}_{x} \rho \biggl( \frac{xy}{r} \biggr)v(y)\, dy \\ = & \int_{B_{1}(x_{0})} r^{N4}\rho \biggl( \frac{xy}{r} \biggr) \bigl(h(y)  V(y)v(y) \bigr)\, dy =: \bar{h}(x), \end{aligned}$$
where \(\bar{h} \geq0\) as for each \(y\in B_{r+1}(0)\), \(h(y)  V(y)v(y) \geq0\). Then \(\tilde{v}\) satisfies the following problem:
$$ \left \{ \begin{array}{l@{\quad}l} \Delta^{2} \tilde{v} = \bar{h} & \mbox{in }B_{r+1}(0), \\ \tilde{v} = \Delta\tilde{v} = 0 & \mbox{on }\partial B_{r+1}(0). \end{array} \right . $$
(2.9)
Let us recall that from standard minimization arguments and elliptic regularity theory, as proved in [15][Lemma 3], for all bounded smooth domain \(\Omega\subset\mathbb{R}^{N}\) and for all \(g \in L^{2N/(N4)}(\Omega) \cap C^{0,\alpha}(\Omega)\), \(0 < \alpha< 1\), there exists a classical solution \(w > 0\) of
$$ \left \{ \begin{array}{l@{\quad}l} \Delta^{2} w = g & \mbox{in }\Omega, \\ w = \Delta w = 0 & \mbox{on }\partial\Omega. \end{array} \right . $$
(2.10)
For any \(L^{2N/(N4)}(B_{r+1}(x_{0})) \cap C^{0,\alpha}(B_{r+1}(x_{0}))\) positive function g, let w be the positive smooth solution of (2.10) with \(\Omega= B_{r+1}(x_{0})\). Using w as a test function in (2.9) we obtain
$$\begin{aligned} \begin{aligned} \int_{B_{r+1}(x_{0})}\bar{h}w\, dx & = \int_{B_{r+1}(x_{0})} \Delta^{2} \tilde{v}w \, dx \\ & = \int_{B_{r+1}(x_{0})}\tilde{v}\Delta^{2}w \, dx \\ & = \int_{B_{r+1}(x_{0})}\tilde{v} g\, dx. \end{aligned} \end{aligned}$$
However, while the last integral is negative, the first one is positive, which gives us a contradiction. □
Using the last lemma it is possible to prove the following claim.
Claim
\(\mathcal{K}^{*} \subset\mathcal{K}\).
Proof
Let \(u \in\mathcal{K}^{*}\). For each \(h \in L^{2}(\mathbb{R}^{N})\), \(h \geq0\) a.e. in \(\mathbb{R}^{N}\) let \((h_{n})\) in \(C^{\infty}_{0}(\mathbb {R}^{N})\), \(h_{n} \geq0\), such that \(h_{n} \to h\) in \(L^{2}(\mathbb{R}^{N})\). For each \(n \in\mathbb{N}\), let \(v_{n}\) be the positive solution of the linear problem (2.8) with \(h = h_{n}\), given by Lemma 2.10. Then \(v_{n} \in\mathcal{K}\) and
$$ 0 \geq\langle u,v_{n} \rangle= \int \bigl(\Delta u \Delta v_{n} + V(x)u v_{n} \bigr) = \int u h_{n}. $$
(2.11)
Hence, by (2.11) and Fatou’s lemma,
$$\int uh \leq\liminf_{n\to\infty} \int u(x)h_{n}(x) \leq0. $$
Then it follows that \(u \leq0\) a.e. in \(\mathbb{R}^{N}\) and therefore \(u \in\mathcal{K}\). □
Remark 2.11
It is worth pointing out that if \(u \in H\), then \(u = Pu + P^{*}u\) where \(Pu \geq0\) and \(P^{*}u \leq0\) a.e. in \(\mathbb{R}^{N}\). Then \(u \leq Pu\), and consequently, \(u^{+} \leq Pu\) a.e. in \(\mathbb{R}^{N}\). In the same way one can prove that \(P^{*} u \leq u^{}\), \(Qu \leq u^{}\), and \(u^{+} \leq Q^{*} u\) a.e. in \(\mathbb{R}^{N}\).
The following is a very important result to prove the invariance of \(\mathcal{K}\) and \(\mathcal{K}\) under the flow φ.
Lemma 2.12
The operator
A
satisfies the following conditions:

(i)
\(\langle A(u),v \rangle\leq\langle A(P^{*}u),v \rangle \), for all
\(u \in H\)
and
\(v\in\mathcal{K}^{*}\);

(ii)
\(\langle A(u),v \rangle\leq\langle A(Q^{*}u),v \rangle\), for all
\(u \in H\)
and
\(v\in(\mathcal{K})^{*}\).
Proof
Since (ii) can be proved in the same way, we just prove (i). Let \(u \in H\) and \(v \in\mathcal{K}^{*}\). By Remark 2.11, \(P^{*}u \leq u^{}\), and by (f_{5})
$$f\bigl(x,P^{*}u(x)\bigr) \leq f\bigl(x,u^{}(x)\bigr). $$
Since \(v \leq0\), once more by (f_{5}) it follows that
$$\begin{aligned} \bigl\langle A(u) , v \bigr\rangle = & \int f(x,u)v \\ \leq& \int f\bigl(x,u^{}\bigr)v \\ \leq& \int f\bigl(x,P^{*}u\bigr)v = \bigl\langle A\bigl(P^{*}u\bigr) , v \bigr\rangle . \end{aligned}$$
□
Lemma 2.13

(i)
\(A(\mathcal{K}) \subset\mathcal{K}\)
and
\(A(\mathcal{K}) \subset\mathcal{K}\).

(ii)
For sufficiently small
\(\alpha> 0\), the
αneighborhood of
\(\mathcal{K}\), \(B_{\alpha}(\mathcal{K})\)
is positively invariant under
φ. Moreover, all critical points of
I
in
\(\overline{B_{\alpha}(\mathcal{K})}\)
belong to
\(\mathcal{K}\). The same holds for the cone
\(\mathcal{K}\).

(iii)
\(\mathcal{K}\)
and
\(\mathcal{K}\)
are positively invariant under
φ.
Proof
Let us prove the results just for \(\mathcal{K}\), since for \(\mathcal {K}\) the arguments are the same.
(i) By Lemma 2.12(i), for \(u\in\mathcal{K}\),
$$\begin{aligned} \bigl\Vert P^{*}\bigl(A(u)\bigr)\bigr\Vert ^{2} = & \bigl\langle A(u)  P\bigl(A(u)\bigr) , A(u)  P\bigl(A(u)\bigr) \bigr\rangle \\ = & \bigl\langle A(u), P^{*}\bigl(A(u)\bigr) \bigr\rangle \\ \leq& \bigl\langle A\bigl(P^{*}u\bigr), P^{*}\bigl(A(u)\bigr) \bigr\rangle = 0, \end{aligned}$$
which implies that \(A(u) \in\mathcal{K}\).
(ii) Let \(u\in H\), by Lemma 2.3 and Lemma 2.12(i), for \(0 < \epsilon< 1\), we have
$$\begin{aligned} \bigl\Vert P^{*}\bigl(A(u)\bigr)\bigr\Vert ^{2} = & \bigl\langle A(u), P^{*}\bigl(A(u)\bigr) \bigr\rangle \\ \leq& \bigl\langle A\bigl(P^{*}u\bigr), P^{*}\bigl(A(u)\bigr) \bigr\rangle \\ \leq& \bigl\Vert P^{*}\bigl(A(u)\bigr)\bigr\Vert \bigl(\epsilon\bigl\Vert P^{*}u\bigr\Vert + A(\epsilon)\bigl\Vert P^{*}u\bigr\Vert ^{p+1} \bigr), \end{aligned}$$
which implies that
$$ \bigl\Vert P^{*}\bigl(A(u)\bigr)\bigr\Vert \leq\epsilon\bigl\Vert P^{*}u\bigr\Vert + A(\epsilon)\bigl\Vert P^{*}u\bigr\Vert ^{p+1} = \bigl\Vert P^{*}u\bigr\Vert \bigl(\epsilon+ A(\epsilon)\bigl\Vert P^{*}u\bigr\Vert ^{p} \bigr), $$
(2.12)
since \(\P^{*}(A(u))\\neq0\). Then, if \(0 < \P^{*}u\ < (\frac{1\epsilon}{2A(\epsilon )} )^{\frac{1}{p}} =: \alpha_{0}\),
$$ \bigl\Vert P^{*}\bigl(A(u)\bigr)\bigr\Vert < \bigl\Vert P^{*}u\bigr\Vert . $$
(2.13)
Hence, for all \(\alpha< \alpha_{0}\), every fixed point of A in \(\overline{B_{\alpha}(\mathcal{K})}\) belongs to \(\mathcal{K}\). In fact, if \(u\in\overline{B_{\alpha}(\mathcal{K})}\backslash\mathcal {K}\) and \(A(u) = u\) then \(0 < \P^{*}u\ \leq\alpha< \alpha_{0}\), which implies by (2.13) that
$$\bigl\Vert P^{*}u\bigr\Vert = \bigl\Vert P^{*}\bigl(A(u)\bigr)\bigr\Vert < \bigl\Vert P^{*}u\bigr\Vert , $$
which is a contradiction.
Now let us prove that \(B_{\alpha}(\mathcal{K})\) is positively invariant.
Note that by (2.13)
$$ A\bigl(\partial B_{\alpha}(\mathcal{K})\bigr) \subset \operatorname{int} \bigl(B_{\alpha}(\mathcal {K})\bigr). $$
(2.14)
Suppose, contrary to our claim, that there exists \(u_{0}\in B_{\alpha}(\mathcal{K})\) such that \(\varphi(t_{0},u_{0}) \in\partial B_{\alpha}(\mathcal{K})\) where \(t_{0}\in[0,T(u_{0}))\) is the least positive real with this property. As \(B_{\alpha}(\mathcal{K})\) is a convex open set and \(\{\varphi(t_{0},u_{0})\}\) is compact, by Mazur’s separation theorem, there exist a linear functional \(\rho\in H^{*}\) and a real number β such that \(\rho(\varphi(t_{0},u_{0})) = \beta\) and \(\rho(u) > \beta\) for all \(u \in B_{\alpha}(\mathcal{K})\).
By (2.14) it follows that
$$\begin{aligned} {\biggl.\frac{\partial}{\partial t} \rho\bigl(\varphi(t,u_{0})\bigr) \biggr_{t=t_{0}}} = & \rho\bigl(  \nabla I\bigl(\varphi(t_{0},u_{0})\bigr) \bigr) \\ = & \rho\bigl(A\bigl(\varphi(t_{0},u_{0})\bigr)\bigr)  \beta> 0. \end{aligned}$$
Therefore, there exists \(\eta> 0\) such that \(\rho(\varphi(t,u_{0})) < \beta\) as long as \(t\in(t_{0}\eta,t_{0})\). Then \(\varphi(t,u_{0}) \notin B_{\alpha}(\mathcal{K})\) for all \(t \in(t_{0}  \eta, t_{0})\), which contradicts the minimality of \(t_{0}\).
(iii) This item follows straightforwardly observing that \(\mathcal{K} = \bigcap_{\alpha>0} B_{\alpha}(\mathcal{K})\). □
From now on let us consider \(\alpha> 0\) such that the statement of Lemma 2.13 holds for \(\mathcal{K}\) and \(\mathcal{K}\).
To obtain the signed solutions we will use the following result.
Proposition 2.14
Assume that there exists
\(u_{0}\in\mathcal{K}\)
such that
\(I(u_{0}) < 0\), then there exists a nontrivial critical point of
I
in
\(\mathcal{K}\). The same holds for
\(\mathcal{K}\).
Proof
First note that by definition of \(\mathcal{A}_{0}\), \(I(u) \geq0\) for all \(u\in\mathcal{A}_{0}\). Then by continuity, \(I(u) \geq0\) for all \(u\in\overline{\mathcal{A}}_{0}\).
Since \(I(u_{0}) < 0\) then \(u_{0} \notin\overline{\mathcal{A}}_{0}\). As \(\mathcal{A}_{0}\) is an open neighborhood of the origin, there exists \(s\in(0,1)\) such that \(su_{0} \in\partial\mathcal{A}_{0}\cap\mathcal {K}\). Since \(\partial\mathcal{A}_{0}\cap\mathcal{K}\) is a closed positively invariant set, by Proposition 2.4, \(\omega(su_{0}) \subset\partial\mathcal{A}_{0}\cap\mathcal{K}\) is nonempty and any of its points are critical points of I. □
Let us denote by
$$\mathcal{A}_{+} = \mathcal{A}\bigl(B_{\alpha}(\mathcal{K})\bigr)\cap\partial \mathcal{A}_{0} \quad \mbox{and}\quad \mathcal{A}_{} = \mathcal {A} \bigl(B_{\alpha}(\mathcal{K})\bigr)\cap\partial\mathcal{A}_{0}. $$
Lemma 2.15
\(\mathcal{A}_{+}\)
and
\(\mathcal{A}_{}\)
are disjoint relatively open sets of
\(\partial\mathcal{A}_{0}\).
Proof
Since \(B_{\alpha}(\mathcal{K})\) and \(B_{\alpha}(\mathcal{K})\) are open sets, then so are \(\mathcal{A}(B_{\alpha}(\mathcal{K}))\) and \(\mathcal {A}(B_{\alpha}(\mathcal{K}))\).
Suppose, contrary to our claim, that there exists \(u\in\mathcal {A}_{+}\cap\mathcal{A}_{}\). Since \(u\in\partial\mathcal{A}_{0}\), then \(T(u)=\infty\) and \(\omega(u)\neq\emptyset\). Further, since \(u\in \mathcal{A}(B_{\alpha}(\mathcal{K})) \cap\mathcal{A}(B_{\alpha}(\mathcal{K}))\), then \(\omega(u)\subset\overline{B_{\alpha}(\mathcal{K})} \cap\overline{B_{\alpha}(\mathcal{K})}\). But since \(\omega(u)\) consists of critical points of I, by Lemma 2.13, \(\omega(u) \in\mathcal{K} \cap\mathcal{K} = \{0\}\), which contradicts the fact that \(u\in\partial\mathcal{A}_{0}\). □
Proposition 2.16
Suppose that there exists a continuous path
\(h:[0,1] \rightarrow H\), such that
\(h(0)\in\mathcal{K}\), \(h(1)\in \mathcal{K}\), and
\(I(h(t)) < 0\)
for all
\(t\in[0,1]\). Then
I
has at least three nontrivial critical points, these being
\(u_{1}\in\mathcal{K}\), \(u_{2}\in  \mathcal{K}\), and
\(u_{3}\in H\backslash(\mathcal{K}\cup\mathcal{K})\).
Proof
Proposition 2.14 gives the existence of the signed critical points \(u_{1} \in\mathcal{K}\) and \(u_{2} \in\mathcal{K}\).
To get the nodal one, let us first highlight that \(h([0,1]) \cap \overline{\mathcal{A}}_{0} = \emptyset\).
Let \(Q = [0,1]^{2}\) and \(\mathcal{B}\subset Q\) be defined by
$$\mathcal{B} = \bigl\{ (s_{1},s_{2})\in Q; s_{1}h(s_{2}) \in\mathcal{A}_{0}\bigr\} . $$
Note that B is relatively open in Q and the following hold:

\(\{0\} \times[0,1] \subset\mathcal{B}\), since \(0h(s) = 0\in \mathcal{A}_{0}\), \(\forall s\in[0,1]\);

\(\{1\} \times[0,1] \cap\overline{\mathcal{B}} = \emptyset\), since \(I(h(s))<0\), \(\forall s\in[0,1]\).
By recalling the LeraySchauder continuation principle, it follows that there exists a connected component Γ of \(\partial\mathcal {B}\), such that
$$\Gamma\cap\bigl([0,1]\times\{0\}\bigr) \neq\emptyset\quad \mbox{and}\quad \Gamma \cap\bigl([0,1]\times\{1\}\bigr) \neq\emptyset. $$
Now, let us consider Σ the closure of the connected component of \(\Gamma\backslash\partial Q\) that intersects \([0,1]\times\{0\}\) and \([0,1]\times\{1\}\).
Denoting by \(\Gamma_{0} = \{s_{1}h(s_{2}); (s_{1},s_{2})\in\Sigma\}\), we find that \(\Gamma_{0}\) is a connected subset of \(\partial\mathcal {A}_{0}\) such that \(\Gamma_{0} \cap\pm\mathcal{K} \neq\emptyset\). Since by Lemma 2.15, \(\mathcal{A}_{\pm}\) are disjoint open subsets of \(\partial\mathcal{A}_{0}\), \(\Gamma_{0} \cap\mathcal{A}_{\pm}\) are open disjoints subsets of \(\Gamma_{0}\). By connectedness of \(\Gamma_{0}\), there exists \(u\in\Gamma_{0}\backslash(\mathcal{A}_{+} \cup\mathcal{A}_{})\) and once \(\partial\mathcal{A}_{0} \backslash (\mathcal{A}_{+} \cup\mathcal{A}_{})\) is positively invariant, then \(\{ \varphi(t,u); t \geq0\} \subset\partial\mathcal{A}_{0} \backslash (\mathcal{A}_{+} \cup\mathcal{A}_{})\). Using the fact that this is a closed subset in \(\partial\mathcal{A}_{0}\), it follows that \(\omega (u)\subset\partial\mathcal{A}_{0} \backslash(\mathcal{A}_{+} \cup \mathcal{A}_{})\). In particular, \(\omega(u) \cap(\mathcal{K} \cup \mathcal{K}) \neq\emptyset\) and any of his points are nodal critical points of I. □
The next result will be useful to put the energy functional I in the context of the last proposition.
Lemma 2.17
If
\(S\subset H\backslash\{0\}\)
is a compact subset and
\(\tilde{S} = \{ tu; u\in S \textit{ and } t\geq0\}\), then
$$I(u) \rightarrow\infty \quad \textit{as }u\in\tilde{S} \textit{ and }\u\ \rightarrow\infty. $$
Proof
Let \((u_{n})\subset\tilde{S}\) be a sequence such that \(\u_{n}\ \rightarrow\infty\). Then there exists a sequence in \(\mathbb{R}_{+}\)
\((t_{n})\) such that \(u_{n} = t_{n} v_{n}\) and \((v_{n})\subset S\). Since S is compact, we can suppose that along a subsequence \(v_{n} \rightarrow v\), as \(n\rightarrow\infty\), for some \(v\in S\). As \(\u_{n}\\rightarrow \infty\), then one trivially sees that \(t_{n}\rightarrow\infty\).
Now let us prove that
$$\lim_{n\to\infty}\int\frac{F(x,t_{n}v_{n})}{t_{n}^{2}} = +\infty. $$
In fact, let \(\Gamma= \{x\in\mathbb{R}^{N}; v(x) \neq0\}\), then by Fatou’s lemma and (f_{4}), it follows that
$$\begin{aligned} \lim_{n\to\infty}\int\frac{F(x,t_{n}v_{n})}{t_{n}^{2}} = & \lim _{n\to\infty}\int\frac {F(x,t_{n}v_{n})}{t_{n}^{2}v_{n}^{2}}v_{n}^{2} \\ \geq& \lim_{n\to\infty}\int_{\Gamma} \frac {F(x,ts_{n}v_{n})}{t_{n}^{2}v_{n}^{2}}v_{n}^{2} = +\infty. \end{aligned}$$
(2.15)
Let \(M > 0\) be such that \(\v_{n}\ \leq M\), for all \(n \in\mathbb{N}\). Then we have
$$\begin{aligned} I(u_{n}) = & I(t_{n} v_{n}) \\ = & t_{n}^{2} \biggl(\frac{\v_{n}\^{2}}{2}  \int \frac {F(x,t_{n}v_{n})}{t_{n}^{2}} \biggr) \\ \leq& t_{n}^{2} \biggl(\frac{M^{2}}{2}  \int \frac {F(x,t_{n}v_{n})}{t_{n}^{2}} \biggr) \\ \to& \infty \end{aligned}$$
as \(n \to\infty\). □
Now, let us prove the main result of this work
Proof of Theorem 1.3
Let \(u\in\mathcal{K}\) and \(v\in\mathcal{K}^{*}\), such that \(u,v\neq 0\) and u and v are linearly independent in H. For each \(s>0\), let us define \(h_{s}:[0,1] \rightarrow H\) as \(h_{s}(t) = s(tu + (1t)v)\).
For each \(s>0\),

\(h_{s}(1) = su \in\mathcal{K}\backslash\{0\}\),

\(h_{s}(0) = sv \in\mathcal{K}^{*}\backslash\{0\}\).
By Lemma 2.17 applied to the compact set \(S = \{tu+(1t)v; t\in[0,1]\}\), we see that if s is large enough, then \(I(h_{s}(t)) < 0\), \(\forall t\in[0,1]\). Hence, Proposition 2.16 gives us the existence of \(u_{1}\), \(u_{2}\), and \(u_{3}\), respectively, a positive, a negative, and a nodal critical point of I restricted to H. The existence of the critical points of I in all space \(H^{2}(\mathbb {R}^{N})\) follows just by applying the principle of symmetric criticality of Palais to the functional I, once we have observed that I is invariant by the action of the group \(O(N)\), and
$$H^{2}_{\mathrm{rad}}\bigl(\mathbb{R}^{N}\bigr) = \bigl\{ u \in H^{2}\bigl(\mathbb{R}^{N}\bigr); u\bigl(g(x)\bigr) = u(x), \forall g \in O(N)\bigr\} . $$
Hence the theorem follows. □