In order to apply Theorem 2.1 to prove our main result, we define A, B, and \(\varphi_{\lambda}\) on our working space E by
$$ A(u,v)=\frac{1}{2}\bigl\| (u,v)\bigr\| ^{2},\qquad B(u,v)= \int_{\mathbb{R}^{N}}H(x)F(x, u, v)\,dx, $$
(3.1)
and
$$ \varphi_{\lambda}(u,v)=A(u,v)-\lambda B(u,v)=\frac{1}{2}\bigl\| (u,v)\bigr\| ^{2}-\lambda \int_{\mathbb{R}^{N}}H(x)F(x, u, v)\,dx $$
(3.2)
for all \((u,v)\in E\) and \(\lambda\in[1,2]\). Obviously, \(\varphi _{\lambda}(u,v)\in C^{1}(E, R)\) for all \(\lambda\in[1,2]\). We choose a completely orthonormal basis \(\{e_{j}: j\in N\}\) of X and let \(X_{j}=\operatorname{span}\{e_{j}\}\) for all \(j \in N\). Then \(Y_{k}=\operatorname{span} \{e_{1},\ldots,e_{k}\}\), \(Z_{k}=Y_{k}^{\bot }\), and \(E=(Y_{k}\times Y_{k})\oplus(Z_{k}\times Z_{k})\). Note that \(\varphi_{1}=I\), where I is defined in (2.1).
Lemma 3.1
Suppose that conditions (H1), (H3), and (H4) hold. Then
\(B(u,v)\geq0\). Furthermore, \(B(u,v)\rightarrow\infty\)
as
\(\|(u,v)\|\rightarrow\infty\)
on any finite-dimensional subspace of
E.
Proof
Evidently, by (H3) and (H4), \(B(u,v)=\int _{\mathbb{R}^{N}}H(x)F(x, u, v)\,dx\geq0\), Now we claim that for any finite-dimensional subspace \(\widetilde{E}\subset E\), there exists \(\varepsilon>0\) such that
$$ \operatorname{meas}\bigl\{ x\in\mathbb{R}^{N}: H(x)\bigl|(u,v)\bigr|^{\mu}\geq \varepsilon \bigl\| (u,v)\bigr\| ^{\mu}\bigr\} \geq\varepsilon, \quad\forall(u,v)\in \widetilde{E}. $$
(3.3)
Arguing by contradiction, we assume that there exists a sequence \(\{ (u_{n},v_{n})\}_{n\in\mathbb{N}}\subset\widetilde{E}\setminus\{(0,0)\} \) such that
$$ \operatorname{meas}\biggl\{ x\in\mathbb{R}^{N}: H(x)\bigl|(u_{n},v_{n})\bigr|^{\mu} \geq\frac {1}{n}\bigl\| (u_{n},v_{n})\bigr\| ^{\mu}\biggr\} < \frac{1}{n}. $$
(3.4)
Set \((s_{n},w_{n})=\frac{(u_{n},v_{n})}{\|(u_{n},v_{n})\|}\subset \widetilde{E}\setminus\{(0,0)\}\), then \(\|(s_{n},w_{n})\|=1\) for all \(n\in\mathbb{N}\), and
$$ \operatorname{meas} \biggl\{ x\in\mathbb{R}^{N}: H(x)\bigl|(s_{n},w_{n})\bigr|^{\mu} \geq\frac {1}{n}\biggr\} < \frac{1}{n}. $$
(3.5)
Since \(\operatorname{dim} \widetilde{E}<\infty\), it follows from the compactness of the unit sphere of Ẽ that there exists a subsequence, say \(\{(s_{n},w_{n})\}\), such that \((s_{n},w_{n})\rightarrow(s_{0},w_{0})\) in Ẽ. It is easy to verify that \(\|(s_{0},w_{0})\|=1\). In view of the equivalence of the norms on the finite-dimensional space Ẽ, we have \((s_{n},w_{n})\rightarrow(s_{0},w_{0})\) in \(L^{2}(\mathbb{R}^{N})\), that is,
$$ \int_{\mathbb{R}^{N}}\bigl|(s_{n},w_{n})-(s_{0},w_{0})\bigr|^{2} \,dx\rightarrow0 \quad\mbox{as } n\rightarrow\infty. $$
(3.6)
By (3.6) and the Hölder inequality we have
$$\begin{aligned} &\int_{\mathbb{R}^{N}}H(x)\bigl|(s_{n},w_{n})- (s_{0},w_{0})\bigr|^{\mu}\,dx \\ &\quad\leq\bigl\| H(x) \bigr\| _{{\frac{2}{2-\mu}}}\biggl( \int_{\mathbb {R}^{N}}\bigl|(s_{n},w_{n})- (s_{0},w_{0})\bigr|^{2}\,dx\biggr)^{\frac{\mu}{2}} \rightarrow0 \quad\mbox{as } n\rightarrow\infty. \end{aligned}$$
(3.7)
Therefore, there exist \(\xi_{1}, \xi_{2}>0\) such that
$$ \operatorname{meas} \bigl\{ x\in\mathbb{R}^{N}: H(x)\bigl|(s_{0},w_{0})\bigr|^{\mu} \geq\xi _{1}\bigr\} \geq\xi_{2}. $$
(3.8)
Otherwise, we get
$$ \operatorname{meas} \biggl\{ x\in\mathbb{R}^{N}: H(x)\bigl|(s_{0},w_{0})\bigr|^{\mu} \geq\frac {1}{n}\biggr\} =0, $$
(3.9)
which implies that
$$0\leq \int_{\mathbb{R}^{N}}H(x)\bigl|(s_{0},w_{0})\bigr|^{\mu+2} \,dx \leq\frac{\|(s_{0},w_{0})\|^{2}_{{2}}}{n}\leq\frac {C^{2}\|(s_{0},w_{0})\|^{2}}{n}=\frac{C^{2}}{n}\rightarrow0 \quad\mbox{as } n\rightarrow\infty. $$
Hence, \((s_{0},w_{0})=0\), which contradicts with \(\|(s_{0},w_{0})\|=1\). Therefore, (3.8) holds.
Now let
$$\begin{aligned}& \Omega_{0}=\operatorname{meas} \bigl\{ x\in\mathbb{R}^{N}: H(x)\bigl|(s_{0},w_{0})\bigr|^{\mu }\geq\xi_{1}\bigr\} , \\& \Omega_{n}=\operatorname{meas} \biggl\{ x\in\mathbb{R}^{N}: H(x)\bigl|(s_{0},w_{0})\bigr|^{\mu }< \frac{1}{n}\biggr\} , \\& \Omega_{n}^{c}=\mathbb{R}^{N}\setminus \Omega_{n}=\operatorname{meas} \biggl\{ x\in \mathbb{R}^{N}: H(x)\bigl|(s_{0},w_{0})\bigr|^{\mu}\geq\frac{1}{n} \biggr\} . \end{aligned}$$
Then by (3.5) and (3.8) we get
$$\begin{aligned}[b] \operatorname{meas} (\Omega_{n}\cap \Omega_{0}) &=\operatorname{meas} \bigl(\Omega_{0}\setminus \bigl(\Omega_{n}^{c}\cap\Omega_{0}\bigr)\bigr) \\ &\geq\operatorname{meas} (\Omega_{0})-\operatorname{meas} \bigl( \Omega_{n}^{c}\cap\Omega _{0}\bigr) \\ &\geq\xi_{2}-\frac{1}{n} \end{aligned} $$
for all positive integer n. Let n be large enough such that \(\xi _{2}-\frac{1}{n}\geq\frac{\xi_{2}}{2}\) and \(\frac{\xi_{1}}{2^{\mu}}-\frac{1}{n}\geq\frac{\xi_{1}}{2^{\mu+1}}\). Then we have
$$\begin{aligned}[b] &\int_{\mathbb{R}^{N}}H(x)\bigl|(s_{n},w_{n})-(s_{0},w_{0})\bigr|^{\mu} \,dx \\ &\quad\geq \int_{\Omega_{n}\cap\Omega _{0}}H(x)\bigl|(s_{n},w_{n})-(s_{0},w_{0})\bigr|^{\mu} \,dx\\ &\quad\geq\frac{1}{2^{\mu}} \int_{\Omega_{n}\cap\Omega _{0}}H(x)\bigl|(s_{0},w_{0})\bigr|^{\mu} \,dx- \int_{\Omega_{n}\cap\Omega _{0}}H(x)\bigl|(s_{n},w_{n})\bigr|^{\mu} \,dx \\ &\quad\geq\biggl(\frac{\xi_{1}}{2^{\mu}}-\frac{1}{n}\biggr) \operatorname{meas} ( \Omega_{n}\cap \Omega_{0})>0, \end{aligned} $$
which is a contradiction with (3.7). Therefore, (3.3) holds. For the ε given in (3.3), let
$$\Omega_{(u,v)}=\operatorname{meas} \bigl\{ x\in\mathbb{R}^{N}: H(x)\bigl|(u,v)\bigr|^{\mu}\geq \varepsilon\bigl\| (u,v)\bigr\| ^{\mu}\bigr\} ,\quad \forall(u,v)\in\widetilde{E}\setminus\bigl\{ (0,0)\bigr\} . $$
Then by (3.3)
$$ \operatorname{meas} (\Omega_{(u,v)})\geq\varepsilon,\quad \forall(u,v)\in \widetilde{E}\setminus\bigl\{ (0,0)\bigr\} . $$
(3.10)
Combining (H3) and (3.10), we have
$$B(u,v)\geq \int_{\mathbb{R}^{N}}c_{1}H(x)\bigl|(u,v)\bigr|^{\mu}\,dx\geq \int_{\Omega _{(u,v)}}\varepsilon c_{1}\bigl\| (u,v)\bigr\| ^{\mu} \,dx \geq\varepsilon^{2} c_{1}\bigl\| (u,v)\bigr\| ^{\mu}, $$
which implies that \(B(u,v)\rightarrow\infty\) as \(\|(u,v)\|\rightarrow \infty\) on any finite-dimensional space of E. The proof is completed. □
Lemma 3.2
Suppose that (H1)-(H2) and (H4) are satisfied. Then there exists a sequence
\(\rho_{k}\rightarrow0^{+}\)
as
\(k\rightarrow\infty\)
such that
\(a_{k}(\lambda) :=\inf_{(u,v)\in Z_{k}\times Z_{k}, \|(u,v)\|=\rho_{k}}\varphi _{\lambda}(u,v)\geq0\)
and
\(d_{k}(\lambda):=\inf_{(u,v)\in Z_{k}\times Z_{k}, \|(u,v)\|\leq\rho_{k}} \varphi_{\lambda }(u,v)\rightarrow0\)
as
\(k\rightarrow\infty\)
uniformly for
\(\lambda\in [1,2]\), where
\(Z_{k}=\overline{\bigoplus_{j=k+1}^{\infty }X_{j}}=\overline{\operatorname{span}\{e_{k},\ldots\}}\)
for all
\(k\in N\).
Proof
Let
$$ \alpha_{k}(r):=\sup_{(u,v)\in Z_{k}\times Z_{k}, \|(u,v)\|=1}\bigl\| (u,v)\bigr\| _{r},\quad \forall k\in N, $$
(3.11)
where \(\|(u,v)\|_{r}=(\int_{\mathbb {R}^{N}}|(u,v)|^{r})^{\frac{1}{r}}\). Then \(\alpha_{k}(r)\rightarrow0\) as \(k\rightarrow\infty\). Indeed, \(\alpha_{k}(r)\) is convergent since \(\alpha_{k}(r)\) are decreasing in k and \(\alpha_{k}(r)\geq0\). Furthermore, for any k, there exists \((u_{k},v_{k})\in Z_{k}\times Z_{k}\) such that \(\|(u_{k},v_{k})\|=1\) and \(\|(u_{k},v_{k})\|_{r}\geq\frac{\alpha_{k}(r)}{2}\).
For any \(\varphi\in X\), \(\varphi=\sum_{n=1}^{\infty}a_{n}e_{n}\), we get
$$\bigl|\langle u_{k},\varphi\rangle_{X}\bigr|= \Biggl| \Biggl\langle u_{k},\sum_{n=k+1}^{\infty}a_{n}e_{n} \Biggr\rangle _{X} \Biggr|\leq\|u_{k}\|_{X} \Biggl\| \sum _{n=k+1} ^{\infty}a_{n}e_{n} \Biggr\| _{X}\leq \Biggl\| \sum_{n=k+1} ^{\infty}a_{n}e_{n} \Biggr\| _{X}\rightarrow0 $$
as \(k\rightarrow\infty\), which implies that \(u_{k}\rightharpoonup0\) in X. Since the embedding \(X\hookrightarrow L^{r}(\mathbb {R}^{N})\) is compact, \(u_{k}\rightarrow0\) in \(L^{r}(\mathbb {R}^{N})\) for \(r\in[2, 2^{*})\). The same argument implies that \(v_{k}\rightarrow0\) in \(L^{r}(\mathbb {R}^{N})\) for \(r\in[2, 2^{*})\). Consequently,
$$\bigl\| (u_{k},v_{k})\bigr\| _{r}^{r} \leq2^{\frac {r}{2}}\bigl(\|u_{k}\|_{r}^{r}+ \|v_{k}\|_{r}^{r}\bigr)\rightarrow \quad\mbox{as } k \rightarrow\infty, $$
that is,
$$ \alpha_{k}(r)\rightarrow0 \quad\mbox{as } k\rightarrow\infty. $$
(3.12)
By (H2) we have
$$ \begin{aligned}[b] \bigl|F(x,u,v)\bigr| &=\bigl|F(x,u,v)-F(x,0,0)\bigr|\\ &\leq \int_{0}^{1}\bigl|F_{u}(x,tu,tv)\bigr||u|\,dt+ \int _{0}^{1}\bigl|F_{v}(x,tu,tv)\bigr||v|\,dt\\ &\leq C\bigl(\bigl|(u,v)\bigr|^{2}+\bigl|(u,v)\bigr|^{p}+\bigl|(u,v)\bigr|^{q} \bigr). \end{aligned} $$
(3.13)
Therefore, by (3.2), (3.11)-(3.13), and the Hölder inequality we get
$$\begin{aligned} \varphi_{\lambda}(u,v) &=\frac{1}{2}\bigl\| (u,v) \bigr\| ^{2}-\lambda \int_{\mathbb{R}^{N}}H(x)F(x, u, v)\,dx \\ &\geq\frac{1}{2}\bigl\| (u,v)\bigr\| ^{2}-2C\bigl(\bigl\| (u,v) \bigr\| _{2}^{2}+\bigl\| H(x)\bigr\| _{\frac {2}{2-p}}\bigl\| (u,v)\bigr\| _{2}^{p}+ \bigl\| H(x)\bigr\| _{\frac{2}{2-q}}\bigl\| (u,v)\bigr\| _{2}^{q}\bigr) \\ &\geq\frac{1}{2}\bigl\| (u,v)\bigr\| ^{2}-2C\bigl(\alpha_{k}^{2}(2) \bigl\| (u,v)\bigr\| ^{2}+\alpha _{k}^{p}(2)\bigl\| (u,v) \bigr\| ^{p}+\alpha_{k}^{q}(2)\bigl\| (u,v)\bigr\| ^{q} \bigr). \end{aligned}$$
(3.14)
By (3.13) there exist a positive integer \(k_{1}\) such that
$$ 2C\alpha_{k}^{2}(2)\leq\frac{1}{8},\quad \forall k\geq k_{1}. $$
(3.15)
Then, by (3.14), we have
$$ \varphi_{\lambda}(u,v)\geq\frac{3}{8}\bigl\| (u,v)\bigr\| ^{2}-2C \bigl(\alpha _{k}^{p}(2)\bigl\| (u,v)\bigr\| ^{p}+ \alpha_{k}^{q}(2)\bigl\| (u,v)\bigr\| ^{q} \bigr). $$
(3.16)
Let
$$ \rho_{k}=\max\bigl\{ \bigl(16C\alpha_{k}^{p}(2) \bigr)^{\frac{1}{2-p}}, \bigl(16C\alpha _{k}^{q}(2) \bigr)^{\frac{1}{2-q}}\bigr\} . $$
(3.17)
Obviously, \(\rho_{k}\rightarrow0\) as \(k\rightarrow\infty\) since \(p, q\in(1,2)\). By (3.16) and (3.17) direct computation shows that
$$a_{k}(\lambda) :=\inf_{(u,v)\in Z_{k}\times Z_{k}, \|(u,v)\|= \rho_{k}}\varphi_{\lambda}(u,v) \geq\frac{\rho_{k}^{2}}{8}>0,\quad \forall k\geq k_{1}. $$
Moreover, by (3.14), for any \((u,v)\in Z_{k}\times Z_{k}\) with \(\|(u,v)\|= \rho_{k}\), we have
$$\varphi_{\lambda}(u,v)\geq-2C\bigl(\alpha_{k}^{2}(2) \bigl\| (u,v)\bigr\| ^{2}+\alpha _{k}^{p}(2)\bigl\| (u,v) \bigr\| ^{p}+\alpha_{k}^{q}(2)\bigl\| (u,v)\bigr\| ^{q} \bigr). $$
Therefore,
$$0\geq\inf_{(u,v)\in Z_{k}\times Z_{k}, \|(u,v)\|\leq\rho _{k}} \varphi_{\lambda}(u,v)\geq-2C\bigl( \alpha_{k}^{2}(2)\bigl\| (u,v)\bigr\| ^{2}+\alpha _{k}^{p}(2)\bigl\| (u,v)\bigr\| ^{p}+\alpha_{k}^{q}(2) \bigl\| (u,v)\bigr\| ^{q}\bigr). $$
Since \(\alpha_{k}(2)\rightarrow0\) as \(k\rightarrow\infty\), we have
$$d_{k}(\lambda):=\inf_{(u,v)\in Z_{k}\times Z_{k}, \|(u,v)\|\leq \rho_{k}}\varphi_{\lambda}(u,v) \rightarrow0 \quad\mbox{as } k\rightarrow \infty \mbox{ uniformly for } \lambda\in[1,2]. $$
The proof is completed. □
Lemma 3.3
Suppose that (H1)-(H4) hold. Then for the positive integer
\(k_{1}\)
and the sequence
\(\{\rho_{k}\}\)
obtained in Lemma
3.2, for all
\(k\geq k_{1}\), there exist
\(0< r_{k}<\rho_{k}\)
such that
$$b_{k}(\lambda):=\max_{(u,v)\in Y_{k}\times Y_{k}, \|(u,v)\|=r_{k}}\varphi_{\lambda}(u,v)< 0\quad \textit{for all } \lambda\in [1,2], \forall k\geq k_{1}, $$
where
\(Y_{k}=\bigoplus_{j=1}^{k}X_{j}=\operatorname{span}\{e_{1},\ldots e_{k}\}\)
for all
\(k\in N\).
Proof
Note that \(Y_{k}\times Y_{k}\) is a finite-dimensional subspace of E. Then by (3.3) there exists a constant \(\varepsilon_{k}\) such that
$$ \operatorname{meas} \bigl(\Omega_{(u,v)}^{k}\bigr)\geq \varepsilon_{k}, \quad\forall(u,v)\in Y_{k}\times Y_{k}\setminus\bigl\{ (0,0)\bigr\} , $$
(3.18)
where
$$\Omega_{(u,v)}^{k}=\operatorname{meas} \bigl\{ x\in \mathbb{R}^{N}: H(x)\bigl|(u,v)\bigr|^{\mu }\geq\varepsilon_{k} \bigl\| (u,v)\bigr\| ^{\mu}\bigr\} ,\quad \forall(u,v)\in Y_{k}\times Y_{k}\setminus\bigl\{ (0,0)\bigr\} . $$
Combining (3.2), (H3), (H4), and (3.18), for any \(k\in N\) and \(\lambda\in[1,2]\), we have
$$ \begin{aligned}[b] \varphi_{\lambda}(u,v) &=\frac{1}{2}\bigl\| (u,v) \bigr\| ^{2}-\lambda \int_{\mathbb{R}^{N}}H(x)F(x, u, v)\,dx \\ &\leq\frac{1}{2}\bigl\| (u,v)\bigr\| ^{2}-c_{1} \int_{\mathbb{R}^{N}}H(x)\bigl|(u,v)\bigr|^{\mu }\,dx \\ &\leq\frac{1}{2}\bigl\| (u,v)\bigr\| ^{2}-c_{1} \int_{\Omega _{(u,v)}^{k}}H(x)\bigl|(u,v)\bigr|^{\mu}\,dx \\ &\leq\frac{1}{2}\bigl\| (u,v)\bigr\| ^{2}-c_{1} \varepsilon_{k}\bigl\| (u,v)\bigr\| ^{\mu} \operatorname{meas} \bigl( \Omega_{(u,v)}^{k}\bigr) \\ &\leq\frac{1}{2}\bigl\| (u,v)\bigr\| ^{2}-c_{1} \varepsilon_{k}^{2}\bigl\| (u,v)\bigr\| ^{\mu}. \end{aligned} $$
(3.19)
For \(\|(u,v)\|=r_{k}<\rho_{k}\) small enough, we have
$$b_{k}(\lambda):=\max_{(u,v)\in Y_{k}\times Y_{k}, \|(u,v)\|=r_{k}}\varphi_{\lambda}(u,v)< 0 \quad\mbox{for all } \lambda\in [1,2], \forall k\geq k_{1}, $$
since \(\mu\in(1,2)\). The proof is completed. □
Now we give the proof of Theorem 1.1.
Proof
Obviously, condition (C1) in Theorem 2.1 holds. By Lemmas 3.1-3.3 conditions (C2) and (C3) in Theorem 2.1 are also satisfied. Furthermore, by Theorem 2.1, there exist \(\lambda_{n}\rightarrow1\) and \((u(\lambda_{n}),v(\lambda _{n}))\in Y_{n}\times Y_{n}\) such that
$$ \begin{aligned} &\varphi_{\lambda_{n}}'|_{Y_{n}\times Y_{n}}\bigl(u( \lambda_{n}),v(\lambda_{n})\bigr)=0, \\ &\varphi_{\lambda_{n}} \bigl(u(\lambda_{n}),v(\lambda_{n})\bigr) \rightarrow c_{k}\in\bigl[d_{k}(2),b_{k}(1)\bigr] \quad\mbox{as } n\rightarrow\infty . \end{aligned} $$
(3.20)
For simplicity, in what follows, we always set \((u_{n},v_{n})=(u(\lambda_{n}),v(\lambda_{n}))\) for all \(n\in\mathbb {N}\).
Now we claim that the sequence \(\{(u_{n},v_{n})\}\) obtained in (3.19) is bounded in E. Indeed, by (H2), (H4), (3.2), (3.20), and the Hölder inequality we have
$$ \begin{aligned}[b]\bigl\| (u_{n},v_{n})\bigr\| ^{2} &\leq2 \varphi_{\lambda_{n}}\bigl((u_{n},v_{n})\bigr)+2 \lambda_{n} \int_{\mathbb {R}^{N}}H(x)F(x,u_{n},v_{n})\,dx \\ &\leq C_{0}+C\bigl(\bigl\| (u_{n},v_{n}) \bigr\| ^{2}+\bigl\| (u,v)\bigr\| _{p}^{p}+\bigl\| (u,v) \bigr\| _{q}^{q}\bigr) \end{aligned} $$
(3.21)
for some \(C_{0}>0\). Since \(p, q\in(1,2)\), (3.21) implies that \(\{(u_{n},v_{n})\}\) is bounded in E.
Finally, we show that \(\{(u_{n},v_{n})\}\) possesses a strong convergent sequence in E. Indeed, since \(\{(u_{n},v_{n})\}\) is bounded, there exists \((u_{0},v_{0})\in E\) such that
$$\begin{aligned}& (u_{n},v_{n})\rightharpoonup(u_{0},v_{0}) \quad\mbox{in } E,\\& (u_{n},v_{n})\rightarrow(u_{0},v_{0}) \quad\mbox{in } L^{p}\bigl(\mathbb {R}^{N}\bigr)\times L^{p}\bigl(\mathbb {R}^{N}\bigr), p\in\bigl[2,2^{*}\bigr),\\& (u_{n},v_{n})\rightarrow(u_{0},v_{0}) \quad\mbox{a.e. on } \mathbb {R}^{N}. \end{aligned}$$
By (2.2) we easily get
$$ \begin{aligned}[b] \bigl\| (u_{n},v_{n})-(u_{0},v_{0}) \bigr\| ^{2} ={}&\bigl\langle \varphi_{\lambda_{n}}'(u_{n},v_{n})- \varphi _{1}'(u_{0},v_{0}),(u_{n},v_{n})-(u_{0},v_{0}) \bigr\rangle \\ &{} + \int_{\mathbb {R}^{N}}H(x) \bigl(\lambda _{n}F_{u}(x,u_{n},v_{n})-F_{u}(x,u_{0},v_{0}) \bigr) (u_{n}-u_{0})\,dx \\ &{} + \int_{\mathbb {R}^{N}}H(x) \bigl(\lambda _{n}F_{v}(x,u_{n},v_{n})-F_{v}(x,u_{0},v_{0}) \bigr) (v_{n}-v_{0})\,dx. \end{aligned} $$
(3.22)
Clearly,
$$ \bigl\langle \varphi_{\lambda_{n}}'(u_{n},v_{n})- \varphi _{1}'(u_{0},v_{0}),(u_{n},v_{n})-(u_{0},v_{0}) \bigr\rangle \rightarrow0. $$
(3.23)
Denote
$$\begin{aligned}& M:= \int_{\mathbb {R}^{N}}H(x) \bigl(\lambda _{n}F_{u}(x,u_{n},v_{n})-F_{u}(x,u_{0},v_{0}) \bigr) (u_{n}-u_{0})\,dx,\\& N:= \int_{\mathbb {R}^{N}}H(x) \bigl(\lambda _{n}F_{v}(x,u_{n},v_{n})-F_{v}(x,u_{0},v_{0}) \bigr) (v_{n}-v_{0})\,dx. \end{aligned}$$
Then by (H2), (H4), and the Hölder and Minkowski inequalities we have
$$\begin{aligned} M \leq{}& C\|u_{n}-u_{0}\|_{2} \biggl( \int_{\mathbb {R}^{N}}H^{2}(x) \bigl(2\bigl|(u_{n},v_{n})\bigr| +2\bigl|(u_{n},v_{n})\bigr|^{p-1}+2\bigl|(u_{n},v_{n})\bigr|^{q-1} \\ &{} +\bigl|(u_{0},v_{0})\bigr|+\bigl|(u_{0},v_{0})\bigr|^{p-1}+\bigl|(u_{0},v_{0})\bigr|^{q-1} \bigr)^{2}\,dx \biggr)^{\frac{1}{2}} \\ \leq{}& C\|u_{n}-u_{0}\|_{2} \biggl( \int_{\mathbb {R}^{N}}H^{2}(x) \bigl(2\bigl|(u_{n},v_{n})\bigr|+\bigl|(u_{0},v_{0})\bigr| \bigr)^{2}+H^{2}(x) \bigl(2\bigl|(u_{n},v_{n})\bigr|^{p-1} \\ &{} +\bigl|(u_{0},v_{0})\bigr|^{p-1}\bigr)^{2}+H^{2}(x) \bigl(2\bigl|(u_{n},v_{n})\bigr|^{q-1}+\bigl|(u_{0},v_{0})\bigr|^{q-1} \bigr)^{2}\,dx \biggr)^{\frac {1}{2}} \\ \leq{}& C\|u_{n}-u_{0}\|_{2} \biggl( \int_{\mathbb {R}^{N}}H^{2}(x) \bigl(4\bigl|(u_{n},v_{n})\bigr|^{2} +\bigl|(u_{0},v_{0})\bigr|^{2} \bigr)+H^{2}(x) \bigl(4\bigl|(u_{n},v_{n})\bigr|^{2p-2} \\ &{} +\bigl|(u_{0},v_{0})\bigr|^{2p-2}\bigr)+H^{2}(x) \bigl(4\bigl|(u_{n},v_{n})\bigr|^{2q-2}+\bigl|(u_{0},v_{0})\bigr|^{2q-2} \bigr)\,dx \biggr)^{\frac{1}{2}} \\ \leq{}& C\|u_{n}-u_{0}\|_{2} \biggl[ \biggl( \int_{\mathbb {R}^{N}}4H^{2}(x)\bigl|(u_{n},v_{n})\bigr|^{2} \,dx \biggr)^{\frac{1}{2}}+ \biggl( \int _{\mathbb {R}^{N}}H^{2}(x)\bigl|(u_{0},v_{0})\bigr|^{2} \,dx \biggr)^{\frac{1}{2}} \\ &{} + \biggl( \int_{\mathbb {R}^{N}}4H^{2}(x)\bigl|(u_{n},v_{n})\bigr|^{2p-2} \biggr)^{\frac{1}{2}}+ \biggl( \int_{\mathbb {R}^{N}}H^{2}(x)\bigl|(u_{0},v_{0})\bigr|^{2p-2} \,dx \biggr)^{\frac{1}{2}} \\ &{} + \biggl( \int_{\mathbb {R}^{N}}4H^{2}(x)\bigl|(u_{n},v_{n})\bigr|^{2q-2} \,dx \biggr)^{\frac{1}{2}}\,dx+ \biggl( \int_{\mathbb {R}^{N}}H^{2}(x)\bigl|(u_{0},v_{0})\bigr|^{2q-2} \,dx \biggr)^{\frac{1}{2}} \biggr] \\ \leq{}& C\|u_{n}-u_{0}\|_{2} \bigl[2\|H \|_{\infty }\bigl\| (u_{n},v_{n})\bigr\| _{2}^{2}+ \|H\|_{\infty}\bigl\| (u_{0},v_{0})\bigr\| _{2}^{2} +2\|H\|_{\frac{2}{2-p}}\bigl\| (u_{n},v_{n})\bigr\| _{2}^{p-1} \\ &{} +\|H\|_{\frac{2}{2-p}}\bigl\| (u_{0},v_{0})\bigr\| _{2}^{p-1}+2 \|H\|_{\frac {2}{2-q}}\bigl\| (u_{n},v_{n})\bigr\| _{2}^{q-1}+ \|H\|_{\frac{2}{2-q}}\bigl\| (u_{0},v_{0})\bigr\| _{2}^{q-1} \bigr] \\ \leq{}& C\|u_{n}-u_{0}\|_{2} \bigl( \bigl\| (u_{n},v_{n})\bigr\| _{2}^{2}+ \bigl\| (u_{0},v_{0})\bigr\| _{2}^{2}+ \bigl\| (u_{n},v_{n})\bigr\| _{2}^{p-1} + \bigl\| (u_{0},v_{0})\bigr\| _{2}^{p-1} \\ &{} +\bigl\| (u_{n},v_{n})\bigr\| _{2}^{q-1}+ \bigl\| (u_{0},v_{0})\bigr\| _{2}^{q-1} \bigr). \end{aligned}$$
(3.24)
Since \((u_{n},v_{n})\rightarrow(u_{0},v_{0})\) in \(L^{p}(\mathbb {R}^{N})\times L^{p}(\mathbb {R}^{N})\), for any \(p\in[2,2^{*})\), we obtain
$$ \int_{\mathbb {R}^{N}}H(x) \bigl(\lambda _{n}F_{u}(x,u_{n},v_{n})-F_{u}(x,u_{0},v_{0}) \bigr) (u_{n}-u_{0})\,dx\rightarrow 0 \quad\mbox{as } n\rightarrow \infty. $$
(3.25)
Similarly, we can also obtain
$$ \int_{\mathbb {R}^{N}}H(x) \bigl(\lambda _{n}F_{v}(x,u_{n},v_{n})-F_{v}(x,u_{0},v_{0}) \bigr) (v_{n}-v_{0})\,dx\rightarrow 0 \quad\mbox{as } n\rightarrow \infty. $$
(3.26)
Therefore, by (3.22)-(3.26) we get \(\|(u_{n},v_{n})-(u_{0},v_{0})\| \rightarrow0\) as \(n\rightarrow\infty\).
Now from the last assertion of Theorem 2.1 we know that \(I=\varphi_{1}\) has infinitely many nontrivial critical points. Therefore, system (1.1) possesses infinitely many small negative-energy solutions. The proof is completed. □