Theorem 3.1
Assume that
\((\phi_{1})\), \((\phi_{2})\), \((V_{1})\), \((V_{2})\), \((F_{1})\)
and the following conditions hold:
-
\((F_{2})\)
:
-
$$\begin{aligned} &\lim_{ \vert (u,v) \vert \rightarrow0}\frac{F_{u}(x,u,v)}{\phi_{1}( \vert u \vert )+{\widetilde {\Phi}_{1}}^{-1}(\Phi_{2}( \vert v \vert ))}=0,\qquad \lim_{ \vert (u,v) \vert \rightarrow0} \frac{F_{v}(x,u,v)}{{\widetilde{\Phi }_{2}}^{-1}(\Phi_{1}( \vert u \vert ))+\phi_{2}( \vert v \vert )}=0, \\ &\lim_{ \vert (u,v) \vert \rightarrow\infty}\frac{F_{u}(x,u,v)}{\Phi _{1*}'( \vert u \vert )+{\widetilde{\Phi}_{1*}}^{-1}(\Phi_{2*}( \vert v \vert ))}=0, \qquad\lim_{ \vert (u,v) \vert \rightarrow\infty} \frac{F_{v}(x,u,v)}{{\widetilde{\Phi }_{2*}}^{-1}(\Phi_{1*}( \vert u \vert ))+\Phi_{2*}'( \vert v \vert )}=0, \end{aligned}$$
uniformly in
\(x\in\mathbb{R}^{N}\), where and in the sequel
\({\widetilde {\Phi}_{i}}^{-1}\)
denotes the inverse of
\(\widetilde{\Phi}_{i} (i=1,2)\), \(\Phi_{i*}'\)
denotes the derivative of
\(\Phi_{i*}(i=1,2)\)
and
\({\widetilde{\Phi}_{i*}}^{-1}\)
denotes the inverse of
\(\widetilde{\Phi }_{i*}(i=1,2)\);
-
\((F_{3})\)
:
-
there exist
\(\mu_{i}>m_{i} (i=1,2)\)
such that
$$0< F(x,u,v)\leq\frac{1}{\mu_{1}}uF_{u}(x,u,v)+\frac{1}{\mu_{2}}vF_{v}(x,u,v)\quad \textit{for all }(u,v)\neq(0,0). $$
Then system (1.1) has a ground state, that is, a nontrivial solution
\((u_{0},v_{0})\)
such that
$$I(u_{0},v_{0})=\inf\bigl\{ I(u,v): (u,v)\in W\setminus \bigl\{ (\mathbf{0,0})\bigr\} \textit{ and } I'(u,v)=0\bigr\} , $$
where
\(W=W^{1, \Phi_{1}}(\mathbb{R}^{N})\times W^{1, \Phi_{2}}(\mathbb{R}^{N})\)
and
$$\begin{aligned} I(u,v) ={}& \int_{\mathbb{R}^{N}}\Phi_{1}\bigl( \vert \nabla u \vert \bigr)\,dx+ \int_{\mathbb {R}^{N}}V_{1}(x)\Phi_{1}\bigl( \vert u \vert \bigr)\,dx \\ & {}+ \int_{\mathbb{R}^{N}}\Phi_{2}\bigl( \vert \nabla v \vert \bigr)\,dx+ \int_{\mathbb {R}^{N}}V_{2}(x)\Phi_{2}\bigl( \vert v \vert \bigr)\,dx- \int_{\mathbb{R}^{N}}F(x,u,v)\,dx. \end{aligned}$$
Theorem 3.2
Assume that
\((\phi_{1})\), \((\phi_{2})\), \((V_{1})\), \((V_{2})\), \((F_{1})\), \((F_{2})\)
and the following conditions hold:
-
\((\phi_{3})\)
:
-
$$\limsup_{t\rightarrow0}\frac{ \vert t \vert ^{l_{i}}}{\Phi_{i}( \vert t \vert )}< \infty, \quad i=1,2; $$
-
\((F_{4})\)
:
-
$$\lim_{ \vert (u,v) \vert \rightarrow\infty}\frac{F(x,u,v)}{\Phi_{1}( \vert u \vert )+\Phi _{2}( \vert v \vert )}=+\infty $$
uniformly in
\(x\in\mathbb{R}^{N}\);
-
\((F_{5})\)
:
-
\(\overline{F}(x,u,v)>0\)
for all
\((u,v)\neq(0,0)\)
and there exists
\(k>\max\{\frac{N}{l_{1}},\frac{N}{l_{2}}\}\)
such that
$$\limsup_{ \vert (u,v) \vert \rightarrow\infty} \biggl(\frac {F(x,u,v)}{ \vert u \vert ^{l_{1}}+ \vert v \vert ^{l_{2}}} \biggr)^{k} \frac{1}{\overline {F}(x,u,v)}< \infty, $$
where
$$\overline{F}(x,u,v)=\frac{1}{m_{1}}u F_{u}(x,u,v)+ \frac{1}{m_{2}}v F_{v}(x,u,v)-F(x,u,v). $$
Then system (1.1) has a ground state.
By Lemmas 2.1 and 2.4, it is easy to check that the following conditions \((F_{2})'\) and \((F_{4})'\) imply \((F_{2})\) and \((F_{4})\), respectively.
-
\((F_{2})'\)
:
-
$$\begin{aligned} &\lim_{ \vert (u,v) \vert \rightarrow0}\frac{F_{u}(x,u,v)}{ \vert u \vert ^{m_{1}-1}+ \vert v \vert ^{\frac {m_{2}(m_{1}-1)}{m_{1}}}}=0,\qquad \lim_{ \vert (u,v) \vert \rightarrow0} \frac{F_{v}(x,u,v)}{ \vert u \vert ^{\frac {m_{1}(m_{2}-1)}{m_{2}}}+ \vert v \vert ^{m_{2}-1}}=0, \\ &\lim_{ \vert (u,v) \vert \rightarrow\infty}\frac {F_{u}(x,u,v)}{ \vert u \vert ^{l_{1}^{*}-1}+ \vert v \vert ^{\frac{l_{2}^{*}(l_{1}^{*}-1)}{l_{1}^{*}}}}=0,\\ & \lim_{ \vert (u,v) \vert \rightarrow\infty} \frac{F_{v}(x,u,v)}{ \vert u \vert ^{\frac {l_{1}^{*}(l_{2}^{*}-1)}{l_{2}^{*}}}+ \vert v \vert ^{l_{2}^{*}-1}}=0,\quad\textit{uniformly in }x\in \mathbb{R}^{N}; \end{aligned}$$
-
\((F_{4})'\)
:
-
$$\lim_{ \vert (u,v) \vert \rightarrow\infty}\frac {F(x,u,v)}{ \vert u \vert ^{m_{1}}+ \vert v \vert ^{m_{2}}}=+\infty, \quad\textit{uniformly in }x \in \mathbb{R}^{N}. $$
Thus, we have the following corollary.
Corollary 3.3
In Theorems
3.1
and
3.2, if conditions
\((F_{2})\)
and
\((F_{4})\)
are replaced by
\((F_{2})'\)
and
\((F_{4})'\), respectively, then the conclusions still hold.
Remark 3.4
We point out that Theorems 3.1 and 3.2 are complementary, which is based on the fact that there are functions satisfying \((F_{4})\) and \((F_{5})\) but not satisfying \((F_{3})\) (see Example 4.2 in Section 4) and there are also functions \(\phi _{i}(i=1,2)\) defined by (1.2) satisfying \((\phi_{1})\) and \((\phi_{2})\) but not satisfying \((\phi_{3})\) (see Case 4 in Section 4).
When system (1.1) reduces to equation (1.3), we present the following results which correspond to Theorems 3.1 and 3.2.
Corollary 3.5
Assume that functions
\(a_{1}\), \(V_{1}\)
and
f
satisfy
\((\phi_{1})\)-\((\phi_{2})\), \((V_{1})\)-\((V_{2})\)
and
-
\((f_{1})^{*}\)
:
-
\(f\in C(\mathbb{R}^{N}, \mathbb{R})\)
is 1-periodic in
x,
-
\((f_{2})^{*}\)
:
-
$$\lim_{ \vert u \vert \rightarrow0}\frac{f(x,u)}{\phi_{1}( \vert u \vert )}=0,\qquad \lim_{ \vert u \vert \rightarrow\infty} \frac{f(x,u)}{\Phi_{1*}'( \vert u \vert )}=0 $$
uniformly in
\(x\in\mathbb{R}^{N}\);
-
\((f_{3})^{*}\)
:
-
there exists
\(\mu>m_{1}\)
such that
$$0< \mu F(x,u)\leq uf(x,u) \quad\textit{for all } u\neq0. $$
Then equation (1.3) has a ground state in
\(W^{1,\Phi_{1}}(\mathbb{R}^{N})\).
Corollary 3.6
Assume that functions
\(a_{1}\), \(V_{1}\)
and
f
satisfy
\((\phi_{1})\)-\((\phi_{3})\), \((V_{1})\)-\((V_{2})\), \((f_{1})^{*}\)-\((f_{2})^{*}\)
and
-
\((f_{4})^{*}\)
:
-
$$\lim_{ \vert u \vert \rightarrow\infty}\frac{F(x,u)}{\Phi_{1}( \vert u \vert )}=+\infty $$
uniformly in
\(x\in\mathbb{R}^{N}\);
-
\((f_{5})^{*}\)
:
-
\(\overline{F}(x,u)>0\)
for all
\(u\neq0\)
and there exists
\(k>\frac{N}{l_{1}}\)
such that
$$\limsup_{ \vert (u,v) \vert \rightarrow\infty} \biggl(\frac{F(x,u)}{ \vert u \vert ^{l_{1}}} \biggr)^{k} \frac{1}{\overline{F}(x,u)}< \infty, $$
where
$$\overline{F}(x,u)=u f(x,u)-m_{1}F(x,u,v). $$
Then equation (1.3) has a ground state in
\(W^{1,\Phi_{1}}(\mathbb{R}^{N})\).
Remark 3.7
It is easy to see that our results are different from Theorem 3.4 and Theorem 4.11 in [12].
Remark 3.8
For the nonlinearity f, our subcritical growth condition in the Orlicz-Sobolev space
$$\begin{aligned} \lim_{ \vert u \vert \rightarrow\infty}\frac{f(x,u)}{\Phi_{1*}'( \vert u \vert )}=0, \quad\textit{uniformly in } x\in\mathbb{R}^{N} \end{aligned}$$
(3.1)
in \((f_{2})^{*}\) is weaker than the following one which is usually assumed in many papers in order to consider ϕ-Laplacian problems (for example, see [9–12]):
\((SC)\)
there exist a constant
\(C>0\)
and an
N-function defined by
\(\Psi(t):=\int_{0}^{t}\psi(s)\,ds, t\in[0,+\infty)\)
satisfying
$$m_{1}< l_{\Psi}:=\inf_{t>0} \frac{t\psi(t)}{\Psi(t)}\leq\sup_{t>0}\frac {t\psi(t)}{\Psi(t)} =:m_{\Psi}< l_{1}^{*} $$
or increasing essentially more slowly than
\(\Phi_{1*}\)
near infinity, such that
$$\limsup_{ \vert u \vert \rightarrow\infty} \biggl\vert \frac{f(x,u)}{\psi(u)} \biggr\vert < \infty,\quad \textit{uniformly in } x\in\mathbb{R}^{N}. $$
Condition (3.1) was introduced by Alves et al. [8] for the autonomous nonlinearity
f
in the Orlicz-Sobolev space. When
\(a_{1}( \vert t \vert )t= \vert t \vert ^{p-2}t (p>1)\)
, (3.1) reduces to
$$\begin{aligned} \lim_{ \vert u \vert \rightarrow\infty}\frac{f(x,u)}{ \vert u \vert ^{p^{*}-1}}=0, \quad\mbox{uniformly in } x\in\mathbb{R}^{N}, \end{aligned}$$
(3.2)
which was first introduced by Liu and Wang [22] instead of the usual subcritical growth condition, that is, there exist constants
\(C>0\)
and
\(q\in(p,p^{*})\)
such that
$$\begin{aligned} \bigl\vert f(x,u) \bigr\vert \leq C\bigl( \vert u \vert ^{p-1}+ \vert u \vert ^{q-1}\bigr)\quad \textit{for all }(x,u)\in \mathbb {R}^{N}\times\mathbb{R}. \end{aligned}$$
(3.3)
Remark 3.9
A condition similar to \((f_{5})^{*}\) was introduced by Carvalho et al. [11] for the ϕ-Laplacian equation in the bounded domain \(\Omega\subset\mathbb{R}^{N}\). In this paper, because we consider problems on the whole space \(\mathbb {R}^{N}\) where the Sobolev spaces lack compactness of the Sobolev embedding, we claim \(\overline{F}(x,u)>0\) for all \(u\neq0\) in \((f_{5})^{*}\).
When \(a_{1}( \vert t \vert )t= \vert t \vert ^{p-2}t (1< p< N)\), it is obvious that \((\phi _{1})\)-\((\phi_{3})\) hold, and then we also present the corresponding results for equation (1.4).
Corollary 3.10
Assume that
\(N>p\)
and functions
\(V_{1}\)
and
f
satisfy
\((V_{1})\)-\((V_{2})\), \((f_{1})^{*}\), \((AR)\)
and
-
\((f_{2})'\)
:
-
$$\lim_{ \vert u \vert \rightarrow0}\frac{f(x,u)}{ \vert u \vert ^{p-1}}=0,\qquad \lim_{ \vert u \vert \rightarrow\infty} \frac{f(x,u)}{ \vert u \vert ^{p^{*}-1}}=0 $$
uniformly in
\(x\in\mathbb{R}^{N}\).
Then equation (1.4) has a ground state in
\(W^{1,p}(\mathbb{R}^{N})\).
Corollary 3.11
Assume that
\(N>p\)
and functions
\(V_{1}\)
and
f
satisfy
\((V_{1})\)-\((V_{2})\), \((f_{1})^{*}\), \((f_{2})'\)
and
-
\((f_{4})'\)
:
-
$$\lim_{ \vert u \vert \rightarrow\infty}\frac{F(x,u)}{ \vert u \vert ^{p}}=+\infty $$
uniformly in
\(x\in\mathbb{R}^{N}\);
-
\((f_{5})'\)
:
-
\(\overline{F}(x,u)>0\)
for all
\(u\neq0\)
and there exists
\(k>\frac{N}{p}\)
such that
$$\limsup_{ \vert (u,v) \vert \rightarrow\infty} \biggl(\frac{F(x,u)}{ \vert u \vert ^{p}} \biggr)^{k} \frac{1}{\overline{F}(x,u)}< \infty, $$
where
$$\overline{F}(x,u)=u f(x,u)-pF(x,u,v). $$
Then equation (1.4) has a ground state in
\(W^{1,p}(\mathbb{R}^{N})\).
Remark 3.12
If the subcritical growth condition (3.2) in \((f_{2})'\) is replaced by (3.3), Corollary 3.10 becomes a corollary of Corollary 3.11 based on the fact that \((AR)\) and (3.3) imply \((f_{4})'\) and \((f_{5})'\) (see [3, 4]). However, we are not sure whether \((AR)\) and (3.2) imply \((f_{4})'\) and \((f_{5})'\) so that we do not know whether Corollary 3.10 is a corollary of Corollary 3.11. It is remarkable that our Corollaries 3.10 and 3.11 are different from Theorem 1.1 in [5] because there are examples satisfying \((AR)\) and \((f_{5})'\) but not satisfying \((f_{3})\) (see example (1.5) for \(p=2\)).
Next, we start to present our proofs. By \((\phi_{1})\) and \((\phi_{2})\), we define the space \(W:=W^{1, \Phi_{1}}(\mathbb{R}^{N})\times W^{1, \Phi _{2}}(\mathbb{R}^{N})\) with the norm
$$\bigl\Vert (u, v) \bigr\Vert = \Vert u \Vert _{1,\Phi_{1}}+ \Vert v \Vert _{1,\Phi_{2}}= \Vert \nabla u \Vert _{\Phi_{1}}+ \Vert u \Vert _{\Phi_{1}}+ \Vert \nabla v \Vert _{\Phi_{2}}+ \Vert v \Vert _{\Phi_{2}}. $$
Then W is a separable and reflexive Banach space by Remark 2.6.
On W, define a functional I by
$$\begin{aligned} I(u,v):={}& \int_{\mathbb{R}^{N}}\Phi_{1}\bigl( \vert \nabla u \vert \bigr)\,dx+ \int_{\mathbb {R}^{N}}V_{1}(x)\Phi_{1}\bigl( \vert u \vert \bigr)\,dx \\ & {}+ \int_{\mathbb{R}^{N}}\Phi_{2}\bigl( \vert \nabla v \vert \bigr)\,dx+ \int_{\mathbb {R}^{N}}V_{2}(x)\Phi_{2}\bigl( \vert v \vert \bigr)\,dx- \int_{\mathbb{R}^{N}}F(x,u,v)\,dx. \end{aligned}$$
(3.4)
Standard arguments show that I is well defined and of class \(C^{1}(W,\mathbb{R})\) and
$$\begin{aligned} \bigl\langle I'(u,v),(\tilde{u},\tilde{v})\bigr\rangle = {}& \int_{\mathbb {R}^{N}}a_{1}\bigl( \vert \nabla u \vert \bigr) \nabla u\nabla\tilde{u}\,dx + \int_{\mathbb{R}^{N}}V_{1}(x)a_{1}\bigl( \vert u \vert \bigr)u\tilde{u}\,dx \\ & {}+ \int_{\mathbb{R}^{N}}a_{2}\bigl( \vert \nabla v \vert \bigr) \nabla v\nabla\tilde{v}\,dx + \int_{\mathbb{R}^{N}}V_{2}(x)a_{2}\bigl( \vert v \vert \bigr)v\tilde{v}\,dx \\ & {}- \int_{\mathbb{R}^{N}}F_{u}(x,u,v)\tilde{u}\,dx- \int_{\mathbb {R}^{N}}F_{v}(x,u,v)\tilde{v}\,dx \end{aligned}$$
(3.5)
for all \((\tilde{u},\tilde{v})\in W\). For the sake of completeness, we give the proof in the Appendix. Thus, the critical points of I in W are weak solutions of system (1.1). Denote by \(I_{i}(i=1,2): W\rightarrow\mathbb{R}\) the functionals
$$\begin{aligned} I_{1}(u,v)={}& \int_{\mathbb{R}^{N}}\Phi_{1}\bigl( \vert \nabla u \vert \bigr)\,dx+ \int_{\mathbb {R}^{N}}V_{1}(x)\Phi_{1}\bigl( \vert u \vert \bigr)\,dx + \int_{\mathbb{R}^{N}}\Phi_{2}\bigl( \vert \nabla v \vert \bigr)\,dx \\ &{}+ \int_{\mathbb{R}^{N}}V_{2}(x)\Phi_{2}\bigl( \vert v \vert \bigr)\,dx \end{aligned}$$
(3.6)
and
$$\begin{aligned} I_{2}(u,v)= \int_{\mathbb{R}^{N}}F(x,u,v)\,dx. \end{aligned}$$
(3.7)
Then
$$I(u,v)=I_{1}(u,v)-I_{2}(u,v). $$
Lemma 3.13
If
\((F_{1})\)
and
\((F_{2})\)
hold, then there exist positive constants
\(C_{i}\)
\((i=1,2,3)\)
such that
$$\begin{aligned} & \bigl\vert F_{u}(x,u,v) \bigr\vert \leq C_{1} \bigl(\phi_{1}\bigl( \vert u \vert \bigr)+{ \widetilde{\Phi}_{1}}^{-1}\bigl(\Phi _{2}\bigl( \vert v \vert \bigr)\bigr)+\Phi_{1*}'\bigl( \vert u \vert \bigr)+{\widetilde{\Phi}_{1*}}^{-1}\bigl(\Phi _{2*}\bigl( \vert v \vert \bigr)\bigr) \bigr), \end{aligned}$$
(3.8)
$$\begin{aligned} &\bigl\vert F_{v}(x,u,v) \bigr\vert \leq C_{2} \bigl({\widetilde{\Phi}_{2}}^{-1}\bigl( \Phi_{1}\bigl( \vert u \vert \bigr)\bigr)+\phi _{2}\bigl( \vert v \vert \bigr)+{\widetilde{\Phi}_{2*}}^{-1}\bigl( \Phi_{1*}\bigl( \vert u \vert \bigr)\bigr)+\Phi _{2*}'\bigl( \vert v \vert \bigr) \bigr), \end{aligned}$$
(3.9)
$$\begin{aligned} & \bigl\vert F(x,u,v) \bigr\vert \leq C_{3} \bigl( \Phi_{1}\bigl( \vert u \vert \bigr)+\Phi_{2}\bigl( \vert v \vert \bigr)+\Phi_{1*}\bigl( \vert u \vert \bigr)+\Phi _{1*}\bigl( \vert v \vert \bigr) \bigr) \end{aligned}$$
(3.10)
for all
\((x,u,v)\in\mathbb{R}^{N}\times\mathbb{R}\times\mathbb{R}\), and
$$\begin{aligned} \lim_{ \vert (u,v) \vert \rightarrow0}\frac{F(x,u,v)}{\Phi_{1}( \vert u \vert )+\Phi_{2}( \vert v \vert )}=0, \qquad \lim _{ \vert (u,v) \vert \rightarrow\infty}\frac{F(x,u,v)}{\Phi_{1*}( \vert u \vert )+\Phi_{2*}( \vert v \vert )}=0 \end{aligned}$$
(3.11)
uniformly in
\(x\in\mathbb{R}^{N}\).
Proof
The proof can be easily completed by virtue of Young’s inequality (2.1) and the fact
$$F(x,u,v)= \int_{0}^{u}F_{s}(x,s,v)\,ds+ \int_{0}^{v}F_{t}(x,0,t)\,dt+F(x,0,0)\quad \mbox{for all } (x,u,v)\in\mathbb{R}^{N}\times\mathbb{R}\times \mathbb{R}. $$
We omit the details. □
Notation
\(C_{a}\) denotes a positive constant which depends on the real number a.
Lemma 3.14
Assume that
\((\phi_{1})\), \((\phi_{2})\), \((V_{2})\), \((F_{1})\)
and
\((F_{2})\)
hold. Then there exist two positive constants
\(\rho, \eta\)
such that
\(I(u,v)\geq\eta\)
for all
\((u,v)\in W\)
with
\(\Vert (u,v) \Vert =\rho\).
Proof
By (3.11), for any given \(\varepsilon\in (0,\alpha_{1})\), there exists a constant \(C_{\varepsilon}>0\) such that
$$\begin{aligned} \bigl\vert F(x,u,v) \bigr\vert \leq{}&\varepsilon\bigl(\Phi_{1}\bigl( \vert u \vert \bigr)+\Phi_{2}\bigl( \vert v \vert \bigr) \bigr)+C_{\varepsilon}\bigl(\Phi _{1*}\bigl( \vert u \vert \bigr)+\Phi_{2*}\bigl( \vert v \vert \bigr)\bigr) \\ &\mbox{for all } (x,u,v)\in\mathbb{R}^{N}\times \mathbb{R}\times\mathbb{R}. \end{aligned}$$
Then, by (3.4), \((V_{2})\), Lemma 2.2, (3) in Lemma 2.4, (2.8) and (2.7), when \(\Vert (u,v) \Vert = \Vert u \Vert _{1,\Phi_{1}}+ \Vert v \Vert _{1,\Phi _{2}}= \Vert \nabla u \Vert _{\Phi_{1}}+ \Vert u \Vert _{\Phi_{1}}+ \Vert \nabla v \Vert _{\Phi_{2}}+ \Vert v \Vert _{\Phi_{2}}\leq1\), we have
$$\begin{aligned} I(u,v)\geq{}& \int_{\mathbb{R}^{N}}\Phi_{1}\bigl( \vert \nabla u \vert \bigr)\,dx+ \int_{\mathbb {R}^{N}}V_{1}(x)\Phi_{1}\bigl( \vert u \vert \bigr)\,dx \\ & {} + \int_{\mathbb{R}^{N}}\Phi_{2}\bigl( \vert \nabla v \vert \bigr)\,dx+ \int_{\mathbb {R}^{N}}V_{2}(x)\Phi_{2}\bigl( \vert v \vert \bigr)\,dx- \int_{\mathbb{R}^{N}} \bigl\vert F(x,u,v) \bigr\vert \,dx \\ \geq{}& \int_{\mathbb{R}^{N}}\Phi_{1}\bigl( \vert \nabla u \vert \bigr)\,dx+\alpha_{1} \int_{\mathbb {R}^{N}}\Phi_{1}\bigl( \vert u \vert \bigr)\,dx + \int_{\mathbb{R}^{N}}\Phi_{2}\bigl( \vert \nabla v \vert \bigr)\,dx+\alpha_{1} \int_{\mathbb {R}^{N}}\Phi_{2}\bigl( \vert v \vert \bigr)\,dx \\ & {} -\varepsilon \int_{\mathbb{R}^{N}}\Phi_{1}\bigl( \vert u \vert \bigr)\,dx- \varepsilon \int _{\mathbb{R}^{N}}\Phi_{2}\bigl( \vert v \vert \bigr)\,dx\\ &{}-C_{\varepsilon} \int_{\mathbb{R}^{N}}\Phi_{1*}\bigl( \vert u \vert \bigr)\,dx-C_{\varepsilon} \int _{\mathbb{R}^{N}}\Phi_{2*}\bigl( \vert v \vert \bigr)\,dx \\ \geq{}& \Vert \nabla u \Vert _{\Phi_{1}}^{m_{1}}+( \alpha_{1}-\varepsilon) \Vert u \Vert _{\Phi _{1}}^{m_{1}}+ \Vert \nabla v \Vert _{\Phi_{2}}^{m_{2}}+(\alpha_{1}- \varepsilon) \Vert v \Vert _{\Phi _{2}}^{m_{2}} \\ & {} -C_{\varepsilon}\max\bigl\{ \Vert u \Vert _{\Phi_{1*}}^{l_{1}^{*}}, \Vert u \Vert _{\Phi _{1*}}^{m_{1}^{*}}\bigr\} -C_{\varepsilon}\max \bigl\{ \Vert v \Vert _{\Phi_{2*}}^{l_{2}^{*}}, \Vert v \Vert _{\Phi _{2*}}^{m_{2}^{*}}\bigr\} \\ \geq{}& \min\{1,\alpha_{1}-\varepsilon\}C_{m_{1}} \Vert u \Vert _{1,\Phi _{1}}^{m_{1}}+\min\{1,\alpha_{1}-\varepsilon \}C_{m_{2}} \Vert v \Vert _{1,\Phi _{2}}^{m_{2}} \\ & {}-C_{\varepsilon}C_{\Phi_{1*}}^{l_{1}^{*}} \Vert u \Vert _{1,\Phi _{1}}^{l_{1}^{*}}-C_{\varepsilon}C_{\Phi_{1*}}^{m_{1}^{*}} \Vert u \Vert _{1,\Phi_{1}}^{m_{1}^{*}} -C_{\varepsilon}C_{\Phi_{2*}}^{l_{2}^{*}} \Vert v \Vert _{1,\Phi _{2}}^{l_{2}^{*}}-C_{\varepsilon}C_{\Phi_{2*}}^{m_{2}^{*}} \Vert v \Vert _{1,\Phi_{2}}^{m_{2}^{*}}. \end{aligned}$$
Note that \(m_{i}< l_{i}^{*}\leq m_{i}^{*} (i=1,2)\). It is easy to see that the foregoing inequality implies that there exist positive constants ρ and η small enough such that \(I(u,v)\geq\eta\) for all \((u,v)\in W\) with \(\Vert (u,v) \Vert =\rho\). □
Lemma 3.15
Assume that
\((\phi_{1})\), \((\phi_{2})\), \((V_{2})\), \((F_{1})\)
and
\((F_{3})\) (or
\((F_{4})\)) hold. Then there exists
\((u_{0},v_{0})\in W\)
such that
\(I(tu_{0},tv_{0})\rightarrow-\infty\)
as
\(t\rightarrow+\infty\).
Proof
First, we prove that under assumptions \((F_{1})\) and \((F_{3})\) (or \((F_{4})\)), for any given constant \(M>\alpha_{2}\), there exists a constant \(C_{M}>0\) such that
$$\begin{aligned} F(x,u,0)\geq M\Phi_{1}\bigl( \vert u \vert \bigr)-C_{M} \quad\mbox{for all } (x,u)\in\mathbb {R}^{N}\times \mathbb{R}. \end{aligned}$$
(3.12)
In fact, it is obvious by \((F_{1})\) and \((F_{4})\). Let \(v=0\) in \((F_{3})\). Then \((F_{3})\) reduces to
$$0< F(x,u,0)\leq\frac{1}{\mu_{1}}uF_{u}(x,u,0)\quad \mbox{for all } u \neq0, $$
where \(\mu_{1}>m_{1}\), which implies that \(F(x,u,0)\geq C( \vert u \vert ^{\mu_{1}}-1)\) for some \(C>0\) and all \((x,u)\in\mathbb{R}^{N}\times\mathbb{R}\). Moreover, it follows from (2) in Lemma 2.1 that \(\Phi_{1}( \vert u \vert )\leq\Phi _{1}(1)\max\{ \vert u \vert ^{l_{1}}, \vert u \vert ^{m_{1}}\}\) for all \(u\in\mathbb{R}\). Since \(\mu _{1}>m_{1}\), then for any given constant \(M>\alpha_{2}\), there exists a constant \(C_{M}>0\) such that (3.12) holds.
Now, choose \(u_{0}\in C_{0}^{\infty}(B_{r})\setminus\{\mathbf{0}\}\) with \(0\leq u_{0}(x)\leq1\), where \(r>0\). Then \((u_{0},0)\in W\), and by (3.4), \((V_{2})\), \((F_{1})\), (3.12) and (2) in Lemma 2.1, when \(t>0\), we have
$$\begin{aligned} I(tu_{0},\mathbf{0})= {}& \int_{\mathbb{R}^{N}}\Phi_{1}\bigl( \vert t\nabla u_{0} \vert \bigr)\,dx+ \int _{\mathbb{R}^{N}}V_{1}(x)\Phi_{1}\bigl( \vert tu_{0} \vert \bigr)\,dx- \int_{\mathbb{R}^{N}}F(x,tu_{0},0)\,dx \\ ={} & \int_{\mathbb{R}^{N}}\Phi_{1}\bigl( \vert t\nabla u_{0} \vert \bigr)\,dx+ \int_{B_{r}}V_{1}(x)\Phi _{1}\bigl( \vert tu_{0} \vert \bigr)\,dx- \int_{B_{r}}F(x,tu_{0},0)\,dx \\ \leq{}& \int_{\mathbb{R}^{N}}\Phi_{1}\bigl( \vert t\nabla u_{0} \vert \bigr)\,dx+\alpha_{2} \int_{B_{r}}\Phi _{1}\bigl( \vert tu_{0} \vert \bigr)\,dx-M \int_{B_{r}}\Phi_{1}\bigl( \vert tu_{0} \vert \bigr)+C_{M} \vert B_{r} \vert \\ \leq{}&\Phi_{1}(t) \int_{\mathbb{R}^{N}}\max\bigl\{ \vert \nabla u_{0} \vert ^{l_{1}}, \vert \nabla u_{0} \vert ^{m_{1}}\bigr\} \,dx\\ &{}-\Phi_{1}(t) (M-\alpha_{2}) \int_{B_{r}}\min\bigl\{ \vert u_{0} \vert ^{l_{1}}, \vert u_{0} \vert ^{m_{1}}\bigr\} \,dx+C_{M} \vert B_{r} \vert \\ \leq{}&\Phi_{1}(t) \bigl[ \bigl\Vert \vert \nabla u_{0} \vert \bigr\Vert _{l_{1}}^{l_{1}}+ \bigl\Vert \vert \nabla u_{0} \vert \bigr\Vert _{m_{1}}^{m_{1}}-(M- \alpha_{2}) \Vert u_{0} \Vert _{m_{1}}^{m_{1}} \bigr]+C_{M} \vert B_{r} \vert . \end{aligned}$$
Since \(\lim_{t\rightarrow+\infty}\Phi_{1}(t)=+\infty\), we can choose \(M>\frac{ \Vert \vert \nabla u_{0} \vert \Vert _{l_{1}}^{l_{1}}+ \Vert \vert \nabla u_{0} \vert \Vert _{m_{1}}^{m_{1}}}{ \Vert u_{0} \Vert _{m_{1}}^{m_{1}}}+\alpha_{2}\) such that \(I(tu_{0},\mathbf{0})\rightarrow -\infty\) as \(t\rightarrow+\infty\). □
Lemmas 3.14, 3.15 and the fact \(I(\mathbf{0,0})=0\) show that I has a mountain pass geometry, that is, setting
$$\Gamma=\bigl\{ \gamma\in C\bigl([0,1],W\bigr):\gamma(0)=\mathbf{0} \mbox{ and } I \bigl(\gamma (1)\bigr)< 0\bigr\} , $$
we have \(\Gamma\neq\emptyset\). By a special version of the mountain pass lemma (see [23]), for the mountain pass level
$$\begin{aligned} c=\inf_{\gamma\in\Gamma}\max_{t\in[0,1]}I \bigl(\gamma(t)\bigr), \end{aligned}$$
(3.13)
there exists a \((C)_{c}\)-sequence \(\{(u_{n},v_{n})\}\) of I in W. Moreover, Lemma 3.14 implies that \(c>0\). We recall that \((C)_{c}\)-sequence \(\{u_{n}, v_{n}\}\) of I in W means
$$\begin{aligned} I(u_{n},v_{n})\rightarrow c \quad\mbox{and}\quad \bigl(1+ \bigl\Vert (u_{n},v_{n}) \bigr\Vert \bigr) \bigl\Vert I'(u_{n},v_{n}) \bigr\Vert _{W^{*}}\rightarrow0 \quad\mbox{as } n\rightarrow\infty. \end{aligned}$$
(3.14)
Lemma 3.16
Assume that
\((\phi_{1})\), \((\phi_{2})\), \((V_{2})\), \((F_{1})\)-\((F_{3})\)
hold. Then any
\((C)_{c}\)-sequence of
I
in
W
is bounded for all
\(c\geq0\).
Proof
Let \(\{(u_{n},v_{n})\}\) be a \((C)_{c}\)-sequence of I in W for \(c\geq0\). By (3.14), we have
$$\begin{aligned} I(u_{n},v_{n})\rightarrow c \quad\mbox{as } n \rightarrow\infty \end{aligned}$$
(3.15)
and
$$\begin{aligned} \bigl\Vert I'(u_{n},v_{n}) \bigr\Vert _{W^{*}} \bigl\Vert (u_{n},v_{n}) \bigr\Vert = \bigl\Vert I'(u_{n},v_{n}) \bigr\Vert _{W^{*}}\bigl( \Vert u_{n} \Vert _{1,\Phi_{1}}+ \Vert v_{n} \Vert _{1,\Phi_{2}}\bigr)\rightarrow0 \quad\mbox{as } n \rightarrow \infty, \end{aligned}$$
which implies
$$\begin{aligned} \begin{aligned} &\biggl\vert \biggl\langle I'(u_{n},v_{n}), \biggl(\frac{1}{\mu_{1}}u_{n},\frac{1}{\mu _{2}}v_{n} \biggr) \biggr\rangle \biggr\vert \leq \bigl\Vert I'(u_{n},v_{n}) \bigr\Vert _{W^{*}} \biggl(\frac{1}{\mu_{1}} \Vert u_{n} \Vert _{1,\Phi_{1}}+\frac {1}{\mu_{2}} \Vert v_{n} \Vert _{1,\Phi_{2}} \biggr)\rightarrow0 \\ &\quad\mbox{as } n\rightarrow\infty. \end{aligned} \end{aligned}$$
(3.16)
Then, by (3.15), (3.16), (3.4), (3.5), \((\phi _{2})\), \((V_{2})\), \((F_{3})\) and Lemma 2.2, for n large, we have
$$\begin{aligned} c+1 \geq{}&I(u_{n},v_{n})- \biggl\langle I'(u_{n},v_{n}), \biggl( \frac{1}{\mu _{1}}u_{n},\frac{1}{\mu_{2}}v_{n} \biggr) \biggr\rangle \\ = {}& \int_{\mathbb{R}^{N}} \biggl(\Phi_{1}\bigl( \vert \nabla u_{n} \vert \bigr)-\frac{1}{\mu _{1}}a_{1}\bigl( \vert \nabla u_{n} \vert \bigr) \vert \nabla u_{n} \vert ^{2} \biggr)\,dx \\ &{}+ \int_{\mathbb{R}^{N}}V_{1}(x) \biggl(\Phi_{1}\bigl( \vert u_{n} \vert \bigr)-\frac{1}{\mu _{1}}a_{1}\bigl( \vert u_{n} \vert \bigr) \vert u_{n} \vert ^{2} \biggr)\,dx \\ & {}+ \int_{\mathbb{R}^{N}} \biggl(\Phi_{2}\bigl( \vert \nabla v_{n} \vert \bigr)-\frac{1}{\mu _{2}}a_{2}\bigl( \vert \nabla v_{n} \vert \bigr) \vert \nabla v_{n} \vert ^{2} \biggr)\,dx \\ &{}+ \int_{\mathbb{R}^{N}}V_{2}(x) \biggl(\Phi_{2}\bigl( \vert v_{n} \vert \bigr)-\frac{1}{\mu _{2}}a_{2}\bigl( \vert v_{n} \vert \bigr) \vert v_{n} \vert ^{2} \biggr)\,dx \\ & {} + \int_{\mathbb{R}^{N}} \biggl(\frac{1}{\mu_{1}}u_{n}F_{u}(x,u_{n},v_{n})+ \frac {1}{\mu_{2}}v_{n}F_{v}(x,u_{n},v_{n})-F(x,u_{n},v_{n}) \biggr)\,dx \\ \geq{}& \biggl(1-\frac{m_{1}}{\mu_{1}} \biggr) \int_{\mathbb{R}^{N}}\Phi _{1}\bigl( \vert \nabla u_{n} \vert \bigr)\,dx+ \biggl(1-\frac{m_{1}}{\mu_{1}} \biggr) \alpha_{1} \int_{\mathbb {R}^{N}}\Phi_{1}\bigl( \vert u_{n} \vert \bigr)\,dx \\ & {} + \biggl(1-\frac{m_{2}}{\mu_{2}} \biggr) \int_{\mathbb{R}^{N}}\Phi_{2}\bigl( \vert \nabla v_{n} \vert \bigr)\,dx+ \biggl(1-\frac{m_{2}}{\mu_{2}} \biggr) \alpha_{1} \int_{\mathbb{R}^{N}}\Phi _{2}\bigl( \vert v_{n} \vert \bigr)\,dx \\ \geq{}& \biggl(1-\frac{m_{1}}{\mu_{1}} \biggr)\min\bigl\{ \Vert \nabla u_{n} \Vert _{\Phi _{1}}^{l_{1}}, \Vert \nabla u_{n} \Vert _{\Phi_{1}}^{m_{1}}\bigr\} + \biggl(1- \frac{m_{1}}{\mu_{1}} \biggr)\alpha_{1}\min\bigl\{ \Vert u_{n} \Vert _{\Phi _{1}}^{l_{1}}, \Vert u_{n} \Vert _{\Phi_{1}}^{m_{1}}\bigr\} \\ & {} + \biggl(1-\frac{m_{2}}{\mu_{2}} \biggr)\min\bigl\{ \Vert \nabla v_{n} \Vert _{\Phi _{2}}^{l_{2}}, \Vert \nabla v_{n} \Vert _{\Phi_{2}}^{m_{2}}\bigr\} + \biggl(1- \frac{m_{2}}{\mu_{2}} \biggr)\alpha_{1}\min\bigl\{ \Vert v_{n} \Vert _{\Phi _{2}}^{l_{2}}, \Vert v_{n} \Vert _{\Phi_{2}}^{m_{2}}\bigr\} , \end{aligned}$$
which implies that \(\Vert (u_{n}, v_{n}) \Vert = \Vert \nabla u_{n} \Vert _{\Phi_{1}}+ \Vert u_{n} \Vert _{\Phi _{1}}+ \Vert \nabla v_{n} \Vert _{\Phi_{2}}+ \Vert v_{n} \Vert _{\Phi_{2}}\leq C\) for some \(C>0\), that is, \(\{(u_{n},v_{n})\}\) is bounded in W. □
Lemma 3.17
Assume that
\((\phi_{1})\)-\((\phi_{3})\), \((V_{1})\), \((V_{2})\), \((F_{1})\), \((F_{2})\), \((F_{4})\)
and
\((F_{5})\)
hold. Then any
\((C)_{c}\)-sequence of
I
in
W
is bounded for all
\(c\geq0\).
Proof
Let \(\{(u_{n},v_{n})\}\) be a \((C)_{c}\)-sequence of I in W for \(c\geq0\). By (3.14), we have
$$\begin{aligned} \begin{aligned} &I(u_{n},v_{n})=I_{1}(u_{n},v_{n})-I_{2}(u_{n},v_{n}) \rightarrow c\quad\mbox{and}\quad \biggl\vert \biggl\langle I'(u_{n},v_{n}), \biggl(\frac{1}{m_{1}}u_{n},\frac {1}{m_{2}}v_{n} \biggr) \biggr\rangle \biggr\vert \rightarrow0\\ &\quad \mbox{as } n\rightarrow\infty. \end{aligned} \end{aligned}$$
(3.17)
Then, by (3.4), (3.5), \((\phi_{2})\) and \((V_{2})\), for n large, we have
$$\begin{aligned} c+1 \geq{}& I(u_{n},v_{n})- \biggl\langle I'(u_{n},v_{n}), \biggl( \frac {1}{m_{1}}u_{n},\frac{1}{m_{2}}v_{n} \biggr) \biggr\rangle \\ = {}& \int_{\mathbb{R}^{N}} \biggl(\Phi_{1}\bigl( \vert \nabla u_{n} \vert \bigr)-\frac {1}{m_{1}}a_{1}\bigl( \vert \nabla u_{n} \vert \bigr) \vert \nabla u_{n} \vert ^{2} \biggr)\,dx \\ &{}+ \int_{\mathbb{R}^{N}}V_{1}(x) \biggl(\Phi_{1}\bigl( \vert u_{n} \vert \bigr)-\frac {1}{m_{1}}a_{1}\bigl( \vert u_{n} \vert \bigr) \vert u_{n} \vert ^{2} \biggr)\,dx \\ & {}+ \int_{\mathbb{R}^{N}} \biggl(\Phi_{2}\bigl( \vert \nabla v_{n} \vert \bigr)-\frac {1}{m_{2}}a_{2}\bigl( \vert \nabla v_{n} \vert \bigr) \vert \nabla v_{n} \vert ^{2} \biggr)\,dx \\ &{}+ \int_{\mathbb{R}^{N}}V_{2}(x) \biggl(\Phi_{2}\bigl( \vert v_{n} \vert \bigr)-\frac {1}{m_{2}}a_{2}\bigl( \vert v_{n} \vert \bigr) \vert v_{n} \vert ^{2} \biggr)\,dx \\ & {} + \int_{\mathbb{R}^{N}} \biggl(\frac{1}{m_{1}}u_{n}F_{u}(x,u_{n},v_{n})+ \frac {1}{m_{2}}v_{n}F_{v}(x,u_{n},v_{n})-F(x,u_{n},v_{n}) \biggr)\,dx \\ \geq{}& \int_{\mathbb{R}^{N}}\overline{F}(x,u_{n},v_{n})\,dx. \end{aligned}$$
(3.18)
To prove the boundedness of \(\{(u_{n},v_{n})\}\), arguing by contradiction, we suppose that there exists a subsequence of \(\{(u_{n},v_{n})\}\), still denoted by \(\{(u_{n},v_{n})\}\), such that \(\Vert (u_{n},v_{n}) \Vert = \Vert u_{n} \Vert _{1,\Phi _{1}}+ \Vert v_{n} \Vert _{1,\Phi_{2}}\rightarrow\infty\). Next, we discuss the problem in three cases.
Case 1. Suppose that \(\Vert u_{n} \Vert _{1,\Phi_{1}}\rightarrow\infty\) and \(\Vert v_{n} \Vert _{1,\Phi_{2}}\rightarrow\infty\). Let \(\tilde{u}_{n}=\frac{u_{n}}{ \Vert u_{n} \Vert _{1,\Phi_{1}}}\) and \(\tilde{v}_{n}=\frac{v_{n}}{ \Vert v_{n} \Vert _{1,\Phi_{2}}}\). Then \(\{{\tilde{u}_{n}}\}\) and \(\{{\tilde{v}_{n}}\}\) are bounded in \(W^{1,\Phi_{1}}(\mathbb{R}^{N})\) and \(W^{1,\Phi_{2}}(\mathbb{R}^{N})\), respectively. We claim that
$$\begin{aligned} \lambda_{1}:=\lim_{n\rightarrow\infty}\sup_{y\in\mathbb{R}^{N}} \int _{B_{2}(y)}\bigl(\Phi_{1}\bigl( \vert \tilde{u}_{n} \vert \bigr)+\Phi_{2}\bigl( \vert \tilde{v}_{n} \vert \bigr)\bigr)\,dx=0. \end{aligned}$$
Indeed, if \(\lambda_{1}\neq0\), there exist a constant \(\delta>0\), a subsequence of \(\{(\tilde{u}_{n},\tilde{v}_{n})\}\), still denoted by \(\{ (\tilde{u}_{n},\tilde{v}_{n})\}\), and a sequence \(\{z_{n}\}\in\mathbb{Z}^{N}\) such that
$$\begin{aligned} \int_{B_{2}(z_{n})}\bigl(\Phi_{1}\bigl( \vert \tilde{u}_{n} \vert \bigr)+\Phi_{2}\bigl( \vert \tilde{v}_{n} \vert \bigr)\bigr)\,dx>\delta \quad\mbox{for all } n\in \mathbb{N}. \end{aligned}$$
(3.19)
Let \(\bar{u}_{n}=\tilde{u}_{n}(\cdot+z_{n})\) and \(\bar{v}_{n}=\tilde {v}_{n}(\cdot+z_{n})\). Then \(\Vert \bar{u}_{n} \Vert _{1,\Phi_{1}}= \Vert \tilde{u}_{n} \Vert _{1,\Phi_{1}}\) and \(\Vert \bar{v}_{n} \Vert _{1,\Phi_{2}}= \Vert \tilde{v}_{n} \Vert _{1,\Phi_{2}}\), that is, \(\{{\bar{u}_{n}}\}\) and \(\{{\bar{v}_{n}}\}\) are bounded in \(W^{1,\Phi_{1}}(\mathbb{R}^{N})\) and \(W^{1,\Phi_{2}}(\mathbb{R}^{N})\), respectively. Passing to a subsequence of \(\{({\bar{u}_{n}},{\bar{v}_{n}})\} \), still denoted by \(\{({\bar{u}_{n}},{\bar{v}_{n}})\}\), by Remark 2.7, there exists \((\bar{u},\bar{v})\in W\) such that
- ⋆:
-
\(\bar{u}_{n}\rightharpoonup\bar{u}\) in \(W^{1,\Phi_{1}}(\mathbb{R}^{N})\), \(\bar{u}_{n}\rightarrow\bar{u}\) in \(L^{\Phi_{1}}(B_{2})\) and \(\bar{u}_{n}(x)\rightarrow\bar{u}(x)\) a.e. in \(B_{2}\);
- ⋆:
-
\(\bar{v}_{n}\rightharpoonup\bar{v}\) in \(W^{1,\Phi_{2}}(\mathbb{R}^{N})\), \(\bar{v}_{n}\rightarrow\bar{v}\) in \(L^{\Phi_{2}}(B_{2})\) and \(\bar{v}_{n}(x)\rightarrow\bar{v}(x)\) a.e. in \(B_{2}\).
Since
$$\begin{aligned} \int_{B_{2}}\bigl(\Phi_{1}\bigl( \vert \bar{u}_{n} \vert \bigr)+\Phi_{2}\bigl( \vert \bar{v}_{n} \vert \bigr)\bigr)\,dx= \int _{B_{2}(z_{n})}\bigl(\Phi_{1}\bigl( \vert \tilde{u}_{n} \vert \bigr)+\Phi_{2}\bigl( \vert \tilde{v}_{n} \vert \bigr)\bigr)\,dx, \end{aligned}$$
then, by (3.19), ⋆ and (2.3), we obtain that \(\bar {u}\neq\mathbf{0}\) in \(L^{\Phi_{1}}(B_{2})\) or \(\bar{v}\neq\mathbf{0}\) in \(L^{\Phi_{2}}(B_{2})\). Without loss of generality, we can assume that \(\bar {u}\neq\mathbf{0}\) in \(L^{\Phi_{1}}(B_{2})\), that is, \([\bar{u}\neq0]:=\{x \in B_{2}: \bar{u}(x)\neq0\}\) has nonzero Lebesgue measure. Let \(u_{n}^{*}=u_{n}(\cdot+z_{n})\) and \(v_{n}^{*}=v_{n}(\cdot+z_{n})\). Then \(\Vert (u_{n}^{*},v_{n}^{*}) \Vert = \Vert (u_{n},v_{n}) \Vert \), and it follows from that fact that \(V_{i}(i=1,2)\) and F are 1-periodic in x that
$$\begin{aligned} I\bigl(u_{n}^{*},v_{n}^{*}\bigr)=I(u_{n},v_{n})\quad \mbox{and} \quad\bigl\Vert I'\bigl(u_{n}^{*},v_{n}^{*} \bigr) \bigr\Vert _{W^{*}}= \bigl\Vert I'(u_{n},v_{n}) \bigr\Vert _{W^{*}} \quad\mbox{for all } n\in\mathbb{N}, \end{aligned}$$
that is, \(\{(u_{n}^{*},v_{n}^{*})\}\) is also a \((C)_{c}\)-sequence of I. Then, by (3.18), for n large, we have
$$\begin{aligned} \int_{\mathbb{R}^{N}}\overline{F}\bigl(x,u_{n}^{*},v_{n}^{*} \bigr)\,dx\leq c+1. \end{aligned}$$
(3.20)
However, by (2) in Lemma 2.1, \((F_{4})\) and \((F_{5})\) imply
$$\begin{aligned} \lim_{ \vert (u,v) \vert \rightarrow\infty}\overline{F}(x,u,v)=+\infty\quad \mbox{uniformly in } x\in\mathbb{R}^{N}, \end{aligned}$$
(3.21)
and by ⋆, \(\bar{u}_{n}=\tilde{u}_{n}(\cdot+z_{n})=\frac{u_{n}(\cdot +z_{n})}{ \Vert u_{n} \Vert _{1,\Phi_{1}}}=\frac{u_{n}^{*}}{ \Vert u_{n} \Vert _{1,\Phi_{1}}}\) implies
$$\begin{aligned} \bigl\vert u_{n}^{*}(x) \bigr\vert = \bigl\vert \bar{u}_{n}(x) \bigr\vert \Vert u_{n} \Vert _{1,\Phi_{1}}\rightarrow\infty, \quad\mbox{a.e. } x\in[\bar{u}\neq0]. \end{aligned}$$
(3.22)
Then, it follows from \((F_{5})\), (3.21), (3.22) and Fatou’s lemma that
$$\begin{aligned} \int_{\mathbb{R}^{N}}\overline{F}\bigl(x,u_{n}^{*},v_{n}^{*} \bigr)\,dx\geq \int_{[\bar{u}\neq 0]}\overline{F}\bigl(x,u_{n}^{*},v_{n}^{*} \bigr)\,dx\rightarrow+\infty, \end{aligned}$$
which contradicts (3.20). Therefore, \(\lambda_{1}=0\) and
$$\begin{aligned} \lim_{n\rightarrow\infty}\sup_{y\in\mathbb{R}^{N}} \int_{B_{2}(y)}\Phi _{1}\bigl( \vert \tilde{u}_{n} \vert \bigr)\,dx =\lim_{n\rightarrow\infty}\sup _{y\in\mathbb{R}^{N}} \int_{B_{2}(y)}\Phi _{2}\bigl( \vert \tilde{v}_{n} \vert \bigr)\,dx=0. \end{aligned}$$
(3.23)
By Lemma 2.5, \((\phi_{3})\) and the fact that
$$\limsup_{t\rightarrow+\infty}\frac{t^{l_{i}}}{\Phi_{i*}(t)}\leq\limsup _{t\rightarrow+\infty}\frac{t^{l_{i}}}{\Phi_{i*}(1)\min\{ t^{l_{i}^{*}},t^{m_{i}^{*}}\}}=0, \quad i=1,2, $$
imply that the embeddings \(W^{1, \Phi_{i}}(\mathbb{R}^{N})\hookrightarrow L^{l_{i}}(\mathbb{R}^{N})(i=1,2)\) are continuous. Hence, there exists a constant \(M_{1}>0\) such that
$$\begin{aligned} \Vert \tilde{u}_{n} \Vert _{l_{1}}^{l_{1}}+ \Vert \tilde{v}_{n} \Vert _{l_{2}}^{l_{2}}\leq M_{1} \quad\mbox{for all } n\in\mathbb{N}. \end{aligned}$$
(3.24)
For \(p_{i}\in(l_{i},l_{i}^{*})(i=1,2)\), by \((\phi_{2})\) and \((\phi_{3})\), we have
$$\begin{aligned} \lim_{t\rightarrow0^{+}}\frac{t^{p_{i}}}{\Phi_{i}(t)}=0 \quad\mbox{and}\quad \lim _{t\rightarrow+\infty}\frac{t^{p_{i}}}{\Phi_{i*}(t)}\leq\lim_{t\rightarrow +\infty} \frac{t^{p_{i}}}{\Phi_{i*}(1)\min\{t^{l_{i}^{*}},t^{m_{i}^{*}}\}}=0,\quad i=1,2. \end{aligned}$$
(3.25)
Then, by the Lions type result for Orlicz-Sobolev spaces (see Theorem 1.3 in [8]), (3.23) and (3.25) imply that
$$\begin{aligned} \begin{aligned} &\tilde{u}_{n}\rightarrow\mathbf{0} \quad\mbox{in } L^{p_{1}}\bigl(\mathbb{R}^{N}\bigr) \quad\mbox{and}\\ & \tilde{v}_{n}\rightarrow\mathbf{0} \quad\mbox{in } L^{p_{2}}\bigl( \mathbb{R}^{N}\bigr) \mbox{ for all } p_{1}\in \bigl(l_{1},l_{1}^{*}\bigr), p_{2}\in \bigl(l_{2},l_{2}^{*}\bigr). \end{aligned} \end{aligned}$$
(3.26)
Now, by (3.6), \((V_{2})\) and Lemma 2.2, we have
$$\begin{aligned} & \frac{I_{1}(u_{n},v_{n})}{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+ \Vert v_{n} \Vert _{1,\Phi _{2}}^{l_{2}}} \\ &\quad\geq \frac{\int_{\mathbb{R}^{N}}\Phi_{1}( \vert \nabla u_{n} \vert )\,dx+\alpha_{1}\int _{\mathbb{R}^{N}}\Phi_{1}( \vert u_{n} \vert )\,dx +\int_{\mathbb{R}^{N}}\Phi_{2}( \vert \nabla v_{n} \vert )\,dx+\alpha_{1}\int_{\mathbb {R}^{N}}\Phi_{2}( \vert v_{n} \vert )\,dx}{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+ \Vert v_{n} \Vert _{1,\Phi _{2}}^{l_{2}}} \\ &\quad\geq \frac{\min\{ \Vert \nabla u_{n} \Vert _{\Phi_{1}}^{l_{1}}, \Vert \nabla u_{n} \Vert _{\Phi _{1}}^{m_{1}}\}+\alpha_{1}\min\{ \Vert u_{n} \Vert _{\Phi_{1}}^{l_{1}}, \Vert u_{n} \Vert _{\Phi _{1}}^{m_{1}}\}}{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+ \Vert v_{n} \Vert _{1,\Phi_{2}}^{l_{2}}} \\ & \qquad{} +\frac{\min\{ \Vert \nabla v_{n} \Vert _{\Phi_{2}}^{l_{2}}, \Vert \nabla v_{n} \Vert _{\Phi _{2}}^{m_{2}}\}+\alpha_{1}\min\{ \Vert v_{n} \Vert _{\Phi_{2}}^{l_{2}}, \Vert v_{n} \Vert _{\Phi _{2}}^{m_{2}}\}}{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+ \Vert v_{n} \Vert _{1,\Phi_{2}}^{l_{2}}} \\ &\quad \geq \frac{ \Vert \nabla u_{n} \Vert _{\Phi_{1}}^{l_{1}}+\alpha_{1} \Vert u_{n} \Vert _{\Phi _{1}}^{l_{1}}+ \Vert \nabla v_{n} \Vert _{\Phi_{2}}^{l_{2}}+\alpha_{1} \Vert v_{n} \Vert _{\Phi _{2}}^{l_{2}}-2-2\alpha_{1}}{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+ \Vert v_{n} \Vert _{1,\Phi_{2}}^{l_{2}}} \\ &\quad \geq \frac{\min\{1,\alpha_{1}\}C_{l_{1}} \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+\min\{ 1,\alpha_{1}\}C_{l_{2}} \Vert v_{n} \Vert _{1,\Phi_{2}}^{l_{2}}-2-2\alpha_{1}}{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+ \Vert v_{n} \Vert _{1,\Phi_{2}}^{l_{2}}} \\ &\quad \geq \min\{1,\alpha_{1}\}\min\{C_{l_{1}},C_{l_{2}} \}+o_{n}(1). \end{aligned}$$
(3.27)
Moreover, (3.11) and (2) in Lemma 2.1 imply that
$$\begin{aligned} \lim_{ \vert (u,v) \vert \rightarrow0}\frac{F(x,u,v)}{ \vert u \vert ^{l_{1}}+ \vert v \vert ^{l_{2}}}=0 \end{aligned}$$
uniformly in \(x\in\mathbb{R}^{N}\). Then, for any given constant \(\varepsilon>0\), there exists a constant \(R_{\varepsilon}>0\) such that
$$\begin{aligned} \frac{ \vert F(x,u,v) \vert }{ \vert u \vert ^{l_{1}}+ \vert v \vert ^{l_{2}}}\leq\varepsilon \quad\mbox{for all } x\in \mathbb{R}^{N}, \bigl\vert (u,v) \bigr\vert \leq R_{\varepsilon}, \end{aligned}$$
(3.28)
and by \((F_{1})\) and \((F_{5})\), for above \(R_{\varepsilon}>0\), there exists a constant \(C_{R}>0\) such that
$$\begin{aligned} \biggl(\frac{ \vert F(x,u,v) \vert }{ \vert u \vert ^{l_{1}}+ \vert v \vert ^{l_{2}}} \biggr)^{k}\leq C_{R}\overline {F}(x,u,v)\quad \mbox{for all } x\in\mathbb{R}^{N}, \bigl\vert (u,v) \bigr\vert > R_{\varepsilon}. \end{aligned}$$
(3.29)
Let
$$X_{n}=\bigl\{ x\in\mathbb{R}^{N}: \bigl\vert \bigl(u_{n}(x),v_{n}(x)\bigr) \bigr\vert \leq R_{\varepsilon}\bigr\} \quad \mbox{and}\quad Y_{n}=\bigl\{ x\in \mathbb{R}^{N}: \bigl\vert \bigl(u_{n}(x),v_{n}(x) \bigr) \bigr\vert > R_{\varepsilon}\bigr\} . $$
Then
$$\begin{aligned} \frac{ \vert I_{2}(u_{n},v_{n}) \vert }{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+ \Vert v_{n} \Vert _{1,\Phi_{2}}^{l_{2}}} \leq& \int_{X_{n}}\frac{ \vert F(x,u_{n},v_{n}) \vert }{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+ \Vert v_{n} \Vert _{1,\Phi_{2}}^{l_{2}}}\,dx + \int_{Y_{n}}\frac{ \vert F(x,u_{n},v_{n}) \vert }{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+ \Vert v_{n} \Vert _{1,\Phi_{2}}^{l_{2}}}\,dx. \end{aligned}$$
(3.30)
By (3.28) and (3.24), we have
$$\begin{aligned} \int_{X_{n}}\frac{ \vert F(x,u_{n},v_{n}) \vert }{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+ \Vert v_{n} \Vert _{1,\Phi_{2}}^{l_{2}}}\,dx ={}& \int_{X_{n}}\frac{ \vert F(x,u_{n},v_{n}) \vert }{\frac{ \vert u_{n} \vert ^{l_{1}}}{ \vert \tilde {u}_{n} \vert ^{l_{1}}}+\frac{ \vert v_{n} \vert ^{l_{2}}}{ \vert \tilde{v}_{n} \vert ^{l_{2}}}}\,dx \\ \leq{}& \int_{X_{n}}\frac{ \vert F(x,u_{n},v_{n}) \vert }{ \vert u_{n} \vert ^{l_{1}}+ \vert v_{n} \vert ^{l_{2}}}\bigl( \vert \tilde {u}_{n} \vert ^{l_{1}}+ \vert \tilde{v}_{n} \vert ^{l_{2}}\bigr)\,dx \leq\varepsilon M_{1}. \end{aligned}$$
(3.31)
Since \(k>\max\{\frac{N}{l_{1}},\frac{N}{l_{2}}\}\), then \(\frac{l_{i}k}{k-1}\in (l_{i},l_{i}^{*})(i=1,2)\). Hence, by (3.29), (3.18), (3.26) and the fact \(\overline{F}(x,u,v)\geq0\), for n large, we have
$$\begin{aligned} &\int_{Y_{n}}\frac{ \vert F(x,u_{n},v_{n}) \vert }{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+ \Vert v_{n} \Vert _{1,\Phi_{2}}^{l_{2}}}\,dx \\ &\quad\leq \int_{Y_{n}}\frac{ \vert F(x,u_{n},v_{n}) \vert }{ \vert u_{n} \vert ^{l_{1}}+ \vert v_{n} \vert ^{l_{2}}}\bigl( \vert \tilde {u}_{n} \vert ^{l_{1}}+ \vert \tilde{v}_{n} \vert ^{l_{2}}\bigr)\,dx \\ &\quad\leq \biggl( \int_{Y_{n}} \biggl(\frac { \vert F(x,u_{n},v_{n}) \vert }{ \vert u_{n} \vert ^{l_{1}}+ \vert v_{n} \vert ^{l_{2}}} \biggr)^{k}\,dx \biggr)^{\frac{1}{k}} \biggl( \int_{Y_{n}} \bigl( \vert \tilde{u}_{n} \vert ^{l_{1}}+ \vert \tilde{v}_{n} \vert ^{l_{2}} \bigr)^{\frac{k}{k-1}}\,dx \biggr)^{\frac{k-1}{k}} \\ &\quad\leq \biggl( \int_{Y_{n}}C_{R}\overline{F}(x,u_{n},v_{n})\,dx \biggr)^{\frac{1}{k}} \biggl( \int_{\mathbb{R}^{N}}C_{\frac{k}{k-1}} \bigl( \vert \tilde{u}_{n} \vert ^{\frac {l_{1}k}{k-1}}+ \vert \tilde{v}_{n} \vert ^{\frac{l_{2}k}{k-1}} \bigr)\,dx \biggr)^{\frac {k-1}{k}} \\ &\quad\leq\bigl[C_{R}(c+1)\bigr]^{\frac{1}{k}} \bigl[C_{\frac{k}{k-1}} \bigl( \Vert \tilde{u}_{n} \Vert _{\frac{l_{1}k}{k-1}}^{\frac{l_{1}k}{k-1}}+ \Vert \tilde {v}_{n} \Vert _{\frac{l_{2}k}{k-1}}^{\frac{l_{2}k}{k-1}} \bigr) \bigr]^{\frac {k-1}{k}} = o_{n}(1). \end{aligned}$$
(3.32)
Since ε is arbitrary, it follows from (3.30)-(3.32) that
$$\begin{aligned} \frac{I_{2}(u_{n},v_{n})}{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+ \Vert v_{n} \Vert _{1,\Phi _{2}}^{l_{2}}}\rightarrow0 \quad\mbox{as } n\rightarrow\infty. \end{aligned}$$
(3.33)
By dividing (3.17) by \(\Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+ \Vert v_{n} \Vert _{1,\Phi _{2}}^{l_{2}}\) and letting \(n\rightarrow\infty\), we get a contradiction via (3.27) and (3.33).
Case 2. Suppose that \(\Vert u_{n} \Vert _{1,\Phi_{1}}\rightarrow\infty\) and \(\Vert v_{n} \Vert _{1,\Phi_{2}}\leq M_{2}\) for some constant \(M_{2}>0\). Let \(\tilde {u}_{n}=\frac{u_{n}}{ \Vert u_{n} \Vert _{1,\Phi_{1}}}\) and \(\tilde{v}_{n}=\frac{v_{n}}{ \Vert u_{n} \Vert _{1,\Phi_{1}}}\). Then \(\{{\tilde{u}_{n}}\}\) is bounded in \(W^{1,\Phi _{1}}(\mathbb{R}^{N})\) and \({\tilde{v}_{n}}\rightarrow{\mathbf{0} }\) in \(W^{1,\Phi _{2}}(\mathbb{R}^{N})\). We claim that
$$\begin{aligned} \lambda_{2}:=\lim_{n\rightarrow\infty}\sup_{y\in\mathbb{R}^{N}} \int _{B_{2}(y)}\bigl(\Phi_{1}\bigl( \vert \tilde{u}_{n} \vert \bigr)+\Phi_{2}\bigl( \vert \tilde{v}_{n} \vert \bigr)\bigr)\,dx=0. \end{aligned}$$
Indeed, if \(\lambda_{2}\neq0\), there exist a constant \(\delta>0\), a subsequence of \(\{(\tilde{u}_{n},\tilde{v}_{n})\}\), still denoted by \(\{ (\tilde{u}_{n},\tilde{v}_{n})\}\), and a sequence \(\{z_{n}\}\in\mathbb{Z}^{N}\) such that
$$\begin{aligned} \int_{B_{2}(z_{n})}\bigl(\Phi_{1}\bigl( \vert \tilde{u}_{n} \vert \bigr)+\Phi_{2}\bigl( \vert \tilde{v}_{n} \vert \bigr)\bigr)\,dx>\delta \quad\mbox{for all } n\in \mathbb{N}. \end{aligned}$$
(3.34)
Let \(\bar{u}_{n}=\tilde{u}_{n}(\cdot+z_{n})\) and \(\bar{v}_{n}=\tilde {v}_{n}(\cdot+z_{n})\). Then \(\Vert \bar{u}_{n} \Vert _{1,\Phi_{1}}= \Vert \tilde{u}_{n} \Vert _{1,\Phi_{1}}\) and \(\Vert \bar{v}_{n} \Vert _{1,\Phi_{2}}= \Vert \tilde{v}_{n} \Vert _{1,\Phi_{2}}\), that is, \(\{{\bar{u}_{n}}\}\) is bounded in \(W^{1,\Phi_{1}}(\mathbb{R}^{N})\) and \({\bar{v}_{n}}\rightarrow\mathbf{0}\) in \(W^{1,\Phi_{2}}(\mathbb {R}^{N})\). Passing to a subsequence of \(\{({\bar{u}_{n}},{\bar{v}_{n}})\}\), still denoted by \(\{({\bar{u}_{n}},{\bar{v}_{n}})\}\), by Remark 2.7, there exists \((\bar{u},\mathbf{0})\in W\) such that
- ⋆:
-
\(\bar{u}_{n}\rightharpoonup\bar{u}\) in \(W^{1,\Phi_{1}}(\mathbb{R}^{N})\), \(\bar{u}_{n}\rightarrow\bar{u}\) in \(L^{\Phi_{1}}(B_{2})\) and \(\bar{u}_{n}(x)\rightarrow\bar{u}(x)\) a.e. in \(B_{2}\);
- ⋆:
-
\(\bar{v}_{n}\rightarrow\mathbf{0}\) in \(W^{1,\Phi_{2}}(\mathbb{R}^{N})\), \(\bar{v}_{n}\rightarrow\mathbf{0}\) in \(L^{\Phi_{2}}(B_{2})\) and \(\bar{v}_{n}(x)\rightarrow\mathbf{0}\) a.e. in \(B_{2}\).
Since
$$\begin{aligned} \int_{B_{2}}\bigl(\Phi_{1}\bigl( \vert \bar{u}_{n} \vert \bigr)+\Phi_{2}\bigl( \vert \bar{v}_{n} \vert \bigr)\bigr)\,dx= \int _{B_{2}(z_{n})}\bigl(\Phi_{1}\bigl( \vert \tilde{u}_{n} \vert \bigr)+\Phi_{2}\bigl( \vert \tilde{v}_{n} \vert \bigr)\bigr)\,dx, \end{aligned}$$
then, by (3.34), ⋆ and (2.3), we obtain that \(\bar {u}\neq\mathbf{0}\) in \(L^{\Phi_{1}}(B_{2})\), that is, \([\bar{u}\neq0]:=\{x \in B_{2}: \bar{u}(x)\neq0\}\) has nonzero Lebesgue measure. Let \(u_{n}^{*}=u_{n}(\cdot+z_{n})\) and \(v_{n}^{*}=v_{n}(\cdot+z_{n})\). Then \(\Vert (u_{n}^{*},v_{n}^{*}) \Vert = \Vert (u_{n},v_{n}) \Vert \) and
$$\begin{aligned} \bigl\vert u_{n}^{*}(x) \bigr\vert = \bigl\vert \bar{u}_{n}(x) \bigr\vert \Vert u_{n} \Vert _{1,\Phi_{1}}\rightarrow\infty, \quad\mbox{a.e. } x\in[\bar{u}\neq0]. \end{aligned}$$
(3.35)
Since \(V_{i}(i=1,2)\) and F are 1-periodic in x, \(\{(u_{n}^{*},v_{n}^{*})\}\) is also a \((C)_{c}\)-sequence of I. Then, by (3.18), for n large, we have
$$\begin{aligned} \int_{\mathbb{R}^{N}}\overline{F}\bigl(x,u_{n}^{*},v_{n}^{*} \bigr)\,dx\leq c+1. \end{aligned}$$
(3.36)
However, it follows from \((F_{5})\), (3.35), (3.21) and Fatou’s lemma that
$$\begin{aligned} \int_{\mathbb{R}^{N}}\overline{F}\bigl(x,u_{n}^{*},v_{n}^{*} \bigr)\,dx\geq \int_{[\bar {u}\neq0]}\overline{F}\bigl(x,u_{n}^{*},v_{n}^{*} \bigr)\,dx=+\infty, \end{aligned}$$
which contradicts (3.36). Therefore, \(\lambda_{2}=0\) and
$$\begin{aligned} \lim_{n\rightarrow\infty}\sup_{y\in\mathbb{R}^{N}} \int_{B_{2}(y)}\Phi _{1}\bigl( \vert \tilde{u}_{n} \vert \bigr)\,dx=0. \end{aligned}$$
(3.37)
Then, by the Lions type result for Orlicz-Sobolev spaces (see Theorem 1.3 in [8]) again, (3.37), (3.25) and the fact \(\frac{l_{1}k}{k-1}\in(l_{1},l_{1}^{*})\) imply that
$$\begin{aligned} \tilde{u}_{n}\rightarrow\mathbf{0} \quad\mbox{in } L^{\frac {l_{1}k}{k-1}}\bigl(\mathbb{R}^{N}\bigr). \end{aligned}$$
(3.38)
Since the embeddings \(W^{1, \Phi_{i}}(\mathbb{R}^{N})\hookrightarrow L^{l_{i}}(\mathbb{R}^{N})(i=1,2)\) are continuous, there exists a constant \(M_{3}>0\) such that
$$\begin{aligned} \Vert \tilde{u}_{n} \Vert _{l_{1}}^{l_{1}}+ \Vert v_{n} \Vert _{l_{2}}^{l_{2}}\leq M_{3} \quad\mbox{for all } n\in\mathbb{N}. \end{aligned}$$
(3.39)
Moreover, \(\frac{l_{2}k}{k-1}\in(l_{2},l_{2}^{*})\), (3.25) and Lemma 2.5 imply that the embedding \(W^{1, \Phi_{2}}(\mathbb{R}^{N})\hookrightarrow L^{\frac{l_{2}k}{k-1}}(\mathbb{R}^{N})\) is continuous. Hence, there exists a constant \(M_{4}>0\) such that
$$\begin{aligned} \Vert v_{n} \Vert _{\frac{l_{2}k}{k-1}}^{l_{2}} \leq M_{4} \quad\mbox{for all } n\in\mathbb{N}. \end{aligned}$$
(3.40)
So, for any given constant \(M>1\), by (3.6), \((V_{2})\) and Lemma 2.2, we have
$$\begin{aligned} \frac{I_{1}(u_{n},v_{n})}{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+M} \geq{}& \frac{\min\{ \Vert \nabla u_{n} \Vert _{\Phi_{1}}^{l_{1}}, \Vert \nabla u_{n} \Vert _{\Phi _{1}}^{m_{1}}\}+\alpha_{1}\min\{ \Vert u_{n} \Vert _{\Phi_{1}}^{l_{1}}, \Vert u_{n} \Vert _{\Phi _{1}}^{m_{1}}\}}{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+M} \\ \geq{}& \frac{ \Vert \nabla u_{n} \Vert _{\Phi_{1}}^{l_{1}}+\alpha_{1} \Vert u_{n} \Vert _{\Phi _{1}}^{l_{1}}-1-\alpha_{1}}{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+M} \\ \geq{}& \frac{\min\{1,\alpha_{1}\}C_{l_{1}} \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}-1-\alpha_{1}}{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+M} \\ = {}& \min\{1,\alpha_{1}\}C_{l_{1}}+o_{n}(1). \end{aligned}$$
(3.41)
It is obvious that (3.28) and (3.29) still hold for this case. Based on this fact, let
$$\begin{aligned} &X_{n}=\bigl\{ x\in\mathbb{R}^{N}: \bigl\vert \bigl(u_{n}(x),v_{n}(x)\bigr) \bigr\vert \leq R_{\varepsilon}\bigr\} \quad\mbox{and}\\ & Y_{n}=\bigl\{ x\in \mathbb{R}^{N}: \bigl\vert \bigl(u_{n}(x),v_{n}(x) \bigr) \bigr\vert > R_{\varepsilon}\bigr\} . \end{aligned}$$
Then
$$\begin{aligned} \frac{ \vert I_{2}(u_{n},v_{n}) \vert }{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+M} \leq \int_{X_{n}}\frac{ \vert F(x,u_{n},v_{n}) \vert }{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+M}\,dx + \int_{Y_{n}}\frac{ \vert F(x,u_{n},v_{n}) \vert }{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+M}\,dx. \end{aligned}$$
(3.42)
By (3.28) and (3.39), we have
$$\begin{aligned} \int_{X_{n}}\frac{ \vert F(x,u_{n},v_{n}) \vert }{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+M}\,dx &\leq \int_{X_{n}}\frac{ \vert F(x,u_{n},v_{n}) \vert }{ \vert u_{n} \vert ^{l_{1}}+ \vert v_{n} \vert ^{l_{2}}}\biggl( \vert \tilde {u}_{n} \vert ^{l_{1}}+\frac{1}{M} \vert v_{n} \vert ^{l_{2}}\biggr)\,dx \\ &\leq\varepsilon \biggl( \Vert \tilde{u}_{n} \Vert _{l_{1}}^{l_{1}}+ \frac{1}{M} \Vert v_{n} \Vert _{l_{2}}^{l_{2}} \biggr) \leq\varepsilon M_{3}. \end{aligned}$$
(3.43)
Note that \(\frac{l_{i}k}{k-1}\in(l_{i},l_{i}^{*})(i=1,2)\). By (3.29), (3.18), (3.38), (3.40) and the fact \(\overline {F}(x,u,v)\geq0\), for n large, we have
$$\begin{aligned} &\int_{Y_{n}}\frac{ \vert F(x,u_{n},v_{n}) \vert }{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+M}\,dx \\ &\quad\leq \int_{Y_{n}}\frac{ \vert F(x,u_{n},v_{n}) \vert }{ \vert u_{n} \vert ^{l_{1}}+ \vert v_{n} \vert ^{l_{2}}}\biggl( \vert \tilde {u}_{n} \vert ^{l_{1}}+\frac{1}{M} \vert v_{n} \vert ^{l_{2}}\biggr)\,dx \\ &\quad \leq \biggl( \int_{Y_{n}} \biggl(\frac { \vert F(x,u_{n},v_{n}) \vert }{ \vert u_{n} \vert ^{l_{1}}+ \vert v_{n} \vert ^{l_{2}}} \biggr)^{k}\,dx \biggr)^{\frac{1}{k}} \biggl( \int_{Y_{n}} \biggl( \vert \tilde{u}_{n} \vert ^{l_{1}}+\frac{1}{M} \vert v_{n} \vert ^{l_{2}} \biggr)^{\frac{k}{k-1}}\,dx \biggr)^{\frac{k-1}{k}} \\ &\quad\leq \biggl( \int_{Y_{n}}C_{R}\overline{F}(x,u_{n},v_{n})\,dx \biggr)^{\frac{1}{k}} \biggl[C_{\frac{k}{k-1}} \biggl( \Vert \tilde{u}_{n} \Vert _{\frac{l_{1}k}{k-1}}^{\frac{l_{1}k}{k-1}}+ \biggl( \frac {1}{M} \biggr)^{\frac{k}{k-1}} \Vert v_{n} \Vert _{\frac{l_{2}k}{k-1}}^{\frac {l_{2}k}{k-1}} \biggr) \biggr]^{\frac{k-1}{k}} \\ &\quad \leq \bigl[C_{R}(c+1)\bigr]^{\frac{1}{k}}C_{\frac{k}{k-1}}^{\frac {k-1}{k}}C_{\frac{k-1}{k}} \biggl( \Vert \tilde{u}_{n} \Vert _{\frac{l_{1}k}{k-1}}^{l_{1}}+ \frac{1}{M} \Vert v_{n} \Vert _{\frac{l_{2}k}{k-1}}^{l_{2}} \biggr) \\ &\quad\leq \bigl[C_{R}(c+1)\bigr]^{\frac{1}{k}}C_{\frac{k}{k-1}}^{\frac {k-1}{k}}C_{\frac{k-1}{k}} \biggl(o_{n}(1)+\frac{M_{4}}{M} \biggr). \end{aligned}$$
(3.44)
Since \(\varepsilon>0\) and \(M>1\) are arbitrary, it follows from (3.42)-(3.44) that
$$\begin{aligned} \frac{I_{2}(u_{n},v_{n})}{ \Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+M}\rightarrow0 \quad\mbox{as } n\rightarrow\infty. \end{aligned}$$
(3.45)
By dividing (3.17) by \(\Vert u_{n} \Vert _{1,\Phi_{1}}^{l_{1}}+M\) and letting \(n\rightarrow\infty\), we get a contradiction via (3.41) and (3.45).
Case 3. Suppose that \(\Vert \nabla u_{n} \Vert _{\Phi_{1}}\leq M_{5}\) for some constant \(M_{5}>0\) and \(\Vert \nabla v_{n} \Vert _{\Phi_{2}}\rightarrow\infty\). For this case, with the same discussion as Case 2, we can also get a contradiction. □
Lemma 3.18
System (1.1) has a nontrivial solution under the assumptions of Theorems
3.1
and
3.2, respectively.
Proof
For the level \(c>0\) given in (3.13), there exists a \((C)_{c}\)-sequence \(\{(u_{n},v_{n})\}\) for I in W. Moreover, Lemmas 3.16 and 3.17 show that the sequence \(\{(u_{n},v_{n})\}\) is bounded in W. We claim that
$$\begin{aligned} \lambda_{3}:=\lim_{n\rightarrow\infty}\sup_{y\in\mathbb{R}^{N}} \int _{B_{2}(y)}\bigl(\Phi_{1}\bigl( \vert u_{n} \vert \bigr)+\Phi_{2}\bigl( \vert v_{n} \vert \bigr)\bigr)\,dx>0. \end{aligned}$$
Indeed, if \(\lambda_{3}=0\), then
$$\begin{aligned} \lim_{n\rightarrow\infty}\sup_{y\in\mathbb{R}^{N}} \int_{B_{2}(y)}\Phi_{1}\bigl( \vert u_{n} \vert \bigr)\,dx =\lim_{n\rightarrow\infty}\sup_{y\in\mathbb{R}^{N}} \int_{B_{2}(y)}\Phi _{2}\bigl( \vert v_{n} \vert \bigr)\,dx=0. \end{aligned}$$
By using the Lions type result for Orlicz-Sobolev spaces (see Theorem 1.3 in [8]) again, we have
$$\begin{aligned} \begin{aligned}& u_{n}\rightarrow\mathbf{0} \quad\mbox{in } L^{q_{1}}\bigl(\mathbb{R}^{N}\bigr) \quad\mbox{and}\\ &v_{n}\rightarrow\mathbf{0} \quad\mbox{in } L^{q_{2}}\bigl( \mathbb{R}^{N}\bigr), \mbox{ for all } q_{1}\in \bigl(m_{1},l_{1}^{*}\bigr), q_{2}\in \bigl(m_{2},l_{2}^{*}\bigr). \end{aligned} \end{aligned}$$
(3.46)
Given \(q_{i}\in(m_{i},l_{i}^{*})(i=1,2)\), by \((F_{1})\), \((F_{2})\), \((\phi_{1})\), \((\phi_{2})\) and (2.1), for any given constant \(\varepsilon>0\), there exists a constant \(C_{\varepsilon}>0\) such that
$$\begin{aligned} &\bigl\vert F(x,u_{n},v_{n}) \bigr\vert \leq\varepsilon \bigl(\Phi_{1}\bigl( \vert u_{n} \vert \bigr)+\Phi_{2}\bigl( \vert v_{n} \vert \bigr)+ \Phi _{1*}\bigl( \vert u_{n} \vert \bigr)+ \Phi_{2*}\bigl( \vert v_{n} \vert \bigr) \bigr)+C_{\varepsilon}\bigl( \vert u_{n} \vert ^{q_{1}}+ \vert v_{n} \vert ^{q_{2}}\bigr), \\ &\bigl\vert u_{n}F_{u}(x,u_{n},v_{n}) \bigr\vert \leq\varepsilon \bigl(\Phi_{1}\bigl( \vert u_{n} \vert \bigr)+\Phi _{2}\bigl( \vert v_{n} \vert \bigr)+\Phi_{1*}\bigl( \vert u_{n} \vert \bigr)+\Phi_{2*}\bigl( \vert v_{n} \vert \bigr) \bigr) \\ &\phantom{\bigl\vert u_{n}F_{u}(x,u_{n},v_{n}) \bigr\vert \leq}{}+C_{\varepsilon}\bigl( \vert u_{n} \vert ^{q_{1}}+ \vert v_{n} \vert ^{q_{2}}\bigr), \\ &\bigl\vert v_{n}F_{v}(x,u_{n},v_{n}) \bigr\vert \leq\varepsilon \bigl(\Phi_{1}\bigl( \vert u_{n} \vert \bigr)+\Phi _{2}\bigl( \vert v_{n} \vert \bigr)+\Phi_{1*}\bigl( \vert u_{n} \vert \bigr)+\Phi_{2*}\bigl( \vert v_{n} \vert \bigr) \bigr) \\ &\phantom{\bigl\vert v_{n}F_{v}(x,u_{n},v_{n}) \bigr\vert \leq}{}+C_{\varepsilon}\bigl( \vert u_{n} \vert ^{q_{1}}+ \vert v_{n} \vert ^{q_{2}}\bigr), \end{aligned}$$
(3.47)
for all \(x\in\mathbb{R}^{N}\). Then it follows from Lemma 2.2, (2) in Lemma 2.4, (2.8), (3.46) and the arbitrariness of \(\varepsilon >0\) that
$$\begin{aligned} \lim_{n\rightarrow\infty} \int_{\mathbb{R}^{N}}F(x,u_{n},v_{n})\,dx={}&\lim _{n\rightarrow\infty} \int_{\mathbb{R}^{N}}u_{n}F_{u}(x,u_{n},v_{n})\,dx \\ ={}&\lim_{n\rightarrow\infty} \int_{\mathbb{R}^{N}}v_{n}F_{v}(x,u_{n},v_{n})\,dx=0. \end{aligned}$$
(3.48)
Hence, by (3.4), (3.5), (3.14), \((\phi_{2})\), \((V_{2})\) and (3.48), we have
$$\begin{aligned} c = {}& \lim_{n\rightarrow\infty} \biggl\{ I(u_{n},v_{n})- \biggl\langle I'(u_{n},v_{n}), \biggl( \frac{1}{l_{1}}u_{n},\frac{1}{l_{2}}v_{n} \biggr) \biggr\rangle \biggr\} \\ = {}& \lim_{n\rightarrow\infty} \biggl\{ \int_{\mathbb{R}^{N}}\Phi _{1}\bigl( \vert \nabla u_{n} \vert \bigr)\,dx- \int_{\mathbb{R}^{N}}\frac{1}{l_{1}}a_{1}\bigl( \vert \nabla u_{n} \vert \bigr) \vert \nabla u_{n} \vert ^{2}\,dx \\ & {} + \int_{\mathbb{R}^{N}}V_{1}(x)\Phi_{1}\bigl( \vert u_{n} \vert \bigr)\,dx- \int_{\mathbb{R}^{N}}\frac {1}{l_{1}}V_{1}(x)a_{1} \bigl( \vert u_{n} \vert \bigr) \vert u_{n} \vert ^{2}\,dx \\ & {}+ \int_{\mathbb{R}^{N}}\Phi_{2}\bigl( \vert \nabla v_{n} \vert \bigr)\,dx- \int_{\mathbb{R}^{N}}\frac {1}{l_{2}}a_{2}\bigl( \vert \nabla v_{n} \vert \bigr) \vert \nabla v_{n} \vert ^{2}\,dx \\ & {} + \int_{\mathbb{R}^{N}}V_{2}(x)\Phi_{2}\bigl( \vert v_{n} \vert \bigr)\,dx- \int_{\mathbb{R}^{N}}\frac {1}{l_{2}}V_{2}(x)a_{2} \bigl( \vert v_{n} \vert \bigr) \vert v_{n} \vert ^{2}\,dx \\ & {}+ \int_{\mathbb{R}^{N}} \biggl(\frac {1}{l_{1}}u_{n}F_{u}(x,u_{n},v_{n})+ \frac {1}{l_{2}}v_{n}F_{v}(x,u_{n},v_{n})-F(x,u_{n},v_{n}) \biggr)\,dx \biggr\} \\ \leq{}& \lim_{n\rightarrow\infty} \biggl\{ \int_{\mathbb{R}^{N}} \biggl(\frac {1}{l_{1}}u_{n}F_{u}(x,u_{n},v_{n})+ \frac {1}{l_{2}}v_{n}F_{v}(x,u_{n},v_{n})-F(x,u_{n},v_{n}) \biggr)\,dx \biggr\} =0, \end{aligned}$$
which contradicts \(c>0\). Therefore, \(\lambda_{3}> 0\), which implies that there exist a constant \(\delta>0\), a subsequence of \(\{(u_{n},v_{n})\}\), still denoted by \(\{(u_{n},v_{n})\}\), and a sequence \(\{z_{n}\}\in\mathbb {Z}^{N}\) such that
$$\begin{aligned} \begin{aligned} &\int_{B_{2}(z_{n})}\bigl(\Phi_{1}\bigl( \vert u_{n} \vert \bigr)+\Phi_{2}\bigl( \vert v_{n} \vert \bigr)\bigr)\,dx= \int_{B_{2}}\bigl(\Phi _{1}\bigl( \bigl\vert u_{n}^{*} \bigr\vert \bigr)+\Phi_{2}\bigl( \bigl\vert v_{n}^{*} \bigr\vert \bigr)\bigr)\,dx>\delta\\ &\quad \mbox{for all } n\in \mathbb{N}, \end{aligned} \end{aligned}$$
(3.49)
where \(u_{n}^{*}:=u_{n}(\cdot+z_{n})\) and \(v_{n}^{*}:=v_{n}(\cdot+z_{n})\). Since \(\Vert u_{n}^{*} \Vert _{1,\Phi_{1}}= \Vert u_{n} \Vert _{1,\Phi_{1}}\) and \(\Vert v_{n}^{*} \Vert _{1,\Phi_{2}}= \Vert v_{n} \Vert _{1,\Phi_{2}}\), then \(\{{u_{n}^{*}}\}\) and \(\{{v_{n}^{*}}\}\) are bounded in \(W^{1,\Phi_{1}}(\mathbb{R}^{N})\) and \(W^{1,\Phi_{2}}(\mathbb{R}^{N})\), respectively. Passing to a subsequence of \(\{({u_{n}^{*}},{v_{n}^{*}})\}\), still denoted by \(\{({u_{n}^{*}},{v_{n}^{*}})\}\), there exists \((u^{*},v^{*})\in W\) such that \(u_{n}^{*}\rightharpoonup u^{*}\) in \(W^{1,\Phi_{1}}(\mathbb{R}^{N})\) and \(v_{n}^{*}\rightharpoonup v^{*}\) in \(W^{1,\Phi_{2}}(\mathbb{R}^{N})\), respectively. Moreover, for any given constant \(r>0\), by Remark 2.7 and the similar arguments as those in Lemma 4.3 in [8], we can assume that
- ⋆:
-
\(u_{n}^{*}\rightarrow u^{*}\) in \(L^{\Phi_{1}}(B_{r})\) and \(u_{n}^{*}(x)\rightarrow u^{*}(x), \nabla u_{n}^{*}(x)\rightarrow\nabla u^{*}(x)\) a.e. in \(B_{r}\);
- ⋆:
-
\(v_{n}^{*}\rightarrow v^{*}\) in \(L^{\Phi_{2}}(B_{r})\) and \(v_{n}^{*}(x)\rightarrow v^{*}(x), \nabla v_{n}^{*}(x)\rightarrow\nabla v^{*}(x)\) a.e. in \(B_{r}\).
Then, by (3.49), ⋆ and (2.3), we obtain that \((u^{*},v^{*})\neq(\mathbf{0,0})\). Since \(V_{i}(i=1,2)\) and F are 1-periodic in x, \(\{(u_{n}^{*},v_{n}^{*})\}\) is also a \((C)_{c}\)-sequence of I. Then, for any given point \((w_{1},w_{2})\in C_{0}^{\infty}(\mathbb {R}^{N})\times C_{0}^{\infty}(\mathbb{R}^{N})\) with \(\operatorname{supp}\{w_{1}\}\cup \operatorname{supp}\{w_{2}\}\subset B_{r}\) for some \(r>0\), we have
$$\lim_{n\rightarrow\infty}\bigl\langle I'\bigl(u_{n}^{*},v_{n}^{*} \bigr),(w_{1},w_{2})\bigr\rangle =0. $$
We claim that
$$\begin{aligned} \lim_{n\rightarrow\infty}\bigl\langle I' \bigl(u_{n}^{*},v_{n}^{*}\bigr),(w_{1},w_{2}) \bigr\rangle =\bigl\langle I'\bigl(u^{*},v^{*}\bigr),(w_{1},w_{2}) \bigr\rangle . \end{aligned}$$
(3.50)
First, we claim
$$\begin{aligned} \lim_{n\rightarrow\infty} \int_{\mathbb {R}^{N}}V_{1}(x)a_{1}\bigl( \bigl\vert u_{n}^{*} \bigr\vert \bigr)u_{n}^{*}w_{1}\,dx= \int_{\mathbb{R}^{N}}V_{1}(x)a_{1}\bigl( \bigl\vert u^{*} \bigr\vert \bigr)u^{*}w_{1}\,dx. \end{aligned}$$
(3.51)
Indeed, it follows from \((\phi_{1})\), \((\phi_{2})\), \((V_{2})\), ⋆ and the boundedness of sequence \(\{u_{n}^{*}\}\) in \(W^{1,\Phi_{1}}(\mathbb{R}^{N})\) that the sequence \(\{V_{1}(x)a_{1}( \vert u_{n}^{*} \vert )u_{n}^{*}\}\) is bounded in \(L^{\widetilde{\Phi}_{1}}(B_{r})\) and \(V_{1}(x)\times a_{1}( \vert u_{n}^{*}(x) \vert )u_{n}^{*}(x)\rightarrow V_{1}(x)a_{1}( \vert u^{*}(x) \vert )u^{*}(x)\) a.e. \(x\in B_{r}\). Then, by applying Lemma 2.1 in [8], we get (3.51) because \(w_{1}\in L^{\Phi_{1}}(B_{r})\). Similarly, we can get
$$\begin{aligned} \lim_{n\rightarrow\infty} \int_{\mathbb {R}^{N}}V_{2}(x)a_{2}\bigl( \bigl\vert v_{n}^{*} \bigr\vert \bigr)v_{n}^{*}w_{2}\,dx= \int_{\mathbb{R}^{N}}V_{2}(x)a_{2}\bigl( \bigl\vert v^{*} \bigr\vert \bigr)v^{*}w_{2}\,dx. \end{aligned}$$
(3.52)
Next, we claim
$$\begin{aligned} \lim_{n\rightarrow\infty} \int_{\mathbb{R}^{N}} F_{u}\bigl(x,u_{n}^{*},v_{n}^{*} \bigr)w_{1}\,dx= \int_{\mathbb{R}^{N}}F_{u}\bigl(x,u^{*},v^{*}\bigr)w_{1}\,dx. \end{aligned}$$
(3.53)
Indeed, it follows from \((\phi_{1})\), \((\phi_{2})\), \((F_{1})\), \((F_{2})\), ⋆, the boundedness of sequence \(\{(u_{n}^{*},v_{n}^{*})\}\) in W and Remark 2.7 that the sequence \(\{F_{u}(x,u_{n}^{*},v_{n}^{*})\}\) is bounded in \(L^{\widetilde{\Phi}_{1*}}(B_{r})\) and \(F_{u}(x,u_{n}^{*}(x),v_{n}^{*}(x))\rightarrow F_{u}(x,u^{*}(x),v^{*}(x))\) a.e. \(x\in B_{r}\). Then, by applying Lemma 2.1 in [8] again, we get (3.53) because \(w_{1}\in L^{\Phi_{1*}}(B_{r})\). Similarly, we can get
$$\begin{aligned} \lim_{n\rightarrow\infty} \int_{\mathbb{R}^{N}} F_{v}\bigl(x,u_{n}^{*},v_{n}^{*} \bigr)w_{2}\,dx= \int_{\mathbb{R}^{N}}F_{v}\bigl(x,u^{*},v^{*}\bigr)w_{2}\,dx. \end{aligned}$$
(3.54)
Finally, we claim
$$\begin{aligned} \lim_{n\rightarrow\infty} \int_{\mathbb{R}^{N}}a_{1}\bigl( \bigl\vert \nabla u_{n}^{*} \bigr\vert \bigr)\nabla u_{n}^{*}\nabla w_{1}\,dx= \int_{\mathbb{R}^{N}}a_{1}\bigl( \bigl\vert \nabla u^{*} \bigr\vert \bigr)\nabla u^{*}\nabla w_{1}\,dx \end{aligned}$$
(3.55)
and
$$\begin{aligned} \lim_{n\rightarrow\infty} \int_{\mathbb{R}^{N}}a_{2}\bigl( \bigl\vert \nabla v_{n}^{*} \bigr\vert \bigr)\nabla v_{n}^{*}\nabla w_{2}\,dx= \int_{\mathbb{R}^{N}}a_{2}\bigl( \bigl\vert \nabla v^{*} \bigr\vert \bigr)\nabla v^{*}\nabla w_{2}\,dx. \end{aligned}$$
(3.56)
In fact, the boundedness of sequence \(\{(u_{n}^{*},v_{n}^{*})\}\) implies that sequences \(\{a_{1}( \vert \nabla u_{n}^{*} \vert )\frac{\partial u_{n}^{*}}{\partial x_{j}} \}\ (j=1,2,\ldots,N)\) are bounded in \(L^{\widetilde{\Phi }_{1}}(B_{r})\). Moreover, \((\phi_{1})\) and ⋆ imply that \(a_{1}( \vert \nabla u_{n}^{*}(x) \vert )\frac{\partial u_{n}^{*}(x)}{\partial x_{j}}\rightarrow a_{1}( \vert \nabla u^{*}(x) \vert )\frac{\partial u^{*}(x)}{\partial x_{j}}(j=1,2,\ldots,N)\) a.e. \(x\in B_{r}\). Then, by applying Lemma 2.1 in [8] again, we get (3.55) because \(\frac{\partial w_{1}}{\partial x_{j}}\in L^{\Phi _{1}}(B_{r})(j=1,2,\ldots,N)\). Similarly, we can get (3.56). Hence, it follows from (3.51)-(3.56) that (3.50) holds, that is, \(\langle I'(u^{*},v^{*}),(w_{1},w_{2})\rangle=0\) for all \((w_{1},w_{2})\in C_{0}^{\infty}(\mathbb{R}^{N})\times C_{0}^{\infty}(\mathbb{R}^{N})\). Now, we can conclude that \(I'(u^{*},v^{*})=0\) because \(C_{0}^{\infty}(\mathbb {R}^{N})\times C_{0}^{\infty}(\mathbb{R}^{N})\) is dense in W. □
Proof of Theorem 3.1
Lemma 3.18 shows that system (1.1) has at least a nontrivial solution. Next, we prove that system (1.1) has a ground state. Let
$$d=\inf\bigl\{ I(u,v): (u,v)\neq(\mathbf{0,0}) \mbox{ and } I'(u,v)=0 \bigr\} . $$
First, we claim that \(d\geq0\). Indeed, for any given nontrivial critical point \((u,v)\) of I, by (3.4), (3.5), \((\phi _{2})\), \((V_{2})\) and \((F_{3})\), we have
$$\begin{aligned} I(u,v)={}& I(u,v)- \biggl\langle I'(u,v), \biggl( \frac{1}{\mu_{1}}u,\frac {1}{\mu_{2}}v \biggr) \biggr\rangle \\ ={} & \int_{\mathbb{R}^{N}} \biggl(\Phi_{1}\bigl( \vert \nabla u \vert \bigr)-\frac{1}{\mu _{1}}a_{1}\bigl( \vert \nabla u \vert \bigr) \vert \nabla u \vert ^{2} \biggr)\,dx \\ &{}+ \int_{\mathbb{R}^{N}}V_{1}(x) \biggl(\Phi_{1}\bigl( \vert u \vert \bigr)-\frac{1}{\mu _{1}}a_{1}\bigl( \vert u \vert \bigr) \vert u \vert ^{2} \biggr)\,dx \\ & {}+ \int_{\mathbb{R}^{N}} \biggl(\Phi_{2}\bigl( \vert \nabla v \vert \bigr)-\frac{1}{\mu _{2}}a_{2}\bigl( \vert \nabla v \vert \bigr) \vert \nabla v \vert ^{2} \biggr)\,dx\\ &{} + \int_{\mathbb{R}^{N}}V_{2}(x) \biggl(\Phi_{2}\bigl( \vert v \vert \bigr)-\frac{1}{\mu _{2}}a_{2}\bigl( \vert v \vert \bigr) \vert v \vert ^{2} \biggr)\,dx \\ & {} + \int_{\mathbb{R}^{N}} \biggl(\frac{1}{\mu_{1}}uF_{u}(x,u,v)+ \frac{1}{\mu _{2}}vF_{v}(x,u,v)-F(x,u,v) \biggr)\,dx \\ \geq{}& \biggl(1-\frac{m_{1}}{\mu_{1}} \biggr) \int_{\mathbb{R}^{N}} \bigl(\Phi _{1}\bigl( \vert \nabla u \vert \bigr)+\alpha_{1}\Phi_{1}\bigl( \vert u \vert \bigr) \bigr)\,dx \\ &{}+ \biggl(1-\frac{m_{2}}{\mu_{2}} \biggr) \int_{\mathbb{R}^{N}} \bigl(\Phi _{2}\bigl( \vert \nabla v \vert \bigr)+\alpha_{1}\Phi_{2}\bigl( \vert v \vert \bigr) \bigr)\,dx \\ & {} + \int_{\mathbb{R}^{N}} \biggl(\frac{1}{\mu_{1}}uF_{u}(x,u,v)+ \frac{1}{\mu _{2}}vF_{v}(x,u,v)-F(x,u,v) \biggr)\,dx\geq0. \end{aligned}$$
Since the nontrivial critical point \((u,v)\) of I is arbitrary, we conclude \(d\geq0\). Choose a sequence \(\{(u_{n},v_{n})\}\subset\{(u,v)\in W: (u,v)\neq(\mathbf{0,0}) \mbox{ and } I'(u,v)=0\}\) such that \(I(u_{n},v_{n})\rightarrow d\) as \(n\rightarrow\infty\). Then it is obvious that \(\{(u_{n},v_{n})\}\) is a \((C)_{d}\)-sequence of I for the level \(d\geq0\). Lemma 3.16 shows that \(\{(u_{n},v_{n})\}\) is bounded in W. Moreover, Lemma A.3 in the Appendix implies that there exists a constant \(M_{6}>0\) such that
$$\begin{aligned} \bigl\Vert (u_{n},v_{n}) \bigr\Vert \geq M_{6} \quad\mbox{for all } n\in\mathbb{N}. \end{aligned}$$
(3.57)
We claim that
$$\begin{aligned} \lambda_{4}:=\lim_{n\rightarrow\infty}\sup_{y\in\mathbb{R}^{N}} \int _{B_{2}(y)}\bigl(\Phi_{1}\bigl( \vert u_{n} \vert \bigr)+\Phi_{2}\bigl( \vert v_{n} \vert \bigr)\bigr)\,dx>0. \end{aligned}$$
Indeed, if \(\lambda_{4}=0\), similar to (3.48), we get
$$\begin{aligned} \lim_{n\rightarrow\infty} \int_{\mathbb{R}^{N}}u_{n}F_{u}(x,u_{n},v_{n})\,dx= \lim_{n\rightarrow\infty} \int_{\mathbb{R}^{N}}v_{n}F_{v}(x,u_{n},v_{n})\,dx=0. \end{aligned}$$
(3.58)
Then, by (3.5), \((\phi_{2})\), \((V_{2})\) and (3.58), we have
$$\begin{aligned} 0 = {}& \lim_{n\rightarrow\infty} \biggl\{ \bigl\langle I'(u_{n},v_{n}),(u_{n},v_{n}) \bigr\rangle + \int_{\mathbb{R}^{N}}u_{n}F_{u}(x,u_{n},v_{n})\,dx+ \int _{\mathbb{R}^{N}}v_{n}F_{v}(x,u_{n},v_{n})\,dx \biggr\} \\ = {}& \lim_{n\rightarrow\infty} \biggl\{ \int_{\mathbb{R}^{N}}a_{1}\bigl( \vert \nabla u_{n} \vert \bigr) \vert \nabla u_{n} \vert ^{2}\,dx + \int_{\mathbb {R}^{N}}V_{1}(x)a_{1}\bigl( \vert u_{n} \vert \bigr) \vert u_{n} \vert ^{2}\,dx \\ & {}+ \int_{\mathbb{R}^{N}}a_{2}\bigl( \vert \nabla v_{n} \vert \bigr) \vert \nabla v_{n} \vert ^{2}\,dx + \int _{\mathbb{R}^{N}}V_{2}(x)a_{2}\bigl( \vert v_{n} \vert \bigr) \vert v_{n} \vert ^{2}\,dx \biggr\} \\ \geq{}& \lim_{n\rightarrow\infty} \biggl\{ l_{1} \int_{\mathbb{R}^{N}}\Phi _{1}\bigl( \vert \nabla u_{n} \vert \bigr)\,dx+l_{1}\alpha_{1} \int_{\mathbb{R}^{N}}\Phi_{1}\bigl( \vert u_{n} \vert \bigr)\,dx\\ &{} +l_{2} \int_{\mathbb{R}^{N}}\Phi_{2}\bigl( \vert \nabla v_{n} \vert \bigr)\,dx+l_{2}\alpha_{1} \int_{\mathbb {R}^{N}}\Phi_{2}\bigl( \vert v_{n} \vert \bigr)\,dx \biggr\} \\ \geq{}&0, \end{aligned}$$
which, together with (2.3), implies that \(\Vert (u_{n}, v_{n}) \Vert = \Vert \nabla u_{n} \Vert _{\Phi_{1}}+ \Vert u_{n} \Vert _{\Phi_{1}}+ \Vert \nabla v_{n} \Vert _{\Phi_{2}}+ \Vert v_{n} \Vert _{\Phi_{2}}\rightarrow0\), which contradicts (3.57). Therefore, \(\lambda_{4}> 0\), which implies that there exist a constant \(\delta>0\), a subsequence of \(\{(u_{n},v_{n})\}\), still denoted by \(\{(u_{n},v_{n})\}\), and a sequence \(\{z_{n}\}\in\mathbb{Z}^{N}\) such that
$$\begin{aligned} \begin{aligned} &\int_{B_{2}(z_{n})}\bigl(\Phi_{1}\bigl( \vert u_{n} \vert \bigr)+\Phi_{2}\bigl( \vert v_{n} \vert \bigr)\bigr)\,dx= \int_{B_{2}}\bigl(\Phi _{1}\bigl( \bigl\vert u_{n}^{*} \bigr\vert \bigr)+\Phi_{2}\bigl( \bigl\vert v_{n}^{*} \bigr\vert \bigr)\bigr)\,dx>\delta \\ &\quad\mbox{for all } n\in \mathbb{N}, \end{aligned} \end{aligned}$$
(3.59)
where \(u_{n}^{*}:=u_{n}(\cdot+z_{n})\) and \(v_{n}^{*}:=v_{n}(\cdot+z_{n})\). Since \(\Vert u_{n}^{*} \Vert _{1,\Phi_{1}}= \Vert u_{n} \Vert _{1,\Phi_{1}}\) and \(\Vert v_{n}^{*} \Vert _{1,\Phi_{2}}= \Vert v_{n} \Vert _{1,\Phi_{2}}\), then \(\{{u_{n}^{*}}\}\) and \(\{{v_{n}^{*}}\}\) are bounded in \(W^{1,\Phi_{1}}(\mathbb{R}^{N})\) and \(W^{1,\Phi_{2}}(\mathbb{R}^{N})\), respectively. Passing to a subsequence of \(\{({u_{n}^{*}},{v_{n}^{*}})\}\), still denoted by \(\{({u_{n}^{*}},{v_{n}^{*}})\}\), there exists \((u_{0},v_{0})\in W\) such that \(u_{n}^{*}\rightharpoonup u_{0}\) in \(W^{1,\Phi_{1}}(\mathbb{R}^{N})\) and \(v_{n}^{*}\rightharpoonup v_{0}\) in \(W^{1,\Phi_{2}}(\mathbb{R}^{N})\), respectively. Moreover, for any given constant \(r>0\), by Remark 2.7, we can assume that
- ⋆:
-
\(u_{n}^{*}\rightarrow u_{0}\) in \(L^{\Phi_{1}}(B_{r})\) and \(u_{n}^{*}(x)\rightarrow u_{0}(x)\) a.e. in \(B_{r}\);
- ⋆:
-
\(v_{n}^{*}\rightarrow v_{0}\) in \(L^{\Phi_{2}}(B_{r})\) and \(v_{n}^{*}(x)\rightarrow v_{0}(x)\) a.e. in \(B_{r}\).
Then, by (3.59), ⋆ and (2.3), we obtain that \((u_{0},v_{0})\neq(\mathbf{0,0})\). Since \(V_{i}(i=1,2)\) and F are 1-periodic in x, \(\{(u_{n}^{*},v_{n}^{*})\}\) is also a \((C)_{d}\)-sequence of I with \(\{(u_{n}^{*},v_{n}^{*})\}\subset\{(u,v)\in W: (u,v)\neq(\textbf {0,0}) \mbox{ and } I'(u,v)=0\}\). Then similar arguments as those in Lemma 3.18 show that \(I'(u_{0},v_{0})=0\), and thus \(I(u_{0},v_{0})\geq d\). However, for any given constant \(r>0\), it follows from (3.4), (3.5), \((\phi_{2})\), \((V_{2})\), \((F_{3})\), ⋆ and Fatou’s lemma that
$$\begin{aligned} & \int_{B_{r}} \biggl(\Phi_{1}\bigl( \vert \nabla u_{0} \vert \bigr)-\frac{1}{\mu_{1}}a_{1}\bigl( \vert \nabla u_{0} \vert \bigr) \vert \nabla u_{0} \vert ^{2} \biggr)\,dx \\ &\qquad{}+ \int_{B_{r}}V_{1}(x) \biggl(\Phi_{1}\bigl( \vert u_{0} \vert \bigr)-\frac{1}{\mu _{1}}a_{1}\bigl( \vert u_{0} \vert \bigr) \vert u_{0} \vert ^{2} \biggr)\,dx \\ & \qquad{}+ \int_{B_{r}} \biggl(\Phi_{2}\bigl( \vert \nabla v_{0} \vert \bigr)-\frac{1}{\mu_{2}}a_{2}\bigl( \vert \nabla v_{0} \vert \bigr) \vert \nabla v_{0} \vert ^{2} \biggr)\,dx \\ &\qquad{}+ \int_{B_{r}}V_{2}(x) \biggl(\Phi_{2}\bigl( \vert v_{0} \vert \bigr)-\frac{1}{\mu _{2}}a_{2}\bigl( \vert v_{0} \vert \bigr) \vert v_{0} \vert ^{2} \biggr)\,dx \\ & \qquad{} + \int_{B_{r}} \biggl(\frac{1}{\mu_{1}}u_{0}F_{u}(x,u_{0},v_{0})+ \frac{1}{\mu _{2}}v_{0}F_{v}(x,u_{0},v_{0})-F(x,u_{0},v_{0}) \biggr)\,dx \\ &\quad \leq \liminf_{n\rightarrow\infty} \biggl\{ \int_{B_{r}} \biggl(\Phi _{1}\bigl( \bigl\vert \nabla u_{n}^{*} \bigr\vert \bigr)-\frac{1}{\mu_{1}}a_{1}\bigl( \bigl\vert \nabla u_{n}^{*} \bigr\vert \bigr) \bigl\vert \nabla u_{n}^{*} \bigr\vert ^{2} \biggr)\,dx\\ &\qquad{} + \int_{B_{r}}V_{1}(x) \biggl(\Phi_{1}\bigl( \bigl\vert u_{n}^{*} \bigr\vert \bigr)-\frac{1}{\mu _{1}}a_{1} \bigl( \bigl\vert u_{n}^{*} \bigr\vert \bigr) \bigl\vert u_{n}^{*} \bigr\vert ^{2} \biggr)\,dx \\ & \qquad{} + \int_{B_{r}} \biggl(\Phi_{2}\bigl( \bigl\vert \nabla v_{n}^{*} \bigr\vert \bigr)-\frac{1}{\mu_{2}}a_{2}\bigl( \bigl\vert \nabla v_{n}^{*} \bigr\vert \bigr) \bigl\vert \nabla v_{n}^{*} \bigr\vert ^{2} \biggr)\,dx \\ &\qquad{}+ \int_{B_{r}}V_{2}(x) \biggl(\Phi_{2}\bigl( \bigl\vert v_{n}^{*} \bigr\vert \bigr)-\frac{1}{\mu _{2}}a_{2} \bigl( \bigl\vert v_{n}^{*} \bigr\vert \bigr) \bigl\vert v_{n}^{*} \bigr\vert ^{2} \biggr)\,dx \\ & \qquad{} + \int_{B_{r}} \biggl(\frac{1}{\mu _{1}}u_{n}^{*}F_{u} \bigl(x,u_{n}^{*},v_{n}^{*}\bigr)+\frac{1}{\mu _{2}}v_{n}^{*}F_{v} \bigl(x,u_{n}^{*},v_{n}^{*}\bigr)-F\bigl(x,u_{n}^{*},v_{n}^{*} \bigr) \biggr)\,dx \biggr\} \\ &\quad \leq\liminf_{n\rightarrow\infty} \biggl\{ I\bigl(u_{n}^{*},v_{n}^{*} \bigr)- \biggl\langle I'\bigl(u_{n}^{*},v_{n}^{*} \bigr), \biggl(\frac{1}{\mu_{1}}u_{n}^{*},\frac{1}{\mu_{2}}v_{n}^{*} \biggr) \biggr\rangle \biggr\} \\ &\quad=d. \end{aligned}$$
Since \(r>0\) is arbitrary, then \(I(u_{0},v_{0})=I(u_{0},v_{0})- \langle I'(u_{0},v_{0}), (\frac{1}{\mu_{1}}u_{0},\frac{1}{\mu_{2}}v_{0} ) \rangle\leq d\). Therefore, \(I(u_{0},v_{0})=d\), that is, \((u_{0},v_{0})\) is a ground state of system (1.1). □
Proof of Theorem 3.2
Lemma 3.18 shows that system (1.1) has at least a nontrivial solution under the assumptions of Theorem 3.2. Moreover, following the same steps as in the above proof of Theorem 3.1 but replacing \(\mu_{i}\) with \(m_{i}(i=1,2)\), we can find a ground state of system (1.1). □