Existence of positive ground states for some nonlinear Schrödinger systems

Abstract

We prove the existence of positive ground states for the nonlinear Schrödinger system

$$\{\begin{array}{c}-\mathrm{\Delta }u+(1+a(x))u=F_u(u,v)+\lambda v,\hfill \\ -\mathrm{\Delta }v+(1+b(x))v=F_v(u,v)+\lambda u,\hfill \end{array}$$

where a, b are periodic or asymptotically periodic and F satisfies some superlinear conditions in $(u,v)$. The proof is based on the method of Nehari manifold and the concentration-compactness principle.

MSC:35J05, 35J50, 35J61.

1 Introduction and statement of the main result

This paper was motivated by the following two-component system of nonlinear Schrödinger equations:

$$\{\begin{array}{c}-i{\partial }_t\varphi =\mathrm{\Delta }\varphi +{\mu }_1{|\varphi |}^2\varphi +\beta {|\phi |}^2\varphi ,x\in {\mathbb{R}}^n,t>0,\hfill \\ -i{\partial }_t\phi =\mathrm{\Delta }\phi +{\mu }_2{|\phi |}^2\phi +\beta {|\varphi |}^2\phi ,x\in {\mathbb{R}}^n,t>0,\hfill \\ \varphi =\varphi (x,t),\phi =\phi (x,t)\in \mathbb{C},\hfill \\ \varphi (x,t),\phi (x,t)\to 0\text{as }|x|\to \mathrm{\infty },t>0,\hfill \end{array}$$

(1.1)

where ${\mu }_i>0$, $i=1,2$, $\beta \in \mathbb{R}$ and $n=2,3$. The system (1.1) has applications in many physical problems, especially in nonlinear optics (see [1]). To obtain standing wave solutions of (1.1) of the form $\varphi (x,t)=e^{i{\lambda }_{1}t}u(x)$, $\phi (x,t)=e^{i{\lambda }_{2}t}v(x)$ with ${\lambda }_1,{\lambda }_2>0$, the system (1.1) turns out to be

$$\{\begin{array}{c}-\mathrm{\Delta }u+{\lambda }_{1}u={\mu }_1{u}^3+\beta u{v}^2,\hfill \\ -\mathrm{\Delta }v+{\lambda }_{2}v={\mu }_2{v}^3+\beta v{u}^2.\hfill \end{array}$$

(1.2)

Following the work [2] by Lin and Wei about the existence of ground states for the problem (1.2), there are many results on the existence of ground states relevant to five parameters (${\lambda }_1$, ${\lambda }_2$, ${\mu }_1$, ${\mu }_2$ and β); see [3–9] and the references therein. Later in [10], assuming ${\lambda }_1={\lambda }_2=1$, Pomponio and Secchi established the existence of radially symmetric ground states for (1.2) with general nonlinearities ($f(u)$ and $g(v)$).

On the other hand, some authors considered the existence of ground states for non-autonomous similar problems. We recall the results about non-autonomous case for two subcases. For periodic case, in [11] Szulkin and Weth referred that treating as periodic Schrödinger equations, it is possible to deduce that there are ground states for the following system using the method of Nehari manifold:

$$\{\begin{array}{c}-\mathrm{\Delta }u+u=G_u(x,u,v),\hfill \\ -\mathrm{\Delta }v+v=G_v(x,u,v),\hfill \end{array}$$

(1.3)

where G is periodic in x and satisfies some superlinear conditions in $(u,v)$. For non-periodic case, we refer to [8, 12–14] for instance. As we can observe, most of the previous results on ground states for the non-periodic system have used the condition that there exists a limit system (or the problem at infinity; for precise statement, refer to [15]). Moreover, the limit system is autonomous. Here we mainly deal with an asymptotically periodic Schrödinger system which has a periodic non-autonomous limit system, roughly speaking. In this paper, we are concerned with the existence of positive ground states for the nonlinear Schrödinger system in $W^{1,2}({\mathbb{R}}^N)\times W^{1,2}({\mathbb{R}}^N)$ ($N\ge 2$)

$$\{\begin{array}{c}-\mathrm{\Delta }u+(1+a(x))u=F_u(u,v)+\lambda v,\hfill \\ -\mathrm{\Delta }v+(1+b(x))v=F_v(u,v)+\lambda u,\hfill \end{array}$$

(NLS)

where $\lambda >0$ is a real parameter. For simplicity, we denote +∞ by ∞

$$2^{\ast }=\{\begin{array}{c}\frac{2N}{N-2},N\ge 3,\hfill \\ \mathrm{\infty },N=2.\hfill \end{array}$$

Moreover, in what follows, the notation inf (sup) is understood as the essential infimum (supremum). In the sequel, let $a,b\in L^{\mathrm{\infty }}({\mathbb{R}}^N)$ and $F\in C^1({\mathbb{R}}^2,\mathbb{R})$ with $F(0,0)=0$, we always assume that

(V1) ${inf}_{{\mathbb{R}}^N}\{1+a(x)\}>\lambda$, ${inf}_{{\mathbb{R}}^N}\{1+b(x)\}>\lambda$,

(F1) $|\mathrm{\nabla }F(u,v)|\le C_0(1+{|(u,v)|}^{q-1})$ for some $C_0>0$ and $2<q<2^{\ast }$,

(F2) $|\mathrm{\nabla }F(u,v)|=o(|(u,v)|)$ as $|(u,v)|\to 0$,

(F3) $\mathrm{\forall }(u,v)\ne (0,0)$, $s>0$, $s\mapsto \frac{\mathrm{\nabla }F(su,sv)(u,v)}{s}$ is strictly increasing,

(F4) $\frac{F(u,v)}{u^2+v^2}\to \mathrm{\infty }$ as $|(u,v)|\to \mathrm{\infty }$,

(F5) $F_u(u,v)\ge 0$, $F_v(u,v)\ge 0$, $F_u(0,v)=F_v(u,0)=0$, $u\ge 0$, $v\ge 0$,

(F6) $F(u,v)\le F(|u|,|v|)$, $u,v\in \mathbb{R}$.

(F1)-(F4) are similar to the conditions of the nonlinearities for the periodic system (1.3) as considered in [11]. We divide the study of (NLS) into two cases as follows.

First, we consider the periodic case

(V2) $a(x)=a(x+y)$, $b(x)=b(x+y)$, $\mathrm{\forall }x\in {\mathbb{R}}^N$, $y\in {\mathbb{Z}}^N$.

We have the following result.

Theorem 1.1 Let (V1), (V2) and (F1)-(F6) hold. Then the system (NLS) has a positive ground state.

Remark 1.1 It is observed that the system (NLS) with periodic a and b is a particular case of the problem (1.3) with

$$G(x,u,v)=F(u,v)+\lambda uv-\frac{1}{2}(a(x)u^2+b(x)v^2),$$

and G is periodic in x. The problem (1.3) is mentioned in [11] when G is periodic in x. However, in [11] the conditions on the function G are not made explicit.

Next, we consider the asymptotically periodic case. We assume that there are functions $a_p,b_p\in L^{\mathrm{\infty }}({\mathbb{R}}^N)$ satisfying (V1) and (V2) and a, b satisfies that

(V3) ${lim}_{|x|\to \mathrm{\infty }}|a(x)-a_p(x)|=0$, ${lim}_{|x|\to \mathrm{\infty }}|b(x)-b_p(x)|=0$,

(V4) $a\le a_p$, $b\le b_p$.

We have the following result.

Theorem 1.2 Assume that $a_p$ and $b_p$ satisfy (V2). Let (V1), (V3), (V4) and (F1)-(F6) hold. Then the system (NLS) has a positive ground state.

Remark 1.2 Conditions (V1) and (V4) imply that $a_p$ and $b_p$ satisfy (V1).

In addition, we consider the following conditions:

(V5) $a_p=b_p:=V$, $a+b\le 2V$,

(F7) $F_u(u,v)=F_v(v,u)$, $u>0$, $v>0$.

We have the following result.

Theorem 1.3 Suppose that $a_p$ and $b_p$ satisfy (V1) and (V2). Let (V1), (V3), (V5) and (F1)-(F7) hold. Then the system (NLS) has a positive ground state.

We will prove Theorems 1.1, 1.2 and 1.3 using the method of Nehari manifold. We first reduce the problem of seeking for ground states of (NLS) into that of looking for minimizers of the functional constrained on the Nehari manifold. Then we apply the concentration-compactness principle to solve the minimization problem. Since the Nehari manifold for (NLS) may not be smooth, in the same way as [11], we will make use of the differential structure of a unit sphere in $W^{1,2}({\mathbb{R}}^N)\times W^{1,2}({\mathbb{R}}^N)$ to find a ${(\mathit{PS})}_c$ sequence (c is the infimum of the functional constrained on the Nehari manifold). When (NLS) is periodic, we will use the invariance of the functional under translation to recover the compactness of the ${(\mathit{PS})}_c$ sequence. When the system (NLS) is asymptotically periodic, the difficulty is to recover the compactness for the ${(\mathit{PS})}_c$ sequence. By comparing c with the infimum of the functional of the related periodic limit system constrained on the corresponding Nehari manifold, we will restore the compactness.

The paper is organized as follows. In Section 2 we give some preliminaries. In Section 3 we introduce the variational setting. In Section 4 we consider the periodic case and prove Theorem 1.1. Section 5 is devoted to studying the asymptotically periodic case and showing Theorems 1.2 and 1.3.

2 Notation and preliminaries

We use the following notation:

• For simplicity, we denote $\int h:={\int }_{{\mathbb{R}}^N}h(x)dx$ and ${\int }_{E}h:={\int }_{E}h(x)dx$, where $E\subset {\mathbb{R}}^N$ is measurable.

• X denotes the Sobolev space $W^{1,2}({\mathbb{R}}^N)$ ($N\ge 2$), with the standard scalar product ${〈u,v〉}_X=\int (\mathrm{\nabla }u\cdot \mathrm{\nabla }v+uv)$ and the norm ${\parallel u\parallel }_X^2={〈u,u〉}_X$. $H=X\times X$ with the norm ${\parallel (u,v)\parallel }_H^2={\parallel u\parallel }_X^2+{\parallel v\parallel }_X^2$. When there is no possible misunderstanding, the subscripts could be omitted.

• The usual norm in $L^r({\mathbb{R}}^N)$ ($2\le r\le \mathrm{\infty }$) will be denoted by ${|\cdot |}_r$.

• $S=\{(u,v)\in H:{\parallel (u,v)\parallel }^2=1\}$.

• For any $\varrho >0$ and $z\in {\mathbb{R}}^N$, $B_{\varrho }(z)$ denotes the ball of radius ϱ centered at z.

Note that $a,b\in L^{\mathrm{\infty }}({\mathbb{R}}^N)$ and $F\in C^1({\mathbb{R}}^2,\mathbb{R})$. Then by conditions (F1) and (F2), the functional

$$\mathrm{\Phi }(u,v)=\frac{1}{2}({\parallel u\parallel }^2+{\parallel v\parallel }^2+\int a(x)u^2+\int b(x)v^2)-\lambda \int uv-\int F(u,v)$$

is of class $C^1$ and its critical points are solutions of (NLS). Moreover, by (V1) we have

$$\mu {\parallel (u,v)\parallel }^2\le {\parallel u\parallel }^2+{\parallel v\parallel }^2+\int a(x)u^2+\int b(x)v^2-2\lambda \int uv\le \nu {\parallel (u,v)\parallel }^2,$$

(2.1)

with $\mu ,\nu >0$.

A solution $(\tilde{u},\tilde{v})$ of (NLS) is called a ground state if

$$\mathrm{\Phi }(\tilde{u},\tilde{v})=min\{\mathrm{\Phi }(u,v):(u,v)\in H\setminus \{(0,0)\},{\mathrm{\Phi }}^{\prime }(u,v)=0\}.$$

A ground state $(u,v)$ such that $u>0$, $v>0$ ($u\ge 0$, $v\ge 0$) is called a positive (non-negative) ground state. Below we give some lemmas useful for studying our problem.

Lemma 2.1 (F1) and (F2) imply that for all $ϵ>0$, there exists $C_{ϵ}>0$ such that

$$|\mathrm{\nabla }F(u,v)|\le ϵ|(u,v)|+C_{ϵ}{|(u,v)|}^{q-1}\mathrm{\forall }u,v\in \mathbb{R}.$$

(2.2)

(F2) and (F3) yield that

$$F(u,v)>0,2F(u,v)<\mathrm{\nabla }F(u,v)(u,v)\mathrm{\forall }(u,v)\ne (0,0).$$

(2.3)

Moreover, (F3) implies the function $\alpha (s)=\frac{1}{2}\mathrm{\nabla }F(su,sv)(su,sv)-F(su,sv)$ is increasing in $(0,\mathrm{\infty })$ for all $u,v\in \mathbb{R}$.

Proof The inequalities (2.2) and (2.3) are easily inferred from the corresponding assumptions. We just prove the last conclusion. Indeed, let $s_2\ge s_1>0$. Then by (F3) we obtain

$$\begin{array}{c}\alpha (s_2)-\alpha (s_1)\hfill \\ =\frac{1}{2}\mathrm{\nabla }F(s_{2}u,s_{2}v)(s_{2}u,s_{2}v)-\frac{1}{2}\mathrm{\nabla }F(s_{1}u,s_{1}v)(s_{1}u,s_{1}v)+F(s_{1}u,s_{1}v)-F(s_{2}u,s_{2}v)\hfill \\ ={\int }_0^{s_2}t\cdot \frac{\mathrm{\nabla }F(s_{2}u,s_{2}v)(u,v)}{s_2}dt-{\int }_0^{s_1}t\cdot \frac{\mathrm{\nabla }F(s_{1}u,s_{1}v)(u,v)}{s_1}dt-{\int }_{s_1}^{s_2}\mathrm{\nabla }F(tu,tv)(u,v)dt\hfill \\ ={\int }_0^{s_1}t\cdot [\frac{\mathrm{\nabla }F(s_{2}u,s_{2}v)(u,v)}{s_2}-\frac{\mathrm{\nabla }F(s_{1}u,s_{1}v)(u,v)}{s_1}]dt\hfill \\ +{\int }_{s_1}^{s_2}t\cdot [\frac{\mathrm{\nabla }F(s_{2}u,s_{2}v)(u,v)}{s_2}-\frac{\mathrm{\nabla }F(tu,tv)(u,v)}{t}]dt\ge 0.\hfill \end{array}$$

 □

Lemma 2.2 Let (F1) and (F2) hold. Then ${\mathrm{\Phi }}^{\prime }$ is weakly sequentially continuous. Namely, if $(u_n,v_n)\rightharpoonup (u,v)$ in H, then ${\mathrm{\Phi }}^{\prime }(u_n,v_n)\rightharpoonup {\mathrm{\Phi }}^{\prime }(u,v)$ in H.

Proof Suppose $(u_n,v_n)\rightharpoonup (u,v)$ in H. After passing to a subsequence, we assume $(u_n,v_n)\to (u,v)$ in $L_{\mathrm{loc}}^q({\mathbb{R}}^N)\times L_{\mathrm{loc}}^q({\mathbb{R}}^N)$. By (F1), we get

$$F_u(u_n,v_n)\to F_u(u,v),F_v(u_n,v_n)\to F_v(u,v)\text{in }{L}_{\mathrm{loc}}^{\frac{q}{q-1}}\left({\mathbb{R}}^N\right)\times L_{\mathrm{loc}}^{\frac{q}{q-1}}\left({\mathbb{R}}^N\right).$$

Then for all $(\varphi ,\phi )\in C_0^{\mathrm{\infty }}({\mathbb{R}}^N)\times C_0^{\mathrm{\infty }}({\mathbb{R}}^N)$, we have

$$\int F_u(u_n,v_n)\varphi \to \int F_u(u,v)\varphi ,\int F_v(u_n,v_n)\phi \to \int F_v(u,v)\phi .$$

So, one easily has that

$$〈{\mathrm{\Phi }}^{\prime }(u_n,v_n),(\varphi ,\phi )〉\to 〈{\mathrm{\Phi }}^{\prime }(u,v),(\varphi ,\phi )〉.$$

(2.4)

Now, we claim that ${\mathrm{\Phi }}^{\prime }(u_n,v_n)$ is bounded in H. Indeed, for $(h,k)\in H$, using (2.2) and the Hölder inequality, we obtain that

$$\begin{array}{rcl}|\int F_u(u_n,v_n)h|& \le & C\int (|u_n|+|v_n|)|h|+\overline{C}\int ({|u_n|}^{q-1}+{|v_n|}^{q-1})|h|\\ \le & C({|u_n|}_2+{|v_n|}_2){|h|}_2+C_1\overline{C}({|u_n|}_q^{q-1}+{|v_n|}_q^{q-1}){|h|}_q\\ \le & C(\parallel u_n\parallel +\parallel v_n\parallel )\parallel h\parallel +C_2\overline{C}({\parallel u_n\parallel }^{q-1}+{\parallel v_n\parallel }^{q-1})\parallel h\parallel <C^{\prime }\parallel h\parallel .\end{array}$$

Similarly, we get $|\int F_v(u_n,v_n)k|\le C\parallel k\parallel$. Then we easily have

$$\left|〈{\mathrm{\Phi }}^{\prime }(u_n,v_n),(h,k)〉\right|\le C\parallel (h,k)\parallel .$$

Hence, ${\mathrm{\Phi }}^{\prime }(u_n,v_n)$ is bounded in H. Combining with the fact that $C_0^{\mathrm{\infty }}({\mathbb{R}}^N)\times C_0^{\mathrm{\infty }}({\mathbb{R}}^N)$ is dense in H, we easily deduce that (2.4) holds for any $(\varphi ,\phi )\in H$. Therefore, ${\mathrm{\Phi }}^{\prime }(u_n,v_n)\rightharpoonup {\mathrm{\Phi }}^{\prime }(u,v)$ in H. □

3 Variational setting

This section is devoted to describing the variational framework for the study of ground states for (NLS).

It is easy to see that Φ is bounded neither from above nor from below. So, it is convenient to consider Φ on the Nehari manifold that contains all nontrivial critical points of Φ and on which Φ turns out to be bounded from below. The Nehari manifold M corresponding to Φ is defined by

$$M=\{(u,v)\in H\setminus \{(0,0)\}:〈{\mathrm{\Phi }}^{\prime }(u,v),(u,v)〉=0\},$$

where

$$〈{\mathrm{\Phi }}^{\prime }(u,v),(u,v)〉={\parallel u\parallel }^2+{\parallel u\parallel }^2+\int a(x)u^2+\int b(x)v^2-2\lambda \int uv-\int \mathrm{\nabla }F(u,v)(u,v).$$

Below we investigate the main properties of Φ on M.

Lemma 3.1 Let (F2) and (F3) hold. Then Φ is bounded from below on M by 0.

Proof

Note that

$$\mathrm{\Phi }{|}_M(u,v)=\int [\frac{1}{2}\mathrm{\nabla }F(u,v)(u,v)-F(u,v)].$$

(3.1)

By (2.3) we have $\mathrm{\Phi }{|}_M(u,v)>0$. □

Define the least energy of (NLS) on M by $c:=inf\mathrm{\Phi }{|}_M$, then $c\ge 0$. Next, we prove M is a manifold. First, we give the following two lemmas, which will be important when proving M is a manifold.

Lemma 3.2 Let (V1) and (F2)-(F4) hold. Assume $(u_n,v_n)\rightharpoonup (u,v)$ in H and $(u,v)\ne (0,0)$. Then for any $\{t_n\}$ with $t_n>0$ and $t_n\to \mathrm{\infty }$, we have

$$\int \frac{F(t_n{u}_n,t_n{v}_n)}{t_n^2}\to \mathrm{\infty }.$$

Moreover, $\mathrm{\Phi }(t_n{u}_n,t_n{v}_n)\to -\mathrm{\infty }$.

Proof Since $(u_n,v_n)\rightharpoonup (u,v)$ in H, we assume that $(u_n,v_n)\to (u,v)$ in $L_{\mathrm{loc}}^2({\mathbb{R}}^N)\times L_{\mathrm{loc}}^2({\mathbb{R}}^N)$, and $(u_n,v_n)\to (u,v)$ a.e. on ${\mathbb{R}}^{2N}$ for a subsequence. By $(u,v)\ne (0,0)$, there exists a positive measure set Ω such that $(u(x),v(x))\ne (0,0)$, $\mathrm{\forall }x\in \mathrm{\Omega }$. Then $t_n|(u_n(x),v_n(x))|\to \mathrm{\infty }$, $x\in \mathrm{\Omega }$. By (F4) we have

$${\int }_{\mathrm{\Omega }}\underline{lim}\frac{F(t_n{u}_n,t_n{v}_n)}{t_n^2(u_n^2+v_n^2)}(u_n^2+v_n^2)=\mathrm{\infty }.$$

Therefore, (2.3) and the Fatou lemma yield that

$$\underline{lim}\int \frac{F(t_n{u}_n,t_n{v}_n)}{t_n^2}=\mathrm{\infty }.$$

Using (2.1) we have

$$\mathrm{\Phi }(t_n(u_n,v_n))\le \frac{t_n^2}{2}[\nu ({\parallel u_n\parallel }^2+{\parallel v_n\parallel }^2)-\int \frac{2F(t_n{u}_n,t_n{v}_n)}{t_n^2}]\to -\mathrm{\infty },$$

since $\{(u_n,v_n)\}$ is bounded in H. □

Lemma 3.3 Let (V1) and (F1)-(F4) hold. Then

• (i) for each $(u,v)\in H\setminus \{(0,0)\}$, there exists $t_{(u,v)}$ such that if $g_{(u,v)}(t):=\mathrm{\Phi }(tu,tv)$, then $g_{(u,v)}^{\mathrm{\prime }}(t)>0$ for $0<t<t_{(u,v)}$ and $g_{(u,v)}^{\mathrm{\prime }}(t)<0$ for $t>t_{(u,v)}$;

• (ii) there exists $\rho >0$ such that $t_{(w,z)}\ge \rho$ for all $(w,z)\in S$;

• (iii) for each compact subset $W\subset S$, there exists a constant $C_W$ such that $t_{(u,v)}\le C_W$ for all $(u,v)\in W$.

Proof

• (i) Note that

  $$g_{(u,v)}^{\mathrm{\prime }}(t)=t[{\parallel u\parallel }^2+{\parallel v\parallel }^2+\int a(x)u^2+\int b(x)v^2-2\lambda \int uv-\int \frac{\mathrm{\nabla }F(tu,tv)(u,v)}{t}].$$

Using (F2), we infer that when t is small enough, $g_{(u,v)}^{\mathrm{\prime }}(t)>0$. By Lemma 3.2 and (2.3), we have

$$\int \frac{\mathrm{\nabla }F(tu,tv)(u,v)}{t}\ge \int \frac{2F(tu,tv)}{t^2}\to \mathrm{\infty },t\to \mathrm{\infty }.$$

Then when t is large enough, $g_{(u,v)}^{\mathrm{\prime }}(t)<0$. Then $g_{(u,v)}$ has maximum points in $(0,\mathrm{\infty })$. Moreover, from (F3) one easily deduces that the critical point of $g_{(u,v)}$ is unique in $(0,\mathrm{\infty })$, and then it is the maximum point. We denote it by $t_{(u,v)}$. Then $g_{(u,v)}^{\mathrm{\prime }}(t)>0$ for $0<t<t_{(u,v)}$ and $g_{(u,v)}^{\mathrm{\prime }}(t)<0$ for $t>t_{(u,v)}$.

• (ii) If $(u,v)\in M$, then

  $${\parallel u\parallel }^2+{\parallel v\parallel }^2+\int a(x)u^2+\int b(x)v^2-2\lambda \int uv=\int \mathrm{\nabla }F(u,v)(u,v).$$

By (2.1) and (2.2), we get

$$\mu ({\parallel u\parallel }^2+{\parallel v\parallel }^2)\le ϵ({|u|}_2^2+{|v|}_2^2)+C_{ϵ}({|u|}_q^q+{|v|}_q^q)\le ϵ({\parallel u\parallel }^2+{\parallel v\parallel }^2)+C{C}_{ϵ}({\parallel u\parallel }^q+{\parallel v\parallel }^q),$$

where $ϵ>0$ is arbitrary. Then

$${\parallel u\parallel }^{q-2}+{\parallel v\parallel }^{q-2}\ge \tilde{C}.$$

So, there exists $\rho >0$ such that

$${\parallel u\parallel }^2+{\parallel v\parallel }^2\ge {\rho }^2\text{for all }(u,v)\in M.$$

(3.2)

Using (i), for $(w,z)\in S$, there exists $t_{(w,z)}>0$ such that $g_{(w,z)}^{\mathrm{\prime }}(t_{(w,z)})=0$. Then $t_{(w,z)}(w,z)\in M$. Then (3.2) yields the conclusion (ii).

• (iii) We argue by contradiction. Suppose that there exist a compact set W and a sequence $\{(u_n,v_n)\}$ such that $\{(u_n,v_n)\}\subset W\subset S$ and $t_{(u_n,v_n)}\to \mathrm{\infty }$. Since W is compact, there exists $(u,v)\in W$ such that $(u_n,v_n)\to (u,v)$ in H. Then Lemma 3.2 implies that $\mathrm{\Phi }(t_{(u_n,v_n)}(u_n,v_n))\to -\mathrm{\infty }$. Contrary to Lemma 3.1 since $t_{(u_n,v_n)}(u_n,v_n)\in M$. This ends the proof. □

Remark 3.1 Lemma 3.3(i) implies that for each $(u,v)\in H\setminus \{(0,0)\}$, there exists a unique $t_{(u,v)}>0$ such that

$$t_{(u,v)}(u,v)\in M\text{and}\mathrm{\Phi }(t_{(u,v)}(u,v))=\underset{t>0}{max}\mathrm{\Phi }(tu,tv).$$

(3.3)

As a consequence of Lemma 3.3(i), we can define the mapping $m:S\to M$ by $m(u,v)=t_{(u,v)}(u,v)$. By Lemma 3.3, [[11], Proposition 3.1(b)] yields the following result.

Lemma 3.4 If (V1) and (F1)-(F4) are satisfied, then m is a homeomorphism between S and M, and M is a manifold.

If M is a $C^1$ manifold, we can make use of the differential structure of M to reduce the problem of finding a ground state for (NLS) into that of looking for a minimizer of $\mathrm{\Phi }{|}_M$ and solve the minimizing problem. However, since $F\in C^1({\mathbb{R}}^2,\mathbb{R})$, M may not be a $C^1$ manifold. Noting that M and S are homeomorphic, we will take advantage of the differential structure of S to seek for ground states for (NLS) as [11]. Therefore, as in [11], we introduce the functional $\mathrm{\Psi }:S\to \mathbb{R}$ defined by $\mathrm{\Psi }(u,v):=\mathrm{\Phi }(m(u,v))$, and we have the following conclusion.

Proposition 3.1 Let (V1) and (F1)-(F4) hold. Then the following results hold:

• (i) If $\{(w_n,z_n)\}$ is a PS sequence for Ψ, then $\{m(w_n,z_n)\}$ is a PS sequence for Φ.

• (ii) $(w,z)$ is a critical point of Ψ if and only if $m(w,z)$ is a nontrivial critical point of Φ. Moreover, ${inf}_S\mathrm{\Psi }={inf}_M\mathrm{\Phi }$.

• (iii) A minimizer of $\mathrm{\Phi }{|}_M$ is a solution of (NLS).

Proof As in the proof of [[11], Corollary 3.3], we can show (i) and (ii). Now, we prove the conclusion (iii). Indeed, let $(u,v)\in M$ such that $\mathrm{\Phi }(u,v)=c$. Then $\mathrm{\Psi }(w,z)=c$, where $(w,z)=m^{-1}(u,v)\in S$. By the conclusion (ii), we have $\mathrm{\Psi }(w,z)={inf}_S\mathrm{\Psi }$. So, ${\mathrm{\Psi }}^{\prime }(w,z)=0$. Using the conclusion (ii) again, we deduce that ${\mathrm{\Phi }}^{\prime }(u,v)=0$. □

From the definition of a ground state, we translate the problem of looking for a ground state for (NLS) into that of seeking for a solution for (NLS) which is a minimizer of $\mathrm{\Phi }{|}_M$. By Proposition 3.1(iii), in order to look for a ground state for (NLS), we just need to seek for a minimizer of $\mathrm{\Phi }{|}_M$.

4 The periodic case

In this section, we consider the periodic case and prove Theorem 1.1. In [11], Szulkin and Weth considered the existence of ground states for periodic single Schrödinger equations. Treating as in [11], we find ground states for a periodic case for the system (NLS). In addition, under conditions (F5) and (F6), we deduce that there are positive ground states.

From the statement in Section 3, it suffices to solve the minimizing problem. By conclusions (i) and (ii) of Proposition 3.1, we first make use of the minimizing sequence of Ψ to obtain a ${(\mathit{PS})}_c$ sequence of Φ. Then we use the invariant of the functional under translation of the form $v\mapsto v(\cdot -y)$, $y\in {\mathbb{Z}}^N$ to recover the compactness for the ${(\mathit{PS})}_c$ sequence.

Proof of Theorem 1.1 Let $({\overline{w}}_n,{\overline{z}}_n)\in S$ be a minimizing sequence of Ψ. By the Ekeland variational principle [[16], Theorem 8.5], we may assume that ${\mathrm{\Psi }}^{\prime }({\overline{w}}_n,{\overline{z}}_n)\to 0$. Using Proposition 3.1(i), we have that ${\mathrm{\Phi }}^{\prime }(u_n,v_n)\to 0$, where $(u_n,v_n)=m({\overline{w}}_n,{\overline{z}}_n)\in M$. Proposition 3.1(ii) implies that $\mathrm{\Phi }(u_n,v_n)=\mathrm{\Psi }({\overline{w}}_n,{\overline{z}}_n)\to {inf}_S\mathrm{\Psi }=c$.

We claim that $\{(u_n,v_n)\}$ is bounded in H. Otherwise, suppose $s_n:=\parallel (u_n,v_n)\parallel \to \mathrm{\infty }$ up to a subsequence. Set $(w_n,z_n)=\frac{(u_n,v_n)}{\parallel (u_n,v_n)\parallel }$. Then we assume $(w_n,z_n)\rightharpoonup (w,z)$ in H, $(w_n,z_n)\to (w,z)$ in $L_{\mathrm{loc}}^2({\mathbb{R}}^N)\times L_{\mathrm{loc}}^2({\mathbb{R}}^N)$ and $(w_n,z_n)\to (w,z)$ a.e. on ${\mathbb{R}}^{2N}$ after passing to a subsequence. Moreover, the Sobolev embedding theorem implies that $\{(w_n,z_n)\}$ is bounded in $L^q({\mathbb{R}}^N)\times L^q({\mathbb{R}}^N)$, namely, $\{{|w_n|}_q+{|z_n|}_q\}$ is bounded. Taking a subsequence, we suppose ${|w_n|}_q+{|z_n|}_q\to A\in [0,\mathrm{\infty })$.

• (i) If $A=0$, then for any $ϵ>0$, there exists $K\in \mathbb{N}$ such that ${|w_n|}_q^q+{|z_n|}_q^q<Cϵ$, for $n>K$. Combining with (2.2), for $n>K$ and $s>0$, we have

  $$\int F(s{w}_n,s{z}_n)\le ϵs^2({|w_n|}_2^2+{|z_n|}_2^2)+s^{q}C{C}_{ϵ}({|w_n|}_q^q+{|z_n|}_q^q)<Cϵ.$$

Then $\int F(s{w}_n,s{z}_n)\to 0$. Hence, by (2.1) we get

$$c+o_n(1)=\mathrm{\Phi }(u_n,v_n)\ge \mathrm{\Phi }(s{w}_n,s{z}_n)\ge \frac{s^2\mu }{2}({\parallel w_n\parallel }^2+{\parallel z_n\parallel }^2)-\int F(s{w}_n,s{z}_n)\to \frac{s^2\mu }{2},$$

a contradiction for $s>\sqrt{\frac{2c}{\mu }}$.

• (ii) If $A\ne 0$, then we can assume that in $L^q({\mathbb{R}}^N)$. From the Lions compactness lemma [[16], Lemma 1.21], it follows that there exist ${\delta }_0>0$ and $x_n\in {\mathbb{R}}^N$ such that

  $${\int }_{B_1(x_n)}{|w_n|}^2>{\delta }_0.$$

  (4.1)

Since Φ and M are invariant by translation of the form $v\mapsto v(\cdot -y)$, $y\in {\mathbb{Z}}^N$, translating $w_n$ if necessary, we may assume $\{x_n\}$ is bounded. Since $w_n\to w$ in $L_{\mathrm{loc}}^2({\mathbb{R}}^N)$, then (4.1) implies $w\ne 0$. Then from Lemma 3.2, we deduce that $\mathrm{\Phi }(s_n{w}_n,s_n{z}_n)\to -\mathrm{\infty }$. This is impossible since $\mathrm{\Phi }(s_n{w}_n,s_n{z}_n)=\mathrm{\Phi }(u_n,v_n)\to c$.

Hence, $\{(u_n,v_n)\}$ is bounded in H. Suppose that $(u_n,v_n)\rightharpoonup (\check{u},\check{v})$ in H, $(u_n,v_n)\to (\check{u},\check{v})$ in $L_{\mathrm{loc}}^2({\mathbb{R}}^N)\times L_{\mathrm{loc}}^2({\mathbb{R}}^N)$ and $(u_n,v_n)\to (\check{u},\check{v})$ a.e. on ${\mathbb{R}}^{2N}$ for a subsequence. Since ${\mathrm{\Phi }}^{\prime }(u_n,v_n)\to 0$, Lemma 2.2 yields ${\mathrm{\Phi }}^{\prime }(\check{u},\check{v})=0$.

We will show that $(\check{u},\check{v})\ne (0,0)$. Similarly, suppose ${|u_n|}_q+{|v_n|}_q\to B\in [0,\mathrm{\infty })$. If $B=0$, then as before, combining with (2.2), we obtain that $\int \mathrm{\nabla }F(u_n,v_n)(u_n,v_n)\to 0$. Hence, by (2.1) we have

$$\begin{array}{rcl}{o}_n(1)& =& 〈{\mathrm{\Phi }}^{\prime }(u_n,v_n),(u_n,v_n)〉\\ \ge & \mu ({\parallel u_n\parallel }^2+{\parallel v_n\parallel }^2)-\int \mathrm{\nabla }F(u_n,v_n)(u_n,v_n)\\ =& \mu ({\parallel u_n\parallel }^2+{\parallel v_n\parallel }^2)+o_n(1).\end{array}$$

Then $(u_n,v_n)\to (0,0)$ in H. This is impossible since $(u_n,v_n)\in M$ and (3.2) holds. Therefore, $B\ne 0$. So, we can assume in $L^q({\mathbb{R}}^N)$. Then the Lions compactness lemma implies that there exist $y_n\in {\mathbb{R}}^N$, $\tilde{\delta }>0$ such that

$${\int }_{B_1(y_n)}{|u_n|}^2>\tilde{\delta }.$$

(4.2)

As before, translating $u_n$ if necessary, we may assume $\{y_n\}$ is bounded. Since (4.2) and $u_n\to \check{u}$ in $L_{\mathrm{loc}}^2({\mathbb{R}}^N)$, we get $\check{u}\ne 0$. Note that ${\mathrm{\Phi }}^{\prime }(\check{u},\check{v})=0$. So, $(\check{u},\check{v})\in M$. Then by (3.1) we get

$$\begin{array}{rcl}c+o_n(1)& =& \mathrm{\Phi }(u_n,v_n)\\ =& \int [\frac{1}{2}\mathrm{\nabla }F(u_n,v_n)(u_n,v_n)-F(u_n,v_n)]\\ \ge & \int [\frac{1}{2}\mathrm{\nabla }F(\check{u},\check{v})(\check{u},\check{v})-F(\check{u},\check{v})]+o_n(1)\\ =& \mathrm{\Phi }(\check{u},\check{v})+o_n(1),\end{array}$$

(4.3)

where (4.3) follows from the Fatou lemma and (2.3). Then $\mathrm{\Phi }(\check{u},\check{v})\le c$. According to $(\check{u},\check{v})\in M$, we have $\mathrm{\Phi }(\check{u},\check{v})\ge c$. Thus, $\mathrm{\Phi }(\check{u},\check{v})=c$. Consequently, $(\check{u},\check{v})$ is a ground state of (NLS).

It remains to look for a positive ground state for (NLS). First, we can assume that $(\check{u},\check{v})$ is non-negative. In fact, note that ${|\mathrm{\nabla }u|}_2^2=|\mathrm{\nabla }|u{||}_2^2$ and ${|\mathrm{\nabla }v|}_2^2=|\mathrm{\nabla }|v{||}_2^2$ for all $(u,v)\in H$. Then $(|\check{u}|,|\check{v}|)\in H$. Let $\tau >0$ be such that $\tau (|\check{u}|,|\check{v}|)\in M$. By (F6) we easily have that $\mathrm{\Phi }(\tau |\check{u}|,\tau |\check{v}|)\le \mathrm{\Phi }(\tau \check{u},\tau \check{v})$. Moreover, $\mathrm{\Phi }(\tau \check{u},\tau \check{v})\le \mathrm{\Phi }(\check{u},\check{v})$ since $(\check{u},\check{v})\in M$. Then $\mathrm{\Phi }(\tau |\check{u}|,\tau |\check{v}|)\le \mathrm{\Phi }(\check{u},\check{v})$. So, $(\tau |\check{u}|,\tau |\check{v}|)$ is also a minimizer of Φ on M. Then $(\tau |\check{u}|,\tau |\check{v}|)$ is also a ground state of (NLS). Thus we can assume that $(\check{u},\check{v})$ is a non-negative ground state for (NLS). Now, we claim that $\check{u}\ne 0$, $\check{v}\ne 0$. Indeed, if $\check{u}=0$, then from (F5) and $\lambda >0$, the first equation of (NLS) yields that $\check{v}=0$. Then $(\check{u},\check{v})=(0,0)$. This is impossible. So, $\check{u}\ne 0$. Similarly, $\check{v}\ne 0$. By (F5), applying the maximum principle to each equation of (NLS), we infer that $\check{u}>0$, $\check{v}>0$. The proof is complete. □

5 The asymptotically periodic case

In this section, we will consider the asymptotically periodic case and prove Theorems 1.2 and 1.3. As in the proof of Theorem 1.1, we first take advantage of the minimizing sequence of Ψ to find a ${(\mathit{PS})}_c$ sequence of Φ. In what follows, the important thing is to recover the compactness for the ${(\mathit{PS})}_c$ sequence. For this purpose, we need to estimate the functional levels of the problem (NLS) and those of a related periodic problem of (NLS) (roughly speaking, the limit system of (NLS) by (V3))

Hence, first we introduce some definitions and look for solutions for the problem (NLS) _p . The functional of (NLS) _p is defined by

$${\mathrm{\Phi }}_p(u,v)=\frac{1}{2}({\parallel u\parallel }^2+{\parallel v\parallel }^2+\int a_p(x)u^2+\int b_p(x)v^2)-\lambda \int uv-\int F(u,v).$$

The Nehari manifold of (NLS) _p is

$$M_p=\{(u,v)\in H\setminus \{(0,0)\}:〈{\mathrm{\Phi }}_p^{\mathrm{\prime }}(u,v),(u,v)〉=0\},$$

and $c_p={inf}_{M_p}{\mathrm{\Phi }}_p$ is the least energy of (NLS) _p on $M_p$. Note that

$${\mathrm{\Phi }}_p{|}_{M_p}(u,v)=\int [\frac{1}{2}\mathrm{\nabla }F(u,v)(u,v)-F(u,v)].$$

(5.1)

As for c, we have $c_p\ge 0$.

Lemma 5.1 Suppose that $a_p$, $b_p$ satisfy (V1) and (V2). Let (F1)-(F6) hold. Then the problem (NLS) _p has a positive ground state $(u,v)\in M_p$ such that ${\mathrm{\Phi }}_p(u,v)=c_p$.

Proof As a corollary of Theorem 1.1, we infer that the problem (NLS) _p has a positive ground state. Moreover, from the argument of Theorem 1.1, we find that the ground state of the problem (NLS) _p we obtained is a minimizer of ${\mathrm{\Phi }}_p$ on $M_p$. □

The existence of a positive ground state for the problem (NLS) _p implies that (NLS) has a positive ground state when $a=a_p$ and $b=b_p$. So, it remains to consider

$$a\ne a_p\text{or}b\ne b_p.$$

(5.2)

Next, we prove that $c<c_p$ under some conditions.

Lemma 5.2 Suppose that $a_p$, $b_p$ satisfy (V2). Let (V1), (V4), (5.2) and (F1)-(F6) hold. Then $c<c_p$.

Proof Let $(u_0,v_0)\in M_p$ be a positive ground state of (NLS) _p such that ${\mathrm{\Phi }}_p(u_0,v_0)=c_p$. Assume $t>0$ satisfies $t(u_0,v_0)\in M$. By (V4), we get

$$\int [(a(x)-a_p(x))u_0^2+(b(x)-b_p(x))v_0^2]\le 0.$$

Then $\mathrm{\Phi }(t{u}_0,t{v}_0)\le {\mathrm{\Phi }}_p(t{u}_0,t{v}_0)$.

Replacing Φ and M by ${\mathrm{\Phi }}_p$ and $M_p$ respectively, (3.3) also holds. Noting that $(u_0,v_0)\in M_p$, we infer that

$${\mathrm{\Phi }}_p(t{u}_0,t{v}_0)\le {\mathrm{\Phi }}_p(u_0,v_0)\text{and}{\mathrm{\Phi }}_p(t{u}_0,t{v}_0)={\mathrm{\Phi }}_p(u_0,v_0)\text{if and only if }t=1.$$

(5.3)

Therefore,

$$c\le \mathrm{\Phi }(t{u}_0,t{v}_0)\le {\mathrm{\Phi }}_p(t{u}_0,t{v}_0)\le {\mathrm{\Phi }}_p(u_0,v_0)=c_p.$$

(5.4)

If $c<c_p$, we are done. Otherwise, $c=c_p$. Then by (5.3) and (5.4), we get $t=1$ and $\mathrm{\Phi }(u_0,v_0)=c$. Then $(u_0,v_0)$ is a ground state for (NLS). Note that $(u_0,v_0)$ is a solution of (NLS) _p . From the first equations of (NLS) and (NLS) _p , we infer that $a=a_p$. Similarly, $b=b_p$ contrary to (5.2). The proof is now complete. □

Lemma 5.3 Suppose that $a_p$, $b_p$ satisfy (V1) and (V2). Let (V1), (V5), (5.2) and (F1)-(F7) hold. Then $c<c_p$.

Proof Let $(u_0,v_0)\in M_p$ be a positive ground state of (NLS) _p such that ${\mathrm{\Phi }}_p(u_0,v_0)=c_p$. By (V5) and (F7), we find that $(v_0,u_0)$ is also a minimizer of ${\mathrm{\Phi }}_p$ on $M_p$. Let $t,\tau >0$ be such that $t(u_0,v_0),\tau (v_0,u_0)\in M$. Using (V5), we have $\int (a(x)+b(x)-2V(x))(u_0^2+v_0^2)\le 0$. Then

$$\begin{array}{c}\int [(a(x)-V(x))u_0^2+(b(x)-V(x))v_0^2]\le 0\text{or}\hfill \\ \int [(a(x)-V(x))v_0^2+(b(x)-V(x))u_0^2]\le 0.\hfill \end{array}$$

Without loss of generality, we assume that

$$\int [(a(x)-V(x))u_0^2+(b(x)-V(x))v_0^2]\le 0.$$

Then $\mathrm{\Phi }(t{u}_0,t{v}_0)\le {\mathrm{\Phi }}_p(t{u}_0,t{v}_0)$. Below we argue analogously with the proof of Lemma 5.2 to infer that $c<c_p$. This ends the proof. □

Now, we are ready to prove Theorems 1.2 and 1.3. The proof is partially inspired by [17], where the authors dealt with Schrödinger-Poisson equations.

Proof of Theorem 1.2 As the argument of Theorem 1.1, we infer that there exists a sequence $(u_n,v_n)\in M$ such that ${\mathrm{\Phi }}^{\prime }(u_n,v_n)\to 0$ and $\mathrm{\Phi }(u_n,v_n)\to c$.

We claim that $\{(u_n,v_n)\}$ is bounded in H. Otherwise, suppose $t_n:=\parallel (u_n,v_n)\parallel \to \mathrm{\infty }$ up to a subsequence. Set $(w_n,z_n)=\frac{(u_n,v_n)}{\parallel (u_n,v_n)\parallel }$. As in the proof of Theorem 1.1, taking a subsequence, we suppose ${|w_n|}_q+{|z_n|}_q\to A\in [0,\mathrm{\infty })$ and exclude the case that $A=0$. So, $A\ne 0$, then we can assume that in $L^q({\mathbb{R}}^N)$. From the Lions compactness lemma, it follows that there exist ${\delta }_0>0$ and $y_n\in {\mathbb{R}}^N$ such that

$${\int }_{B_1(y_n)}{|w_n|}^2>{\delta }_0.$$

Set ${\tilde{w}}_n=w_n(\cdot +y_n)$ and ${\tilde{z}}_n=z_n(\cdot +y_n)$. We assume that $({\tilde{w}}_n,{\tilde{z}}_n)\rightharpoonup (\tilde{w},\tilde{z})$ in H, $({\tilde{w}}_n,{\tilde{z}}_n)\to (\tilde{w},\tilde{z})$ in $L_{\mathrm{loc}}^2({\mathbb{R}}^N)\times L_{\mathrm{loc}}^2({\mathbb{R}}^N)$ and $({\tilde{w}}_n,{\tilde{z}}_n)\to (\tilde{w},\tilde{z})$ a.e. on ${\mathbb{R}}^{2N}$ up to a subsequence. Then by

$${\int }_{B_1(y_n)}{|w_n|}^2>{\delta }_0,$$

we obtain $\tilde{w}\ne 0$. So, Lemma 3.2 implies that

$$\int \frac{F(t_n{\tilde{w}}_n,t_n{\tilde{z}}_n)}{t_n^2}\to \mathrm{\infty }.$$

Then by (2.1), we get

$$\begin{array}{rcl}0& \le & \frac{\mathrm{\Phi }(u_n,v_n)}{{\parallel (u_n,v_n)\parallel }^2}\\ =& \frac{1}{2}\left[\frac{{\parallel u_n\parallel }^2+{\parallel v_n\parallel }^2+\int a(x)u_n^2+\int b(x)v_n^2-2\lambda \int u_n{v}_n}{{\parallel (u_n,v_n)\parallel }^2}\right]-\int \frac{F(t_n{w}_n,t_n{z}_n)}{t_n^2}\\ \le & \frac{\nu }{2}-\int \frac{F(t_n{\tilde{w}}_n,t_n{\tilde{z}}_n)}{t_n^2}\to -\mathrm{\infty }.\end{array}$$

This is a contradiction.

Hence, $\{(u_n,v_n)\}$ is bounded in H. Up to a subsequence, we assume that $(u_n,v_n)\rightharpoonup (\tilde{u},\tilde{v})$ in H, $(u_n,v_n)\to (\tilde{u},\tilde{v})$ in $L_{\mathrm{loc}}^2({\mathbb{R}}^N)\times L_{\mathrm{loc}}^2({\mathbb{R}}^N)$ and $(u_n,v_n)\to (\tilde{u},\tilde{v})$ a.e. on ${\mathbb{R}}^{2N}$. By Lemma 2.2, we have ${\mathrm{\Phi }}^{\prime }(\tilde{u},\tilde{v})=0$. Namely, $(\tilde{u},\tilde{v})$ is a solution of (NLS).

Below we prove that $(\tilde{u},\tilde{v})\ne (0,0)$. We argue by contradiction. Suppose that $(\tilde{u},\tilde{v})=(0,0)$. By (V3), for any $ϵ>0$, there exists $r>0$ such that

$$|a(x)-a_p(x)|<ϵ,|b(x)-b_p(x)|<ϵ\text{for all }|x|>r.$$

(5.5)

Note that $(\tilde{u},\tilde{v})=(0,0)$, after passing to a subsequence, we assume $(u_n,v_n)\to (0,0)$ in $L^2(B_r(0))\times L^2(B_r(0))$. So, for the above ϵ, there exists $J_1\in \mathbb{N}$ such that for $n>J_1$, we have

$${\int }_{B_r(0)}{u}_n^2<ϵ,{\int }_{B_r(0)}{v}_n^2<ϵ.$$

Combining with (5.5), for $n>J_1$, we get

$$|\int (a(x)-a_p(x))u_n^2|\le {\int }_{B_r(0)}|a(x)-a_p(x)|u_n^2+ϵ{\int }_{{\mathbb{R}}^N\setminus B_r(0)}{u}_n^2<({|a|}_{\mathrm{\infty }}+{|a_p|}_{\mathrm{\infty }})ϵ+Cϵ.$$

Then $\int (a(x)-a_p(x))u_n^2\to 0$. Similarly, $\int (b(x)-b_p(x))v_n^2\to 0$. Therefore,

$${\mathrm{\Phi }}_p(u_n,v_n)=\mathrm{\Phi }(u_n,v_n)+o_n(1),〈{\mathrm{\Phi }}_p^{\mathrm{\prime }}(u_n,v_n),(u_n,v_n)〉=〈{\mathrm{\Phi }}^{\prime }(u_n,v_n),(u_n,v_n)〉+o_n(1).$$

Hence,

$${\mathrm{\Phi }}_p(u_n,v_n)=c+o_n(1),〈{\mathrm{\Phi }}_p^{\mathrm{\prime }}(u_n,v_n),(u_n,v_n)〉=o_n(1).$$

(5.6)

Let $s_n>0$ be such that $s_n(u_n,v_n)\in M_p$. We claim that $s_n\ge 1$ for large n and $s_n\to 1$.

First, we prove that

$$\underset{n\to \mathrm{\infty }}{lim\hspace{0.17em}sup}{s}_n\le 1.$$

(5.7)

Otherwise, there exist $\delta >0$ and a subsequence of $s_n$, still denoted by $s_n$, such that $s_n\ge 1+\delta$ for all $n\in \mathbb{N}$. From (5.6) we have

$${\parallel u_n\parallel }^2+{\parallel v_n\parallel }^2+\int a_p(x)u_n^2+\int b_p(x)v_n^2-2\lambda \int u_n{v}_n=\int \mathrm{\nabla }F(u_n,v_n)(u_n,v_n)+o_n(1).$$

Moreover, by $s_n(u_n,v_n)\in M_p$, we get

$$\begin{array}{c}{s}_n^2[{\parallel u_n\parallel }^2+{\parallel v_n\parallel }^2+\int a_p(x)u_n^2+\int b_p(x)v_n^2-2\lambda \int u_n{v}_n]\hfill \\ =\int \mathrm{\nabla }F(s_n{u}_n,s_n{v}_n)(s_n{u}_n,s_n{v}_n).\hfill \end{array}$$

Hence,

$$\int [\frac{\mathrm{\nabla }F(s_n{u}_n,s_n{v}_n)(u_n,v_n)}{s_n}-\mathrm{\nabla }F(u_n,v_n)(u_n,v_n)]=o_n(1).$$

By $s_n\ge 1+\delta$ and (F3), we obtain

$$\int [\frac{\mathrm{\nabla }F((1+\delta )u_n,(1+\delta )v_n)(u_n,v_n)}{1+\delta }-\mathrm{\nabla }F(u_n,v_n)(u_n,v_n)]\le o_n(1).$$

(5.8)

Similar to the proof of Theorem 1.1, if $(u_n,v_n)\to (0,0)$ in $L^q({\mathbb{R}}^N)\times L^q({\mathbb{R}}^N)$, then $(u_n,v_n)\to (0,0)$ in H. Contrary to (3.2), since $(u_n,v_n)\in M$, therefore, in $L^q({\mathbb{R}}^N)\times L^q({\mathbb{R}}^N)$. Suppose in $L^q({\mathbb{R}}^N)$. Then from the Lions compactness lemma, it follows that there exist $x_n\in {\mathbb{R}}^N$ and ${\delta }_1>0$ such that

$${\int }_{B_1(x_n)}{u}_n^2>{\delta }_1.$$

(5.9)

We denote ${\overline{u}}_n$ and ${\overline{v}}_n$ by ${\overline{u}}_n=u_n(\cdot +x_n)$ and ${\overline{v}}_n=v_n(\cdot +x_n)$. Similarly, we assume that $({\overline{u}}_n,{\overline{v}}_n)\rightharpoonup (\overline{u},\overline{v})$ in H, $({\overline{u}}_n,{\overline{v}}_n)\to (\overline{u},\overline{v})$ in $L_{\mathrm{loc}}^2({\mathbb{R}}^N)\times L_{\mathrm{loc}}^2({\mathbb{R}}^N)$ and $({\overline{u}}_n,{\overline{u}}_n)\to (\overline{u},\overline{v})$ a.e. on ${\mathbb{R}}^{2N}$ up to a subsequence. By (5.9), we have

$${\int }_{B_1(0)}{\overline{u}}_n^2>{\delta }_1.$$

So, $\overline{u}\ne 0$. From (5.8), (F3) and the Fatou lemma, we obtain

$$0<\int [\frac{\mathrm{\nabla }F((1+\delta )\overline{u},(1+\delta )\overline{v})(\overline{u},\overline{v})}{1+\delta }-\mathrm{\nabla }F(\overline{u},\overline{v})(\overline{u},\overline{v})]\le 0,$$

which is impossible. Consequently, (5.7) holds.

Now, we show that $s_n\ge 1$ for large n. Indeed, on the contrary, passing to a subsequence, we assume that $s_n<1$. Using (3.1) and (5.1), we have

$$\begin{array}{rcl}{c}_p& \le & {\mathrm{\Phi }}_p(s_n{u}_n,s_n{v}_n)=\int [\frac{1}{2}\mathrm{\nabla }F(s_n{u}_n,s_n{v}_n)(s_n{u}_n,s_n{v}_n)-F(s_n{u}_n,s_n{v}_n)]\\ \le & \int [\frac{1}{2}\mathrm{\nabla }F(u_n,v_n)(u_n,v_n)-F(u_n,v_n)]\\ =& \mathrm{\Phi }(u_n,v_n)+o_n(1)=c+o_n(1),\end{array}$$

(5.10)

where (5.10) follows from the fact that α is increasing in $(0,\mathrm{\infty })$ by Lemma 2.1. Then $c_p\le c$, contrary to Lemma 5.2. Therefore, combining with (5.7), we may assume that

$$s_n\ge 1,\text{for large }n\text{and}\underset{n\to \mathrm{\infty }}{lim}{s}_n=1.$$

(5.11)

For $ϵ>0$ and $1\le s\le s_n$, using (2.2) we get

$$|\int \mathrm{\nabla }F(s{u}_n,s{v}_n)(u_n,v_n)|\le ϵs_n({|u_n|}_2^2+{|v_n|}_2^2)+C_{ϵ}{s}_n^{q-1}({|u_n|}_q^q+{|v_n|}_q^q).$$

(5.12)

Combining (5.11) with (5.12), one easily has that

$$\int [F(s_n{u}_n,s_n{v}_n)-F(u_n,v_n)]={\int }_1^{s_n}[\int \mathrm{\nabla }F(s{u}_n,s{v}_n)(u_n,v_n)dx]ds=o_n(1).$$

Since $a_p,b_p\in L^{\mathrm{\infty }}({\mathbb{R}}^N)$ and $\{(u_n,v_n)\}$ is bounded, we get

$$\frac{s_n^2-1}{2}[{\parallel u_n\parallel }^2+{\parallel v_n\parallel }^2+\int a_p(x)u_n^2+\int b_p(x)v_n^2-2\lambda \int u_n{v}_n]=o_n(1).$$

Hence, ${\mathrm{\Phi }}_p(s_n{u}_n,s_n{v}_n)={\mathrm{\Phi }}_p(u_n,v_n)+o_n(1)$. Then using (5.6), we have $c_p\le {\mathrm{\Phi }}_p(s_n{u}_n,s_n{v}_n)=c+o_n(1)$. Then $c_p\le c$. However, Lemma 5.2 implies that $c<c_p$. This is a contradiction. Note that this contradiction follows from the hypothesis that $(\tilde{u},\tilde{v})=(0,0)$. So, $(\tilde{u},\tilde{v})\ne (0,0)$. Then $(\tilde{u},\tilde{v})\in M$.

It suffices to show that $\mathrm{\Phi }(\tilde{u},\tilde{v})=c$. By (3.1) we have

$$\begin{array}{rcl}c+o_n(1)& =& \mathrm{\Phi }(u_n,v_n)=\int [\frac{1}{2}\mathrm{\nabla }F(u_n,v_n)(u_n,v_n)-F(u_n,v_n)]\\ \ge & \int [\frac{1}{2}\mathrm{\nabla }F(\tilde{u},\tilde{v})(\tilde{u},\tilde{v})-F(\tilde{u},\tilde{v})]+o_n(1)\\ =& \mathrm{\Phi }(\tilde{u},\tilde{v})+o_n(1),\end{array}$$

(5.13)

where the inequality (5.13) holds by (2.3) and the Fatou lemma. Then $\mathrm{\Phi }(\tilde{u},\tilde{v})\le c$. According to $(\tilde{u},\tilde{v})\in M$, we have $\mathrm{\Phi }(\tilde{u},\tilde{v})=c$. Then $(\tilde{u},\tilde{v})$ is a ground state for (NLS). Below we argue analogously with the proof of Theorem 1.1 to get a positive ground state for (NLS). The proof is complete. □

Proof of Theorem 1.3 By Lemma 5.3, repeating the argument of Theorem 1.2, we show the existence of a ground state for (NLS) and then look for a positive ground state as the argument of Theorem 1.1. □