Existence of weak solutions for a class of $(p,q)$-Laplacian systems

Abstract

The existence of weak solutions for a class of $(p,q)$-Laplacian systems is obtained by using a new linking theorem on the product space $W^{1,p}_{0}(\Omega)\times W^{1,q}_{0}(\Omega)$.

1 Introduction and main results

In this paper, we study the existence of weak solutions for the following $(p,q)$-Laplacian system:

$$ \textstyle\begin{cases} -\Delta_{p}u=\frac{1}{\alpha+1}G_{u}(x,u,v) & \mbox{in } \Omega,\\ -\Delta_{q}v=\frac{1}{\beta+1}G_{v}(x,u,v) & \mbox{in } \Omega,\\ u=v=0 &\mbox{on } \partial\Omega, \end{cases} $$

(1)

where Ω is a bounded domain with smooth boundary in $R^{N}$ ($N\geq 3$), $1< p,q< N$, $-\Delta_{p}u=-\operatorname{div}( \vert \nabla u \vert ^{p-2}\nabla u)$ is the p-Laplacian operator, the nonlinearity $G\in C(\bar{\Omega}\times R^{2}, R)$ has the continuous derivatives $G_{s}(x,s,t),G_{t}(x,s,t)$ with respect to s and t for any $x\in\Omega$, and there exist $p< p_{1}< p^{*}$, $q< q_{1}< q^{*}$, and $c_{0}>0$ such that

$$\begin{aligned} \bigl\vert G_{s}(x,s,t) \bigr\vert \leq&c_{0}\bigl(1+ \vert s \vert ^{p_{1}-1}+ \vert t \vert ^{q_{1}(p_{1}-1)/p_{1}}\bigr), \end{aligned}$$

(2)

$$\begin{aligned} \bigl\vert G_{t}(x,s,t) \bigr\vert \leq&c_{0}\bigl(1+ \vert s \vert ^{p_{1}(q_{1}-1)/q_{1}}+ \vert t \vert ^{q_{1}-1}\bigr), \end{aligned}$$

(3)

for any $(x,s,t)\in\Omega\times R^{2}$, where $p^{*}:=\frac{Np}{N-p}$ is the critical Sobolev exponent of p, as is the case for $q^{*}$.

Let $W=W^{1,p}_{0}(\Omega)\times W^{1,q}_{0}(\Omega)$ be the product space with the norm

$$\bigl\Vert (u,v) \bigr\Vert = \Vert u \Vert _{p}+ \Vert v \Vert _{q} \quad \mbox{for any }(u,v)\in W, $$

where $W^{1,p}_{0}(\Omega)$ is the usual Banach space with the norm $\Vert u \Vert _{p}= (\int_{\Omega} \vert \nabla u \vert ^{p}\,dx )^{1/p}$ for any $u\in W^{1,p}_{0}(\Omega)$. From Sobolev’s embedding theorem, the embedding $W^{1,p}_{0}(\Omega)\hookrightarrow L^{\theta}(\Omega)$ is continuous and compact for any $\theta\in(1,p^{*})$ and there is a constant $C=C(N,\theta,\Omega)>0$ such that

$$ \Vert u \Vert _{L^{\theta}}\leq C \Vert u \Vert _{p}\quad \forall u\in W^{1,p}_{0}(\Omega), $$

(4)

where $\Vert \cdot \Vert _{L^{\theta}}$ denotes the norm of $L^{\theta}(\Omega)$. Throughout this paper, C always denotes an embedding constant with relation to (4).

In the past decades, many authors have considered the existence and multiplicity of weak solutions for the elliptic equations and the elliptic systems by the variational method (see [1–3] for the semilinear elliptic equations, [4, 5] for the semilinear elliptic systems, [6–9] for p-Laplacian equations, [10, 11] for p-Laplacian systems, [12–16] for $(p,q)$-elliptic systems, and references therein). The well known mountain pass theorem, the saddle point theorem, and the linking theorem by Rabinowitz (see [17]) are the three very important abstract critical point theorems to study the existence of weak solutions for a class of semilinear elliptic equations and elliptic systems with variational structure. However, because $-\Delta_{p}$ is no longer a linear operator and $W^{1,p}_{0}(\Omega)$ is a Banach space, the saddle point theorem and the linking theorem fail for the p-Laplacian equations and p-Laplacian systems. In 2007, with the aid of the $Z_{2}$-cohomological index of Fadell and Rabinowitz, Degiovanni and Lancelotti [18] established new linking structures over cones, corresponding to the saddle point theorem and the linking theorem by Rabinowitz, which have been widely applied to investigate the existence of weak solutions for p-Laplacian equations and systems, where a set E of $W^{1,p}_{0}(\Omega)$ is said to be a cone, if $tu\in E$ for any $u\in E$ and $t>0$. The problem becomes more complicated for $(p,q)$-Laplacian systems, but there are many papers to study the existence and multiplicity of nontrivial solutions for $(p,q)$-Laplacian systems (see, for example, [15, 16] for the resonance case, [12, 13] for the superquadratic case, and [14] for the critical case). Especially, in [4], under the asymptotic noncrossing conditions and the nonquadraticity conditions, Costa proved that there was at least a weak solution for the semilinear elliptic systems by using the saddle point theorem and they proved there was at least a nontrivial solution for the semilinear elliptic systems under the crossing condition by using the linking theorem. In [13], with the aid of the mountain pass theorem, Boccardo and Guedes De Figueiredo proved that there exists a nontrivial solution of system (1) with $\alpha=\beta=0$ when the crossing condition happens at the first eigenvalue of some kind of the eigenvalue problem with the weights. In [19], Ou and Tang also proved that there was a nontrivial solution of system (1) where the nonlinearity crossed two eigenvalues of the corresponding eigenvalue problem.

Inspired by [4, 13, 18], in this paper, we will extend the linking structures over cones to the product space $W=W^{1,p}_{0}(\Omega)\times W^{1,q}_{0}(\Omega)$ and then study the existence of weak solutions for system (1) under the asymptotic noncrossing condition and the crossing condition, respectively.

We first recall the following nonlinear eigenvalue problem (the details can be found in [17]):

$$ \textstyle\begin{cases} -\Delta_{p}u=\lambda \vert u \vert ^{p-2}u+\lambda \vert u \vert ^{\alpha-1} \vert v \vert ^{\beta+1}u & \mbox{in } \Omega ,\\ -\Delta_{q}v=\lambda \vert v \vert ^{q-2}v+\lambda \vert u \vert ^{\alpha+1} \vert v \vert ^{\beta-1}v & \mbox{in } \Omega ,\\ u=v=0 & \mbox{on } \partial\Omega, \end{cases} $$

(5)

where $\alpha\geq0$, $\beta\geq0$ satisfy

$$\frac{\alpha+1}{p}+\frac{\beta+1}{q}=1. $$

Define the functionals $\phi,\varphi$ on W as follows:

$$\begin{aligned} &\phi(u,v)=\frac{\alpha+1}{p} \int_{\Omega} \vert \nabla u \vert ^{p}\,dx + \frac{\beta+1}{q} \int_{\Omega} \vert \nabla v \vert ^{q}\,dx, \\ &\varphi(u,v)=\frac{\alpha+1}{p} \int_{\Omega} \vert u \vert ^{p}\,dx + \frac{\beta+1}{q} \int_{\Omega} \vert v \vert ^{q}\,dx+ \int_{\Omega} \vert u \vert ^{\alpha+1} \vert v \vert ^{\beta+1}\,dx. \end{aligned}$$

Define the manifold

$$\Sigma=\bigl\{ (u,v)\in W:\varphi(u,v)=1\bigr\} . $$

We can easily prove that $\phi(u,v),\varphi(u,v)$ are $(p,q)$-homogeneous, i.e.,

$$\phi\bigl(t^{1/p}u,t^{1/q}v\bigr)=t\phi(u,v),\qquad \varphi \bigl(t^{1/p}u,t^{1/q}v\bigr)=t\varphi(u,v)\quad \mbox{for any } t>0 \mbox{ and }(u,v)\in W, $$

and Σ is a symmetric nonempty manifold in W. Denote by i the $Z_{2}$-cohomological index of Fadell and Rabinowitz (see [20]) and ${\mathcal{A}}=\{A\subset\Sigma: A \mbox{ is a compact symmetric set}\}$. Define

$$\Sigma_{k}=\bigl\{ A\in{\mathcal{A}}: i(A)\geq k\bigr\} . $$

The eigenvalues of problem (5) can be variationally characterized as follows:

$$\lambda_{k}=\inf_{A\in\Sigma_{k}}\sup_{(u,v)\in A} \phi(u,v), $$

where $0<\lambda_{1}<\lambda_{2}\leq\cdots\leq\lambda_{k}\leq\cdots$ , $\lambda _{k}\to\infty$ as $k\to\infty$. It is not clear whether the sequence $\{ \lambda_{k}\}_{k\in N}$ contains all the eigenvalues of problem (5), but we will refer to $\{\lambda_{k}\}_{k\in N}$ as the variational eigenvalues of problem (5). Since $\phi(u,v),\varphi(u,v)$ are $(p,q)$-homogeneous, the eigenfunction space corresponding to $\lambda _{k}$

$$E_{\lambda_{k}}:=\bigl\{ (u,v)\in W:\phi(u,v)=\lambda_{k}\varphi(u,v) \bigr\} $$

is not the cone, but is a $(p,q)$-set, that is,

$$\bigl(t^{\frac{1}{p}}u,t^{\frac{1}{q}}v\bigr)\in E_{\lambda_{k}}\quad \mbox{for any } t\geq0 \mbox{ and }(u,v)\in E_{\lambda_{k}}. $$

Let

$$\begin{aligned} &F(s,t):=\frac{\alpha+1}{p} \vert s \vert ^{p}+\frac{\beta+1}{q} \vert t \vert ^{q}+ \vert s \vert ^{\alpha+1} \vert t \vert ^{\beta+1} \quad \mbox{for any }(s,t)\in R^{2}, \\ &H(x,s,t):=\frac{1}{p}G_{s}(x,s,t)s+\frac{1}{q}G_{t}(x,s,t)t-G(x,s,t) \quad \mbox{for any }(x,s,t)\in\Omega\times R^{2}. \end{aligned}$$

The main results of this paper are the following theorems.

Theorem 1

Assume (2) and (3) and suppose $\lambda _{k}<\lambda_{k+1}$ are two consecutive eigenvalues of (5). If the following conditions hold:

$$\begin{aligned} &\lim_{ \vert (s,t) \vert \to\infty}H(x,s,t)=+\infty\quad \textit{uniformly for }x\in\Omega, \end{aligned}$$

(6)

$$\begin{aligned} &\lambda_{k}< \liminf_{ \vert (s,t) \vert \to\infty} \frac{G(x,s,t) }{F(s,t)}\leq\limsup_{ \vert (s,t) \vert \to\infty}\frac{G(x,s,t) }{F(s,t)}\leq \lambda_{k+1} \quad \textit{uniformly for }x\in\Omega, \end{aligned}$$

(7)

then system (1) has at least a weak solution.

Theorem 2

Assume (2) and (3) and suppose $\lambda _{k}<\lambda_{k+1}$ are two consecutive eigenvalues of (5). If the nonlinearity G satisfies the following conditions:

$$\begin{aligned} &\lim_{ \vert (s,t) \vert \to\infty}H(x,s,t)=-\infty\quad \textit{uniformly for }x\in\Omega, \end{aligned}$$

(8)

$$\begin{aligned} &\lambda_{k}\leq\liminf_{ \vert (s,t) \vert \to\infty} \frac{G(x,s,t) }{F(s,t)}\leq\limsup_{ \vert (s,t) \vert \to\infty}\frac {G(x,s,t)}{F(s,t)}< \lambda_{k+1} \quad \textit{uniformly for }x\in\Omega, \end{aligned}$$

(9)

then system (1) has at least a weak solution.

Let $\mu_{0}=\max\{\frac{p^{*}(p_{1}-1)}{p^{*}-1}, \frac {(q_{1}-1)p_{1}q^{*}}{q_{1}(q^{*}-1)}\}, \nu_{0}=\max\{\frac{q^{*}(q_{1}-1)}{q^{*}-1}, \frac{(p_{1}-1)q_{1}p^{*}}{p_{1}(p^{*}-1)}\}$. We assume

$$ \mu> \textstyle\begin{cases} \max\{\frac{(q_{1}-1)p_{1}}{q_{1}(p-1)},\mu_{0}\} & \mbox{if } p_{1}\leq q_{1},\\ \max\{\frac{p_{1}-1}{p-1},\mu_{0}\} & \mbox{if } p_{1}< q_{1}, \end{cases}\displaystyle \qquad \nu> \textstyle\begin{cases} \max\{\frac{q_{1}-1}{p-1},\nu_{0}\} & \mbox{if } p_{1}\leq q_{1},\\ \max\{\frac{(p_{1}-1)q_{1}}{p_{1}(p-1)}\nu_{0}\} & \mbox{if } p_{1}< q_{1}, \end{cases} $$

(10)

for the case $p\leq q$, and

$$ \mu> \textstyle\begin{cases} \max\{\frac{(q_{1}-1)p_{1}}{q_{1}(q-1)},\mu_{0}\} & \mbox{if } p_{1}\leq q_{1},\\ \max\{\frac{p_{1}-1}{q-1},\mu_{0}\} & \mbox{if } p_{1}< q_{1}, \end{cases}\displaystyle \qquad \nu> \textstyle\begin{cases} \max\{\frac{q_{1}-1}{q-1},\nu_{0}\} & \mbox{if } p_{1}\leq q_{1}, \\ \max\{\frac{(p_{1}-1)q_{1}}{p_{1}(q-1)},\nu_{0}\} & \mbox{if } p_{1}< q_{1}, \end{cases} $$

(11)

for the case $p>q$.

Theorem 3

Assume that $\lambda_{k}<\lambda_{k+1}$ are two consecutive eigenvalues of (5) and suppose the nonlinearity G satisfies (2), (3), and the crossing conditions

$$\begin{aligned} &G(x,s,t)\geq\lambda_{k}F(s,t) \quad \forall(x,s,t)\in \bar{\Omega}\times R^{2}, \end{aligned}$$

(12)

$$\begin{aligned} &\limsup_{ \vert (s,t) \vert \to0}\frac{G(x,s,t)}{F(s,t)}\leq\alpha _{0}< \lambda_{k+1} < \beta_{0}\leq\liminf _{ \vert (s,t) \vert \to\infty}\frac {G(x,s,t)}{F(s,t)}\quad \textit{uniformly for } x\in\Omega, \end{aligned}$$

(13)

where $\alpha_{0}, \beta_{0}$ are the two constants. If one of the following conditions holds:

$$ \liminf_{ \vert (s,t) \vert \to\infty}\frac{H(x,s,t)}{ \vert s \vert ^{\mu}+ \vert t \vert ^{\nu}}\geq a>0 \quad \textit{uniformly for }x\in\Omega, $$

(14)

or

$$ \limsup_{ \vert (s,t) \vert \to\infty}\frac{H(x,s,t)}{ \vert s \vert ^{\mu}+ \vert t \vert ^{\nu}} \leq-a< 0 \quad \textit{uniformly for }x\in\Omega, $$

(15)

where a is a positive constant and $\mu,\nu$ satisfy (10) or (11), then system (1) has at least a nontrivial weak solution in W.

Remark

The conditions (6), (8), (14), and (15) are the generalizations of the nonquadraticity conditions from the semilinear elliptic equation to the $(p,q)$-elliptic systems, (7) and (9) are the asymptotic noncrossing conditions, and (13) is the crossing condition, introduced by Costa and Magalhaés in [2] and used to study the existence of weak solutions for the elliptic equations and the elliptic systems (see [4, 6–9] and the references therein).

Costa and Magalhaés [2] have proved the existence of a weak solution for the semilinear elliptic equation under the nonquadraticity conditions and the asymptotic noncrossing conditions by using the saddle point theorem by Rabinowitz (see [17]) and the existence of a nontrivial solution under the nonquadraticity conditions and the crossing condition by using the linking theorem by Rabinowitz (see [17]). As for the quasilinear elliptic equation, by using an abstract critical point theory, El Amrouss and Moussaoui in [6] proved the same result with Theorem 2 with $k=1$. Ou and Li in [8] obtained the same result with Theorem 1 by using the G-linking theorem, where the eigenvalues of $-\Delta_{p}$ are defined by the cogenus. In [9] Yuan and Ou proved the same conclusions as in [2] by using the linking theorem over cones by Degiovanni and Lancelotti (see [18]), where the eigenvalues of $-\Delta_{p}$ are defined by the $Z_{2}$-cohomological index. However, as for the $(p,q)$-elliptic systems, because the functional $\phi(u,v)$, $\varphi(u,v)$ are $(p,q)$-homogeneous, the eigenfunction space corresponding to the eigenvalue $\lambda_{k}$ ($k\geq1$) of problem (5) is not a cone. Hence the linking theorem over cones cannot be applied to prove our theorems and we must prove a new linking theorem on the product space $W^{1,p}_{0}(\Omega)\times W^{1,q}_{0}(\Omega)$ (see Lemma 3).

As is well known, Perera and Schechter in [11, 21, 22] also proved similar results with Theorem 1 and Theorem 2 by using the notion of sandwich pairs, in [21, 22] for p-Laplacian problems and in [11] for p-Laplacian systems. However, our results are different from Theorem 4.3 of [11]. Condition (7) is about the growth of $G(x,s,t)$ when $\vert (s,t) \vert \rightarrow\infty$, while condition (4.4) of Theorem 4.3 in [11] is a global condition, that is,

$$\lambda_{k}J(x,u)-W(x)\leq F(x,u)\leq\lambda_{k+1}J(x,u)+W(x) \quad \forall(x,u)\in\Omega\times R^{m} $$

for some $W(x)\in L^{1}(\Omega)$. On the other hand, our conditions (6) and (8) are weaker than conditions (i) and (ii) of Lemma 4.2 of [11], which result in the $(Ce)_{c}$ condition and the $(PS)_{c}$ condition, respectively.

2 Proofs of theorems

We define the functional $J:W\to R$ as follows:

$$\begin{aligned} J(u,v)=\frac{\alpha+1}{p} \int_{\Omega} \vert \nabla u \vert ^{p}\,dx + \frac{\beta+1}{q} \int_{\Omega} \vert \nabla v \vert ^{q}\,dx- \int_{\Omega}G(x,u,v)\,dx. \end{aligned}$$

(16)

From (2) and (3), by a standard argument, the functional J is well defined and $J\in C^{1}(W,R)$. From the variational point of view, a weak solution of system (1) corresponds to a critical point of the functional J in W. Theorem 2 is parallel to Theorem 1, hence we will only prove Theorem 1 and Theorem 3. In the following, we will introduce an abstract critical point theorem, which is based on a compactness condition - the $(PS)$ condition or the $(Ce)$ condition - and on a linking structure.

Definition 1

Let X be a real Banach space. The functional I satisfies the $(PS)_{c}$ condition at the level $c\in R$, if any sequence $\{u_{n}\}\subset X$ such that $I(u_{n})\to c$, $I'(u_{n})\to0$ as $n\to\infty$ has a convergent subsequence. The functional I satisfies the $(PS)$ condition if I satisfies the $(PS)_{c}$ condition at any $c\in R$.

Definition 2

The functional I satisfies the $(Ce)_{c}$ condition at the level $c\in R$, if any sequence $\{u_{n}\}\subset X$ such that $I(u_{n})\to c$, $(1+ \Vert u_{n} \Vert ) \Vert I'(u_{n}) \Vert _{X^{*}}\to0$ as $n\to\infty$ has a convergent subsequence. The functional I satisfies the $(Ce)$ condition if I satisfies the $(Ce)_{c}$ condition at any $c\in R$.

The $(Ce)$ condition was introduced by Cerami [1] and it is a weaker version of the $(PS)$ condition. Next, we will introduce the notions of the relative homotopical linking and the relative cohomotopical linking.

Definition 3

(see [3])

Let X be a metric space and $S\subset P$ and $B\subset A$ be four subsets of X. We say that $(P,S)$ links $(A,B)$, if $S\cap A=B\cap P={\O}$ and for every deformation $\eta:P\times[0,1]\to X\backslash B$ with $\eta(S\times [0,1])\cap A={\O}$ we have $\eta(P\times\{1\})\cap A\neq{\O}$.

Definition 4

(see [18])

Let X be a metric space and $S\subset P$ and $B\subset A$ be four subsets of X. Let m be a nonnegative integer and $\mathcal{K}$ be a field. We say that $(P,S)$ links $(A,B)$ cohomologically in dimension m over $\mathcal{K}$, if $S\cap A=B\cap P={\O}$ and the restriction homomorphism $H^{m}(X\backslash B,X\backslash A,{\mathcal{K}})\to H^{m}(P,S,{\mathcal{K}})$ is not identically zero, where $H^{*}$ denotes Alexander-Spanier cohomology (see [23]).

Following from [18], if $(P,S)$ links $(A,B)$ cohomologically, then $(P,S)$ links $(A,B)$.

Theorem A

(see [3])

Let X be a Banach space and $f\in C^{1}(X, R)$. Let $A,B,P,S$ be four subsets of X with $S\subset P$ and $B\subset A$ such that $(P,S)$ links $(A,B)$ and

$$\sup_{S}f\leq\inf_{A}f,\qquad \sup_{P}f\leq\inf_{B}f, $$

where we agree that $\sup{\o}=-\infty$, $\inf{\o}=+\infty$. Define

$$c=\inf_{\eta\in{\mathcal{N}}}\sup f\bigl(\eta\bigl(P\times\{1\}\bigr)\bigr), $$

where ${\mathcal{N}}$ is the set of deformations $\eta:P\times [0,1]\to X\backslash B$ with $\eta(S\times[0,1])\cap A={\o}$. Then we have

$$\inf_{A} f\leq c\leq\sup_{P} f. $$

Moreover, if f satisfies the $(PS)_{c}$ condition or the $(Ce)_{c}$ condition, then c is a critical value of f.

In the following, we will introduce the examples of the relative cohomological linking on the product space $W=W^{1,p}_{0}(\Omega)\times W^{1,q}_{0}(\Omega)$ which are used to prove our theorems. The main ideas come from [18].

For $\lambda_{k}<\lambda_{k+1}(k\geq1)$, let

$$\begin{aligned} &C_{\Sigma,-}=\bigl\{ (u,v)\in\Sigma: \phi(u,v)\leq\lambda_{k}\bigr\} ,\qquad C_{\Sigma,+}=\bigl\{ (u,v)\in\Sigma: \phi(u,v)\geq \lambda_{k+1}\bigr\} , \\ &C_{-}=\bigl\{ (u,v)\in W: \phi(u,v)\leq\lambda_{k}\varphi(u,v)\bigr\} ,\qquad C_{+}=\bigl\{ (u,v)\in W: \phi(u,v)\geq\lambda_{k+1}\varphi(u,v)\bigr\} . \end{aligned}$$

It is easy to see that $C_{\Sigma,-}$, $C_{\Sigma,+}$, $C_{-}$, $C_{+}$ are the symmetric subsets such that $C_{-}\cap C_{+}=\{(0,0)\}$ and $C_{-}$, $C_{+}$ are the $(p,q)$-sets. From Theorem 10.10 of [24] and Theorem 2.7 of [18], we have the following conclusion.

Lemma 1

Let $\lambda_{k}<\lambda_{k+1}$ for some $k\geq1$. Then

$$\begin{aligned} &i(C_{\Sigma,-})=i(C_{\Sigma,+})=k, \\ &i\bigl(C_{-}\backslash\bigl\{ (0,0)\bigr\} \bigr)=i\bigl(\bigl\{ (u,v)\in W: \phi(u,v)< \lambda_{k+1}\varphi (u,v)\bigr\} \bigr)=i(W\setminus C_{+})=k. \end{aligned}$$

Moreover, $(W,C_{-}\backslash\{(0,0)\})$ links $C_{+}$ cohomologically in dimension $k=i$ ($C_{-}\backslash\{(0,0)\}$) over $Z_{2}$.

For the sake of convenience of the reader, we introduce the following the so-called five lemma (see [23]).

Lemma 2

(Five lemma)

Given a commutative diagram of Abelian groups and homomorphisms

$$\textstyle\begin{array}{ccccccccc} G_{1} &\stackrel{\alpha_{1} }{\longrightarrow} & G_{2} &\stackrel{\alpha_{2} }{\longrightarrow} &G_{3}& \stackrel{\alpha_{3} }{\longrightarrow}& G_{4}&\stackrel{\alpha_{4} }{\longrightarrow} &G_{5}\\ \downarrow& & \downarrow& & \downarrow& & \downarrow& & \downarrow\\ H_{1} & \stackrel{\beta_{1} }{\longrightarrow} & H_{2} &\stackrel{\beta_{2} }{\longrightarrow} &H_{3}& \stackrel{\beta_{3} }{\longrightarrow} & H_{4}&\stackrel{\beta_{4} }{\longrightarrow} &H_{5} \end{array} $$

in which each row is exact and $\gamma_{i}:G_{i}\rightarrow H_{i}$ are isomorphisms $(i=1,2,4,5)$, $\gamma_{3}:G_{3}\rightarrow H_{3}$ is an isomorphism.

Lemma 3

For $r_{-}>0$, let

$$D_{-}=\bigl\{ (u,v)\in C_{-}:\phi(u,v)\leq r_{-}\bigr\} ,\qquad S_{-}=\bigl\{ (u,v)\in C_{-}:\phi (u,v)=r_{-}\bigr\} . $$

Then $(D_{-},S_{-})$ links $C_{+}$ cohomologically in dimension $k=i(C_{-}\backslash\{(0,0)\})$ over $Z_{2}$. Moreover, for some $(e_{1},e_{2})\in W\backslash C_{-}$, let $r_{-}>r_{+}>0$ and

$$\begin{aligned} &Q=\bigl\{ (u+te_{1},v+te_{2}):(u,v)\in C_{-},t\geq0, \phi(u+te_{1},v+te_{2})\leq r_{-}\bigr\} , \\ &H=\bigl\{ (u+te_{1},v+te_{2}):(u,v)\in C_{-},t\geq0, \phi(u+te_{1},v+te_{2})=r_{-}\bigr\} , \\ &D_{+}=\bigl\{ (u,v)\in C_{+}:\phi(u,v)\leq r_{+}\bigr\} ,\qquad S_{+}=\bigl\{ (u,v)\in C_{+}: \phi (u,v)=r_{+}\bigr\} . \end{aligned}$$

Then $(Q,D_{-}\cup H)$ links $S_{+}$ cohomologically in dimension $k=i(C_{-}\backslash\{(0,0)\})+1$ over $Z_{2}$.

Proof

For the sake of convenience, we will miss the coefficient field $Z_{2}$.

• (a) Since $S_{-}$, $C_{-}\backslash\{(0,0)\}$, $D_{-}$, and W are homotopy equivalences, from Lemma 2, the restriction homomorphism

  $$H^{k}\bigl(W,C_{-}\backslash\bigl\{ (0,0)\bigr\} \bigr)\to H^{k}(D_{-},S_{-}) $$

  is an isomorphism. Following from Lemma 1, $(D_{-},S_{-})$ links $C_{+}$ cohomologically in dimension $k=i(C_{-}\backslash\{(0,0)\})$.

• (b) Let

  $$E_{+}=\bigl\{ (u,v)\in C_{+}:\phi(u,v)\geq r_{+}\bigr\} . $$

  Since $W\backslash E_{+}$ is homeomorphic with a star shaped subset of W with respect to the origin, $H^{*}(W,W\backslash E_{+})$ is trivial. From the exact sequence of triple $(W,W\backslash E_{+},W\backslash C_{+})$, the restriction homomorphism

  $$H^{k}(W,W\backslash C_{+})\to H^{k}(W\backslash E_{+},W \backslash C_{+}) $$

  is an isomorphism. Hence, it follows from (a) that the restriction homomorphism

  $$ H^{k}(W\backslash E_{+},W\backslash C_{+})\to H^{k}(D_{-},S_{-}) $$

  (17)

  is not identically zero.

On the other hand, since $E_{+}\cap(W\backslash S_{+})$ is a closed subset of $W\backslash S_{+}$ contained in the open set $W\backslash D_{+}$, we have the excision isomorphism $H^{k}(W\backslash S_{+},W\backslash D_{+})\to H^{k}(W\backslash E_{+}, W\backslash C_{+})$. Following from (17), $(D_{-},S_{-})$ links $(D_{+},S_{+})$ cohomologically in dimension $k= i(C_{-}\backslash \{(0,0)\})$. Consider the diagram

$$\textstyle\begin{array}{ccccc} H^{k}(W,W\backslash D_{+}) & \longrightarrow& H^{k}(W\backslash S_{+},W\backslash D_{+}) &\longrightarrow &H^{k+1}(W,W\backslash S_{+})\\ \downarrow& & \downarrow& & \downarrow\\ H^{k}(Q,H) & \longrightarrow& H^{k}(D_{-}\cup H,H) & \longrightarrow & H^{k+1}(Q,D_{-}\cup H) \\ & & \downarrow& & \\ & & H^{k}(D_{-},S_{-}) & & \end{array} $$

where vertical rows are restriction homomorphisms and horizontal rows come from exact sequences of the triples $(W,W\backslash S_{+},W\backslash D_{+})$ and $(Q,D_{-}\cup H,H)$. The restriction homomorphism

$$H^{k}(W\backslash S_{+},W\backslash D_{+})\to H^{k}(D_{-}\cup H,H) $$

does the same. Let $(e_{1},e_{2})\notin C_{-}$, $(\tilde{u},\tilde{v})=((1-t)u+te_{1},(1-t)v+te_{2})$ and define a contraction $\psi: H\times[0,1]\to H$ by

$$\psi(u,v,t)= \biggl( \biggl(\frac{r_{-}}{\phi(\tilde{u},\tilde{v})} \biggr)^{1/p}\tilde{u}, \biggl(\frac{r_{-}}{\phi(\tilde{u},\tilde{v})} \biggr)^{1/q}\tilde{v} \biggr). $$

Since Q is also contractible in itself, $H^{k}(Q,H)$ is trivial. Consequently, from the exactness of the second row, the map

$$H^{k}(D_{-}\cup H,H)\to H^{k+1}(Q,D_{-}\cup H) $$

is injective. Therefore, from the commutativity of the right square, the restriction homomorphism $H^{k+1}(W,W\backslash S_{+})\to H^{k+1}(Q,D_{-}\cup H)$ is not identically zero. □

Proof of Theorem 1

• (1) The functional J satisfies the $(Ce)$ condition. Let $(u_{n},v_{n})$ be a $(Ce)$ sequence for the functional J, i.e., there is a positive constant $M_{0}$ such that

  $$ \bigl\vert J(u_{n},v_{n}) \bigr\vert \leq M_{0}\quad \mbox{and}\quad \bigl\Vert J'(u_{n},v_{n}) \bigr\Vert \bigl(1+ \Vert u_{n} \Vert _{p}+ \Vert v_{n} \Vert _{q}\bigr)\to0\quad \mbox{as }n\to \infty. $$

  (18)

  Since the nonlinearity G satisfies the subcritical growth conditions (2) and (3), if $(u_{n},v_{n})$ is bounded in W, by a standard argument, it follows that $(u_{n},v_{n})$ converges strongly in W. Hence we only prove that $(u_{n},v_{n})$ is bounded in W. Arguing by contradiction, we assume $\Vert (u_{n},v_{n}) \Vert = \Vert u_{n} \Vert _{p}+ \Vert v_{n} \Vert _{q}\to\infty$ as $n\to\infty$. Let $K_{n}:= \Vert u_{n} \Vert ^{p}_{p}+ \Vert v_{n} \Vert ^{q}_{q}$ and it is easy to see that $K_{n}\to\infty$ as $n\to\infty$. Let $\bar{u}_{n}=u_{n}\setminus K_{n}^{1/p}, \bar{v}_{n}=v_{n}\setminus K_{n}^{1/q}$. Then $(\bar{u}_{n},\bar{v}_{n})$ is bounded in W, i.e.,

  $$\Vert \bar{u}_{n} \Vert _{p}^{p}+ \Vert \bar{v}_{n} \Vert _{q}^{q}=1 \quad \mbox{for all }n. $$

  We can choose a subsequence of $(u_{n},v_{n})$ if necessary, also denoted by $(u_{n},v_{n})$, and there exists $(\bar{u},\bar{v})\in W$ such that

  $$\begin{aligned} &(\bar{u}_{n},\bar{v}_{n})\rightharpoonup( \bar{u},\bar{v}) \quad \mbox{weakly in } W, \end{aligned}$$

  (19)
  $$\begin{aligned} &(\bar{u}_{n},\bar{v}_{n})\to(\bar{u}, \bar{v}) \quad \mbox{strongly in }L^{\theta_{1}}(\Omega)\times L^{\theta_{2}}( \Omega), \end{aligned}$$

  (20)
  $$\begin{aligned} &\bigl(\bar{u}_{n}(x),\bar{v}_{n}(x)\bigr)\to \bigl(\bar{u}(x),\bar{v}(x)\bigr)\quad \mbox{for a.e. }x\in\Omega, \end{aligned}$$

  (21)

  where $\theta_{1}\in(1,p^{*})$, $\theta_{2}\in(1,q^{*})$. Following from (18), there is a constant $M_{1}>0$ such that

  $$\begin{aligned} M_{1} \geq& \liminf_{n\to\infty}J(u_{n},v_{n})- \biggl\langle J'(u_{n},v_{n}),\biggl( \frac {1}{p}u_{n},\frac{1}{q}v_{n}\biggr)\biggr\rangle \\ =&\liminf_{n\to\infty} \int_{\Omega}\biggl(\frac{1}{p}G_{s}(x,u_{n},v_{n})u_{n}+ \frac{1}{q}G_{t}(x,u_{n},v_{n})v_{n}-G(x,u_{n},v_{n}) \biggr)\,dx \\ =&\liminf_{n\to\infty} \int_{\Omega}H(x,u_{n},v_{n})\,dx. \end{aligned}$$

  (22)

  In view of the right side of (7) and the continuity of G, for any $\varepsilon>0$, there is $M_{2}:=M_{2}(\varepsilon)>0$ such that

  $$\bigl\vert G(x,s,t) \bigr\vert \leq(\lambda_{k+1}+ \varepsilon)F(s,t)+M_{2}\quad \forall(x,s,t)\in\bar{\Omega}\times R^{2}. $$

  Hence, from (16) and the Young inequality, we obtain

  $$\begin{aligned} &\min \biggl\{ \frac{\alpha+1}{p},\frac{\beta+1}{q} \biggr\} \bigl( \Vert u_{n} \Vert _{p}^{p}+ \Vert v_{n} \Vert _{q}^{q}\bigr) \\ &\quad \leq \frac{\alpha+1}{p} \Vert u_{n} \Vert _{p}^{p}+ \frac{\beta+1}{q} \Vert v_{n} \Vert _{q}^{q} \\ &\quad \leq J(u_{n},v_{n})+ \int_{\Omega}G(x,u_{n},v_{n})\,dx \\ &\quad \leq M_{3}+(\lambda_{k+1}+\varepsilon) \int_{\Omega}\biggl(\frac{\alpha +1}{p} \vert u_{n} \vert ^{p}+\frac{\beta+1}{q} \vert v_{n} \vert ^{q}+ \vert u_{n} \vert ^{\alpha+1} \vert v_{n} \vert ^{\beta|+1} \biggr)\,dx \\ &\quad \leq M_{3}+M_{4}(\lambda_{k+1}+\varepsilon) \bigl( \Vert u_{n} \Vert ^{p}_{L^{p}}+ \Vert v_{n} \Vert ^{q}_{L^{q}}\bigr), \end{aligned}$$

  where $M_{3}=M_{0}+M_{2} \vert \Omega \vert$, $M_{4}=2\max\{\frac{\alpha+1}{p}, \frac{\beta+1}{q}\}$. Dividing the above inequality by $K_{n}$ and letting $n\to\infty$, it follows from (20) and (21) that

  $$\min \biggl\{ \frac{\alpha+1}{p},\frac{\beta+1}{q} \biggr\} \leq M_{4}( \lambda _{k+1}+\varepsilon) \bigl( \Vert \bar{u} \Vert ^{p}_{L^{p}}+ \Vert \bar{v} \Vert ^{q}_{L^{q}} \bigr). $$

  Therefore, there exists a subset Ω̃ of Ω with positive measure, such that $\bar{u}(x)\neq0$ or $\bar{v}(x)\neq0$ for all $x\in\tilde{\Omega}$. From the definitions of $\bar{u}_{n}$ and $\bar{v}_{n}$, we have $\vert (u_{n}(x),v_{n}(x)) \vert \to \infty$ as $n\to\infty$ for any $x\in\tilde{\Omega}$. From (6), there exists a positive constant $M_{5}$ such that

  $$H(x,s,t)\geq M_{5}\quad \mbox{for any } (x,s,t)\in\bar{\Omega}\times R^{2}. $$

  From Fatou’s lemma, (6), and the above inequality, we have

  $$\liminf_{n\to\infty} \int_{\Omega}H(x,u_{n},v_{n})\,dx\geq \int_{\tilde{\Omega }}\liminf_{n\to\infty} H(x,u_{n},v_{n})\,dx+M_{5} \vert \Omega\setminus\tilde {\Omega} \vert =+\infty, $$

  which is a contradiction to (22). Therefore, we have proved that $(u_{n},v_{n})$ is bounded in W.

• (2) There is a positive constant $r_{-}$ such that

  $$\sup_{(u,v)\in S_{-}}J(u,v)< \inf_{(u,v)\in C_{+}}J(u,v)\quad \mbox{and}\quad \sup_{(u,v)\in D_{-}}J(u,v)< +\infty, $$

  where $D_{-}=\{(u,v)\in C_{-}:\phi(u,v)\leq r_{-}\}, S_{-}=\{(u,v)\in C_{-}:\phi(u,v)=r_{-}\}, C_{+}=\{(u,v)\in W: \phi(u,v)\geq\lambda _{k+1}\varphi(u,v)\}$. From the left side of (7) and the continuity of G, for any $\varepsilon>0$, there exists $M_{6}=M_{6}(\varepsilon)>0$ such that

  $$G(x,s,t)\geq(\lambda_{k}+\varepsilon)F(s,t)-M_{6}\quad \mbox{for any }(x,s,t)\in\bar{\Omega}\times R^{2}. $$

  Hence, for any $(u,v)\in C_{-}$, we obtain

  $$\begin{aligned} J(u,v) \leq&\frac{\alpha+1}{p} \int_{\Omega} \vert \nabla u \vert ^{p}\,dx + \frac{\beta+1}{q} \int_{\Omega} \vert \nabla v \vert ^{q}\,dx \\ &{} -( \lambda_{k}+\varepsilon) \int_{\Omega}F(u,v)\,dx+M_{6} \vert \Omega \vert \\ \leq&-\frac{\varepsilon}{\lambda_{k}} \biggl(\frac{\alpha+1}{p} \Vert u \Vert ^{p}_{p}+\frac{\beta+1}{q} \Vert v \Vert ^{q}_{q}\biggr)+M_{6} \vert \Omega \vert . \end{aligned}$$

  (23)

  On the other hand, define

  $$L(x,s,t)=G(x,s,t)-\lambda_{k+1}F(s,t). $$

  By a simple calculation, we have

  $$\begin{aligned} H(x,s,t) =&\frac{1}{p}G_{s}(x,s,t)s+ \frac{1}{q}G_{t}(x,s,t)t-G(x,s,t) \\ =& \frac{1}{p}L_{s}(x,s,t)s+\frac{1}{q}L_{t}(x,s,t)t-L(x,s,t). \end{aligned}$$

  From (6), for any $M_{7}>0$, there exists a positive constant $L_{0}$ such that

  $$ H(x,s,t)=\frac{1}{p}G_{s}(x,s,t)s+ \frac{1}{q}G_{t}(x,s,t)t-G(x,s,t)\geq M_{7}, $$

  (24)

  for any $x\in\Omega$ and $\vert (s,t) \vert \geq L_{0}$. Moreover, for any $(\tilde{s},\tilde{t})\in R^{2}$ with $\vert (\tilde {s},\tilde{t}) \vert =1$, from the right side of (7), we have

  $$ \limsup_{\tau\to+\infty}\frac{L(x,\tau^{\frac{1}{p}}\tilde{s},\tau ^{\frac{1}{q}}\tilde{t})}{\tau}\leq0\quad \mbox{uniformly for }x\in \Omega. $$

  (25)

  Hence, for $\tau\geq L_{0}$, we have

  $$\begin{aligned} &\frac{d}{d\tau} \biggl(\frac{L(x,\tau^{\frac{1}{p}}\tilde{s},\tau^{\frac {1}{q}}\tilde{t})}{\tau} \biggr) \\ &\quad = \frac{1}{\tau^{2}} \biggl(\frac{1}{p}L_{s}\bigl(x, \tau^{\frac{1}{p}}\tilde{s},\tau^{\frac{1}{q}}\tilde {t}\bigr) \tau^{\frac{1}{p}}\tilde{s}+ \frac{1}{q}L_{t}\bigl(x, \tau^{\frac{1}{p}}\tilde{s},\tau^{\frac{1}{q}}\tilde {t}\bigr) \tau^{\frac{1}{q}}\tilde{t}- L\bigl(x,\tau^{\frac{1}{p}}\tilde{s}, \tau^{\frac{1}{q}}\tilde{t}\bigr) \biggr) \\ &\quad \geq \frac{M_{7}}{\tau^{2}}. \end{aligned}$$

  Integrating the above inequality over the interval $[T_{1},T_{2}]\subset [L_{0},+\infty)$, we obtain

  $$\frac{L(x,T_{2}^{\frac{1}{p}}\tilde{s},T_{2}^{\frac{1}{q}}\tilde{t})}{T_{2}} -\frac{L(x,T_{1}^{\frac{1}{p}}\tilde{s},T_{1}^{\frac{1}{q}}\tilde{t})}{T_{1}} \geq M_{7} \biggl( \frac{1}{T_{1}}-\frac{1}{T_{2}} \biggr). $$

  Letting $T_{2}\to+\infty$, from (25), we obtain

  $$L\bigl(x,\tau^{\frac{1}{p}}\tilde{s},\tau^{\frac{1}{q}}\tilde{t}\bigr)\leq- \frac {M_{7}}{2}\quad \mbox{for any }x\in\Omega\mbox{ and }\tau>L_{0}. $$

  Therefore, it follows that

  $$\lim_{\tau\to+\infty}L\bigl(x,\tau^{\frac{1}{p}}\tilde{s}, \tau^{\frac {1}{q}}\tilde{t}\bigr)=-\infty\quad \mbox{uniformly for any } x\in \Omega. $$

  Picking $\tau>0$, it follows from the above expression that, for any $(u,v)\in C_{\Sigma,+}$,

  $$\begin{aligned} &J\bigl(\tau^{\frac{1}{p}}u,\tau^{\frac{1}{q}}v\bigr) \\ &\quad = \frac{\tau(\alpha +1)}{p} \int_{\Omega} \vert \nabla u \vert ^{p}\,dx + \frac{\tau(\beta+1)}{q} \int_{\Omega} \vert \nabla v \vert ^{q}\,dx- \int _{\Omega}G\bigl(x,\tau^{\frac{1}{p}}u,\tau^{\frac{1}{q}}v \bigr)\,dx \\ &\quad =\frac{\tau(\alpha+1)}{p} \int_{\Omega} \vert \nabla u \vert ^{p}\,dx+ \frac {\tau(\beta+1)}{q} \int_{\Omega} \vert \nabla v \vert ^{q}\,dx- \lambda_{k+1}\tau \int_{\Omega}F(u,v)\,dx \\ &\qquad {} - \int_{\Omega}\bigl(G\bigl(x,\tau^{\frac{1}{p}}u, \tau^{\frac{1}{q}}v\bigr)-\lambda _{k+1}\tau F(u,v)\bigr)\,dx \\ &\quad \geq- \int_{\Omega}L\bigl(x,\tau^{\frac{1}{p}}u,\tau^{\frac {1}{p}}v \bigr)\,dx \\ &\quad \to+\infty, \end{aligned}$$

  as $\tau\to+\infty$. From (23) and the above inequality, for fixed $\varepsilon>0$, there exists a positive constant $r_{-}$ such that

  $$\max_{(u,v)\in S_{-}}J(u,v)< \inf_{(u,v)\in C_{+}}J(u,v), $$

  which together with Theorem A and Lemma 3 implies that the functional J has a critical point. It follows that Theorem 1 is proved.

 □

Proof of Theorem 3

• (1) The functional J satisfies the $(PS)$ condition. Without loss of generality, we consider the case (14). Let $(u_{n},v_{n})$ be a $(PS)$ sequence of the functional J, that is,

  $$ J(u_{n},v_{n})\to c\in R\quad \mbox{and}\quad J'(u_{n},v_{n})\to0\quad \mbox{as }n\to\infty. $$

  (26)

  Similar to Theorem 1, we only prove that $(u_{n},v_{n})$ is bounded in W. First of all, from (14), there is a positive constant $M_{8}$ such that

  $$H(x,s,t)\geq a\bigl( \vert s \vert ^{\mu}+ \vert t \vert ^{\nu}\bigr)-M_{8} \quad \forall (x,s,t)\in\Omega\times R^{2}. $$

  Hence, it follows that

  $$\begin{aligned} J(u_{n},v_{n})-\biggl\langle J'(u_{n},v_{n}), \biggl(\frac{1}{p}u_{n},\frac{1}{q}v_{n} \biggr)\biggr\rangle =& \int_{\Omega}H(x,u_{n},v_{n})\,dx \\ \geq& a \int_{\Omega}\bigl( \vert u_{n} \vert ^{\mu}+ \vert v_{n} \vert ^{\nu}\bigr)\,dx-M_{8} \vert \Omega \vert . \end{aligned}$$

  Combining (26) and the above inequality, we obtain

  $$ \frac{\int_{\Omega}( \vert u_{n} \vert ^{\mu}\,dx+ \vert v_{n} \vert ^{\nu})\,dx}{ \Vert u_{n} \Vert _{p}+ \Vert v_{n} \Vert _{q}}\to0 \quad \mbox{as }n\to\infty. $$

  (27)

  From the Hölder inequality, (2), (3), and (26), we have

  $$\begin{aligned} &\bigl\langle J'(u_{n},v_{n}),(u_{n},v_{n}) \bigr\rangle \\ &\quad = (\alpha+1) \Vert u_{n} \Vert ^{p}_{p}+( \beta+1) \Vert v_{n} \Vert ^{q}_{q}- \int _{\Omega}\bigl(G_{s}(x,u_{n},v_{n})u_{n}+G_{t}(x,u_{n},v_{n})v_{n} \bigr)\,dx \\ &\quad \geq (\alpha+1) \Vert u_{n} \Vert ^{p}_{p}+( \beta+1) \Vert v_{n} \Vert ^{q}_{q}- \biggl\vert \int_{\Omega}G_{s}(x,u_{n},v_{n}) \cdot u_{n}\,dx \biggr\vert \\ &\qquad {}- \biggl\vert \int _{\Omega}G_{t}(x,u_{n},v_{n}) \cdot v_{n}\,dx \biggr\vert \\ &\quad \geq (\alpha+1) \Vert u_{n} \Vert ^{p}_{p}+( \beta+1) \Vert v_{n} \Vert ^{q}_{q}-c_{0} \int_{\Omega}\bigl( \vert u_{n} \vert + \vert u_{n} \vert ^{p_{1}-1} \vert u_{n} \vert + \vert u_{n} \vert \vert v_{n} \vert ^{\frac {(p_{1}-1)q_{1}}{p_{1}}} \bigr)\,dx \\ &\qquad {}-c_{0} \int_{\Omega}\bigl( \vert v_{n} \vert + \vert v_{n} \vert ^{q_{1}-1} \vert v_{n} \vert + \vert v_{n} \vert \vert u_{n} \vert ^{\frac {(q_{1}-1)p_{1}}{q_{1}}} \bigr)\,dx \\ &\quad \geq (\alpha+1) \Vert u_{n} \Vert ^{p}_{p}+( \beta+1) \Vert v_{n} \Vert ^{q}_{q}-c_{0} \Vert u_{n} \Vert _{L^{1}} \\ &\qquad {}-c_{0} \biggl( \int_{\Omega} \vert u_{n} \vert ^{(p_{1}-1)\cdot\frac{\mu }{p_{1}-1}}\,dx \biggr)^{\frac{p_{1}-1}{\mu}} \cdot \biggl( \int_{\Omega} \vert u_{n} \vert ^{\frac{\mu}{\mu+1-p_{1}}}\,dx \biggr)^{\frac{\mu+1-p_{1}}{\mu}} \\ &\qquad {}-c_{0} \biggl( \int_{\Omega} \vert v_{n} \vert ^{\frac{(p_{1}-1)q_{1}}{p_{1}}\cdot \frac{p_{1}\nu}{(p_{1}-1)q_{1}}}\,dx \biggr)^{\frac{(p_{1}-1)q_{1}}{p_{1}\nu}} \cdot \biggl( \int_{\Omega} \vert u_{n} \vert ^{\frac{p_{1}\nu}{p_{1}\nu +(1-p_{1})q_{1}}}\,dx \biggr)^{\frac{p_{1}\nu+(1-p_{1})q_{1}}{p_{1}\nu}} \\ &\qquad {}-c_{0} \Vert v_{n} \Vert _{L^{1}}-c_{0} \biggl( \int_{\Omega} \vert v_{n} \vert ^{(q_{1}-1)\cdot\frac{\nu}{q_{1}-1}}\,dx \biggr)^{\frac{q_{1}-1}{\nu}} \cdot \biggl( \int_{\Omega} \vert v_{n} \vert ^{\frac{\nu}{\nu+1-q_{1}}}\,dx \biggr)^{\frac{\nu+1-q_{1}}{\nu}} \\ &\qquad {}-c_{0} \biggl( \int_{\Omega} \vert u_{n} \vert ^{\frac{(q_{1}-1)p_{1}}{q_{1}}\cdot \frac{q_{1}\mu}{(q_{1}-1)p_{1}}}\,dx \biggr)^{\frac{(q_{1}-1)p_{1}}{q_{1}\mu}} \cdot \biggl( \int_{\Omega} \vert v_{n} \vert ^{\frac{q_{1}\mu}{q_{1}\mu +(1-q_{1})p_{1}}}\,dx \biggr)^{\frac{q_{1}\mu+(1-q_{1})p_{1}}{q_{1}\mu}} \\ &\quad \geq (\alpha+1) \Vert u_{n} \Vert ^{p}_{p}+( \beta+1) \Vert v_{n} \Vert ^{q}_{q}-c_{0}C \bigl( \Vert u_{n} \Vert _{p}+ \Vert v_{n} \Vert _{q}\bigr) \\ &\qquad {}- c_{0}C \Vert u_{n} \Vert _{p} \bigl( \Vert u_{n} \Vert _{L^{\mu}}^{p_{1}-1}+ \Vert v_{n} \Vert _{L^{\nu}}^{\frac{(p_{1}-1)q_{1}}{p_{1}}} \bigr) \\ &\qquad {} -c_{0}C \Vert v_{n} \Vert _{q} \bigl( \Vert u_{n} \Vert _{L^{\mu}}^{\frac {(q_{1}-1)p_{1}}{q_{1}}}+ \Vert v_{n} \Vert _{L^{\nu}}^{q_{1}-1} \bigr), \end{aligned}$$

  (28)

  for all n. Here we consider the case $p< q$ and $p_{1}\leq q_{1}$. The other cases can be proved similarly. By a simple computation, we have

  $$p_{1}-1< \frac{(q_{1}-1)p_{1}}{q_{1}},\qquad \frac{(p_{1}-1)q_{1}}{p_{1}}< q_{1}-1, $$

  and we assume that

  $$\Vert u_{n} \Vert _{L^{\mu}}^{p_{1}-1}< \Vert u_{n} \Vert _{L^{\mu}}^{\frac {(q_{1}-1)p_{1}}{q_{1}}},\qquad \Vert v_{n} \Vert _{L^{\nu}}^{\frac {(p_{1}-1)q_{1}}{p_{1}}}< \Vert v_{n} \Vert _{L^{\nu}}^{q_{1}-1}. $$

  Then in view of $\mu>\frac{(q_{1}-1)p_{1}}{q_{1}}\frac{1}{p-1}$ and (27), we have

  $$\begin{aligned} &\frac{ \Vert u_{n} \Vert _{p} \Vert u_{n} \Vert _{L^{\mu}}^{p_{1}-1}+ \Vert v_{n} \Vert _{q} \Vert u_{n} \Vert _{L^{\mu}}^{\frac{(q_{1}-1)p_{1}}{q_{1}}}}{ \Vert u_{n} \Vert _{p}^{p}+ \Vert v_{n} \Vert ^{q}_{q}} \\ &\quad \leq \frac {2^{p-1}}{2^{p-1}} \frac{( \Vert u_{n} \Vert _{p}+ \Vert v_{n} \Vert _{q}) \Vert u_{n} \Vert _{L^{\mu}}^{\frac{(q_{1}-1)p_{1}}{q_{1}}}}{ \Vert u_{n} \Vert _{p}^{p}+ \Vert v_{n} \Vert ^{p}_{q}} \\ &\quad =2^{p-1}\frac{ \Vert u_{n} \Vert _{L^{\mu}}^{\frac{(q_{1}-1)p_{1}}{q_{1}}}}{ ( \Vert u_{n} \Vert _{p}+ \Vert v_{n} \Vert _{q})^{p-1}} \\ &\quad \to 0\quad \mbox{as }n\to\infty. \end{aligned}$$

  (29)

  Similarly, we have

  $$\frac{ \Vert u_{n} \Vert _{p} \Vert v_{n} \Vert _{L^{\nu}}^{\frac {(p_{1}-1)q_{1}}{p_{1}}}+ \Vert v_{n} \Vert _{q} \Vert v_{n} \Vert _{L^{\nu}}^{q_{1}-1}}{ \Vert u_{n} \Vert _{p}^{p}+ \Vert v_{n} \Vert ^{q}_{q}} \to0\quad \mbox{as }n\to\infty. $$

  Hence, following from (28), (29), and the above limit, we see that $\{(u_{n},v_{n})\}$ is bounded in W.

• (2) For some $(e_{1}, e_{2})\in W\setminus C_{-}$, there are two positive constants $r_{-}>r_{+}$ such that

  $$\sup_{(u,v)\in D_{-}\cup H}J(u,v)< \inf_{(u,v)\in S_{+}}J(u,v) $$

  and

  $$\sup_{(u,v)\in Q}J(u,v)< +\infty, $$

  where $Q=\{(u+te_{1},v+te_{2}):(u,v)\in C_{-},t\geq0, \phi (u+te_{1},v+te_{2})\leq r_{-}\}$, $D_{-}\cup H=\{(u,v)\in C_{-}:\phi(u,v)\leq r_{-}\} \cup\{(u+te_{1},v+te_{2}):(u,v)\in C_{-},t>0, \phi(u+te_{1},v+te_{2})= r_{-}\}$, $S_{+}=\{(u,v)\in C_{+}: \phi(u,v)=r_{+}\}$.

  From (12) and (16), for any $(u,v)\in C_{-}$, we obtain

  $$\begin{aligned} J(u,v) \leq&\frac{\alpha+1}{p} \int_{\Omega} \vert \nabla u \vert ^{p}\,dx + \frac{\beta+1}{q} \int_{\Omega} \vert \nabla v \vert ^{q}\,dx- \lambda_{k} \int_{\Omega}F(u,v)\,dx \leq0. \end{aligned}$$

  (30)

  Picking a constant β̂ such that $\lambda_{k+1}<\hat{\beta }<\beta_{0}$, it follows from the right side of (13) that there exists a positive constant $M_{9}$ such that

  $$G(x,s,t)\geq\hat{\beta}F(s,t)-M_{9},\quad \forall(x,s,t)\in \Omega \times R^{2}. $$

  Letting $(e_{1},e_{2})\in W\backslash C_{-}$, it follows from (16) and the above inequality that, for all $(u,v)\in \{(u+te_{1},v+te_{2}):(u,v)\in C_{-},t>0\}$,

  $$\begin{aligned} J(u,v) \leq& \frac{\alpha+1}{p} \Vert u \Vert ^{p}_{p}+\frac{\beta +1}{q} \Vert u \Vert ^{q}_{q}-\hat{\beta} \int_{\Omega}F(u,v)\,dx+M_{9} \vert \Omega \vert \\ \leq&\frac{\lambda_{k+1}-\hat{\beta}}{\lambda_{k+1}} \biggl(\frac{\alpha +1}{p} \Vert u \Vert ^{p}_{p}+\frac{\beta+1}{q} \Vert u \Vert ^{q}_{q} \biggr)+M_{9} \vert \Omega \vert , \end{aligned}$$

  which together with (30) shows there is $r_{-}>0$ such that

  $$J(u,v)\leq0, $$

  for any $(u,v)\in D_{-}\cup H=\{(u,v)\in C_{-}:\phi(u,v)\leq r_{-}\}\cup\{ (u+te_{1},v+te_{2}):(u,v)\in C_{-},t>0, \phi(u+te_{1},v+te_{2})= r_{-}\}$.

  On the other hand, choosing a constant α̂ such that $\alpha _{0}<\hat{\alpha}<\lambda_{k+1}$, it follows from the left side of (13) that there is $\delta>0$ such that

  $$ G(x,s,t)\leq\hat{\alpha} F(s,t),\quad \forall x\in\Omega, \bigl\vert (s,t) \bigr\vert \leq\delta. $$

  (31)

  From (2), (3), and the Young inequality, for any $(s,t)\in R^{2}$, we have

  $$\begin{aligned} \bigl\vert G(x,s,t) \bigr\vert =& \biggl\vert \int _{0}^{1}\bigl(G_{s}(x,rs,rt)s+G_{t}(x,rs,rt)t \bigr)\,dr \biggr\vert \\ \leq& \int_{0}^{1}\bigl( \bigl\vert G_{s}(x,rs,rt) \bigr\vert \vert s \vert +G_{t}(x,rs,rt)| \vert t \vert \bigr)\,dr \\ \leq&c_{0} \int_{0}^{1}\bigl( \vert s \vert + \vert s \vert ^{p_{1}}r^{p_{1}-1}+ \vert rt \vert ^{\frac{q_{1}(p_{1}-1)}{p_{1}}} \vert s \vert \\ &{}+ \vert t \vert + \vert t \vert ^{q_{1}}r^{q_{1}-1}+ \vert rs \vert ^{\frac{p_{1}(q_{1}-1)}{q_{1}}} \vert t \vert \bigr)\,dr \\ \leq&c_{0} \biggl( \vert s \vert + \vert t \vert + \frac{1}{p_{1}} \vert s \vert ^{p_{1}}+\frac{1}{q_{1}} \vert t \vert ^{q_{1}} \\ &{} +\frac{p_{1}}{p_{1}+q_{1}(p_{1}-1)} \vert t \vert ^{\frac {q_{1}(p_{1}-1)}{p_{1}}} \vert s \vert + \frac{q_{1}}{p_{1}(q_{1}-1)+q_{1}} \vert s \vert ^{\frac {p_{1}(q_{1}-1)}{q_{1}}} \vert t \vert \biggr) \\ \leq&M_{10}\bigl( \vert s \vert + \vert t \vert + \vert s \vert ^{p_{1}}+ \vert t \vert ^{q_{1}}\bigr), \end{aligned}$$

  where $M_{10}$ is a positive constant independent of $(s,t)$. From (31) and the above inequality, there is a positive constant $M_{11}$ such that

  $$G(x,s,t)\leq\hat{\alpha} F(s,t)+M_{11}\bigl( \vert s \vert ^{p_{1}}+ \vert t \vert ^{q_{1}}\bigr)\quad \mbox{for all }(x,s,t) \in\Omega\times R^{2}. $$

  Hence, from the above inequality, for any $(u,v)\in C_{+}$, we have

  $$\begin{aligned} J(u,v) \geq&\frac{\alpha+1}{p} \Vert u \Vert ^{p}_{p}+ \frac{\beta +1}{q} \Vert v \Vert ^{q}_{q}-\hat{\alpha} \int_{\Omega}F(u,v)\,dx-M_{11}\bigl( \Vert u \Vert ^{p_{1}}_{L^{p_{1}}}+ \Vert v \Vert ^{q_{1}}_{L^{q_{1}}} \bigr) \\ \geq&\frac{\lambda_{k+1}-\hat{\alpha}}{\lambda_{k+1}}\biggl(\frac{\alpha +1}{p} \Vert u \Vert ^{p}_{p}+\frac{\beta+1}{q} \Vert u \Vert ^{q}_{q}\biggr)- CM_{11}\bigl( \Vert u \Vert ^{p_{1}}_{p}+ \Vert v \Vert ^{q_{1}}_{q} \bigr). \end{aligned}$$

  In view of $\lambda_{k+1}>\hat{\alpha}$ and $p< p_{1}$, $q< q_{1}$, there is a positive constant $r_{-}>r_{+}>0$ such that

  $$J(u,v)>0 $$

  for any $(u,v)\in S_{+}=\{(u,v)\in C_{+}: \phi(u,v)=r_{+}\}$.

Finally, from Theorem A and Lemma 3, Theorem 3 is proved. □