## Boundary Value Problems

Impact Factor 0.642

Open Access

# Multiple solutions for p-Laplacian systems with critical homogeneous nonlinearity

Boundary Value Problems20122012:27

DOI: 10.1186/1687-2770-2012-27

Accepted: 28 February 2012

Published: 28 February 2012

## Abstract

In this article, we deal with existence and multiplicity of solutions to the p-Laplacian system of the type

$\left\{\begin{array}{cc}-{\Delta }_{p}u=\frac{1}{{p}^{*}}\frac{\partial F\left(x,u,v\right)}{\partial u}+\lambda {\left|u\right|}^{q-2}u,\hfill & \hfill x\in \Omega ,\hfill \\ -{\Delta }_{p}v=\frac{1}{{p}^{*}}\frac{\partial F\left(x,u,v\right)}{\partial v}+\delta {\left|v\right|}^{q-2}v,\hfill & \hfill x\in \Omega ,\hfill \\ u=v=0,\hfill & \hfill x\in \partial \Omega ,\hfill \end{array}\right\$

where Ω N is a bounded domain with smooth boundary ∂Ω, Δ p u = div(|u|p-2u) is the p-Laplacian operator, $N\ge {p}^{2},2\le p\le q<{p}^{*},{p}^{*}=\frac{Np}{N-p}$ denotes the Sobolev critical exponent, $F\in {C}^{1}\left(\stackrel{̄}{\Omega }×{ℝ}^{+}×{ℝ}^{+},{ℝ}^{+}\right)$ is a homogeneous function of degree p*. By using the variational method and Ljusternik-Schnirelmann theory, we prove that the system has at least catΩ(Ω) distinct nonnegative solutions.

AMS 2010 Mathematics Subject Classifications: 35J50; 35B33.

### Keywords

p-Laplacian system Ljusternik-Schnirelmann theory critical exponent multiple solutions

## 1 Introduction and main results

In this article, we consider the existence and multiplicity of solutions for the following critical p-Laplacian system:
$\left\{\begin{array}{cc}-{\Delta }_{p}u=\frac{1}{{p}^{*}}\frac{\partial F\left(x,u,v\right)}{\partial u}+\lambda {\left|u\right|}^{q-2}u,\hfill & \hfill x\in \Omega ,\hfill \\ -{\Delta }_{p}v=\frac{1}{{p}^{*}}\frac{\partial F\left(x,u,v\right)}{\partial v}+\delta {\left|v\right|}^{q-2}v,\hfill & \hfill x\in \Omega ,\hfill \\ u=v=0,\hfill & \hfill x\in \partial \Omega ,\hfill \end{array}\right\$
(1.1)

where Ω N is a bounded domain with smooth boundary ∂Ω, Δ p u = div(|u|p-2u) is the p-Laplacian operator, $N\ge {p}^{2},2\le p\le q<{p}^{*},{p}^{*}=\frac{Np}{N-p}$ denotes the Sobolev critical exponent, $F\in {C}^{1}\left(\stackrel{̄}{\Omega }×{ℝ}^{+}×{ℝ}^{+},{ℝ}^{+}\right)$ is a homogeneous function of degree ${p}^{*},\left(\frac{\partial F\left(x,u,v\right)}{\partial u},\frac{\partial F\left(x,u,v\right)}{\partial v}\right)=\nabla F$ and λ, δ are positive parameters.

The starting point on the study of the system (1.1) is its scalar version:
$\left\{\begin{array}{cc}-{\Delta }_{p}u={\left|u\right|}^{{p}^{*}-2}u+\lambda {\left|u\right|}^{q-2}u,\hfill & \hfill x\in \Omega \hfill \\ u=0,\hfill & \hfill x\in \partial \Omega ,\hfill \end{array}\right\$
(1.2)

with 2 ≤ pq < p*. In a pioneer work Brezis and Nirenberg [1] showed that, if p = q = 2, the equation (1.2) has at least one positive solution provided N ≥ 4 and 0 < λ < λ1, where λ1 is the first eigenvalue of the operator $\left(-\Delta ,{H}_{0}^{1}\left(\Omega \right)\right)$. In particular, the first multiplicity result for (1.2) has been achieved by Rey [2] in the semilinear case. Precisely Rey proved that if N ≥ 5, p = q = 2, for λ small enough equation (1.2) has at least catΩ(Ω) solutions, where catΩ(Ω) denotes the Ljusternik-Schnirelmann category of Ω in itself. Furthermore, Alves and Ding [3] obtained the existence of catΩ(Ω) positive solutions to equation (1.2) with p ≥ 2, pq < p*.

In recent years, more and more attention have been paid to the elliptic systems. In particular, Ding and Xiao [4] concerned the case F(x, u, v) = 2|u| α |v| β ,α > 1, β >1 satisfying α + β = p*, i.e., the following elliptic system
$\left\{\begin{array}{cc}-{\Delta }_{p}u=\frac{2\alpha }{\alpha +\beta }{\left|u\right|}^{\alpha -2}u{\left|v\right|}^{\beta }+\lambda {\left|u\right|}^{q-2}u,\hfill & \hfill x\in \Omega ,\hfill \\ -{\Delta }_{p}v=\frac{2\beta }{\alpha +\beta }{\left|u\right|}^{\alpha }{\left|v\right|}^{\beta -2}v+\delta {\left|v\right|}^{q-2}v,\hfill & \hfill x\in \Omega ,\hfill \\ u=v=0,\hfill & \hfill x\in \partial \Omega .\hfill \end{array}\right\$
(1.3)

Using standard tools of the variational theory and the Ljusternik-Schnirelmann category theory, Ding and Xiao [4] have proved that system (1.3) has at least catΩ(Ω) positive solutions if λ, δ satisfied a certain condition. Hsu [5] obtained the existence of two positive solutions of system (1.3) with the sublinear perturbation of 1 < q < p < N. Recently, Shen and Zhang [6] extended the results in [5] to the case (1.1) with 1 < q < p < N and obtained similar results. In this article, we study (1.1) and complement the results of [5, 6] to the case 2 ≤ pq < p*, also extend the results of [4, 7]. To the best of our knowledge, problem (1.1) has not been considered before. Thus it is necessary for us to investigate the critical p-Laplacian systems (1.1) deeply. For more similar problems, we refer to [817], and references therein.

Before stating our results, we need the following assumptions:

(F0) $F\in {C}^{1}\left(\stackrel{̄}{\Omega }×{ℝ}^{+}×{ℝ}^{+},{ℝ}^{+}\right)$ and $F\left(x,tu,tv\right)={t}^{{p}^{*}}F\left(x,u,v\right)\left(t>0\right)$ holds for all $\left(x,u,v\right)\in \stackrel{̄}{\Omega }×{ℝ}^{+}×{ℝ}^{+}$;

(F1) $F\left(x,u,0\right)=F\left(x,0,v\right)=\frac{\partial F\left(x,u,0\right)}{\partial u}=\frac{\partial F\left(x,0,v\right)}{\partial v}=0$, where u, v +;

(F2) $\frac{\partial F\left(x,u,v\right)}{\partial u},\frac{\partial F\left(x,u,v\right)}{\partial v}$ are strictly increasing functions about u and v for all u, v > 0.

The main results we get are the following:

Theorem 1.1. Suppose Np2and F satisfies (F0)-(F2), then the problem (1.1) has at least one nonnegative solution for 2 ≤ p < q < p* and λ, δ > 0, or q = p and λ, δ (0, Λ1), where Λ1is the first eigenvalue of$\left(-{\Delta }_{p},{W}_{0}^{1,p}\left(\Omega \right)\right)$.

Theorem 1.2. Suppose Np2, 2 ≤ pq < p* and F satisfies (F0)-(F2), then there exists Λ > 0 such that the problem (1.1) has at least catΩ(Ω) distinct nonnegative solutions for λ, δ (0,Λ).

Remark 1.1. Theorem 1 in[4]is the special case of our Theorem 1.2 corresponding to F(x,u,v) = 2|u| α |v| β ,α > 1,β > 1,α + β = p*. There are functions F(x,u,v) satisfying the conditions of our Theorems 1.1 and 1.2. Some typical examples are:

(i)$F\left(x,u,v\right)={\sum }_{i=1}^{k}{f}_{i}\left(x\right){\left|u\right|}^{{\alpha }_{i}}{\left|v\right|}^{{\beta }_{i}};$;

(ii) $F\left(x,u,v\right)=\left\{\begin{array}{c}{f}_{1}\left(x\right){\left|u\right|}^{\frac{3}{2}}{\left|v\right|}^{\frac{5}{2}}+{f}_{2}\left(x\right)\frac{{u}^{3}{v}^{3}}{{u}^{2}+{v}^{2}},\phantom{\rule{2.77695pt}{0ex}}\phantom{\rule{2.77695pt}{0ex}}\left(u,v\right)\ne \left(0,0\right),\hfill \\ 0,\phantom{\rule{2.77695pt}{0ex}}\phantom{\rule{2.77695pt}{0ex}}\left(u,v\right)=\left(0,0\right),\hfill \end{array}\right\$

where${f}_{i}\left(x\right)\ge 0,{f}_{i}\left(x\right)\not\equiv 0,{f}_{i}\left(x\right)\in C\left(\stackrel{̄}{\Omega }\right)\cap {L}^{\infty }\left(\Omega \right),{\alpha }_{i},{\beta }_{i}>1,{\alpha }_{i}+{\beta }_{i}={p}^{*}$. Obviously, F(x, u, v) satisfies (F0)-(F2).

This article is organized as follows. In Section 2, some notations and the Mountain-Pass levels are established and the Theorem 1.1 is proved. We present some technical lemmas which are crucial in the proof of the Theorem 1.2 in Section 3. Theorem 1.2 is proved in Section 4.

## 2 Notations and proof of Theorem 1.1

Throughout this article, C, C i will denote various positive constants whose exact values are not important, → (respectively ) denotes strong (respectively weak) convergence. O(ε t ) denotes |O(ε t )|/ε t C, o m (1) denotes o m (1) → 0 as m → ∞. L s (Ω)(1 ≤ s < +∞) denotes Lebesgue spaces, the norm L s is denoted by | · | s for 1 ≤ s < + ∞. Let B r (x) denotes a ball centered at x with radius r, the dual space of a Banach space E will be denoted by E-1. We define the product space $E:={W}_{0}^{1,p}\left(\Omega \right)×{W}_{0}^{1,p}\left(\Omega \right)$ endowed with the norm ${∥\left(u,v\right)∥}_{E}={\left({∥u∥}_{{W}_{0}^{1,p}\left(\Omega \right)}^{p}+{∥v∥}_{{W}_{0}^{1,p}\left(\Omega \right)}^{p}\right)}^{\frac{1}{p}}$, and the norm ${∥u∥}_{{W}_{0}^{1,p}\left(\Omega \right)}={\left({\int }_{\Omega }{\left|\nabla u\right|}^{p}dx\right)}^{\frac{1}{p}}$.

Using assumption of (F1), we have the so-called Euler identity
$\frac{\partial F\left(x,u,v\right)}{\partial u}u+\frac{\partial F\left(x,u,v\right)}{\partial v}v={p}^{*}F\left(x,u,v\right).$
(2.1)

In addition, we can extend the function F(x,u,v) to the whole $\stackrel{̄}{\Omega }×{ℝ}^{2}$ by considering $\stackrel{̃}{F}\left(x,u,v\right)=F\left(x,{u}^{+},{v}^{+}\right)$, where u+ = max{u,0}. It is easy to check that $\stackrel{̃}{F}\left(x,u,v\right)$ is of class C1 and its restriction to $\stackrel{̄}{\Omega }×{ℝ}^{+}×{ℝ}^{+}$ coincides with F(x,u,v). In order to simplify the notation we shall write, from now on, only F(x,u,v) to denote the above extension.

A pair of functions (u, v) E is said to be a weak solution of problem (1.1) if
$\begin{array}{l}\underset{\Omega }{\int }\left({\left|\nabla u\right|}^{p-2}\nabla u\nabla {\phi }_{1}+{\left|\nabla v\right|}^{p-2}\nabla v\nabla {\phi }_{2}\right)dx-\frac{1}{{p}^{*}}\underset{\Omega }{\int }\left(\frac{\partial F\left(x,u,v\right)}{\partial u}{\phi }_{1}+\frac{\partial F\left(x,u,v\right)}{\partial v}{\phi }_{2}\right)dx\phantom{\rule{2em}{0ex}}\\ \phantom{\rule{1em}{0ex}}-\underset{\Omega }{\int }\left(\lambda {\left|u\right|}^{q-2}u{\phi }_{1}+\delta {\left|v\right|}^{q-2}v{\phi }_{2}\right)dx=0,\phantom{\rule{1em}{0ex}}\forall \left({\phi }_{1},{\phi }_{2}\right)\in E.\phantom{\rule{2em}{0ex}}\end{array}$
Thus, by (2.1) the corresponding energy functional of problem (1.1) is defined on E by
${I}_{\lambda ,\delta }\left(u,v\right)=\frac{1}{p}\underset{\Omega }{\int }\left({\left|\nabla u\right|}^{p}+{\left|\nabla v\right|}^{p}\right)dx-\frac{1}{{p}^{*}}\underset{\Omega }{\int }F\left(x,u,v\right)dx-\frac{1}{q}\underset{\Omega }{\int }\left(\lambda {\left|u\right|}^{q}+\delta {\left|v\right|}^{q}\right)dx.$

Using (F0)-(F2), we can verify I λ, δ (u, v) C1(E, ) (see [6]). It is well known that the weak solutions of problem (1.1) are the critical points of the energy functional I λ, δ (u, v).

The functional I C1(E, ) is said to satisfy the (PS) c condition if any sequence {u m } E such that as m → ∞, I(u m ) → c, I'(u m ) → 0 strongly in E-1 contains a subsequence converging in E to a critical point of I. In this article, we will take I = I λ, δ (u, v) and $E:={W}_{0}^{1,p}\left(\Omega \right)×{W}_{0}^{1,p}\left(\Omega \right)$.

As the energy functional Iλ,δis not bounded below on E, we need to study Iλ,δon the Nehari manifold
${\mathcal{N}}_{\lambda ,\delta }=\left\{\left(u,v\right)\in E\\left\{\left(0,0\right)\right\}:⟨{I}_{\lambda ,\delta }^{\prime }\left(u,v\right),\left(u,v\right)⟩=0\right\}.$
Note that ${\mathcal{N}}_{\lambda ,\delta }$ contains every nonzero solution of problem (1.1), and define the minimax cλ,δas
${c}_{\lambda ,\delta }=\underset{\left(u,v\right)\in {\mathcal{N}}_{\lambda ,\delta }}{\text{inf}}{I}_{\lambda ,\delta }\left(u,v\right).$
Next, we present some properties of cλ,δand ${\mathcal{N}}_{\lambda ,\delta }$. Its proofs can be done as [18, Theorem 4.2]. First of all, we note that there exists ρ > 0, such that
${∥\left(u,v\right)∥}_{E}\ge \rho >0,\phantom{\rule{2.77695pt}{0ex}}\phantom{\rule{2.77695pt}{0ex}}\forall \left(u,v\right)\in {\mathcal{N}}_{\lambda ,\delta }.$
It is standard to check that Iλ,δsatisfies Mountain-Pass geometry, so we can use the homogeneity of F to prove that cλ,δcan be alternatively characterized by
${c}_{\lambda ,\delta }=\underset{\gamma \in \Gamma }{\text{inf}}\underset{t\in \left[0,1\right]}{\text{max}}{I}_{\lambda ,\delta }\left(\gamma \left(t\right)\right)=\underset{\left(u,v\right)\in E\\left\{\left(0,0\right)\right\}}{\text{inf}}\underset{t\ge 0}{\text{max}}{I}_{\lambda ,\delta }\left(t\left(u,v\right)\right)>0,$
(2.2)

where Γ = {γ C([0, 1],E) : γ(0) = 0,Iλ,δ(γ(1)) < 0}. Moreover, for each (u, v) E\{(0,0)}, there exists a unique t* > 0 such that ${t}^{*}\left(u,v\right)\in {\mathcal{N}}_{\lambda ,\delta }$. The maximum of the function t Iλ,δ(t(u, v)), for t ≥ 0, is achieved at t = t*.

In this section, we will find the range of c where the (PS) c condition holds for the functional Iλ,δ. First let us define
${S}_{F}=\underset{\left(u,v\right)\in E\\left\{\left(0,0\right)\right\}}{\text{inf}}\left\{\frac{{\int }_{\Omega }{\left|\nabla u\right|}^{p}+{\left|\nabla v\right|}^{p}dx}{{\left({\int }_{\Omega }F\left(x,u,v\right)dx\right)}^{\frac{p}{{p}^{*}}}}:\underset{\Omega }{\int }F\left(x,u,v\right)dx>0\right\}.$
(2.3)

Lemma 2.1. If Np2and F satisfies (F0)-(F2), then the functional Iλ,δsatisfies the (PS) c condition for all$c<\frac{1}{N}{S}_{F}^{\frac{N}{p}}$, provide one of the following conditions holds

(i) 2 ≤ p < q < p* and λ, δ > 0;

(ii) q = p, and λ, δ (0, Λ1), where Λ1 > 0 denotes the first eigenvalue of$\left(-{\Delta }_{p},{W}_{0}^{1,p}\left(\Omega \right)\right)$.

Proof. Let {(u m , v m )} E such that ${I}_{\lambda ,\delta }^{\prime }\left({u}_{m},{v}_{m}\right)\to 0$ and ${I}_{\lambda ,\delta }\left({u}_{m},{v}_{m}\right)\to c<\frac{1}{N}{S}_{F}^{\frac{N}{p}}$. Now, we first prove that {(u m , v m )} is bounded in E. If the above item (i) is true it suffices to use the definition of Iλ,δto obtain C1 > 0 such that
$\begin{array}{ll}\hfill c+{C}_{1}{∥\left({u}_{m},{v}_{m}\right)∥}_{E}+{o}_{m}\left(1\right)& \ge {I}_{\lambda ,\delta }\left({u}_{m},{v}_{m}\right)-\frac{1}{q}⟨{I}_{\lambda ,\delta }^{\prime }\left({u}_{m},{v}_{m}\right),\left({u}_{m},{v}_{m}\right)⟩\phantom{\rule{2em}{0ex}}\\ =\left(\frac{1}{p}-\frac{1}{q}\right){∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}+\left(\frac{1}{q}-\frac{1}{{p}^{*}}\right)\underset{\Omega }{\int }F\left(x,{u}_{m},{v}_{m}\right)dx\phantom{\rule{2em}{0ex}}\\ \ge \frac{q-p}{pq}{∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}.\phantom{\rule{2em}{0ex}}\end{array}$
The above expression implies that {(u m , v m )} E is bounded. When (ii) occurs, in this case, it follows that
$\underset{\Omega }{\int }\left(\lambda {\left|{u}_{m}\right|}^{p}+\delta {\left|{v}_{m}\right|}^{p}\right)dx\le \text{max}\left\{\lambda ,\delta \right\}\underset{\Omega }{\int }\left({\left|{u}_{m}\right|}^{p}+{\left|{v}_{m}\right|}^{p}\right)dx\le \frac{\text{max}\left\{\lambda ,\delta \right\}}{{\Lambda }_{1}}\underset{E}{\overset{p}{∥\left({u}_{m},{v}_{m}\right)∥}},$
and therefore we get
$\begin{array}{ll}\hfill c+{C}_{1}{∥\left({u}_{m},{v}_{m}\right)∥}_{E}+{o}_{m}\left(1\right)& \ge {I}_{\lambda ,\delta }\left({u}_{m},{v}_{m}\right)-\frac{1}{{p}^{*}}⟨{I}_{\lambda ,\delta }^{\prime }\left({u}_{m},{v}_{m}\right),\left({u}_{m},{v}_{m}\right)⟩\phantom{\rule{2em}{0ex}}\\ =\left(\frac{1}{p}-\frac{1}{{p}^{*}}\right){∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}+\left(\frac{1}{{p}^{*}}-\frac{1}{p}\right)\underset{\Omega }{\int }\left(\lambda {\left|{u}_{m}\right|}^{p}+\delta {\left|{v}_{m}\right|}^{p}\right)dx\phantom{\rule{2em}{0ex}}\\ \ge \frac{1}{N}\left(1-\frac{\text{max}\left\{\lambda ,\delta \right\}}{{\Lambda }_{1}}\right){∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}.\phantom{\rule{2em}{0ex}}\end{array}$

Since λ, δ (0,Λ1) the boundedness of {(u m , v m )} follows as the first case.

So, {(u m , v m )} is bounded in E. Going if necessary to a subsequence, we can assume that
$\left\{\begin{array}{cc}\left({u}_{m},{v}_{m}\right)⇀\left(u,v\right),\hfill & \hfill \mathsf{\text{in}}\phantom{\rule{2.77695pt}{0ex}}E,\hfill \\ \left({u}_{m},{v}_{m}\right)\to \left(u,v\right),\hfill & \hfill \mathsf{\text{a}}\mathsf{\text{.e}}\mathsf{\text{.}}\phantom{\rule{2.77695pt}{0ex}}\mathsf{\text{in}}\phantom{\rule{2.77695pt}{0ex}}\Omega \hfill \\ \left({u}_{m},{v}_{m}\right)\to \left(u,v\right),\hfill & \hfill \mathsf{\text{in}}\phantom{\rule{2.77695pt}{0ex}}{L}^{s}\left(\Omega \right)×{L}^{s}\left(\Omega \right),1\le s<{p}^{*},\hfill \end{array}\right\$
as m → ∞. Clearly, we have
$\underset{\Omega }{\int }\left(\lambda {\left|{u}_{m}\right|}^{q}+\delta {\left|{v}_{m}\right|}^{q}\right)dx=\underset{\Omega }{\int }\left(\lambda {\left|u\right|}^{q}+\delta {\left|v\right|}^{q}\right)dx+{o}_{m}\left(1\right).$
(2.4)
Moreover, a standard argument shows that ${I}_{\lambda ,\delta }^{\prime }\left(u,v\right)=0$. Thus we get
$\begin{array}{ll}\hfill {I}_{\lambda ,\delta }\left(u,v\right)& =\frac{1}{p}{∥\left(u,v\right)∥}_{E}^{p}-\frac{1}{{p}^{*}}\underset{\Omega }{\int }F\left(x,u,v\right)dx-\frac{1}{q}\underset{\Omega }{\int }\left(\lambda {\left|u\right|}^{q}+\delta {\left|v\right|}^{q}\right)dx\phantom{\rule{2em}{0ex}}\\ =\left(\frac{1}{p}-\frac{1}{q}\right){∥\left(u,v\right)∥}_{E}^{p}+\left(\frac{1}{q}-\frac{1}{{p}^{*}}\right)\underset{\Omega }{\int }F\left(x,u,v\right)dx\phantom{\rule{2em}{0ex}}\\ \ge 0.\phantom{\rule{2em}{0ex}}\end{array}$
(2.5)
Let $\left({ũ}_{m},{ṽ}_{m}\right)=\left({u}_{m}-u,{v}_{m}-v\right)$, then by Brezis-Lieb Lemma in [19] implies
${∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}={∥\left(u,v\right)∥}_{E}^{p}+{∥\left({ũ}_{m},{ṽ}_{m}\right)∥}_{E}^{p}+{o}_{m}\left(1\right).$
(2.6)
By the same method of [8, Lemma 5] (or [6, Lemma 3.4]), we obtain
$\underset{\Omega }{\int }F\left(x,{u}_{m},{v}_{m}\right)dx=\underset{\Omega }{\int }F\left(x,u,v\right)dx+\underset{\Omega }{\int }F\left(x,{ũ}_{m},{ṽ}_{m}\right)dx+{o}_{m}\left(1\right).$
(2.7)
By (2.4)-(2.7) and the weak convergence of (u m , v m ), we have
$\begin{array}{ll}\hfill c+{o}_{m}\left(1\right)& ={I}_{\lambda ,\delta }\left(u,v\right)+\frac{1}{p}{∥\left({ũ}_{m},{ṽ}_{m}\right)∥}_{E}^{p}-\frac{1}{{p}^{*}}\underset{\Omega }{\int }F\left(x,{ũ}_{m},{ṽ}_{m}\right)dx\phantom{\rule{2em}{0ex}}\\ \ge \frac{1}{p}{∥\left({ũ}_{m},{ṽ}_{m}\right)∥}_{E}^{p}-\frac{1}{{p}^{*}}\underset{\Omega }{\int }F\left(x,{ũ}_{m},{ṽ}_{m}\right)dx.\phantom{\rule{2em}{0ex}}\end{array}$
(2.8)
By using ${I}_{\lambda ,\delta }^{\prime }\left({u}_{m},{v}_{m}\right)\to 0$ and (2.4), (2.6), and (2.7), we get
$\begin{array}{ll}\hfill {o}_{m}\left(1\right)& =⟨{I}_{\lambda ,\delta }^{\prime }\left({u}_{m},{v}_{m}\right),\left({u}_{m},{v}_{m}\right)⟩\phantom{\rule{2em}{0ex}}\\ ={∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}-\underset{\Omega }{\int }F\left(x,{u}_{m},{v}_{m}\right)dx-\underset{\Omega }{\int }\left(\lambda {\left|{u}_{m}\right|}^{q}+\delta {\left|{v}_{m}\right|}^{q}\right)dx\phantom{\rule{2em}{0ex}}\\ =⟨{I}_{\lambda ,\delta }^{\prime }\left(u,v\right),\left(u,v\right)⟩+{∥\left({ũ}_{m},{ṽ}_{m}\right)∥}_{E}^{p}-\underset{\Omega }{\int }F\left(x,{ũ}_{m},{ṽ}_{m}\right)dx.\phantom{\rule{2em}{0ex}}\end{array}$
Recalling that ${I}_{\lambda ,\delta }^{\prime }\left(u,v\right)=0$, we can use the above equality and (2.8) to obtain
$\underset{m\to \infty }{\text{lim}}{∥\left({ũ}_{m},{ṽ}_{m}\right)∥}_{E}^{p}=k=\underset{m\to \infty }{\text{lim}}\underset{\Omega }{\int }F\left(x,{ũ}_{m},{ṽ}_{m}\right)dx,\phantom{\rule{1em}{0ex}}c\ge \left(\frac{1}{p}-\frac{1}{{p}^{*}}\right)k=\frac{1}{N}k,$

where k is a nonnegative number.

In view of the definition of S F , we have that
${∥\left({ũ}_{m},{ṽ}_{m}\right)∥}_{E}^{p}\ge {S}_{F}{\left(\underset{\Omega }{\int }F\left(x,{ũ}_{m},{ṽ}_{m}\right)dx\right)}^{\frac{p}{{p}^{*}}}.$
Taking the limit we get $k\ge {S}_{F}{k}^{\frac{p}{p*}}$. So, if k > 0, we conclude that $k\ge {S}_{F}^{\frac{N}{p}}$ and therefore
$\frac{1}{N}{S}_{F}^{\frac{N}{p}}\le \frac{1}{N}k\le c<\frac{1}{N}{S}_{F}^{\frac{N}{p}},$

which is a contradiction. Hence k = 0 and therefore (u m , v m ) → (u, v) strongly in E.

Before presenting our next result we recall that, for each ε > 0, the function
${U}_{\epsilon }\left(x\right)=\frac{{C}_{N}\cdot {\epsilon }^{\frac{N-p}{{p}^{2}}}}{{\left(\epsilon +{\left|x\right|}^{\frac{p}{p-1}}\right)}^{\frac{N-p}{p}}},\phantom{\rule{1em}{0ex}}{C}_{N}={\left(N{\left(\frac{N-p}{p-1}\right)}^{p-1}\right)}^{\frac{N-p}{{p}^{2}}},\phantom{\rule{1em}{0ex}}x\in {ℝ}^{N}$
(2.9)
satisfies
${\left|\nabla {U}_{\epsilon }\left(x\right)\right|}_{p}^{p}={\left|{U}_{\epsilon }\left(x\right)\right|}_{{p}^{*}}^{{p}^{*}}={S}^{\frac{N}{p}},$
(2.10)
where S is the best constant of the Sobolev embedding ${D}^{1,p}\left({ℝ}^{N}\right)↪{L}^{{p}^{*}}\left({ℝ}^{N}\right)$. Thus, using [8, Lemma 3] and the homogeneity of F, we obtain A, B > 0 such that
${S}_{F}=\frac{{∥\left(A{U}_{\epsilon },B{U}_{\epsilon }\right)∥}_{E}^{p}}{{\left({\int }_{{ℝ}^{N}}F\left(x,A{U}_{\epsilon }B{U}_{e}\right)dx\right)}^{\frac{p}{{p}^{*}}}}=\frac{{A}^{p}+{B}^{p}}{{\left(F\left(x,A,B\right)\right)}^{\frac{p}{{p}^{*}}}}\cdot \frac{{S}^{\frac{N}{p}}}{{\left|{U}_{\epsilon }\right|}_{{p}^{*}}^{p}},$
from which and (2.10) it follows that
${S}_{F}=\frac{{A}^{p}+{B}^{p}}{{\left(F\left(x,A,B\right)\right)}^{\frac{p}{{p}^{*}}}}S.$
(2.11)
We define a cut-off function $\varphi \left(x\right)\in {C}_{0}^{\infty }\left({ℝ}^{N}\right)$ such that ϕ(x) = 1 if |x| ≤ R; ϕ(x) = 0 if |x| ≥ 2R and 0 ≤ ϕ(x) ≤ 1, where B2R(0) Ω, set ${u}_{\epsilon }=\frac{\varphi \left(x\right){U}_{\epsilon }}{{\left|\varphi {U}_{\epsilon }\right|}_{{p}^{*}}}$, where U ε was defined in (2.9). So that ${\left|{u}_{\epsilon }\right|}_{{p}^{*}}=1$. Then, we can get the following results from [[20], Lemma 11.1]:
${∥{u}_{\epsilon }∥}_{{W}_{0}^{1,p}\left(\Omega \right)}^{p}=S+O\left({\epsilon }^{\frac{N-p}{p}}\right),$
(2.12)
$\underset{\Omega }{\int }{\left|{u}_{\epsilon }\right|}^{\xi }dx\approx \left\{\begin{array}{cc}{\epsilon }^{\frac{N-p}{{p}^{2}}\xi },\hfill & \hfill \mathsf{\text{if}}1<\xi <{p}^{*}\left(1-\frac{1}{p}\right),\hfill \\ {\epsilon }^{\frac{N-p}{{p}^{2}}\xi }\left|\text{ln}\epsilon \right|,\hfill & \hfill \mathsf{\text{if}}\phantom{\rule{2.77695pt}{0ex}}\xi ={p}^{*}\left(1-\frac{1}{p}\right),\hfill \\ {\epsilon }^{\frac{\left(p-1\right)\left(Np-\xi \left(N-p\right)\right)}{{p}^{2}}},\hfill & \hfill \mathsf{\text{if}}\phantom{\rule{2.77695pt}{0ex}}{p}^{*}\left(1-\frac{1}{p}\right)<\xi <{p}^{*},\hfill \end{array}\right\$
(2.13)

where AB means C1BAC2B.

Lemma 2.2. Suppose that F satisfies (F0)-(F2), 2 ≤ p < q < p* and λ > 0, δ > 0, then${c}_{\lambda ,\delta }<\frac{1}{N}{S}_{F}^{\frac{N}{p}}$. The same result holds if q = p and λ, δ (0,Λ1), where Λ1 > 0 denotes the first eigenvalue of$\left(-{\Delta }_{p},{W}_{0}^{1,p}\left(\Omega \right)\right)$.

Proof. We can use the homogeneity of F to get, for any t ≥ 0,
${I}_{\lambda ,\delta }\left(tA{u}_{\epsilon },tB{u}_{\epsilon }\right)=\frac{{t}^{p}}{p}\left({A}^{p}+{B}^{p}\right){∥{u}_{\epsilon }∥}_{{W}_{0}^{1,p}\left(\Omega \right)}^{p}-\frac{{t}^{{p}^{*}}}{{p}^{*}}F\left(x,A,B\right)-\frac{{t}^{q}}{q}\left(\lambda {A}^{q}+\delta {B}^{q}\right){\left|{u}_{\epsilon }\right|}_{q}^{q}.$

We shall denote by h(t) the right-hand side of the above equality and consider two distinct cases.

Case 1. 2 ≤ p < q < p*.

From the fact that $\underset{t\to +\infty }{\text{lim}}h\left(t\right)=-\infty$ and h(t) > 0 when t is close to 0, there exists t ε > 0 such that
$h\left({t}_{\epsilon }\right)=\underset{t\ge 0}{\text{max}}h\left(t\right).$
(2.14)
Let
$g\left(t\right)=\frac{{t}^{p}}{p}\left({A}^{p}+{B}^{p}\right){∥{u}_{\epsilon }∥}_{{W}_{0}^{1,p}\left(\Omega \right)}^{p}-\frac{{t}^{{p}^{*}}}{{p}^{*}}F\left(x,A,B\right),\phantom{\rule{2.77695pt}{0ex}}\phantom{\rule{2.77695pt}{0ex}}t\ge 0,$
and notice that the maximum value of g(t) occurs at the point
${\stackrel{̃}{t}}_{\epsilon }={\left(\frac{\left({A}^{p}+{B}^{p}\right){∥{u}_{\epsilon }∥}_{{W}_{0}^{1,p}\left(\Omega \right)}^{p}}{F\left(x,A,B\right)}\right)}^{\frac{1}{{p}^{*}-p}}.$
So, for each t ≥ 0,
$g\left(t\right)\le g\left({\stackrel{̃}{t}}_{\epsilon }\right)=\frac{1}{N}{\left(\frac{\left({A}^{p}+{B}^{p}\right){∥{u}_{\epsilon }∥}_{{W}_{0}^{1,p}\left(\Omega \right)}^{p}}{{\left(F\left(x,A,B\right)\right)}^{\frac{p}{{p}^{*}}}}\right)}^{\frac{N}{p}},$
and therefore
$h\left({t}_{\epsilon }\right)\le \frac{1}{N}{\left(\frac{\left({A}^{p}+{B}^{p}\right){∥{u}_{\epsilon }∥}_{{W}_{0}^{1,p}\left(\Omega \right)}^{p}}{{\left(F\left(x,A,B\right)\right)}^{\frac{p}{{p}^{*}}}}\right)}^{\frac{N}{p}}-\frac{{t}_{\epsilon }^{q}}{q}\left(\lambda {A}^{q}+\delta {B}^{q}\right){\left|{u}_{\epsilon }\right|}_{q}^{q}.$
(2.15)
We claim that, for some C2 > 0, there holds
${t}_{\epsilon }^{q}\left(\lambda {A}^{q}+\delta {B}^{q}\right)\ge {C}_{2}.$
Indeed, if this is not the case, we have that ${t}_{{\epsilon }_{m}}\to 0$ for some sequence ε m → 0+, then,
$0<{c}_{\lambda ,\delta }\le \underset{t\ge 0}{\text{sup}}{I}_{\lambda ,\delta }\left(tA{u}_{{\epsilon }_{m}},tB{u}_{{\epsilon }_{m}}\right)={I}_{\lambda ,\delta }\left({t}_{{\epsilon }_{m}}A{u}_{{\epsilon }_{m}},{t}_{{\epsilon }_{m}}B{u}_{{\epsilon }_{m}}\right)\to 0,$
which is a contradiction. So, the claim holds and we infer from (2.15) and (2.11)-(2.13) that
$\begin{array}{ll}\hfill h\left({t}_{\epsilon }\right)& \le \frac{1}{N}{\left(\frac{{A}^{p}+{B}^{p}}{{\left(F\left(x,A,B\right)\right)}^{\frac{p}{{p}^{*}}}}\left(S+O\left({\epsilon }^{\frac{N-p}{p}}\right)\right)\right)}^{\frac{N}{p}}-{C}_{3}{\left|{u}_{\epsilon }\right|}_{q}^{q}\phantom{\rule{2em}{0ex}}\\ \le \frac{1}{N}{S}_{F}^{\frac{N}{p}}+O\left({\epsilon }^{\frac{N-p}{p}}\right)-{C}_{3}{\left|{u}_{\epsilon }\right|}_{q}^{q}\phantom{\rule{2em}{0ex}}\\ \le \frac{1}{N}{S}_{F}^{\frac{N}{p}}+O\left({\epsilon }^{\frac{N-p}{p}}\right)-O\left({\epsilon }^{\frac{\left(p-1\right)\left(Np-q\left(N-p\right)\right)}{{p}^{2}}}\right),\phantom{\rule{2em}{0ex}}\end{array}$
(2.16)
where ${C}_{3}=\frac{{C}_{2}}{q}$. We know ${p}^{*}\left(1-\frac{1}{p}\right)\le p if Np2. By Np2 and 2 ≤ p < q < p* we obtain $\frac{N-p}{p}>\frac{\left(p-1\right)\left(Np-q\left(N-p\right)\right)}{{p}^{2}}$. Thus from the above inequality we conclude that, for each ε > 0 small, there holds
${c}_{\lambda ,\delta }\le \underset{t\ge 0}{\text{sup}}{I}_{\lambda ,\delta }\left(tA{u}_{{\epsilon }_{m}},tB{u}_{{\epsilon }_{m}}\right)=h\left({t}_{\epsilon }\right)<\frac{1}{N}{S}_{F}^{\frac{N}{p}}.$

Case 2. q = p.

In this case, we have that h'(t) = 0 if and only if,
$\left({A}^{p}+{B}^{p}\right){∥{u}_{\epsilon }∥}_{{W}_{0}^{1,p}\left(\Omega \right)}^{p}-\left(\lambda {A}^{p}-\delta {B}^{p}\right){\left|{u}_{\epsilon }\right|}_{p}^{p}={t}^{p*-p}F\left(x,A,B\right).$
Since we suppose λ, δ (0,Λ1), we can use Poincaré's inequality to obtain
$\begin{array}{ll}\hfill \left(\lambda {A}^{p}+\delta {B}^{p}\right){\left|{u}_{\epsilon }\right|}_{p}^{p}& \le \text{max}\left\{\lambda ,\delta \right\}\left({A}^{p}+{B}^{p}\right)\underset{p}{\overset{p}{\left|{u}_{\epsilon }\right|}}\phantom{\rule{2em}{0ex}}\\ <{\Lambda }_{1}\left({A}^{p}+{B}^{p}\right){\left|{u}_{\epsilon }\right|}_{p}^{p}\phantom{\rule{2em}{0ex}}\\ \le \left({A}^{p}+{B}^{p}\right){∥{u}_{\epsilon }∥}_{{W}_{0}^{1,p}\left(\Omega \right)}^{p}.\phantom{\rule{2em}{0ex}}\end{array}$

Thus, there exists t ε > 0 satisfying (2.14).

Arguing as in the first case we conclude that, from (2.16) for ε > 0 small, there holds
$\begin{array}{ll}\hfill h\left({t}_{\epsilon }\right)& \le \frac{1}{N}{S}_{F}^{\frac{N}{p}}+O\left({\epsilon }^{\frac{N-p}{p}}\right)-{C}_{3}{\left|{u}_{\epsilon }\right|}_{p}^{p}\phantom{\rule{2em}{0ex}}\\ =\left\{\begin{array}{c}\frac{1}{N}{S}_{F}^{\frac{N}{p}}+O\left({\epsilon }^{p-1}\right)-O\left({\epsilon }^{p-1}\left|\text{ln}\epsilon \right|\right),\phantom{\rule{2.77695pt}{0ex}}N={p}^{2},\hfill \\ \frac{1}{N}{S}_{F}^{\frac{N}{p}}+O\left({\epsilon }^{\frac{N-p}{p}}\right)-O\left({\epsilon }^{p-1}\right),\phantom{\rule{2.77695pt}{0ex}}\phantom{\rule{2.77695pt}{0ex}}N>{p}^{2}.\hfill \end{array}\right\\phantom{\rule{2em}{0ex}}\end{array}$
Because ${p}^{*}\left(1-\frac{1}{p}\right) if N > p2 and ${p}^{*}\left(1-\frac{1}{p}\right)=p$ if N = p2, then εp-1= o(εp-1| ln ε|). If N > p2, then $\frac{N-p}{p}>p-1$, so ${\epsilon }^{\frac{N-p}{p}}=o\left({\epsilon }^{p-1}\right)$. Choosing ε > 0 small enough, we have
${c}_{\lambda ,\delta }\le \underset{t\ge 0}{\text{sup}}{I}_{\lambda ,\delta }\left(tA{u}_{\epsilon },tB{u}_{\epsilon }\right)=h\left({t}_{\epsilon }\right)<\frac{1}{N}{S}_{F}^{\frac{N}{p}}.$

This concludes the proof.

By Lemmas 2.1 and 2.2 we can prove our first result.

Proof of Theorem 1.1.

Since Iλ,δsatisfies the geometric conditions of the Mountain-Pass theorem, there exists {(u m , v m )} E such that ${I}_{\lambda ,\delta }\left({u}_{m},{v}_{m}\right)\to {c}_{\lambda ,\delta },{I}_{\lambda ,\delta }^{\prime }\left({u}_{m},{v}_{m}\right)\to 0$. It follows from Lemmas 2.1 and 2.2 that {(u m , v m )} converges, along a subsequence, to a nonzero critical point (u,v) E of Iλ,δ. Then, if we denote by u- = max{-u,0} and v- = max{-v,0} the negative part of u and v, respectively, we get
$\begin{array}{ll}\hfill 0=⟨{I}_{\lambda ,\delta }^{\prime }\left(u,v\right),\left({u}^{-},{v}^{-}\right)⟩& =-{∥\left({u}^{-},{v}^{-}\right)∥}_{E}^{p}-\frac{1}{{p}^{*}}\underset{\Omega }{\int }\left(\frac{\partial F\left(x,u,v\right)}{\partial u}{u}^{-}+\frac{\partial F\left(x,u,v\right)}{\partial v}{v}^{-}\right)dx\phantom{\rule{2em}{0ex}}\\ \phantom{\rule{1em}{0ex}}-\underset{\Omega }{\int }\left(\lambda {\left|u\right|}^{q-2}u{u}^{-}+\delta {\left|v\right|}^{q-2}v{v}^{-}\right)dx\phantom{\rule{2em}{0ex}}\\ \le -{∥\left({u}^{-},{v}^{-}\right)∥}_{E}^{p},\phantom{\rule{2em}{0ex}}\end{array}$

it follows that (u-,v-) = (0,0). Hence, u,v ≥ 0 in Ω. The Theorem 1.1 is proved.

We finalize this section with the study of the asymptotic behavior of the minimax level cλ,δas both the parameters λ, δ approach zero.

Lemma 2.3. $\underset{\lambda ,\delta \to {0}^{+}}{\text{lim}}{c}_{\lambda ,\delta }={c}_{0,0}=\frac{1}{N}{S}_{F}^{\frac{N}{p}}.$.

Proof. We first prove the second equality. It follows from λ = δ = 0 that λ|u| q + δ|v| q ≡ 0. If A, B, u ε , g ε , and t ε are the same as those in the proof of Lemma 2.2, we have that $\left({t}_{\epsilon }A{u}_{\epsilon },{t}_{\epsilon }B{u}_{\epsilon }\right)\in {\mathcal{N}}_{0,0}$. Thus
$\begin{array}{ll}\hfill {c}_{0,0}& \le {I}_{0,0}\left({t}_{\epsilon }A{u}_{\epsilon },{t}_{\epsilon }B{u}_{\epsilon }\right)\phantom{\rule{2em}{0ex}}\\ =\frac{1}{N}{\left(\frac{{A}^{p}+{B}^{p}}{{\left(F\left(x,A,B\right)\right)}^{\frac{p}{{p}^{*}}}}{∥{u}_{\epsilon }∥}_{{W}_{0}^{1,p}\left(\Omega \right)}^{p}\right)}^{\frac{N}{p}}\phantom{\rule{2em}{0ex}}\\ =\frac{1}{N}{\left(\frac{{A}^{p}+{B}^{p}}{{\left(F\left(x,A,B\right)\right)}^{\frac{p}{{p}^{*}}}}\left(S+O\left({\epsilon }^{\frac{N-p}{p}}\right)\right)\right)}^{\frac{N}{p}}.\phantom{\rule{2em}{0ex}}\end{array}$

Taking the limit as ε →0+ and using (2.11), we conclude that ${c}_{0,0}\le \frac{1}{N}{S}_{F}^{\frac{N}{p}}$.

In order to obtain the reverse inequality we consider {(u m , v m )} E such that I0,0 (u m , v m ) → c0,0 and ${I}_{0,0}^{\prime }\left({u}_{m},{v}_{m}\right)\to 0$. It is easy to show that the sequence {(u m , v m )} is bounded in E and therefore $⟨{I}_{0,0}^{\prime }\left({u}_{m},{v}_{m}\right),\left({u}_{m},{v}_{m}\right)⟩={∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}-{\int }_{\Omega }F\left(x,{u}_{m},{v}_{m}\right)dx={o}_{m}\left(1\right)$. It follows that
$\underset{m\to \infty }{\text{lim}}{∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}=l=\underset{m\to \infty }{\text{lim}}\underset{\Omega }{\int }F\left(x,{u}_{m},{v}_{m}\right)dx.$
Taking the limit in the inequality ${S}_{F}{\left({\int }_{\Omega }F\left(x,{u}_{m},{v}_{m}\right)dx\right)}^{\frac{p}{{p}^{*}}}\le {∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}$ we conclude, as in the proof of Lemma 2.1, that $N{c}_{0,0}=l\ge {S}_{F}^{\frac{N}{p}}$. Hence,
${c}_{0,0}=\underset{m\to \infty }{\text{lim}}{I}_{0,0}\left({u}_{m},{v}_{m}\right)=\underset{m\to \infty }{\text{lim}}\left(\frac{1}{p}{∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}-\frac{1}{{p}^{*}}\underset{\Omega }{\int }F\left(x,{u}_{m},{v}_{m}\right)dx\right)=\frac{1}{N}l\ge \frac{1}{N}{S}_{F}^{\frac{N}{p}},$

and therefore ${c}_{0,0}=\frac{1}{N}{S}_{F}^{\frac{N}{p}}$.

We proceed now with the calculation of $\underset{\lambda ,\delta \to {0}^{+}}{\text{lim}}{c}_{\lambda ,\delta }$. Let {λ m },{δ m } + such that λ m , δ m → 0+. Since λ m , δ m are positive, we have that ${\int }_{\Omega }\left({\lambda }_{m}{\left|u\right|}^{q}+{\delta }_{m}{\left|v\right|}^{q}\right)dx\ge 0$ whenever (u, v) is nonnegative. Thus, for this kind of function, we have that ${I}_{{\lambda }_{m},{\delta }_{m}}\left(u,v\right)\le {I}_{0,0}\left(u,v\right)$.

It follows that
$\begin{array}{ll}\hfill {c}_{{\lambda }_{m},{\delta }_{m}}& =\underset{\left(u,v\right)\ne \left(0,0\right)}{\text{inf}}\underset{t\ge 0}{\text{max}}{I}_{{\lambda }_{m},{\delta }_{m}}\left(t\left(u,v\right)\right)\phantom{\rule{2em}{0ex}}\\ \le \underset{\underset{\left(u,v\right)\ge 0}{\left(u,v\right)\ne \left(0,0\right),}}{\text{inf}}\underset{t\ge 0}{\text{max}}{I}_{{\lambda }_{m},{\delta }_{m}}\left(t\left(u,v\right)\right)\phantom{\rule{2em}{0ex}}\\ \le \underset{\underset{\left(u,v\right)\ge 0}{\left(u,v\right)\ne \left(0,0\right),}}{\text{inf}}\underset{t\ge 0}{\text{max}}{I}_{0,0}\left(t\left(u,v\right)\right)={c}_{0,0},\phantom{\rule{2em}{0ex}}\end{array}$
in the last equality, we have used the infimum c0,0 which can be attained at a nonnegative solution. The above inequality implies that
$\underset{m\to \infty }{\text{lim}\text{sup}}{c}_{{\lambda }_{m,}{\delta }_{m}}\le {c}_{0,0}.$
(2.17)
On the other hand, it follows from Theorem 1.1 that there exists {(u m , v m )} E such that
${I}_{{\lambda }_{m},{\delta }_{m}}\left({u}_{m},{v}_{m}\right)={c}_{{\lambda }_{m},{\delta }_{m}},\phantom{\rule{1em}{0ex}}{I}_{{\lambda }_{m},{\delta }_{m}}^{\prime }\left({u}_{m},{v}_{m}\right)\to 0.$
Since ${c}_{{\lambda }_{m},{\delta }_{m}}$ is bounded, the same argument performed in the proof of Lemma 2.1 implies that {(u m , v m )} is bounded in E. Since
$\underset{m\to \infty }{\text{lim}}\underset{\Omega }{\int }\left({\lambda }_{m}{\left|{u}_{m}\right|}^{q}+{\delta }_{m}{\left|{v}_{m}\right|}^{q}\right)dx=0.$
(2.18)
Let t m > 0 be such that ${t}_{m}\left({u}_{m},{v}_{m}\right)\in {\mathcal{N}}_{0,0}$. Since $\left({u}_{m},{v}_{m}\right)\in {\mathcal{N}}_{{\lambda }_{m},{\delta }_{m}}$, we have that
$\begin{array}{ll}\hfill {c}_{0,0}& \le {I}_{0,0}\left({t}_{m}\left({u}_{m},{v}_{m}\right)\right)\phantom{\rule{2em}{0ex}}\\ ={I}_{{\lambda }_{m},{\delta }_{m}}\left({t}_{m}\left({u}_{m},{v}_{m}\right)\right)+\frac{{t}_{m}^{q}}{q}\underset{\Omega }{\int }\left({\lambda }_{m}{\left|{u}_{m}\right|}^{q}+{\delta }_{m}{\left|{v}_{m}\right|}^{q}\right)dx\phantom{\rule{2em}{0ex}}\\ \le {I}_{{\lambda }_{m},{\delta }_{m}}\left({u}_{m},{v}_{m}\right)+\frac{{t}_{m}^{q}}{q}\underset{\Omega }{\int }\left({\lambda }_{m}{\left|{u}_{m}\right|}^{q}+{\delta }_{m}{\left|{v}_{m}\right|}^{q}\right)dx\phantom{\rule{2em}{0ex}}\\ ={c}_{{\lambda }_{m},{\delta }_{m}}+\frac{{t}_{m}^{q}}{q}\underset{\Omega }{\int }\left({\lambda }_{m}{\left|{u}_{m}\right|}^{q}+{\delta }_{m}{\left|{v}_{m}\right|}^{q}\right)dx.\phantom{\rule{2em}{0ex}}\end{array}$
If {t m } is bounded, we can use the above estimate and (2.18) to get
${c}_{0,0}\le \underset{m\to \infty }{\text{lim}\text{inf}}{c}_{{\lambda }_{m},{\delta }_{m}}.$
This and (2.17) we get
${c}_{0,0}\le \underset{m\to \infty }{\text{lim}\text{inf}}{c}_{{\lambda }_{m},{\delta }_{m}}\le \underset{m\to \infty }{\text{lim}\text{sup}}{c}_{{\lambda }_{m},{\delta }_{m}}\le {c}_{0,0},$

that is ${c}_{0,0}=\underset{m\to \infty }{\text{lim}}{c}_{{\lambda }_{m},{\delta }_{m}}$.

It remains to check that {t m } is bounded. A straightforward calculation shows that
${t}_{m}={\left(\frac{{∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}}{{\int }_{\Omega }F\left(x,{u}_{m},{v}_{m}\right)dx}\right)}^{\frac{1}{{p}^{*}-p}}.$
(2.19)
Since $\left({u}_{m},{v}_{m}\right)\in {\mathcal{N}}_{{\lambda }_{m},{\delta }_{m}}$, we obtain
${∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}=\underset{\Omega }{\int }F\left(x,{u}_{m},{v}_{m}\right)dx+\underset{\Omega }{\int }\left({\lambda }_{m}{\left|{u}_{m}\right|}^{q}+{\delta }_{m}{\left|{v}_{m}\right|}^{q}\right)dx\le {S}_{F}^{-\frac{{p}^{*}}{p}}{∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{{p}^{*}}+{o}_{m}\left(1\right).$

Hence ${∥\left({u}_{m,}{v}_{m}\right)∥}_{E}^{p}\ge {C}_{4}>0$, and therefore from the above expression it follows that ∫ΩF(x, u m , v m )dxC5 > 0. Thus, the boundedness of {(u m , v m )} and (2.19) imply that {t m } is bounded. This completes the proof.

## 3 Some technical lemmas

In this section, we denote by $ℳ\left(\Omega \right)$ the Banach space of finite Radon measures over Ω equipped with the norm $‖\sigma ‖={\mathrm{sup}}_{\underset{{|\phi |}_{\infty }=1}{\phi \in {C}_{0}\left(\Omega \right)}}|\sigma \left(\phi \right)|$. A sequence $\left\{{\sigma }_{m}\right\}\subset ℳ\left(\Omega \right)$ is said to converge weakly to $\sigma \in ℳ\left(\Omega \right)$ provided σ m (φ) → σ(φ) for all φ C0(Ω). By [18, Theorem 1.39], every bounded sequence $\left\{{\sigma }_{m}\right\}\subset ℳ\left(\Omega \right)$ contains a weakly convergent subsequence.

The next lemma is a version of the second concentration-compactness lemma of Lions [21]. It is also inspired by [18, Lemma 1.40] and [[22], Lemma 2.4].

Lemma 3.1. Suppose that the sequence {(u m ,v m )} D1,p( N ) × D1,p( N ) satisfies
$\begin{array}{l}\left({u}_{m},{v}_{m}\right)⇀\left(u,v\right)\phantom{\rule{2.77695pt}{0ex}}in\phantom{\rule{2.77695pt}{0ex}}{D}^{1,p}\left({ℝ}^{N}\right)×{D}^{1,p}\left({ℝ}^{N}\right),\phantom{\rule{2em}{0ex}}\\ \left({u}_{m},{v}_{m}\right)\to \left(u,v\right)\phantom{\rule{2.77695pt}{0ex}}a.e.\phantom{\rule{2.77695pt}{0ex}}x\in {ℝ}^{N},\phantom{\rule{2em}{0ex}}\\ \left(\nabla {u}_{m},\nabla {v}_{m}\right)\to \left(\nabla u,\nabla v\right)\phantom{\rule{2.77695pt}{0ex}}a.e.\phantom{\rule{2.77695pt}{0ex}}x\in {ℝ}^{N},\phantom{\rule{2em}{0ex}}\\ {\left|\nabla \left({u}_{m}-u\right)\right|}^{p}⇀\mu ,\phantom{\rule{1em}{0ex}}{\left|\nabla \left({v}_{m}-v\right)\right|}^{p}⇀\sigma \phantom{\rule{2.77695pt}{0ex}}in\phantom{\rule{2.77695pt}{0ex}}ℳ\left({ℝ}^{N}\right),\phantom{\rule{2em}{0ex}}\\ F\left(x,{u}_{m}-u,{v}_{m}-v\right)⇀\nu \phantom{\rule{2.77695pt}{0ex}}in\phantom{\rule{2.77695pt}{0ex}}ℳ\left({ℝ}^{N}\right),\phantom{\rule{2em}{0ex}}\end{array}$
and define
$\left\{\begin{array}{c}{\mu }_{\infty }=\underset{R\to \infty }{\text{lim}}\underset{m\to \infty }{\text{lim}\text{sup}}\underset{\left|x\right|>R}{\int }{\left|\nabla {u}_{m}\right|}^{p}dx,\hfill \\ {\sigma }_{\infty }=\underset{R\to \infty }{\text{lim}}\underset{m\to \infty }{\text{lim}\text{sup}}\underset{\left|x\right|>R}{\int }{\left|\nabla {v}_{m}\right|}^{p}dx,\hfill \\ {\nu }_{\infty }=\underset{R\to \infty }{\text{lim}}\underset{m\to \infty }{\text{lim}\text{sup}}\underset{\left|x\right|>R}{\int }F\left(x,{u}_{m},{v}_{m}\right)dx,\hfill \end{array}\right\$
(3.1)
then it follows that
$\underset{m\to \infty }{\text{lim}\text{sup}}\underset{{ℝ}^{N}}{\int }{\left|\nabla {u}_{m}\right|}^{p}dx=∥\mu ∥+{\mu }_{\infty }+\underset{{ℝ}^{N}}{\int }{\left|\nabla u\right|}^{p}dx,$
(3.2)
$\underset{m\to \infty }{\text{lim}\text{sup}}\underset{{ℝ}^{N}}{\int }{\left|\nabla {v}_{m}\right|}^{p}dx=∥\sigma ∥+{\sigma }_{\infty }+\underset{{ℝ}^{N}}{\int }{\left|\nabla u\right|}^{p}dx,$
(3.3)
$\underset{m\to \infty }{\text{lim}\text{sup}}\underset{{ℝ}^{N}}{\int }F\left(x,{u}_{m},{v}_{m}\right)dx=∥\nu ∥+{\nu }_{\infty }+\underset{{ℝ}^{N}}{\int }F\left(x,u,v\right)dx,$
(3.4)
${∥\nu ∥}^{\frac{p}{{p}^{*}}}\le {S}_{F}^{-1}\left(∥\mu ∥+∥\sigma ∥\right),\phantom{\rule{1em}{0ex}}{\nu }_{\infty }^{\frac{p}{{p}^{*}}}\le {S}_{F}^{-1}\left({\mu }_{\infty }+{\sigma }_{\infty }\right).$
(3.5)

Moreover, if (u,v) = (0,0) and${∥\nu ∥}^{\frac{p}{{p}^{*}}}={S}_{F}^{-1}\left(∥\mu ∥+∥\sigma ∥\right)$, then the measures μ,ν, and σ are concentrated at a single point, respectively.

Proof. We first recall that, in view of the definition of S F , for each nonnegative function $\phi \in {C}_{0}^{\infty }\left({ℝ}^{N}\right)$ we have
${\left(\underset{{ℝ}^{N}}{\int }{\left|\phi \right|}^{{p}^{*}}F\left(x,{u}_{m},{v}_{m}\right)dx\right)}^{\frac{p}{{p}^{*}}}={\left(\underset{{ℝ}^{N}}{\int }F\left(x,\phi {u}_{m},\phi {v}_{m}\right)dx\right)}^{\frac{p}{{p}^{*}}}\le {S}_{F}^{-1}{∥\left(\phi {u}_{m},\phi {v}_{m}\right)∥}_{E}^{p}.$
Moreover, arguing as [8, Lemma 5], we have that
$\underset{{ℝ}^{N}}{\int }\phi F\left(x,{u}_{m}-u,{v}_{m}-v\right)dx=\underset{{ℝ}^{N}}{\int }\phi F\left(x,{u}_{m},{v}_{m}\right)dx-\underset{{ℝ}^{N}}{\int }\phi F\left(x,u,v\right)dx+{o}_{m}\left(1\right).$

Since F is p*-homogeneous, we can use the two above expressions and argue along the same line of the proof of Lemma 1.40 in [18] to conclude that (3.2)-(3.5) hold. If (u, v) = (0,0) and ${∥\nu ∥}^{\frac{p}{{p}^{*}}}={S}_{F}^{-1}\left(∥\mu ∥+∥\sigma ∥\right)$, the same argument of step 3 of the proof of Lemma 1.40 in [18] implies that the measures μ, ν and σ are concentrated at a single point, respectively.

Remark 3.1. We notice that the last conclusion of the above result holds even if (u, v) (0,0). Indeed, in this case we can define$\left({ũ}_{m},{ṽ}_{m}\right)=\left({u}_{m}-u,{v}_{m}-v\right)$and notice that
$\begin{array}{c}\left({ũ}_{m},{ṽ}_{m}\right)⇀\left(ũ,ṽ\right)\phantom{\rule{2.77695pt}{0ex}}in\phantom{\rule{2.77695pt}{0ex}}{D}^{1,p}\left({ℝ}^{N}\right)×{D}^{1,p}\left({ℝ}^{N}\right),\\ \left({ũ}_{m},{ṽ}_{m}\right)\to \left(0,0\right)\phantom{\rule{2.77695pt}{0ex}}a.e.\phantom{\rule{2.77695pt}{0ex}}x\in {ℝ}^{N},\\ \left(\nabla {ũ}_{m},\nabla {ṽ}_{m}\right)\to \left(\nabla ũ,\nabla ṽ\right)\phantom{\rule{2.77695pt}{0ex}}a.e.\phantom{\rule{2.77695pt}{0ex}}x\in {ℝ}^{N},\\ {\left|\nabla \left({ũ}_{m}-ũ\right)\right|}^{p}⇀\stackrel{̃}{\mu },\phantom{\rule{1em}{0ex}}{\left|\nabla \left({ṽ}_{m}-ṽ\right)\right|}^{p}⇀\stackrel{̃}{\sigma }\phantom{\rule{2.77695pt}{0ex}}in\phantom{\rule{2.77695pt}{0ex}}ℳ\left({ℝ}^{N}\right),\\ F\left(x,{ũ}_{m}-ũ,{ṽ}_{m}-ṽ\right)⇀\stackrel{̃}{\nu }\phantom{\rule{2.77695pt}{0ex}}in\phantom{\rule{2.77695pt}{0ex}}ℳ\left({ℝ}^{N}\right),\end{array}$

Since$\left({ũ}_{m}-ũ,{ṽ}_{m}-ṽ\right)=\left({u}_{m}-u,{v}_{m}-v\right)$and therefore$\stackrel{̃}{\mu }=\mu ,\stackrel{̃}{\sigma }=\sigma$, and$ṽ=v$, where μ,σ, and ν are the same as those in Lemma 3.1. Thus, if${∥\nu ∥}^{\frac{p}{{p}^{*}}}={S}_{F}^{-1}\left(∥\mu ∥+∥\sigma ∥\right)$we also have that${∥\stackrel{̃}{\nu }∥}^{\frac{p}{{p}^{*}}}={S}_{F}^{-1}\left(∥\stackrel{̃}{\mu }∥+∥\stackrel{̃}{\sigma }∥\right)$and the result follows from the last part of Lemma 3.1.

Now, we introduce the following Lemma.

Lemma 3.2. Suppose {(u m , v m )} E such thatΩF(x, u m , v m )dx = 1 and$\underset{m\to \infty }{\text{lim}}{∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}={S}_{F}$. Then there exist {r m } (0, +∞) and {y m } N such that
$\left({\omega }_{m}^{1}\left(x\right),{\omega }_{m}^{2}\left(x\right)\right)={r}_{m}^{\frac{N-p}{p}}\left({u}_{m}\left({r}_{m}x+{y}_{m}\right),{v}_{m}\left({r}_{m}x+{y}_{m}\right)\right)$
(3.6)

contains a convergent subsequence denoted again by$\left\{\left({\omega }_{m}^{1}\left(x\right),{\omega }_{m}^{2}\left(x\right)\right)\right\}$such that$\left({\omega }_{m}^{1}\left(x\right),{\omega }_{m}^{2}\left(x\right)\right)\to \left({\omega }_{1},{\omega }_{2}\right)$in D1,p( N ) × D1,p( N ). Moreover, as m → ∞, we have r m → 0 and${y}_{m}\to y\in \stackrel{̄}{\Omega }$.

Proof. For each r > 0, we consider the Lévy concentration functions
${H}_{m}\left(r\right)=\underset{y\in {ℝ}^{N}}{\text{sup}}\underset{{B}_{r}\left(y\right)}{\int }F\left(x,{u}_{m},{v}_{m}\right)dx.$
Since for every m,
$\underset{r\to {0}^{+}}{\text{lim}}{H}_{m}\left(r\right)=0,\phantom{\rule{1em}{0ex}}\underset{r\to \infty }{\text{lim}}{H}_{m}\left(r\right)=1,$
there exist r m > 0 and a sequence $\left\{{y}_{m}^{k}\right\}\subset {ℝ}^{N}$ satisfying
$\frac{1}{2}={H}_{m}\left({r}_{m}\right)=\underset{k\to \infty }{\text{lim}}\underset{{B}_{{r}_{m}}\left({y}_{m}^{k}\right)}{\int }F\left(x,{u}_{m},{v}_{m}\right)dx.$
Recalling that $\underset{\left|y\right|\to \infty }{\text{lim}}{\int }_{{B}_{{r}_{m}}\left(y\right)}F\left(x,{u}_{m},{v}_{m}\right)dx=0$, we conclude that $\left\{{y}_{m}^{k}\right\}$ is bounded. Hence, up to a subsequence, $\underset{k\to \infty }{\text{lim}}{y}_{m}^{k}={y}_{m}\in {ℝ}^{N}$ and we obtain
$\frac{1}{2}=\underset{{B}_{{r}_{m}}\left({y}_{m}\right)}{\int }F\left(x,{u}_{m},{v}_{m}\right)dx.$
We shall prove that the above sequences {r m } and {y m } satisfy the statements of the lemma. First notice that
$\frac{1}{2}=\underset{{B}_{{r}_{m}}\left({y}_{m}\right)}{\int }F\left(x,{u}_{m},{v}_{m}\right)dx=\underset{{B}_{1}\left(0\right)}{\int }F\left(x,{\omega }_{m}^{1},{\omega }_{m}^{2}\right)dx=\underset{y\in {ℝ}^{N}}{\text{sup}}\underset{{B}_{1}\left(y\right)}{\int }F\left(x,{\omega }_{m}^{1},{\omega }_{m}^{2}\right)dx.$
(3.7)
By (3.6), a straightforward calculation provides
$\underset{m\to \infty }{\text{lim}}{∥\left({\omega }_{m}^{1},{\omega }_{m}^{2}\right)∥}_{E}^{p}=\underset{m\to \infty }{\text{lim}}{∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}={S}_{F},\underset{{ℝ}^{N}}{\int }F\left(x,{\omega }_{m}^{1},{\omega }_{m}^{2}\right)dx=1.$
Hence, we can apply Lemma 3.1 to obtain (ω1,ω2) D1,p( N ) × D1,p( N ) satisfying
${S}_{F}=∥\mu ∥+{\mu }_{\infty }+∥\sigma ∥+{\sigma }_{\infty }+{∥\left({\omega }_{1},{\omega }_{2}\right)∥}_{E}^{p},\phantom{\rule{1em}{0ex}}1=∥\nu ∥+{\nu }_{\infty }+\underset{{ℝ}^{N}}{\int }F\left(x,{\omega }_{1},{\omega }_{2}\right)dx,$
(3.8)
${∥\nu ∥}^{\frac{p}{{p}^{*}}}\le {S}_{F}^{-1}\left(∥\mu ∥+∥\sigma ∥\right),\phantom{\rule{1em}{0ex}}{\nu }_{\infty }^{\frac{p}{{p}^{*}}}\le {S}_{F}^{-1}\left({\mu }_{\infty }+{\sigma }_{\infty }\right).$
(3.9)
The second equality in (3.8) implies that $∥\nu ∥,{\nu }_{\infty },{\int }_{{ℝ}^{N}}F\left(x,{\omega }_{1},{\omega }_{2}\right)dx\in \left[0,1\right]$. If one of these values belongs to the open interval (0,1), we can use (3.8), $\frac{p}{{p}^{*}}<1,\left({{\int }_{{ℝ}^{N}}F\left(x,{\omega }_{1},{\omega }_{2}\right)dx\right)}^{\frac{p}{{p}^{*}}}\le {S}_{F}^{-1}{‖\left({\omega }_{1},{\omega }_{2}\right)‖}_{E}^{p}$ and (3.9) to get
$\begin{array}{ll}\hfill {S}_{F}& ={S}_{F}\left(∥\nu ∥+{\nu }_{\infty }+\underset{{ℝ}^{N}}{\int }F\left(x,{\omega }_{1},{\omega }_{2}\right)dx\right)\phantom{\rule{2em}{0ex}}\\ <{S}_{F}\left({∥\nu ∥}^{\frac{p}{{p}^{*}}}+{\nu }_{\infty }^{\frac{p}{{p}^{*}}}+{\left(\underset{{ℝ}^{N}}{\int }F\left(x,{\omega }_{1},{\omega }_{2}\right)dx\right)}^{\frac{p}{{p}^{*}}}\right)\phantom{\rule{2em}{0ex}}\\ \le {S}_{F},\phantom{\rule{2em}{0ex}}\end{array}$

which is a contradiction. Thus $∥\nu ∥,{\nu }_{\infty },{\int }_{{ℝ}^{N}}F\left(x,{\omega }_{1},{\omega }_{2}\right)dx\in \left\{0,1\right\}$. Actually, it follows from (3.7) that ${\int }_{\left|x\right|>R}F\left(x,{\omega }_{m}^{1},{\omega }_{m}^{2}\right)dx\le \frac{1}{2}$ for any R > 1. Thus, we conclude that ν = 0.

Let us prove that ||ν|| = 0. Arguing by contradiction, then ||ν|| = 1. It follows from the first equality in (3.8) that S F ≥ ||μ|| + ||σ||. On the other hand, the first inequality in (3.9) provides ||μ|| + ||σ|| ≥ S F . Hence, we conclude that ||μ|| + ||σ|| = S F . Since we suppose that ||ν|| = 1 we obtain ${∥\nu ∥}^{\frac{p}{{p}^{*}}}={S}_{F}^{-1}\left(∥\mu ∥+∥\sigma ∥\right)$. It follows from Remark 3.1 that $\nu ={\delta }_{{x}_{0}}$ for some x0 N . Thus, from (3.7), we get
$\frac{1}{2}\ge \underset{m\to \infty }{\text{lim}}\underset{{B}_{1}\left({x}_{0}\right)}{\int }F\left(x,{\omega }_{m}^{1},{\omega }_{m}^{2}\right)dx=\underset{{B}_{1}\left({x}_{0}\right)}{\int }d\nu =∥\nu ∥=1.$

This contradiction proves that ν = 0.

Since ν = ν = 0, we have that ${\int }_{{ℝ}^{N}}F\left(x,{\omega }_{1},{\omega }_{2}\right)dx=1$. This and (3.8) provide
$\underset{m\to \infty }{\text{lim}}{∥\left({\omega }_{m}^{1},{\omega }_{m}^{2}\right)∥}_{E}^{p}={S}_{F}\ge {∥\left({\omega }_{1},{\omega }_{2}\right)∥}_{E}^{p}\ge {S}_{F}{\left(\underset{{ℝ}^{N}}{\int }F\left(x,{\omega }_{1},{\omega }_{2}\right)dx\right)}^{\frac{p}{{p}^{*}}}={S}_{F}.$
So, ${∥\left({\omega }_{1},{\omega }_{2}\right)∥}_{E}^{p}={S}_{F}$ and therefore $\left({\omega }_{m}^{1},{\omega }_{m}^{2}\right)\to \left({\omega }_{1},{\omega }_{2}\right)\not\equiv \left(0,0\right)$ strongly in D1,p( N ) × D1,p( N ) and $\left({\omega }_{m}^{1}\left(x\right),{\omega }_{m}^{2}\left(x\right)\right)\to \left({\omega }_{1}\left(x\right),{\omega }_{2}\left(x\right)\right)$ for a.e. x N . In order to conclude the proof we notice that
${∥\left({\omega }_{m}^{1},{\omega }_{m}^{2}\right)∥}_{{L}^{p}\left({ℝ}^{N}\right)×{L}^{p}\left({ℝ}^{N}\right)}=\frac{1}{{r}_{m}^{p}}{∥\left({u}_{m},{v}_{m}\right)∥}_{{L}^{p}\left(\Omega \right)×{L}^{p}\left(\Omega \right).}$

Since {(u m , v m )} is bounded and (ω1, ω2) (0,0), we infer from the above equality that, up to a subsequence, r m r0 ≥ 0. If |y m | → ∞, for each fixed x N , we have that there exists m x N such that r m x + y m Ω for mm x . For such values of m we have that $\left({\omega }_{m}^{1}\left(x\right),{\omega }_{m}^{2}\left(x\right)\right)=\left(0,0\right)$. Taking the limit and recalling that x is arbitrary, we conclude that (ω1, ω2) = (0,0), which is a contradiction. So, along a subsequence, y m y N .

We claim that r0 = 0. Indeed, suppose by contradiction that r0 > 0. Then, as m becomes large, the set Ω m = (Ω-y m )/r m approaches Ω0 = (Ω -y)/r0 N . This implies that ω1,ω2 has compact support in N . On the other hand, since (ω1,ω2) achieves the infimum in (2.3) and F is homogeneous, we can use the Lagrange Multiplier Theorem to conclude that ( ω1, ω2) satisfies
$-{\Delta }_{p}u=\theta \frac{\partial F\left(x,u,v\right)}{\partial u},-{\Delta }_{p}v=\theta \frac{\partial F\left(x,u,v\right)}{\partial u},x\in {ℝ}^{N},$

for $\theta =\frac{{S}_{F}}{{p}^{*}}>0$. It follows from (F2) and the maximum principle that at least one of the functions ω1,ω2 is positive in N . But this contradicts supp (ω1,ω2) Ω0. Hence, we conclude that r0 = 0. Finally, if $y\notin \stackrel{̄}{\Omega }$ we obtain r m x + y m Ω for large values of m, and therefore we should have (ω1, ω2) ≡ (0, 0) again. Thus, $y\in \stackrel{̄}{\Omega }$ and the proof is completed.

Up to translations, we may assume that 0 Ω, since Ω is a smooth bounded domain of N , we can choose r > 0 small enough such that B r = B r (0) = {x N : d(x, 0) < r} Ω and the sets
${\Omega }_{r}^{+}=\left\{x\in {ℝ}^{N}:\mathsf{\text{dist}}\left(x,\Omega \right)r\right\},$
are homotopically equivalent to Ω. Let
${W}_{0,\mathsf{\text{rad}}}^{1,p}\left({B}_{r}\right)=\left\{u\in {W}_{0}^{1,p}\left({B}_{r}\right):u\phantom{\rule{2.77695pt}{0ex}}\mathsf{\text{is}}\phantom{\rule{2.77695pt}{0ex}}\mathsf{\text{radial}}\right\}$
and
${E}_{\mathsf{\text{rad}}}\left({B}_{r}\right)={W}_{0,\mathsf{\text{rad}}}^{1,p}\left({B}_{r}\right)×{W}_{0,\mathsf{\text{rad}}}^{1,p}\left({B}_{r}\right).$
We define the functional
${I}_{{B}_{r}}\left(u,v\right)=\frac{1}{p}\underset{{B}_{r}}{\int }\left({\left|\nabla u\right|}^{p}+{\left|\nabla v\right|}^{p}\right)dx-\frac{1}{{p}^{*}}\underset{{B}_{r}}{\int }F\left(x,u,v\right)dx-\frac{1}{q}\underset{{B}_{r}}{\int }\left(\lambda {\left|u\right|}^{q}+\delta {\left|v\right|}^{q}\right)dx,\phantom{\rule{2.77695pt}{0ex}}\left(u,v\right)\in {E}_{\mathsf{\text{rad}}}\left({B}_{r}\right),$
and set
${m}_{\lambda ,\delta }=\underset{\left(u,v\right)\in {\mathcal{N}}_{\lambda ,\delta }^{{B}_{r}}}{\text{inf}}{I}_{{B}_{r}}\left(u,v\right),$
where
${\mathcal{N}}_{\lambda ,\delta }^{{B}_{r}}:=\left\{\left(u,v\right)\in {E}_{\mathsf{\text{rad}}}\left({B}_{r}\right)\\left\{\left(0,0\right)\right\}:⟨{I}_{{B}_{r}}^{\prime }\left(u,v\right),\left(u,v\right)⟩=0\right\}.$

Clearly, m λ , δ is nonincreasing in λ, δ. Note that m λ , δ > 0 for all λ, δ > 0.

Arguing as in the proof of Lemma 2.3 and Theorem 1.1, we obtain the following result.

Lemma 3.3. Suppose F satisfies (F0)-(F2), then the infimum mλ,δis attained by a nonneg-ative radial function (uλ,δ, vλ,δ) Eradwhenever 2 ≤ p < q < p* and λ,δ > 0, or q = p and λ,δ (0,Λ1,rad), where Λ1,rad > 0 is the first eigenvalue of the operator$\left(-{\Delta }_{p},{W}_{0,\mathsf{\text{rad}}}^{1,p}\left({B}_{r}\right)\right)$. Moreover,
${m}_{\lambda ,\delta }<\frac{1}{N}{S}_{F}^{\frac{N}{p}},\underset{\lambda ,\delta \to {0}^{+}}{\text{lim}}{m}_{\lambda ,\delta }=\frac{1}{N}{S}_{F}^{\frac{N}{p}}.$
We introduce the barycenter map $\beta :{\mathcal{N}}_{\lambda ,\delta }\to {ℝ}^{N}$ as follows
$\beta \left(u,v\right)={S}_{F}^{-\frac{N}{p}}\underset{\Omega }{\int }F\left(x,u,v\right)xdx.$

This map has the following property.

Lemma 3.4. If Np2,2 ≤ pq < p* and F satisfies (F0)-(F2), then there exists λ* > 0 such that$\beta \left(u,v\right)\in {\Omega }_{r}^{+}$whenever$\left(u,v\right)\in {\mathcal{N}}_{\lambda ,\delta },\lambda ,\delta \in \left(0,{\lambda }^{*}\right)$and Iλ,δ(u, v) ≤ mλ,δ.

Proof. By way of contradiction, we suppose that there exist {λ m }, {δ m } + and $\left\{\left({u}_{m},{v}_{m}\right)\right\}\subset {\mathcal{N}}_{{\lambda }_{m},{\delta }_{m}}$ such that λ m , δ m → 0+ as $m\to \infty ,{I}_{{\lambda }_{m},{\delta }_{m}}\left({u}_{m},{v}_{m}\right)\le {m}_{{\lambda }_{m},{\delta }_{m}}$ but $\beta \left({u}_{m},{v}_{m}\right)\notin {\Omega }_{r}^{+}$.

From $\left\{\left({u}_{m},{v}_{m}\right)\right\}\subset {\mathcal{N}}_{{\lambda }_{m},{\delta }_{m}}$ and ${I}_{{\lambda }_{m},{\delta }_{m}}\left({u}_{m},{v}_{m}\right)\le {m}_{{\lambda }_{m},{\delta }_{m}}$ we have that {(u m , v m )} is bounded in E. Moreover,
$0=〈{I}_{{\lambda }_{m},}^{\text{'}}{}_{{\delta }_{m}}\left({u}_{m},{v}_{m}\right),\left({u}_{m},{v}_{m}\right)〉={‖\left({u}_{m},{v}_{m}\right)‖}_{E}^{p}-\underset{\Omega }{\int }F\left(x,{u}_{m},{v}_{m}\right)dx-\underset{\Omega }{\int }\left({\lambda }_{m}{|{u}_{m}|}^{q}+{\delta }_{m}{|{v}_{m}|}^{q}\right)dx.$
Since λ m , δ m → 0+, we can use the boundedness of {(u m , v m )} to get
$0\le \underset{\Omega }{\int }\left({\lambda }_{m}{\left|{u}_{m}\right|}^{q}+{\delta }_{m}{\left|{v}_{m}\right|}^{q}\right)dx\to 0,$
from which it follows that
$\underset{m\to \infty }{\text{lim}}{∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}=\underset{m\to \infty }{\text{lim}}\underset{\Omega }{\int }F\left(x,{u}_{m},{v}_{m}\right)dx=k\ge 0.$
Notice that
$\begin{array}{ll}\hfill {c}_{{\lambda }_{m},{\delta }_{m}}& \le {I}_{{\lambda }_{m},{\delta }_{m}}\left({u}_{m},{v}_{m}\right)\phantom{\rule{2em}{0ex}}\\ =\frac{1}{p}{∥\left({u}_{m},{v}_{m}\right)∥}_{E}^{p}-\frac{1}{{p}^{*}}\underset{\Omega }{\int }F\left(x,{u}_{m},{v}_{m}\right)dx-\frac{1}{q}\underset{\Omega }{\int }\left({\lambda }_{m}{\left|{u}_{m}\right|}^{q}+{}_{}\end{array}$