Multiplicity of solutions for a quasilinear elliptic equation with $(p,q)$-Laplacian and critical exponent on $\mathbb{R}^{N}$

Abstract

The multiplicity of solutions for a $(p,q)$-Laplacian equation involving critical exponent

$$ -\Delta_{p}u-\Delta_{q}u=\lambda V(x) \vert u \vert ^{k-2}u+K(x) \vert u \vert ^{p^{\ast }-2}u,\quad x\in \mathbb{R}^{N} $$

is considered. By variational methods and the concentration–compactness principle, we prove that the problem possesses infinitely many weak solutions with negative energy for $\lambda\in(0,\lambda^{\ast})$. Moreover, the existence of infinitely many solutions with positive energy is also given for all $\lambda>0$ under suitable conditions.

1 Introduction

In this paper, we consider multiple nontrivial weak solutions to the following nonlinear elliptic problem of $(p,q)$-Laplacian type involving critical Sobolev exponent:

$$ -\Delta_{p}u-\Delta_{q}u=\lambda V(x) \vert u \vert ^{k-2}u+K(x) \vert u \vert ^{p^{\ast }-2}u,\quad x\in \mathbb{R}^{N}, $$

(1.1)

where $\Delta_{m}u=\operatorname{div}(|\nabla|^{m-2}\nabla u)$ is the m-Laplacian of u, $\lambda>0$, $1< k< q< p< N$ and $p^{\ast }=\frac{Np}{N-p}$.

The $(p,q)$-Laplacian problem (1.1) comes from a general reaction–diffusion system

$$\begin{aligned} u_{t}=\operatorname{div}\bigl[E(u)\nabla u\bigr]+c(x,u), \end{aligned}$$

(1.2)

where $E(u)=(|\nabla u|^{p-2}+|\nabla u|^{q-2})$. The system has a wide range of applications in physics and related sciences, such as biophysics, chemical reaction and plasma physics. In such applications, the function u describes a concentration, the first term on the right-hand side of (1.2) corresponds to the diffusion with a diffusion coefficient $E(u)$; whereas the second one is the reaction and relates to sources and loss processes. Typically, in chemical and biological applications, the reaction term $c(x,u)$ has a polynomial form with respect to the concentration u. Specially, taking $q=2$, we note that $(p, 2)$-equations arise in many physical applications (see [2] and [5]), and recently such equations were studied by Papageorgiou et al. [10–13]. For example, in [11], they studied the existence and multiplicity of the following parametric nonlinear nonhomogeneous Dirichlet problem:

$$-\Delta_{p}u(z)-\Delta(z)=\lambda \bigl\vert u(z) \bigr\vert ^{p-2}u(z)+f\bigl(z,u(z)\bigr)\quad \mbox{in } \Omega, u|\partial \Omega=0, 2< p< \infty, $$

where $\Omega\subset\mathbb{R}^{N}$ and the parameter $\lambda>0$ is near the principal eigenvalue $\lambda_{1}(p)>0$ of $(-\Delta _{p},W_{0}^{1,p}(\Omega))$.

For general $q\in(1,p)$ and concave–convex nonlinearities, the stationary solution of (1.2) was studied by many authors and fruitful multiplicity results were obtained for the following problem:

$$\begin{aligned} -\operatorname{div}\bigl[E(u)\nabla u\bigr]=c(x,u). \end{aligned}$$

(1.3)

For example, in [7], G. Li and G. Zhang considered problem (1.3) with the critical exponent

$$\begin{aligned} c(x,u)= \vert u \vert ^{p^{\ast}-2}u+\theta \vert u \vert ^{r-2}u \end{aligned}$$

(1.4)

by using Lusternik–Schnirelman’s theory. They proved that when $\theta >0$, $1< r< q< p< N$ and $\Omega\subset\mathbb{R}^{N}$ is bounded, there is a $\theta_{0}>0$ such that problem (1.3) possesses infinitely many weak solutions in $W^{1,p}_{0}(\Omega)$ for any $\theta\in(0,\theta_{0})$.

Moreover, H. Yin and Z. Yang in [17] studied the equation

$$\begin{aligned} -\Delta_{p}u-\mu\Delta_{q}u=\theta V(x) \vert u \vert ^{r-2}u+ \vert u \vert ^{p^{\ast }-2}u+\lambda f(x,u) \end{aligned}$$

(1.5)

for the multiplicity of solutions on a bounded domain $\Omega\subset \mathbb{R}^{N}$ with $1< r< q< p$ and $\lambda\in(0,\lambda^{\ast})$.

But they only considered infinitely many weak solutions on a bounded domain Ω. Different from [7] and [17], our work is developed in the whole space $\mathbb{R}^{N}$ and the existence of infinitely many solutions with positive energy for problem (1.1) is also discussed, which are not mentioned in the references.

Our main results can be described as follows.

Theorem 1.1

Suppose $1< k< q< p< N$, $N\geq3$, $K(x)\in C(\mathbb{R}^{N})\cap L^{\infty}(\mathbb{R}^{N})$ and $0\leq V(x)\in C(\mathbb{R}^{N})\cap L^{r}(\mathbb{R}^{N})$ with $r=\frac {p^{\ast}}{p^{\ast}-k}$. Moreover, $V(x)>0$ is bounded on some open subset $\Omega\subset\mathbb{R}^{N}$, with $|\Omega|>0$. Then there exists a $\lambda^{\ast}>0$ such that, for all $\lambda\in (0,\lambda^{\ast})$, problem (1.1) has a sequence of weak solutions with negative energy.

Denote the group of orthogonal linear transformations in $\mathbb {R}^{N}$ by $O(N)$ and let $T\subset O(N)$ be a subgroup. Set $|T|:=\inf_{x\in\mathbb{R}^{N},x\neq0}|T_{x}|$, where $T_{x}:=\{\tau x:\tau\in O(N)\}$ for $x\neq0$ (see [16]). Moreover, a function $f: \mathbb{R}^{N}\to\mathbb{R}$ is called T-invariant if $f(\tau x)=f(x)$ for all $\tau\in T$ and $x\in\mathbb{R}^{N}$.

Theorem 1.2

Suppose $1< k< q< p< N$, $N\geq3$, and assume $V(x)$ and $K(x)$ are T-invariant. Moreover, let $|T|=\infty$, $K(0)=0$, $\lim_{|x|\to\infty}K(x)=0$, $K(x)\in C(\mathbb {R}^{N})\cap L^{\infty}(\mathbb{R}^{N})$, $K(x)>0$ a.e. in $\mathbb{R}^{N}$and $0\leq V(x)\in L^{r}(\mathbb{R}^{N})\cap L^{r'}(\mathbb{R}^{N})$ with $r=\frac{p^{\ast}}{p^{\ast}-k}$ and $r'=\frac{q^{\ast}}{q^{\ast}-k}$. Then, for all $\lambda>0$, problem (1.1) possesses infinitely many solutions with positive energy.

This paper is organized as follows. In Sect. 2, for the reader’s convenience, we describe the main mathematical tools which we shall use. The existence theorem for $\lambda\in(0,\lambda^{\ast})$ is proved in Sect. 3 via the application of genus. In Sect. 4, under suitable conditions, we show that problem (1.1) possesses infinitely many solutions with positive energy for every $\lambda>0$.

2 Preliminary results

We now recall some known results and state our basic assumptions.

In this paper $\Vert\cdot\Vert_{p}$ denotes the usual $L^{p}$ norm and

$$D^{1,p}\bigl(\mathbb{R}^{N}\bigr):=\bigl\{ \nabla u\in L^{p}\bigl(\mathbb{R}^{N}\bigr)\bigr\} , $$

with the norm defined by

$$\Vert u \Vert _{D^{1,p}}= \biggl( \int_{\mathbb{R}^{N}} \vert \nabla u \vert ^{p}\,dx \biggr)^{1/p}. $$

We deal with problem (1.1) in the reflexive Banach space [3]

$$X:=D^{1,p}\bigl(\mathbb{R}^{N}\bigr)\cap D^{1,q} \bigl(\mathbb{R}^{N}\bigr), $$

which is endowed with the norm

$$\Vert u \Vert _{X}= \Vert u \Vert _{D^{1,p}}+ \Vert u \Vert _{D^{1,q}}. $$

Throughout this paper the function $K(x)\in C(\mathbb{R}^{N})\cap L^{\infty}(\mathbb{R}^{N})$. We consider the following functional

$$ \begin{aligned}[b] E_{\lambda}(u)&=\frac{1}{p} \int_{\mathbb{R}^{N}} \vert \nabla u \vert ^{p}\,dx+ \frac{1}{q} \int_{\mathbb{R}^{N}} \vert \nabla u \vert ^{q}\,dx\\ &\quad {}- \frac {\lambda}{k} \int_{\mathbb{R}^{N}}V(x) \vert u \vert ^{k}\,dx- \frac{1}{p^{\ast }} \int_{\mathbb{R}^{N}}K(x) \vert u \vert ^{p^{\ast}}\,dx. \end{aligned} $$

(2.1)

From the following Lemmas 2.1–2.2 the functional $E_{\lambda}$ is well defined in X. Obviously, a critical point of $E_{\lambda}$ in X is a weak solution of (1.1).

The value S is the best Sobolev constant, i.e.,

$$\begin{aligned} S=\inf \biggl\{ \frac{ \Vert \nabla u \Vert _{p}^{p}}{ \Vert u \Vert _{p^{\ast}}^{p}}: u\in D^{1,p}\bigl( \mathbb{R}^{N}\bigr), u\neq0 \biggr\} . \end{aligned}$$

(2.2)

Lemma 2.1

Suppose that $V(x)\in L^{r}(\mathbb{R}^{N})$ with $r=\frac{p^{\ast }}{p^{\ast}-k}$, then the functional

$$J(u)= \int_{\mathbb{R}^{N}}V(x) \vert u \vert ^{k}\,dx $$

is well defined and weakly continuous on $D^{1,p}(\mathbb{R}^{N})$. Moreover, $J(u)$ is continuously differentiable, its derivative $J': D^{1,p}(\mathbb{R}^{N})\rightarrow(D^{1,p}(\mathbb{R}^{N}))^{\ast}$ is given by

$$J'(u)\psi=k \int_{\mathbb{R}^{N}}V(x) \vert u \vert ^{k-2}u\cdot\psi \,dx, \quad\forall\psi\in D^{1,p}\bigl(\mathbb{R}^{N}\bigr). $$

Proof

For $u\in X\subset L^{p^{\ast}}(\mathbb{R}^{N})$, by Hölder inequality, we have

$$\int_{\mathbb{R}^{N}}V(x) \vert u \vert ^{k}\,dx\leq \Vert V \Vert _{r} \Vert u \Vert _{p^{\ast}}. $$

This implies that $J(u)$ is well defined.

Let $\{u_{n}\}$ converge weakly to u in $D^{1,p}(\mathbb{R}^{N})$. Then $\{u_{n}\}$ is bounded in $L^{p^{\ast}}(\mathbb{R}^{N})$ and $\{ |u_{n}|^{k}\}$ is bounded in $L^{\frac{p^{\ast}}{k}}(\mathbb {R}^{N})$. Hence, $\{|u_{n}|^{k}\}$ converges weakly to $|u|^{k}$ in $L^{\frac{p^{\ast}}{k}}(\mathbb{R}^{N})$. Since $V(x)\in L^{\frac {p^{\ast}}{p^{\ast}-k}}(\mathbb{R}^{N})$, we have

$$\int_{\mathbb{R}^{N}}V(x) \vert u_{n} \vert ^{k}\,dx\to \int_{\mathbb {R}^{N}}V(x) \vert u \vert ^{k}\,dx, $$

which implies weak continuity. The proof of the rest is similar to that of Lemma 2.6 in [15], we omit it. □

Using a similar argument as in the proof of Lemma 2.1, we have

Lemma 2.2

Suppose that $K(x)\in L^{\infty}(\mathbb{R}^{N})$, then the functional

$$H(u)= \int_{\mathbb{R}^{N}}K(x) \vert u \vert ^{p^{\ast}}\,dx $$

is well defined on $D^{1,p}(\mathbb{R}^{N})$. Moreover, $H(u)$ is continuously differentiable, its derivative $H': D^{1,p}(\mathbb {R}^{N})\to (D^{1,p}(\mathbb{R}^{N}) )^{\ast}$ is given by

$$H'(u)\psi=p^{\ast} \int_{\mathbb{R}^{N}}K(x) \vert u \vert ^{p^{\ast}-2}u\cdot \psi \,dx, \quad\forall\psi\in D^{1,p}\bigl(\mathbb{R}^{N}\bigr). $$

The following lemmas and definitions are also needed in our discussion.

Lemma 2.3

([6])

Let $s>1$ and Ω be an open set in $\mathbb{R}^{N}$. Consider $u_{n}$, $u\in W^{1,s}(\Omega)$, $n=1,2,3,\dots$ . Let $a(x,\xi)\in C^{0}(\Omega\times\mathbb{R}^{N},\mathbb{R}^{N})$ have, for positive numbers $\alpha,\beta>0$, the following properties:

• (i) $\alpha|\xi|^{s}\leq a(x,\xi)\xi$ for all $\xi\in \mathbb{R}^{N}$,

• (ii) $|a(x,\xi)|\leq\beta|\xi|^{s-1}$ for all $(x,\xi)\in \Omega\times\mathbb{R}^{N}$,

• (iii) $(a(x,\xi)-a(x,\eta) )(\xi-\eta)>0$ for all $(x,\xi)\in\Omega\times\mathbb{R}^{N}$ with $\xi\neq\eta$.

Then $\nabla u_{n}\to\nabla u$ in $L^{s}(\Omega)$ if and only if

$$\lim_{n\to\infty} \int_{\Omega} \bigl(a\bigl(x,\nabla u_{n}(x)\bigr)-a \bigl(x,\nabla u(x)\bigr) \bigr) \bigl(\nabla u_{n}(x)-\nabla u(x) \bigr)\,dx=0. $$

Lemma 2.4

([8, 9])

Let $\{u_{n}\}$ converge weakly to u in $D^{1,p}(\mathbb{R}^{N})$ such that $\{|u_{n}|^{p^{\ast}}\}$ converges weakly to a nonnegative measure ν on $\mathbb{R}^{N}$. Then, for some at most countable set J, we have

$$\begin{aligned} \nu= \vert u \vert ^{p^{\ast}}+\sum_{j\in J} \nu_{j}\delta_{x_{j}}\quad \textit{and}\quad \sum _{j\in J}\nu_{j}^{\frac{p}{p^{\ast}}}< \infty, \end{aligned}$$

(2.3)

where $x_{j}\in\mathbb{R}^{N}$, $\delta_{x_{j}}$ denotes the Dirac measure at $x_{j}$, and $\nu_{j}$ are constants.

Definition 2.1

• (i) Let X be a Banach space and $E: X\rightarrow\mathbb{R}$ be a differentiable functional. A sequence $\{u_{k}\}\subseteq X$ is called a $(\mathrm{PS})_{\mathrm{c}}$ sequence for E if $E(u_{k})\to c$ and $E'(u_{k})\rightarrow0$ (in $X^{\ast}$) as $k\rightarrow\infty$.

• (ii) If every $(\mathrm{PS})_{\mathrm{c}}$ sequence for E has a converging subsequence (in X), we say that E satisfies the $(\mathrm{PS})_{\mathrm{c}}$-conditions.

In the rest of this section, we introduce some preparatory work for the proof of Theorem 1.1.

Lemma 2.5

Let $\{u_{n}\}\subset X$ be a $(\mathrm{PS})_{\mathrm{c}}$ sequence for $E_{\lambda}(u)$. Then $\{u_{n}\}$ is bounded in X.

Proof

Suppose $\{u_{n}\}\subset X$ is a $(\mathrm{PS})_{\mathrm{c}}$ sequence of $E_{\lambda}(u)$, i.e.,

$$\begin{aligned} E_{\lambda}(u_{n})=c+o(1), \qquad E'_{\lambda}(u_{n})=o(1), \end{aligned}$$

(2.4)

where $o(1)\to0$ as $n\to\infty$. By (2.4), for n large enough, we have

$$ \begin{aligned} c+1+ \Vert u_{n} \Vert &\geq E_{\lambda}(u_{n})-\frac{1}{p^{\ast}}E_{\lambda }'(u_{n})u_{n} \\ &= \biggl(\frac{1}{p}-\frac{1}{p^{\ast}} \biggr) \int_{\mathbb {R}^{N}} \vert \nabla u_{n} \vert ^{p}\,dx+ \biggl(\frac{1}{q}-\frac{1}{p\ast } \biggr) \int_{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{q}\,dx\\ &\quad {} - \biggl(\frac{\lambda}{k}-\frac{\lambda}{p^{\ast}} \biggr) \int _{\mathbb{R}^{N}}V(x) \vert u_{n} \vert ^{k}\,dx \\ &\geq \biggl(\frac{1}{p}-\frac{1}{p^{\ast}} \biggr) \int_{\mathbb {R}^{N}} \vert \nabla u_{n} \vert ^{p}\,dx+ \biggl(\frac{1}{q}-\frac{1}{p^{\ast }} \biggr) \int_{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{q}\,dx \\ & \quad{}- \biggl(\frac{\lambda}{k}-\frac{\lambda}{p^{\ast}} \biggr)S^{-\frac{k}{p}} \bigl\Vert V(x) \bigr\Vert _{L^{\frac{p^{\ast}}{p^{\ast}-k}}} \biggl( \int_{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{p}\,dx \biggr)^{\frac{k}{p}}. \end{aligned} $$

That is, for all large n, we have

$$c_{1}\bigl(1+ \Vert u_{n} \Vert + \Vert u_{n} \Vert ^{k}\bigr)\geq c_{2} \Vert u_{n} \Vert ^{p}_{D^{1,p}}+c_{3} \Vert u_{n} \Vert ^{q}_{D^{1,q}}, $$

where $c_{1}$, $c_{2}$ and $c_{3}$ are positive constants independent of n.

Suppose $\|u_{n}\|\to\infty$. We distinguish the following three cases:

• (1) $\|u_{n}\|_{D^{1,p}}\to\infty$ and $\|u_{n}\| _{D^{1,q}}\to\infty$;

• (2) $\|u_{n}\|_{D^{1,p}}\to\infty$ and $\{\|u_{n}\| _{D^{1,q}}\}$ is bounded;

• (3) $\{\|u_{n}\|_{D^{1,p}}\}$ is bounded and $\|u_{n}\| _{D^{1,q}}\to\infty$.

If case (1) occurs, for all large n, we get

$$ \begin{aligned} c_{1}\bigl(1+ \Vert u_{n} \Vert + \Vert u_{n} \Vert ^{k}\bigr)&\geq c_{2} \Vert u_{n} \Vert ^{p}_{D^{1,p}}+c_{3} \Vert u_{n} \Vert ^{q}_{D^{1,q}} \\ &\geq c_{2} \Vert u_{n} \Vert ^{q}_{D^{1,p}}+c_{3} \Vert u_{n} \Vert ^{q}_{D^{1,q}} \\ &\geq c_{4} \bigl( \Vert u_{n} \Vert ^{q}_{D^{1,p}}+ \Vert u_{n} \Vert ^{q}_{D^{1,q}} \bigr) \\ &\geq c_{5} \Vert u_{n} \Vert ^{q}, \end{aligned} $$

which is a contradiction to the fact $k < q$.

If case (2) is true, for all large n, we have

$$ \begin{aligned} &c_{1} \bigl(1+ \Vert u_{n} \Vert _{D^{1,p}}+ \Vert u_{n} \Vert _{D^{1,q}}+2^{k-1} \Vert u_{n} \Vert ^{k}_{D^{1,p}}+2^{k-1} \Vert u_{n} \Vert ^{k}_{D^{1,q}} \bigr) \\ &\quad \geq c_{2} \Vert u_{n} \Vert ^{p}_{D^{1,p}}+c_{3} \Vert u_{n} \Vert ^{q}_{D^{1,q}} \\ &\quad \geq c_{2} \Vert u_{n} \Vert ^{p}_{D^{1,p}}, \end{aligned} $$

thus

$$0< \frac{c_{2}}{c_{1}}\leq\lim_{n\to\infty} \biggl(\frac{1}{ \Vert u_{n} \Vert ^{p}_{D^{1,p}}}+ \frac{1}{ \Vert u_{n} \Vert ^{p-1}_{D^{1,p}}}+\frac{ \Vert u_{n} \Vert _{D^{1,q}}}{ \Vert u_{n} \Vert ^{p}_{D^{1,p}}} +\frac{2^{k-1}}{ \Vert u_{n} \Vert ^{p-k}_{D^{1,p}}}+\frac{2^{k-1} \Vert u_{n} \Vert ^{k}_{D^{1,q}}}{ \Vert u_{n} \Vert ^{p}_{D^{1,p}}} \biggr)=0. $$

This is impossible.

Proceeding as in the second case, one can also verify that the third case cannot happen. Hence, the proof is completed. □

Lemma 2.6

If $c<0$, then there exists a $\lambda^{\ast}>0$ such that $E_{\lambda}$ satisfies $(\mathrm{PS})_{\mathrm{c}}$-conditions for all $0<\lambda <\lambda^{\ast}$.

Proof

For $\varphi\in C^{\infty}_{0}(\mathbb{R}^{N})$ and $w\in X$, from (2.2) we have

$$ \begin{aligned}[b] &S^{\frac{1}{p}} \biggl( \int_{\mathbb{R}^{N}} \vert w\varphi \vert ^{p^{\ast }}\,dx \biggr)^{\frac{1}{p^{\ast}}}\\ &\quad \leq \biggl( \int_{\mathbb {R}^{N}} \bigl\vert \nabla(w\varphi) \bigr\vert ^{p}\,dx \biggr)^{\frac{1}{p}} \\ &\quad \leq \biggl( \int_{\mathbb{R}^{N}} \vert w \vert ^{p} \vert \nabla \varphi \vert ^{p}\,dx \biggr)^{\frac{1}{p}}+ \biggl( \int_{\mathbb{R}^{N}} \vert \nabla w \vert ^{p} \vert \varphi \vert ^{p}\,dx \biggr)^{\frac{1}{p}} \\ &\quad \leq \biggl( \int_{\mathbb{R}^{N}} \vert w \vert ^{p} \vert \nabla \varphi \vert ^{p}\,dx \biggr)^{\frac{1}{p}}+ \biggl( \int_{\mathbb{R}^{N}}\bigl( \vert \nabla w \vert ^{p}+ \vert \nabla w \vert ^{q}\bigr) \vert \varphi \vert ^{p}\,dx \biggr)^{\frac{1}{p}}. \end{aligned} $$

(2.5)

Suppose $\{u_{n}\}$ is a $(\mathrm{PS})_{\mathrm{c}}$ sequence. As a consequence of the boundedness of $\{u_{n}\}$, given by Lemma 2.5, there exists a $u\in X$ such that, up to subsequence, $u_{n}\rightharpoonup u$ in X.

Let $\psi\in C^{\infty}_{0}(\mathbb{R}^{N})$ satisfy $\psi(x)=0$ for $|x|>1$, $\psi(x)=1$ for $|x|\leq\frac{1}{2}$, $0\leq\psi (x)\leq1$, $x\in\mathbb{R}^{N}$.

Applying Lemma 2.4 gives

$$\vert u_{n} \vert ^{p^{\ast}}\rightharpoonup \vert u \vert ^{p^{\ast}}+\sum_{j\in J}\nu _{j}\delta_{x_{j}}. $$

Since $\{u_{n}\}$ is bounded, there exists a nonnegative measure μ such that

$$ \vert \nabla u_{n} \vert ^{p}+ \vert \nabla u_{n} \vert ^{q}\rightharpoonup\mu. $$

(2.6)

For each index j and each $0<\varepsilon<1$, define

$$\psi_{\varepsilon}(x):=\psi \biggl(\frac{x-x_{j}}{\varepsilon } \biggr). $$

It follows from inequality (2.5) that

$$\begin{aligned} S^{\frac{1}{p}} \biggl( \int_{\mathbb{R}^{N}} \vert u_{n}\psi_{\varepsilon } \vert ^{p^{\ast}}\,dx \biggr)^{\frac{1}{p^{\ast}}}&\leq \biggl( \int _{\mathbb{R}^{N}} \vert u_{n} \vert ^{p} \vert \nabla\psi_{\varepsilon} \vert ^{p}\,dx \biggr)^{\frac{1}{p}}\\ &\quad{}+ \biggl( \int_{\mathbb{R}^{N}}\bigl( \vert \nabla u_{n} \vert ^{p}+ \vert \nabla u_{n} \vert ^{q}\bigr) \vert \psi_{\varepsilon} \vert ^{p}\,dx \biggr)^{\frac{1}{p}}. \end{aligned} $$

Furthermore, letting $n\to\infty$, Lemma 2.4 and (2.6) together imply that

$$S^{\frac{1}{p}} \biggl( \int_{\mathbb{R}^{N}} \vert \psi_{\varepsilon } \vert ^{p^{\ast}}\,d \nu \biggr)^{\frac{1}{p^{\ast}}}\leq \biggl( \int _{\mathbb{R}^{N}} \vert u \vert ^{p} \vert \nabla \psi_{\varepsilon} \vert ^{p}\,dx \biggr)^{\frac{1}{p}}+ \biggl( \int_{\mathbb{R}^{N}} \vert \psi_{\varepsilon } \vert ^{p}\,d \mu \biggr)^{\frac{1}{p}}, $$

and then, by taking $\varepsilon\to0$,

$$S^{\frac{1}{p}} \biggl( \int_{x_{j}}\,d\nu \biggr)^{\frac{1}{p^{\ast }}}\leq \biggl( \int_{x_{j}}\,d\mu \biggr)^{\frac{1}{p}}, $$

which yields

$$\begin{aligned} S\nu^{\frac{p}{p^{\ast}}}_{j}\leq\mu_{j}:= \int_{x_{j}}\,d\mu. \end{aligned}$$

(2.7)

On the other hand, from the fact that $E'_{\lambda}(u_{n})\psi _{\varepsilon}u_{n}\to0$ we have

$$ \begin{aligned}[b] & \int_{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{p-2}u_{n}\nabla\psi _{\varepsilon}\nabla u_{n}\,dx+ \int_{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{q-2}u_{n}\nabla\psi_{\varepsilon}\nabla u_{n}\,dx \\ &\quad=\lambda \int_{\mathbb{R}^{N}}V(x) \vert u_{n} \vert ^{k} \psi_{\varepsilon }\,dx+ \int_{\mathbb{R}^{N}}K(x) \vert u_{n} \vert ^{p^{\ast}} \psi_{\varepsilon }\,dx \\ &\qquad{}- \int_{\mathbb{R}^{N}}\bigl( \vert \nabla u_{n} \vert ^{p}+ \vert \nabla u_{n} \vert ^{q}\bigr) \psi_{\varepsilon}\,dx+o(1),\quad \text{as } n\to\infty, \end{aligned} $$

(2.8)

and since $V(x)\psi_{\varepsilon}\in L^{r}(\mathbb{R}^{N})$, Lemma 2.1 and (2.8) show that

$$ \begin{aligned}[b] &\lim_{n\to\infty} \biggl( \int_{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{p-2}u_{n}\nabla\psi_{\varepsilon}\nabla u_{n}\,dx+ \int _{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{q-2}u_{n}\nabla\psi_{\varepsilon }\nabla u_{n}\,dx \biggr) \\ &\quad=\lambda \int_{\mathbb{R}^{N}}V(x) \vert u \vert ^{k} \psi_{\varepsilon }\,dx+ \int_{\mathbb{R}^{N}}K(x)\psi_{\varepsilon}\,d\nu- \int_{\mathbb {R}^{N}}\psi_{\varepsilon}\,d\mu. \end{aligned} $$

(2.9)

From Hölder inequality with p, $p/(p-1)$, we have

$$ \begin{aligned} \biggl\vert \int_{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{p-2}u_{n}\nabla\psi _{\varepsilon}\nabla u_{n}\,dx \biggr\vert &\leq \int_{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{p-1} \vert u_{n}\nabla\psi_{\varepsilon} \vert \,dx \\ &\leq \Vert u_{n} \Vert ^{p-1} \biggl( \int_{B_{\varepsilon }(x_{j})} \vert u_{n} \vert ^{p} \vert \nabla\psi_{\varepsilon} \vert ^{p}\,dx \biggr)^{1/p}. \end{aligned} $$

Furthermore, since $|u_{n}\nabla\psi_{\varepsilon}|\to|u\nabla\psi _{\varepsilon}|$ in $L^{p}(\mathbb{R}^{N})$,

$$ \begin{aligned}[b] &\lim_{n\to\infty} \biggl( \Vert u_{n} \Vert ^{p-1} \biggl( \int_{B_{\varepsilon }(x_{j})} \vert u_{n} \vert ^{p} \vert \nabla\psi_{\varepsilon} \vert ^{p}\,dx \biggr)^{1/p} \biggr)\\ &\quad \leq C \biggl( \int_{B_{\varepsilon }(x_{j})} \vert u \vert ^{p} \vert \nabla \psi_{\varepsilon} \vert ^{p}\,dx \biggr)^{1/p} \\ &\quad \leq C \biggl( \int_{B_{\varepsilon}(x_{j})} \vert u \vert ^{p^{\ast}}\,dx \biggr)^{1/p^{\ast}} \biggl( \int_{B_{\varepsilon}(x_{j})} \vert \nabla\psi _{\varepsilon} \vert ^{N}\,dx \biggr)^{1/N} \\ &\quad \leq C \biggl( \int_{B_{\varepsilon}(x_{j})} \vert u \vert ^{p^{\ast}}\,dx \biggr)^{1/p^{\ast}}. \end{aligned} $$

(2.10)

Now by replacing p with q, (2.10) reveals

$$\lim_{n\to\infty} \int_{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{q-2}u_{n}\nabla\psi_{\varepsilon}\nabla u_{n}\,dx\leq C \biggl( \int_{B_{\varepsilon}(x_{j})} \vert u \vert ^{q^{\ast}}\,dx \biggr)^{1/q^{\ast}}. $$

In (2.9),

$$ K(x_{j})\nu_{j}=\mu_{j} $$

(2.11)

is valid if $\varepsilon\to0$. Besides, if $K(x_{j})\leq0$, one gets $\mu_{j}=\nu_{j}=0$; while if $K(x_{j})>0$, by (2.7), we have

• (i) $\nu_{j}=0$;

• (ii) $\nu_{j}\geq (\frac{S}{K(x_{j})} )^{\frac{N}{P}}$.

Define

$$\begin{gathered} \nu_{\infty}:=\lim_{R\to\infty} \limsup_{n\to\infty} \int _{ \vert x \vert >R} \vert u_{n} \vert ^{p^{\ast}}\,dx; \\ \mu_{\infty}:=\lim_{R\to\infty}\limsup_{n\to\infty} \int _{ \vert x \vert >R} \vert \nabla u_{n} \vert ^{p}+ \vert \nabla u_{n} \vert ^{q}\,dx. \end{gathered} $$

By the concentration–compactness principle at infinity, $\nu_{\infty }$ and $\mu_{\infty}$ exist and satisfy:

($a_{1}$):

$\limsup_{n\to\infty} \int_{\mathbb {R}^{N}} \vert u_{n} \vert ^{p^{\ast}}\,dx= \int_{\mathbb{R}^{N}}\,d\nu+\nu_{\infty}$;

($a_{2}$):

$\limsup_{n\to\infty} \int_{\mathbb{R}^{N}} ( \vert \nabla u_{n} \vert ^{p}+ \vert \nabla u_{n} \vert ^{q} )\,dx= \int_{\mathbb {R}^{N}} d\mu+\mu_{\infty}$;

($a_{3}$):

$S\nu_{\infty}^{\frac{p}{p^{\ast}}}\leq \mu_{\infty}$.

Let $\psi_{R}\in C^{\infty}(\mathbb{R}^{N})$ satisfy $\psi _{R}(x)=0$ for $|x|< R$, $\psi_{R}(x)=1$ for $|x|>2R$, $0\leq\psi _{R}(x)\leq1$, $x\in\mathbb{R}^{N}$. Then we get

$$ \begin{aligned}[b] & \int_{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{p-2}u_{n}\nabla\psi_{R}\nabla u_{n}\,dx+ \int_{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{q-2}u_{n}\nabla\psi _{R}\nabla u_{n}\,dx \\ &\quad=\lambda \int_{\mathbb{R}^{N}}V(x) \vert u_{n} \vert ^{k} \psi_{R}\,dx+ \int _{\mathbb{R}^{N}}K(x) \vert u_{n} \vert ^{p^{\ast}} \psi_{R}\,dx \\ &\qquad{}- \int_{\mathbb{R}^{N}}\bigl( \vert \nabla u_{n} \vert ^{p}+ \vert \nabla u_{n} \vert ^{q}\bigr) \psi_{R}\,dx+o(1). \end{aligned} $$

(2.12)

Similar to the proof of (2.10), we have

$$\limsup_{n\to\infty} \biggl\vert \int_{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{p-2}u_{n}\nabla\psi_{R}\nabla u_{n}\,dx \biggr\vert \leq C \biggl( \int_{R< \vert x \vert < 2R} \vert u \vert ^{p^{\ast}}\,dx \biggr)^{1/p^{\ast}}\to0,\quad \mbox{as } R\to\infty. $$

Let $R\to\infty$ in (2.12), then

$$ \Vert K \Vert _{\infty}\nu_{\infty}=\mu_{\infty}, $$

(2.13)

which in turn means, by ($a_{3}$),

• (iii) $\nu_{\infty}=0$;

• (iv) $\nu_{\infty}\geq (\frac{S}{\|K\|_{\infty }} )^{\frac{N}{p}}$.

We now claim that (ii) and (iv) are impossible if λ is chosen small enough. Indeed, since $\{u_{n}\}$ is a $(\mathrm{PS})_{\mathrm {c}}$ sequence, for n large enough, we have

$$ \begin{aligned}[b] 0&>c+o(1) \Vert u_{n} \Vert =E_{\lambda}(u_{n})-\frac{1}{p^{\ast}} E'_{\lambda }(u_{n})u_{n} \\ &\geq \biggl(\frac{1}{p}-\frac{1}{p^{\ast}} \biggr) \int_{\mathbb {R}^{N}} \vert \nabla u_{n} \vert ^{p}\,dx-\frac{(p^{\ast}-k)\lambda}{kp^{\ast}} \int _{\mathbb{R}^{N}}V(x) \vert u_{n} \vert ^{k}\,dx \\ &\geq\frac{S}{N} \Vert u_{n} \Vert ^{p}_{p\ast}- \frac{(p^{\ast}-k)\lambda }{kp^{\ast}} \bigl\Vert V(x) \bigr\Vert _{r} \Vert u_{n} \Vert _{p\ast}^{k}. \end{aligned} $$

(2.14)

This yields that

$$ \Vert u_{n} \Vert _{p\ast}\leq C\lambda^{\frac{1}{p-k}}. $$

(2.15)

On the other hand, for n and R large enough and if (iv) occurs, we get

$$ \begin{aligned}[b] 0&>c+o(1) \Vert u_{n} \Vert \\ &\geq \biggl(\frac{1}{p}-\frac{1}{p^{\ast}} \biggr) \int _{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{p}\,dx+ \biggl(\frac{1}{q}-\frac {1}{p^{\ast}} \biggr) \int_{\mathbb{R}^{N}} \vert \nabla u_{n} \vert ^{q}\,dx\\ &\quad{}-\frac {(p^{\ast}-k)\lambda}{kp^{\ast}} \int_{\mathbb {R}^{N}}V(x) \vert u_{n} \vert ^{k}\,dx \\ &\geq\frac{1}{N} \int_{\mathbb{R}^{N}} \bigl( \vert \nabla u_{n} \vert ^{p}+ \vert \nabla u_{n} \vert ^{q} \bigr) \psi_{R}\,dx-\frac{(p^{\ast}-k)\lambda }{kp^{\ast}} \bigl\Vert V(x) \bigr\Vert _{r} \Vert u_{n} \Vert _{p\ast}^{k} \\ &\geq\frac{1}{N}\mu_{\infty}+o(1)-\frac{(p^{\ast}-k)\lambda }{kp^{\ast}} \bigl\Vert V(x) \bigr\Vert _{r} \Vert u_{n} \Vert _{p\ast}^{k} \\ &\geq\frac{1}{N}S^{\frac{N}{p}} \Vert K \Vert _{\infty}^{\frac {p-N}{p}}-C\lambda^{\frac{p}{p-k}}, \end{aligned} $$

(2.16)

where we use (2.15) and ($a_{3}$). Therefore we can choose $\lambda^{\ast}>0$ such that for every $\lambda\in(0,\lambda^{\ast})$

$$\frac{1}{N}S^{\frac{N}{p}} \Vert K \Vert _{\infty}^{\frac{p-N}{p}}-C \lambda ^{\frac{p}{p-k}}>0, $$

which is a contradiction to (2.16).

A similar argument shows that (ii) cannot occur if $\lambda^{\ast}$ is chosen properly. Thus, $\mu_{i}=\nu_{i}=\mu_{\infty}=\nu_{\infty}=0$. From ($a_{1}$) and (2.3),

$$ \lim_{n\to\infty} \int_{\mathbb{R}^{N}} \vert u_{n} \vert ^{p^{\ast}}\,dx= \int _{\mathbb{R}^{N}} \vert u \vert ^{p^{\ast}}\,dx. $$

(2.17)

And Brezis–Lieb Lemma [16] implies

$$ \lim_{n\to\infty} \int_{\mathbb{R}^{N}} \vert u_{n}-u \vert ^{p^{\ast}}\,dx=0. $$

(2.18)

Since $K(x)\in L^{\infty}(\mathbb{R}^{N})$,

$$ \biggl\vert \int_{\mathbb{R}^{N}}K(x)|u_{n} \biggr\vert ^{p^{\ast }-1}|u_{n}-u \vert \,dx \vert \leq \Vert K \Vert _{\infty} \Vert u_{n} \Vert _{p^{\ast}}^{p^{\ast }-1} \biggl( \int_{\mathbb{R}^{N}} \vert u_{n}-u \vert ^{p^{\ast}}\,dx \biggr)^{\frac{1}{p^{\ast}}}. $$

(2.19)

Then from (2.18) and (2.19), one gets

$$ \lim_{n\to\infty} \int_{\mathbb{R}^{N}}K(x) \vert u_{n} \vert ^{p^{\ast }-1} \vert u_{n}-u \vert \,dx=0. $$

(2.20)

A similar argument shows that

$$ \lim_{n\to\infty} \int_{\mathbb{R}^{N}}V(x) \vert u_{n} \vert ^{k-1} \vert u_{n}-u \vert \,dx=0. $$

(2.21)

Now we define

$$\begin{aligned} \bigl\langle A_{r}(u),\varphi\bigr\rangle := \int_{\mathbb{R}^{N}} \vert \nabla u \vert ^{r-2}\langle \nabla u,\nabla\varphi \rangle _{\mathbb {R}^{N}}\,dx, \quad \forall u, \varphi\in X. \end{aligned}$$

Considering $\langle E'_{\lambda}(u_{n}),u_{n}-u\rangle$ as $n\to \infty$, we have

$$\begin{aligned} &\lim_{n\to\infty}\biggl[\bigl\langle A_{p}(u_{n}),u_{n}-u \bigr\rangle +\bigl\langle A_{q}(u_{n}),u_{n}-u \bigr\rangle \\ &\quad {}-\lambda \int_{\mathbb{R}^{N}}V(x) \vert u_{n} \vert ^{k-2}u_{n}(u_{n}-u)\,dx- \int _{\mathbb{R}^{N}}K(x) \vert u_{n} \vert ^{p^{\ast}-2}u_{n}(u_{n}-u)\,dx\biggr]=0. \end{aligned} $$

It means

$$\lim_{n\to\infty}\bigl[\bigl\langle A_{p}(u_{n}),u_{n}-u \bigr\rangle +\bigl\langle A_{q}(u_{n}),u_{n}-u \bigr\rangle \bigr]=0. $$

From the monotonicity of $A_{q}(u)$ (see [4]), the following is true:

$$\limsup_{n\to\infty}\bigl[\bigl\langle A_{p}(u_{n}),u_{n}-u \bigr\rangle +\bigl\langle A_{q}(u),u_{n}-u\bigr\rangle \bigr] \leq0. $$

Notice that $u_{n}\rightharpoonup u$ in $D^{1,q}(\mathbb{R}^{N})$,

$$\lim_{n\to\infty}\bigl\langle A_{q}(u),u_{n}-u \bigr\rangle =0. $$

Therefore

$$ \limsup_{n\to\infty}\bigl\langle A_{p}(u_{n}),u_{n}-u \bigr\rangle \leq0. $$

(2.22)

Consequently,

$$\lim_{n\to\infty}\bigl\langle A_{p}(u_{n})-A_{p}(u),u_{n}-u \bigr\rangle =0. $$

Finally, the following two results can be obtained by taking $a(x,\xi )=|\xi|^{p-2}\xi$ and using Lemma 2.3:

$$\begin{gathered} \lim_{n\to\infty} \int_{\mathbb{R}^{N}} \bigl\vert \nabla(u_{n}-u) \bigr\vert ^{p}\,dx=0, \\ \lim_{n\to\infty} \int_{\mathbb{R}^{N}} \bigl\vert \nabla(u_{n}-u) \bigr\vert ^{q}\,dx=0. \end{gathered} $$

The proof is complete. □

Now truncate the energy functional of problem (1.1). By Sobolev embedding theorem, for all $u\in X$, we have

$$ \begin{aligned}[b] E_{\lambda}(u)&=\frac{1}{p} \int_{\mathbb{R}^{N}} \vert \nabla u \vert ^{p}\,dx+ \frac{1}{q} \int_{\mathbb{R}^{N}} \vert \nabla u \vert ^{q}\,dx\\&\quad{}- \frac {\lambda}{k} \int_{\mathbb{R}^{N}}V(x) \vert u \vert ^{k}\,dx- \frac{1}{p^{\ast }} \int_{\mathbb{R}^{N}}K(x) \vert u \vert ^{p^{\ast}}\,dx \\ &\geq\frac{1}{p} \Vert u \Vert _{D^{1,p}}^{p}-\lambda c_{1} \Vert u \Vert _{D^{1,p}}^{k}-c_{2} \Vert u \Vert _{D^{1,p}}^{p^{\ast}}. \end{aligned} $$

(2.23)

Let $h(t)=c_{3}t^{p}-\lambda c_{4}t^{k}-c_{5}t^{p^{\ast}}$, we need to discuss the further properties of $h(t)$. Firstly, it is easy to see that there exist $\lambda^{\ast}$, $T_{0}$ and $T_{1}$, with $0< T_{0}< T_{1}$, such that

$$ \begin{gathered} h(T_{0})=h(T_{1})=0, \\ h(t)\leq0,\quad\forall0\leq t\leq T_{0}, \\ h(t)>0,\quad\forall T_{0}< t< T_{1}, \\ h(t)\leq0, \quad\forall t\geq T_{1}. \end{gathered} $$

for every $\lambda\in(0,\lambda^{\ast})$.

Secondly, let $\tau: R^{+}\rightarrow[0,1]$ be a $C^{\infty}$ non-increasing function such that

$$\tau(t)=1,\quad\mbox{if } t\leq T_{0} ; \qquad \tau(t)=0,\quad \mbox{if } t\geq T_{1}. $$

We consider the truncated functional

$$\begin{aligned} E_{\infty}(u)&=\frac{1}{p} \int_{\mathbb{R}^{N}} \vert \nabla u \vert ^{p}\,dx+ \frac{1}{q} \int_{\mathbb{R}^{N}} \vert \nabla u \vert ^{q}\,dx- \frac {\lambda}{k} \int_{\mathbb{R}^{N}}V(x) \vert u \vert ^{k}\,dx\\&\quad{}- \frac{\tau( \Vert u \Vert _{D^{1,p}})}{p^{\ast}} \int_{\mathbb{R}^{N}}K(x) \vert u \vert ^{p^{\ast}}\,dx \end{aligned} $$

and suppose

$$\overline{h}(t)=c_{3}t^{p}-\lambda c_{4}t^{k}-c_{5}t^{p^{\ast}} \tau(t). $$

Then

$$E_{\infty}(u)\geq\overline{h}\bigl( \Vert u \Vert _{D^{1,p}} \bigr). $$

At the same time, we notice that $\overline{h}(t)\geq h(t)$, if $t>0$; $\overline{h}(t)= h(t)$ if $0\leq t\leq T_{0}$; $0\leq h(t)\leq \overline{h}(t)$, if $T_{0}< t< T_{1}$; $\overline{h}(t)>0$, if $t>T_{1}$. Thus we get that $E_{\lambda}(u)=E_{\infty}(u)$ when $0\leq\| u\|_{D^{1,p}}\leq T_{0}$. Furthermore, for $\tau\in C^{\infty}$ we have $E_{\infty}(u)\in C^{1}(X,\mathbb{R})$ and obtain the following lemma.

Lemma 2.7

• (a) If $E_{\infty}(u)<0$, then $\|u\|_{D^{1,p}}< T_{0}$, and $E_{\lambda}(v)=E_{\infty}(v)$ for all v in a small enough neighborhood of u.

• (b) For all $\lambda\in(0,\lambda^{\ast})$, $E_{\infty }(u)$ satisfies the $(\mathrm{PS})_{\mathrm{c}}$-conditions for $c<0$.

Proof

We prove (a) by contradiction. If $\|u\|_{D^{1,p}}\in[T_{0},+\infty )$, by the above analysis we see that

$$E_{\infty}(u)\geq\overline{h}\bigl( \Vert u \Vert _{D^{1,p}} \bigr)\geq0. $$

This is a contradiction to $E_{\infty}(u)<0$, thus $\|u\| _{D^{1,p}}< T_{0}$ and (a) holds.

Claim (b) can be proved by the $(\mathrm{PS})_{\mathrm {c}}$-conditions for $E_{\lambda}$ as $\lambda\in(0,\lambda^{\ast })$ (see Lemma 2.6). □

The following is the classical Deformation Lemma (see [14]):

Lemma 2.8

Let Y be a Banach space and consider an $f\in C^{1}(Y,\mathbb{R})$, satisfying the (PS)-conditions. If $c\in\mathbb{R}$ and N is any neighborhood of $K_{c}\triangleq\{u\in Y: f(u)=c, f'(u)=0\}$, there exist $\eta(t,u)\equiv\eta_{t}(u)\in C([0,1]\times Y,Y)$ and constants $\overline{\epsilon}>\epsilon>0$ such that

• (1) $\eta_{0}(u)=u$ $\forall u\in Y$;

• (2) $\eta_{t}(u)=u$ $\forall u\notin f^{-1}[c-\overline {\epsilon},c+\overline{\epsilon}]$;

• (3) $\eta_{t}(u)=u$ is a homeomorphism of Y onto Y $\forall t\in[0,1]$;

• (4) $f(\eta_{t}(u))\leq f(u)$ $\forall u\in Y$ $\forall t\in[0,1]$;

• (5) $\eta_{1}(f^{c+\epsilon}\setminus N)\subset f^{c-\epsilon }$, where $f^{c}=\{u\in Y: f(u)\leq c\}$ $\forall c\in\mathbb{R}$;

• (6) If $K_{c}=\emptyset$, $\eta_{1}(f^{c+\epsilon})\subset f^{c-\epsilon}$;

• (7) If f is even, $\eta_{t}$ is odd in u.

We end up this section by pointing out some concepts and results about $Z_{2}$ index theory.

Let Y be a Banach space and set

$$\Sigma=\bigl\{ A\subset Y\setminus\{0\}: A \mbox{ is closed}, -A=A\bigr\} . $$

For $A\in\Sigma$, we define the $Z_{2}$ genus of A by

$$\gamma(A)=\min\bigl\{ n\in\mathbb{N}: \mbox{there exists an odd, continuous } \phi:~A\to\mathbb{R}^{n}\setminus\{0\}\bigr\} , $$

if such a minimum does not exist, then $\gamma(A)=+\infty$.

The main properties of genus are given in the following lemma (see [14]).

Lemma 2.9

Let $A,B\in\Sigma$. Then

• (1) If there exists $f\in C(A,B)$, odd, then $\gamma(A)\leq \gamma(B)$;

• (2) If $A\subset B$, then $\gamma(A)\leq\gamma(B)$;

• (3) If there exists an odd homeomorphism between A and B, then $\gamma(A)=\gamma(B)$;

• (4) If $S^{N-1}$ is the unit sphere in $\mathbb{R}^{N}$, then $\gamma(S^{N-1})=N$;

• (5) $\gamma(A\cup B)\leq\gamma(A)+\gamma(B)$;

• (6) If $\gamma(A)<\infty$, then $\gamma(\overline{A-B})\geq \gamma(A)-\gamma(B)$;

• (7) If A is compact, then $\gamma(A)<\infty$, and there exists a $\delta>0$ such that $\gamma(A)=\gamma(N_{\delta}(A))$, where $N_{\delta}(A)=\{x\in Y: d(x,A)\leq\delta\}$;

• (8) If $Y_{0}$ is a subspace of Y with codimension k, and $\gamma(A)>k$, then $A\cap Y_{0}\neq\emptyset$.

3 Proof of Theorem 1.1

Now we are ready to prove Theorem 1.1 via genus argument.

For $1\leq j\leq n$, we define

$$c_{j}=\inf_{A\in\sum_{j}}\sup_{u\in A}E_{\infty}(u), $$

where

$$\Sigma_{j}=\bigl\{ A\subset X\setminus\{0\}: A\mbox{ is closed in }X, -A=A, \gamma(A)\geq j\bigr\} . $$

Let $K_{c}=\{u\in X : E_{\infty}(u)=c, E'_{\infty}(u)=0\}$ and suppose that $\lambda\in(0,\lambda^{\ast})$, where $\lambda^{\ast }$ is given by Lemma 2.6.

Firstly, we claim that if $j\in\mathbb{N}$, there is an $\varepsilon _{j}=\varepsilon(j)>0$ such that

$$\gamma\bigl(E_{\infty}^{-\varepsilon_{j}}\bigr)\geq j, $$

where $E_{\infty}^{-\varepsilon}=\{u\in X: E_{\infty}(u)\leq -\varepsilon\}$.

Here $W^{1,p}_{0}(\Omega)$ is the closure of $C^{\infty}_{0}(\Omega )$ with $\|u\|_{W_{0}^{1,p}(\Omega)}= (\int_{\Omega}|\nabla u|^{p} )^{\frac{1}{p}}$, and $\Omega\subset\mathbb{R}^{N}$ is an open bounded subset with $|\Omega|>0$ and $C^{1}$-boundary, $V(x)>0$ in Ω. Extending functions in $W^{1,p}_{0}(\Omega)$ by 0 outside Ω, we can assume that $W^{1,p}_{0}(\Omega)\subset X$.

Let $W_{j}$ be a j-dimensional subspace of $W^{1,p}_{0}(\Omega)$. For every $v\in W_{j}$ with $\|v\|_{W_{0}^{1,p}(\Omega)}=1$, from the assumptions of $V(x)$, it is easy to see that there exists a $d_{j}>0$ such that

$$\int_{\Omega}V(x) \vert v \vert ^{k}\,dx\geq d_{j}. $$

Since $W_{j}$ is a finite-dimensional space, all the norms in $W_{j}$ are equivalent. Thus we can define

$$ \begin{gathered} a_{j}=\sup\bigl\{ \vert \nabla v \vert ^{q}_{q}:~v\in W_{j}, \Vert v \Vert _{W_{0}^{1,p}(\Omega )}=1\bigr\} < \infty, \\ b_{j}=\sup\bigl\{ \vert v \vert ^{p^{\ast}}_{p^{\ast}}:~v \in W_{j}, \Vert v \Vert _{W_{0}^{1,p}(\Omega)}=1\bigr\} < \infty. \end{gathered} $$

(3.1)

On the other hand, for $0< t< T_{0}$, since

$$E_{\infty}(tv)=E_{\lambda}(tv)=\frac{1}{p}t^{p}+ \frac {t^{q}}{q} \vert \nabla v \vert _{q}^{q}- \frac{\lambda t^{k}}{k} \int_{\Omega }V(x) \vert v \vert ^{k}\,dx- \frac{t^{p^{\ast}}}{p^{\ast}} \int_{\Omega }K(x) \vert v \vert ^{p^{\ast}}\,dx, $$

for every $v\in W_{j}$ with $\|v\|_{W_{0}^{1,p}(\Omega)}=1$, we obtain

$$ \begin{aligned}[b] E_{\infty}(tv)&\leq\frac{1}{p}t^{p}+ \frac{a_{j}}{q}t^{q}-\frac {\lambda d_{j}}{k}t^{k}- \frac{t^{p^{\ast}}}{p^{\ast}} \int_{\Omega }K(x) \vert v \vert ^{p^{\ast}}\,dx \\ &\leq\frac{1}{p}t^{p}+\frac{a_{j}}{q}t^{q}- \frac{\lambda d_{j}}{k}t^{k}+\frac{b_{j} \vert K \vert _{\infty}}{p^{\ast}}t^{p^{\ast}}. \end{aligned} $$

(3.2)

Therefore for $\lambda\in(0,\lambda^{\ast})$, there must be a $t_{0}\in(0,T_{0})$ sufficiently small such that $E_{\infty }(t_{0}v)\leq-\varepsilon_{j}<0$, where $\varepsilon_{j}=-\frac {1}{p}t_{0}^{p}-\frac{a_{j}}{q}t_{0}^{q}+\frac{\lambda d_{j}}{k}t_{0}^{k}-\frac{b_{j}|K|_{\infty}}{p^{\ast}}t_{0}^{p^{\ast}}$. Denote $S_{t_{0}}=\{v\in X:~\|v\|_{W_{0}^{1,p}(\Omega)}=t_{0}\}$, then $S_{t_{0}}\cap W_{j}\subset E_{\infty}^{-\varepsilon_{j}}$. By Lemma 2.9,

$$\gamma\bigl(E^{-\varepsilon_{j}}_{\infty}\bigr)\geq\gamma(S_{t_{0}} \cap X_{j})\geq j. $$

As $E_{\infty}$ is continuous and even, $E_{\infty}^{-\varepsilon _{j}}\in\Sigma_{j}$ and $c_{j}\leq-\varepsilon_{j}<0$. Since $E_{\infty}$ is bounded from below, $c_{j}>-\infty$ (that is why we consider $E_{\infty}$ instead of $E_{\lambda}$). Then from Lemma 2.6 we see that $E_{\infty}$ satisfies the $(\mathrm{PS})_{\mathrm{c}}$-conditions (for $c<0$) and this implies that $K_{c}$ is a compact set.

Secondly, we claim that if for some $j\in\mathbb{N}$ there is an $i\geq0$ such that $c=c_{j}=c_{j+1}=\cdots=c_{j+i}$, then $\gamma (K_{c})\geq i+1$.

We now prove the main claim by contradiction. If $\gamma(K_{c})\leq i$, there exists a closed and symmetric set U with $K_{c}\subset U$ and $\gamma(U)\leq i$. Since $c<0$, we can also assume that the closed set $U\subset E^{0}_{\infty}$. Using Lemma 2.8, there is an odd homeomorphism

$$\eta: [0,1]\times X\rightarrow X $$

such that $\eta(E^{c+\delta}_{\infty}\backslash U)\subset E^{c-\delta}_{\infty}$ for some $\delta\in(0,-c)$.

From the hypothesis of $c=c_{j+i}$, there exists an $A\in\Sigma _{j+i}$ such that

$$\sup_{u\in A}E_{\infty}(u)< c+\delta. $$

Thus

$$\eta(A\backslash U)\subset\eta\bigl(E_{\infty}^{c+\delta}\backslash U \bigr)\subset E_{\infty}^{c-\delta}, $$

which means

$$\sup_{u\in\eta(A\backslash U)}E_{\infty}(u)\leq c-\delta. $$

But Lemma 2.9 reveals

$$\gamma\bigl(\overline{\eta(A\backslash U)}\bigr)\geq\gamma(\overline {A \backslash U})\geq\gamma(A)-\gamma(U)\geq j. $$

Hence $\overline{\eta(A\backslash U)}\in\Sigma_{j}$ and

$$c=c_{j}\leq\sup_{u\in\overline{\eta(A\backslash U)}}E_{\infty }(u)=\sup _{u\in\eta(A\backslash U)}E_{\infty}(u)\leq c-\delta. $$

So we have proved the main claim.

We now complete the proof of Theorem 1.1. For all $j\in \mathbb{N}$, we have $\Sigma_{j+1}\subset\Sigma_{j}$ and $c_{j}\leq c_{j+1}<0$. If all $c_{j}$s are distinct, then $\gamma (K_{c_{j}})\geq1$, and we know that $\{c_{j}\}$ is a sequence of distinct negative critical values of $E_{\infty}$. If for some $j_{0}$, there exists an $i\geq1$ such that

$$c=c_{j_{0}}=c_{j_{0}+1}=\cdots=c_{j_{0}+i}, $$

from the main claim, we have

$$\gamma(K_{c_{j_{0}}})\geq i+1, $$

which shows that $K_{c_{j_{0}}}$ has infinitely many distinct elements.

By Lemma 2.7, we know $E_{\lambda}(u)=E_{\infty}(u)$ when $E_{\infty}(u)<0$, and we see that there exist 2n critical points of $E_{\lambda}(u)$ with negative critical values. Therefore problem (1.1) has 2n weak solutions with negative critical energy.

4 Proof of Theorem 1.2

We denote $X_{T}=\{u\in X: u(\tau x)=u(x), \tau\in O(N)\}$ and $L^{p^{\ast}}_{T}=\{u\in L^{p^{\ast}}(\mathbb{R}^{N}): u(\tau x)=u(x), \tau\in O(N)\}$. By the principle of symmetric criticality, we have

Lemma 4.1

([15])

If $E_{\lambda}'(u)=0$ in $X_{T}^{\ast}$, then $E_{\lambda}'(u)=0$ in $X^{\ast}$.

Lemma 4.2

If $1< k< q< p< N$, $|T|=\infty$, $K(0)=0$ and $\lim_{|x|\to\infty }K(x)=0$, then $E_{\lambda}$ in $X_{T}$ satisfies the $(\mathrm {PS})_{\mathrm{c}}$-conditions for all $c\in\mathbb{R}$.

Proof

We only give a sketch of the proof because it is analogous to that of Lemma 2.6. Let $\{u_{n}\}\subset X_{T}$ be a $(\mathrm {PS})_{\mathrm{c}}$ sequence of $E_{\lambda}$. An argument similar to the one used in proving Lemma 2.5 shows that $\{u_{n}\}$ is bounded. Using Lemma 2.4, there exists a measure ν such that (2.3) holds. We claim that the concentration of ν cannot occur at any $x\neq0$. Assuming that $x_{k}\neq0$ is a singular point of ν, we have $\nu_{k}=\nu(x_{k})>0$ and since ν is T-invariant, $\nu(\tau x_{k})=\nu_{k}>0$ for all $\tau\in T$. Since $|T|=\infty$, the sum in (2.3) (see Lemma 2.4) is infinite, which is a contradiction. On the other hand, by (2.11) and since $K(0)=0$, we get $\nu_{0}=0$.

The next step in the proof is showing that the concentration of ν cannot occur at infinity. Since $\lim_{|x|\to\infty}K(x)=0$, we have

$$\lim_{R\to\infty}\limsup_{n\to\infty} \int _{ \vert x \vert >R}K(x) \vert u_{n} \vert ^{p^{\ast}}\,dx=0. $$

By the same arguments as when proving (2.13), we have $\mu _{\infty}=0$, then from $(a_{3})$ (see Lemma 2.6), we obtain $\nu_{\infty}=0$. Thus $u_{n}\to u$ in $L^{p^{\ast}}_{T}(\mathbb {R}^{N})$, and the argument at the end of the proof of Lemma 2.6 implies that $u_{n}\to u$ in $X_{T}$. □

Since $X_{T}$ is a separable Banach space (see [1]), there is a linearly independent sequence $\{e_{j}\}$ such that

$$X_{T}=\overline{\bigoplus_{j\geq1}X_{j}}, X_{j}:=\operatorname {span}\{e_{j}\}. $$

Denote $Y_{k}=\bigoplus_{j\leq k}X_{j}$ and $Z_{k}=\overline {\bigoplus_{j\geq k}X_{j}}$.

Lemma 4.3

([15])

Let $E\in C^{1}(X_{T},\mathbb{R})$ be an even functional satisfying the $(\mathrm{PS})_{\mathrm{c}}$-conditions for every $c>0$. If for every $k\in\mathbb{N}$ there exist $\rho_{k}>r_{k}>0$ such that

• (a) $\alpha_{k}:=\max_{u\in Y_{k},\|u\|=\rho_{k}}E(u)\leq0$,

• (b) $\beta_{k}:=\inf_{u\in Z_{k},\|u\|=r_{k}}E(u)\to\infty$, as $k\to\infty$,

then E has a sequence of critical values tending to ∞.

Proof of Theorem 1.2

Obviously, $E_{\lambda}$ is even and $E_{\lambda}\in C^{1}(X_{T},\mathbb{R})$. By Lemma 4.2, $E_{\lambda}$ satisfies the $(\mathrm{PS})_{\mathrm{c}}$ conditions for every $c\in \mathbb{R}$. Since $Y_{k}$ is a finite-dimensional subspace of $X_{T}$ for each $k\in\mathbb{N}$ and $K(x)>0$ a.e. in $\mathbb{R}^{N}$, this implies that there exists a constant $\varepsilon_{k}>0$ such that for all $v\in Y_{k}$ with $\|v\|=1$ we have

$$ \int_{\mathbb{R}^{N}}K(x) \vert v \vert ^{p^{\ast}}\,dx\geq \varepsilon_{k}. $$

(4.1)

On the other hand,

$$ \begin{aligned}[b] E_{\lambda}(u)&=\frac{1}{p} \int_{\mathbb{R}^{N}} \vert \nabla u \vert ^{p}\,dx+ \frac{1}{q} \int_{\mathbb{R}^{N}} \vert \nabla u \vert ^{q}\,dx\\&\quad{}- \frac {\lambda}{k} \int_{\mathbb{R}^{N}}V(x) \vert u \vert ^{k}\,dx- \frac{1}{p^{\ast }} \int_{\mathbb{R}^{N}}K(x) \vert u \vert ^{p^{\ast}}\,dx \\ &\leq\frac{1}{p} \Vert u \Vert ^{p}+\frac{1}{q} \Vert u \Vert ^{q}-\frac{1}{p^{\ast}} \int _{\mathbb{R}^{N}}K(x) \vert u \vert ^{p^{\ast}}\,dx. \end{aligned} $$

(4.2)

Therefore, if $u\in Y_{k}$, $u\neq0$, and writing $u=t_{k}v$ with $\|v\| =1$, from (4.1) and (4.2) we get

$$E_{\lambda}(u)\leq\frac{1}{p}t_{k}^{p}+ \frac{1}{q}t_{k}^{q}-\frac {\varepsilon_{k}}{p^{\ast}}t_{k}^{p^{\ast}} \leq0 $$

for large $t_{k}$. This proves (a) of Lemma 4.3.

Define

$$ \beta_{k}:=\sup_{u\in Z_{k}, \Vert u \Vert =1} \biggl( \int_{\mathbb {R}^{N}}K(x) \vert u \vert ^{p^{\ast}}\,dx \biggr)^{\frac{1}{p^{\ast}}}. $$

(4.3)

It is clear that $0\leq\beta_{k+1}\leq\beta_{k}$ and $\beta_{k}\to \beta_{0}\geq0$. Then for every $k\geq1$ there exists a $u_{k}\in Z_{k}$ such that $\|u_{k}\|=1$ and

$$ \biggl( \int_{\mathbb{R}^{N}}K(x) \vert u_{k} \vert ^{p^{\ast}}\,dx \biggr)^{\frac {1}{p^{\ast}}}\geq\frac{\beta_{0}}{2}. $$

(4.4)

By the definition of $Z_{k}$, we get $u_{k}\rightharpoonup0$ in $X_{T}$. Thus, there exists a ν such that (2.3) holds. Combining the arguments proving Lemma 4.2 and the fact that $|T|=\infty$, we see that a concentration of the measure ν can only occur at 0 and ∞. Thus, $u_{k}\to0$ in $L^{p^{\ast}}(\Omega)$, where $\Omega=\{x\in\mathbb{R}^{N}: r<|x|<R\}$ for each $0< r< R$. Due to $K(x)$ being continuous, $K(0)=0$ and $\lim_{|x|\to\infty}K(x)=0$, for any $\varepsilon>0$, we can choose small r and large R such that

$$\biggl( \int_{\{x\in\mathbb{R}^{N}: \vert x \vert < r\}}K(x) \vert u_{k} \vert ^{p^{\ast }}\,dx \biggr)^{\frac{1}{p^{\ast}}}< \frac{\varepsilon}{2},\qquad \biggl( \int_{\{x\in\mathbb{R}^{N}: \vert x \vert >R\}}K(x) \vert u_{k} \vert ^{p^{\ast}}\,dx \biggr)^{\frac{1}{p^{\ast}}}< \frac{\varepsilon}{2}. $$

Therefore, from $K(x)\in L^{\infty}(\mathbb{R}^{N})$, we have

$$\biggl( \int_{\mathbb{R}^{N}}K(x) \vert u_{k} \vert ^{p^{\ast}}\,dx \biggr)^{\frac {1}{p^{\ast}}}\to0, $$

as $k\to\infty$. Hence, by (4.4), we get $\beta_{0}=0$.

If we take $\|u\|=r_{k}$, by the definition of $\|\cdot\|$, either $\|u\| _{D^{1,p}}$ or $\|u\|_{D^{1,q}}$ is not less than $r_{k}/2$. Without loss of generality, we let $\|u\|_{D^{1,p}}\geq r_{k}/2$. Since $V(x)\geq0$ and $K(x)>0$ a.e. in $\mathbb{R}^{N}$ and $\lambda>0$, for $u\in Z_{k}$, by Sobolev inequality and (4.3), we have

$$E_{\lambda}(u)\geq\frac{1}{p} \Vert u \Vert _{D^{1,p}}^{p}- \frac{\lambda C}{k} \Vert u \Vert _{D^{1,p}}^{k}- \frac{\beta_{k}^{p^{\ast}}}{p^{\ast}} \Vert u \Vert ^{p^{\ast}}. $$

On the other hand, there exists an $R>0$ such that for all $\|u\| _{D^{1,p}}\geq R$, we have

$$\frac{1}{2p} \Vert u \Vert _{D^{1,p}}^{p}\geq \frac{\lambda C}{k} \Vert u \Vert _{D^{1,p}}^{k}. $$

Hence, taking $\|u\|=r_{k}:= (\frac{p^{\ast}}{p2^{p+2}\beta _{k}^{p^{\ast}}} )^{\frac{1}{p^{\ast}-p}}$, since $\beta_{k}\to 0$, we get $r_{k}\to\infty$ and

$$ \begin{aligned} E_{\lambda}(u)&\geq\frac{1}{2p} \Vert u \Vert _{D^{1,p}}^{p}-\frac{\beta _{k}^{p^{\ast}}}{p^{\ast}} \Vert u \Vert ^{p^{\ast}} \\ &\geq\frac{1}{p2^{p+1}} \Vert u \Vert ^{p}-\frac{\beta_{k}^{p^{\ast }}}{p^{\ast}} \Vert u \Vert ^{p^{\ast}} \\ &=\frac{1}{p2^{p+2}}r_{k}^{p}\to\infty,\quad \mbox{as } k \to \infty. \end{aligned} $$

This concludes the proof of Theorem 1.2. □