Existence, nonexistence, and multiplicity of solutions for the fractional $p\&q$-Laplacian equation in $\mathbb{R}^{N}$

Abstract

In this paper, we study the existence, nonexistence, and multiplicity of solutions to the following fractional $p\&q$-Laplacian equation:

$$\begin{aligned} &(-\Delta)^{s}_{p}u + a(x)\vert u\vert ^{p-2}u +(-\Delta)^{s}_{q}u + b(x)\vert u\vert ^{q-2}u +\mu(x)\vert u\vert ^{r-2}u \\ &\quad= \lambda h(x)\vert u \vert ^{m-2}u,\quad x\in {\mathbb {R}}^{N}, \end{aligned}$$

(0.1)

where λ is a real parameter, $(-\Delta)_{p}^{s}$ and $(-\Delta )_{q}^{s}$ are the fractional $p\&q$-Laplacian operators with $0< s<1<q<p, r>1$ and $sp< N$, and the functions $a(x), b(x),\mu(x)$, and $h(x)$ are nonnegative in ${\mathbb {R}}^{N}$. Three cases on $p,q,r,m$ are considered: $p< m< r< p_{s}^{*}$, $\max\{p,r\}< m< p_{s}^{*}$, and $1< m< q< r< p_{s}^{*}$. Using variational methods, we prove the existence, nonexistence, and multiplicity of solutions to Eq. (0.1) depending on $\lambda, p,q,r,m$ and the integrability properties of the ratio $h^{r-p}/\mu^{m-p}$. Our results extend the previous work in Bartolo et al. (J. Math. Anal. Appl. 438:29-41, 2016) and Chaves et al. (Nonlinear Anal. 114:133-141, 2015) to the fractional $p\&q$-Laplacian equation (0.1).

1 Introduction and the main result

In this paper, we study the existence, nonexistence, and multiplicity of solutions to the following fractional $p\&q$-Laplacian equation:

$$\begin{aligned} (-\Delta)^{s}_{p}u + a(x)\vert u\vert ^{p-2}u +(-\Delta)^{s}_{q}u + b(x)\vert u\vert ^{q-2}u=f(x,u),\quad x\in {\mathbb {R}}^{N}, \end{aligned}$$

(1.1)

where $(-\Delta)_{p}^{s}$ and $(-\Delta)_{q}^{s}$ are the fractional $p\&q$-Laplacian operators with $0< s<1<q<p, r>1$ and $sp< N$. The nonlinearity $f(x,u)=\lambda h(x)\vert u\vert ^{m-2}u-\mu(x)\vert u\vert ^{r-2}u$ can be seen as a competitive interplay of two nonlinearities. The coefficients $a(x), b(x),\mu(x), h(x)$ are assumed to be positive in ${\mathbb {R}}^{N}$, and other exact assumptions will be given further.

The fractional t-Laplacian operator $(-\Delta)_{t}^{s}$ with $0< s<1<t$ and $st< N$ is defined along a function $\varphi\in C_{0}^{\infty}({\mathbb {R}}^{N})$ as

$$\begin{aligned} (-\Delta)_{t}^{s}\varphi(x)=2\lim _{\varepsilon\to 0^{+}} \int_{{\mathbb {R}}^{N}\setminus B_{\varepsilon}(x)} \frac{\vert \varphi(x)-\varphi(y)\vert ^{t-2}(\varphi (x)-\varphi(y))}{\vert x-y\vert ^{N+ts}}\,dy,\quad \forall x\in {\mathbb {R}}^{N}, \end{aligned}$$

(1.2)

where $B_{\varepsilon}(x):=\{y\in {\mathbb {R}}^{N}:\vert x-y\vert <\varepsilon\}$; see [3–6] and the references therein.

When $p=q$, Eq. (1.1) is reduced to the fractional p-Laplacian equation

$$\begin{aligned} (-\Delta)^{s}_{p}u+V(x)\vert u\vert ^{p-2}u=f(x,u),\quad x\in {\mathbb {R}}^{N}, \end{aligned}$$

(1.3)

and when $s=1$, Eq. (1.1) is the $p\&q$-Laplacian equation

$$\begin{aligned} -\Delta_{p}u+a(x)\vert u\vert ^{p-2}u- \Delta_{q}u+b(x)\vert u\vert ^{q-2}u=f(x,u),\quad x\in {\mathbb {R}}^{N}. \end{aligned}$$

(1.4)

Equation (1.4) comes from a general reaction-diffusion system

$$\begin{aligned} u_{t}=\operatorname{div}\bigl(D(u)\nabla u\bigr)+f(x,u),\quad x\in {\mathbb {R}}^{N}, t>0, \end{aligned}$$

(1.5)

where $D(u)=\vert \nabla u\vert ^{p-2}+\vert \nabla u\vert ^{q-2}$. This system has a wide range of applications in physics and related sciences such as biophysics, plasma physics, and chemical reaction design. In such applications, the function u describes a concentration, and the first term on the right-hand side of (1.5) corresponds to the diffusion with a diffusion coefficient $D(u)$, whereas the second one is the reaction and relates to sources and loss processes. Typically, in chemical and biological applications, the reaction term $f(x, u)$ is a polynomial of u with variable coefficients [7, 8].

The solution of (1.4) has been studied by many authors; for example, see [1, 2, 7, 9–13] and the references therein. In the literature cited, the authors always assume that the potentials $a(x), b(x)$ satisfy one of the following conditions:

(A₁):

$a(x),b(x)\in C(\mathbb{R}^{N})$ and $a(x),b(x)\ge c_{0}$ in ${\mathbb {R}}^{N}$ for some constant $c_{0}>0$. Furthermore, for each $d>0$, $\operatorname{meas}(\{x\in\mathbb{ R}^{N}: a(x), b(x)\le d\})<\infty$.

(A₂):

$\lim _{\vert x\vert \to\infty}a(x)=+\infty, \lim _{\vert x\vert \to\infty}b(x)=+\infty$.

(A₃):

$a(x),b(x)\ge c_{0}>0$ in ${\mathbb {R}}^{N}$, and $a(x)^{-1}, b(x)^{-1}\in L^{1}(\mathbb{R}^{N})$.

It is well known that one of assumptions (A₁), (A₂), and (A₃) guarantees that the embedding $W^{1,t}({\mathbb {R}}^{N})\hookrightarrow L^{r}({\mathbb {R}}^{N})$ is compact for each $t\le r< t^{*}=\frac{tN}{N-t}$ with $1< t< N$. As far as we know, there are few papers that deal with a general bounded potential case for problem (1.4).

Now let us recall some advances of our problem. Pucci and Rădulescu [14] first studied the nonnegative solutions of the equation

$$\begin{aligned} -\Delta_{p}u+\vert u\vert ^{p-2}u+h(x) \vert u\vert ^{r-2}u=\lambda \vert u\vert ^{m-2}u,\quad x\in {\mathbb {R}}^{N}, \end{aligned}$$

(1.6)

where $h(x)>0$ satisfies

$$\begin{aligned} \int_{{\mathbb {R}}^{N}}\bigl[h(x)\bigr]^{\frac{m}{m-r}}\,dx=H_{1}\in {\mathbb {R}}^{+}=(0,\infty), \end{aligned}$$

(1.7)

and $\lambda>0$, and $2\le p< m<\min\{r,p^{*}\}$ with $p^{*}=pN/(N-p)$ if $N>p$ and $p*=\infty$ if $N\le p$. They showed the nonexistence of nontrivial solutions to (1.6) if λ is small enough and the existence of at least two nontrivial solutions for (1.6) if λ is large enough.

Autuori and Pucci [15] generalized (1.6) to the quasilinear elliptic equation

$$\begin{aligned} -\operatorname{div} A(x,\nabla u)+ a(x)\vert u\vert ^{p-2}u+h(x) \vert u\vert ^{r-2}u=\lambda\omega(x)\vert u\vert ^{m-2}u,\quad x\in {\mathbb {R}}^{N}, \end{aligned}$$

(1.8)

where $A(x,\nabla u)$ acts like the p-Laplacian, $\max\{2,p\}< m<\min\{r,p^{*}\}$, and the coefficients ω and h are related by the integrability condition

$$\begin{aligned} \int_{{\mathbb {R}}^{N}} \biggl[\frac{\omega(x)^{r}}{h(x)^{m}} \biggr]^{\frac {1}{r-m}}\,dx=H_{2} \in {\mathbb {R}}^{+}. \end{aligned}$$

(1.9)

By imposing a strong convexity condition of the p-Laplacian type on the potential of A, the authors extend completely the result of [14]. Moreover, Autuori and Pucci [15] proposed two open questions: the deletion of the restriction $\max\{2,p\}< m$ and the replacement of (1.9) by the assumption that $\omega(\omega/h)^{(m-p)/(r-m)}$ is in $L^{N/p}({\mathbb {R}}^{N})$.

Later, Autuori and Pucci [16] studied the existence and multiplicity of solution to the following elliptic equation involving the fractional Laplacian:

$$\begin{aligned} (-\Delta)^{s}u+ a(x)u+h(x)\vert u\vert ^{r-2}u=\lambda\omega(x)\vert u\vert ^{m-2}u,\quad x\in {\mathbb {R}}^{N}, \end{aligned}$$

(1.10)

where $\lambda>0$, $0< s<1$, $2s< N$, $2< m<\min\{r,2_{s}^{*}\}$, $2_{s}^{*}=2N/(N-2s)$, and $(-\Delta)^{s}$ is the fractional Laplacian operator. The coefficients ω and h are related by condition (1.9). The authors proved the existence of entire solutions of (1.10) by using a direct variational method and the mountain pass theorem.

More recently, Xiang et al. [6] investigated the fractional p-Laplacian equation

$$\begin{aligned} (-\Delta)_{p}^{s}u+V(x)\vert u\vert ^{p-2}u+b(x)\vert u\vert ^{r-2}u=\lambda a(x)\vert u \vert ^{m-2}u,\quad x\in {\mathbb {R}}^{N}, \end{aligned}$$

(1.11)

where $\lambda>0$, $p< m<\min\{r,p_{s}^{*}\}$, $p_{s}^{*}=pN/(N-ps)$, and $a(x)$ and $b(x)$ are related by the condition $a(a/b)^{(r-p)/(m-r)}\in L^{N/ps}({\mathbb {R}}^{N})$.

Up to now, it is worth noting that there is much attention on equations like (1.6), (1.8), and (1.11) with $1< m< r$. From the papers mentioned, it is natural to ask whether the existence, nonexistence, and multiplicity of solutions to Eq. (1.1) is admitted if $1< r< m< p_{s}^{*}$ and $1< m< r< p_{s}^{*}$ ? Clearly, equations like (1.6), (1.8), and (1.11) are contained in (1.1).

In this paper, motivated by [5, 6], we will answer this interesting question, extend the $p\&q$-Laplacian (1.4), which has been studied deeply in [1, 2], to the fractional $p\&q$-Laplacian equation (1.1), and investigate the existence, nonexistence, and multiplicity of solutions depending on λ and according to the integrability properties of the ratio $h^{r-p}/\mu^{m-p}$.

For this purpose, we apply a version of symmetric mountain pass lemma in [17]. Also, we adapt some ideas developed by Pucci et al. [18] and Xiang et al. [6]. Note that although the idea was earlier used for other problems, the adaptation to the procedure to our problem is not trivial at all since the parameters $r,m$ satisfy $1< r< m$ and we must consider our problem for a suitable space, and so we need more delicate estimates and new technique. Our results, which are new even in the canonical case $p=q=2$, generalize the main results of [1, 2] in several directions. Furthermore, we weaken the conditions in those papers, and assumptions (A₁)-(A₃) are not necessary for our results.

In order to state our main theorems, we recall some fractional Sobolev spaces and norms. The fractional Sobolev space $W^{s,t}({\mathbb {R}}^{N})$ ($0< s<1<t$) with $st< N$ is defined by

$$\begin{aligned} W^{s,t}\bigl({\mathbb {R}}^{N}\bigr)= \biggl\{ u\in L^{t}\bigl({\mathbb {R}}^{N}\bigr): \frac{\vert u(x)-u(y)\vert }{\vert x-y\vert ^{\frac{N}{t}+s}} \in L^{t}\bigl({\mathbb {R}}^{2N}\bigr) \biggr\} . \end{aligned}$$

(1.12)

This space is endowed with the natural norm

$$\begin{aligned} \Vert u\Vert _{W^{s,t}}= \bigl([u]_{s,t}^{t}+ \Vert u\Vert _{t}^{t} \bigr)^{1/t}, \end{aligned}$$

(1.13)

whereas $[u]_{s,t}$ denotes the Gagliardo seminorm given by

$$\begin{aligned}{} [u]_{s,t}= \biggl( \int \int _{{\mathbb {R}}^{2N}} \frac {\vert u(x)-u(y)\vert ^{t}}{\vert x-y\vert ^{N+ts}}\,dx\,dy \biggr)^{1/t}. \end{aligned}$$

(1.14)

The spaces $X_{p}$ and $X_{q}$ denote the completion of $C_{0}^{\infty}({\mathbb {R}}^{N})$ with respect to the norms

$$\begin{aligned} \Vert u\Vert _{X_{p}}= \bigl([u]_{s,p}^{p}+ \Vert u\Vert _{p,a}^{p} \bigr)^{1/p},\qquad \Vert u \Vert _{X_{q}}= \bigl([u]_{s,q}^{q}+\Vert u\Vert _{q,b}^{q} \bigr)^{1/q}, \end{aligned}$$

(1.15)

respectively, in which the functions $a(x), b(x)$ satisfy

(H₀):

$a(x),b(x)\in C({\mathbb {R}}^{N})$ and $a(x),b(x)\ge c_{0}>0$ in ${\mathbb {R}}^{N}$ for some constant $c_{0}$.

In general, let $\Vert u\Vert _{t,\rho}=(\int_{{\mathbb {R}}^{N}}\rho \vert u\vert ^{t}\,dx)^{1/t}$ with $t\ge1$ and $\rho=\rho(x)\ge0, \neq0$ a.e. in ${\mathbb {R}}^{N}$. In particular, denote $\Vert u\Vert _{t}=(\int_{{\mathbb {R}}^{N}}\vert u\vert ^{t}\,dx)^{1/t}$ or $\Vert u\Vert _{L^{t}(\Omega)}=(\int_{\Omega} \vert u\vert ^{t}\,dx)^{1/t}$ with the domain $\Omega\subset {\mathbb {R}}^{N}$. Let $E=X_{p}\cap X_{q}$ with $1< q< p< N$. The norm of $u\in E$ is defined by

$$\begin{aligned} \Vert u\Vert _{E}=\Vert u\Vert _{X_{p}}+\Vert u\Vert _{X_{q}}. \end{aligned}$$

(1.16)

Lemma 1.1

[3, 19]

Let $0< s<1<t$ with $st< N$. In addition, assume (H₀). Then, $Y_{t}\equiv W^{s,t}({\mathbb {R}}^{N})$ is a uniformly convex Banach space, and there exists a positive constant $S_{0}=S_{0}(N,t,s)$ such that

$$\begin{aligned} \Vert u\Vert _{t^{*}_{s}}\le S_{0}[u]_{s,t},\quad \forall u\in Y_{t}, \end{aligned}$$

(1.17)

and

$$\begin{aligned} \Vert u\Vert _{r}\le S_{r}\Vert u \Vert _{Y_{t}}, \quad\forall u\in Y_{t}, \end{aligned}$$

(1.18)

where $t=p$ or q, $t_{s}^{*}=\frac{tN}{N-ts}$ is the fractional critical exponent, and $S_{r}$ is a constant depending on $s,r,t,N$. For convenience, we denote $S_{t_{s}^{*}}$ by $S_{0}$. Consequently, the space $Y_{t}$ is continuously embedded in $L^{r}({\mathbb {R}}^{N})$ for any $r\in [t,t_{s}^{*}]$. Moreover, the embedding $Y_{t}\hookrightarrow L^{r}({\mathbb {R}}^{N})$ is locally compact whenever $1< r< t_{s}^{*}$.

Clearly, from definitions (1.13) and (1.15) and assumption (H₀), we see that

$$\begin{aligned} \min\{1,c_{0}\}\Vert u\Vert _{Y_{t}}\le \Vert u\Vert _{X_{t}}, \quad\forall u\in Y_{t}, \mbox{where } t=p,q. \end{aligned}$$

(1.19)

Let $J(u):E\rightarrow\mathbb{R}$ be the energy functional associated to Eq. (1.1) defined by

$$\begin{aligned} J(u)=\frac{1}{p}\Vert u\Vert _{X_{p}}^{p}+ \frac{1}{q}\Vert u\Vert _{X_{q}}^{q}+ \frac{1}{r}\Vert u\Vert _{r,\mu}^{r}- \frac{\lambda}{m} \Vert u\Vert _{m,h}^{m},\quad \forall u\in E, \end{aligned}$$

(1.20)

where the norms $\Vert \cdot \Vert _{X_{p}}$ and $\Vert \cdot \Vert _{X_{p}}$ are defined by (1.15).

From the embedding inequalities (1.18) and assumptions (H₀)-(H₄) below, we see that the functional J is well defined and $J\in C^{1}(E,\mathbb{R})$ with

$$\begin{aligned} J'(u)\varphi={}& \int \int _{{\mathbb {R}}^{2N}} \biggl[\frac{\vert u(x) - u(y)\vert ^{p-2}}{\vert x-y\vert ^{N+ps}}+\frac{\vert u(x) - u(y)\vert ^{q-2}}{\vert x-y\vert ^{N+qs}} \biggr] \bigl(u(x) - u(y)\bigr) \bigl(\varphi(x) - \varphi(y)\bigr)\,dx\,dy \\ &{}+ \int_{\mathbb{R}^{N}} \bigl[a(x)\bigl\vert u(x)\bigr\vert ^{p-2}+b(x)\vert u\vert ^{q-2}+\mu (x)\vert u\vert ^{r-2}-\lambda h(x)\vert u\vert ^{m-2} \bigr] \\ &{}\times u(x) \varphi(x)\,dx,\quad \forall\varphi\in E. \end{aligned}$$

(1.21)

A function $u\in E$ is said to be a (weak) solution of Eq. (1.1) if $J'(u)\varphi=0$ for any $\varphi\in E$.

Throughout this paper, we let $0< s<1<q<p$ with $sp< N$. Our main results are as follows.

Theorem 1.2

Assume (H₀) and

(H₁):

$p< m< p_{s}^{*}$; $h(x)$ is a positive weight satisfying $h(x)\in L^{\gamma}({\mathbb {R}}^{N})$ with $\gamma=\frac{p_{s}^{*}}{p_{s}^{*}-m}$.

(H₂):

$p< m< r< p_{s}^{*}$; the functions $\mu(x)$ and $h(x)$ are positive and $\mu(x), h(x)\in L_{\mathrm{loc}}^{1}({\mathbb {R}}^{N})$. Furthermore, $h(x)$ and $\mu(x)$ are related by the condition

$$\begin{aligned} \int_{{\mathbb {R}}^{N}} \biggl[\frac{h(x)^{(r-p)/(r-m)}}{\mu (x)^{(m-p)/(r-m)}} \biggr]^{N/ps}\,dx=H_{1} \in {\mathbb {R}}^{+}. \end{aligned}$$

(1.22)

Then there exist constants $\lambda_{2}\ge\lambda_{1}>0$ such that Eq. (1.1) has

• (i) only the trivial weak solution if $\lambda<\lambda_{1}$;

• (ii) at least two nontrivial weak solutions if $\lambda\ge \lambda_{2}$.

Theorem 1.3

Let $\max\{p,r\}< m< p_{s}^{*}$. Assume that (H₀) and $(H_{1})$ hold. In addition, suppose that $\mu(x)$ are nonnegative and $\mu(x)\in L_{\mathrm{loc}}^{1}({\mathbb {R}}^{N})$. Then Eq. (1.1) admits

• (i) only the trivial solution if $\lambda\le0$;

• (ii) infinitely many weak solutions $u_{n}\in E$ such that $J(u_{n})\to\infty$ as $n\to\infty$ if $\lambda>0$.

Theorem 1.4

Let $0< s<1<m<q\le p<p_{s}^{*}$ and $q\le r< p_{s}^{*}$. Assume (H₀) and

(H₃):

$\mu(x)\ge0$ in ${\mathbb {R}}^{N}$ and $\mu(x)\in L^{\sigma }_{\mathrm{loc}}({\mathbb {R}}^{N})$ with $\sigma=\frac{p_{s}^{*}}{p_{s}^{*}-r}$;

(H₄):

$h(x)\in L^{\delta}(\mathbb{R}^{N})$ with $\delta =\frac {q}{q-m}$, and there exist $d_{0}>0$ and $x_{0}=(x^{0}_{1},x^{0}_{2},\ldots,x^{0}_{N})\in \mathbb{R}^{N}$ such that $h(x)>0$ in $B_{d_{0}}(x_{0})$, where $B_{d_{0}}(x_{0})=\{x\in\mathbb{R}^{N}: \vert x-x_{0}\vert < d_{0}\}$.

Then Eq. (1.1) with $\lambda>0$ admits infinitely many solutions $u_{n}\in E$ with $u_{n}\to0$ in E.

Remark 1.5

From Theorem 1.2, we know that it still remains an open problem to verify whether $\lambda_{1}=\lambda_{2}$. In addition, the nonlinear function $f(x,u)=\lambda h(x)\vert u\vert ^{m-2}u-\mu(x)\vert u\vert ^{r-2}u$ with $p< m< r$ fails to satisfy the Ambrosetti-Rabinowitz condition. Furthermore, for $s=1$ in (1.1), our results and context are more general than those in [1, 2].

The paper is organized as follows: In Section 2, we give some preliminaries, will set up the variational framework for problem (1.1), and prove that the functional associated to (1.1) satisfies the $(PS)_{c}$ condition. The proofs of Theorems 1.2 and 1.3 are given in Section 2. Finally, Section 3 is devoted to the proof of Theorem 1.4.

2 Preliminaries

To prove our main results, we need to establish some lemmas.

Lemma 2.1

Let (H₀) and one of assumptions (H₁) and (H₄) hold. Then, if $\{u_{n}\}$ is a bounded sequence in E, then there exists $u\in E\cap L^{m}({\mathbb {R}}^{N},h)$ such that, up to a subsequence, $u_{n}\to u$ strongly in $L^{m}({\mathbb {R}}^{N},h)$ as $n\to\infty$.

Proof

We first choose a constant $\beta>0$ such that $\Vert u_{n}\Vert _{E}\le\beta$ for all $n\ge1$. If (H₁) is satisfied, then for any $\varepsilon>0$, there exists $R_{0}>0$ such that

$$\begin{aligned} \biggl( \int_{B_{R}^{c}} \bigl\vert h(x)\bigr\vert ^{\gamma}\,dx \biggr)^{1/\gamma}< 2^{-m}\beta ^{-m}\varepsilon \quad\mbox{for all } R\ge R_{0}. \end{aligned}$$

(2.1)

Then, it follows from the Hölder inequality and Lemma 1.1 that, for $R\ge R_{0}$,

$$\begin{aligned} \int_{B_{R}^{c}} h(x)\bigl\vert u_{n}(x)-u(x)\bigr\vert ^{m}\,dx\le \Vert h\Vert _{L^{\gamma}(B_{R}^{c})}\Vert u_{n}-u\Vert ^{m}_{L^{p_{s}^{*}}(B_{R}^{c})} \le 2^{m} \beta^{m}\Vert h\Vert _{L^{\gamma}(B_{R}^{c})}< \varepsilon. \end{aligned}$$

(2.2)

By Lemma 1.1, up to a subsequence, we obtain $u_{n}\to u$ strongly in $L^{m}(B_{R_{0}})$ and $u_{n}(x)\to u(x)$ a.e. in $B_{R_{0}}$ as $n\to\infty$. Thus $h(x)\vert u_{n}(x)-u(x)\vert ^{m}\to0$ a.e. in $B_{R_{0}}$ as $n\to\infty$. Similarly, for each measurable subset $\Omega\subset B_{R_{0}}$, we have

$$\begin{aligned} \int_{\Omega}h(x)\bigl\vert u_{n}(x)-u(x)\bigr\vert ^{m}\,dx\le \Vert h\Vert _{L^{\gamma}(\Omega)}\Vert u_{n}-u \Vert ^{m}_{L^{p_{s}^{*}}(\Omega)} \le 2^{m}\beta^{m} \Vert h\Vert _{L^{\gamma}(\Omega)}. \end{aligned}$$

(2.3)

Since $h(x)\in L^{\gamma}({\mathbb {R}}^{N})$, we obtain that the sequence $\{h(x)\vert u_{n}(x)-u(x)\vert ^{m}\}$ is uniformly integrable and bounded in $L^{1}(B_{R_{0}})$. Furthermore, an application of the Vitali convergence theorem gives

$$\begin{aligned} \lim_{n\to\infty} \int_{B_{R_{0}}} h(x)\bigl\vert u_{n}(x)-u(x)\bigr\vert ^{m}\,dx=0. \end{aligned}$$

(2.4)

Then the conclusion that $u_{n}\to u$ strongly in $L^{m}({\mathbb {R}}^{N},h)$ follows from (2.2) and (2.4).

If (H₄) is satisfied, then for any $\varepsilon>0$, there exists $R_{0}>0$ such that

$$\begin{aligned} \Vert h\Vert _{L^{\delta}(B_{R}^{c})}= \biggl( \int_{B_{R}^{c}} \bigl\vert h(x)\bigr\vert ^{\delta}\,dx \biggr)^{1/\delta}< 2^{-m}\beta^{-m}\varepsilon \quad\mbox{for all } R\ge R_{0} \end{aligned}$$

(2.5)

and

$$\begin{aligned} \int_{B_{R}^{c}} h(x)\bigl\vert u_{n}(x)-u(x)\bigr\vert ^{m}\,dx\le \Vert h\Vert _{L^{\delta}(B_{R}^{c})}\Vert u_{n}-u\Vert ^{m}_{L^{q}(B_{R}^{c})} \le 2^{m} \beta^{m}\Vert h\Vert _{L^{\delta}(B_{R}^{c})}< \varepsilon. \end{aligned}$$

(2.6)

Similarly, we can derive (2.4). Then combining (2.4) with (2.6), we have $u_{n}\to u$ in $L^{m}({\mathbb {R}}^{N},h)$. □

Lemma 2.2

Let (H₀) and one of assumptions (H₁) and (H₄) hold. If $\{u_{n}\}$ is a bounded $(PS)_{c}$ sequence of the functional J defined by (1.20), then the functional J satisfies $(PS)_{c}$ condition.

Proof

Let $\{u_{n}\}$ be a $(PS)_{c}$ sequence, that is,

$$\begin{aligned} J(u_{n})\to c \quad\mbox{and}\quad \bigl\Vert J'(u_{n})\bigr\Vert _{E'}\to0 \quad\mbox{as } n \to\infty. \end{aligned}$$

(2.7)

Since the sequence $\{u_{n}\}$ is bounded in E, there exists a subsequence, still denoted by $\{u_{n}\}$, such that

$$ \begin{aligned} &u_{n}\rightharpoonup u \quad\mbox{weakly in } E,\qquad u_{n}(x)\to u(x) \quad\mbox{a.e. in } {\mathbb {R}}^{N},\\ & u_{n}\to u \quad\mbox{strongly in } L^{t}_{\mathrm{loc}}\bigl({\mathbb {R}}^{N} \bigr), \end{aligned} $$

(2.8)

where $t=p$ or q. We now prove that $u_{n}\to u$ in E. Let $\varphi \in E$ be fixed and denote by $T_{\varphi}$ the linear functional on E defined by

$$\begin{aligned} T_{\varphi}(v)=A_{\varphi}(v)+B_{\varphi}(v),\quad \forall\varphi\in E, \end{aligned}$$

(2.9)

where $A_{\varphi}(v)$ and $B_{\varphi}(v)$ are the linear functionals defined by

$$ \begin{aligned} &A_{\varphi}(v)= \int \int _{{\mathbb {R}}^{2N}} \frac{\vert \varphi (x)-\varphi(y)\vert ^{p-2}(\varphi(x)-\varphi(y))}{\vert x-y\vert ^{N+ps}}\bigl(v(x)-v(y)\bigr)\,dx\,dy, \quad\forall \varphi\in E, \\ &B_{\varphi}(v)= \int \int _{{\mathbb {R}}^{2N}} \frac{\vert \varphi (x)-\varphi(y)\vert ^{q-2}(\varphi(x)-\varphi(y))}{\vert x-y\vert ^{N+qs}}\bigl(v(x)-v(y)\bigr)\,dx\,dy,\quad \forall \varphi\in E, \end{aligned} $$

(2.10)

respectively. Clearly, by the Hölder inequality, $T_{\varphi}$ is also continuous, and

$$\begin{aligned} \bigl\vert T_{\varphi}(v)\bigr\vert &\le\bigl\vert A_{\varphi}(v)\bigr\vert + \bigl\vert B_{\varphi}(v)\bigr\vert \le \Vert \varphi \Vert _{X_{p}}^{p-1}\Vert v\Vert _{X_{p}} + \Vert \varphi \Vert _{X_{q}}^{q-1}\Vert v\Vert _{X_{q}} \\ &\le \bigl(\Vert \varphi \Vert _{E}^{p-1} + \Vert \varphi \Vert _{E}^{q-1}\bigr)\Vert v\Vert _{E}, \quad\forall v\in E. \end{aligned}$$

(2.11)

Furthermore, the fact that $u_{n}\rightharpoonup u$ weakly in E implies that $\lim _{n\to\infty}A_{u}(u_{n}-u)= \lim _{n\to\infty }B_{u}(u_{n}-u)=0$, and so

$$\begin{aligned} \lim_{n\to\infty}T_{u}(u_{n}-u)=0. \end{aligned}$$

(2.12)

On the other hand, as $n\to\infty$, we have

$$\begin{aligned} o_{n}(1)&=\bigl(J'(u_{n})-J'(u) \bigr) (u_{n}-u) \\ &=T_{u_{n}}(u_{n}-u)-T_{u}(u_{n}-u) +\Phi_{n}+\Psi_{n} -\lambda P_{n}+Z_{n}, \end{aligned}$$

(2.13)

where

$$\begin{aligned} &\Phi_{n}= \int_{{\mathbb {R}}^{N}} a(x) \bigl(\vert u_{n}\vert ^{p-2}u_{n} - \vert u\vert ^{p-2}u\bigr) (u_{n} - u)\,dx, \\ &\Psi_{n} = \int_{{\mathbb {R}}^{N}} b(x) \bigl(\vert u_{n}\vert ^{q-2}u_{n} - \vert u\vert ^{q-2}u\bigr) (u_{n} - u)\,dx,\\ &Z_{n}= \int_{{\mathbb {R}}^{N}} \mu(x) \bigl(\vert u_{n}\vert ^{r-2}u_{n} - \vert u\vert ^{r-2}u\bigr) (u_{n} - u)\,dx, \\ & P_{n} = \int_{{\mathbb {R}}^{N}} h(x) \bigl(\vert u_{n}\vert ^{m-2}u_{n} - \vert u\vert ^{m-2}u\bigr) (u_{n} - u)\,dx. \end{aligned}$$

(2.14)

From (2.13) and $Z_{n}\ge0$, we obtain, for large n,

$$\begin{aligned} T_{u_{n}}(u_{n}- u)-T_{u}(u_{n}-u)+ \Phi_{n}+\Psi_{n}\le\lambda P_{n}+o_{n}(1). \end{aligned}$$

(2.15)

Note that, by Lemma 2.1, $P_{n} \to0$ as $n\to\infty$.

Let us now recall the well-known vector inequalities: for all $\xi,\eta\in {\mathbb {R}}^{N}$,

$$ \begin{aligned} &\vert \xi-\eta \vert ^{p} \le c_{p}\bigl(\vert \xi \vert ^{p-2}\xi-\vert \eta \vert ^{p-2}\eta\bigr) (\xi-\eta) \quad\mbox{for } p\ge2,\quad \mbox{and}\\ &\vert \xi-\eta \vert ^{p}\le C_{p} \bigl[\bigl(\vert \xi \vert ^{p-2}\xi-\vert \eta \vert ^{p-2}\eta\bigr) (\xi- \eta) \bigr]^{p/2}\bigl(\vert \xi \vert ^{p}+\vert \eta \vert ^{p}\bigr)^{(2-p)/2} \quad\mbox{for } 1< p< 2, \end{aligned} $$

(2.16)

where $c_{p}$ and $C_{p}$ are positive constants depending only on p. Assume first that $p>q\ge2$. Then by (2.16) we have $\Vert u_{n}-u\Vert _{p,a}^{p}\le c_{p}\Phi_{n}$ and

$$\begin{aligned}{} [u_{n}-u]^{p}_{s,p}={}& \int \int _{{\mathbb {R}}^{2N}} \bigl\vert u_{n}(x)-u_{n}(y)-u(x)+u(y) \bigr\vert ^{p}\vert x-y\vert ^{-(N+sp)}\,dx\,dy \\ \le{}& c_{p} \int \int _{{\mathbb {R}}^{2N}} \bigl[\bigl\vert u_{n}(x)-u_{n}(y) \bigr\vert ^{p-2}\bigl(u_{n}(x)-u_{n}(y)\bigr)- \bigl\vert u(x)-u(y)\bigr\vert ^{p-2} \\ &{} \times \bigl(u(x)-u(y)\bigr) \bigr]\bigl(u_{n}(x)-u(x)-u_{n}(y)+u(y) \bigr)\vert x-y\vert ^{-(N+sp)}\,dx\,dy \\ ={}&c_{p} \bigl[A_{u_{n}}(u_{n}-u)-A_{u}(u_{n}-u) \bigr]. \end{aligned}$$

(2.17)

Similarly, we have $\Vert u_{n}-u\Vert _{q,b}^{q}\le c_{q} \Psi_{n}$ and

$$\begin{aligned}{} [u_{n}-u]^{q}_{s,q}={}& \int \int _{{\mathbb {R}}^{2N}} \bigl\vert u_{n}(x)-u_{n}(y)-u(x)+u(y) \bigr\vert ^{q}\vert x-y\vert ^{-(N+sq)}\,dx\,dy \\ \le{}& c_{q} \int \int _{{\mathbb {R}}^{2N}} \bigl[\bigl\vert u_{n}(x)-u_{n}(y) \bigr\vert ^{q-2}\bigl(u_{n}(x)-u_{n}(y)\bigr)- \bigl\vert u(x)-u(y)\bigr\vert ^{q-2} \\ &{} \times \bigl(u(x)-u(y)\bigr) \bigr]\bigl(u_{n}(x)-u(x)-u_{n}(y)+u(y) \bigr)\vert x-y\vert ^{-(N+sq)}\,dx\,dy \\ ={}&c_{q} \bigl[B_{u_{n}}(u_{n}-u)-B_{u}(u_{n}-u) \bigr]. \end{aligned}$$

(2.18)

Let $C_{0}=\min\{c_{p}^{-1},c_{q}^{-1}\}$. By (2.17) and (2.18) we see that

$$\begin{aligned} T_{u_{n}}(u_{n}-u)-T_{u}(u_{n}-u)&=A_{u_{n}} (u_{n}-u)-A_{u}(u_{n}-u)+B_{u_{n}}(u_{n}-u)-B_{u}(u_{n}-u) \\ &\ge C_{0} \bigl([u_{n}-u]_{s,p}^{p}+[u_{n}-u]_{s,q}^{q} \bigr). \end{aligned}$$

(2.19)

Then the application of (2.15) yields

$$\begin{aligned} C_{0}\bigl(\Vert u_{n}-u\Vert _{X_{p}}^{p}+\Vert u_{n}-u\Vert _{X_{q}}^{q}\bigr)\le\lambda P_{n}+o_{n}(1) \to 0 \quad\mbox{as } n\to\infty. \end{aligned}$$

(2.20)

In conclusion, $u_{n}\to u$ in E as $n\to\infty$.

Finally, it remains to consider the case $1< p<2$. By (2.8) there exists $\beta>0$ such that $\Vert u_{n}\Vert _{E}\le\beta$ for all $n\ge 1$. Now from (2.16) and the Hölder inequality it follows that

$$\begin{aligned}{} [u_{n}-u]^{p}_{s,p}& \le C_{p}\bigl[A_{u_{n}}(u_{n}-u)-A_{u}(u_{n}-u) \bigr]^{p/2}\bigl([u_{n}]_{s,p}^{p}+[u]_{s,p}^{p} \bigr)^{(2-p)/2} \\ &\le C_{p}\bigl[A_{u_{n}}(u_{n}-u)-A_{u}(u_{n}-u) \bigr]^{p/2}\bigl([u_{n}]_{s,p}^{p(2-p)/2}+[u]_{s,p}^{p(2-p)/2} \bigr) \\ &\le D_{p}\bigl[A_{u_{n}}(u_{n}-u)-A_{u}(u_{n}-u) \bigr]^{p/2} \end{aligned}$$

(2.21)

and

$$\begin{aligned} \Vert u_{n}-u\Vert _{p,a}^{p} \le D_{p}\Phi_{n}^{p/2}, \end{aligned}$$

(2.22)

where we have applied the inequality

$$\begin{aligned} (x+y)^{(2-p)/2}\le x^{(2-p)/2}+y^{(2-p)/2} \quad\mbox{for all } x,y\ge0 \mbox{ and } 1< p< 2, \end{aligned}$$

(2.23)

and $D_{p}=2C_{p}\beta^{p(2-p)/2}$. Similarly, for $1< q<2$, we have

$$\begin{aligned}{} [u_{n}-u]^{q}_{s,q}\le D_{q}\bigl[B_{u_{n}}(u_{n}-u)-B_{u}(u_{n}-u) \bigr]^{q/2}, \qquad \Vert u_{n}-u\Vert _{q,b}^{q} \le D_{q}\Psi_{n}^{q/2} \end{aligned}$$

(2.24)

with $D_{q}=2C_{q}\beta^{q(2-q)/2}$. Then, by (2.21), (2.22), and (2.24) we get

$$\begin{aligned} &T_{u_{n}}(u_{n} - u) - T_{u}(u_{n} - u) + \Phi_{n} + \Psi_{n} \\ &\quad\ge C_{1} \bigl([u_{n} - u]^{2}_{s,q} + [u_{n} - u]_{s,p}^{2} + \Vert u_{n} - u\Vert _{p,a}^{2} + \Vert u_{n} - u\Vert _{q,b}^{2}\bigr) \end{aligned}$$

(2.25)

with some $C_{1}>0$. Then (2.15) and (2.25) imply that $u_{n}\to u$ in E as $n\to\infty$. Therefore, J satisfies the $(PS)_{c}$ condition, and we complete the proof of Lemma 2.2. □

Lemma 2.3

Under the assumptions of Theorem  1.2, suppose that $u\in E$ is a nontrivial weak solution of (1.1). Then there exists $\lambda_{1}>0$ such that $\lambda\ge\lambda_{1}$.

Proof

Since $u\in E$ is a nontrivial weak solution of (1.1), we have $J'(u)\varphi=0$ for all $\varphi\in E$. In particular, choosing $\varphi=u$, we have

$$\begin{aligned} \Vert u\Vert _{X_{p}}^{p}+\Vert u\Vert _{X_{q}}^{q}+\Vert u\Vert _{r,\mu}^{r} = \lambda \Vert u\Vert _{m,h}^{m}. \end{aligned}$$

(2.26)

By the Young inequality with $\epsilon>0$ we see that

$$\begin{aligned} cd\le\epsilon\theta^{-1} c^{p}+ \tau^{-1}\epsilon^{1/(1-\theta)}d^{\tau}, \quad\tau^{-1}+ \theta^{-1}=1, \theta>1. \end{aligned}$$

(2.27)

Taking $0<\alpha<\beta, c=k_{1}>0, d=t^{\alpha}, \tau=\frac{\beta}{\alpha}$, $\epsilon=(k_{2}\beta/\alpha)^{-\alpha/(\beta-\alpha)}$, $k_{2}>0$, it follows from (2.27) that

$$\begin{aligned} k_{1}t^{\alpha}-k_{2}t^{\beta} \le k_{0}k_{1}(k_{1}/k_{2})^{\alpha/(\beta-\alpha)},\quad \forall t\ge0, \end{aligned}$$

(2.28)

with $k_{0}=(1-\alpha/\beta)(\beta/\alpha)^{-\alpha/(\beta-\alpha)}<1$. Furthermore, let $k_{1}=\lambda h(x), k_{2}=\frac{1}{2}\mu(x)$, $\alpha=m-p$, and $\beta=r-p$. Then from (2.28) we obtain

$$\begin{aligned} \lambda h(x)\vert u\vert ^{m-p}-\frac{1}{2} \mu(x)\vert u\vert ^{r-p}\le c_{1}\lambda^{\frac{r-p}{r-m}}g(x),\quad \forall (x,u)\in {\mathbb {R}}^{N}\times {\mathbb {R}}, \end{aligned}$$

(2.29)

where $c_{1}=2^{(m-p)/(r-m)}$ and $g(x)=[h(x)^{r-p}/\mu(x)^{m-p}]^{\frac{1}{r-m}}$. By (H₂) we know $g(x)\in L^{\frac{N}{sp}}({\mathbb {R}}^{N})$. So, the application of (1.17) and (2.29) yields

$$\begin{aligned} \lambda \int_{{\mathbb {R}}^{N}} h(x)\vert u\vert ^{m}\,dx- \frac{1}{2} \int_{{\mathbb {R}}^{N}} \mu(x)\vert u\vert ^{r}\,dx \le& c_{1}\lambda^{\frac{r-p}{r-m}} \int_{{\mathbb {R}}^{N}} g(x)\vert u\vert ^{p}\,dx \\ \le& c_{1}GS_{0}^{p}\lambda^{\frac{r-p}{r-m}}[u]_{s,p}^{p} \end{aligned}$$

(2.30)

with $G=\Vert g\Vert _{L^{\frac{N}{sp}}({\mathbb {R}}^{N})}$. Then, from (2.26) and (2.30) we see that

$$\begin{aligned}{} [u]_{s,p}^{p}\le c_{1}GS_{0}^{p} \lambda^{\frac{r-p}{r-m}}[u]_{s,p}^{p}. \end{aligned}$$

(2.31)

This implies that $\lambda\ge\lambda_{1}\equiv (c_{1}^{-1}S_{0}^{-p}G^{-1})^{(r-m)/(r-p)}$ and completes the proof of Lemma 2.3. □

Lemma 2.4

Under the assumptions of Theorem  1.2, the functional J is coercive in E.

Proof

Letting $k_{1}=\frac{\lambda}{m}h(x), k_{2}=\frac{1}{2r}\mu(x)$, $\alpha=m-p, \beta=r-p$, and $t=\vert u(x)\vert$ in (2.28), we conclude that

$$\begin{aligned} f(x,u):=\frac{\lambda}{m}h(x)\vert u\vert ^{m}- \frac{1}{2r}\mu(x)\vert u\vert ^{r}\le c_{2}g(x) \vert u\vert ^{p},\quad \forall(x,u)\in {\mathbb {R}}^{N}\times {\mathbb {R}}, \end{aligned}$$

(2.32)

where $c_{2}=(2r)^{\frac{m-p}{r-m}}m^{\frac{p-r}{r-m}}\lambda^{\frac{r-p}{r-m}}$ and $g(x)=[h(x)^{r-p}/\mu(x)^{m-p}]^{\frac{1}{r-m}}$. Since $g(x)\in L^{\frac{N}{ps}}({\mathbb {R}}^{N})$, for any small $\varepsilon>0$, there exists $R_{1}>0$ such that

$$\begin{aligned} c_{2} \biggl( \int_{B_{R_{1}}^{c}} \bigl\vert g(x)\bigr\vert ^{N/ps}\,dx \biggr)^{ps/N}\le \varepsilon \end{aligned}$$

(2.33)

and

$$\begin{aligned} c_{2} \int_{B_{R_{1}}^{c}} \bigl\vert g(x)\bigr\vert \vert u\vert ^{p}\,dx\le c_{2} \biggl( \int_{B_{R_{1}}^{c}} \bigl\vert g(x)\bigr\vert ^{N/ps}\,dx \biggr)^{ps/N}\Vert u\Vert ^{p}_{L^{p_{s}^{*}}({\mathbb {R}}^{N})} \le \varepsilon S_{0}^{p}[u]_{s,p}^{p}, \end{aligned}$$

(2.34)

where $S_{0}$ is the embedding constant in (1.17). So, it follows from (2.32)-(2.34) that

$$\begin{aligned} J(u)&=\frac{1}{p}\Vert u\Vert _{X_{p}}^{p}+\frac{1}{q}\Vert u\Vert _{X_{q}}^{q}+ \frac{1}{r}\Vert u\Vert _{r,\mu}^{r}-\frac{\lambda}{m} \Vert u\Vert _{m,h}^{m} \\ &\ge\frac{1}{p}\Vert u\Vert _{X_{p}}^{p}+ \frac{1}{q}\Vert u\Vert _{X_{q}}^{q} - \int_{B_{R}} f(x,u)\,dx- c_{2} \int_{B_{R}^{c}} g\vert u\vert ^{p}\,dx. \end{aligned}$$

(2.35)

For fixed $R_{1}>0$ and for any $\tau>0$ and $\omega>0$, we decompose $B_{R_{1}}=X\cup Y\cup Z$ as follows:

$$ \begin{aligned} &X=\bigl\{ x\in B_{R_{1}}: 0\le h(x)< \omega\mbox{ and } \mu(x)>\tau\bigr\} ,\qquad Z=\bigl\{ x\in B_{R_{1}}: h(x) \ge\omega\bigr\} , \\ &Y=\bigl\{ x\in B_{R_{1}}: 0\le h(x)< \omega\mbox{ and } 0\le \mu(x)< \tau \bigr\} . \end{aligned} $$

(2.36)

Obviously, the sets $X, Y$, and Z are Lebesgue measurable. Note that the assumption $h(x),\mu(x)\in L_{\mathrm{loc}}^{1}({\mathbb {R}}^{N})$ implies that $\operatorname{meas}(Y)\to0$ as $\tau\to0$ and $\operatorname{meas}(Z)\to0$ as $\omega\to\infty$.

On the other hand, letting $k_{1}= \frac{\lambda}{m}h(x), k_{2}=\frac{1}{2r}\mu(x), t=\vert u(x)\vert , \alpha=m$, and $\beta=r$ in (2.28), we derive

$$\begin{aligned} f(x,u):=\frac{\lambda}{m}h(x)\vert u\vert ^{m}- \frac{1}{2r}\mu(x)\vert u\vert ^{r}\le c_{3}g_{1}(x) \end{aligned}$$

(2.37)

with $c_{3}=(2r)^{m/r}(\lambda/m)^{1+m/r}, g_{1}(x)=[h(x)/\mu(x)]^{m/r}$. Then,

$$\begin{aligned} \int_{X}f(x,u)\,dx\le c_{3} \int_{X}g_{1}(x)\,dx\le C_{1}, \end{aligned}$$

(2.38)

where $C_{1}=C_{1}(\omega,\tau,R)>0$ is a constant. Furthermore, it follows from (2.32) and (2.34) that

$$\begin{aligned} \int_{Y\cup Z}f(x,u)\,dx\le c_{2} \int_{Y\cup Z}g(x)\vert u\vert ^{p}\,dx\le c_{2} \biggl( \int_{Y\cup Z}\vert g\vert ^{N/ps}\,dx \biggr)^{ps/N}\Vert u\Vert ^{p}_{L^{p_{s}^{*}}(B_{R_{1}})}. \end{aligned}$$

(2.39)

For any $\varepsilon>0$, we can choose large $\omega>0$ and small $\tau>0$ such that $\operatorname{meas}(Y\cup Z)$ is so small that

$$\begin{aligned} c_{2} \biggl( \int_{Y\cup Z}\vert g\vert ^{N/ps}\,dx \biggr)^{ps/N}\le\varepsilon. \end{aligned}$$

(2.40)

From (1.13) and (2.38)-(2.40) we obtain

$$\begin{aligned} \int_{B_{R}}f(x,u)\,dx\le C_{1}+\varepsilon \Vert u \Vert ^{p}_{L^{p_{s}^{*}}(B_{R_{1}})}\le C_{1}+\varepsilon S_{0}^{p}[u]_{s,p}^{p}. \end{aligned}$$

(2.41)

Thus, combining (2.34) and (2.35) with (2.41) yields

$$\begin{aligned} J(u)\ge\frac{1}{p}\Vert u\Vert _{X_{p}}^{p}+ \frac{1}{q}\Vert u\Vert _{X_{q}}^{q} -2\varepsilon S_{0}^{p}[u]_{s,p}^{p}-C_{1} \ge \frac{1}{2p}\Vert u\Vert _{X_{p}}^{p}+ \frac{1}{q}\Vert u\Vert _{X_{q}}^{q}-C_{1}, \end{aligned}$$

(2.42)

where $0<2\varepsilon S_{0}^{p}\le1/2p$. Hence, J is coercive in E. □

Lemma 2.5

Under the assumptions of Theorem  1.2, there exists $u\in E$ such that $d=J(u)=\inf_{v\in E}J(v)$ and u is a weak solution of (1.1).

Proof

By Lemma 2.4 we see that $d>-\infty$. Let $\{u_{n}\}$ be a minimizing sequence for d in E, which is bounded in E by Lemma 2.4. Without loss of generality, we may assume that $\{u_{n}\}$ is nonnegative, converges to weakly to some u in E, and $u_{n}(x)\to u(x)$ a.e. in ${\mathbb {R}}^{N}$. Moreover, by the weak lower semicontinuity of the norms we have

$$\begin{aligned} \frac{1}{p}\Vert u\Vert _{X_{p}}^{p}+ \frac{1}{q}\Vert u\Vert _{X_{q}}^{q}+ \frac{1}{r}\Vert u\Vert _{r,\mu}^{r} \le \liminf _{n\to\infty} \biggl[\frac{1}{p}\Vert u_{n}\Vert _{X_{p}}^{p}+\frac{1}{q}\Vert u_{n}\Vert _{X_{q}}^{q} +\frac{1}{r}\Vert u_{n}\Vert _{r,\mu}^{r} \biggr]. \end{aligned}$$

(2.43)

Then from Lemma 2.1 and (2.43) it follows

$$\begin{aligned} J(u)\le\liminf_{n\to\infty}J(u_{n})=d. \end{aligned}$$

(2.44)

On the other hand, since $u\in E$, we have that $J(u)\ge d$, which shows that $J(u)=d$. Therefore, u is a global minimum for J, and hence it is a critical point, namely a weak solution of (1.1). □

Lemma 2.6

Under the assumptions of Theorem  1.2, there exists $\lambda_{2}>0$ such that for all $\lambda>\lambda_{2}$, Eq. (1.1) admits a global nontrivial minimum $u_{0}\in E$ of J with $J(u_{0})<0$.

Proof

Clearly, $J(0)=0$. Consider the constrained minimization problem

$$\begin{aligned} \lambda_{2}=\inf \biggl\{ \frac{1}{p}\Vert u \Vert _{X_{p}}^{p}+\frac{1}{q}\Vert u\Vert _{X_{q}}^{q}+\frac{1}{r}\Vert u\Vert _{r,\mu}^{r} : u\in E \mbox{ and } \Vert u\Vert _{m,h}^{m}=m \biggr\} . \end{aligned}$$

(2.45)

Let $u_{n}$ be a minimizing sequence of (2.45), which is clearly bounded in E, so that we can assume, without loss of generality, that it converges weakly to some $u_{0}\in E$ with $\Vert u_{0}\Vert _{m,h}^{m}=m$ and

$$\begin{aligned} \lambda_{2}=\frac{1}{p}\Vert u_{0} \Vert _{X_{p}}^{p}+\frac{1}{q}\Vert u_{0} \Vert _{X_{q}}^{q}+\frac {1}{r}\Vert u_{0} \Vert _{r,\mu}^{r}>0. \end{aligned}$$

(2.46)

Thus, $J(u_{0})=\lambda_{2}-\lambda<0$ for any $\lambda>\lambda_{2}$, and

$$\begin{aligned} d=J(u_{0})=\inf_{u\in E}J(u)< 0 \quad\mbox{for all } \lambda>\lambda_{2}. \end{aligned}$$

(2.47)

This completes the proof. □

Next, we show that if $\lambda>\lambda_{2}$, then problem (1.1) admits a second nontrivial weak solution $e\neq u_{0}$ by the mountain pass theorem.

Lemma 2.7

Suppose that assumptions (H₀)-(H₁) are satisfied. Then, for all $e\in E$ and $\lambda>0$, there exist $\alpha>0$ and $\rho\in (0,\Vert e\Vert _{E})$ such that $J(u)\ge\alpha$ for all $u\in E$ with $\Vert u\Vert _{E}=\rho$.

Proof

Let $u\in E$. From (H₁), (1.18), and (1.19) with $t=p$ we obtain

$$\begin{aligned} \int_{{\mathbb {R}}^{N}} h(x)\vert u\vert ^{m}\,dx\le \Vert h \Vert _{\gamma} \Vert u\Vert _{p_{s}^{*}}^{m}\le S_{0}^{m}\Vert h\Vert _{\gamma} \Vert u\Vert _{E}^{m}. \end{aligned}$$

(2.48)

Then,

$$\begin{aligned} J(u)\ge p^{-1}\Vert u\Vert _{X_{p}}^{p}+q^{-1} \Vert u\Vert _{X_{q}}^{q}-\lambda S_{0}^{m}H \Vert u\Vert _{E}^{m}\ge p^{-1}\Vert u \Vert _{E}^{p}-\lambda S_{0}^{m}H \Vert u\Vert _{E}^{m}, \end{aligned}$$

(2.49)

where $H=\Vert h\Vert _{\gamma}, \Vert u\Vert _{E}=\rho$, and

$$\begin{aligned} 0< \rho< \min \bigl\{ 1, \Vert e\Vert _{E}, \bigl( \lambda pS_{0}^{m}H\bigr)^{\frac{1}{p-m}} \bigr\} , \end{aligned}$$

(2.50)

so that

$$\begin{aligned} J(u)\ge\rho^{p} \bigl(p^{-1}-\lambda S_{0}^{m}H\rho^{m-p}\bigr)\equiv\alpha>0. \end{aligned}$$

(2.51)

Thus, we finish the proof of Lemma 2.7. □

Lemma 2.8

Under the assumptions of Theorem  1.2 and $\lambda>\lambda_{2}$, Eq. (1.1) admits a nontrivial weak solution $u\in E$ such that $J(u)>0$.

Proof

By Lemma 2.6, for all $\lambda>\lambda_{2}$, there exists a nontrivial weak solution $u_{0}\in E$ with $J(u_{0})<0$. Taking $e=u_{0}$ in Lemma 2.7, we get that J satisfies the geometrical structure of Theorem A.3 of [15]. Thus, for all $\lambda>\lambda_{2}$ there exists a sequence $\{u_{n}\}\subset E$ such that

$$\begin{aligned} J(u_{n})\to c>0\quad \mbox{and}\quad \bigl\Vert J'(u_{n})\bigr\Vert _{E'}\to0 \quad\mbox{as } n \to\infty, \end{aligned}$$

(2.52)

where

$$\begin{aligned} c=\inf_{\gamma\in\Gamma}\max_{0\le t\le1}J \bigl(\gamma(t)\bigr) \quad\mbox{with } \Gamma=\bigl\{ \gamma\in C\bigl([0,1]; E\bigr): \gamma(0)=0, \gamma(1)=u_{0}\bigr\} . \end{aligned}$$

(2.53)

Since J is coercive in E, the sequence $\{u_{n}\}$ is bounded in E. By Lemma 2.2 there exists a subsequence, still denoted by $\{u_{n}\}$, such that $u_{n}\to u$ in E as $n\to\infty$. Therefore, $J(u)=\lim_{n\to\infty}J(u_{n})=c>0$, and $J'(u)\varphi=\lim_{n\to\infty}J'(u_{n})\varphi=0$ for all $\varphi \in E$. So, u is a weak solution of (1.1) with $J(u)>0$. □

Proof of Theorem 1.2

The application of Lemma 2.2 shows that problem (1.1) has only a trivial solution if $\lambda<\lambda_{1}$. By Lemmas 2.6 and 2.8 it follows that, for all $\lambda>\lambda_{2}$, problem (1.1) admits at least two nontrivial weak solutions in E, one with negative energy and the other with positive energy. This completes the proof of Theorem 1.2. □

Proof of Theorem 1.3

We first prove, under the assumptions in Theorem 1.3, that any $(PS)_{c}$ sequence $\{u_{n}\}$ is bounded in E. Let the sequence $\{u_{n}\}$ satisfy (2.7). Then, for large n, we have

$$\begin{aligned} c+1+\Vert u_{n}\Vert _{E} \ge& J(u_{n}) - \frac{1}{m}J'(u_{n})u_{n} \\ =& \biggl(\frac{1}{p}-\frac{1}{m} \biggr)\Vert u_{n} \Vert _{X_{p}}^{p}+ \biggl(\frac{1}{q}- \frac{1}{m} \biggr)\Vert u_{n}\Vert _{X_{q}}^{q} + \biggl(\frac{1}{r}-\frac{1}{m} \biggr)\Vert u_{n} \Vert _{r,\mu}^{r}. \end{aligned}$$

(2.54)

Since $m>\max\{p,r\}$, it follows from (1.19) that $\{\Vert u_{n}\Vert _{E}\}$ is bounded. Furthermore, by Lemma 2.2 there exists a subsequence of $\{u_{n}\}$, still denoted by $\{u_{n}\}$, and $u\in E$ such that $u_{n}\to u$ in E and J satisfies the $(PS)_{c}$ condition.

From (2.26) it follows that if $u\in E$ is a nontrivial solution, then $\lambda>0$. This proves part (i). In the following, we prove part (ii). We now verify the conditions in Theorem 6.5 in [17]. Clearly, the functional J defined by (1.20) is even, and $J(0)=0$. By Lemma 2.7 there exist $\alpha,\rho>0$ such that $J(u)\ge\alpha$ for all $u\in E$ with $\Vert u\Vert _{E}=\rho$.

On the other hand, for any finite-dimensional subspace $E_{0}\subset E$, it is well known that any norms in $E_{0}$ are equivalent. So, there exist $d_{1},d_{2}>0$ such that

$$\begin{aligned} d_{1}\Vert u\Vert _{E}\le \Vert u\Vert _{r,\mu}\le d_{2}\Vert u\Vert _{E},\qquad d_{1}\Vert u\Vert _{E}\le \Vert u\Vert _{m,h}\le d_{2}\Vert u\Vert _{E},\quad \forall u \in E_{0}. \end{aligned}$$

(2.55)

Then, from (1.20) we have

$$\begin{aligned} J(u)\le \frac{1}{q}\bigl(\Vert u\Vert _{E}^{p}+\Vert u\Vert _{E}^{q} \bigr)+\frac{1}{r}d_{2}^{r}\Vert u\Vert _{E}^{r}-\frac {\lambda }{m}d_{1}^{m} \Vert u\Vert _{E}^{m},\quad \forall u\in E_{0}. \end{aligned}$$

(2.56)

Since $\lambda>0$ and $m>\max\{p,r\}$, there exists $R=R(E_{0})>\rho$ such that $J(u)<0$ for $u\in E_{0}$ and $\Vert u\Vert _{E}\ge R$. Therefore, all conditions are verified. Then an application of Theorem 6.5 in [17] shows that Eq. (1.1) admits infinitely many solutions $u_{n}\in E$ with $J(u_{n})\to\infty$ as $n\to\infty$. This completes the proof of Theorem 1.3. □

3 Proof of Theorem 1.4

In this section, we give a proof of Theorem 1.4. The main tool for this purpose is the following symmetric mountain pass lemma. First, we introduce the concept of genus.

Definition 3.1

[17]

Let E be a Banach space, and A a subset of E. The set A is said to be symmetric if $u\in E$ implies $-u\in E$. For a closed symmetric set A that does not contain the origin, we define the genus $\gamma(A)$ of A as the smallest integer k such that there exists an odd continuous mapping from A to $\mathbb{R}^{k}\setminus\{0\}$. If such k does not exist, then we define $\gamma(A)=\infty$. We set $\gamma(\emptyset)=0$. Let $\Gamma_{k}$ denote the family of closed symmetric subsets A of E such that $0\notin A$ and $\gamma(A)\ge k$.

Lemma 3.1

[20] (Symmetric mountain pass lemma)

Let E be an infinite-dimensional Banach space and $J\in C^{1}(E,\mathbb{R})$ such that:

• (I) J is even and bounded from below, $J(0)=0$, and J verifies the $(PS)_{c}$ condition.

• (II) for each $k\in\mathbb{N}$, there exists $A_{k}\in\Gamma_{k}$ such that $\sup _{u\in A_{k}}J(u)<0$.

Then one of the following two results holds:

• (1) there exists a sequence $\{u_{k}\}$ such that $J'(u_{k})=0, J(u_{k})<0$, and $\{u_{k}\}$ converges to zero.

• (2) there exist two sequences $\{u_{k}\}$ and $\{v_{k}\}$ such that $J'(u_{k})=0, J(u_{k})=0, u_{k}\neq0, \lim_{k\to\infty}u_{k}=0, J'(v_{k})=0, J(v_{k})<0, \lim_{k\to\infty}J(v_{k})=0$, and $\{v_{k}\}$ converges to a nonzero limit.

We now establish the following:

Lemma 3.2

Let the assumptions in Theorem  1.4 be satisfied. Then, for each $k\in\mathbb{N}$, there exists $A_{k}\in\Gamma_{k}$ such that

$$\begin{aligned} \sup_{u\in A_{k}}J(u)< 0. \end{aligned}$$

(3.1)

Proof

We use the following geometric construction introduced by Kajikiya [20]. Let $d_{0}$ and $x_{0}=(x_{1}^{0},x_{2}^{0},\ldots,x_{N}^{0})$ be fixed by assumption (H₄) and consider the cube

$$\begin{aligned} D(d) = \bigl\{ (x_{1},x_{2}, \ldots,x_{N})\in\mathbb{R}^{N} : \bigl\vert x_{i}-x_{i}^{0}\bigr\vert < d, 1\le i\le N\bigr\} . \end{aligned}$$

(3.2)

We choose small $d>0$ such that the cube $D(d)\subset\Omega_{0}:= B_{d_{0}}(x_{0})$. Note that $h(x)>0$ in $D(d)$. Fix $k\in\mathbb{N}$ arbitrarily. Let $n\in\mathbb{N}$ be the smallest integer such that $n^{N}\ge k$. We divide $D(d)$ equally into $n^{N}$ small cubes, denoted $D_{i}$, $1\le i\le n^{N}$, by planes parallel to each face of $D(d)$. The edge of $D_{i}$ has the length of $z=\frac{2d}{n}$. We construct new cubes $E_{i}$ in $D_{i}$ such that $E_{i}$ has the same center as that of $D_{i}$. The faces of $E_{i}$ and $D_{i}$ are parallel, and the edge of $E_{i}$ has the length $\frac{d}{n}$. Then, let the functions $\psi_{i}(x)\in C^{1}(\mathbb{R}^{N}), 1\le i\le k$, be such that

$$ \begin{aligned} &\operatorname{supp}(\psi_{i})\subset D_{i},\ \operatorname{supp}( \psi_{i})\cap \operatorname{supp}(\psi_{j})=\emptyset\ (i\neq j),\\ &\psi_{i}(x)=1,\ x\in E_{i},\ 0\le\psi_{i}(x)\le1,\ x \in \mathbb{R}^{N}. \end{aligned} $$

(3.3)

Denote

$$\begin{aligned} V_{k}= \Bigl\{ (t_{1},t_{2}, \ldots,t_{k})\in\mathbb{R}^{k} : \max_{1\le i\le k} \vert t_{i}\vert =1 \Bigr\} \end{aligned}$$

(3.4)

and

$$\begin{aligned} W_{k}= \Biggl\{ \sum_{i=1}^{k}t_{i} \psi_{i}(x): (t_{1},t_{2},\ldots,t_{k}) \in V_{k} \Biggr\} \subset E. \end{aligned}$$

(3.5)

Clearly, $V_{k}$ is the surface of k-dimensional, cube and $W_{k}$ is a closed symmetric set in E such that $0\notin W_{k}$. It is easy to see that $V_{k}$ is homeomorphic to the sphere $S^{k-1}$ by an odd mapping (take, e.g., the radial projection $V_{k}\to S^{k-1}$). Hence, $\gamma(V_{k})=k$. Moreover, $\gamma(W_{k})=\gamma(V_{k})=k$ because the mapping $(t_{1},t_{2},\ldots,t_{k})\longmapsto\sum_{i=1}^{k}t_{i}\psi_{i}(x)$ is homeomorphic and odd. On the other hand, since $W_{k}$ is bounded in E, there is a constant $\alpha_{k}>0$ such that

$$\begin{aligned} \Vert u\Vert _{E}\le\alpha_{k},\quad \forall u\in W_{k}. \end{aligned}$$

(3.6)

Let $z>0$ and $u=\sum_{i=1}^{k}t_{i}\psi_{i}(x)\in W_{k}$. Then,

$$\begin{aligned} J(zu)&=\frac{z^{p}}{p}\Vert u\Vert _{X_{p}}^{p}+\frac{z^{q}}{q}\Vert u\Vert _{X_{q}}^{q}+\frac {z^{r}}{r}\Vert u\Vert _{r,\mu}^{r}-\frac{1}{m} \int_{\mathbb{R}^{N}} h\vert zu\vert ^{m}\,dx \\ &\le \frac{z^{p}}{p}\alpha_{k}^{p}+\frac{z^{q}}{q} \alpha_{k}^{q}+\frac {z^{r}}{r}\alpha _{k}^{r}S_{r}^{r} \Vert \mu \Vert _{L^{\sigma}(\Omega_{0})} -\frac{1}{m}\sum _{i=1}^{k} \int_{D_{i}} h\vert zt_{i}\psi_{i}\vert ^{m}\,dx, \end{aligned}$$

(3.7)

where $S_{r}$ is the embedding constant in (1.18), and $\sigma =p_{s}^{*}/(p_{s}^{*}-r)$. By (3.7) there exists an integer $j\in[1,k]$ such that $\vert t_{j}\vert =1$ and $\vert t_{i}\vert \le1$ for $i\neq j$. Hence,

$$\begin{aligned} \sum_{i=1}^{k} \int_{D_{i}} h\vert zt_{i}\psi_{i}\vert ^{m}\,dx=\sum_{i\neq j} \int_{D_{i}} h\vert zt_{i}\psi_{i}\vert ^{m}\,dx + \int_{D_{j}\setminus E_{j}}h\vert zt_{j}\psi_{j}\vert ^{m}\,dx+ \int_{E_{j}} h\vert zt_{j}\psi_{j}\vert ^{m}\,dx. \end{aligned}$$

(3.8)

Since $\psi_{j}(x)=1$ for $x\in E_{j}$ and $\vert t_{j}\vert =1$, we have

$$\begin{aligned} \int_{E_{j}} h\vert zt_{j}\psi_{j}\vert ^{m}\,dx=\vert z\vert ^{m} \int_{E_{j}} h\,dx. \end{aligned}$$

(3.9)

On the over hand, since $D(d)\subset\Omega_{0}$, by $(H_{4})$ we obtain

$$\begin{aligned} \sum_{i\neq j} \int_{D_{i}} h\vert zt_{i}\psi_{i}\vert ^{m}\,dx + \int_{D_{j}\setminus E_{j}} h\vert zt_{j}\psi_{j}\vert ^{m}\,dx\ge0. \end{aligned}$$

(3.10)

Then, it follows from (3.7)-(3.10) that

$$\begin{aligned} \frac{J(zu)}{z^{q}}\le \frac{z^{p-q}}{p}\alpha_{k}^{p}+ \frac{1}{q}\alpha_{k}^{q}+\frac {z^{r-q}}{r} \alpha_{k}^{r}S_{r}^{r}\Vert \mu \Vert _{L^{\sigma}(\Omega_{0})}-\frac {1}{m}z^{m-q}\inf_{1\le i\le k} \biggl( \int_{E_{i}} h\,dx\biggr). \end{aligned}$$

(3.11)

Since $h(x)>0$ in $E_{i}$ and $m\in(1,q)$, we have

$$\begin{aligned} \lim_{z\to0^{+}}\sup_{u\in W_{k}} \frac{J(zu)}{z^{q}}=-\infty. \end{aligned}$$

(3.12)

We fix $z>0$ small such that

$$\begin{aligned} \sup\bigl\{ J(u): u\in A_{k}\bigr\} < 0, \quad\mbox{where } A_{k}=zW_{k} \in\Gamma_{k}, \end{aligned}$$

(3.13)

which completes the proof of (3.1) and thus of Lemma 3.2. □

Proof of Theorem 1.4

Evidently, $J(0)=0$, and J is an even functional. Then, by Lemma 2.2, J satisfies the $(PS)_{c}$ condition. Furthermore, by Lemma 3.2 conditions (I) and (II) in Lemma 3.1 are satisfied. Thus, by Lemma 3.1 problem (1.1) admits infinitely many solutions $u_{n}\in E$ with $u_{n}\to0$ in E. Thus, the proof of Theorem 1.4 is finished. □