To prove our main results, we need to establish some lemmas.
Lemma 2.1
Let (H0) and one of assumptions (H1) and (H4) hold. Then, if
\(\{u_{n}\}\)
is a bounded sequence in
E, then there exists
\(u\in E\cap L^{m}({\mathbb {R}}^{N},h)\)
such that, up to a subsequence, \(u_{n}\to u\)
strongly in
\(L^{m}({\mathbb {R}}^{N},h)\)
as
\(n\to\infty\).
Proof
We first choose a constant \(\beta>0\) such that \(\Vert u_{n}\Vert _{E}\le\beta\) for all \(n\ge1\). If (H1) is satisfied, then for any \(\varepsilon>0\), there exists \(R_{0}>0\) such that
$$\begin{aligned} \biggl( \int_{B_{R}^{c}} \bigl\vert h(x)\bigr\vert ^{\gamma}\,dx \biggr)^{1/\gamma}< 2^{-m}\beta ^{-m}\varepsilon \quad\mbox{for all } R\ge R_{0}. \end{aligned}$$
(2.1)
Then, it follows from the Hölder inequality and Lemma 1.1 that, for \(R\ge R_{0}\),
$$\begin{aligned} \int_{B_{R}^{c}} h(x)\bigl\vert u_{n}(x)-u(x)\bigr\vert ^{m}\,dx\le \Vert h\Vert _{L^{\gamma}(B_{R}^{c})}\Vert u_{n}-u\Vert ^{m}_{L^{p_{s}^{*}}(B_{R}^{c})} \le 2^{m} \beta^{m}\Vert h\Vert _{L^{\gamma}(B_{R}^{c})}< \varepsilon. \end{aligned}$$
(2.2)
By Lemma 1.1, up to a subsequence, we obtain \(u_{n}\to u\) strongly in \(L^{m}(B_{R_{0}})\) and \(u_{n}(x)\to u(x)\) a.e. in \(B_{R_{0}}\) as \(n\to\infty\). Thus \(h(x)\vert u_{n}(x)-u(x)\vert ^{m}\to0\) a.e. in \(B_{R_{0}}\) as \(n\to\infty\). Similarly, for each measurable subset \(\Omega\subset B_{R_{0}}\), we have
$$\begin{aligned} \int_{\Omega}h(x)\bigl\vert u_{n}(x)-u(x)\bigr\vert ^{m}\,dx\le \Vert h\Vert _{L^{\gamma}(\Omega)}\Vert u_{n}-u \Vert ^{m}_{L^{p_{s}^{*}}(\Omega)} \le 2^{m}\beta^{m} \Vert h\Vert _{L^{\gamma}(\Omega)}. \end{aligned}$$
(2.3)
Since \(h(x)\in L^{\gamma}({\mathbb {R}}^{N})\), we obtain that the sequence \(\{h(x)\vert u_{n}(x)-u(x)\vert ^{m}\}\) is uniformly integrable and bounded in \(L^{1}(B_{R_{0}})\). Furthermore, an application of the Vitali convergence theorem gives
$$\begin{aligned} \lim_{n\to\infty} \int_{B_{R_{0}}} h(x)\bigl\vert u_{n}(x)-u(x)\bigr\vert ^{m}\,dx=0. \end{aligned}$$
(2.4)
Then the conclusion that \(u_{n}\to u\) strongly in \(L^{m}({\mathbb {R}}^{N},h)\) follows from (2.2) and (2.4).
If (H4) is satisfied, then for any \(\varepsilon>0\), there exists \(R_{0}>0\) such that
$$\begin{aligned} \Vert h\Vert _{L^{\delta}(B_{R}^{c})}= \biggl( \int_{B_{R}^{c}} \bigl\vert h(x)\bigr\vert ^{\delta}\,dx \biggr)^{1/\delta}< 2^{-m}\beta^{-m}\varepsilon \quad\mbox{for all } R\ge R_{0} \end{aligned}$$
(2.5)
and
$$\begin{aligned} \int_{B_{R}^{c}} h(x)\bigl\vert u_{n}(x)-u(x)\bigr\vert ^{m}\,dx\le \Vert h\Vert _{L^{\delta}(B_{R}^{c})}\Vert u_{n}-u\Vert ^{m}_{L^{q}(B_{R}^{c})} \le 2^{m} \beta^{m}\Vert h\Vert _{L^{\delta}(B_{R}^{c})}< \varepsilon. \end{aligned}$$
(2.6)
Similarly, we can derive (2.4). Then combining (2.4) with (2.6), we have \(u_{n}\to u\) in \(L^{m}({\mathbb {R}}^{N},h)\). □
Lemma 2.2
Let (H0) and one of assumptions (H1) and (H4) hold. If
\(\{u_{n}\}\)
is a bounded
\((PS)_{c}\)
sequence of the functional
J
defined by (1.20), then the functional
J
satisfies
\((PS)_{c}\)
condition.
Proof
Let \(\{u_{n}\}\) be a \((PS)_{c}\) sequence, that is,
$$\begin{aligned} J(u_{n})\to c \quad\mbox{and}\quad \bigl\Vert J'(u_{n})\bigr\Vert _{E'}\to0 \quad\mbox{as } n \to\infty. \end{aligned}$$
(2.7)
Since the sequence \(\{u_{n}\}\) is bounded in E, there exists a subsequence, still denoted by \(\{u_{n}\}\), such that
$$ \begin{aligned} &u_{n}\rightharpoonup u \quad\mbox{weakly in } E,\qquad u_{n}(x)\to u(x) \quad\mbox{a.e. in } {\mathbb {R}}^{N},\\ & u_{n}\to u \quad\mbox{strongly in } L^{t}_{\mathrm{loc}}\bigl({\mathbb {R}}^{N} \bigr), \end{aligned} $$
(2.8)
where \(t=p\) or q. We now prove that \(u_{n}\to u\) in E. Let \(\varphi \in E\) be fixed and denote by \(T_{\varphi}\) the linear functional on E defined by
$$\begin{aligned} T_{\varphi}(v)=A_{\varphi}(v)+B_{\varphi}(v),\quad \forall\varphi\in E, \end{aligned}$$
(2.9)
where \(A_{\varphi}(v)\) and \(B_{\varphi}(v)\) are the linear functionals defined by
$$ \begin{aligned} &A_{\varphi}(v)= \int \int _{{\mathbb {R}}^{2N}} \frac{\vert \varphi (x)-\varphi(y)\vert ^{p-2}(\varphi(x)-\varphi(y))}{\vert x-y\vert ^{N+ps}}\bigl(v(x)-v(y)\bigr)\,dx\,dy, \quad\forall \varphi\in E, \\ &B_{\varphi}(v)= \int \int _{{\mathbb {R}}^{2N}} \frac{\vert \varphi (x)-\varphi(y)\vert ^{q-2}(\varphi(x)-\varphi(y))}{\vert x-y\vert ^{N+qs}}\bigl(v(x)-v(y)\bigr)\,dx\,dy,\quad \forall \varphi\in E, \end{aligned} $$
(2.10)
respectively. Clearly, by the Hölder inequality, \(T_{\varphi}\) is also continuous, and
$$\begin{aligned} \bigl\vert T_{\varphi}(v)\bigr\vert &\le\bigl\vert A_{\varphi}(v)\bigr\vert + \bigl\vert B_{\varphi}(v)\bigr\vert \le \Vert \varphi \Vert _{X_{p}}^{p-1}\Vert v\Vert _{X_{p}} + \Vert \varphi \Vert _{X_{q}}^{q-1}\Vert v\Vert _{X_{q}} \\ &\le \bigl(\Vert \varphi \Vert _{E}^{p-1} + \Vert \varphi \Vert _{E}^{q-1}\bigr)\Vert v\Vert _{E}, \quad\forall v\in E. \end{aligned}$$
(2.11)
Furthermore, the fact that \(u_{n}\rightharpoonup u\) weakly in E implies that \(\lim _{n\to\infty}A_{u}(u_{n}-u)= \lim _{n\to\infty }B_{u}(u_{n}-u)=0\), and so
$$\begin{aligned} \lim_{n\to\infty}T_{u}(u_{n}-u)=0. \end{aligned}$$
(2.12)
On the other hand, as \(n\to\infty\), we have
$$\begin{aligned} o_{n}(1)&=\bigl(J'(u_{n})-J'(u) \bigr) (u_{n}-u) \\ &=T_{u_{n}}(u_{n}-u)-T_{u}(u_{n}-u) +\Phi_{n}+\Psi_{n} -\lambda P_{n}+Z_{n}, \end{aligned}$$
(2.13)
where
$$\begin{aligned} &\Phi_{n}= \int_{{\mathbb {R}}^{N}} a(x) \bigl(\vert u_{n}\vert ^{p-2}u_{n} - \vert u\vert ^{p-2}u\bigr) (u_{n} - u)\,dx, \\ &\Psi_{n} = \int_{{\mathbb {R}}^{N}} b(x) \bigl(\vert u_{n}\vert ^{q-2}u_{n} - \vert u\vert ^{q-2}u\bigr) (u_{n} - u)\,dx,\\ &Z_{n}= \int_{{\mathbb {R}}^{N}} \mu(x) \bigl(\vert u_{n}\vert ^{r-2}u_{n} - \vert u\vert ^{r-2}u\bigr) (u_{n} - u)\,dx, \\ & P_{n} = \int_{{\mathbb {R}}^{N}} h(x) \bigl(\vert u_{n}\vert ^{m-2}u_{n} - \vert u\vert ^{m-2}u\bigr) (u_{n} - u)\,dx. \end{aligned}$$
(2.14)
From (2.13) and \(Z_{n}\ge0\), we obtain, for large n,
$$\begin{aligned} T_{u_{n}}(u_{n}- u)-T_{u}(u_{n}-u)+ \Phi_{n}+\Psi_{n}\le\lambda P_{n}+o_{n}(1). \end{aligned}$$
(2.15)
Note that, by Lemma 2.1, \(P_{n} \to0\) as \(n\to\infty\).
Let us now recall the well-known vector inequalities: for all \(\xi,\eta\in {\mathbb {R}}^{N}\),
$$ \begin{aligned} &\vert \xi-\eta \vert ^{p} \le c_{p}\bigl(\vert \xi \vert ^{p-2}\xi-\vert \eta \vert ^{p-2}\eta\bigr) (\xi-\eta) \quad\mbox{for } p\ge2,\quad \mbox{and}\\ &\vert \xi-\eta \vert ^{p}\le C_{p} \bigl[\bigl(\vert \xi \vert ^{p-2}\xi-\vert \eta \vert ^{p-2}\eta\bigr) (\xi- \eta) \bigr]^{p/2}\bigl(\vert \xi \vert ^{p}+\vert \eta \vert ^{p}\bigr)^{(2-p)/2} \quad\mbox{for } 1< p< 2, \end{aligned} $$
(2.16)
where \(c_{p}\) and \(C_{p}\) are positive constants depending only on p. Assume first that \(p>q\ge2\). Then by (2.16) we have \(\Vert u_{n}-u\Vert _{p,a}^{p}\le c_{p}\Phi_{n}\) and
$$\begin{aligned}{} [u_{n}-u]^{p}_{s,p}={}& \int \int _{{\mathbb {R}}^{2N}} \bigl\vert u_{n}(x)-u_{n}(y)-u(x)+u(y) \bigr\vert ^{p}\vert x-y\vert ^{-(N+sp)}\,dx\,dy \\ \le{}& c_{p} \int \int _{{\mathbb {R}}^{2N}} \bigl[\bigl\vert u_{n}(x)-u_{n}(y) \bigr\vert ^{p-2}\bigl(u_{n}(x)-u_{n}(y)\bigr)- \bigl\vert u(x)-u(y)\bigr\vert ^{p-2} \\ &{} \times \bigl(u(x)-u(y)\bigr) \bigr]\bigl(u_{n}(x)-u(x)-u_{n}(y)+u(y) \bigr)\vert x-y\vert ^{-(N+sp)}\,dx\,dy \\ ={}&c_{p} \bigl[A_{u_{n}}(u_{n}-u)-A_{u}(u_{n}-u) \bigr]. \end{aligned}$$
(2.17)
Similarly, we have \(\Vert u_{n}-u\Vert _{q,b}^{q}\le c_{q} \Psi_{n}\) and
$$\begin{aligned}{} [u_{n}-u]^{q}_{s,q}={}& \int \int _{{\mathbb {R}}^{2N}} \bigl\vert u_{n}(x)-u_{n}(y)-u(x)+u(y) \bigr\vert ^{q}\vert x-y\vert ^{-(N+sq)}\,dx\,dy \\ \le{}& c_{q} \int \int _{{\mathbb {R}}^{2N}} \bigl[\bigl\vert u_{n}(x)-u_{n}(y) \bigr\vert ^{q-2}\bigl(u_{n}(x)-u_{n}(y)\bigr)- \bigl\vert u(x)-u(y)\bigr\vert ^{q-2} \\ &{} \times \bigl(u(x)-u(y)\bigr) \bigr]\bigl(u_{n}(x)-u(x)-u_{n}(y)+u(y) \bigr)\vert x-y\vert ^{-(N+sq)}\,dx\,dy \\ ={}&c_{q} \bigl[B_{u_{n}}(u_{n}-u)-B_{u}(u_{n}-u) \bigr]. \end{aligned}$$
(2.18)
Let \(C_{0}=\min\{c_{p}^{-1},c_{q}^{-1}\}\). By (2.17) and (2.18) we see that
$$\begin{aligned} T_{u_{n}}(u_{n}-u)-T_{u}(u_{n}-u)&=A_{u_{n}} (u_{n}-u)-A_{u}(u_{n}-u)+B_{u_{n}}(u_{n}-u)-B_{u}(u_{n}-u) \\ &\ge C_{0} \bigl([u_{n}-u]_{s,p}^{p}+[u_{n}-u]_{s,q}^{q} \bigr). \end{aligned}$$
(2.19)
Then the application of (2.15) yields
$$\begin{aligned} C_{0}\bigl(\Vert u_{n}-u\Vert _{X_{p}}^{p}+\Vert u_{n}-u\Vert _{X_{q}}^{q}\bigr)\le\lambda P_{n}+o_{n}(1) \to 0 \quad\mbox{as } n\to\infty. \end{aligned}$$
(2.20)
In conclusion, \(u_{n}\to u\) in E as \(n\to\infty\).
Finally, it remains to consider the case \(1< p<2\). By (2.8) there exists \(\beta>0\) such that \(\Vert u_{n}\Vert _{E}\le\beta\) for all \(n\ge 1\). Now from (2.16) and the Hölder inequality it follows that
$$\begin{aligned}{} [u_{n}-u]^{p}_{s,p}& \le C_{p}\bigl[A_{u_{n}}(u_{n}-u)-A_{u}(u_{n}-u) \bigr]^{p/2}\bigl([u_{n}]_{s,p}^{p}+[u]_{s,p}^{p} \bigr)^{(2-p)/2} \\ &\le C_{p}\bigl[A_{u_{n}}(u_{n}-u)-A_{u}(u_{n}-u) \bigr]^{p/2}\bigl([u_{n}]_{s,p}^{p(2-p)/2}+[u]_{s,p}^{p(2-p)/2} \bigr) \\ &\le D_{p}\bigl[A_{u_{n}}(u_{n}-u)-A_{u}(u_{n}-u) \bigr]^{p/2} \end{aligned}$$
(2.21)
and
$$\begin{aligned} \Vert u_{n}-u\Vert _{p,a}^{p} \le D_{p}\Phi_{n}^{p/2}, \end{aligned}$$
(2.22)
where we have applied the inequality
$$\begin{aligned} (x+y)^{(2-p)/2}\le x^{(2-p)/2}+y^{(2-p)/2} \quad\mbox{for all } x,y\ge0 \mbox{ and } 1< p< 2, \end{aligned}$$
(2.23)
and \(D_{p}=2C_{p}\beta^{p(2-p)/2}\). Similarly, for \(1< q<2\), we have
$$\begin{aligned}{} [u_{n}-u]^{q}_{s,q}\le D_{q}\bigl[B_{u_{n}}(u_{n}-u)-B_{u}(u_{n}-u) \bigr]^{q/2}, \qquad \Vert u_{n}-u\Vert _{q,b}^{q} \le D_{q}\Psi_{n}^{q/2} \end{aligned}$$
(2.24)
with \(D_{q}=2C_{q}\beta^{q(2-q)/2}\). Then, by (2.21), (2.22), and (2.24) we get
$$\begin{aligned} &T_{u_{n}}(u_{n} - u) - T_{u}(u_{n} - u) + \Phi_{n} + \Psi_{n} \\ &\quad\ge C_{1} \bigl([u_{n} - u]^{2}_{s,q} + [u_{n} - u]_{s,p}^{2} + \Vert u_{n} - u\Vert _{p,a}^{2} + \Vert u_{n} - u\Vert _{q,b}^{2}\bigr) \end{aligned}$$
(2.25)
with some \(C_{1}>0\). Then (2.15) and (2.25) imply that \(u_{n}\to u\) in E as \(n\to\infty\). Therefore, J satisfies the \((PS)_{c}\) condition, and we complete the proof of Lemma 2.2. □
Lemma 2.3
Under the assumptions of Theorem
1.2, suppose that
\(u\in E\)
is a nontrivial weak solution of (1.1). Then there exists
\(\lambda_{1}>0\)
such that
\(\lambda\ge\lambda_{1}\).
Proof
Since \(u\in E\) is a nontrivial weak solution of (1.1), we have \(J'(u)\varphi=0\) for all \(\varphi\in E\). In particular, choosing \(\varphi=u\), we have
$$\begin{aligned} \Vert u\Vert _{X_{p}}^{p}+\Vert u\Vert _{X_{q}}^{q}+\Vert u\Vert _{r,\mu}^{r} = \lambda \Vert u\Vert _{m,h}^{m}. \end{aligned}$$
(2.26)
By the Young inequality with \(\epsilon>0\) we see that
$$\begin{aligned} cd\le\epsilon\theta^{-1} c^{p}+ \tau^{-1}\epsilon^{1/(1-\theta)}d^{\tau}, \quad\tau^{-1}+ \theta^{-1}=1, \theta>1. \end{aligned}$$
(2.27)
Taking \(0<\alpha<\beta, c=k_{1}>0, d=t^{\alpha}, \tau=\frac{\beta}{\alpha}\), \(\epsilon=(k_{2}\beta/\alpha)^{-\alpha/(\beta-\alpha)}\), \(k_{2}>0\), it follows from (2.27) that
$$\begin{aligned} k_{1}t^{\alpha}-k_{2}t^{\beta} \le k_{0}k_{1}(k_{1}/k_{2})^{\alpha/(\beta-\alpha)},\quad \forall t\ge0, \end{aligned}$$
(2.28)
with \(k_{0}=(1-\alpha/\beta)(\beta/\alpha)^{-\alpha/(\beta-\alpha)}<1\). Furthermore, let \(k_{1}=\lambda h(x), k_{2}=\frac{1}{2}\mu(x)\), \(\alpha=m-p\), and \(\beta=r-p\). Then from (2.28) we obtain
$$\begin{aligned} \lambda h(x)\vert u\vert ^{m-p}-\frac{1}{2} \mu(x)\vert u\vert ^{r-p}\le c_{1}\lambda^{\frac{r-p}{r-m}}g(x),\quad \forall (x,u)\in {\mathbb {R}}^{N}\times {\mathbb {R}}, \end{aligned}$$
(2.29)
where \(c_{1}=2^{(m-p)/(r-m)}\) and \(g(x)=[h(x)^{r-p}/\mu(x)^{m-p}]^{\frac{1}{r-m}}\). By (H2) we know \(g(x)\in L^{\frac{N}{sp}}({\mathbb {R}}^{N})\). So, the application of (1.17) and (2.29) yields
$$\begin{aligned} \lambda \int_{{\mathbb {R}}^{N}} h(x)\vert u\vert ^{m}\,dx- \frac{1}{2} \int_{{\mathbb {R}}^{N}} \mu(x)\vert u\vert ^{r}\,dx \le& c_{1}\lambda^{\frac{r-p}{r-m}} \int_{{\mathbb {R}}^{N}} g(x)\vert u\vert ^{p}\,dx \\ \le& c_{1}GS_{0}^{p}\lambda^{\frac{r-p}{r-m}}[u]_{s,p}^{p} \end{aligned}$$
(2.30)
with \(G=\Vert g\Vert _{L^{\frac{N}{sp}}({\mathbb {R}}^{N})}\). Then, from (2.26) and (2.30) we see that
$$\begin{aligned}{} [u]_{s,p}^{p}\le c_{1}GS_{0}^{p} \lambda^{\frac{r-p}{r-m}}[u]_{s,p}^{p}. \end{aligned}$$
(2.31)
This implies that \(\lambda\ge\lambda_{1}\equiv (c_{1}^{-1}S_{0}^{-p}G^{-1})^{(r-m)/(r-p)}\) and completes the proof of Lemma 2.3. □
Lemma 2.4
Under the assumptions of Theorem
1.2, the functional
J
is coercive in
E.
Proof
Letting \(k_{1}=\frac{\lambda}{m}h(x), k_{2}=\frac{1}{2r}\mu(x)\), \(\alpha=m-p, \beta=r-p\), and \(t=\vert u(x)\vert \) in (2.28), we conclude that
$$\begin{aligned} f(x,u):=\frac{\lambda}{m}h(x)\vert u\vert ^{m}- \frac{1}{2r}\mu(x)\vert u\vert ^{r}\le c_{2}g(x) \vert u\vert ^{p},\quad \forall(x,u)\in {\mathbb {R}}^{N}\times {\mathbb {R}}, \end{aligned}$$
(2.32)
where \(c_{2}=(2r)^{\frac{m-p}{r-m}}m^{\frac{p-r}{r-m}}\lambda^{\frac{r-p}{r-m}}\) and \(g(x)=[h(x)^{r-p}/\mu(x)^{m-p}]^{\frac{1}{r-m}}\). Since \(g(x)\in L^{\frac{N}{ps}}({\mathbb {R}}^{N})\), for any small \(\varepsilon>0\), there exists \(R_{1}>0\) such that
$$\begin{aligned} c_{2} \biggl( \int_{B_{R_{1}}^{c}} \bigl\vert g(x)\bigr\vert ^{N/ps}\,dx \biggr)^{ps/N}\le \varepsilon \end{aligned}$$
(2.33)
and
$$\begin{aligned} c_{2} \int_{B_{R_{1}}^{c}} \bigl\vert g(x)\bigr\vert \vert u\vert ^{p}\,dx\le c_{2} \biggl( \int_{B_{R_{1}}^{c}} \bigl\vert g(x)\bigr\vert ^{N/ps}\,dx \biggr)^{ps/N}\Vert u\Vert ^{p}_{L^{p_{s}^{*}}({\mathbb {R}}^{N})} \le \varepsilon S_{0}^{p}[u]_{s,p}^{p}, \end{aligned}$$
(2.34)
where \(S_{0}\) is the embedding constant in (1.17). So, it follows from (2.32)-(2.34) that
$$\begin{aligned} J(u)&=\frac{1}{p}\Vert u\Vert _{X_{p}}^{p}+\frac{1}{q}\Vert u\Vert _{X_{q}}^{q}+ \frac{1}{r}\Vert u\Vert _{r,\mu}^{r}-\frac{\lambda}{m} \Vert u\Vert _{m,h}^{m} \\ &\ge\frac{1}{p}\Vert u\Vert _{X_{p}}^{p}+ \frac{1}{q}\Vert u\Vert _{X_{q}}^{q} - \int_{B_{R}} f(x,u)\,dx- c_{2} \int_{B_{R}^{c}} g\vert u\vert ^{p}\,dx. \end{aligned}$$
(2.35)
For fixed \(R_{1}>0\) and for any \(\tau>0\) and \(\omega>0\), we decompose \(B_{R_{1}}=X\cup Y\cup Z\) as follows:
$$ \begin{aligned} &X=\bigl\{ x\in B_{R_{1}}: 0\le h(x)< \omega\mbox{ and } \mu(x)>\tau\bigr\} ,\qquad Z=\bigl\{ x\in B_{R_{1}}: h(x) \ge\omega\bigr\} , \\ &Y=\bigl\{ x\in B_{R_{1}}: 0\le h(x)< \omega\mbox{ and } 0\le \mu(x)< \tau \bigr\} . \end{aligned} $$
(2.36)
Obviously, the sets \(X, Y\), and Z are Lebesgue measurable. Note that the assumption \(h(x),\mu(x)\in L_{\mathrm{loc}}^{1}({\mathbb {R}}^{N})\) implies that \(\operatorname{meas}(Y)\to0\) as \(\tau\to0\) and \(\operatorname{meas}(Z)\to0\) as \(\omega\to\infty\).
On the other hand, letting \(k_{1}= \frac{\lambda}{m}h(x), k_{2}=\frac{1}{2r}\mu(x), t=\vert u(x)\vert , \alpha=m\), and \(\beta=r\) in (2.28), we derive
$$\begin{aligned} f(x,u):=\frac{\lambda}{m}h(x)\vert u\vert ^{m}- \frac{1}{2r}\mu(x)\vert u\vert ^{r}\le c_{3}g_{1}(x) \end{aligned}$$
(2.37)
with \(c_{3}=(2r)^{m/r}(\lambda/m)^{1+m/r}, g_{1}(x)=[h(x)/\mu(x)]^{m/r}\). Then,
$$\begin{aligned} \int_{X}f(x,u)\,dx\le c_{3} \int_{X}g_{1}(x)\,dx\le C_{1}, \end{aligned}$$
(2.38)
where \(C_{1}=C_{1}(\omega,\tau,R)>0\) is a constant. Furthermore, it follows from (2.32) and (2.34) that
$$\begin{aligned} \int_{Y\cup Z}f(x,u)\,dx\le c_{2} \int_{Y\cup Z}g(x)\vert u\vert ^{p}\,dx\le c_{2} \biggl( \int_{Y\cup Z}\vert g\vert ^{N/ps}\,dx \biggr)^{ps/N}\Vert u\Vert ^{p}_{L^{p_{s}^{*}}(B_{R_{1}})}. \end{aligned}$$
(2.39)
For any \(\varepsilon>0\), we can choose large \(\omega>0\) and small \(\tau>0\) such that \(\operatorname{meas}(Y\cup Z)\) is so small that
$$\begin{aligned} c_{2} \biggl( \int_{Y\cup Z}\vert g\vert ^{N/ps}\,dx \biggr)^{ps/N}\le\varepsilon. \end{aligned}$$
(2.40)
From (1.13) and (2.38)-(2.40) we obtain
$$\begin{aligned} \int_{B_{R}}f(x,u)\,dx\le C_{1}+\varepsilon \Vert u \Vert ^{p}_{L^{p_{s}^{*}}(B_{R_{1}})}\le C_{1}+\varepsilon S_{0}^{p}[u]_{s,p}^{p}. \end{aligned}$$
(2.41)
Thus, combining (2.34) and (2.35) with (2.41) yields
$$\begin{aligned} J(u)\ge\frac{1}{p}\Vert u\Vert _{X_{p}}^{p}+ \frac{1}{q}\Vert u\Vert _{X_{q}}^{q} -2\varepsilon S_{0}^{p}[u]_{s,p}^{p}-C_{1} \ge \frac{1}{2p}\Vert u\Vert _{X_{p}}^{p}+ \frac{1}{q}\Vert u\Vert _{X_{q}}^{q}-C_{1}, \end{aligned}$$
(2.42)
where \(0<2\varepsilon S_{0}^{p}\le1/2p\). Hence, J is coercive in E. □
Lemma 2.5
Under the assumptions of Theorem
1.2, there exists
\(u\in E\)
such that
\(d=J(u)=\inf_{v\in E}J(v)\)
and
u
is a weak solution of (1.1).
Proof
By Lemma 2.4 we see that \(d>-\infty\). Let \(\{u_{n}\}\) be a minimizing sequence for d in E, which is bounded in E by Lemma 2.4. Without loss of generality, we may assume that \(\{u_{n}\}\) is nonnegative, converges to weakly to some u in E, and \(u_{n}(x)\to u(x)\) a.e. in \({\mathbb {R}}^{N}\). Moreover, by the weak lower semicontinuity of the norms we have
$$\begin{aligned} \frac{1}{p}\Vert u\Vert _{X_{p}}^{p}+ \frac{1}{q}\Vert u\Vert _{X_{q}}^{q}+ \frac{1}{r}\Vert u\Vert _{r,\mu}^{r} \le \liminf _{n\to\infty} \biggl[\frac{1}{p}\Vert u_{n}\Vert _{X_{p}}^{p}+\frac{1}{q}\Vert u_{n}\Vert _{X_{q}}^{q} +\frac{1}{r}\Vert u_{n}\Vert _{r,\mu}^{r} \biggr]. \end{aligned}$$
(2.43)
Then from Lemma 2.1 and (2.43) it follows
$$\begin{aligned} J(u)\le\liminf_{n\to\infty}J(u_{n})=d. \end{aligned}$$
(2.44)
On the other hand, since \(u\in E\), we have that \(J(u)\ge d\), which shows that \(J(u)=d\). Therefore, u is a global minimum for J, and hence it is a critical point, namely a weak solution of (1.1). □
Lemma 2.6
Under the assumptions of Theorem
1.2, there exists
\(\lambda_{2}>0\)
such that for all
\(\lambda>\lambda_{2}\), Eq. (1.1) admits a global nontrivial minimum
\(u_{0}\in E\)
of
J
with
\(J(u_{0})<0\).
Proof
Clearly, \(J(0)=0\). Consider the constrained minimization problem
$$\begin{aligned} \lambda_{2}=\inf \biggl\{ \frac{1}{p}\Vert u \Vert _{X_{p}}^{p}+\frac{1}{q}\Vert u\Vert _{X_{q}}^{q}+\frac{1}{r}\Vert u\Vert _{r,\mu}^{r} : u\in E \mbox{ and } \Vert u\Vert _{m,h}^{m}=m \biggr\} . \end{aligned}$$
(2.45)
Let \(u_{n}\) be a minimizing sequence of (2.45), which is clearly bounded in E, so that we can assume, without loss of generality, that it converges weakly to some \(u_{0}\in E\) with \(\Vert u_{0}\Vert _{m,h}^{m}=m\) and
$$\begin{aligned} \lambda_{2}=\frac{1}{p}\Vert u_{0} \Vert _{X_{p}}^{p}+\frac{1}{q}\Vert u_{0} \Vert _{X_{q}}^{q}+\frac {1}{r}\Vert u_{0} \Vert _{r,\mu}^{r}>0. \end{aligned}$$
(2.46)
Thus, \(J(u_{0})=\lambda_{2}-\lambda<0\) for any \(\lambda>\lambda_{2}\), and
$$\begin{aligned} d=J(u_{0})=\inf_{u\in E}J(u)< 0 \quad\mbox{for all } \lambda>\lambda_{2}. \end{aligned}$$
(2.47)
This completes the proof. □
Next, we show that if \(\lambda>\lambda_{2}\), then problem (1.1) admits a second nontrivial weak solution \(e\neq u_{0}\) by the mountain pass theorem.
Lemma 2.7
Suppose that assumptions (H0)-(H1) are satisfied. Then, for all
\(e\in E\)
and
\(\lambda>0\), there exist
\(\alpha>0\)
and
\(\rho\in (0,\Vert e\Vert _{E})\)
such that
\(J(u)\ge\alpha\)
for all
\(u\in E\)
with
\(\Vert u\Vert _{E}=\rho\).
Proof
Let \(u\in E\). From (H1), (1.18), and (1.19) with \(t=p\) we obtain
$$\begin{aligned} \int_{{\mathbb {R}}^{N}} h(x)\vert u\vert ^{m}\,dx\le \Vert h \Vert _{\gamma} \Vert u\Vert _{p_{s}^{*}}^{m}\le S_{0}^{m}\Vert h\Vert _{\gamma} \Vert u\Vert _{E}^{m}. \end{aligned}$$
(2.48)
Then,
$$\begin{aligned} J(u)\ge p^{-1}\Vert u\Vert _{X_{p}}^{p}+q^{-1} \Vert u\Vert _{X_{q}}^{q}-\lambda S_{0}^{m}H \Vert u\Vert _{E}^{m}\ge p^{-1}\Vert u \Vert _{E}^{p}-\lambda S_{0}^{m}H \Vert u\Vert _{E}^{m}, \end{aligned}$$
(2.49)
where \(H=\Vert h\Vert _{\gamma}, \Vert u\Vert _{E}=\rho\), and
$$\begin{aligned} 0< \rho< \min \bigl\{ 1, \Vert e\Vert _{E}, \bigl( \lambda pS_{0}^{m}H\bigr)^{\frac{1}{p-m}} \bigr\} , \end{aligned}$$
(2.50)
so that
$$\begin{aligned} J(u)\ge\rho^{p} \bigl(p^{-1}-\lambda S_{0}^{m}H\rho^{m-p}\bigr)\equiv\alpha>0. \end{aligned}$$
(2.51)
Thus, we finish the proof of Lemma 2.7. □
Lemma 2.8
Under the assumptions of Theorem
1.2
and
\(\lambda>\lambda_{2}\), Eq. (1.1) admits a nontrivial weak solution
\(u\in E\)
such that
\(J(u)>0\).
Proof
By Lemma 2.6, for all \(\lambda>\lambda_{2}\), there exists a nontrivial weak solution \(u_{0}\in E\) with \(J(u_{0})<0\). Taking \(e=u_{0}\) in Lemma 2.7, we get that J satisfies the geometrical structure of Theorem A.3 of [15]. Thus, for all \(\lambda>\lambda_{2}\) there exists a sequence \(\{u_{n}\}\subset E\) such that
$$\begin{aligned} J(u_{n})\to c>0\quad \mbox{and}\quad \bigl\Vert J'(u_{n})\bigr\Vert _{E'}\to0 \quad\mbox{as } n \to\infty, \end{aligned}$$
(2.52)
where
$$\begin{aligned} c=\inf_{\gamma\in\Gamma}\max_{0\le t\le1}J \bigl(\gamma(t)\bigr) \quad\mbox{with } \Gamma=\bigl\{ \gamma\in C\bigl([0,1]; E\bigr): \gamma(0)=0, \gamma(1)=u_{0}\bigr\} . \end{aligned}$$
(2.53)
Since J is coercive in E, the sequence \(\{u_{n}\}\) is bounded in E. By Lemma 2.2 there exists a subsequence, still denoted by \(\{u_{n}\}\), such that \(u_{n}\to u\) in E as \(n\to\infty\). Therefore, \(J(u)=\lim_{n\to\infty}J(u_{n})=c>0\), and \(J'(u)\varphi=\lim_{n\to\infty}J'(u_{n})\varphi=0\) for all \(\varphi \in E\). So, u is a weak solution of (1.1) with \(J(u)>0\). □
Proof of Theorem 1.2
The application of Lemma 2.2 shows that problem (1.1) has only a trivial solution if \(\lambda<\lambda_{1}\). By Lemmas 2.6 and 2.8 it follows that, for all \(\lambda>\lambda_{2}\), problem (1.1) admits at least two nontrivial weak solutions in E, one with negative energy and the other with positive energy. This completes the proof of Theorem 1.2. □
Proof of Theorem 1.3
We first prove, under the assumptions in Theorem 1.3, that any \((PS)_{c}\) sequence \(\{u_{n}\}\) is bounded in E. Let the sequence \(\{u_{n}\}\) satisfy (2.7). Then, for large n, we have
$$\begin{aligned} c+1+\Vert u_{n}\Vert _{E} \ge& J(u_{n}) - \frac{1}{m}J'(u_{n})u_{n} \\ =& \biggl(\frac{1}{p}-\frac{1}{m} \biggr)\Vert u_{n} \Vert _{X_{p}}^{p}+ \biggl(\frac{1}{q}- \frac{1}{m} \biggr)\Vert u_{n}\Vert _{X_{q}}^{q} + \biggl(\frac{1}{r}-\frac{1}{m} \biggr)\Vert u_{n} \Vert _{r,\mu}^{r}. \end{aligned}$$
(2.54)
Since \(m>\max\{p,r\}\), it follows from (1.19) that \(\{\Vert u_{n}\Vert _{E}\}\) is bounded. Furthermore, by Lemma 2.2 there exists a subsequence of \(\{u_{n}\}\), still denoted by \(\{u_{n}\}\), and \(u\in E\) such that \(u_{n}\to u\) in E and J satisfies the \((PS)_{c}\) condition.
From (2.26) it follows that if \(u\in E\) is a nontrivial solution, then \(\lambda>0\). This proves part (i). In the following, we prove part (ii). We now verify the conditions in Theorem 6.5 in [17]. Clearly, the functional J defined by (1.20) is even, and \(J(0)=0\). By Lemma 2.7 there exist \(\alpha,\rho>0\) such that \(J(u)\ge\alpha\) for all \(u\in E\) with \(\Vert u\Vert _{E}=\rho\).
On the other hand, for any finite-dimensional subspace \(E_{0}\subset E\), it is well known that any norms in \(E_{0}\) are equivalent. So, there exist \(d_{1},d_{2}>0\) such that
$$\begin{aligned} d_{1}\Vert u\Vert _{E}\le \Vert u\Vert _{r,\mu}\le d_{2}\Vert u\Vert _{E},\qquad d_{1}\Vert u\Vert _{E}\le \Vert u\Vert _{m,h}\le d_{2}\Vert u\Vert _{E},\quad \forall u \in E_{0}. \end{aligned}$$
(2.55)
Then, from (1.20) we have
$$\begin{aligned} J(u)\le \frac{1}{q}\bigl(\Vert u\Vert _{E}^{p}+\Vert u\Vert _{E}^{q} \bigr)+\frac{1}{r}d_{2}^{r}\Vert u\Vert _{E}^{r}-\frac {\lambda }{m}d_{1}^{m} \Vert u\Vert _{E}^{m},\quad \forall u\in E_{0}. \end{aligned}$$
(2.56)
Since \(\lambda>0\) and \(m>\max\{p,r\}\), there exists \(R=R(E_{0})>\rho\) such that \(J(u)<0\) for \(u\in E_{0}\) and \(\Vert u\Vert _{E}\ge R\). Therefore, all conditions are verified. Then an application of Theorem 6.5 in [17] shows that Eq. (1.1) admits infinitely many solutions \(u_{n}\in E\) with \(J(u_{n})\to\infty\) as \(n\to\infty\). This completes the proof of Theorem 1.3. □