We will denote by \(H^{s}(\mathbb{R}^{N})\) the usual fractional Sobolev space endowed with the natural norm
$$\Vert u\Vert _{ H^{s}(\mathbb{R}^{N})}=\Vert u\Vert _{L^{2}(\mathbb{R}^{N})}+ \biggl( \iint _{\mathbb{R}^{2N}}\frac{\vert u(x)-u(y)\vert ^{2}}{\vert x-y\vert ^{N+2s}}\,dx\,dy \biggr) ^{ \frac{1}{2}}. $$
We consider the function space
$$X_{0}^{s}= \bigl\{ u\in H^{s} \bigl( \mathbb{R}^{N} \bigr):u=0 \mbox{ a.e. in }\mathbb{R}^{N} \backslash \Omega \bigr\} , $$
with the norm
$$\Vert u\Vert _{X_{0}^{s}}= \biggl( \int_{Q}\frac{\vert u(x)-u(y)\vert ^{2}}{\vert x-y\vert ^{N+2s}}\,dx\,dy-\mu \int_{ \Omega }\frac{u^{2}}{\vert z\vert ^{2s}}\,dx \biggr) ^{\frac{1}{2}}, $$
which is equivalent to its general norm due to the Hardy inequality
$$ a_{k,s} \int_{\mathbb{R}^{N}}\frac{u^{2}}{\vert z\vert ^{2s}}\leq \int_{Q} \frac{\vert u(x)-u(y)\vert ^{2}}{\vert x-y\vert ^{N+2s}}\,dx\,dy, $$
(3)
where \(Q=\mathbb{R}^{2N}\setminus (C\Omega \times C\Omega)\) with \(C\Omega =\mathbb{R}^{N}\setminus \Omega \). We can introduce the best fractional critical Hardy-Sobolev constant \(S_{\mu,\alpha }\), given by
$$ S_{\mu,\alpha } = \inf_{u\in X_{0}^{s}({\mathbb{R}^{N}}\setminus (0,w^{0})),u\neq 0}\frac{ \int_{Q}\frac{\vert u(x)-u(y)\vert ^{2}}{\vert x-y\vert ^{N+2s}}\,dx\,dy- \mu \int_{\Omega }\frac{u ^{2}}{\vert z\vert ^{2s}}\,dx}{ ( \int_{\mathbb{R}^{n}}\frac{\vert u(x)\vert ^{2_{s}^{*}( \alpha)}}{\vert z\vert ^{\alpha }}\,dx) ^{2/ 2_{s}^{*}(\alpha)}}. $$
(4)
From [20], we know that \(S_{\mu,\alpha }\) is attained by functions
$$U_{\varepsilon }(x)= \frac{\varepsilon^{\frac{(N-2s)}{2}}}{(\varepsilon^{2}+\vert x-x_{0}\vert ^{2})^{ \frac{(N-2s)}{2}}}. $$
Let \(u^{+}=\max \{u,0\}\), the energy functional \(J_{\lambda }:X_{0} ^{s}\rightarrow \mathbb{R}\) associated to the problem (1) is defined as
$$\begin{aligned} J_{\lambda }(u) =& \frac{1}{2} \int_{Q}\frac{\vert u(x)-u(y)\vert ^{2}}{\vert x-y\vert ^{N+2s}}\,dx\,dy- \frac{ \mu }{2} \int_{\Omega }\frac{(u^{+})^{2}}{\vert z\vert ^{2s}}\,dx \\ &{}- \frac{1}{2_{s}^{*}(\alpha)} \int_{\Omega }\frac{(u^{+})^{2_{s}^{*}( \alpha)}}{\vert z\vert ^{\alpha }}\,dx -\lambda \int_{\Omega }F \bigl(x,u^{+} \bigr)\,dx, \end{aligned}$$
(5)
for all \(u\in X_{0}^{s}\), where \(F(x,t)\) is a primitive function of \(f(x,t)\) defined by \(F(x,t)=\int_{0}^{t}f(x,\tau)\,d\tau \). Obviously, \(J_{\lambda }\) is a \(C^{1}(X_{0}^{s})\) functional, and it is well known that the solutions of problem (1) are the critical points of the energy functional \(J_{\lambda }\). In fact, if u is a weak solution of problem (1), we have
$$\begin{aligned} \bigl\langle J'_{\lambda }(u),\varphi \bigr\rangle =& \int_{Q}\frac{(u(x)-u(y))(\varphi (x)-\varphi (y))}{\vert x-y\vert ^{N+2s}}\,dx\,dy -\mu \int_{\Omega }\frac{u^{+}\varphi }{\vert z\vert ^{2s}}\,dx \\ &{}- \int_{\Omega }\frac{(u^{+})^{2_{s}^{*}(\alpha)-1}\varphi }{\vert z\vert ^{ \alpha }}\,dx-\lambda \int_{\Omega } f \bigl(x,u^{+} \bigr)\varphi \,dx \\ =&0, \end{aligned}$$
(6)
for all \(\varphi \in X_{0}^{s}\). Now, we will give some essential lemmas as follows.
Lemma 1
Let
\(\lambda >0\)
and
f
satisfies assumptions
\((f_{1})\)
and
\((f_{2})\). We can deduce that:
-
(i)
there exist
\(\varsigma,\rho >0\)
such that
\(J_{\lambda }(u)\geq \varsigma >0\)
for any
\(u\in X_{0}^{s}\), with
\(\Vert u\Vert _{X_{0} ^{s}}=\rho \);
-
(ii)
there exists
\(e\in X_{0}^{s}\), with
\(e\geq 0\)
in
\(\mathbb{R}^{N}\)
such that
\(J_{\lambda }(e)<0\)
and
\(\Vert e\Vert _{X_{0}^{s}} > \rho \).
Proof of Lemma 1
(i) Fixing \(\lambda >0\), from \((f_{2})\), for \(\varepsilon >0\), there exists \(C_{1}>0\), one has
$$ \bigl\vert F(x,t) \bigr\vert \leq \varepsilon \vert t\vert ^{2}+C_{1} \vert t\vert ^{2_{s}^{*}(\alpha)},\quad \forall (x,t)\in \Omega \times \mathbb{R}^{+}. $$
(7)
It is evident that \(X_{0}^{s}\hookrightarrow L^{q}(\Omega)\) for \(1\leq q\leq 2_{s}^{*}(\alpha)\), then there exists \(C_{2}>0\) such that
$$ \int_{\Omega }\vert u\vert ^{q}\,dx\leq C_{2} \Vert u\Vert _{X_{0}^{s}}^{q}. $$
(8)
Take \(u\in X_{0}^{s}\). Combining (4), (5), (7) and (8), we have
$$\begin{aligned} J_{\lambda }(u) =& \frac{1}{2}\Vert u\Vert _{X_{0}^{s}}^{2}- \frac{1}{2^{*}(s)} \int_{\Omega }\frac{\vert u\vert ^{2_{s} ^{*}(\alpha)}}{\vert z\vert ^{\alpha }}\,dx -\lambda \int_{\Omega }F(x,u)\,dx \\ \geq & \frac{1}{2}\Vert u\Vert _{X_{0}^{s}}^{2}- \frac{1}{2_{s}^{*}(\alpha)} \int_{\Omega }\frac{\vert u\vert ^{2_{s}^{*}(\alpha)}}{\vert x\vert ^{\alpha }}\,dx- \lambda \int_{\Omega } \bigl(\varepsilon \vert u\vert ^{2}+C_{1}\vert u\vert ^{2_{s}^{*}(\alpha)} \bigr)\,dx \\ \geq & \frac{1}{2}\Vert u\Vert _{X_{0}^{s}}^{2}- \frac{C_{3}}{2_{s}^{*}(\alpha)}\Vert u \Vert _{X_{0}^{s}}^{2_{s}^{*}(\alpha)}-\lambda \varepsilon C_{4} \Vert u\Vert _{X_{0}^{s}}^{2}- \lambda C_{5} \Vert u\Vert _{X_{0}^{s}}^{2_{s}^{*}(\alpha)}, \end{aligned}$$
where \(C_{i}\), \(i=3,4,5\), are positive constants. For \(\varepsilon >0\) small and according to the fact \(2< 2^{*}(s)\), then there exists \(\rho >0\) small enough such that \(J_{\lambda }(u)\geq \varsigma >0\), for any \(\Vert u\Vert _{X_{0}^{s}}=\rho \).
(ii) Given \(\lambda >0\). Take \(v \in X_{0}^{s}\), with \(v\geq 0\) in \(\mathbb{R}^{N}\) and \(\Vert v\Vert _{X_{0}^{s}}=1\). From \((f_{1})\), we get
$$\begin{aligned} J_{\lambda }(tv) =& \frac{1}{2}\Vert tv\Vert _{X_{0}^{s}}^{2}- \frac{1}{2_{s}^{*}(\alpha)} \int_{\Omega }\frac{\vert tv\vert ^{2_{s}^{*}(\alpha)}}{\vert x\vert ^{\alpha }}\,dx- \lambda \int_{\Omega }F(x,tv)\,dx \\ \leq& \frac{t^{2}}{2}\Vert v\Vert _{X_{0}^{s}}^{2}- \frac{t^{2_{s}^{*}(\alpha)}}{2_{s}^{*}(\alpha)} \int_{\Omega }\frac{\vert v\vert ^{2_{s} ^{*}(\alpha)}}{\vert x\vert ^{\alpha }}\,dx, \end{aligned}$$
then \(J_{\lambda }(tv)\rightarrow -\infty \) as \(t\rightarrow +\infty \). Choosing \(e=t_{*}v\) with \(t_{*}>0\) large enough, we get \(\Vert e\Vert _{X _{0}^{s}}>\rho \) and \(J_{\lambda }(e)<0\). This completes the proof of the Lemma 1. □
We recall that a sequence \(\{u_{j}\}_{j\in \mathbb{N}} \subset X_{0} ^{s}\) is a Palais-Smale sequence for the functional \(J_{\lambda }\) at level \(c_{\lambda }\) if
$$J_{\lambda }(u_{j})\rightarrow c_{\lambda } \quad \mbox{and} \quad J'_{\lambda }(u_{j})\rightarrow 0\quad \mbox{in } \bigl(X_{0}^{s} \bigr)', $$
as \(j\rightarrow \infty \). We say that \(J_{\lambda }\) satisfies the Palais-Smale condition if every Palais-Smale sequence of \(J_{\lambda }\) has a convergent subsequence in \(X_{0}^{s}\). Now put
$$c_{\lambda }=\inf_{g \in \Gamma } \max_{t\in [0,1]} J_{\lambda } \bigl(g(t) \bigr), $$
where
$$\Gamma = \bigl\{ g \in C \bigl([0,1],X_{0}^{s} \bigr):g(0)=0,J_{\lambda } \bigl(g(1) \bigr)< 0 \bigr\} . $$
Obviously, \(c_{\lambda }>0\) from Lemma 1. Next, we introduce an asymptotic condition for the level \(c_{\lambda }\).
Lemma 2
Under the conditions of Lemma
1, \(\lim_{\lambda \rightarrow \infty } c_{\lambda }=0\).
Proof of Lemma 2
Fix \(\lambda >0\). Since the functional \(J_{\lambda }\) satisfies the Mountain pass geometry, there exists \(t_{\lambda }>0\) verifying \(J_{\lambda }(t_{\lambda }e)=\max_{t\geq 0}J _{\lambda }(te)\), where \(e \in X_{0}^{s}\) is the function given in Lemma 1. Hence, by (6), we have \(\langle J'_{\lambda }(t_{\lambda }e),t _{\lambda }e\rangle =0\), that is,
$$ t_{\lambda }^{2}\Vert e\Vert _{X_{0}^{s}}^{2}= t_{\lambda }^{2_{s}^{*}(\alpha)} \int_{\Omega }\frac{(e^{+})^{2_{s}^{*}(\alpha)}}{\vert z\vert ^{\alpha }}\,dx+ \lambda \int_{\Omega }f \bigl(x,t_{\lambda }e^{+} \bigr)t_{\lambda }e^{+}\,dx. $$
(9)
From \((f_{1})\), we have
$$t_{\lambda }^{2}\Vert e\Vert _{X_{0}^{s}}^{2}\geq t_{\lambda }^{2_{s}^{*}( \alpha)} \int_{\Omega }\frac{(e^{+})^{2_{s}^{*}(\alpha)}}{\vert z\vert ^{ \alpha }}\,dx, $$
which implies that \(\{t_{\lambda }\}\) is bounded. Hence, there exist a number \(t_{0}\geq 0\) and a subsequence of \(\{\lambda_{j}\}_{j\in \mathbb{N}}\), which we still denote by \(\{\lambda_{j}\}_{j\in \mathbb{N}}\), such that \(\lambda_{j} \rightarrow +\infty \) and \(t_{\lambda_{j}}\rightarrow t_{0}\) as \(j\rightarrow \infty \). So by (9) there exists \(D>0\) such that \(t_{\lambda }^{2}\Vert e\Vert _{X_{0} ^{s}}^{2}\leq D \) for any \(j\in \mathbb{N}\), then
$$ \lambda_{j} \int_{\Omega }f \bigl(x,t_{\lambda_{j}} e^{+} \bigr)t_{\lambda_{j}} e ^{+}\,dx+t_{\lambda_{j}}^{2_{s}^{*}(\alpha)} \int_{\Omega }\frac{(e^{+})^{2_{s} ^{*}(\alpha)}}{\vert z\vert ^{\alpha }}\,dx \leq D. $$
(10)
If \(t_{0}>0\), by \((f_{1})\) and the Lebesgue dominated convergence theorem, we obtain
$$\lim_{j\rightarrow \infty } \int_{\Omega }f \bigl(x,t_{\lambda_{j}} e^{+} \bigr)t _{\lambda_{j}} e^{+}\,dx = \int_{\Omega }f \bigl(x,t_{0} e^{+} \bigr)t_{0}e^{+}\,dx>0. $$
Recalling that \(\lambda_{j}\rightarrow \infty \), we have
$$\lim_{j\rightarrow \infty } \biggl( \lambda_{j} \int_{\Omega }f \bigl(x,t_{ \lambda_{j}}e^{+} \bigr)t_{\lambda_{j}}e^{+}\,dx+t_{\lambda_{j}}^{p_{s}^{*}( \alpha)} \int_{\Omega }\frac{(e^{+})^{2_{s}^{*}(\alpha)}}{\vert z\vert ^{ \alpha }}\,dx \biggr) =\infty, $$
which contradicts (10). Thus \(t_{0}=0\) for \(\lambda_{j}\rightarrow \infty \). Now, let us consider the path \(g(t)=te\), for \(t\in [0,1]\), which belongs to Γ. By Lemma 1 and \((f_{1})\), we get
$$0< c_{\lambda }\leq \max_{t \in [0,1]} J_{\lambda } \bigl(g(t) \bigr)\leq J_{ \lambda }(t_{\lambda }e)\leq \frac{t_{\lambda }^{2}}{2}\Vert e \Vert _{X_{0} ^{s}}^{2}. $$
Notice that \(t_{\lambda_{j}}\rightarrow t_{0}=0\) as \(j\rightarrow \infty \), one has
$$\lim_{\lambda \rightarrow +\infty }\frac{t_{\lambda }^{2}}{2}\Vert e\Vert _{X_{0}^{s}}^{2}=0, $$
which leads to \(\lim_{\lambda \rightarrow \infty } c_{\lambda }=0\). This completes the proof of the Lemma 2. □
Lemma 3
Assume that conditions
\((f_{1})\)
and
\((f_{2})\)
hold. If
\(\{u_{j}\}\subset X_{0}^{s}\)
is a
\((PS)_{c_{\lambda }}\)
condition of
\(J_{\lambda }\), then
\(\{u_{j}\}\)
is bounded in
\(X_{0}^{s}\).
Proof of Lemma 3
By \((f_{2})\) and the boundedness of Ω, for any \(\varepsilon >0\), there exists \(T>0\), such that
$$\begin{aligned}& \bigl\vert F(x,t) \bigr\vert \leq \varepsilon \vert t\vert ^{2_{s}^{*}(\alpha)},\quad x\in \Omega, t\geq T;\qquad \bigl\vert F(x,t) \bigr\vert \leq C_{6}(\varepsilon),\quad t\in (0,T]; \\& \bigl\vert f(x,t)t \bigr\vert \leq \varepsilon \vert t\vert ^{2_{s}^{*}(\alpha)}, \quad x\in \Omega, t\geq T; \qquad \bigl\vert f(x,t)t \bigr\vert \leq C_{7} (\varepsilon),\quad t\in (0,T], \end{aligned}$$
for \(C_{i}(\varepsilon)>0\), \(i=6,7\). Furthermore, for any \((x,t)\in \Omega \times \mathbb{R}^{+}\), we have
$$\bigl\vert F(x,t) \bigr\vert \leq C_{6}(\varepsilon)+\varepsilon \vert t\vert ^{2_{s}^{*}(\alpha)},\qquad \bigl\vert f(x,t)t \bigr\vert \leq C_{7}(\varepsilon)+\varepsilon \vert t\vert ^{2_{s}^{*}(\alpha)}. $$
Then, for \(\xi \in (2,2_{s}^{*}(\alpha))\), one has
$$ F(x,t)-\frac{1}{2}f(x,t)t\leq F(x,t)-\frac{1}{\xi }f(x,t)t \leq C_{8}( \varepsilon)+\varepsilon \vert t\vert ^{2_{s}^{*}(\alpha)}, $$
(11)
for \(C_{8}(\varepsilon)>0\) and any \((x,t)\in \overline{\Omega } \times \mathbb{R}^{+}\). Set \(l(x,t):=\frac{\vert t\vert ^{2_{s}^{*}(\alpha)-1}}{\vert z\vert ^{ \alpha }}+\lambda f(x,t)\), we claim that \(l(x,t)\) satisfies the \(\mathrm{(AR)}\) condition. By (11), one easily gets
$$\begin{aligned} \xi L(x,t)-l(x,y)t =& \biggl(\frac{\xi }{2_{s}^{*}(\alpha)}-1 \biggr)\frac{\vert t\vert ^{2_{s}^{*}( \alpha)}}{\vert z\vert ^{\alpha }}+ \lambda \bigl( \xi F(x,t)-f(x,t)t \bigr) \\ \leq & \biggl(\frac{\xi }{2_{s}^{*}(\alpha)}-1 \biggr)\frac{\vert t\vert ^{2_{s}^{*}( \alpha)}}{\vert z\vert ^{\alpha }}+\lambda \xi C_{8}(\varepsilon)+\lambda \xi \varepsilon \vert t\vert ^{2_{s}^{*}(\alpha)} \\ =& \biggl( \biggl(\frac{\xi }{2_{s}^{*}(\alpha)}-1 \biggr)\vert z\vert ^{-\alpha }+ \lambda \xi \varepsilon \biggr)\vert t\vert ^{2_{s}^{*}(\alpha)}+\lambda \xi C_{8}( \varepsilon), \end{aligned}$$
where \(L(x,t)=\int_{0}^{t}l(x,\tau)\,d\tau \). Thus, for a fixed \(\lambda >0\) and \(\varepsilon >0\) sufficiently small, there exists \(T'_{\lambda }>0\), such that
$$0\leq \xi L(x,t)\leq l(x,t)t, \quad t\geq T'_{\lambda }. $$
Moreover, by \((f_{2})\), we obtain
$$L(x,t)-\frac{1}{\xi }l(x,t)t\leq \max_{x\in \Omega,0\leq t\leq T'_{\lambda }} \biggl( F(x,t)-\frac{1}{ \xi }f(x,t)t \biggr):=T_{\lambda }, $$
for any \(0\leq t\leq T'_{\lambda }\). Notice that \(\xi <2_{s}^{*}( \alpha)\), we obtain \(T_{\lambda }>0\). It follows from the above inequalities that
$$ L(x,t)-\frac{1}{\xi }l(x,t)t\leq T_{\lambda },\quad \mbox{for all } x\in \overline{\Omega }\backslash \bigl\{ \bigl(0,w^{0} \bigr) \bigr\} , t \geq 0. $$
(12)
Combining \((f_{2})\), (6) and (12), it follows that
$$\begin{aligned} c_{\lambda }+1 \geq& J_{\lambda }(u_{j})-\frac{1}{\xi } \bigl\langle J'_{\lambda }(u_{j}),u_{j} \bigr\rangle \\ \geq& \biggl(\frac{1}{2}-\frac{1}{\xi } \biggr)\Vert u_{j}\Vert _{X_{0}^{s}}^{2}+ \biggl( \frac{1}{ \xi }-\frac{1}{2_{s}^{*}(\alpha)} \biggr) \int_{\Omega }\frac{(u_{j}^{+})^{2_{s} ^{*}(\alpha)}}{\vert z\vert ^{\alpha }}\,dx \\ &{}-\lambda \int_{\Omega } \biggl(F \bigl(x,u_{j}^{+} \bigr)-\frac{1}{\xi }f \bigl(x,u_{j} ^{+} \bigr)u_{j}^{+} \biggr)\,dx \\ \geq & \biggl( \frac{1}{2}-\frac{1}{\xi } \biggr) \Vert u_{j}\Vert _{X_{0}^{s}}^{2} - \int_{\Omega } \biggl(L \bigl(x,u_{j}^{+} \bigr)-\frac{1}{\xi }l \bigl(x,u _{j}^{+} \bigr)u_{j}^{+} \biggr)\,dx \\ \geq& \biggl( \frac{1}{2}-\frac{1}{\xi } \biggr) \Vert u_{j}\Vert _{X_{0}^{s}}^{2}-T _{\lambda }\vert \Omega \vert . \end{aligned}$$
Hence, we obtain \(\{u_{j}\}_{j\in \mathbb{N}}\) is bounded in \(X_{0}^{s}\). This completes the proof of Lemma 3. □
Lemma 4
Assume that conditions
\((f_{1})\)
and
\((f_{2})\)
hold. Then
\(J_{\lambda }\)
satisfies the
\((PS)_{c_{\lambda }}\)
condition with
\(c_{\lambda }<\frac{2s-\alpha }{2(N-\alpha)} S_{ \mu,\alpha }^{\frac{N-\alpha }{2s-\alpha }} -T_{\lambda }\vert \Omega \vert \), where
\(T_{\lambda }\)
is a bounded constant given in Lemma
3.
Proof of Lemma 4
Let \(\{u_{j}\}_{j\in \mathbb{N}}\subset X_{0}^{s}\) be a \((PS)_{c_{\lambda }}\) sequence of \(J_{\lambda }\). From Lemma 3, we know that \(\{u_{j}\}_{j\in \mathbb{N}}\) is bounded. Thus there exist a subsequence (still denoted by \(\{u_{j}\}_{j\in \mathbb{N}}\)) and \(u_{\lambda }\in X_{0}^{s}\) such that
$$\begin{aligned} \textstyle\begin{cases} u_{j}\rightharpoonup u_{\lambda }, & \mbox{weakly in }X_{0}^{s}, \\ u_{j}\rightarrow u_{\lambda }, & \mbox{strongly in }L^{p},2\leq p< 2_{s}^{*}(\alpha), \\ u_{j}\rightharpoonup u_{\lambda }, & \mbox{weakly in }L^{2_{s}^{*}(\alpha)}(\Omega, \vert z\vert ^{-\alpha }), \\ u_{j}\rightarrow u_{\lambda }, & \mbox{a.e. in } \mathbb{R}^{n}. \end{cases}\displaystyle \end{aligned}$$
(13)
Due to the continuity of embedding \(X_{0}^{s}\hookrightarrow H^{s}( \Omega)\hookrightarrow L^{\nu }(\Omega)\) and the boundedness of \(\{u_{j}\}_{j\in \mathbb{N}}\), there exists a constant \(C_{\nu }\) such that \(\Vert u\Vert _{\nu }\leq \Vert u\Vert _{X_{0}^{s}}\leq C_{\nu }\) for all \(u\in X_{0}^{s}\) and \(\nu \in [2,2_{s}^{*}(\alpha)]\). Now, according to \((f_{2})\), for any \(\varepsilon >0\), there exists a \(a(\varepsilon)>0\) such that
$$\bigl\vert F(x,t) \bigr\vert \leq \frac{1}{2C_{\nu }}\varepsilon \vert t \vert ^{2_{s}^{*}(\alpha)}+a( \varepsilon)\quad \mbox{for }(x,t)\in \Omega \times \mathbb{R}^{+}. $$
Set \(\delta =\frac{\varepsilon }{2a(\varepsilon)}>0\). When \(E\subset \Omega \), meas \(E<\delta \), one gets
$$\begin{aligned} \biggl\vert \int_{E}F \bigl(x,u_{j}^{+} \bigr)\,dx \biggr\vert \leq& \int_{E} a(\varepsilon)\,dx+ \frac{1}{2C}\varepsilon \int_{E} \bigl\vert u_{j}^{+} \bigr\vert ^{2_{s}^{*}(\alpha)}\,dx \\ \leq& a(\varepsilon) \operatorname{meas} E+ \frac{1}{2C_{\nu }} \varepsilon C _{\nu }\leq \varepsilon. \end{aligned}$$
Obviously, \(\{\int_{\Omega }F(x,u_{j}^{+})\,dx,j\in N\}\) is equi-absolutely continuous. It follows from Vitali’s convergence theorem that
$$ \int_{\Omega }F \bigl(x,u_{j}^{+} \bigr)\,dx \rightarrow \int_{\Omega }F \bigl(x,u_{\lambda }^{+} \bigr)\,dx, \quad \mbox{as } j\rightarrow \infty. $$
(14)
Similarly, we get
$$ \int_{\Omega }f \bigl(x,u_{j}^{+} \bigr)u_{j}\,dx\rightarrow \int_{\Omega }f \bigl(x,u _{\lambda }^{+} \bigr)u_{\lambda }\,dx,\quad \mbox{as } j\rightarrow \infty. $$
(15)
By (13) and (15) we obtain
$$\begin{aligned}& \lim_{j\rightarrow \infty }\bigl\langle J'_{\lambda }(u_{j}),v \bigr\rangle \\& \quad = \int_{Q} \frac{(u_{\lambda }(x)-u_{\lambda }(y))(v(x)-v(y))}{\vert x-y\vert ^{N+2s}}\,dx\,dy -\mu \int_{\Omega }\frac{u_{\lambda }^{+}v}{\vert z\vert ^{2s}}\,dx \\& \qquad {}- \int_{\Omega }\frac{(u_{\lambda }^{+})^{2_{s}^{*}(\alpha)-1}v}{\vert z\vert ^{ \alpha }}\,dx-\lambda \int_{\Omega } f\bigl(x,u_{\lambda }^{+}\bigr)v \,dx \\& \quad =0, \end{aligned}$$
for any \(v \in X_{0}^{s}\). That is, \(\langle J'_{\lambda }(u_{\lambda }),v\rangle =0\) for any \(v \in X_{0}^{s}\). Then \(u_{\lambda }\) is a critical point of \(J_{\lambda }\), thus \(u_{\lambda }\) is a solution of problem (1). It follows from (6) and (12) that
$$\begin{aligned} J_{\lambda }(u_{\lambda }) =& J_{\lambda }(u_{\lambda })- \frac{1}{\xi }\bigl\langle J'_{\lambda }(u_{ \lambda }),u_{\lambda } \bigr\rangle \\ \geq & \biggl(\frac{1}{2}-\frac{1}{\xi } \biggr)\Vert u_{\lambda }\Vert _{X_{0}^{s}}^{2}+ \biggl( \frac{1}{\xi }-\frac{1}{2_{s}^{*}(\alpha)} \biggr) \int_{\Omega }\frac{\vert u_{\lambda }\vert ^{2_{s}^{*}(\alpha)}}{\vert z\vert ^{\alpha }}\,dx \\ &{} -\lambda \int_{\Omega } \biggl(F(x,u_{\lambda })-\frac{1}{\xi }f(x,u_{ \lambda })u_{\lambda } \biggr)\,dx \\ \geq & \biggl( \frac{1}{2}-\frac{1}{\xi } \biggr) \Vert u_{\lambda }\Vert _{X_{0}^{s}} ^{2} - \int_{\Omega } \biggl(L(x,u_{\lambda })-\frac{1}{\xi }l(x,u_{\lambda })u _{\lambda }\biggr)\,dx \\ \geq & \biggl( \frac{1}{2}-\frac{1}{\xi } \biggr) \Vert u_{\lambda }\Vert _{X_{0}^{s}} ^{2}-T_{\lambda }\vert \Omega \vert \\ \geq & -T_{\lambda }\vert \Omega \vert , \end{aligned}$$
for \(\xi \in (2,2_{s}^{*}(\alpha))\). Now, let \(w_{j}=u_{j}-u_{\lambda }\), by the Brezis-Lieb lemma [21], we have
$$\begin{aligned}& \int_{Q}\frac{\vert u_{j}(x)-u_{j}(y)\vert ^{2}}{\vert x-y\vert ^{N+2s}}\,dx\,dy \\& \quad = \int_{Q}\frac{\vert w_{j}(x)-w_{j}(y)\vert ^{2}}{\vert x-y\vert ^{N+2s}}\,dx\,dy+ \int_{Q}\frac{\vert u_{\lambda }(x)-u_{\lambda }(y)\vert ^{2}}{\vert x-y\vert ^{N+2s}}\,dx\,dy+o(1), \end{aligned}$$
(16)
$$\begin{aligned}& \int_{\Omega }\frac{u_{j}^{2}}{\vert z\vert ^{2s}}\,dx= \int_{\Omega }\frac{w_{j} ^{2}}{\vert z\vert ^{2s}}\,dx+ \int_{\Omega }\frac{u_{\lambda }^{2}}{\vert z\vert ^{2s}}\,dx+o(1), \end{aligned}$$
(17)
$$\begin{aligned}& \int_{\Omega }\frac{(u_{j}^{+})^{2_{s}^{*}(\alpha)}}{\vert z\vert ^{\alpha }}\,dx= \int_{\Omega }\frac{(w_{j}^{+})^{2_{s}^{*}(\alpha)}}{\vert z\vert ^{\alpha }}\,dx+ \int_{\Omega }\frac{(u_{\lambda }^{+})^{2_{s}^{*}(\alpha)}}{\vert z\vert ^{ \alpha }}\,dx+o(1). \end{aligned}$$
(18)
Since \(J_{\lambda }(u_{j})=c_{\lambda }+o(1)\), by (14) and (16)-(18), we obtain
$$ J_{\lambda }(u_{j})=J_{\lambda }(u_{\lambda })+ \frac{1}{2}\Vert w_{j}\Vert _{X_{0}^{s}}^{2}- \frac{1}{2_{s}^{*}(\alpha)} \int_{\Omega }\frac{(w _{j}^{+})^{2_{s}^{*}(\alpha)}}{\vert z\vert ^{\alpha }}\,dx=c_{\lambda }+o(1). $$
(19)
According to \(\langle J'_{\lambda }(u_{j}),u_{j}\rangle =o(1)\), (15) and (16)-(18), we get
$$ \Vert w_{j}\Vert _{X_{0}^{s}}^{2}- \int_{\Omega }\frac{(w_{j}^{+})^{2_{s}^{*}( \alpha)}}{\vert z\vert ^{2s}}\,dx=o(1). $$
(20)
Assume that \(\Vert w_{j}\Vert _{X_{0}^{s}}\rightarrow l\), it follows from (20) that
$$\int_{\Omega }\frac{(w_{j}^{+})^{2_{s}^{*}(\alpha)}}{\vert z\vert ^{2s}}\,dx \rightarrow l^{2}, $$
as \(j\rightarrow \infty \). From (4), one has
$$\Vert w_{j}\Vert _{X_{0}^{s}}^{2}\geq S_{\mu,\alpha } \biggl( \int_{\Omega }\frac{(w _{j}^{+})^{2_{s}^{*}(\alpha)}}{\vert z\vert ^{2s}}\,dx \biggr) ^{\frac{2}{2_{s} ^{*}(\alpha)}}. $$
We get \(l\geq S_{\mu,\alpha }^{\frac{N-\alpha }{2s-\alpha }}\). It follows from (19) and (20) that
$$\begin{aligned} c_{\lambda }+o(1) =& J_{\lambda }(u_{\lambda })+\frac{1}{2} \Vert w_{j}\Vert _{X_{0}^{s}}^{2}-\frac{1}{2_{s} ^{*}(\alpha)} \int_{\Omega }\frac{(w_{j}^{+})^{2_{s}^{*}(\alpha)}}{\vert z\vert ^{ \alpha }}\,dx+o(1) \\ \geq & \frac{2s-\alpha }{2(N-\alpha)} S_{\mu,\alpha }^{\frac{N-\alpha }{2s- \alpha }} -T_{\lambda } \vert \Omega \vert +o(1), \end{aligned}$$
which contradicts \(c_{\lambda }<\frac{2s-\alpha }{2(N-\alpha)} S_{ \mu,\alpha }^{\frac{N-\alpha }{2s-\alpha }}-T_{\lambda }\vert \Omega \vert \). Therefore, we have \(l=0\), which implies that \(u_{j}\rightarrow u_{ \lambda }\) in \(X_{0}^{s}\). This completes the proof of the Lemma 4. □