Some results on the eigenvalue problem for a fractional elliptic equation

Abstract

This paper deals with the eigenvalue problem for a fractional variable coefficients elliptic equation defined on a bounded domain. Compared to the previous work, we prove a quite different variational formulation of the first eigenvalue for the above problem. This allows us to give a variational proof of the fractional Faber–Krahn inequality by employing suitable rearrangement techniques.

1 Introduction

In this paper, we study the eigenvalue problem for a fractional elliptic equation

$$ \textstyle\begin{cases} L^{s}u=\lambda u & \text{in }\varOmega , \\ u=0 & \text{in }\mathbb{R}^{N}\setminus \varOmega , \end{cases}\displaystyle \hspace{227pt} {(P_{\lambda })} $$

where $0< s<1$, Ω is an open bounded subset of $\mathbb{R}^{N}$ and $N>2s$; L is an elliptic operator in divergence form $L{u}=- \operatorname{div}(A(x)\nabla u)$. Here $A(x)= \{a_{ij}(x)\}$ is a symmetric matrix with $a_{ij}\in W^{1,\infty }(\mathbb{R}^{N})$, satisfying the uniformly ellipticity condition $A(x)\xi \cdot \xi \geq \varLambda \vert \xi \vert ^{2}$ for all $\xi \in \mathbb{R}^{N}$, a.e. $x\in \mathbb{R} ^{N}$ and for some constant Λ.

Fractional powers of elliptic operators, whose basic case is the fractional Laplacian $(-\Delta )^{s}$, arise naturally in many applications, for instance, the obstacle problem that appears in the study of the configuration of elastic membranes, anomalous diffusion, the so-called quasi-geostrophic flow problem, and pricing of American options, as well as many modern physical problems when considering fractional kinetics and anomalous transport, strange kinetics, and Lévy processes in quantum mechanics; one can see [1,2,3] and the references therein. When working in the whole space domain $\mathbb{R}^{N}$, there are several equivalent definitions of the fractional Laplacian operator $(-\Delta )^{s}$, classical references being [4,5,6]. However, when working on a bounded domain Ω, things get complicated because there are different options for defining $(-\Delta )^{s}$. A particular one is to define the fractional Laplacian as the Dirichlet-to-Neumann map, through an extended function defined in a cylinder $\mathcal{C}_{\varOmega }^{+}=\varOmega \times (0,+\infty )\subset \mathbb{R}^{N+1}$ whose values are assigned to zero on the lateral boundary of $\mathcal{C}_{\varOmega }^{+}$, as was proposed in [7, 8]. This allows reducing nonlocal problems involving $(-\Delta )^{s}$ to suitable local problems, defined in one more space dimension. A similar definition of the fractional elliptic operator $L^{s}$ in a bounded domain is given in [9]. However, this definition seems to be contrary to the nonlocal feature of fractional operators, and thus there are some restrictions on its validity. In [10], the authors take a more usual approach to define $(-\Delta )^{s}$ in a bounded domain. It consists in keeping the definition of fractional Laplacian in $\mathbb{R}^{N}$ through the extension method but asking the functions $u(x)$ to vanish outside of Ω, which seems to be more natural in many applications. In this paper, we will take this approach to define the fractional elliptic operator $L^{s}$ in bounded domains (see Sect. 2).

The study of eigenvalue problems is a classical topic and there are lots of results on local eigenvalue problems (see, for instance, [11,12,13,14] and the references therein). Recently, great attention has been focused on studying of eigenvalue problems involving fractional operators. Results for fractional linear operators were obtained in [15], where variational formulations of eigenvalues and some properties of eigenfunctions were proved. In [16,17,18], the eigenvalue problem associated with the fractional nonlinear operator $(-\Delta )^{s}_{p}$ was studied, and particularly some properties of the first eigenvalue and of the higher order eigenvalues were obtained. Then, Iannizzotto and Squassina [19] proved some Weyl-type estimates for the asymptotic behavior of variational eigenvalues corresponding to $(-\Delta )^{s}_{p}$.

More recently, by employing rearrangement techniques, Sire, Vázquez and Volzone [10] proved the Faber–Krahn inequality in two ways for the first eigenvalue of the fractional Laplacian operator in bounded domains. A variational proof was provided, which seems to be simpler, based on the variational characterization of the first eigenvalue and nonlocal Pólya–Szegö inequality. However, they pointed out that for the fractional variable coefficients problem $(P_{\lambda })$, it is not clear how to use the variational approach to prove such an inequality since in this case the variational formulations of the first eigenvalue given before does not seem to allow Pólya–Szegö inequality to be applied. To solve the above problem, in this paper, with the help of the extension problem of $(P_{\lambda })$ defined in one more space dimension, we will prove a quite different variational formulation of the first eigenvalue, in which the operator $L^{s}$ is associated to a norm satisfying Pólya–Szegö inequality. Then using some properties of rearrangement, a variational proof of fractional Faber–Krahn inequality can be achieved for problem $(P_{\lambda })$.

Many other fractional problems were also actively studied in recent years, such as fractional Kirchhoff type problems, fractional Schrödinger problems, and also Brézis–Nirenberg problem for fractional operators. Interested readers can refer to [20,21,22,23,24,25] for details.

This paper is organized as follows: In Sect. 2, we give all the necessary functional background related to problem $(P_{\lambda })$, which is naturally connected to the very definition of the operator $L^{s}$. In Sect. 3, the variational formulation of the first eigenvalue and some properties of eigenfunctions are obtained, while Sect. 4 is devoted to proving fractional Faber–Krahn inequality.

2 Preliminaries

In this section, we provide a self-contained description of the functional background which is necessary for the well-posedness of problem $(P_{\lambda })$. For further details, one can see [3, 9, 10, 26,27,28] and the references therein.

In the last years, there has been a growing interest in the study of nonlinear problems involving fractional powers of the Laplace operator $(-\bigtriangleup )^{s}$, $0< s<1$. The fractional Laplace of a function u is defined via Fourier transform and it can be expressed by

$$ (-\bigtriangleup )^{s}u(x)=c_{N,s}\mathrm{P.V.} \int _{\mathbb{R}^{N}} \frac{u(x)-u(y)}{ \vert x-y \vert ^{N+2s}}\,dy, $$

(2.1)

where $c_{N,s}$ is a positive constant. Observe from (2.1) that the fractional Laplacian is a nonlocal operator. This fact does not allow us to apply local PDE techniques to treat nonlinear problems for $(-\bigtriangleup )^{s}$. To overcome this difficulty, L. Caffarelli and L. Silvestre showed in [6] that any fractional power of the Laplacian can be determined as an operator that maps a Dirichlet boundary condition to a Neumann-type condition via an extension problem.

However, for a general fractional operator $L^{s}$, the Fourier transform is not available, which necessitates finding a language that can explain in a clear and unified way the concepts of $L^{s}$. Stinga and Torrea [3, 27] gave the following semigroup formula:

$$ L^{s}u(x)=\frac{1}{\varGamma (-s)} \int _{0}^{\infty } \bigl(e^{-tL}f(x)-f(x) \bigr) \frac{dt}{t ^{1+s}}, $$

(2.2)

where $\{e^{-tL}u\}_{t>0}$ is the heat diffusion semigroup generated by L and Γ is the Gamma function. When $L=-\triangle$, the above formula reduces to (2.1). Moreover, they proved that the fractional operators (2.2) can be described as Dirichlet-to-Neumann maps for an extension problem in the spirit of [6], which generalizes the Caffarelli–Silvestre result. In fact, let E be an open subset of $\mathbb{R}^{N}$ and L denote a nonnegative and self-adjoint linear second order partial differential operator defined in $L^{2}(E)$. For $u\in \operatorname{Dom}(L^{s})$, consider the following extension problem to the upper half space:

$$ \textstyle\begin{cases} -Lw+\frac{1-2s}{y}w_{y}+w_{yy}=0 & \text{in }E \times (0,+\infty ), \\ w(x,0)=u(x), & x\in E. \end{cases} $$

(2.3)

Then for the solution $w(x,y)$ of problem (2.3), it is proved that

$$ -\frac{1}{k_{s}}\lim_{y\rightarrow 0^{+}}y^{1-2s} \frac{\partial w}{\partial y}(x,y)=L^{s}u(x), $$

(2.4)

where $k_{s}=\frac{2s\vert \varGamma (-s)\vert }{4^{s}\varGamma (s)}$. See also [3, 9, 27] for a detailed account on this question.

In this paper, as was done for $(-\bigtriangleup )^{s}$ in [10], we keep the definition of fractional operator $L^{s}$ in $\mathbb{R}^{N}$ through the above extension method but ask the functions $u(x)$ to vanish outside of Ω. So, let $E= \mathbb{R}^{N}$ in (2.3) and denote $\mathcal{C}^{+}=\mathbb{R} ^{N}\times (0,+\infty )$. Using the extension problem (2.3) and expression (2.4), the nonlocal problem $(\mathrm{P}_{\lambda })$ is reformulated in a local way as follows:

$$ \textstyle\begin{cases} -Lw+\frac{1-2s}{y}w_{y}+w_{yy}=0 & \text{in }\mathcal{C}^{+}, \\ w(x,0)=0, & x\in \mathbb{R}^{N}\setminus \varOmega , \\ -\frac{1}{k_{s}}\lim_{y\rightarrow 0^{+}}y^{1-2s}\frac{ \partial w}{\partial y}(x,y)=\lambda u(x), & x\in \varOmega . \end{cases} $$

(2.5)

Note that operator L has the form of $-\operatorname{div}(A(x)\nabla )$. Then denoting by $B(x)=\left(\begin{array}{cc}A(x)& 0\\ 0& 1\end{array}\right)$, problem (2.5) is equivalent to the problem

$$ \textstyle\begin{cases} \operatorname{div}(y^{1-2s}B(x)\nabla w)=0 & \text{in }\mathcal{C}^{+}, \\ w(x,0)=0, & x\in \mathbb{R}^{N}\setminus \varOmega , \\ -\frac{1}{k_{s}}\lim_{y\rightarrow 0^{+}}y^{1-2s}\frac{ \partial w}{\partial y}(x,y)=\lambda u(x), & x\in \varOmega . \end{cases} $$

(2.6)

In order to introduce the concept of weak solution to problem (2.6), it is convenient to define the weighted energy space

$$ X^{s} \bigl(\mathcal{C}^{+} \bigr)= \biggl\{ w\in H^{1}_{\mathrm{loc}} \bigl(\mathcal{C}^{+} \bigr): \int _{\mathcal{C}^{+}}y^{1-2s} \bigl\vert \nabla w(x,y) \bigr\vert ^{2} \,dx \,dy< +\infty \biggr\} , $$

equipped with the norm

$$ \Vert w \Vert _{X^{s}(\mathcal{C}^{+})}= \biggl( \int _{\mathcal{C}^{+}}y^{1-2s} \bigl\vert \nabla w(x,y) \bigr\vert ^{2} \,dx \,dy \biggr)^{\frac{1}{2}}. $$

Now, let us define the space of all functions in $X^{s}(\mathcal{C} ^{+})$ whose trace over $\mathbb{R}^{N}$ vanishes outside of Ω, namely

$$ X^{s}_{\varOmega } \bigl(\mathcal{C}^{+} \bigr)= \bigl\{ w \in X^{s} \bigl(\mathcal{C}^{+} \bigr): w\vert _{\mathbb{R}^{N}\times {\{0\}}} \equiv 0\text{ in }\mathbb{R}^{N} \setminus \varOmega \bigr\} , $$

which gives a proper meaning of solutions to problem (2.6) in a bounded domain Ω.

From a detailed discussion in [26], we know that the domain of the natural fractional Laplacian $(-\Delta )^{s}$ is the space $\mathcal{H}(\varOmega )$ defined by

$$ \mathcal{H}(\varOmega )= \textstyle\begin{cases} H^{s}(\varOmega ) & \text{if }0< s< \frac{1}{2}, \\ H_{00}^{\frac{1}{2}}(\varOmega ) & \text{if }s=\frac{1}{2}, \\ H_{0}^{s}(\varOmega ) & \text{if }\frac{1}{2}< s< 1, \end{cases} $$

(2.7)

where $H^{s}(\varOmega )$ and $H_{0}^{s}(\varOmega )$ are classical fractional Sobolev spaces (see [29]) and

$$ H_{00}^{\frac{1}{2}}(\varOmega )= \biggl\{ u\in H^{\frac{1}{2}}( \varOmega ): \int _{\varOmega }\frac{u^{2}(x)}{d^{2}(x)}\, dx< \infty \biggr\} $$

with $d(x)=\operatorname{dist}(x,\partial \varOmega )<\infty$. It turns out that $\mathcal{H}(\varOmega )= \{w\vert _{\varOmega \times {\{0\}}}: w\in X^{s} _{\varOmega }(\mathcal{C}^{+}) \}$. In addition, we have the following compact embedding.

Lemma 2.1

Let $1\leq q<2_{s}^{\sharp }=\frac{2N}{N-2s}$. Then, $\operatorname{Tr}_{ \varOmega }(X^{s}_{\varOmega }(\mathcal{C}^{+}))$ is compactly embedded in $L^{q}(\varOmega )$.

Proof

We know that the trace $\operatorname{Tr}_{\varOmega }(X^{s}_{\varOmega }( \mathcal{C}^{+}))=\mathcal{H}(\varOmega )\subset H^{s}(\varOmega )$ and $H^{s}(\varOmega )\subset \subset L^{q}$ when $1\leq q<2_{s}^{\sharp }$, see [28]. Here ⊂⊂ denotes compact embedding. This completes the proof of the lemma. □

Then according to [9, 10], the following definition of weak solution to problem (2.6) is provided.

Definition 2.1

We say that $w\in X^{s}_{\varOmega }(\mathcal{C}^{+})$ is a weak solution of (2.6) if for any $\varphi \in X^{s}_{\varOmega }(\mathcal{C} ^{+})$, it holds:

$$ \int _{\mathcal{C}^{+}}y^{1-2s}B(x)\nabla w(x,y)\nabla \varphi (x,y) \,dx \,dy= \lambda \int _{\varOmega }w(x,0)\varphi (x,0)\,dx. $$

If w is a solution to the extended problem (2.6), then the trace function $u=\operatorname{Tr}_{\varOmega }(w):=w(x,0)$ will be called a weak solution to problem $(P_{\lambda })$.

Remark 2.1

It is easy to see that the function $u=\operatorname{Tr}_{\varOmega }(w)$ belongs to the space $\mathcal{H}(\varOmega )$.

3 General results about eigenvalues

Theorem 3.1

The first eigenvalue of problem $(P_{\lambda })$ is positive and can be characterized as follows:

$$ \lambda _{1}=\min_{\substack{w\in X_{\varOmega }^{s}(\mathcal{C}^{+}) \\ \Vert w(x,0)\Vert _{L^{2}(\varOmega )}=1}} \int _{\mathcal{C}^{+}}y^{1-2s}B(x) \nabla w\cdot \nabla w \,dx \,dy, $$

(3.1)

or equivalently,

$$ \lambda _{1}=\min_{\substack{w\in X_{\varOmega }^{s}(\mathcal{C}^{+}) \\ w(x,0)\neq 0}}\frac{\int _{\mathcal{C}^{+}}y^{1-2s}B(x)\nabla w\cdot \nabla w \,dx \,dy}{\int _{\varOmega } \vert w(x,0) \vert ^{2}\,dx}. $$

Moreover, there exists a nonegative function $e_{1}\in X_{\varOmega } ^{s}(\mathcal{C}^{+})$ attaining the minimum in (3.1), and then $e_{1}(x,0)$ is a nonnegative eigenfunction of problem $(P_{\lambda })$ corresponding to $\lambda _{1}$.

Proof

Let $J: X_{\varOmega }^{s}(\mathcal{C}^{+})\rightarrow \mathbb{R}$ be the functional defined as follows:

$$ J(w)= \int _{\mathcal{C}^{+}}y^{1-2s}B(x)\nabla w\cdot \nabla w \,dx \,dy, \quad w\in M, $$

(3.2)

where $M=\{w\in X_{\varOmega }^{s}(\mathcal{C}^{+}):\Vert w(x,0)\Vert _{L^{2}( \varOmega )}=1\}$. Take a minimizing sequence $\{w_{j}\}_{j\in N}$ for J on M, that is, a sequence $\{w_{j}\}_{j\in \mathbb{N}}\subset M$ such that

$$ J(w_{j})\rightarrow \inf_{w\in M}J(w). $$

Then the sequence $\{J(w_{j})\}_{j\in N}$ is bounded. So by the uniform ellipticity condition, we have

$$ J(w_{j})\geq \min \{\varLambda , 1\} \Vert w_{j} \Vert ^{2}_{X_{\varOmega }^{s}(C^{+})}, $$

which implies $\{\Vert w_{j}\Vert ^{2}_{X_{\varOmega }^{s}(C^{+})}\}_{j\in N}$ is also bounded.

Since $X_{\varOmega }^{s}(\mathcal{C}^{+})$ is a reflexive space, up to a sequence, still denoted by $w_{j}$, we have that $\{w_{j}\}_{j\in N}$ converges weakly in $X_{\varOmega }^{s}(\mathcal{C}^{+})$ to some e. By Lemma 2.1, we deduce that

$$ w_{j}(x,0)\rightarrow e(x,0) \quad \text{in }L^{2}( \varOmega ) $$

as $j\rightarrow +\infty$. So, $\Vert e(x,0)\Vert _{L^{2}(\varOmega )}=1$, that is, $e\in M$. Observe that the functional J is continuous and convex in $X_{\varOmega }^{s}(\mathcal{C}^{+})$, which guarantees that J is weakly lower semicontinuous in $X_{\varOmega }^{s}(\mathcal{C}^{+})$. Then

$$ \inf_{w\in M}J(w)=\lim_{j\to \infty }J(w_{j}) \geq J(e)\geq \inf_{w \in M}J(w). $$

So that, $J(e)=\inf_{w\in M}J(w)$.

Let $e_{1}=\vert e\vert$, then $e_{1}\geq 0$. Also, $e_{1}\in X_{\varOmega }^{s}( \mathcal{C}^{+})$ and $\Vert e_{1}(x,0)\Vert _{L^{2}(\varOmega )}=\Vert e(x,0)\Vert _{L^{2}(\varOmega )}=1$, which imply $e_{1}\in M$. Moreover,

$$\begin{aligned} J(e_{1}) &= \int _{\mathcal{C}^{+}}y^{1-2s}B(x)\nabla e_{1}\cdot \nabla e_{1} \,dx \,dy \\ &= \int _{\mathcal{C}^{+}}y^{1-2s}B(x)\nabla e\cdot \nabla e \,dx \,dy \\ &=J(e)=\inf_{w\in M}J(w). \end{aligned}$$

Now, let’s prove that the first eigenvalue $\lambda _{1}=J(e_{1})$. Let $\varepsilon \in (-1,1)$, $\varphi \in X_{\varOmega }^{s}(\mathcal{C}^{+})$ and

$$ u_{\varepsilon }=\frac{e_{1}+\varepsilon \varphi }{ \Vert e_{1}(x,0)+ \varepsilon \varphi (x,0) \Vert _{L^{2}(\varOmega )}}\in M. $$

Then

$$ \begin{aligned}[b] J(u_{\varepsilon }) &=\frac{\int _{\mathcal{C}^{+}}y^{1-2s}B(x)\nabla (e _{1}+\varepsilon \varphi )\cdot \nabla (e_{1}+\varepsilon \varphi ) \,dx \,dy}{ \Vert e_{1}(x,0)+\varepsilon \varphi (x,0) \Vert ^{2}_{L^{2}(\varOmega )}} \\ &=\frac{\int _{\mathcal{C}^{+}}y^{1-2s}B(x)\nabla e_{1}\cdot \nabla e _{1} \,dx \,dy+2\varepsilon \int _{\mathcal{C}^{+}}y^{1-2s}B(x)\nabla e_{1} \cdot \nabla \varphi \,dx \,dy+o(\varepsilon )}{1+2\varepsilon \int _{ \varOmega }e(x,0)\varphi (x,0)\,dx+o(\varepsilon )}. \end{aligned} $$

(3.3)

Multiplying the numerator and the denominator by $1-2\varepsilon \int _{\varOmega }e(x,0)\varphi (x,0)\,dx+o(\varepsilon )$ in the above equality, we obtain

$$ \begin{aligned}[b] &J(u_{\varepsilon }) \bigl(1+o(\varepsilon ) \bigr) \\ &\quad = \int _{\mathcal{C}^{+}}y^{1-2s}B(x)\nabla e_{1}\cdot \nabla e_{1} \,dx \,dy+2 \varepsilon \int _{\mathcal{C}^{+}}y^{1-2s}B(x)\nabla e_{1}\cdot \nabla \varphi \,dx \,dy \\ & \quad \quad{} -2\varepsilon \int _{\mathcal{C}^{+}}y^{1-2s}B(x)\nabla e_{1}\cdot \nabla e_{1} \,dx \,dy \int _{\varOmega }e_{1}(x,0)\varphi (x,0)\,dx+o(\varepsilon ). \end{aligned} $$

(3.4)

This and the minimality of $e_{1}$ imply that

$$ \int _{\mathcal{C}^{+}}y^{1-2s}B(x)\nabla e_{1}\cdot \nabla \varphi \,dx \,dy-J(e _{1}) \int _{\varOmega }e_{1}(x,0)\varphi (x,0)\,dx=0. $$

Thus, $\lambda _{1}=J(e_{1})=\min_{w\in M}J(w)$, which shows that (3.1) holds. Notice that $J(e_{1})>0$, because otherwise we would have $e_{1}\equiv 0\notin M$. Hence, $\lambda _{1}>0$ and $e_{1}(x,0)$ is a nonnegative eigenfunction corresponding to $\lambda _{1}$. The proof of Theorem 3.1 is completed. □

4 Faber–Krahn inequality

Theorem 4.1

If $\lambda _{1}$ is the first eigenvalue of problem $(P_{\lambda })$, then

$$ \lambda _{1}\geq \lambda _{1}^{\sharp }, $$

where $\lambda _{1}^{\sharp }$ is the first eigenvalue of the problem

$$ \textstyle\begin{cases} (-\varLambda \Delta )^{s}v=\lambda v & \textit{in }\varOmega ^{\sharp }, \\ v=0 & \textit{on }\mathbb{R}^{N}\setminus \varOmega ^{\sharp } \end{cases}\displaystyle \hspace{201pt} {({Q}_{\lambda })} $$

and $\varOmega ^{\sharp }$ is the ball centered at the origin such that $\vert \varOmega ^{\sharp }\vert =\vert \varOmega \vert$. Furthermore, $\lambda _{1}=\lambda _{1} ^{\sharp }$ if and only if $\varOmega =\varOmega ^{\sharp }$ and $\sum a_{ij}(x)x _{j}=\varLambda x_{i}$ a.e. in $\mathbb{R}^{N}$ modulo translations.

Remark 4.1

In the case $s=1$ and $N=2$, the above result is known as the Faber–Krahn theorem, which can be stated as follows: a membrane with the lowest principle frequency is the circular one.

Remark 4.2

The extension problem associated with $(Q_{\lambda })$ can be written as

$$ \textstyle\begin{cases} \operatorname{div}(y^{1-2s}\widetilde{B}(x)\nabla w)=0 & \text{in }\mathcal{C}^{+}, \\ w(x,0)=0, & x\in \mathbb{R}^{N}\setminus \varOmega ^{\sharp }, \\ -\frac{1}{k_{s}}\lim_{y\rightarrow 0^{+}}y^{1-2s}\frac{\partial w}{\partial y}=\lambda v(x), & x\in \varOmega ^{\sharp }, \end{cases} $$

(4.1)

where matrix $\widetilde{B}(x)$ is diagonal with values Λ for the first N diagonal elements and 1 for the remaining element.

Before proving the theorem, we recall some basic notions of rearrangements and some related fundamental properties. Let E be an open subset of $\mathbb{R}^{N}$ (which may be the whole space) and $f:E\rightarrow \mathbb{R}$ be a measurable function. We define the distribution function $\mu _{f}$ of f as

$$ \mu _{f}(t)= \bigl\vert \bigl\{ x\in E: \bigl\vert f(x) \bigr\vert >t \bigr\} \bigr\vert , \quad t\geq 0, $$

and the decreasing rearrangement of f as

$$ f^{*}(s)=\inf \bigl\{ t\geq 0:\mu _{f}(t)\leq s \bigr\} , \quad s\in \bigl(0, \vert E \vert \bigr). $$

Furthermore, let $E^{\sharp }$ be the ball centered at the origin having the same Lebesgue measure as E and $C_{N}$ denote the measure of the unit ball. We define the function

$$ f^{\sharp }(x)=f^{*} \bigl(C_{N} \vert x \vert ^{N} \bigr), \quad x\in E^{\sharp }, $$

which is called the spherically symmetric rearrangement of f. For an exhaustive treatment of rearrangements, we refer to [30,31,32] and the references therein. Here we only state some properties which will turn useful for what follows.

• (i) Conservation of the $L^{p}$ norms:

  $$ \Vert f \Vert _{L^{p}(E)}= \bigl\Vert f^{\sharp } \bigr\Vert _{L^{p}(E^{\sharp })}, \quad 1\leq p< + \infty . $$

• (ii) Hardy–Littlewood inequality:

  $$ \int _{E} \bigl\vert f(x)g(x) \bigr\vert \,dx\leq \int _{E^{\sharp }}f^{\sharp }(x)g^{\sharp }(x)\,dx, $$

  where f, g are measurable functions on E.

• (iii) Pólya–Szegö inequality:

  $$ \bigl\Vert \nabla f^{\sharp } \bigr\Vert _{L^{p}(E^{\sharp })}\leq \Vert \nabla f \Vert _{L^{p}(E)}, \forall f\in W_{0}^{1,p}(E), \quad 1< p< +\infty . $$

Furthermore, the following lemma holds (see [33, 34]).

Lemma 4.1

Let us suppose that

$$ \Bigl\vert \Bigl\{ x:0< f^{\sharp }(x)< \operatorname{ess}\sup _{x\in E} \bigl\vert f(x) \bigr\vert , \bigl\vert \nabla f^{\sharp }(x) \bigr\vert =0 \Bigr\} \Bigr\vert =0. $$

Then if equality sign holds in Pólya–Szegö inequality, we have $\vert f\vert =f^{\sharp }$ a.e. up to translations.

When we deal with two-variable functions

$$ f:(x,y)\in \mathcal{C}^{+}_{E}\rightarrow \mathbb{R}, $$

where $\mathcal{C}^{+}_{E}=E\times (0,+\infty )$, it will be convenient to define the so-called Steiner symmetrization of f. Denote by $f^{*}(s,y)$ the decreasing rearrangement of f, with respect to x for y fixed. We define the function

$$ f^{\sharp }(x,y)=f^{*} \bigl(C_{N} \vert x \vert ^{N},y \bigr), $$

which is called the Steiner symmetrization of f, with respect to the line $x = 0$. Clearly, $f^{\sharp }$ is a spherically symmetric and decreasing function with respect to x, for any fixed y.

Proof of Theorem 4.1

Let e be a nonnegative eigenfunction corresponding to the first eigenvalue of $(P_{\lambda })$. Then by the uniformly ellipticity condition, we have

$$ \begin{aligned}[b] \lambda _{1} &= \frac{\int _{\mathcal{C}^{+}}y^{1-2s}B(x)\nabla e(x,y) \cdot \nabla e(x,y) \,dx \,dy}{\int _{\varOmega } \vert e(x,0) \vert ^{2}\,dx} \\ &\geq \frac{ \int _{0}^{+\infty }y^{1-2s}\,dy\int _{\mathbb{R}^{N}} (\varLambda \vert \nabla _{x}e(x,y) \vert ^{2} + \vert e_{y}(x,y) \vert ^{2} )\,dx}{ \int _{\varOmega } \vert e(x,0) \vert ^{2}\,dx}. \end{aligned} $$

(4.2)

By Pólya–Szegö inequality, we have

$$ \int _{\mathbb{R}^{N}} \bigl\vert \nabla _{x}e(x,y) \bigr\vert ^{2}\,dx\geq \int _{\mathbb{R}^{N}} \bigl\vert \nabla _{x}e^{\sharp }(x,y) \bigr\vert ^{2}\,dx. $$

(4.3)

Moreover, it follows that

$$ \int _{\mathbb{R}^{N}} \bigl\vert e_{y}(x,y) \bigr\vert ^{2}\,dx\geq \int _{\mathbb{R}^{N}} \bigl\vert e_{y}^{\sharp }(x,y) \bigr\vert ^{2}\,dx. $$

(4.4)

In fact,

$$\begin{aligned} \int _{\mathbb{R}^{N}} \bigl\vert e_{y}(x,y) \bigr\vert ^{2}\,dx &=\lim_{\tilde{y}\rightarrow y} \int _{\mathbb{R}^{N}} \biggl\vert \frac{e(x, \tilde{y})-e(x,y)}{\tilde{y}-y} \biggr\vert ^{2}\,dx \\ &=\lim_{\tilde{y}\rightarrow y}\frac{\int _{\mathbb{R}^{N}}[e ^{2}(x,\tilde{y})-2e(x,\tilde{y}) e(x,y)+e^{2}(x,y)]\,dx}{(\tilde{y}-y)^{2}}. \end{aligned}$$

By Hardy–Littlewood inequality, we conclude that

$$\begin{aligned} \int _{\mathbb{R}^{N}} \bigl\vert e_{y}(x,y) \bigr\vert ^{2}\,dx &\geq \lim_{\tilde{y}\rightarrow y}\frac{\int _{\mathbb{R}^{N}}[e^{\sharp ^{2}}(x,\tilde{y})-2e^{\sharp }(x,\tilde{y}) e^{\sharp }(x,y)+e^{ \sharp ^{2}}(x,y)]\,dx}{(\tilde{y}-y)^{2}} \\ &= \int _{\mathbb{R}^{N}}\lim_{\tilde{y}\rightarrow y} \biggl\vert \frac{e ^{\sharp }(x,\tilde{y})-e^{\sharp }(x,y)}{\tilde{y}-y} \biggr\vert ^{2}\,dx \\ &= \int _{\mathbb{R}^{N}} \bigl\vert e_{y}^{\sharp }(x,y) \bigr\vert ^{2}\,dx. \end{aligned}$$

Then combining (4.2)–(4.4) and the property (i) of rearrangements, we get

$$ \begin{aligned}[b] \lambda _{1} &= \frac{\int _{\mathcal{C}^{+}}y^{1-2s}B(x)\nabla e(x,y) \cdot \nabla e(x,y) \,dx \,dy}{\int _{\varOmega } \vert e(x,0) \vert ^{2}\,dx} \\ &\geq \frac{\int _{0}^{+\infty }y^{1-2s}\,dy \int _{\mathbb{R}^{N}} (\varLambda \vert \nabla _{x} e^{\sharp }(x,y) \vert ^{2} + \vert e_{y}^{\sharp }(x,y) \vert ^{2} ) \,dx}{\int _{\varOmega ^{\sharp }} \vert e^{\sharp }(x,0) \vert ^{2}\,dx} \\ & =\frac{\int _{\mathcal{C}^{+}}y^{1-2s}{\widetilde{B}}(x)\nabla e^{ \sharp }(x,y) \cdot \nabla e^{\sharp }(x,y) \,dx \,dy}{\int _{\varOmega ^{\sharp }} \vert e^{\sharp }(x,0) \vert ^{2}\,dx}. \end{aligned} $$

(4.5)

By Theorem 3.1, we obtain

$$ \lambda _{1}^{\sharp }=\min_{w\in X_{\varOmega ^{\sharp }}^{s}(\mathcal{C}^{+})} \frac{ \int _{\mathcal{C}^{+}}y^{1-2s}\widetilde{B}(x)\nabla w(x,y)\cdot \nabla w(x,y) \,dx \,dy}{\int _{\varOmega ^{\sharp }} \vert w(x,0) \vert ^{2}\,dx}. $$

Note that (4.3) and (4.4) imply that $e^{\sharp }\in X _{\varOmega ^{\sharp }}^{s}(\mathcal{C}^{+})$. Thus $\lambda _{1}\geq \lambda _{1}^{\sharp }$.

On the other hand, we observe that if $\lambda _{1}=\lambda _{1}^{ \sharp }$, then $\lambda _{1}^{\sharp }$ attains the minimum at the function $e^{\sharp }(x,y)$, that is, $e^{\sharp }(x,0)$ is an eigenfunction of problem $(Q_{\lambda })$ corresponding to the first eigenvalue $\lambda _{1}^{\sharp }$. Moreover, equalities hold through (4.2) and (4.5). Then the following equality holds:

$$ \int _{\mathbb{R}^{N}} \bigl\vert \nabla _{x}e(x,y) \bigr\vert ^{2}\,dx= \int _{\mathbb{R}^{N}} \bigl\vert \nabla _{x}e^{\sharp }(x,y) \bigr\vert ^{2}\,dx, \quad y\in (0,+\infty ) $$

(4.6)

because $\Vert e(x,0)\Vert _{L^{2}(\varOmega )}=\Vert e^{\sharp }(x,0)\Vert _{L^{2}( \varOmega ^{\sharp })}$. Since $e^{\sharp }$ is an eigenfunction of $(Q_{\lambda })$, we deduce from the regularity theory (see [7, 9]) that

$$ \Bigl\vert \Bigl\{ x:0< e^{\sharp }(x,y)< \sup_{x\in \mathbb{R}^{N}}e^{\sharp }(x,y), \bigl\vert \nabla _{x}e^{\sharp }(x,y) \bigr\vert =0 \Bigr\} \Bigr\vert =0, \quad \text{for } y\in (0,+ \infty ). $$

By Lemma 4.1, we get that $e^{\sharp }(x,y)=e(x,y)$ modulo translations. Thus, for every fixed $y>0$, function $e(x,y)$ is spherically symmetric up to translations. Taking the limit as $y\rightarrow 0$, we conclude that $e(x,0)$ is also spherically symmetric in $\mathbb{R}^{N}$. This means that the domain Ω, which is the positivity set of $e(x,0)$ in $\mathbb{R}^{N}$, must be a ball, that is, $\varOmega = \varOmega ^{\sharp }$ modulo translations. Finally, since the vector $\nabla _{x} e (x,y)=\nabla _{x} e^{\sharp }(x,y)$ points in the direction x for fixed y, we have that if the equality holds in (4.2), then

$$ A(x)x\cdot x=\varLambda \vert x \vert ^{2} \textit{ a.e.}\quad \text{in } \mathbb{R}^{N}, $$

that is, $A(x)x=\varLambda x$ a.e., which implies that $\sum a_{ij}(x)x _{j}=\varLambda x_{i}$ a.e. in $\mathbb{R}^{N}$ (see also [35, 36]). □

5 Conclusions

For the fractional variable coefficients elliptic operator defined on a bounded domain, it has been pointed out that the variational formulation of the first eigenvalue given before does not allow using a variational approach to prove the fractional Faber–Krahn inequality. In this paper, we proved a different variational formulation of the first eigenvalue for $(P_{\lambda })$. Following this, a variational proof of the fractional Faber–Krahn inequality has been achieved by employing suitable rearrangement techniques. Based on this point, our work is valuable.