Existence of a mountain pass solution for a nonlocal fractional (p,q)$(p, q)$-Laplacian problem

*Correspondence: razani@sci.ikiu.ac.ir 1Department of Pure Mathematics, Faculty of Science, Imam Khomeini International University, P. O. Box 34149-16818, Qazvin, Iran Abstract Here, a nonlocal nonlinear operator known as the fractional (p,q)-Laplacian is considered. The existence of a mountain pass solution is proved via critical point theory and variational methods. To this aim, the well-known theorem on the construction of the critical set of functionals with a weak compactness condition is applied.


Introduction
The quasilinear operator (p, q)-Laplacian has been used to model steady-state solutions of reaction-diffusion problems arising in biophysics, in plasma physics and in the study of chemical reactions. These problems appear, for example, in a general reaction-diffusion system: where D(u) = |∇u| p-2 + |∇u| q-2 is the diffusion coefficient, the function u describes a concentration and the reaction term h(x, u) has a polynomial form with respect to the concentration u. The differential operator p + q is known as the (p, q)-Laplacian operator, if p = q, where j , j > 1 denotes the j-Laplacian and is defined by j u := div(|∇u| j-2 ∇u). It is not homogeneous, thus some technical difficulties arise in applying the usual methods of the theory of elliptic equations, for further details see [1,6,11,12]. When p = q, we obtain the p-Laplacian operator that was extensively studied by many authors; see [8, 14, 21-26, 28, 33-37]. Moreover, Marano et al. [27] studied the existence of solutions for the nonlinear elliptic problem of (p, q)-Laplacian type where ⊂ R N (N ≥ 1) is a bounded domain with boundary of class C 2 , 1 < q ≤ p < N , μ ∈ R + 0 . Recently, a great attention has been focused on the study of problems involving fractional operators of elliptic type, both in pure mathematical research and applications in real world, such as optimization, finance, population dynamics, minimal surfaces, water waves, game theory and so on. The literature on fractional operators and their applications to partially differential equations is quite large; for more details, see [3,29,35,42]. In [5] authors study a new fractional Sobolev space and applications to nonlocal variational problems with variable exponent.
Using the Leray-Schauder nonlinear alternative, Qiu et al. [30] proved the existence of solutions for the fractional p-Laplacian problem where is an open bounded domain in R N with the Lipschitz boundary, 1 < p < N r , 0 < r < 1 and f : × R → R is a Carathéodory function. Also, (-) r p is a nonlocal nonlinear operator known as the fractional p-Laplacian problem and is defined for u smooth enough by for every x ∈ R N , where B (x) is the ball in R N centered at x ∈ R N and with radius > 0. We refer to [10] for the details and history of this operator. For more details on nonlocal operators we refer to [39], where many properties of these operators are investigated. In [13] existence and multiplicity results for fractional (p, q)-Laplacian type equations in R N have been studied. Very recently, Bhakta et al. [7] studied the existence of infinitely many nontrivial weak solutions of the following fractional (p, q)-Laplacian equation: Moreover, the functions V and f satisfy in the following conditions: (A 1 ) V ∈ L ∞ ( ) and there exist σ , η > 0 such that V (x) > σ > 0 for every x ∈ and V (x)|u| m dx ≤ η u m s,m . (A 2 ) |f (x, t)| ≤ a 1 |t| α-1 + a 2 |t| β-1 for all x ∈ , t ∈ R, a 1 , a 2 > 0 and 1 < α, β < p * r .
Notice that, since f is odd in t, the energy functional I is even. It is worth noticing that without any symmetry assumption multiplicity results for some problems are not known, for example consider the following problem: where ⊂ R N is a bounded domain, λ ∈ R, g : R → R is a continuous function and 0 < λ 1 < λ 2 < · · · denote the eigenvalues of -. If g is odd function, then for λ k < λ < λ k+1 problem (1.1) has at least k pairs of distinct nontrivial solutions. If g is not odd, multiplicity results fail (see [41, page 147]). However, Dancer [15] showed that in general even for large enough λ one can expect no more than four nontrivial solutions. If λ = 1, for odd nonlinearities Ambrosetti and Rabinowitz [2] and Rabinowitz [31] studied this kind of problems. In this work, the methods depend on the use of Lusternik-Schnirelman theory or rather on the concept of the genus for symmetric sets. Hence, the fact that energy functional is even is essential for applying these techniques. A natural and open question is to know whether the infinitely many solutions hold under perturbations of the odd equation. For more details of perturbations, we refer the interested reader to [4].
In this paper, we study a quasilinear problem, that is, a fractional (p, q)-Laplacian elliptic problem as 1 where the value of λ 1 is given in Sect. 3. In addition, f : ×R → R is a Carathéodory function with regularity assumptions on . Due to nonlocality of the operator (-) r p , in this kind of problems, Dirichlet boundary condition u = 0 given in R N \ and not simply on ∂ . In this case, more careful analysis is needed. This paper is motivated by results of Servadei et al. [39], where they proved the existence of mountain pass type solutions in a Hilbert space for where λ ∈ R, L k is a nonlocal operator and f satisfies superlinear and subcritical growth conditions at zero and at infinity.
Here, we prove that problem (1.2) has a mountain pass type solution in the fractional Sobolev space which is not a Hilbert space. To this aim, some definitions and propositions are recalled in the sequel.
The norm in L p (R N ) is By [16], for any p ∈ (1, ∞) and r ∈ (0, 1), the fractional Sobolev space W r,p (R N ) can be defined as We consider the standard fractional Sobolev space with respect to the localized norm Specially, restriction to of any function in W r,p 0 ( ) belongs to the closure of C ∞ 0 ( ) in W r,p ( ). From [8], if rp ≤ 1 for the seminorm localized on × Poincaré inequality with |u(x)| p dx is not true.
The space X r,p ( ) is endowed with the following norm: It is worth noticing that in general W r,p ( ) is not the same as X r,p ( ) as × is strictly contained in Q. Now, we define or equivalently as C ∞ 0 ( ) X r,p ( ) . Since u = 0 in R N \ , the Gagliardo norm can be defined as follows: for all measurable functions u : R N − → R. Fractional Sobolev-type spaces are also called Aronszajn, Gagliardo or Slobodeckij spaces, by the names of the ones who first introduced them [40].
Notations: For simplicity of notations, we set · p := · L p ( ) , X := X r,p 0 ( ), · r,p := · X r,p 0 ( ) and X * := X -r,p 0 ( ) the dual space of X, where 1 p + 1 p = 1. The natural function space to study fractional (p, q)-Laplacian problems in the nonlocal framework is fractional Sobolev space X. It is easy to check that (X, · r,p ) is a uniformly convex Banach space. Here we consider the following problem: for all u, v ∈ X. It is clear that (1.4) is the weak formulation of (1.2). The function F is the primitive of f and it is defined as follows: for every (x, ξ ) ∈ × R, then the energy functional I λ : X → R associated with (1.2) is defined by for all u ∈ X. Also we assume that the Carathéodory function f : × R → R satisfies the following conditions: Here is the main result of this paper.

Theorem 1.1
Let be an open bounded set of R N with Lipschitz boundary, 0 < s < r < 1 < q < p < N r and λ < λ 1 , where the value of λ 1 is given in Sect. 3. Consider the following equation: for any v ∈ X. If f is a Carathéodory function verifying (f 1 )-(f 3 ), then for every γ > 0 problem (1.6) admits a mountain pass type solution u ∈ X which is not identically zero.
The paper is organized as follows: Sect. 2, is devoted to proving some regularities on the space X and the function f . In Sect. 3, we recall some properties of fractional eigenvalue problems. Finally, in Sect. 4, we show by some propositions that the conditions of the mountain pass theorem hold and we prove the existence of a solution of problem (1.2).

Preliminaries
In this section, we start with some preliminary lemmas and results on X and the functional f .

Fractional eigenvalue problems
In this section, we consider the generalized eigenvalue problem as follows: where ⊂ R N is a smooth bounded domain, λ ∈ R, 0 < s < r < 1 < q < p, γ > 0 and j ∈ {p, q}.
Finally, we define the concept of eigenfunction.

Definition 3.1
We say that u = 0, u ∈ X, is a (r, p)-eigenfunction of λ 1 (r, p), if for all func- The real number λ 1 (r, p) is called the (r, p)-eigenvalue. Set and The number η 1 is called the first generalized eigenvalue of (3.1). Notice that eigenvalues are positive numbers. One can prove the following proposition by almost the same argument as [17, Proposition 1].
Proof The proof is similar to [39,Lemma 10] and it is omitted.

A mountain pass type solution
Here, we study the existence of a mountain pass type solution of (1.2). Notice that problem (1.2) has a variational structure and the functional I λ is Fréchet differentiable for u ∈ X and every v ∈ X, Hence, weak solutions of (1.2) are critical points of the functional I λ . Thus one can seek weak solutions as critical points by applying mountain pass theorem (see [2,32]). In order to apply mountain pass theorem, we show that the functional I λ satisfies in the Palais-Smale compactness condition and has a particular geometric structure. For this purpose, we start by proving that the functional I λ satisfies the Palais-Smale compactness condition.

Proposition 4.1 Suppose that λ < λ 1 and f is a Carathéodory function satisfying (f 1 )-(f 3 ).
Let c ∈ R and {u n } be a sequence in X such that and as n → +∞. Then {u n } is bounded in X.
Proof Equations (4.1) and (4.2) imply there exists σ > 0 such that for any n and ≺I λ (u n ), u n u n r,p ≤ σ . (4.4) Now, by Lemma 2.2 and = 1 we get Applying Proposition 3.3, then by (f 3 ) and (4.5) we have By (4.6) and (4.7) for every n ∈ N, we have u n p r,p ≤ σ * 1 + where σ * is a suitable positive constant. Thus the proof is completed. Now, we define the operator A : X → X * as follows: for every u, v ∈ X. We know that the function u ∈ X is a weak solution of problem (1.2) such that By [20,28] it is easy to observe that A verifies the compactness condition as follows: (S) If {u n } be a sequence in X such that u n u (weakly) in X and ≺A(u n ), u nu → 0 as n → ∞, then u n → u in X as n → ∞. Now by property (S), one can prove the next result.

Proposition 4.2 Let f be a Carathéodory function satisfying (f 1 )-(f 3 ).
Let {u n } be a sequence in X such that {u n } is bounded in X and (4.2) holds. Then there exists u ∈ X such that, up to a subsequence, u nu r,p → 0 as n → +∞.
Proof Since X is a reflexive space and {u n } is bounded in X, up to a subsequence if necessary, still denoted by {u n }, there exists u ∈ X such that u n u in X. By Remark 2.1, up to a subsequence u n → u in L k R N , u n → u a.e. in R N , (4.8) as n → +∞ and there exists l ∈ L k (R N ) such that u n (x) ≤ l(x) a.e. in R N , for all n ∈ N, (4.9) where k ∈ [1, p * r [; for instance, see [9]. By (f 1 ), (4.8), (4.9), the fact that the map t → f (·, t) is continuous in t ∈ R and the dominated convergence theorem one can obtain Now, we recall Simon's inequality [18]: for any ζ , η ∈ R there exists C m > 0 such that |ζ -η| 2 (|ζ | m + |η| m ) m-2 m if 1 < m < 2. (4.11) From (4.11), it follows that the operator A is strictly monotone. Suppose C p,q = max{C p , C q }. By inequalities (4.11), if q ≥ 2, we get u nu p r,p ≤ u nu p r,p + γ u nu q s,q ≤ C p ≺A p u n -A p u, u nu + C q γ ≺A q u n -A q u, u nu ≤ C p,q ≺Au n -Au, u nu . (4.12) If 1 < q < p < 2, we obtain C p,q ≺Au n -Au, u nu ≥ C p ≺A p u n -A p u, u nu + C q γ ≺A q u n -A q u, u nu where C > 0 is a suitable constant. If 1 < q < 2 ≤ p, we have C p,q ≺Au n -Au, u nu ≥ C p ≺A p u n -A p u, u nu + C q γ ≺A q u n -A q u, u nu ≥ u nu p r,p + γ u nu 2 s,q u n q s,q + u q s,q q-2 q ≥ u nu p r,p . (4.15) Moreover, by (4.2), (4.8) and (4.10), we obtain ≺A(u n ), u nu = ≺I λ (u n ), u nu + λ u nu p p + f (x, u nu)(u nu) dx → 0.