In this section, the existence of nontrivial weak solutions for (\(P_{\lambda }\)) is shown by applying the mountain pass theorem and a variant of Ekeland variational principle under suitable assumptions.
Definition 3.1
We say that \(u\in E\) is a weak solution of (\(P_{\lambda }\)) if
$$\begin{aligned} &M \bigl([u]_{s,p(\cdot,\cdot)} \bigr) \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{{p(x,y)-2}}(u(x)-u(y))(w(x)-w(y))}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy \\ &\qquad+ \int_{\mathbb {R}^{N}} \mathcal{V}(x) \lvert u \rvert ^{p(x)-2}uw \,dx\\&\quad=\lambda \int_{\mathbb {R}^{N}}\rho(x) \vert u \vert ^{r(x)-2}u w\, dx+ \int_{\mathbb {R}^{N}}h(x,u) w\,dx \end{aligned}$$
for any \(w \in E\), where
$$[u]_{s,p(\cdot,\cdot)}:= \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{p(x,y)}}{p(x,y) \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy. $$
Let us define the functional \(\varPhi: E \to\mathbb {R}\) by
$$ \varPhi(u) = \mathcal{M} \bigl([u]_{s,p(\cdot,\cdot)} \bigr) + \int _{\mathbb {R}^{N}}\frac{\mathcal{V}(x)}{p(x)} \lvert u \rvert ^{p(x)}\,dx. $$
The following lemma can be proved by using arguments as in [43, Lemma 2].
Lemma 3.2
If (V) and (M1) hold, then the functional\(\varPhi:E\to\mathbb {R}\)is of class\({C}^{1}(E,\mathbb {R})\)and
$$ \begin{aligned}[b] \bigl\langle {\varPhi^{\prime}(u),w} \bigr\rangle &=M \bigl([u]_{s,p(\cdot,\cdot )} \bigr) \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{{p(x,y)-2}}(u(x)-u(y))(w(x)-w(y))}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy\hspace{-24pt} \\ &\quad+ \int_{\mathbb {R}^{N}}\mathcal{V}(x) \lvert u \rvert ^{p(x)-2}uw \,dx, \end{aligned} $$
(3.1)
for any\(u, w \in E\). Moreover, Φis weakly lower semi-continuous inE.
Proof
It is not difficult to prove that Φ has Fréchet derivative in E and (3.1) holds for any \(u, w \in E\). Now, let \(\{ z_{n} \}_{n} \subset E \) and \(z \in E \) satisfy \(z_{n} \to z\) strongly in E as \(n \to\infty\). Without loss of generality, we assume that \(z_{n} \to z\) a.e. in \(\mathbb {R}^{N}\). Then the sequence
$$\biggl\{ \frac{ \vert z_{n}(x) - z_{n}(y) \vert ^{{p(x,y)-2}}(z_{n}(x)- z_{n}(y))}{ \vert x-y \vert ^{(N+sp(x,y))/p^{\prime}(x,y)}} \biggr\} _{n} $$
is bounded in \(L^{p^{\prime}(\cdot,\cdot)}(\mathbb{R}^{N}\times \mathbb{R}^{N})\), as well as a.e. in \(\mathbb{R}^{N}\times\mathbb{R}^{N} \)
$$\begin{gathered} \mathcal{U}_{n} (x,y):=\frac{ \vert z_{n}(x) - z_{n}(y) \vert ^{{p(x,y)-2}}(z_{n}(x)- z_{n}(y))}{ \vert x-y \vert ^{(N+sp(x,y))/p^{\prime}(x,y)}} \\ \quad\longrightarrow\quad\mathcal{U}(x,y):=\frac{ \vert z(x) - z(y) \vert ^{{p(x,y)-2}}(z(x)- z(y))}{ \vert x-y \vert ^{(N+sp(x,y))/p^{\prime}(x,y)}}.\end{gathered} $$
Thus, the Brezis–Lieb lemma (see [9]) implies
$$ \begin{aligned}[b] &\lim_{n\to\infty} \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \bigl\vert \mathcal{U}_{n} (x,y) - \mathcal{U}(x,y) \bigr\vert ^{p^{\prime}(x,y)} \,dx \,dy \\ &\quad=\lim_{n\to\infty} \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \biggl( \frac{ \vert z_{n} (x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} - \frac { \vert z(x)-z(y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \biggr) \,dx \,dy. \end{aligned} $$
(3.2)
The fact that \(z_{n} \rightarrow z \) strongly in E yields
$$ \lim_{n\to\infty} \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \biggl( \frac{ \vert z_{n} (x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} - \frac { \vert z(x)-z(y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \biggr) \,dx \,dy = 0. $$
Moreover, the continuity of M implies that
$$ \lim_{n\to\infty} M \bigl([ z_{n} ]_{s,p(\cdot,\cdot)} \bigr) = M \bigl([z]_{s,p(\cdot,\cdot)} \bigr). $$
(3.3)
From (3.2) it follows that
$$ \lim_{n\to\infty} \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \bigl\vert \mathcal{U}_{n} (x,y) - \mathcal{U}(x,y) \bigr\vert ^{p^{\prime}(x,y)} \,dx \,dy = 0. $$
(3.4)
Similarly,
$$ \lim_{n\to\infty} \int_{\mathbb {R}^{N}} \mathcal{V}(x) \bigl\vert \bigl\vert z_{n}(x) \bigr\vert ^{{p(x)-2}}z_{n}(x)- \bigl\vert z(x) \bigr\vert ^{{p(x)-2}}z(x) \bigr\vert ^{p^{\prime}(x)} \,dx = 0. $$
(3.5)
Combining (3.3)–(3.5) with the Hölder inequality, we have
$$ \bigl\Vert \varPhi^{\prime}(z_{n} ) - \varPhi^{\prime}(z) \bigr\Vert _{E^{*}} = \sup_{w \in E, \Vert w \Vert _{E}=1}{ \bigl\vert \bigl\langle {\varPhi^{\prime}( z_{n} )- \varPhi^{\prime}(z),w} \bigr\rangle \bigr\vert } \longrightarrow0 $$
as \(n \rightarrow\infty\). Hence, \(\varPhi\in C^{1} (E, \mathbb {R} )\). Finally, notice that the map \(w \mapsto[w]_{s,p(\cdot,\cdot)}\) is lower semi-continuous in the weak topology of \(W^{s,p(\cdot,\cdot )}(\mathbb{R}^{N}) \) and \(\mathcal{M}\) is nondecreasing and continuous on \(\mathbb{R}_{0}^{+}\), so that \(w \mapsto\mathcal{M} ( [w]_{s,p(\cdot,\cdot)} )\) is lower semi-continuous in the weak topology of \(W^{s,p(\cdot,\cdot)}(\mathbb{R}^{N}) \). Indeed, we can define \(\gamma: W^{s,p(\cdot,\cdot)}(\mathbb{R}^{N}) \rightarrow\mathbb {R}\) as follows:
$$ \gamma(w)= \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \bigl\vert w(x)-w(y) \bigr\vert ^{{p(x,y)}} \vert x-y \vert ^{-N-sp(x,y)} \,dx \,dy. $$
It is easy to see that \(\gamma\in C^{1} ( W^{s,p(\cdot,\cdot )}(\mathbb{R}^{N}) )\) and γ is a convex functional in \(W^{s,p(\cdot,\cdot)}(\mathbb{R}^{N})\). By Corollary 3.8 in [13], we obtain \(\gamma(w) \leq\lim\inf_{n \rightarrow\infty} \gamma( w_{n} ) \). Hence, it is easy to see that Φ is weakly lower semi-continuous in E (see [46], Lemma 3.3 for more details). □
Let \(H(x,t)=\int_{0}^{t}h(x,s)\,ds\). Assume that:
-
(A1)
\(p, q, r\in C_{+}(\mathbb {R}^{N})\) and \(1< r_{-}\leq r_{+}< p_{-}\le p_{+}< q_{-}\le q_{+}< p_{s}^{*}(x)\) for all \(x\in\mathbb {R}^{N}\).
-
(A2)
\(0\leq\rho\in L^{\frac{p(\cdot)}{p(\cdot)- r(\cdot )}}(\mathbb {R}^{N})\cap L^{\infty}(\mathbb {R}^{N})\) with meas\(\{x\in \mathbb {R}^{N}:\rho(x)\neq0 \}>0\).
-
(H1)
\(h: \mathbb {R}^{N}\times\mathbb {R} \to\mathbb {R}\) satisfies the Carathéodory condition.
-
(H2)
There exists nonnegative function \(\sigma\in L^{\infty }(\mathbb {R}^{N})\) such that
$$\bigl\lvert h(x,t) \bigr\rvert \le\sigma(x) \lvert t \rvert^{q(x)-1}, $$
for all \((x,t)\in\mathbb {R}^{N}\times\mathbb {R}\).
-
(H3)
There exists a positive constant θ such that \(\theta> \vartheta p^{+}\) and
$$0< \theta H(x,t)\le h(x,t)t, \quad\text{for all } t \in\mathbb {R}\setminus\{0\} \text{ and } x \in\mathbb {R}^{N}, $$
where ϑ is given in (M2).
-
(H4)
\(H(x,t)\geq0\) for all \((x,t)\in\mathbb {R}^{N}\times\mathbb {R}\).
Let the functional \(\varPsi_{\lambda}:E\to\mathbb {R}\) be defined by
$$\varPsi_{\lambda}(u) = \lambda \int_{\mathbb {R}^{N}}\frac{\rho (x)}{r(x)} \lvert u \rvert^{r(x)}\,dx+ \int_{\mathbb {R}^{N}}H(x,u)\,dx. $$
Then it is easy to check that \(\varPsi_{\lambda} \in C^{1}(E,\mathbb {R})\), and its Fréchet derivative is
$$ \bigl\langle {\varPsi_{\lambda}^{\prime}}(u),w \bigr\rangle =\lambda \int_{\mathbb {R}^{N}}\rho \vert u \vert ^{r(x)-2}uw\,dx + \int_{\mathbb {R}^{N}}h(x,u)w\,dx, $$
for any \(u,w \in E\). Subsequently, the functional \(I_{\lambda}:E\to\mathbb {R}\) is defined by
$$ I_{\lambda}(u)=\varPhi(u)-\varPsi_{\lambda}(u). $$
(3.6)
Then according to Lemma 3.2, it follows that the functional \(I_{\lambda}\in C^{1}(E,\mathbb {R})\), and its Fréchet derivative is
$$\begin{aligned} \bigl\langle I_{\lambda}^{\prime}(u),w \bigr\rangle ={}&M \bigl( [u]_{s,p(\cdot,\cdot)} \bigr) \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{{p(x,y)-2}}(u(x)-u(y))(w(x)-w(y))}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy \\ &+ \int_{\mathbb {R}^{N}}\mathcal{V}(x) \lvert u \rvert ^{p(x)-2}uw \,dx-\lambda \int_{\mathbb {R}^{N}}\rho(x) \vert u \vert ^{r(x)-2}u w\, dx- \int_{\mathbb {R}^{N}}h(x,u)w\,dx, \end{aligned}$$
for any \(u,w\in E\).
Lemma 3.3
Assume that (A1)–(A2) and (H1)–(H2) hold. Then\(\varPsi _{\lambda}\)and\(\varPsi_{\lambda}^{\prime}\)are weakly strongly continuous onEfor any\(\lambda>0\).
Proof
Let \(\{z_{n}\}\) be a sequence in E such that \(z_{n}\rightharpoonup z\) in E as \(n\to\infty\). Since \(\{z_{n}\}\) is bounded in E, Lemma 2.5 guarantees that there exists a subsequence such that
$$ \begin{gathered}z_{n_{k}}(x) \to z(x) \quad\text{a.e. in } \mathbb {R}^{N} \quad \text{and} \\ z_{n_{k}}\to z \quad\text{in } L^{p(\cdot)}\bigl(\mathbb {R}^{N}\bigr) \cap L^{q(\cdot)} \bigl(\mathbb {R}^{N}\bigr) \quad\text{as } k\to\infty.\end{gathered} $$
(3.7)
First we prove that \(\varPsi_{\lambda}\) is weakly strongly continuous in E. By the convergence principle, there exists a function \(g\in L^{p(\cdot)}(\mathbb {R}^{N}) \cap L^{q(\cdot)} (\mathbb {R}^{N})\) such that \(|z_{n_{k}}(x)|\leq g(x)\) for all \(k\in\mathbb {N}\) and for almost all \(x\in\mathbb {R}^{N}\). Therefore from (H2) and Lemma 2.1, it follows from the Young inequality that
$$\begin{aligned} &\lambda \int_{\mathbb {R}^{N}}{\frac{\rho(x)}{r(x)} \lvert z_{n_{k}} \rvert^{r(x)}}\, dx+ \int_{\mathbb {R}^{N}} \bigl\vert H(x,z_{n_{k}}) \bigr\vert \, dx \\ &\quad\leq\frac{\lambda}{r_{-}} \int_{\mathbb {R}^{N}} \bigl\lvert \rho (x) \bigr\rvert \bigl\vert z_{n_{k}}(x) \bigr\vert ^{r(x)}\,dx+\frac{1}{q_{-}} \int_{\mathbb {R}^{N}}{\sigma(x)} \bigl\vert z_{n_{k}}(x) \bigr\vert ^{q(x)}\, dx \\ &\quad\leq\frac{\lambda}{r_{-}} \int_{\mathbb {R}^{N}}\frac {2(p(x)-r(x))}{p(x)} \bigl\lvert \rho(x) \bigr\rvert ^{\frac{p(x)}{p(x)-r(x) }}+\frac{r(x)}{p(x)} \bigl\lvert \bigl\lvert z_{n_{k}}(x) \bigr\rvert ^{{r(x)}} \bigr\rvert ^{\frac{p(x)}{r(x)}} \,dx\\&\qquad + \frac{ \Vert \sigma \Vert _{L^{\infty}(\mathbb {R}^{N})}}{q_{-}} \int_{\mathbb {R}^{N}} \bigl\vert z_{n_{k}}(x) \bigr\vert ^{q(x)}\, dx \\ &\quad\leq C_{0} \biggl[ \int_{\mathbb {R}^{N}} \bigl\lvert \rho(x) \bigr\rvert ^{\frac{p(x)}{p(x)-r(x) }}+ \bigl\lvert g(x) \bigr\rvert ^{p(x)}\,dx + \int_{\mathbb {R}^{N}} \bigl\vert g(x) \bigr\vert ^{q(x)} \,dx \biggr], \end{aligned}$$
for some positive constant \(C_{0}\), and so the integral at the left-hand side is dominated by an integrable function. Since the function h satisfies the Carathéodory condition by (H1), it follows from (3.7) that
$$\frac{\rho(x)}{r(x)} \lvert z_{n_{k}} \rvert^{r(x)} \to \frac{\rho(x)}{r(x)} \lvert z \rvert^{r(x)} \quad\text{and} \quad H(x,z_{n_{k}})\to H(x,z) \quad\text{as } k\to\infty, $$
for almost all \(x\in\mathbb {R}^{N}\). Therefore, Lebesgue’s dominated convergence theorem tells us that
$$\begin{gathered} \lambda \int_{\mathbb {R}^{N}}\frac{\rho(x)}{r(x)} \vert z_{n_{k}} \vert ^{r(x)}z_{n_{k}}\,dx+ \int_{\mathbb {R}^{N}} H(x,z_{n_{k}})\,dx \\\quad\to\lambda \int_{\mathbb {R}^{N}}\frac{\rho(x)}{r(x)} \vert z \vert ^{r(x)}z\,dx+ \int_{\mathbb {R}^{N}} H(x,z)\,dx \quad \text{as } k\to\infty,\end{gathered} $$
that is, \(\varPsi_{\lambda}(z_{n_{k}}) \to\varPsi_{\lambda} (z)\) as \(k\to \infty\). Thus \(\varPsi_{\lambda}\) is weakly strongly continuous in E.
Next, we show that \(\varPsi_{\lambda}^{\prime}\) is weakly strongly continuous in \(E^{*}\). First of all, we note that
$$\begin{aligned} & \int_{\mathbb {R}^{N}} \bigl\lvert \rho(x) \vert z_{n_{k}} \vert ^{r(x)-2}z_{n_{k}}-\rho (x) \vert z \vert ^{r(x)-2}z \bigr\rvert ^{r^{\prime}(x)}\,dx \\ &\quad\leq C_{1} \int_{\mathbb {R}^{N}} \bigl\lvert \rho(x) \bigr\rvert ^{\frac {1}{r(x)-1}} \bigl\lvert \rho(x) \bigr\rvert \bigl( \vert z_{n_{k}} \vert ^{r(x)}+ \vert z \vert ^{r(x)}\bigr)\, dx \\ &\quad\leq C_{2} \int_{\mathbb {R}^{N}} \bigl\lvert \rho(x) \bigr\rvert \bigl( \vert z_{n_{k}} \vert ^{r(x)}+ \vert z \vert ^{r(x)}\bigr)\, dx \\ &\quad\leq C_{2} \int_{\mathbb {R}^{N}}\frac{2(p(x)-r(x))}{p(x)} \bigl\lvert \rho (x) \bigr\rvert ^{\frac{p(x)}{p(x)-r(x) }}+\frac{r(x)}{p(x)} \lvert z_{n_{k}} \rvert^{p(x)}+\frac {r(x)}{p(x)} \lvert z \rvert^{p(x)}\,dx, \end{aligned}$$
(3.8)
for some positive constants \(C_{1}\), \(C_{2}\). Due to (H2) and Lemma 2.1, we obtain
$$\begin{aligned} \int_{\mathbb {R}^{N}} \bigl\lvert h(x,z_{n_{k}})-h(x,z) \bigr\rvert ^{q^{\prime}(x)}\,dx &\leq C_{3} \int_{\mathbb {R}^{N}} \bigl\lvert h(x,z_{n_{k}}) \bigr\rvert ^{q^{\prime}(x)}+ \bigl\lvert h(x,z) \bigr\rvert ^{{q^{\prime}(x)}}\, dx \\ &\leq C_{4} \int_{\mathbb {R}^{N}} \lvert z_{n_{k}} \rvert^{q(x)} + \lvert z \rvert^{q(x)}\,dx, \end{aligned}$$
(3.9)
for some positive constants \(C_{3}\), \(C_{4}\). Invoking (3.7)–(3.9) and the convergence principle, one has
$$\bigl\lvert \rho(x) \lvert z_{n_{k}} \rvert^{r(x)-2}-\rho (x) \lvert z \rvert^{r(x)-2} \bigr\rvert ^{r^{\prime}(x)} \leq f_{1}(x) $$
and
$$\bigl\lvert h(x,z_{n_{k}})-h(x,z) \bigr\rvert ^{q^{\prime}(x)} \leq f_{2}(x), $$
for almost all \(x\in\mathbb {R}^{N}\) and for some \(f_{1}, f_{2} \in L^{1}(\mathbb {R}^{N})\), and thus \(\rho(x)|z_{n_{k}}|^{r(x)-2}z_{n_{k}}\to \rho (x)|z|^{r(x)-2}z\) and \(h(x,z_{n_{k}})\to h(x,z)\) as \(k\to\infty\) for almost all \(x\in\mathbb {R}^{N}\). This together with the Lebesgue dominated convergence theorem yields
$$\begin{aligned} & \bigl\Vert \varPsi_{\lambda}^{\prime}(z_{n_{k}})- \varPsi_{\lambda}^{\prime}(z) \bigr\Vert _{E^{*}}\\&\quad=\sup _{ \Vert w \Vert _{E}\leq1} \bigl\lvert \bigl\langle \varPsi_{\lambda }^{\prime}(z_{n_{k}})- \varPsi_{\lambda}^{\prime}(z),w\bigr\rangle \bigr\rvert \\ &\quad=\sup_{ \Vert w \Vert _{E}\leq1} \biggl\lvert \lambda \int_{\mathbb {R}^{N}}\bigl(\rho (x) \vert z_{n_{k}} \vert ^{r(x)-2}z_{n_{k}}- \rho(x) \vert z \vert ^{r(x)-2}z \bigr)w\,dx + \int_{\mathbb {R}^{N}}\bigl(h(x,z_{n_{k}})-h(x,z)\bigr){w}\,dx \biggr\rvert \\ &\quad\leq2 \bigl(\lambda \bigl\Vert \rho(x) \vert z_{n_{k}} \vert ^{r(x)-2}z_{n_{k}}- \rho (x) \vert z \vert ^{r(x)-2}z \bigr\Vert _{L^{{r^{\prime}(\cdot)}}(\mathbb {R}^{N})}+ \bigl\Vert {h(x,z_{n_{k}})-h(x,z)} \bigr\Vert _{L^{{q^{\prime}(\cdot)}}(\mathbb {R}^{N})} \bigr) \to0 \end{aligned}$$
as \(k\to\infty\). Consequently, we derive that \(\varPsi_{\lambda }^{\prime}(z_{n_{k}})\to\varPsi_{\lambda}^{\prime}(z)\) in \({E^{*}}\) as \(k\to\infty\). This completes the proof. □
Combining Lemmas 3.2 and 3.3, we see that \(I_{\lambda} \in{C}^{1}(E,\mathbb {R})\) and \(I_{\lambda}\) is weakly semi-continuous in E. Before going to the proofs of our main results, we consider some useful lemmas and consequences presented below. The following assertion means that \(I_{\lambda}\) satisfies the geometric condition in the mountain pass theorem.
Lemma 3.4
Assume that (V), (M1)–(M2), (A1)–(A2) and (H1)–(H4) hold. Let\(I_{\lambda}\)be defined as in (3.6). Then we have the followings:
-
(1)
There exists a positive constant\(\lambda^{*}\)such that for any\(\lambda\in(0, \lambda^{*} )\)we can choose\(R>0\)and\(0<\delta <1\)such that\(I_{\lambda}(u) \geq R >0\)for all\(u \in E\)with\(\|u\| _{E} = \delta\);
-
(2)
there exists\(\phi\in E\), \(\phi>0 \)such that\(I_{\lambda }(t\phi) \to- \infty\)as\(t \to+\infty\);
-
(3)
there exists\(\psi\in E\), \(\psi>0 \)such that\(I_{\lambda }(t\psi) <0 \)as\(t \to0^{+}\).
Proof
Let us prove condition (1). By Lemma 2.5, there exists a positive constant \(C_{5}\) such that \(\|u\|_{L^{\gamma(\cdot)}(\mathbb {R}^{N})}\le C_{5}\|u\|_{E}\) for \(p(x) \leq\gamma(x) < p_{s}^{\ast}(x)\). Assume that \(\|u\|_{E}<1\). Then it follows from (H2), Proposition 2.3, and Lemmas 2.1, 2.2(2) and 2.5 that
$$\begin{aligned} I_{\lambda}(u)&=\mathcal{M} \bigl( [u ]_{s,p(\cdot,\cdot)} \bigr) + \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal{V}(x)}{p(x)} \lvert u \rvert^{p(x)} \biggr)\,dx -\lambda \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert u \rvert^{r(x)} \biggr)\,dx- \int_{\mathbb {R}^{N}} H(x,u)\,dx \\ &\ge\frac{m_{0}}{\vartheta} \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert u(x)-u(y) \vert ^{p(x,y)}}{p(x,y) \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy + \int _{\mathbb {R}^{N}}\frac{\mathcal{V}(x)}{p(x)} \lvert u \rvert ^{p(x)}\,dx \\ &\quad- 2\frac{\lambda}{r_{+}} \Vert \rho \Vert _{L^{\frac{p(\cdot)}{p(\cdot )-r(\cdot)}}(\mathbb {R}^{N})}C_{5} \max\bigl\{ \Vert u \Vert _{E}^{r_{+}}, \Vert u \Vert _{E}^{r _{-}}\bigr\} \\&\quad-\frac{ \Vert \sigma \Vert _{L^{\infty}(\mathbb {R}^{N})}}{q_{+}}\max\bigl\{ \Vert u \Vert _{L^{q(\cdot)}(\mathbb {R}^{N})}^{q_{+}}, \Vert u \Vert _{L^{q(\cdot)}(\mathbb {R}^{N})}^{q}\bigr\} \\ &\ge\min{ \biggl\{ \frac{m_{0}}{\vartheta}, \frac{1}{p_{+}} \biggr\} } \Vert u \Vert ^{p_{+}}_{E}- 2\frac{\lambda}{r_{+}} \Vert \rho \Vert _{L^{\frac {p(\cdot)}{p(\cdot)-r(\cdot)}}(\mathbb {R}^{N})}C_{5} \Vert u \Vert _{E}^{r _{-}}-\frac{C_{5}}{q_{+}} \Vert \sigma \Vert _{L^{\infty}(\mathbb {R}^{N})} \Vert u \Vert ^{q_{-}}_{E} \\ &\ge \biggl(\min{ \biggl\{ \frac{m_{0}}{\vartheta}, \frac{1}{p_{+}} \biggr\} }- 2\frac{\lambda}{r_{+}}C_{6} \Vert u \Vert _{E}^{r _{-}-p_{+}}-\frac{1}{q_{+}}C_{7} \Vert u \Vert ^{q_{-}-p_{+}}_{E} \biggr) \Vert u \Vert ^{p_{+}}_{E}, \end{aligned}$$
(3.10)
for positive constants \(C_{6}\), \(C_{7}\). Let us define the function \(g_{\lambda}:(0,\infty) \to\mathbb {R} \) by
$$g_{\lambda}(t)=2C_{6}\frac{\lambda}{r_{+}}t^{r _{-}-p_{+}}+C_{7} \frac{1}{q_{+}}t^{q_{-}-p_{+}}. $$
Then it is clear that \(g_{\lambda}\) has a local minimum at the point \(t_{0}= ({\lambda\frac{p_{+}-r_{-}}{q_{-}-p_{+}}\cdot\frac {q_{+}}{r_{+}}\cdot\frac{2C_{6}}{C_{7}}} )^{\frac{1}{q_{+}-r_{+}}}\) and so
$$\lim_{\lambda\to0^{+}}g_{\lambda}(t_{0})=0. $$
Thus there is \(\lambda^{\ast}>0\) such that for each \(\lambda\in (0,\lambda^{\ast})\), there exist \(R>0\) small enough and \(\delta>0\) such that \(I_{\lambda}(u)\ge\delta>0\) for any \(u \in E\) with \(\|u\| _{E} = R\).
Next we show condition (2). Note that condition (H3) implies
$$ H(x,s\eta)\ge s^{\theta}H(x,\eta), $$
(3.11)
for all \(\eta\in\mathbb {R}\), \(x \in\mathbb {R}^{N}\), and \(s \ge1\).
Take \(\phi\in E\) with \(\phi>0\). Since \(\mathcal{M}(\tau)\leq \mathcal{M}({1}) \tau^{\vartheta}\) for any \(\tau\geq1\), it follows from (3.11) that
$$\begin{aligned} I_{\lambda}(t \phi)&=\mathcal{M} \bigl( [t \phi]_{s,p(\cdot,\cdot )} \bigr) + \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal{V}(x)}{p(x)} \lvert t \phi \rvert^{p(x)} \biggr)\,dx\\&\quad-\lambda \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert t \phi \rvert^{r(x)} \biggr)\,dx- \int_{\mathbb {R}^{N}}H(x,t \phi)\,dx \\ &\le t^{\vartheta p^{+}} \biggl(\mathcal{M}(1){[ \phi]}^{\vartheta }_{s,p(\cdot,\cdot)} +\frac{1}{p^{-}} \int_{\mathbb {R}^{N}}\mathcal {V}(x) \lvert\phi \rvert^{p(x)} \,dx \biggr)\\&\quad-\lambda t^{r_{-}} \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert \phi \rvert^{r(x)} \biggr)\,dx-t^{\theta} \int_{\mathbb {R}^{N}}{H(x,\phi)}\,dx, \end{aligned}$$
for sufficiently large \(t\ge1\). Since \(\theta>\vartheta p^{+}>r_{+}\), we see that \(I_{\lambda}(t \phi)\to-\infty\) as \(t\to\infty\).
Finally it remains to prove condition (3). Choose \(\psi\in E \) such that \(\psi>0\). Let λ be fixed. For \(t \in(0,1)\) small enough, from (H4) and Lemma 2.2, we obtain
$$\begin{aligned} I_{\lambda}(t\psi)&=\mathcal{M} \bigl( [t \psi]_{s,p(\cdot,\cdot )} \bigr) + \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal{V}(x)}{p(x)} \lvert t\psi \rvert^{p(x)} \biggr)\,dx\\&\quad-\lambda \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert t\psi \rvert^{r(x)} \biggr)\,dx- \int_{\mathbb {R}^{N}}H(x,t\psi)\,dx \\ &\le t^{p^{-}} \biggl( \Bigl(\sup_{0 \leq\xi\leq\max\{\|u\| _{E}^{p^{-}}, \|u\|_{E}^{p ^{+}}\}} M(\xi) \Bigr) [ \psi]_{s,p(\cdot ,\cdot)} + \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal{V}(x)}{p(x)} \lvert\psi \rvert^{p(x)} \biggr)\,dx \biggr)\\&\quad-\lambda t^{r_{+}} \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert \psi \rvert^{r(x)} \biggr)\,dx. \end{aligned}$$
Since \(p^{-}>r_{+}\), we see that \(I_{\lambda}(t\psi)<0\) as \(t\to 0^{+}\), as claimed. □
With the aid of Lemma 3.3, we will give that the energy functional \(I_{\lambda}\) satisfies the Palais–Smale condition (\((PS)\)-condition for short). This plays a key role in obtaining the existence of a nontrivial weak solution for the given problem. The basic idea of the proof of this assertion comes from [43].
Definition 3.5
We say that \(I_{\lambda}\) satisfies the \((PS)\)-condition in E, if any \((PS)\)-sequence \(\{z_{n}\} \subset E\), namely, \(\{I_{\lambda }(z_{n})\}\) is bounded and \(I^{\prime}_{\lambda}(z_{n})\to0\) as \(n\to\infty\), admits a strongly convergent subsequence in E.
Lemma 3.6
If (V), (M1)–(M2), (A1)–(A2), and (H1)–(H4) hold, then the functional\(I_{\lambda}\)satisfies the\((PS)\)-condition for any\(\lambda>0\).
Proof
Let \(\{z_{n}\}\) be a \((PS)\)-sequence in E, i.e., there exists \(K>0\) such that \(| \langle I^{\prime}_{\lambda }(z_{n}),z_{n} \rangle| \leq K {\|z_{n}\|}_{E} \) and \(| I_{\lambda}(z_{n})|\leq K\). It is first verified that the sequence \(\{z_{n}\}\) is bounded in E. Suppose to the contrary that \(\|z_{n}\|_{E} \to\infty\), in the subsequence sense, as \(n \to\infty \). By assumption (M2), we deduce that
$$\begin{aligned} K + K { \Vert z_{n} \Vert }_{E} &\geq I_{\lambda}(z_{n})-\frac{1}{\theta} \bigl\langle I^{\prime}_{\lambda}(z_{n}),z_{n} \bigr\rangle \\ &=\mathcal{M} \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr)- \frac {1}{\theta} M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr) \biggl( \int _{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert z_{n}(x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy \biggr) \\ &\quad+ \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal{V}(x)}{p(x)} \lvert z_{n} \rvert^{p(x)}-\frac{\mathcal{V}(x)}{\theta} \lvert z_{n} \rvert^{p(x)} \biggr)\,dx\\&\quad+\lambda \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{\theta} \lvert z_{n} \rvert^{r(x)}- \frac {\rho(x)}{r(x)} \lvert z_{n} \rvert^{r(x)} \biggr)\,dx \\ &\quad+ \int_{\mathbb {R}^{N}} \biggl(\frac{1}{\theta}h(x,z_{n})z_{n}- H(x,z_{n}) \biggr)\,dx \\ &\ge \biggl(\frac{1}{\vartheta}-\frac{1}{\theta} \biggr) M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr) \biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert z_{n}(x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \, dy \biggr) \\ &\quad+ \biggl(\frac{1}{p_{+}}-\frac{1}{\theta} \biggr) \int_{\mathbb {R}^{N}}{\mathcal{V}(x)} \lvert z_{n} \rvert^{p(x)}\, dx-\lambda \biggl(\frac{1}{r_{+}}- \frac{1}{\theta} \biggr) \int _{\mathbb {R}^{N}}{\rho(x)} \lvert z_{n} \rvert^{r(x)}\,dx \\ &\quad- \int_{\mathbb {R}^{N}} \biggl(H(x,z_{n})- \frac{1}{\theta }h(x,z_{n})z_{n} \biggr)\,dx, \end{aligned}$$
where θ is the positive constant from (H3). Combining this with conditions (M1) and (H3), we have
$$\begin{aligned} &\min{ \biggl\{ \biggl(\frac{1}{\vartheta}-\frac{1}{\theta} \biggr)m_{0}, \frac{1}{p_{+}}-\frac{1}{\theta} \biggr\} }\\&\qquad\times \frac {1}{p^{+}} \biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert z_{n}(x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy+ \int_{\mathbb {R}^{N}}{\mathcal {V}(x) \lvert z_{n} \rvert^{p(x)}\,dx} \biggr) \\ &\qquad-\lambda \biggl(\frac{1}{r_{+}}-\frac{1}{\theta} \biggr) \int _{\mathbb {R}^{N}}{\rho(x)} \lvert z_{n} \rvert^{r(x)}\,dx \\&\quad\le K + K { \Vert z_{n} \Vert }_{E}. \end{aligned}$$
For n large enough, we may assume that \(\|z_{n}\|_{E}>1\). Then it follows from (H3), Proposition 2.3 and Lemma 2.4(1) that
$$\frac{1}{p^{+}}\min{ \biggl\{ \biggl(\frac{1}{\vartheta}- \frac {1}{\theta} \biggr)m_{0}, \frac{1}{p_{+}}- \frac{1}{\theta} \biggr\} } \Vert z_{n} \Vert ^{p_{-}}_{E}-\lambda \biggl(\frac{1}{r_{+}}- \frac{1}{\theta } \biggr)C_{8} \Vert z_{n} \Vert ^{r_{+}}_{E}\le K + K { \Vert z_{n} \Vert }_{E}. $$
Since \(\theta> \vartheta p^{+} > p_{+}>1\) and \(p_{-}>r_{+} >1\), this is a contradiction. Hence the sequence \(\{z_{n}\}\) is bounded in E. Passing to the limit, if necessary, to a subsequence, by Lemma 2.4, we have
$$ \begin{gathered} z_{n} \rightharpoonup z \quad\text{in }E,\qquad z_{n} \to z \quad\text{a.e. in }\mathbb {R}^{N}\quad \text{and} \\ z_{n} \to z \quad\text{in }L^{p(\cdot)} \bigl(\mathbb {R}^{N}\bigr) \text{ and in } L^{q(\cdot )}\bigl(\mathbb {R}^{N}\bigr)\end{gathered} $$
(3.12)
as \(n\to\infty\). To prove that \(\{z_{n}\}\) converges strongly to z in E, let \(\varphi \in E\) be fixed and let \(\tilde{\varPhi}_{\varphi}\) denote the linear functional on E defined by
$$ \tilde{\varPhi}_{\varphi}(w)= \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert \varphi(x)- \varphi(y) \vert ^{{p(x,y)-2}}(\varphi(x)-\varphi (y))(w(x)-w(y))}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy, $$
for all \(w\in E\). Obviously, by the Hölder inequality, \(\tilde {\varPhi}_{\varphi}\) is also continuous, as
$$\begin{aligned} \bigl\lvert \tilde{\varPhi}_{\varphi}(w) \bigr\rvert &\leq 2\biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert \varphi(x)- \varphi(y) \vert ^{{p(x,y)}}}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy \biggr)^{\frac{\tilde{p}_{1}}{\tilde{p}_{2}}} \\&\quad\times\biggl( \int _{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert w(x)-w(y) \vert ^{{p(x,y)}}}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy \biggr)^{\frac{1}{\tilde{p}_{2}}} \\ &\leq2 \Vert \varphi \Vert ^{\tilde{p}_{1}}_{E} \Vert w \Vert _{E}, \end{aligned}$$
for any \(w\in E\), where \(\tilde{p}_{1}\) is either \(p^{+} -1 \) or \(p^{-}-1\) and \(\tilde{p}_{2}\) is either \(p^{+}\) or \(p^{-}\). Hence, Eq. (3.12) yields
$$ \lim_{n\to\infty} \bigl[M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr)-M \bigl( [z]_{s,p(\cdot,\cdot)} \bigr) \bigr]\tilde{\varPhi}_{u}(z_{n}-z)=0, $$
(3.13)
because the sequence \(\{M ( [z_{n} ]_{s,p(\cdot,\cdot )} )-M ( [z]_{s,p(\cdot,\cdot)} ) \}\) is bounded in \(\mathbb {R}\). Using (H2) and Lemma 2.2, it follows that
$$\begin{aligned} & \int_{\mathbb {R}^{N}} \bigl\lvert \bigl(h(x,z_{n})-h(x,z) \bigr) (z_{n}-z) \bigr\rvert \, dx \\ &\quad\le \int_{\mathbb {R}^{N}}\sigma(x) \bigl( \lvert z_{n} \rvert ^{q(x)-1}+ \lvert z \rvert^{q(x)-1}\bigr) \lvert z_{n}-z \rvert\, dx \\ &\quad\le2 \Vert \sigma \Vert _{L^{\infty}(\mathbb {R}^{N})} \bigl( \Vert z_{n} \Vert _{L^{q(\cdot )}(\mathbb {R}^{N})}^{q_{+}-1}+ \Vert z_{n} \Vert _{L^{q(\cdot)}(\mathbb {R}^{N})}^{q_{-}-1}+ \Vert z \Vert _{L^{q(\cdot)}(\mathbb {R}^{N})}^{q_{+}-1}+ \Vert z \Vert _{L^{q(\cdot)}(\mathbb {R}^{N})}^{q_{-}-1} \bigr)\\&\qquad\times \Vert z_{n}-z \Vert _{L^{q(\cdot)}(\mathbb {R}^{N})}. \end{aligned}$$
Then, due to (3.12), one has
$$ \lim_{n\to\infty} \int_{\mathbb {R}^{N}}\bigl(h(x,z_{n})-h(x,z)\bigr) (z_{n}-z)\, dx=0. $$
(3.14)
Because \(z_{n}\rightharpoonup z\) in E and \(I_{\lambda}'(z_{n})\to0\) in \(E^{*}\) as \(n\to\infty\), we have
$$\bigl\langle I_{\lambda}'(z_{n})-I_{\lambda}'(z), z_{n}-z\bigr\rangle \to0 \quad\text{as } n\to\infty. $$
Hence, Eqs. (3.12)–(3.14) yield as \(n\to\infty\)
$$\begin{aligned} o(1)&=\bigl\langle I_{\lambda}'(z_{n})-I_{\lambda}'(z), z_{n}-z\bigr\rangle \\ &=M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr)\tilde{\varPhi }_{z_{n}}(z_{n}-z)-M \bigl( [z ]_{s,p(\cdot,\cdot)} \bigr) \tilde{\varPhi }_{z}(z_{n}-z) \\ &\quad+ \bigl(M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr)-M \bigl( [z ]_{s,p(\cdot,\cdot)} \bigr) \bigr)\tilde{\varPhi}_{z}(z_{n}-z)\\&\quad+ \int _{\mathbb {R}^{N}}\mathcal{V}(x) \bigl( \lvert z_{n} \rvert ^{p(x)-2}z_{n}- \lvert z \rvert^{p(x)-2}z\bigr) (z_{n}-z)\,dx \\ &\quad-\lambda \int_{\mathbb {R}^{N}}\rho(x) \bigl( \lvert z_{n} \rvert ^{r(x)-2}z_{n}- \lvert z \rvert^{r(x)-2}z\bigr) (z_{n}-z)\,dx\\&\quad- \int _{\mathbb {R}^{N}}\bigl(h(x,z_{n})-h(x,z)\bigr) (z_{n}-z)\,dx \\ &=M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr) \bigl[\tilde{\varPhi }_{z_{n}}(z_{n}-z)-\tilde{\varPhi}_{z}(z_{n}-z) \bigr]\\&\quad+ \int_{\mathbb {R}^{N}}\mathcal{V}(x) \bigl( \lvert z_{n} \rvert^{p(x)-2}z_{n}- \lvert z \rvert^{p(x)-2}z\bigr) (z_{n}-z)\,dx \\ &\quad-\lambda \int_{\mathbb {R}^{N}}\rho(x) \bigl( \lvert z_{n} \rvert ^{r(x)-2}z_{n}- \lvert z \rvert^{r(x)-2}z\bigr) (z_{n}-z)\,dx\\&\quad- \int _{\mathbb {R}^{N}}\bigl(h(x,z_{n})-h(x,z)\bigr) (z_{n}-z)\,dx+o(1), \end{aligned}$$
that is,
$$ \begin{aligned}[b] &\lim_{n\to\infty} \biggl(M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr) \bigl[\tilde{ \varPhi}_{z_{n}}(z_{n}-z)-\tilde{\varPhi}_{z}(z_{n}-z) \bigr]\\&\quad+ \int_{\mathbb {R}^{N}}\mathcal{V}(x) \bigl( \lvert z_{n} \rvert ^{p(x)-2}z_{n}- \lvert z \rvert^{p(x)-2}z\bigr) (z_{n}-z)\,dx \\ &\quad-\lambda \int_{\mathbb {R}^{N}}\rho(x) \bigl( \lvert z_{n} \rvert ^{r(x)-2}z_{n}- \lvert z \rvert^{r(x)-2}z\bigr) (z_{n}-z)\,dx \biggr)=0. \end{aligned} $$
By convexity, (M1), (V), and (H2) we have in particular
$$\begin{gathered} M \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr)\bigl[\tilde{\varPhi }_{z_{n}}(z_{n}-z)-\tilde{\varPhi}_{z}(z_{n}-z) \bigr]\ge0, \\ \mathcal{V}(x) \bigl( \lvert z_{n} \rvert^{p(x)-2}z_{n}- \lvert z \rvert^{p(x)-2}z\bigr) (z_{n}-z)\,dx\ge0,\end{gathered} $$
and
$$\rho(x) \bigl( \lvert z_{n} \rvert^{r(x)-2}z_{n}- \lvert z \rvert^{r(x)-2}z\bigr) (z_{n}-z)\,dx\ge0. $$
It follows that
$$\begin{aligned}& \lim_{n\to\infty}\tilde{\varPhi}_{z_{n}}(z_{n}-z)- \tilde{\varPhi}_{z}(z_{n}-z)=0, \end{aligned}$$
(3.15)
$$\begin{aligned}& \lim_{n\to\infty} \int_{\mathbb {R}^{N}}\mathcal{V}(x) \bigl( \lvert z_{n} \rvert^{p(x)-2}z_{n}- \lvert z \rvert ^{p(x)-2}u\bigr) (z_{n}-z)\,dx=0, \end{aligned}$$
(3.16)
and
$$ \lim_{n\to\infty} \int_{\mathbb {R}^{N}}\rho(x) \bigl( \lvert z_{n} \rvert^{r(x)-2}z_{n}- \lvert z \rvert^{r(x)-2}u\bigr) (z_{n}-z)\,dx=0. $$
(3.17)
It should be noted that we have the well-known useful inequalities
$$ \lvert\xi-\eta \rvert^{p(x,y)}\le \textstyle\begin{cases} C_{9}( \lvert\xi \rvert^{p(x,y)-2}\xi- \lvert\eta \rvert^{p(x,y)-2}\eta)\cdot(\xi-\eta) \quad\text{for } (x,y) \in\Delta_{1}, \\ C_{10}[( \lvert\xi \rvert^{p(x,y)-2}\xi- \lvert \eta \rvert^{p(x,y)-2}\eta)\cdot(\xi-\eta)]^{\frac {p(x,y)}{2}}\\ \quad{}\times( \lvert\xi \rvert^{p(x,y)}+ \lvert \eta \rvert^{p(x,y)})^{\frac{2-p(x,y)}{2}} \quad\text{for } (x,y) \in\Delta_{2} \text{ and } (\xi,\eta)\neq(0,0),\hspace{-12pt} \end{cases} $$
(3.18)
for all \(\xi,\eta\in\mathbb {R}^{N}\), where \(C_{9} \) and \(C_{10}\) are positive constants depending only on \(p(\cdot,\cdot)\), \(\Delta_{1}=\{(x, y) \in\mathbb {R}^{N} \times\mathbb {R}^{N}: p(x,y)\ge 2\}\), and \(\Delta_{2}=\{(x,y) \in\mathbb {R}^{N} \times\mathbb {R}^{N} : 1< p(x,y)<2\} \); see [35, Proposition 3.3].
It is now assumed that \((x,y) \in\Delta_{1}\). Then, by (3.15) and (3.18) as \(n\to\infty\),
$$ \begin{aligned}[b] & \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert ( z_{n} - z)(x) - ( z_{n} - z)(y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \, dx \,dy \\ &\quad= \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \bigl\vert z_{n} (x)- z_{n} (y)- z(x)+ z(y) \bigr\vert ^{{p(x,y)}} \vert x-y \vert ^{-(N+sp(x,y))} \,dx \,dy \\ &\quad\le C_{9} \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \bigl[ \bigl\vert z_{n} (x)-z_{n} (y) \bigr\vert ^{{p(x,y)-2}}\\&\qquad\times\bigl(z_{n} (x)-z_{n} (y)\bigr)- \bigl\vert z(x)-z(y) \bigr\vert ^{{p(x,y)-2}}\bigl(z(x)-z(y)\bigr)\bigr] \\ &\qquad\times\bigl(z_{n} (x)- z_{n} (y)- z(x)+ z(y)\bigr) \vert x-y \vert ^{-(N+sp(x,y))} \,dx \,dy \\ &\quad\le C_{9} \bigl(\tilde{\varPhi}_{z_{n}}(z_{n}-z)- \tilde{\varPhi }_{z}(z_{n}-z) \bigr)=o(1). \end{aligned} $$
(3.19)
Similarly, utilizing (V), (3.16) and (3.18) as \(n\to \infty\),
$$ \int_{\Delta_{1}}\mathcal{V}(x) \lvert z_{n}-z \rvert ^{p(x)}\,dx\le C_{9} \int_{\Delta_{1}}\mathcal{V}(x) \bigl( \lvert z_{n} \rvert ^{p(x)-2}z_{n}- \lvert z \rvert^{p(x)-2}z\bigr) (z_{n}-z)\,dx=o(1). $$
Subsequently, the case \((x, y) \in\Delta_{2}\) is considered. As \(\{ z_{n}\}\) is bounded in E, there exists \(K_{0} >0\) such that
$$\int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert z_{n} (x)-z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy\le K_{0} ,$$
for all \(n\in\mathbb {N}\). By (3.15), (3.18) and Lemma 2.1, we have
$$ \begin{aligned}[b] & \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert ( z_{n} - z)(x) - ( z_{n} - z)(y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \, dx \,dy \\ &\quad\le C_{10} \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \bigl\{ \bigl[ \bigl\vert z_{n} (x)-z_{n} (y) \bigr\vert ^{{p(x,y)-2}}\\&\qquad\times\bigl(z_{n} (x)-z_{n} (y)\bigr)- \bigl\vert z(x)-z(y) \bigr\vert ^{{p(x,y)-2}}\bigl(z(x)-z(y)\bigr)\bigr] \\ &\qquad\times\bigl(z_{n} (x)- z_{n} (y)- z(x)+ z(y)\bigr)\bigr\} ^{\frac{p(x,y)}{2}}\bigl( \bigl\vert z_{n} (x)-z_{n} (y) \bigr\vert ^{p(x,y)}+ \bigl\vert z (x)-z (y) \bigr\vert ^{p(x,y)}\bigr)^{\frac{2-p(x,y)}{2}} \\ &\qquad\times \vert x-y \vert ^{-(N+sp(x,y))} \,dx \,dy \\ &\quad\le2 C_{11} \bigl(\tilde{\varPhi}_{z_{n}}(z_{n}-z)- \tilde{\varPhi }_{z}(z_{n}-z) \bigr)^{\alpha} \\ & \qquad\times \biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert z_{n}(x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy + \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert z(x)- z(y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \,dy \biggr)^{\beta}\\&\quad=o(1), \end{aligned} $$
where \(C_{11} =2C_{10}(2K)^{\beta}\), α is either \(p^{-} /2\) or \(p^{+} /2\), and β is either \((2- p^{+} )/2\) or \((2- p^{-} )/2\). Similarly, by invoking (3.12), there is a positive constant L such that \(\int_{\mathbb {R}^{N}}\mathcal{V}(x) \lvert z_{n} \rvert ^{p(x)}\,dx\le L\) for all \(n\in\mathbb {N}\). Moreover, by Lemma 2.1, (3.16) and (3.18) as \(n\to\infty\),
$$ \begin{aligned}[b] \int_{\Delta_{2}}\mathcal{V}(x) \lvert z_{n}-z \rvert ^{p(x)}\,dx&\le C_{12} \biggl( \int_{\mathbb {R}^{N}}\mathcal{V}(x) \bigl( \lvert z_{n} \rvert^{p(x)-2}z_{n}- \lvert z \rvert^{p(x)-2} u\bigr) (z_{n}-z)\,dx \biggr)^{\alpha}\\&=o(1),\end{aligned} $$
(3.20)
where \(C_{12}=4C_{10}(2L)^{\beta}\). From (3.19) and (3.20), we obtain
$$\int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}} \frac{ \vert z_{n}(x)-z_{n}(y) \vert ^{p(x,y)-2}}{ \vert x-y \vert ^{N+sp(x,y)}} \,dx \, dy+ \int_{\mathbb {R}^{N}}\mathcal{V}(x) \lvert z_{n}-z \rvert ^{p(x)}\,dx\to0 \quad\text{as } n\to\infty. $$
Therefore, \(\|z_{n}-z\|_{E} \to0\) as \(n\to\infty\). Hence, \(I_{\lambda}\) satisfies the \((PS)\)-condition. This completes the proof. □
The proof of the following theorem can be found in [10, 30, 43], however, we will give the proof for the reader’s convenience.
Theorem 3.7
Let (V), (M1)–(M2), (A1)–(A2), and (H1)–(H4) hold. Then there exists a positive constant\(\lambda^{*}\)such that for any\(\lambda\in(0, \lambda^{*} )\), the functional\(I_{\lambda}\)admits at least two nontrivial different solutions inE.
Proof
Thanks to Lemmas 3.4 and 3.6, there exists \(\lambda ^{*}>0\) such that for all \(\lambda\in(0, \lambda^{*})\), \(I_{\lambda }\) satisfies the mountain pass geometry and \((PS)\)-condition. By employing the mountain pass theorem, we infer that there exists a critical point \(u_{0} \in E \) of \(I_{\lambda}\) with \(I_{\lambda }(u_{0})= \overline{d}>0=I_{\lambda}(0)\). Hence \(u_{0}\) is a nontrivial weak solution of the problem (\(P_{\lambda }\)). Let us denote \(d:=\inf_{u \in\overline{B}_{r}}{I_{\lambda}(u)}\) where \({B}_{r} := \{ u \in E : \|u\|_{E} < r \}\) with a boundary \(\partial{B}_{r}\). Then by (3.10) and Lemma 3.4(3), we have \(-\infty< d < 0\). Putting \(0< \epsilon< \inf_{u \in\partial{B}_{r}}{I_{\lambda }(u)}-d\), by Theorem 1.1 in [25] (see also [30]), we can find \(u_{\epsilon} \in\overline{B}_{r}\) such that we have the well-known useful inequalities
$$ \textstyle\begin{cases} I_{\lambda}( u_{\epsilon} ) \leq d + \epsilon,\\ I_{\lambda}( u_{\epsilon} ) < I_{\lambda}(u) + \epsilon \Vert u-u_{\epsilon} \Vert _{E},\quad\text{for all } u \in\overline{B}_{r}, u \neq u_{\epsilon}. \end{cases} $$
(3.21)
This implies that \(u_{\epsilon} \in{B}_{r}\) since \(I_{\lambda}( u_{\epsilon} ) \leq d + \epsilon< \inf_{u \in\partial {B}_{r}}{I_{\lambda}(u)}\). From these facts we see that \(u_{\epsilon}\) is a local minimum of \(\widetilde{I}_{\lambda}( u ) = I_{\lambda}(u) + \epsilon\| u-u_{\epsilon} \|_{E}\). Now by taking \(u = u_{\epsilon} +tw \) for \(w \in{B}_{1}\) and sufficiently small \(t>0\), from (3.21), we deduce
$$0 \leq\frac{\widetilde{I}_{\lambda}( u_{\epsilon} +tw )-\widetilde {I}_{\lambda}( u_{\epsilon} ) }{t} = \frac{{I_{\lambda}}( u_{\epsilon} +tw )-{I_{\lambda}}( u_{\epsilon} ) }{t} + \epsilon \Vert w \Vert _{E}. $$
Therefore, letting \(t \rightarrow0+ \), we get
$$\bigl\langle I_{\lambda}^{\prime}(u_{\epsilon}),w \bigr\rangle + \epsilon \Vert w \Vert _{E} \geq0. $$
Replacing w by −w in the argument above, we have
$$- \bigl\langle I_{\lambda}^{\prime}(u_{\epsilon}),w \bigr\rangle +\epsilon \Vert w \Vert _{E} \geq0. $$
Thus, one has
$$\bigl\vert \bigl\langle I_{\lambda}^{\prime}(u_{\epsilon}),w \bigr\rangle \bigr\vert \leq\epsilon \Vert w \Vert _{E}, $$
for any \(w \in\overline{B}_{1}\). Hence
$$ \bigl\Vert I_{\lambda}^{\prime}(u_{\epsilon}) \bigr\Vert _{E^{*}} \leq\epsilon. $$
(3.22)
Using (3.21) and (3.22), we can choose a sequence \(\{ z_{n} \} \subset B_{r} \) such that
$$ \textstyle\begin{cases} I_{\lambda}( z_{n} ) \rightarrow c \quad\text{as } n \to\infty, \\ \Vert I_{\lambda}^{\prime}(z_{n}) \Vert _{E^{*}} \to0 \quad\text{as } n \to \infty. \end{cases} $$
(3.23)
Thus, \(\{ z_{n} \} \) is a bounded \((PS)\)-sequence in the reflexive Banach space E. According to Lemma 3.6, \(\{ z_{n} \} \) has a subsequence \(\{ z_{n_{k}} \} \) such that \(z_{n_{k}} \to u_{1} \) in E as \(k \to\infty\). This together with (3.23) yields \(I_{\lambda }( u_{1} ) =d\) and \(I_{\lambda}^{\prime}( u_{1} )=0\). Hence \(u_{1}\) is a nontrivial nonnegative solution of the given problem with \(I_{\lambda }( u_{1} )<0\) which is different from \(u_{0}\). This completes the proof. □
The existence of nontrivial solutions for the problem is now investigated if (H3) is replaced with the following condition:
-
(H5)
There exists a constant \(\theta\ge1\) such that
$$\theta{\mathcal {H}}(x,t) \ge{\mathcal {H}}(x,st) ,$$
for \((x,t)\in\mathbb {R}^{N}\times\mathbb {R}\) and \(s\in[0,1]\), where \({\mathcal {H}}(x,t)=h(x,t)t-p_{+}\vartheta H(x,t)\) and ϑ is given in (M2).
This condition originally comes from the work of Jeanjean [31]. As is well known, this is weaker condition than (1.1).
Definition 3.8
We say that \(I_{\lambda}\) satisfies the Cerami condition (\((C)\)-condition for short) in E, if any \((C)\)-sequence \(\{z_{n}\} _{n} \subset E\), i.e. \(\{I_{\lambda}(z_{n})\}\) is bounded and \(\| I_{\lambda}^{\prime}(z_{n})\|_{E^{*}}(1+\|z_{n}\|_{E})\to0\) as \(n\to\infty\), has a convergent subsequence in E.
Lemma 3.9
It is assumed that (V), (M1)–(M2), (A1)–(A2), (H1)–(H2), and (H4)–(H5). hold. Furthermore, assume that
-
(M3)
\(M:\mathbb{R}^{+} \to\mathbb{R}^{+}\)is a differentiable and decreasing function.
Then, the functional\(I_{\lambda}\)satisfies the\((C)\)-condition for any\(\lambda>0\).
Proof
Let \(\{z_{n}\}\) be a \((C)\)-sequence in E, i.e., \(\sup |I_{\lambda}(z_{n})|\leq K_{1}\) and \(\langle I^{\prime}_{\lambda }(z_{n}),z_{n} \rangle=o(1) \to0\), as \(n\to\infty\), and \(K_{1}\) is a positive constant. In view of Lemma 3.6, it needs only to be proved that \(\{z_{n}\}\) is bounded in E. To this end, arguing by contradiction, it is assumed that \(\|z_{n}\|_{E}>1\) and \(\|z_{n}\|_{E}\to\infty\) as \(n\to\infty\), and a sequence \(\{\omega_{n} \}\) is defined by \(\omega_{n}={z_{n}}/{\|z_{n}\|_{E}}\). Then, up to a subsequence, still denoted by \(\{\omega_{n} \}\), we obtain \(\omega_{n}\rightharpoonup\omega\) in E as \(n\to\infty\), and by Lemma 2.5,
$$\begin{gathered} \omega_{n}(x) \to\omega(x) \quad\text{a.e. in }\mathbb {R}^{N}, \qquad \omega_{n} \to\omega\quad\text{in }L^{r(\cdot)} \bigl(\mathbb {R}^{N}\bigr), \quad \text{and}\\ \omega_{n} \to\omega\quad\text{in }L^{p(\cdot)}\bigl(\mathbb {R}^{N}\bigr)\end{gathered} $$
as \(n \to\infty\), where \(p(x)< r(x)<{p_{s}}^{*}(x)\) for all \(x\in \mathbb {R}^{N}\).
Let \(\varOmega_{2}= \{ x\in\mathbb {R}^{N} : \omega(x)\neq0 \}\). By the same argument as in Lemma 3.6, \(\lvert\varOmega_{2} \rvert=0\); thus, \(\omega(x)=0\) for almost all \(x\in \mathbb {R}^{N}\). As \(I_{\lambda}(tz_{n})\) is continuous in \(t\in[0,1]\), for each \(n\in\mathbb {N}\), there exists \(t_{n}\in[0,1]\) such that
$$ I_{\lambda}(t_{n}z_{n}):= \max_{t\in[0,1]} {I_{\lambda}(tz_{n})}. $$
Let \(\{\ell_{k} \}\) be a positive sequence of real numbers such that \(\lim_{k\to\infty}{\ell_{k}}=\infty\) and \(\ell_{k}>1\) for any k. Then, it is clear that \(\|\ell_{k}\omega_{n}\|_{E}=\ell_{k}>1\) for any k and n. Let k be fixed. Because \(\omega_{n}\to0\) strongly in \(L^{q(\cdot)}(\mathbb {R}^{N})\) as \(n\to\infty\), it follows from the continuity of the Nemytskii operator that \(H(x,\ell_{k}\omega_{n})\to0\) in \(L^{1}(\mathbb {R}^{N})\) as \(n\to\infty\). Hence,
$$ \lim_{n\to\infty}{ \int_{\mathbb {R}^{N}}{H(x,\ell_{k}\omega_{n})}} \,dx=0. $$
(3.24)
Because \(\|z_{n}\|_{E}\to\infty\) as \(n\to\infty\), we have \(\|z_{n}\|_{E}>\ell_{k}\) for sufficiently large n. Thus, by (M2), (3.24), and Proposition 2.3 we have
$$\begin{aligned} I_{\lambda}(t_{n}z_{n}) &\geq I_{\lambda} \biggl(\frac{\ell_{k}}{ \Vert z_{n} \Vert _{E}}z_{n} \biggr)=I_{\lambda}(\ell_{k}\omega_{n}) \\ &=\mathcal{M} \bigl([\ell_{k}\omega_{n}]_{s,p(\cdot,\cdot)} \bigr)+ \int_{\mathbb {R}^{N}}\frac{\mathcal{V}(x)}{p(x)} \lvert\ell _{k} \omega_{n} \rvert^{p(x)}\,dx\\&\quad- \lambda \int_{\mathbb {R}^{N}}\frac{\rho(x)}{r(x)} \lvert\ell_{k} \omega_{n} \rvert^{r(x)}\,dx- \int_{\mathbb {R}^{N}}{H(x,\ell_{k}\omega_{n})} \,dx \\ &\ge\min{ \biggl\{ \frac{m_{0} }{\vartheta p^{+}}, \frac{1}{p_{+}} \biggr\} } \\&\quad\times\biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert \ell _{k}\omega_{n}(x)- \ell_{k}\omega_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy+ \int_{\mathbb {R}^{N}}{\mathcal {V}(x) \lvert\ell_{k} \omega_{n} \rvert^{p(x)}\,dx} \biggr) \\ &\quad- \lambda \int_{\mathbb {R}^{N}}\frac{\rho(x)}{r(x)} \lvert\ell _{k} \omega_{n} \rvert^{r(x)}\,dx- \int_{\mathbb {R}^{N}}{H(x,\ell _{k}\omega_{n})} \,dx \\ &\ge\min{ \biggl\{ \frac{m_{0} }{\vartheta p^{+}}, \frac{1}{p_{+}} \biggr\} } \Vert \ell_{k}\omega_{n} \Vert _{E}^{p^{-}}- 2\frac{\lambda}{r_{+}} \Vert \rho \Vert _{L^{\frac{p(\cdot)}{p(\cdot)-r(\cdot)}}(\mathbb {R}^{N})} \Vert \ell_{k}\omega_{n} \Vert _{E}^{r_{+}}\\&\quad- \int_{\mathbb {R}^{N}}{H(x,\ell_{k}\omega _{n})}\,dx \\ &\geq\min{ \biggl\{ \frac{m_{0} }{\vartheta p^{+}}, \frac{1}{p_{+}} \biggr\} } \ell_{k}^{p^{-}}-2C_{13}\frac{\lambda}{r_{+}} \ell_{k}^{r_{+}}, \end{aligned}$$
for sufficiently large n and \(p^{-}>r_{+} >1\). Then, letting n and k tend to infinity, it follows that
$$ \lim_{n\to\infty}{I_{\lambda}(t_{n}z_{n})}= \infty. $$
(3.25)
Because \(I_{\lambda}(0)=0\) and \(|I_{\lambda}(z_{n})| \leq K_{1} \) as \(n\to \infty\), it is obvious that \(t_{n}\in(0,1)\) and \(\langle I_{\lambda}^{\prime}(t_{n}z_{n}), t_{n}z_{n} \rangle=0\). Note that there is a positive constant \(\mathcal{K}\) such that
$$ \int_{\mathbb {R}^{N}} \biggl(\frac{1}{r(x)}-\frac{1}{p_{+}\vartheta} \biggr)\rho (x) \lvert s z \rvert^{r(x)}\,dx\ge\theta \int_{\mathbb {R}^{N}} \biggl(\frac{1}{r(x)}-\frac{1}{p_{+}\vartheta} \biggr)\rho(x) \lvert z \rvert^{r(x)}\,dx-\mathcal{K}, $$
(3.26)
for any \(z\in E\) and \(s\in[0,1]\), where θ and ϑ come from (H5) and (M2), respectively. Therefore, by (M2), (M3), (H5) and (3.26), for all n large enough, we have
$$\begin{aligned} \frac{1}{\theta}I_{\lambda}(t_{n} z_{n})&=\frac{1}{\theta }I_{\lambda}(t_{n} z_{n})-\frac{1}{p_{+}\theta\vartheta} \bigl\langle I^{\prime}_{\lambda}(t_{n} z_{n}),t_{n} z_{n} \bigr\rangle +o(1) \\ &=\frac{1}{\theta}\mathcal{M} \bigl( [t_{n} z_{n} ]_{s,p(\cdot,\cdot )} \bigr)\\&\quad-\frac{1}{p_{+}\theta\vartheta} M \bigl( [t_{n} z_{n} ]_{s,p(\cdot ,\cdot)} \bigr) \biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert t_{n} z_{n}(x)- t_{n} z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy \biggr) \\ &\quad+\frac{1}{\theta} \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal {V}(x)}{p(x)} \lvert t_{n} z_{n} \rvert^{p(x)}- \frac {\mathcal{V}(x)}{p_{+}\vartheta} \lvert t_{n} z_{n} \rvert ^{p(x)} \biggr)\,dx\\&\quad-\frac{\lambda}{\theta} \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert t_{n} z_{n} \rvert ^{r(x)}- \frac{\rho(x)}{p_{+}\vartheta} \lvert t_{n} z_{n} \rvert ^{r(x)} \biggr)\,dx \\ &\quad+\frac{1}{\theta} \int_{\mathbb {R}^{N}} \biggl(\frac{1}{p_{+}\vartheta}h(x,t_{n} z_{n})t_{n} z_{n}- H(x,t_{n} z_{n}) \biggr)\,dx+o(1) \\ &\leq\frac{1}{\theta}\mathcal{M} \bigl( [t_{n} z_{n} ]_{s,p(\cdot ,\cdot)} \bigr)\\&\quad-\frac{1}{p_{+}\theta\vartheta} M \bigl( [t_{n} z_{n} ]_{s,p(\cdot,\cdot)} \bigr) \biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert t_{n} z_{n}(x)- t_{n} z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy \biggr) \\ &\quad+\frac{1}{\theta} \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal {V}(x)}{p(x)} \lvert t_{n} z_{n} \rvert^{p(x)}- \frac {\mathcal{V}(x)}{p_{+}\vartheta} \lvert t_{n} z_{n} \rvert ^{p(x)} \biggr)\,dx\\&\quad-\frac{\lambda}{\theta} \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert t_{n} z_{n} \rvert ^{r(x)}- \frac{\rho(x)}{p_{+}\vartheta} \lvert t_{n} z_{n} \rvert ^{r(x)} \biggr)\,dx \\ &\quad+\frac{1}{p_{+}\theta\vartheta} \int_{\mathbb {R}^{N}} {\mathcal {H}}(x,t_{n} z_{n})\, dx+o(1) \\ &\leq\frac{1}{\theta}\biggl[\mathcal{M} \bigl( [ z_{n} ]_{s,p(\cdot ,\cdot)} \bigr)\\&\quad-\frac{1}{p_{+}\vartheta} M \bigl( [ z_{n} ]_{s,p(\cdot,\cdot)} \bigr) \biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert z_{n}(x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy \biggr)\biggr] \\ &\quad+\frac{1}{\theta} \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal {V}(x)}{p(x)} \lvert z_{n} \rvert^{p(x)}- \frac {\mathcal{V}(x)}{p_{+}\vartheta} \lvert z_{n} \rvert ^{p(x)} \biggr)\,dx\\&\quad-\lambda \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert z_{n} \rvert ^{r(x)}- \frac{\rho(x)}{p_{+}\vartheta} \lvert z_{n} \rvert ^{r(x)} \biggr)\,dx \\ &\quad+\frac{1}{p_{+}\vartheta} \int_{\mathbb {R}^{N}} {\mathcal {H}}(x,z_{n})\, dx+\lambda\mathcal{K}+o(1) \\ &\leq\mathcal{M} \bigl( [z_{n} ]_{s,p(\cdot,\cdot)} \bigr)+ \int _{\mathbb {R}^{N}}\frac{\mathcal{V}(x)}{p(x)} \lvert z_{n} \rvert^{p(x)}\,dx\\&\quad-\lambda \int_{\mathbb {R}^{N}}\frac{\rho (x)}{r(x)} \lvert z_{n} \rvert^{r(x)}\,dx- \int_{\mathbb {R}^{N}}{H(x,z_{n})}\,dx \\ &\quad-\frac{1}{p_{+}\vartheta} M \bigl([z_{n} ]_{s,p(\cdot,\cdot)} \bigr) \biggl( \int_{\mathbb {R}^{N}} \int_{\mathbb {R}^{N}}\frac{ \vert z_{n}(x)- z_{n} (y) \vert ^{p(x,y)}}{ \vert x-y \vert ^{N+sp(x,y)}}\, dx \,dy \biggr)\\&\quad- \frac{1}{p_{+}\vartheta} \int_{\mathbb {R}^{N}}{\mathcal{V}(x)} \lvert z_{n} \rvert ^{p(x)}\,dx \\ &\quad+\frac{\lambda}{p_{+}\vartheta} \int_{\mathbb {R}^{N}}{\rho(x)} \lvert z_{n} \rvert^{r(x)}\,dx+\frac{1}{p_{+}\vartheta} \int_{\mathbb {R}^{N}}h(x,z_{n})z_{n}\,dx+ \lambda\mathcal{K}+o(1) \\ &=I_{\lambda}(z_{n})-\frac{1}{p_{+}\vartheta} \bigl\langle I_{\lambda}^{\prime }(z_{n}),z_{n} \bigr\rangle +\lambda\mathcal{K}+o(1) \le \lambda\mathcal{K}+ K_{1}+o(1), \end{aligned}$$
which contradicts (3.25). This completes the proof. □
We give an example on the function M that fulfils the assumptions (M1)–(M3); see [40].
Example 3.10
Let us consider
$$M(t)=1+\frac{1}{e+t}, \quad t\ge0. $$
Then, it follows from direct calculations that this function M satisfies the assumptions (M1)–(M3).
In the rest of the present paper we establish the existence of at least two distinct nontrivial solutions to the problem (\(P_{\lambda }\)) under the condition on h which is weaker than (H3). In order to obtain this assertion we need to employ the following variational principle of Ekeland’s type in [6, 40], initially developed by Zhong [51].
Lemma 3.11
([6, 40])
LetEbe a Banach space and\(x_{0}\)be a fixed point ofE. Suppose that\(h:{E}\rightarrow\mathbb{R}\cup\{+\infty\}\)is a lower semi-continuous function, not identically +∞, bounded from below. Then, for every\({\varepsilon }>0\)and\(y\in E\)such that
$$h(y)< \inf_{{E}}{h}+{\varepsilon }, $$
and every\(\lambda>0\), there exists some point\(z\in E\)such that
$$ h(z) \le h(y),\qquad \Vert z-x_{0} \Vert _{E}\le \bigl({1+ \Vert y \Vert _{E}}\bigr) \bigl(e^{{\lambda }}-1 \bigr), $$
and
$$ h(x)\ge h(z)-\frac{{\varepsilon }}{\lambda(1+ \Vert z \Vert _{E})}{ \Vert x-z \Vert _{E}}, \quad\textit{for all } x\in E. $$
Theorem 3.12
Let (V), (M1)–(M3), (A1)–(A2), (H1)–(H2), and (H4)–(H5) hold. In addition, assume that
-
(H6)
\(\lim_{ \lvert t \rvert\to\infty}{\frac {H(x,t)}{ \lvert t \rvert^{\vartheta p_{+}}}}=\infty\)uniformly for almost all\(x\in\mathbb {R}^{N}\).
Then there exists a positive constant\(\lambda^{*}\)such that for any\(\lambda\in(0, \lambda^{*} )\), the functional\(I_{\lambda}\)admits at least two nontrivial different solutions inE.
Proof
To apply Lemma 3.4, we first show condition (2) in this lemma. By the assumption (H6), for any \(M_{0}>0\), there exists a constant \(\delta>0\) such that
$$ H(x,t)\ge M_{0} \lvert t \rvert^{\vartheta p^{+}}, $$
(3.27)
for \(\lvert t \rvert>\delta\) and for almost all \(x \in \mathbb {R}^{N}\). Take \(w\in E\setminus \{0 \}\). Then, for large enough \(t>1\), Eq. (3.27) implies that
$$\begin{aligned} I_{\lambda}(tw)&=\mathcal{M} \bigl( [t w ]_{s,p(\cdot,\cdot)} \bigr) + \int_{\mathbb {R}^{N}} \biggl(\frac{\mathcal{V}(x)}{p(x)} \lvert tw \rvert^{p(x)} \biggr)\,dx\\&\quad-\lambda \int_{\mathbb {R}^{N}} \biggl(\frac{\rho(x)}{r(x)} \lvert tw \rvert^{r(x)} \biggr)\, dx- \int_{\mathbb {R}^{N}}H(x,tw)\,dx \\ &\le\mathcal{M}(1) \bigl([t w ]_{s,p(\cdot,\cdot)} \bigr)^{\vartheta} + \frac {1}{p_{-}} \int_{\mathbb {R}^{N}}\mathcal{V}(x) \lvert tw \rvert ^{p(x)} \,dx \\ &\quad-\frac{\lambda}{r_{-}} \int_{\mathbb {R}^{N}}\rho(x) \lvert tw \rvert^{r(x)}\,dx- \int_{\mathbb {R}^{N}}H(x,tw)\,dx \\ &\le \lvert t \rvert^{\vartheta p^{+}} \biggl({\mathcal{M}(1)} {[w ]}^{\vartheta}_{s,p(\cdot,\cdot)} +\frac{1}{p_{-}} \int_{\mathbb {R}^{N}}\mathcal{V}(x) \lvert w \rvert^{p(x)} \,dx- M_{0} \int_{\mathbb {R}^{N}} \lvert w \rvert ^{\vartheta p^{+}}\,dx \biggr), \end{aligned}$$
where ϑ was given in (M2), because \(\mathcal{M}(\tau )\le \mathcal{M}(1)\tau^{\vartheta}\) for \(\tau\ge1\). If \(M_{0}\) is large enough, then we deduce that \(I_{\lambda}(tw)\to-\infty\) as \(t \to \infty\), as required.
Thanks to Lemmas 3.4 and 3.9, there exists a positive number \(\lambda^{*}\) such that for all \(\lambda\in(0, \lambda ^{*})\), \(I_{\lambda}\) satisfies the mountain pass geometry and \((C)\)-condition. By employing the mountain pass theorem, we infer that there exists a critical point \(z_{0} \in E \) of \(I_{\lambda}\) with \(I_{\lambda}(z_{0})= \overline{d}>0=I_{\lambda}(0)\). Hence \(z_{0}\) is a nontrivial weak solution of the problem (\(P_{\lambda }\)). Let us denote \(d:=\inf_{z \in\overline{B}_{r}}{I_{\lambda}(z)}\) where \({B}_{r} := \{ z \in E : \|z\|_{E} < r \}\) with a boundary \(\partial{B}_{r}\). Then by (3.10) and Lemma 3.4(3), we have \(-\infty< d < 0\). Putting \(0< \epsilon< \inf_{z \in\partial{B}_{r}}{I_{\lambda }(z)}-d\), by Lemma 3.11, we can choose \(z_{\epsilon} \in \overline{B}_{r}\) such that
$$ \textstyle\begin{cases} I_{\lambda}( z_{\epsilon} ) \leq d + \epsilon, \\ I_{\lambda}( z_{\epsilon} ) < I_{\lambda}(z) + \frac{\epsilon}{1+ \Vert z_{\epsilon} \Vert _{E} } \Vert z-z_{\epsilon} \Vert _{E},\quad\text{for all } z \in\overline{B}_{r}, z \neq z_{\epsilon}. \end{cases} $$
(3.28)
This implies that \(z_{\epsilon} \in{B}_{r}\) since \(I_{\lambda}( z_{\epsilon} ) \leq d + \epsilon< \inf_{z \in\partial {B}_{r}}{I_{\lambda}(z)}\). From these facts we see that \(z_{\epsilon}\) is a local minimum of \(\widetilde{I}_{\lambda}( z ) = I_{\lambda}(z) + \frac{\epsilon }{1+ \|z_{\epsilon}\|_{E} } \|z-z_{\epsilon} \|_{E}\). Now by taking \(z = z_{\epsilon} +tw \) for \(w \in{B}_{1}\) and sufficiently small \(t>0\), from (3.28), we deduce
$$0 \leq\frac{\widetilde{I}_{\lambda}( z_{\epsilon} +tw )-\widetilde {I}_{\lambda}( z_{\epsilon} ) }{t} = \frac{{I_{\lambda}}( z_{\epsilon} +tw )-{I_{\lambda}}( z_{\epsilon} ) }{t} + \frac {\epsilon}{1+ \Vert z_{\epsilon} \Vert _{E} } \Vert w \Vert _{E}. $$
Therefore, letting \(t \rightarrow0+ \), we get
$$\bigl\langle I_{\lambda}^{\prime}(z_{\epsilon}),w \bigr\rangle + \frac{\epsilon}{1+ \Vert z_{\epsilon} \Vert _{E} } \Vert w \Vert _{E} \geq0. $$
Replacing w by −w in the argument above, we have
$$- \bigl\langle I_{\lambda}^{\prime}(z_{\epsilon}),w \bigr\rangle + \frac{\epsilon}{1+ \Vert z_{\epsilon} \Vert _{E} } \Vert w \Vert _{E} \geq0. $$
Thus, one has
$$\bigl(1+ \Vert z_{\epsilon} \Vert _{E} \bigr) \bigl\vert \bigl\langle I_{\lambda}^{\prime }(z_{\epsilon}),w \bigr\rangle \bigr\vert \leq\epsilon \Vert w \Vert _{E}, $$
for any \(w \in\overline{B}_{1}\). Hence we have
$$ \bigl(1+ \Vert z_{\epsilon} \Vert _{E} \bigr) \bigl\Vert I_{\lambda}^{\prime}(z_{\epsilon}) \bigr\Vert _{E^{*}} \leq\epsilon. $$
(3.29)
Using (3.28) and (3.29), we can choose a sequence \(\{ z_{n} \} \subset B_{r} \) such that
$$ \textstyle\begin{cases} I_{\lambda}( z_{n} ) \rightarrow d \quad\text{as } n \to\infty, \\ (1+ \Vert z_{n} \Vert _{E} ) \Vert I_{\lambda}^{\prime}(z_{n}) \Vert _{E^{*}} \to0 \quad \text{as } n \to\infty. \end{cases} $$
(3.30)
Thus, \(\{ z_{n} \} \) is a bounded Cerami sequence in the reflexive Banach space E. According to Lemma 3.9, \(\{ z_{n} \} \) has a subsequence \(\{ z_{n_{k}} \} \) such that \(z_{n_{k}} \to z_{1} \) in E as \(k \to\infty\). This together with (3.30) shows that \(I_{\lambda }( z_{1} ) =d\) and \(I_{\lambda}^{\prime}(z_{1})=0\). Hence \(z_{1}\) is a nontrivial nonnegative solution of the given problem with \(I_{\lambda }( z_{1} )<0\) which is different from \(z_{0}\). This completes the proof. □