4.1 The proof of statement (i)
First, if \(q>2\), under the weaker assumption (\(\mathrm{H}_{2}'\)), the number \(\lambda_{0}>0\). In fact, if \(u\in E\) is a nontrivial solution of problem (\({P}_{\lambda }\)), then taking \(\varphi=u\) in (2.1), from (1.2) we have that
$$\begin{aligned} \|u\|^{2}_{E}&=\lambda \int_{\Omega}f(x)|u|^{q}\,dx- \int_{\Omega }h(x)g(x,u)u\,dx \\ &\leq\lambda \int_{\Omega}f(x)|u|^{q}\,dx-b_{1} \int_{\Omega}h(x)|u|^{r}\,dx. \end{aligned}$$
(4.1)
Letting \(k_{1}=\lambda f(x)\), \(k_{2}=b_{1}h(x)\), \(\eta=q-2\), \(\beta=r-2\), and \(t=u\) in Lemma 2.3, we get that
$$ \lambda f(x)|u|^{q-2}-b_{1}h(x)|u|^{r-2} \leq\lambda^{\frac{r-2}{r-q}} \biggl(\frac{1}{b_{1}} \biggr)^{\frac{q-2}{r-q}} \biggl(\frac {f^{r-2}}{h^{q-2}} \biggr) ^{\frac{1}{r-q}}. $$
(4.2)
Then it follows from (4.1), (4.2), and Lemma 2.1 that
$$\begin{aligned} \|u\|^{2}_{E} &\leq\lambda^{\frac{r-2}{r-q}} \biggl( \frac{1}{b_{1}} \biggr)^{\frac {q-2}{r-q}} \int_{\Omega} \biggl(\frac{f^{r-2}}{h^{q-2}} \biggr)^{\frac {1}{r-q}} \vert u \vert ^{2}\,dx \\ &\leq C^{2}_{2^{*}_{s}} \lambda^{\frac{r-2}{r-q}} b_{1}^{\frac {2-q}{r-q}} \biggl\Vert \frac{f^{\frac{r-2}{r-q}}}{h^{\frac{q-2}{r-q}}} \biggr\Vert _{L^{\frac{N}{2s}}(\Omega)} \Vert u \Vert ^{2}_{E}. \end{aligned}$$
Thus
$$\biggl(1-C^{2}_{2^{*}_{s}} \lambda^{\frac{r-2}{r-q}} b_{1}^{\frac {2-q}{r-q}} \biggl\Vert \frac{f^{\frac{r-2}{r-q}}}{h^{\frac{q-2}{r-q}}} \biggr\Vert _{L^{\frac{N}{2s}}(\Omega)} \biggr) \Vert u \Vert ^{2}_{E}\leq0, $$
which implies that
$$\lambda\geq \biggl(C^{2}_{2^{*}_{s}} b_{1}^{\frac{2-q}{r-q}} \biggl\Vert \frac {f^{\frac{r-2}{r-q}}}{h^{\frac{q-2}{r-q}}} \biggr\Vert _{L^{\frac{N}{2s}}(\Omega)} \biggr) ^{\frac{q-r}{r-2}}=M_{0}>0. $$
Then \(\lambda_{0}\geq M_{0}>0\).
In addition, since (H2) is replaced by a weaker assumption (\(\mathrm{H}_{2}'\)), the proof of Theorem 1.1 becomes more complicated. The treatment of coercivity of the functional \(\Psi _{\pm}\) has to be adapted. Indeed, for any \(\delta>0\) and \(M>0\), we can decompose \(\Omega=X\cup Y\cup Z\) with measurable sets X, Y, Z defined as follows:
$$\textstyle\begin{cases} X =\{x\in\Omega:f(x)< M \mbox{ and } h(x)>\delta\}, \\ Y =\{x\in\Omega:f(x)< M \mbox{ and } h(x)\leq\delta\}, \\ Z =\{x\in\Omega:f(x)\geq M\}. \end{cases} $$
In Lemma 2.3, setting \(t=u^{+}\), \(k_{1}=\frac{\lambda f(x)}{q}\), \(k_{2}=\frac {b_{1}h(x)}{r}\), \(\eta=q\), and \(\beta=r\) and integrating over X, we have
$$\begin{aligned} \int_{X} \biggl(\frac{\lambda f(x)}{q} \bigl\vert u^{+} \bigr\vert ^{q}-\frac {b_{1}h(x)}{r} \bigl\vert u^{+} \bigr\vert ^{r} \biggr)\,dx&\leq \biggl( \frac{\lambda}{q} \biggr)^{\frac{r}{r-q}} \biggl(\frac {r}{b_{1}} \biggr) ^{\frac{q}{r-q}} \int_{X} \biggl(\frac{f(x)^{r}}{ h(x)^{q}} \biggr)^{\frac{1}{r-q}}\,dx \\ &\leq C_{2}, \end{aligned}$$
(4.3)
where \(C_{2}=C_{2}(M,\delta,\Omega,\lambda)\) is a constant.
Meanwhile, we apply (2.2) with \(\eta=q-2\) and \(\beta=r-2\) and integrate over \(Y\cup Z\) to derive
$$\begin{aligned} & \int_{Y\cup Z} \biggl(\frac{\lambda f(x)}{q} \bigl\vert u^{+} \bigr\vert ^{q} -\frac{b_{1}h(x)}{r} \bigl\vert u^{+} \bigr\vert ^{r} \biggr)\,dx \\ &\quad \leq \biggl(\frac {\lambda}{q} \biggr)^{\frac{r-2}{r-q}} \biggl( \frac{r}{b_{1}} \biggr)^{\frac{q-2}{r-q}} \int_{Y\cup Z} \biggl(\frac{f^{r-2}}{h^{q-2}} \biggr)^{\frac{1}{r-q}} \bigl\vert u^{+} \bigr\vert ^{2}\,dx \\ &\quad \leq C^{2}_{2^{*}_{s}} \biggl(\frac{\lambda}{q} \biggr)^{\frac {r-2}{r-q}} \biggl(\frac{r}{b_{1}} \biggr)^{\frac{q-2}{r-q}} \biggl\Vert \frac{f^{\frac{r-2}{r-q}}}{h^{\frac{q-2}{r-q}}} \biggr\Vert _{L^{\frac{N}{2s}}(Y\cup Z)} \bigl\Vert u^{+} \bigr\Vert ^{2}_{E}. \end{aligned}$$
(4.4)
Since \(f,h\in L^{1}_{\mathrm{loc}}(\Omega)\), this results in that \(|Z|\rightarrow0\) as \(M\rightarrow\infty\) and, for fixed M, \(|Y|\rightarrow0\) as \(\delta\rightarrow0\). Thus, for any \(\varepsilon >0\), we can choose M sufficiently large and then δ sufficiently small such that
$$ C^{2}_{2^{*}_{s}} \biggl(\frac{\lambda}{q} \biggr)^{\frac{r-2}{r-q}} \biggl(\frac{r}{b_{1}} \biggr)^{\frac{q-2}{r-q}} \biggl\Vert \frac{f^{\frac{r-2}{r-q}}}{h^{\frac{q-2}{r-q}}} \biggr\Vert _{L^{\frac{N}{2s}}(Y\cup Z)} < \varepsilon. $$
(4.5)
Combining (4.3)–(4.5), we have
$$ \int_{\Omega} \biggl(\frac{\lambda f(x)}{q} \bigl\vert u^{+} \bigr\vert ^{q} -\frac {b_{1}h(x)}{r} \bigl\vert u^{+} \bigr\vert ^{r} \biggr)\,dx\leq C_{2}+ \varepsilon \bigl\Vert u^{+} \bigr\Vert ^{2}_{E}. $$
(4.6)
Since \(u^{+}=\frac{|u|+u}{2}\), we get
$$\begin{aligned} & \iint_{\Omega}\frac {|u^{+}(x)-u^{+}(y)|^{2}}{|x-y|^{N+2s}}\,dx\,dy \\ &\quad = \iint_{\Omega}\frac{ \vert |u(x)|-|u(y)|+u(x)-u(y) \vert ^{2}}{2^{2}|x-y|^{N+2s}}\,dx\,dy \\ &\quad \leq \iint_{\Omega}\frac{|u(x)-u(y)|^{2}}{|x-y|^{N+2s}}\,dx\,dy. \end{aligned}$$
Furthermore, since \(\int_{\Omega}v(x) |u^{+}(x)|^{2}\,dx\leq\int_{\Omega}v(x) |u(x)|^{2}\,dx\), it follows that
$$ \bigl\Vert u^{+} \bigr\Vert ^{2}_{E} \leq \Vert u \Vert ^{2}_{E}. $$
(4.7)
Taking \(\varepsilon=\frac{1}{4}\) in (4.6) and using (4.7), we obtain
$$\Psi_{+}(u)\geq\frac{1}{4} \Vert u \Vert ^{2}_{E}-C_{2}. $$
Thus \(\Psi_{+}(u)\) is coercive in E. In the same way, \(\Psi_{-}(u)\) is also coercive in E.
So far, all the necessary conditions are satisfied, and we can follow the same lines as in Sect. 3 to get the results of Theorem 1.1.
4.2 The cases \(\lambda=\lambda_{0},\lambda_{*},\lambda^{*}\)
For \(\lambda=\lambda_{0}\), let \(\{\lambda_{n}\}\) be a decreasing sequence converging to \(\lambda_{0}\), and let \(\{u_{n}\}\) be a corresponding sequence of solutions to problems \(\{P_{\lambda_{n}}\}\). By (4.6) we get
$$\frac{\lambda_{n}}{q} \int_{\Omega}f(x) \vert u_{n} \vert ^{q} \,dx-\frac {b_{1}}{r} \int_{\Omega}h(x) \vert u_{n} \vert ^{r} \,dx\leq C_{2n}+\varepsilon \Vert u_{n} \Vert ^{2}_{E}, $$
where \(C_{2n}=C_{2}(M,\delta,\Omega,\lambda_{n})>0\). Then
$$\begin{aligned} \Vert u_{n} \Vert ^{2}_{E}&\leq \lambda_{n} \int_{\Omega}f(x) \vert u_{n} \vert ^{q} \,dx-b_{1} \int_{\Omega}h(x) \vert u_{n} \vert ^{r} \,dx \\ &\leq\frac{b_{1}q}{r} \int_{\Omega}h(x) \vert u_{n} \vert ^{r} \,dx+qC_{2n}+q\varepsilon \Vert u_{n} \Vert ^{2}_{E} -b_{1} \int_{\Omega}h(x) \vert u_{n} \vert ^{r} \,dx \\ &=\frac {b_{1}(q-r)}{r} \int_{\Omega}h(x) \vert u_{n} \vert ^{r} \,dx+qC_{2n}+q\varepsilon \Vert u_{n} \Vert ^{2}_{E} \\ &\leq qC_{2n}+q\varepsilon \Vert u_{n} \Vert ^{2}_{E}. \end{aligned}$$
Setting \(\varepsilon=\frac{1}{2q}\), we obtain
$$\Vert u_{n} \Vert ^{2}_{E} \leq2qC_{2n}. $$
Since \(\{\lambda_{n}\}\) is bounded, it follows that \(C_{2n}\) is bounded by a constant independent of n. Thus by Lemma 2.1 and 2.2, there exist a subsequence of \(\{u_{n}\}\), still denoted by \(\{u_{n}\}\), and \(u_{\lambda_{0}}\in E\) such that
Since \(u_{n}\) are solutions of (\(P_{\lambda_{n}}\)), for all \(\varphi\in E\), we have
$$\langle u_{n},\varphi\rangle_{E}- \lambda_{n} \int_{\Omega} f(x) \vert u_{n} \vert ^{q-2}u_{n}\varphi \,dx - \int_{\Omega}h(x)g(x,u_{n})\varphi \,dx=0. $$
Passing to the limit as \(n\rightarrow\infty\), we deduce that \(u_{\lambda _{0}}\) solves problem (\(P_{\lambda_{0}}\)).
We finally claim that \(u_{\lambda_{0}}\not\equiv0\). Indeed, taking \(\varphi=u_{\lambda_{0}}\) in the definition of weak solution for \(u_{\lambda_{0}}\), we have from (1.2) and Lemma 2.2 that
$$\begin{aligned} \|u_{\lambda_{0}}\|^{2}_{E}&\leq\lambda_{0} \|u_{\lambda_{0}}\|_{f}^{q}-b_{1} \|u_{\lambda_{0}}\|^{r}_{h} \\ &\leq\lambda_{0}\|u_{\lambda_{0}}\|_{f}^{q} \leq\lambda_{0}C_{f}\| u_{\lambda_{0}}\|^{q}_{E}. \end{aligned}$$
Taking into account that \(q>2\), we obtain
$$\Vert u_{\lambda_{0}} \Vert _{E}\geq \biggl(\frac{1}{\lambda _{0}C_{f}} \biggr) ^{\frac{1}{q-2}}>0, $$
which implies \(u_{\lambda_{0}}\not\equiv0\). Hence problem (\(P_{\lambda_{0}}\)) admits at least one nontrivial solution.
For \(\lambda=\lambda_{*}\), without loss of generality, we can assume that \(\lambda_{*}=\lambda_{+}^{*}<\lambda_{-}^{*}\). Then there also exists a decreasing sequence \(\{\lambda_{n}\}\) converging to \(\lambda _{*}\). Let \(u_{n}\in E\) be a nontrivial positive solution of problem (\(P_{\lambda_{n}}\)). It suffices to repeat the previous arguments to conclude that (\(P_{\lambda_{*}}\)) has at least one nontrivial positive solution.
For \(\lambda=\lambda^{*}\), assuming that \(\lambda^{*}=\lambda _{+}^{*}>\lambda_{-}^{*}\), \(\{P_{\lambda_{*}}\}\) admits at least one nontrivial positive solution. Furthermore, since \(\lambda ^{*}>\lambda_{-}^{*}\), we deduce that \(\{P_{\lambda_{*}}\}\) also admits a nontrivial negative solution. So, the second statement of Theorem 1.2 follows.
4.3 The case \(\lambda\geq\tilde{\lambda}\)
Let
$$\tilde{\lambda}=\max\{\lambda_{+},\lambda_{-}\}. $$
For \(\lambda>\tilde{\lambda}\), we already know that Ψ has two nontrivial critical points \(u_{1}\geq0\) and \(u_{2}\leq0\). Now let us show that if \(q>2\), then (\({P}_{\lambda }\)) has two additional nontrivial solutions.
Lemma 4.1
There exist
\(\rho\in(0,\|u_{1}\|_{E})\)
and
\(\alpha>0\)
such that
\(\Psi _{+}(u)\geq\alpha\)
for all
\(u\in E\)
with
\(\|u\|_{E}=\rho\).
Proof
Since \(h(x)G(x,\cdot)\geq0\) in Ω, we have
$$\begin{aligned} \Psi_{+}(u)&=\frac{1}{2} \Vert u \Vert ^{2}_{E}- \frac{\lambda}{q} \int_{\Omega}f(x) \bigl\vert u^{+} \bigr\vert ^{q}\,dx+ \int_{\Omega }h(x)G\bigl(x,u^{+}\bigr)\,dx \\ &\geq\frac{1}{2} \Vert u \Vert ^{2}_{E}- \frac{\lambda}{q} \bigl\Vert u^{+} \bigr\Vert ^{q}_{f,q} \\ &\geq\frac{1}{2} \Vert u \Vert ^{2}_{E}- \frac{\lambda}{q} \Vert u \Vert ^{q}_{f,q}. \end{aligned}$$
From Lemma 2.2 we see that
$$\begin{aligned} \Psi_{+}(u)&\geq\frac{1}{2} \Vert u \Vert ^{2}_{E}- \frac{\lambda}{q}C_{f} \Vert u \Vert ^{q}_{E} \\ &= \Vert u \Vert ^{2}_{E} \biggl(\frac{1}{2}- \frac{\lambda}{q}C_{f} \Vert u \Vert ^{q-2}_{E} \biggr) \\ &=\rho^{2} \biggl(\frac{1}{2}-\frac{\lambda}{q}C_{f} \rho^{q-2} \biggr). \end{aligned}$$
Take \(0<\rho<\min \{ \Vert u_{1} \Vert _{E}, (\frac {q}{2C_{f}\lambda} ) ^{\frac{1}{q-2}} \}\). Then
$$\alpha=\rho^{2} \biggl(\frac{1}{2}-\frac{\lambda}{q}C_{f} \rho^{q-2} \biggr)>0 $$
satisfies the statement. □
Next, we prove that the Palais–Smale condition holds for every sequence \(\{u_{n} \}\subset E\).
Lemma 4.2
Assume that a sequence
\(\{u_{n} \}\subset E\)
such that
\(\{\Psi _{+}(u_{n})\}\)
is bounded and
\(\Psi'_{+}(u_{n})\rightarrow0\)
in
\(E^{*}\)
as
\(n\rightarrow\infty\). Then
\(\{u_{n}\}\)
has a convergent subsequence in
E.
Proof
Since \(\Psi_{+}\) is coercive, we have that \(\{u_{n} \}\) is bounded in E. It is possible to extract a sequence, still relabeled by \(\{u_{n}\}\), such that \(\{u_{n}\}\) converges weakly to some u in E. By Lemma 2.2 we obtain
$$ u_{n}\rightarrow u \quad \mbox{in } L^{q}_{f}(\Omega)\quad \mbox{and}\quad u_{n} \rightarrow u \quad \mbox{in } L^{r}_{h}(\Omega). $$
(4.8)
Then \(\{u_{n}\}\) is bounded in \(L^{q}_{f}(\Omega)\) and \(L^{r}_{h}(\Omega )\) with \(\|u_{n}\|_{f,q}\leq C_{3}\) and \(\|u_{n}\|_{h,r}\leq C_{4}\). Observe that
$$\begin{aligned} \Psi_{+}'(u_{n}) (u_{n}-u)&= \langle u_{n},u_{n}-u\rangle_{E}- \int _{\Omega}j\bigl(x,u_{n}^{+}\bigr) (u_{n}-u)\,dx \\ &=\langle u_{n},u_{n}\rangle_{E}-\langle u_{n},u\rangle_{E}- \int_{\Omega }j\bigl(x,u_{n}^{+}\bigr) (u_{n}-u)\,dx, \end{aligned}$$
(4.9)
where \(j(x,u_{n}^{+})=\lambda f(x) \vert u_{n}^{+} \vert ^{q-2}u_{n}^{+}-h(x)g(x,u_{n}^{+})\). First, note that
$$\langle u_{n},u\rangle_{E} \rightarrow\langle u,u \rangle_{E}. $$
In addition, using Hölder’s inequality and claim (4.8), we have
$$\begin{aligned} \biggl\vert \int_{\Omega}j\bigl(x,u_{n}^{+}\bigr) (u_{n}-u)\,dx \biggr\vert &\leq\lambda C^{q-1}_{3} \Vert u_{n}-u \Vert _{f,q}+b_{2}C_{4}^{r-1} \Vert u_{n}-u \Vert _{h,r} \\ &\rightarrow0\quad \mbox{as } n\rightarrow\infty. \end{aligned}$$
Since \(\Psi_{+}'(u_{n})(u_{n}-u)\rightarrow0\) as \(n\rightarrow\infty \), passing to the limit in (4.9), we obtain \(\Vert u_{n} \Vert _{E}\rightarrow \Vert u \Vert _{E}\) as \(n\rightarrow\infty\). Therefore
$$\begin{aligned} & \Vert u_{n}-u \Vert _{E}^{2} \\ &\quad = \Vert u_{n} \Vert _{E}^{2}+ \Vert u \Vert _{E}^{2}-2 \iint_{\mathbb{R}^{2N}} \frac{(u_{n}(x)-u_{n}(y))(u(x)-u(y))}{|x-y|^{N+2s}}\,dx\,dy \\ &\qquad {} -2 \int_{\Omega}v(x) u_{n}(x)u(x)\,dx \\ &\quad \rightarrow2 \Vert u \Vert _{E}^{2}-2 \Vert u \Vert _{E}^{2}=0 \end{aligned}$$
as \(n\rightarrow\infty\). Then the Palais–Smale condition holds. □
As a consequence, the mountain pass theorem guarantees the existence of a critical point \(u_{3}\) for \(\Psi_{+}\) with \(\Psi_{+}(u_{3})\geq\alpha >0\), which implies that \(u_{3}\) is nontrivial and different from \(u_{1}\) and \(u_{2}\). Moreover, working as we did for \(u_{1}\), we can show that \(u_{3}\geq0\). So, \(u_{3}\) is a critical point of Ψ. The third nontrivial solution of problem (\({P}_{\lambda }\)) is obtained.
Similarly, we apply the mountain pass theorem to \(\Psi_{-}\) to prove the existence of the fourth nontrivial weak solution of problem (\({P}_{\lambda }\)) having negative sign and different from the previous ones. Thus, when \(\lambda>\tilde{\lambda}\), four nontrivial solutions have been obtained.
If \(\lambda=\tilde{\lambda}\), then we can assume that \(\tilde {\lambda}=\lambda_{+}>\lambda_{-}\). Let \(\lambda_{n}=\tilde{\lambda }+\frac{1}{n}\). Then \(\lambda_{n}\rightarrow\tilde{\lambda}\) as \(n\rightarrow\infty\) and \(\lambda_{n}>\tilde{\lambda}\) for \(n\in\mathbb{N}_{+}\). So, for every \(n\in\mathbb{N}_{+}\), there exist two nontrivial positive solutions \(u_{n_{1}}\) and \(u_{n_{2}}\) of problem (\(P_{\lambda_{n}}\)) with \(\Psi(u_{n_{1}})<0\) and \(\Psi(u_{n_{2}})\geq\alpha>0\). Proceeding as in Sect. 4.2, we get that there exist \(u_{\tilde{\lambda}_{1}}\geq0\) and \(u_{\tilde {\lambda}_{2}}\geq0\) such that \(u_{n_{1}}\rightharpoonup u_{\tilde {\lambda}_{1}}\) in E and \(u_{n_{2}}\rightharpoonup u_{\tilde {\lambda}_{2}}\) in E. Furthermore, we can verify that \(u_{\tilde {\lambda}_{1}}\) and \(u_{\tilde{\lambda}_{2}}\) are two weak solutions of problem (\(P_{\tilde{\lambda}}\)) with \(\Psi ({u_{\tilde{\lambda}_{1}}})<0\) and \(\Psi({u_{\tilde{\lambda }_{2}}})\geq\alpha>0\), which implies that \(u_{\tilde{\lambda }_{1}}\neq u_{\tilde{\lambda}_{2}}\). On the other hand, since \(\tilde{\lambda}>\lambda_{-}\), (\(P_{\tilde{\lambda}}\)) admits two different nontrivial negative solutions. Thus, for \(\lambda=\tilde{\lambda}\), (\(P_{\tilde{\lambda}}\)) has at least four nontrivial solutions.