Let \(\mathcal{J}:W^{1,p}_{0}(\varOmega )\rightarrow {\mathbb{R}} \) be the variational functional defined by

$$ \mathcal{J}(u)=\frac{1}{2} \int _{\varOmega }\bigl(A\bigl( \vert \nabla u \vert ^{2} \bigr)-\lambda A\bigl(u^{2}\bigr)\bigr) \,dx-\frac{1}{q} \int _{\varOmega } \vert u \vert ^{q} \,dx. $$

By hypothesis (7) and the Sobolev embedding theorem, \(\mathcal{J}\) is well defined. Moreover, \(\mathcal{J}\in C^{1}(W^{1,p}_{0}(\varOmega ),{\mathbb{R}} )\) and its Gâteux directional derivative is given by

$$ \bigl\langle \mathcal{J}'(u),v\bigr\rangle = \int _{\varOmega }\bigl(a\bigl( \vert \nabla u \vert ^{2} \bigr) \nabla u\nabla v-\lambda a\bigl(u^{2}\bigr)uv\bigr) \,dx - \int _{\varOmega } \vert u \vert ^{q-2}uv \,dx $$

for any *u*, \(v\in W^{1,p}_{0}(\varOmega )\).

Since the problem has a variational structure, then solutions of problem (6) are critical points of the energy functional \(\mathcal{J}\).

We define the truncation

$$ h(t)= \textstyle\begin{cases} t^{q-1}& \text{if $t> 0$}, \\ 0& \text{if $t\geq 0$}, \end{cases} $$

and we set \(H(t):=\int _{0}^{t}h(s)\,ds\).

Consider the variational functional

$$ \mathcal{E}(u)=\frac{1}{2} \int _{\varOmega }\bigl(A\bigl( \vert \nabla u \vert ^{2} \bigr)-\lambda A\bigl(u^{2}\bigr)\bigr) \,dx- \int _{\varOmega }H(u)\,dx. $$

Then \(\mathcal{E}\) is well defined, \(\mathcal{E}\in C^{1}(W_{0}^{1,p}(\varOmega ))\) and, for all \(u,v\in W_{0}^{1,p}(\varOmega )\),

$$ \bigl\langle \mathcal{E}'(u),v\bigr\rangle = \int _{\varOmega }\bigl[a\bigl( \vert \nabla u \vert ^{2} \bigr) \nabla u\nabla v-\lambda a\bigl(u^{2}\bigr)uv\bigr] \,dx- \int _{\varOmega }h(u)v\,dx. $$

### 3.1 Verification of the Palais–Smale condition

Let \((u_{n})\subset W^{1,p}_{0}(\varOmega )\) be a Palais–Smale sequence of \(\mathcal{E}\), that is,

$$ \mathcal{E}(u_{n})=O(1) \quad\text{and}\quad \bigl\Vert \mathcal{E}'(u_{n}) \bigr\Vert _{W^{-1,p'}( \varOmega )}=o(1) \quad\text{as } n\rightarrow \infty. $$

It follows that

$$ \frac{1}{2} \int _{\varOmega }A\bigl( \vert \nabla u_{n} \vert ^{2}\bigr)\,dx-\frac{\lambda }{2} \int _{\varOmega }A\bigl(u_{n}^{2}\bigr)\,dx- \int _{\varOmega }H(u_{n})\,dx=O(1)$$

(10)

and, for all \(v\in W^{1,p}_{0}(\varOmega )\),

$$ \int _{\varOmega }a\bigl( \vert \nabla u_{n} \vert ^{2}\bigr)\nabla u_{n}\nabla v\,dx-\lambda \int _{\varOmega }a\bigl(u_{n}^{2} \bigr)u_{n}v\,dx- \int _{\varOmega }h(u_{n})v\,dx=o(1) \Vert v \Vert .$$

(11)

Taking \(v=u_{n}\) in (11), we obtain

$$ \int _{\varOmega }a\bigl( \vert \nabla u_{n} \vert ^{2}\bigr)|\nabla u_{n}^{2}\,dx-\lambda \int _{\varOmega }a\bigl(u_{n}^{2} \bigr)u_{n}^{2}\,dx- \int _{\varOmega }h(u_{n})u_{n}\,dx=o(1) \Vert u_{n} \Vert .$$

(12)

But, for all \(n\geq 1\),

$$ \int _{\varOmega }h(u_{n})u_{n}\,dx=q \int _{\varOmega }H(u_{n})\,dx.$$

(13)

Relations (10), (11), and (13) yield

$$\begin{aligned} (1-q) \int _{\varOmega }H(u_{n})\,dx={}& \int _{\varOmega } \biggl[ \frac{1}{2} A\bigl( \vert \nabla u_{n} \vert ^{2}\bigr)-a\bigl( \vert \nabla u_{n} \vert ^{2}\bigr) \vert \nabla u_{n} \vert ^{2} \biggr]\,dx \\ &{}-\lambda \int _{\varOmega } \biggl[\frac{1}{2} A\bigl( u_{n}^{2}\bigr)-a\bigl( u_{n}^{2}\bigr) u_{n}^{2} \biggr]\,dx+O(1) \quad\text{as } n\rightarrow \infty. \end{aligned}$$

(14)

But \(A(t^{2})=2\int _{0}^{t}sa(s^{2})\,ds\) and the mapping \((0,+\infty )\ni s\mapsto sa(s^{2})\) is increasing. Therefore

$$ A\bigl(t^{2}\bigr)\leq 2t^{2}a \bigl(t^{2}\bigr) \quad\text{for all } t\geq 0.$$

(15)

*Claim.* The sequence \((u_{n})\subset W^{1,p}_{0}(\varOmega )\) is bounded.

For this purpose, we first show that

$$ \text{the sequence $(u_{n})$ is bounded in $L^{q}(\varOmega )$}.$$

(16)

Assume first that \(\lambda \leq 0\). Thus, from (14) and (15) we deduce that

$$ 0\leq (q-1) \int _{\varOmega }H(u_{n})\,dx\leq O(1) \quad\text{as } n \rightarrow \infty. $$

hence \((u_{n})\) is bounded in \(L^{q}(\varOmega )\).

If \(\lambda >0\), relations (14) and (15) yield

$$ 0\leq (q-1) \int _{\varOmega }H(u_{n})\,dx\leq \lambda \int _{\varOmega } \biggl[ \frac{1}{2} A\bigl( u_{n}^{2}\bigr)-a\bigl( u_{n}^{2}\bigr) u_{n}^{2} \biggr]\,dx.$$

(17)

Since \(1< p< q\), relation (17) shows that \((u_{n})\) is bounded in \(L^{q}(\varOmega )\).

By (8) and since \(\lambda <\varLambda \), there exists \(c_{0}>0\) such that, for all \(u\in W^{1,p}_{0}(\varOmega )\),

$$ \int _{\varOmega }A\bigl( \vert \nabla u \vert ^{2}\bigr) \,dx-\lambda \int _{\varOmega }A\bigl(u^{2}\bigr)\,dx\geq c_{0} \int _{\varOmega }A\bigl( \vert \nabla u \vert ^{2}\bigr) \,dx.$$

(18)

Returning to (10) and using (16) in combination with (18) and the hypothesis \(\lambda <\varLambda \), we obtain the claim.

Next, we prove that \((u_{n})\subset W^{1,p}_{0}(\varOmega )\) contains a strongly convergent subsequence.

Relation (11) yields, for all \(v\in W^{1,p}_{0}(\varOmega )\),

$$ \int _{\varOmega }a\bigl( \vert \nabla u_{n} \vert ^{2}\bigr)\nabla u_{n}\nabla v\,dx= \int _{\varOmega }\psi (u_{n})v\,dx+o(1) \Vert v \Vert \quad\text{as } n\rightarrow \infty,$$

(19)

where

$$ \psi (w)=\lambda a\bigl(w^{2}\bigr)w+h(w) \quad\text{for all } w\in W^{1,p}_{0}( \varOmega ). $$

Obviously, *ψ* is a continuous function.

Assume that \(1< p< N\) (similar arguments work if \(p\geq N\)). It follows that

$$ \bigl\vert \psi (w) \bigr\vert \leq C\bigl(1+ \vert w \vert ^{(Np-N+p)/(N-p)}\bigr) \quad\text{for all } w\in W^{1,p}_{0}( \varOmega )$$

(20)

and

$$ \psi (w)=o\bigl( \vert w \vert ^{Np/(N-p)}\bigr) \quad\text{as } \vert w \vert \rightarrow \infty.$$

(21)

A crucial idea in the proof is to show that the sequence \(\{\psi (u_{n})\}\subset W^{-1,p'}(\varOmega )\) contains a strongly convergent subsequence. Indeed, in this case, relation (19) combined with the Sobolev embedding theorem implies that, up to a subsequence, \(\{\psi (u_{n})\}\) converges strongly in \((L^{Np/(N-p)}(\varOmega ))^{*}= L^{Np/(Np-N+p)}(\varOmega )\).

By our claim and the Rellich–Kondrachov embedding theorem, we can assume, up to a subsequence, that

$$ u_{n}\rightarrow u \quad\text{in } L^{Np/(N-p)}(\varOmega ). $$

Fix \(\delta >0\). Thus, by the Egorov theorem, there exists \(\omega \subset \varOmega \) such that \(|\omega | <\delta \) and

$$ u_{n}\rightarrow u \quad\text{uniformly in $\varOmega \setminus \omega $}. $$

Fix \(\eta >0\) small enough. Thus, to conclude the proof, it is enough to show that

$$ \int _{\omega } \bigl\vert \psi (u_{n})-\psi (u) \bigr\vert ^{Np/(Np-N+p)} \,dx\leq \eta\quad \text{for all $n$ big enough}. $$

Relation (20) implies that

$$ \int _{\omega } \bigl\vert \psi (u) \bigr\vert ^{Np/(Np-N+p)} \,dx\leq C \int _{\omega }\bigl(1+ \vert u \vert ^{Np/(N-p)}\bigr)\,dx $$

and the right-hand side can be made smaller than any positive constant if we choose \(\delta >0\) small enough.

Next, by (21),

$$ \int _{\omega } \bigl\vert \psi (u_{n})-\psi (u) \bigr\vert ^{Np/(Np-N+p)}\,dx\leq \varepsilon \int _{\omega } \vert u_{n}-u \vert ^{Np/(N-p)} \,dx+C_{\varepsilon } \vert \omega \vert , $$

and the right-hand side can be made as small as we wish. This follows by combining our claim with the Sobolev embedding theorem.

We conclude that the energy functional \(\mathcal{E}\) satisfies the Palais–Smale condition.

### 3.2 Proof of Theorem 1(i) concluded

We first prove that \(\mathcal{E}\) satisfies the geometric hypotheses of the mountain pass theorem.

Fix \(\lambda <\varLambda \). We have for all \(u\in W^{1,p}_{0}(\varOmega )\)

$$\begin{aligned} \mathcal{E}(u)&\geq \frac{c_{0}}{2} \int _{\varOmega }A\bigl( \vert \nabla u \vert ^{2}\bigr) \,dx- \int _{\varOmega }H(u)\,dx \\ &\geq \frac{c_{0}}{2} \int _{\varOmega }A\bigl( \vert \nabla u \vert ^{2}\bigr) \,dx- \frac{1}{q} \int _{\varOmega } \vert u \vert ^{q}\,dx, \end{aligned}$$

where \(c_{0}\) is as in (18).

Next, by (3), we obtain for all \(u\in W^{1,p}_{0}(\varOmega )\)

$$ \mathcal{E}(u)\geq \frac{c_{0}\gamma '}{2} \int _{\varOmega } \vert \nabla u \vert ^{p}\,dx- \frac{1}{q} \int _{\varOmega } \vert u \vert ^{q}\,dx. $$

Fix \(\varepsilon >0\) small enough. Thus, by (7) and the Sobolev embedding theorem, there exists \(r>0\) such that

$$ \mathcal{E}(u)\geq \varepsilon \quad\text{for all $u\in W^{1,p}_{0}( \varOmega )$ with $ \Vert u \Vert =r$}. $$

This establishes the existence of a “mountain” near the origin.

Next, we show the existence of a “valley” far from the origin.

Let \(\varphi _{1}>0\) be the first eigenfunction of the Laplace operator, hence \(\varphi _{1}\in W^{1,p}_{0}(\varOmega )\). For all \(t>0\), we have

$$\begin{aligned} \mathcal{E}(t\varphi _{1})={}& \frac{1}{2} \int _{\varOmega }\bigl(A\bigl(t^{2} \vert \nabla \varphi _{1} \vert ^{2}\bigr)-\lambda A\bigl(t^{2}\varphi _{1}^{2}\bigr)\bigr) \,dx- \frac{t^{q}}{q} \int _{\varOmega }\varphi _{1}^{q}\,dx \\ \leq{}& \frac{1}{2} \int _{[t \vert \nabla \varphi _{1} \vert \geq t_{0}]}A\bigl(t^{2} \vert \nabla \varphi _{1} \vert ^{2}\bigr)\,dx + \frac{1}{2} \int _{[t \vert \nabla \varphi _{1} \vert < t_{0}]}A\bigl(t^{2} \vert \nabla \varphi _{1} \vert ^{2}\bigr)\,dx \\ &{}-\frac{\lambda }{2} \int _{[t\varphi _{1}\geq t_{0}]} A\bigl(t^{2} \varphi _{1}^{2} \bigr)\,dx -\frac{\lambda }{2} \int _{[t\varphi _{1}< t_{0}]} A\bigl(t^{2} \varphi _{1}^{2} \bigr)\,dx-\frac{t^{q}}{q} \int _{\varOmega }\varphi _{1}^{q}\,dx, \end{aligned}$$

(22)

where \(t_{0}\) is defined in (3).

Next, we evaluate the terms arising in (22). By (3) we have

$$ \frac{1}{2} \int _{[t \vert \nabla \varphi _{1} \vert \geq t_{0}]}A\bigl(t^{2} \vert \nabla \varphi _{1} \vert ^{2}\bigr)\,dx\leq c_{1}t^{p}, \quad\text{where $c_{1}=\frac{\varGamma '}{2} \int _{\varOmega } \vert \nabla \varphi _{1} \vert ^{p}\,dx>0$}, $$

and

$$ -\frac{\lambda }{2} \int _{[t\varphi _{1}\geq t_{0}]} A\bigl(t^{2}\varphi _{1}^{2} \bigr)\,dx \leq c_{2}t^{p}, \quad\text{where $c_{2}=\frac{ \vert \lambda \vert \varGamma '}{2} \int _{\varOmega }\varphi _{1}^{p}\,dx>0$}. $$

Recall that \(A(t^{2})=2\int _{0}^{t}sa(s^{2})\,ds\). Thus, by (1),

$$ A(t)\leq 2p^{-1}\varGamma \bigl[(\kappa +t)^{p}-\kappa ^{p}\bigr]. $$

It follows that

$$\begin{aligned} \frac{1}{2} \int _{[t \vert \nabla \varphi _{1} \vert < t_{0}]}A\bigl(t^{2} \vert \nabla \varphi _{1} \vert ^{2}\bigr)\,dx\leq{}& \frac{\varGamma }{p} \int _{[t \vert \nabla \varphi _{1} \vert < t_{0}]} \bigl(\kappa +t \vert \nabla \varphi _{1} \vert \bigr)^{p}\,dx-c_{4} \\ &{}\times \frac{\varGamma }{p} \int _{\varOmega }(\kappa +t_{0})^{p} \,dx-c_{4}=O(1). \end{aligned}$$

Similarly, we deduce that

$$ -\frac{\lambda }{2} \int _{[t\varphi _{1}< t_{0}]} A\bigl(t^{2}\varphi _{1}^{2} \bigr)\,dx \leq O(1). $$

Returning now to (22) we obtain

$$ \mathcal{E}(t\varphi _{1})\leq (c_{1}+c_{2})t^{p}-c_{3}t^{q}+O(1), $$

where \(c_{3}=q^{-1}\int _{\varOmega }\varphi _{1}^{q}\,dx>0\). Using (7) we deduce that \(\mathcal{E}(t\varphi _{1})<0\) for *t* large enough.

We have verified all the hypotheses of the mountain pass theorem. It follows that \(\mathcal{E}\) has a nontrivial critical point *u*. Thus, for all \(v\in W^{1,p}_{0}(\varOmega )\),

$$ \int _{\varOmega }a\bigl( \vert \nabla u \vert ^{2}\bigr) \nabla u\nabla v\,dx=\lambda \int _{\varOmega }a\bigl(u^{2}\bigr)uv\,dx + \int _{\varOmega }h(u)v\,dx. $$

Taking \(v=u^{-}\) as a test function, we obtain

$$ \int _{\varOmega }a\bigl( \vert \nabla u \vert ^{2}\bigr) \bigl\vert \nabla u^{-} \bigr\vert ^{2}\,dx-\lambda \int _{\varOmega }a\bigl(u^{2}\bigr) \bigl\vert u^{-} \bigr\vert ^{2}\,dx=0. $$

Finally, by (8) and since \(\lambda <\varLambda \), we conclude that \(u^{-}=0\), hence \(u\geq 0\). This means that \(h(u)=u^{q-1}\), so *u* is a solution of problem (6).

It remains to show that \(u>0\) in *Ω*. For this purpose, we observe that relations (4) and (7) imply that the hypotheses of the generalized maximum principle of Pucci and Serrin [13] are fulfilled. We conclude that \(u>0\) in *Ω*.