Qualitative analysis of a strongly coupled predator–prey system with modified Holling–Tanner functional response

Abstract

In this paper, we deal with a strong coupled predator–prey model with modified Holling–Tanner functional response under homogeneous Neumann boundary conditions. We mainly discuss the following two problems: (1) stability of the positive constant solution; (2) existence and non-existence results as regards the non-constant positive solutions.

1 Introduction

Let $\Omega\subset \mathbb {R}^{N}$ $(N\geq1)$ be a bounded domain with a smooth boundary ∂Ω. We are concerned in this paper with a reaction–diffusion system of the following type:

$$ \textstyle\begin{cases} u_{t}-d_{1}\Delta u=ru(1-u)-\frac{\beta uv}{u+mv}&\mbox{in } \Omega\times \mathbb {R}_{+},\\ v_{t}-d_{2}\Delta [ (1+\frac{d_{3}}{1+\alpha u} )v ]=v (b-\frac{v}{u} )&\mbox{in } \Omega\times \mathbb {R}_{+},\\ \partial_{\nu}u=\partial_{\nu}v=0&\mbox{on } \partial \Omega\times \mathbb {R}_{+},\\ u(x,0)>0,\qquad v(x,0)\geq,\not\equiv0,&\mbox{in } \bar{\Omega}, \end{cases} $$

(1.1)

where u and v represent the species densities of prey and predator, respectively; $r>0$ is the intrinsic growth rate of the prey; $\beta uv/(u+mv)$ with $\beta, m>0$ is called ratio-dependent functional response. $b>0$ is the intrinsic growth rate of the predator; $d_{i}>0$ ($i=1,2$) are the diffusion coefficients for u and v, respectively; $d_{3}\geq0$, $d_{2}d_{3}$ is called the cross-diffusion coefficient; ν is the outward unit normal vector on the boundary ∂Ω and $\partial_{\nu}=\partial/\partial_{\nu}$. The homogeneous Neumann boundary condition means that (1.1) is self-contained, thus it has no population flux across ∂Ω. The initial data $u_{0}(x)$ and $v_{0}(x)$ are smooth functions on Ω̄. In this model, the predator v diffuses with flux

$$J=-\nabla \biggl(d_{2}v+\frac{d_{2}d_{3}v}{1+\alpha u} \biggr)=- \biggl(d_{2}+\frac{d_{2}d_{3}}{1+\alpha u} \biggr)\nabla v+\frac{\alpha d_{2}d_{3} v}{(1+\alpha u)^{2}}\nabla u. $$

We observe that, as $\alpha d_{2}d_{3} v(1+\alpha u)^{-2}\geq0$, the part $\alpha d_{2}d_{3} v(1+\alpha u)^{-2}\nabla u$ of the flux is direct toward the increasing population density of the prey u. On the other hand, $\Delta ({d_{2}d_{3}v}/{(1+\alpha u)} )$ yields a nonlinear diffusion of fractional type. This nonlinear diffusion describes a prey-predator relationship such that the diffusion of predator is prevented by the density of prey, and α represents the prevention. For further details we refer the reader to [1–4].

In this paper, we also study the positive solutions corresponding to the steady states of (1.1), i.e., the following quasilinear elliptic system:

$$ \textstyle\begin{cases} -d_{1}\Delta u=ru(1-u)-\frac{\beta uv}{u+mv}&\mbox{in } \Omega,\\ -d_{2}\Delta [ (1+\frac{d_{3}}{1+\alpha u} )v ]=v (b-\frac{v}{u} )&\mbox{in } \Omega,\\ \partial_{\nu}u=\partial_{\nu}v=0&\mbox{on } \partial \Omega. \end{cases} $$

(1.2)

Among other things, we are interested in positive solutions of (1.2). We call $(u,v)$ a positive solution when $u>0$ and $v>0$ satisfies (1.2). Hence a positive solution is corresponding to a coexistence steady state of prey and predator. It is easy to see that (1.2) has a semi-trivial non-negative solution $\mathbf {u}^{0}=(1,0)$ and a unique positive constant solution $\mathbf {u}^{*}=(u^{*},v^{*})$, where

$$ u^{*}=1-\frac{\beta b}{r(1+bm)},\qquad v^{*}=bu^{*}. $$

(1.3)

In the sequel, we always assume $\beta b< r(1+bm)$, which ensures the existence of $\mathbf{u}^{*}$.

Consider the following predator–prey system with diffusion:

$$ \textstyle\begin{cases} u_{t}-d_{1}\Delta u=ug(u)-p(u,v)&\mbox{in } \Omega\times \mathbb {R}_{+},\\ v_{t}-d_{2}\Delta v=v (b-\frac{v}{ u} )&\mbox{in } \Omega\times \mathbb {R}_{+},\\ \partial_{\nu}u=\partial_{\nu}v=0&\mbox{on } \partial \Omega\times \mathbb {R}_{+},\\ u(x,0)>0,\qquad v(x,0)\geq,\not\equiv0,&\mbox{in } \bar{\Omega}, \end{cases} $$

(1.4)

where $g(u)$ and $p(u,v)$ are $C^{1}$-functions. A typical case of g is the logistic type, namely, $g(u)=r(1-u/k)$ with $r,k>0$. $p(u,v)$ is called the functional response and see [5–8] for classifications of $p(u,v)$. In recent years, there has been considerable interest in investing the system (1.4) with prey-dependent functional response (i.e., $p(u,v)=f(u)v$). In [9, 10], Du, Hsu and Wang investigated the global stability of the unique positive constant steady state and gained some important conclusions about pattern formation for (1.4) with Leslie–Gower functional response (i.e., $p(u,v)=\beta uv$ for $\beta>0$). In [11, 12], Peng and Wang studied the long time behavior of time-dependent solutions and the global stability of the positive constant steady state for (1.4) with Holling–Tanner-type functional response (i.e., $p(u,v)=\beta uv/(m+u)$ with $\beta,m>0$). They also established some results for the existence and non-existence of non-constant positive steady states with respect to diffusion and cross-diffusion rates. In [13], Ko and Ryu investigated system (1.4) with $p(u,v)=f(u)v$ and f satisfies a general hypothesis: $f(0)=0$, and there exists a positive constant M such that $0< f'(u)\leq M$ for all $u>0$. They studied the global stability of the positive constant steady state and derived various conditions for the existence and non-existence of non-constant positive steady states. For the case the function $p(u,v)$ in the system (1.4) takes the form $p(u,v)=\beta uv/(u+mv)$ with $\beta, m>0$, called a ratio-dependent functional response, Peng and Wang [14] studied the global stability of the unique positive constant steady state and gained several results for the non-existence of non-constant positive solutions. Similar results had been obtained by Shi, Li and Lin [15] in the case $p(u,v)$ in the system (1.4) takes the form $p(u,v)=\beta uv/(a+u+mv)$ with $\beta,a,m>0$, called Beddington–DeAngelis functional response.

When we take cross-diffusion into account, a general partial differential prey-predator model takes the following form (see [2, 3]):

$$ \textstyle\begin{cases} u_{t}-\operatorname{div} [K_{11}(u,v)\nabla u + K_{12}(u,v)\nabla v ]=u[r - ku - p(u,v)]&\mbox{in } \Omega\times \mathbb {R}_{+},\\ v_{t}-\operatorname{div} [K_{21}(u,v)\nabla u + K_{22}(u,v)\nabla v ]=v[-b + q(u,v)]&\mbox{in } \Omega\times \mathbb {R}_{+}, \end{cases} $$

(1.5)

where $r,k,b>0$, $p(u,v)$ and $q(u,v)$ are the functional responses, $K_{11},K_{22}$ and $K_{12},K_{21}$, respectively, embody the self-diffusion and cross-diffusion processes. Li, Pang and Wang [16] study the global existence of classical solution of (1.5) with homogeneous Neumann boundary condition and smooth initial datum for the case

$$\begin{aligned} &K_{11}(u,v)=c_{1}+2c_{2}u,\qquad K_{12}(u,v)=0,\qquad K_{21}=\frac{\ell vu^{\ell -1}}{(1+u^{\ell})^{2}},\\ &K_{22}(u,v)=c_{3}+2c_{4}+\frac{1}{1+u^{\ell}}, \end{aligned}$$

where $c_{1},c_{3}>0$, $c_{2},c_{4}\geq0$ and $\ell\geq1$ are constants.

If we take cross-diffusion and Beddington–DeAngelis functional response into account, Zhang and Fu [17] studied the following system:

$$ \textstyle\begin{cases} -d_{1}\Delta u=ru(1-u)-\frac{\beta uv}{1+nu+mv}&\mbox{in } \Omega\times \mathbb {R}_{+},\\ -d_{2}\Delta [(1+d_{3}u)v ]=v (b-\frac{v}{u} )&\mbox{in } \Omega\times \mathbb {R}_{+},\\ \partial_{\nu}u=\partial_{\nu}v=0&\mbox{on } \partial \Omega\times \mathbb {R}_{+}, \end{cases} $$

(1.6)

where $d_{1},d_{2}>0$, $d_{3}\geq0$ and all the other parameters are positive constants. They study the stability of the positive constant solution and the existence and non-existence results about the non-constant steady states of the above system.

When we take cross-diffusion and ratio-dependent functional response into account, Wang, Li and Shi [18] studied the following system:

$$ \textstyle\begin{cases} -d_{1}\Delta u=ru(1-u)-\frac{\beta uv}{u+mv}&\mbox{in }\Omega,\\ -d_{2}\Delta [ (1+d_{3} u )v ]=v (b-\frac {v}{u} )&\mbox{in } \Omega,\\ \partial_{\nu}u=\partial_{\nu}v=0&\mbox{on } \partial \Omega, \end{cases} $$

(1.7)

where $d_{1},d_{2}>0$, $d_{3}\geq0$ and all the other parameters are positive constants. They established the existence and non-existence results about the non-constant positive steady states.

If we take nonlinear diffusion of fractional type into account. Kuto and Yamada [1] considered the following prey-predator model:

$$ \textstyle\begin{cases} -\Delta[(1+c_{1}v)u]=u(a-u-cv)&\mbox{in } \Omega,\\ -\Delta [ (1+\frac{c_{2}}{1+\beta u} )v ]=v(b+du-v)&\mbox{in } \Omega,\\ u=v=0&\mbox{on } \partial\Omega, \end{cases} $$

(1.8)

where $a,c,d$ are positive constants, $c_{1},c_{2},\beta$ are non-negative constants and b is a real constant which is allowed to be non-positive. In a case when the spatial dimension is less than 5, they found a universal bound for coexistence steady states. By using the bound and the bifurcation theorem they obtained the bounded continuum of coexistence steady states.

Finally, we remark that in the past decades, there has been much work on the existence and non-existence of non-constant positive steady states of ecological models with diffusion or cross-diffusion under the homogeneous Neumann boundary conditions. One can refer to [4, 11, 13, 19–33]. The role of diffusion in modeling many physical, chemical and biological processes has been extensively studied. Starting with Turing’s seminal paper [34], diffusion and cross-diffusion have been observed as causes of the spontaneous emergence of ordered structures, called patterns in a variety of non-equilibrium situations. They include the Gierer–Meinhardt model [35–38], the Sel’kov model [26, 39], the Lotka–Volterra competition model [40–42] and the Lotka–Volterra predator–prey model [20, 23, 24, 43–45] and so on.

Based on above reasons, in this paper, we consider problems (1.1) and (1.2). The organization of this paper is as follows. In Sect. 2, we study the stability of constant steady state of (1.1) with $d_{3}=0$. In Sect. 3, we establish a priori upper and lower bounds for the positive solutions of (1.2). Section 4 deals with the non-existence of the non-constant positive solutions of (1.2). Finally, in Sect. 5, we establish the existence of non-constant positive solutions of (1.2) for a range of diffusion and cross-diffusion coefficients.

2 Large time behavior

In this section, we always set $d_{3}=0$ and consider the large time behavior of solution to the special case of (1.1), i.e., the following reaction–diffusion equation:

$$ \textstyle\begin{cases} u_{t}-d_{1}\Delta u=ru(1-u)-\frac{\beta uv}{u+mv}:=f_{1}(u,v)&\mbox{in } \Omega\times \mathbb {R}_{+},\\ v_{t}-d_{2}\Delta v=v (b-\frac{v}{u} ):=f_{2}(u,v)&\mbox{in } \Omega\times \mathbb {R}_{+},\\ \partial_{\nu}u=\partial_{\nu}v=0&\mbox{on } \partial\Omega\times \mathbb {R}_{+},\\ u(x,0)>0,\qquad v(x,0)\geq,\not\equiv0,&\mbox{in } \bar{\Omega}. \end{cases} $$

(2.1)

The main results of this section is the following four theorems.

Theorem 2.1

(Dissipation)

Let $(u,v)$ be the positive solution of (2.1), then we have

$$ \limsup_{t\rightarrow\infty}\max_{\bar{\Omega}}u( \cdot,t)\leq1,\qquad \limsup_{t\rightarrow\infty}\max_{\bar{\Omega}}v( \cdot,t)\leq b. $$

(2.2)

Theorem 2.2

(Persistence)

Let $(u,v)$ be the positive solution of (2.1). If $\beta< rm$, then (2.1) has persistence property, i.e., the following inequalities hold:

$$ \liminf_{t\rightarrow\infty}\min_{\bar{\Omega}}u( \cdot,t)\geq \underline{u}_{1},\qquad \liminf_{t\rightarrow\infty} \min_{\bar{\Omega}}v(\cdot,t)\geq b\underline{u}_{1}, $$

(2.3)

where

$$\underline{u}_{1}=\frac{r(1-mb)+\sqrt{r^{2}(1-mb)^{2}+4rb(rm-\beta)}}{2r}. $$

Theorem 2.3

(Global stability)

If $\beta< rm$, $rbm^{2}< rm^{2}+\beta$ and $r+m(rm-2\beta)b^{2}\geq0$, then $\mathbf{u}^{*}$ defined in (1.3) is globally asymptotically stable for (2.1). In particular, this implies that (1.2) has no non-constant positive solution with $d_{3}=0$.

Remark 2.4

It is easy to see that if $b\leq1$ and $r>\max\{\beta/m, \beta m\}$, then $\mathbf{u}^{*}$ is globally asymptotically stable.

Finally, we consider the extinction results of (2.1).

Theorem 2.5

Assume $d_{1}=d_{2}=d$. Let $\alpha_{1}>0$ such that $r+1/\alpha_{1}\leq b+\beta b/(\alpha_{1}+m)$, then there exists a positive constant $\alpha _{2}\ll\alpha_{1}$ such that $\mathcal{R}:= \{(u,v)\in \mathbb {R}^{2}|u,v\geq0, \alpha_{2}v\leq u\leq\alpha_{1}v \}$ is an invariant region of (2.1). Furthermore, if $r(\alpha _{1}+m)\leq\beta$ or $\alpha_{1}b<1$, then for any $(u_{0},v_{0})\in \mathcal{R}\setminus\{(0,0)\}$ the positive solution $(u,v)$ of (2.1) satisfies $\lim_{t\rightarrow\infty}(u,v)=(0,0)$ uniformly on Ω̄.

In order to prove the above results, we first introduce the following lemma [46, 47].

Lemma 2.6

Assume $f(s)\in C^{1}([0,+\infty))$, $d>0,\beta\geq0,T\in[0,+\infty )$ are constants, $w\in C^{2,1}(\Omega\times(T,+\infty))\cap C^{1,0}(\bar{\Omega}\times[T,+\infty))$ is positive. Then we have

(1) If w satisfies

$$ \textstyle\begin{cases} \frac{\partial w}{\partial t}-d\Delta w\leq(\geq)w^{1+\beta }f(w)(\alpha-w)&\textit{in } \Omega \times(T,+\infty),\\ \frac{\partial w}{\partial\nu}=0&\textit{on } \partial\Omega\times(T,+\infty), \end{cases} $$

(2.4)

where $\alpha>0$ is a constant, we have

$$\limsup_{t\rightarrow+\infty}\max_{\bar{\Omega}}w(\cdot,t)\leq \alpha \qquad \Bigl(\liminf_{t\rightarrow+\infty}\min_{\bar{\Omega}}w(\cdot ,t) \geq\alpha \Bigr). $$

(2) If w satisfies

$$ \textstyle\begin{cases} \frac{\partial w}{\partial t}-d\Delta w\leq w^{1+\beta}f(w)(\alpha -w)&\textit{in } \Omega\times(T,+\infty),\\ \frac{\partial w}{\partial\nu}=0&\textit{on } \partial\Omega \times(T,+\infty), \end{cases} $$

(2.5)

where $\alpha\leq0$ is a constant, we have

$$\limsup_{t\rightarrow+\infty}\max_{\bar{\Omega}}w(\cdot,t)\leq0. $$

Proof of Theorems 2.1–2.3

We divide the prove into three steps.

Step 1 (Dissipation). By (2.1)₁, we obtain $u_{t}-d_{1}\Delta u=ru(1-u)$. So, by virtue of Lemma 2.6, we get

$$ \limsup_{t\rightarrow\infty}\max_{\bar{\Omega}}u( \cdot,t)\leq 1:=\bar{u}_{1}. $$

(2.6)

So, for any $\epsilon>0$, there exists $T_{1}^{\epsilon}\gg1$ such that $u(x,t)\leq\bar{u}_{1}+\epsilon$ for any $x\in\bar{\Omega}$ and $t\geq T_{1}^{\epsilon}$. By (2.1)₂, we get

$$ v_{t}-d_{2}\Delta v\leq v \biggl(b- \frac{v}{\bar{u}_{1}+\epsilon} \biggr)=v\frac{b(\bar{u}_{1}+\epsilon)-v}{\bar{u}_{1}+\epsilon}\quad \mbox{for } (x,t)\in\Omega \times \bigl(T_{1}^{\epsilon},\infty \bigr). $$

(2.7)

By Lemma 2.6, we get $\limsup\max_{\bar{\Omega}}v(\cdot ,t)\leq b(\bar{u}_{1}+\epsilon)$. By the arbitrariness of $\epsilon>0$, we obtain

$$ \limsup\max_{\bar{\Omega}}v(\cdot,t)\leq b\bar{u}_{1}:=\bar{v}_{1}. $$

(2.8)

We get Theorem 2.1 by (2.6) and (2.8).

Step 2 (Persistence). In this step, we assume that $\beta< rm$. Equation (2.8) implies for any $\epsilon>0$, there exists $T_{2}^{\epsilon}\gg1$ such that $v(x,t)\leq\bar{v}_{1}+\epsilon$ for any $x\in\bar{\Omega}$ and $t\geq T_{2}^{\epsilon}$. By (2.1)₁ and for any $(x,t)\in\Omega\times(T_{2}^{\epsilon},\infty)$, we obtain

$$ \begin{aligned}[b] u_{t}-d_{1}\Delta u&\geq ru(1-u)-\frac{\beta u(\bar{v}_{1}+\epsilon )}{u+m(\bar{v}_{1}+\epsilon)} \\ &=-u\frac{ru^{2}+r[m(\bar{v}_{1}+\epsilon)-1]u-(rm-\beta)(\bar{v}_{1}+\epsilon)}{u+m(\bar{v}_{1}+\epsilon)} \\ &=u\frac{(u-u_{2}^{\epsilon})(u_{1}^{\epsilon}-u)}{u+m(\bar{v}_{1}+\epsilon)}, \end{aligned} $$

(2.9)

where

$$\begin{aligned} &u_{1}^{\epsilon}=\frac{r[1-m(\bar{v}_{1}+\epsilon)]+\sqrt{r^{2}[1-m(\bar{v}_{1}+\epsilon)]^{2}+4r(rm-\beta)(\bar{v}_{1}+\epsilon)}}{2r}>0, \\ &u_{2}^{\epsilon}=\frac{r[1-m(\bar{v}_{1}+\epsilon)]-\sqrt{r^{2}[1-m(\bar{v}_{1}+\epsilon)]^{2}+4r(rm-\beta)(\bar{v}_{1}+\epsilon)}}{2r}< 0. \end{aligned} $$

By Lemma 2.6, we obtain $\liminf_{t\rightarrow\infty}\min_{\bar{\Omega}}u(\cdot,t)\geq u_{1}^{\epsilon}$. By the arbitrariness of $\epsilon>0$, we get

$$ \liminf_{t\rightarrow\infty}\min_{\bar{\Omega}}u( \cdot,t)\geq u_{1}^{0}:=\underline{u}_{1}>0, $$

(2.10)

where

$$\underline{u}_{1}=\frac{r(1-m\bar{v}_{1})+\sqrt{r^{2}(1-m\bar{v}_{1})^{2}+4r(rm-\beta)\bar{v}_{1}}}{2r}< \bar{u}_{1}. $$

So, for any $\epsilon\in(0,\underline{u}_{1})$, there exists $T_{3}^{\epsilon}\gg1$ such that $u(x,t)\geq\underline {u}_{1}-\epsilon$ for any $x\in\bar{\Omega}$ and $t\geq T_{3}^{\epsilon}$. By (2.1)₂ we get

$$ v_{t}-d_{2}\Delta v\geq v \biggl(b- \frac{v}{\underline{u}_{1}-\epsilon } \biggr)=v\frac{b(\underline{u}_{1}-\epsilon)-v}{\underline {u}_{1}-\epsilon}\quad \mbox{for } (x,t)\in\Omega \times \bigl(T_{3}^{\epsilon},\infty \bigr). $$

(2.11)

By Lemma 2.6, we obtain $\liminf_{t\rightarrow\infty}\min_{\bar{\Omega}}v(\cdot,t)\geq b(\underline{u}_{1}-\epsilon)$. By the arbitrariness of $\epsilon\in(0,\underline{u}_{1})$, we get

$$ \liminf_{t\rightarrow\infty}\min_{\bar{\Omega}}v( \cdot,t)\geq b\underline{u}_{1}:=\underline{v}_{1}>0. $$

(2.12)

We get Theorem 2.2 by (2.10) and (2.12).

Step 3 (Global stability). In this step, we assume that $\beta < rm$, $rbm^{2}< rm^{2}+\beta$ and $r+m(rm-2\beta)b^{2}\geq0$ and we will use the monotone iterative method to prove Theorem 2.3. Equation (2.12) implies for any $\epsilon\in(0,\underline {v}_{1})$, there exists $T_{4}^{\epsilon}\gg1$ such that $v(x,t)\geq \underline{v}_{1}-\epsilon$ for any $x\in\bar{\Omega}$ and $t\geq T_{4}^{\epsilon}$. So by (2.1)₁ and for any $(x,t)\in\Omega \times(T_{4}^{\epsilon},\infty)$, we have

$$ \begin{aligned}[b] u_{t}-d_{1}\Delta u&\leq ru(1-u)-\frac{\beta u(\underline{v}_{1}-\epsilon )}{u+m(\underline{v}_{1}-\epsilon)} \\ &=-u\frac{ru^{2}+r[m(\underline{v}_{1}-\epsilon)-1]u-(rm-\beta )(\underline{v}_{1}-\epsilon)}{u+m(\underline{v}_{1}-\epsilon)} \\ &=u\frac{(u-u_{4}^{\epsilon})(u_{3}^{\epsilon}-u)}{u+m(\underline {v}_{1}-\epsilon)}, \end{aligned} $$

(2.13)

where

$$\begin{aligned} &u_{3}^{\epsilon}=\frac{r[1-m(\underline{v}_{1}-\epsilon)]+\sqrt {r^{2}[1-m(\underline{v}_{1}-\epsilon)]^{2}+4r(rm-\beta)(\underline {v}_{1}-\epsilon)}}{2r}>0, \\ &u_{4}^{\epsilon}=\frac{r[1-m(\underline{v}_{1}-\epsilon)]-\sqrt {r^{2}[1-m(\underline{v}_{1}-\epsilon)]^{2}+4r(rm-\beta)(\underline {v}_{1}-\epsilon)}}{2r}< 0. \end{aligned} $$

By Lemma 2.6, we get $\limsup_{t\rightarrow\infty}\max_{\bar{\Omega}}u(\cdot,t)\leq u_{3}^{\epsilon}$. By the arbitrariness of $\epsilon\in(0,\underline{v}_{1})$, we get

$$ \limsup_{t\rightarrow\infty}\max_{\bar{\Omega}}u( \cdot,t)\leq u_{3}^{0}:=\bar{u}_{2}, $$

(2.14)

where

$$\bar{u}_{2}=\frac{r(1-m\underline{v}_{1})+\sqrt{r^{2}(1-m\underline {v}_{1})^{2}+4r(rm-\beta)\underline{v}_{1}}}{2r}< \bar{u}_{1}. $$

Define

$$ \textstyle\begin{cases} \varphi(s)=bs,\quad s\in(0,\infty),\\ \psi(s)=\frac{r(1-ms)+\sqrt{r^{2}(1-ms)^{2}+4r(rm-\beta)s}}{2r},\quad s\in (0,\frac{rm^{2}+\beta}{rm^{2}} ). \end{cases} $$

(2.15)

After some simple computations, we get

$$ \textstyle\begin{cases} \varphi'(s)>0,\quad s\in(0,\infty),\\ \psi'(s)< 0,\quad s\in (0,\frac{rm^{2}+\beta}{rm^{2}} ). \end{cases} $$

(2.16)

By virtue of $\bar{u}_{1}>\underline{u}_{1}$, $\bar{u}_{2}<\bar{u}_{1}$, $0<\bar{v}_{1}\leq b<(rm^{2}+\beta)/(rm^{2})$ and (2.16), we get

$$ \textstyle\begin{cases} \bar{v}_{1}=\varphi(\bar{u}_{1})>\varphi(\underline{u}_{1})=\underline {v}_{1},\qquad \underline{u}_{1}=\psi(\bar{v}_{1})< \psi(\underline{v}_{1})=\bar{u}_{2}< \bar{u}_{1},\\ \underline{u}_{1}\leq\liminf_{t\rightarrow\infty}\min_{\bar{\Omega}}u(\cdot,t)\leq u(x,t)\leq\limsup_{t\rightarrow\infty}\max_{\bar{\Omega}}u(\cdot,t)\leq\bar{u}_{2},\\ \underline{v}_{1}\leq\liminf_{t\rightarrow\infty}\min_{\bar{\Omega}}v(\cdot,t)\leq v(x,t)\leq\limsup_{t\rightarrow\infty}\max_{\bar{\Omega}}v(\cdot,t)\leq\bar{v}_{1}. \end{cases} $$

(2.17)

By induction, we can define four sequences $\{\underline{u}_{i}\} _{i=1}^{\infty}$, $\{\underline{v}_{i}\}_{i=1}^{\infty}$, $\{\bar{u}_{i}\} _{i=1}^{\infty}$ and $\{\bar{v}_{i}\}_{i=1}^{\infty}$in the following way: $\bar{v}_{i}=\varphi(\bar{u}_{i})$, $\underline{u}_{i}=\psi(\bar{v}_{i})$, $\underline{v}_{i}=\varphi(\underline{u}_{i})$ and $\bar{u}_{i+1}=\psi (\underline{v}_{i})$ such that

$$ \textstyle\begin{cases} \underline{u}_{i}\leq\liminf_{t\rightarrow\infty}\min_{\bar{\Omega}}u(\cdot,t)\leq u(x,t)\leq\limsup_{t\rightarrow\infty}\max_{\bar{\Omega}}u(\cdot,t)\leq\bar{u}_{i},\\ \underline{v}_{i}\leq\liminf_{t\rightarrow\infty}\min_{\bar{\Omega}}v(\cdot,t)\leq v(x,t)\leq\limsup_{t\rightarrow\infty}\max_{\bar{\Omega}}v(\cdot,t)\leq\bar{v}_{i}. \end{cases} $$

(2.18)

Since (2.16) holds, we can get the following relationships by induction:

$$ \textstyle\begin{cases} \underline{v}_{1}\leq\underline{v}_{i}< \underline{v}_{i+1}=\varphi (\underline{u}_{i+1})< \varphi(\bar{u}_{i+1})=\bar{v}_{i+1}< \bar {v}_{i}\leq\bar{v}_{1},\\ \underline{u}_{1}\leq\underline{u}_{i}< \underline{u}_{i+1}=\psi (\bar{v}_{i+1})< \psi( \underline{v}_{i+1})=\bar{u}_{i+2}< \bar{u}_{i+1}\leq\bar{u}_{1}. \end{cases} $$

(2.19)

Assume $\lim_{i\rightarrow\infty}\underline{u}_{i}=\underline{u}$, $\lim_{i\rightarrow\infty}\underline{v}_{i}=\underline{v}$, $\lim_{i\rightarrow\infty}\bar{u}_{i}=\bar{u}$ and $\lim_{i\rightarrow\infty }\bar{u}_{i}=\bar{v}$. It is obvious that $0<\underline{u}\leq\bar{u}$, $0<\underline{v}\leq\bar{v}$ and $\underline{u},\underline{v},\bar{u}, \bar{v}$ satisfy

$$ \bar{v}=\varphi(\bar{u}),\qquad \underline{u}=\psi(\bar{v}),\qquad \underline {v}=\varphi(\underline{u}),\qquad \bar{u}=\psi(\underline{v}). $$

(2.20)

After some computations, we can see (2.20) is equivalent to

$$ \textstyle\begin{cases} \bar{v}=b\bar{u},\qquad \underline{v}=b\underline{u},\\ -r+r\underline{u}+\frac{\beta\bar{v}}{\underline{u}+m\bar{v}}=0,\\ -r+r\bar{u}+\frac{\beta\underline{v}}{\bar{u}+m\underline{v}}=0. \end{cases} $$

(2.21)

We claim $\bar{u}=\underline{u}$. In the following, we will prove the claim by contradiction. If $\bar{u}\neq\underline{u}$, (2.21)₂ and (2.21)₃ are equivalent to

$$ \textstyle\begin{cases} -r\underline{u}-rmb\bar{u}+r\underline{u}^{2}+rmb\underline{u}\bar{u}+\beta b\bar{u}=0,\\ -r\bar{u}-rmb\underline{u}+r\bar{u}^{2}+rmb\underline{u}\bar{u}+\beta m\underline{u}=0. \end{cases} $$

(2.22)

By (2.22)₁ − (2.22)₂, we get $r-r(\underline{u}+\bar{u})+b(\beta-rm)=0$. Then by virtue of $\beta< rm$, we get

$$ \underline{u}+\bar{u}< 1. $$

(2.23)

By (2.21)₂ + (2.21)₃, we get $r(\underline{u}+\bar{u}))+\frac{\beta\bar{v}}{\underline{u}+m\bar{v}}+\frac{\beta \underline{v}}{\bar{u}+m\underline{v}}=2r$. By virtue of (2.23), we get $\frac{\beta\bar{v}}{\underline{u}+m\bar{v}}+\frac {\beta\underline{v}}{\bar{u}+m\underline{v}}>r$, which is equivalent to

$$ r\underline{u}\bar{u}+(rm-\beta)\underline{u}\underline{v}+(rm- \beta )\bar{u}\bar{v}+m(rm-2\beta)\underline{v}\bar{v}< 0. $$

(2.24)

Since $\beta< rm$, (2.24) implies

$$ r\underline{u}\bar{u}+m(rm-2\beta)\underline{v}\bar{v}< 0. $$

(2.25)

By virtue of (2.21)₁, we get $r+m(rm-2\beta)b^{2}<0$, which contradicts $r+m(rm-2\beta)b^{2}\geq0$. So $\underline{u}=\bar{u}$, and then we get $\underline{v}=\bar{v}$ by (2.21)₁. Theorem 2.3 follows. The proof is completed. □

Proof of Theorem 2.5

Let $G_{1}(u,v)=u-\alpha_{1}v$, $G_{2}(u,v)=\alpha_{2}v-u$, then

$$\mathcal{R}= \bigl\{ (u,v)\in \mathbb {R}^{2}|u,v\geq0, G_{i}(u,v) \leq0, i=1,2 \bigr\} . $$

Since $r+1/\alpha_{1}\leq b+\beta b/(\alpha_{1}+m)$, we get

$$ \nabla G_{1}\cdot(f_{1},f_{2})=u \biggl(r-\frac{b\beta}{\alpha+m}-b-\frac {1}{\alpha} \biggr)-ru^{2}\leq0 $$

(2.26)

on the boundary $u=\alpha_{1}v$. On the other hand, if $u=\alpha_{2}v$, we have

$$ \nabla G_{2} \cdot(f_{1},f_{2})=u \biggl(-r+ru+\frac{\beta}{\alpha _{2}+m}+b- \frac{1}{\alpha_{2}} \biggr)\rightarrow-\infty\quad \mbox{as } \alpha_{2} \rightarrow0^{+}, $$

(2.27)

then we get $\nabla G_{2}\cdot(f_{1},f_{2})\leq0$ for $\alpha_{2}\ll\alpha _{1}$. So $\mathcal{R}$ is an invariant region of (2.1) by [47].

In the following, we assume $r(\alpha_{1}+m)\leq\beta$ or $\alpha _{1}b<1$, and $(u_{0},v_{0})\in\mathcal{R}\setminus\{(0,0)\}$, then $(u(x,t),v(x,t))\in\mathcal{R}$. By the first equation of (2.1), we get

$$ \textstyle\begin{cases} u_{t}-d\Delta u\leq u (r-\frac{\beta}{\alpha_{1}+m}-ru )&\mbox{in } \Omega\times \mathbb {R}_{+},\\ \partial_{\nu}u=0&\mbox{on } \partial\Omega\times \mathbb {R}_{+},\\ u(x,0)>0,&\mbox{in } \bar{\Omega}. \end{cases} $$

(2.28)

Firstly, we consider the case $r(\alpha_{1}+m)\leq\beta$. By Lemma 2.6, it is easy to see $\lim_{t\rightarrow\infty }u(x,t)=0$ uniformly on Ω̄. So for any $\epsilon>0$, there exists $T>0$ such that $u(x,t)\leq\epsilon$ for any $(x,t)\in\bar{\Omega}\times[T,\infty)$, and so v satisfies

$$ \textstyle\begin{cases} v_{t}-d\Delta v\leq v\frac{b\epsilon-v}{\epsilon}&\mbox{in } \Omega\times(T,\infty),\\ \partial_{\nu}v=0&\mbox{on } \partial\Omega\times (T,\infty),\\ v(x,T)>0,&\mbox{in } \bar{\Omega}. \end{cases} $$

(2.29)

By virtue of Lemma 2.6, we get

$$ \limsup_{t\rightarrow\infty}\max_{\bar{\Omega}}v( \cdot,t)\leq b\epsilon, $$

(2.30)

which means $\lim_{t\rightarrow\infty}v(x,t)=0$ uniformly on Ω̄.

Secondly, we consider the case $r(\alpha_{1}+m)>\beta$ and $\alpha _{1}b<1$. By (2.28) and Lemma 2.6, we obtain

$$ \limsup_{t\rightarrow\infty}\max_{\bar{\Omega}}u( \cdot,t)\leq 1-\frac{\beta}{r(\alpha_{1}+m)}:=\sigma>0. $$

(2.31)

Then, for any $\epsilon>0$, there exists $T_{0}>0$ such that, for any $(x,t)\in\bar{\Omega}\times[T_{0},\infty)$, $u(x,t)\leq\sigma +\epsilon$, so v satisfies

$$ \textstyle\begin{cases} v_{t}-d\Delta v\leq v\frac{b(\sigma+\epsilon)-v}{\sigma+\epsilon }&\mbox{in } \Omega\times(T_{0},\infty),\\ \partial_{\nu}v=0&\mbox{on } \partial\Omega\times (T_{0},\infty),\\ v(x,T_{0})>0,&\mbox{in } \bar{\Omega}. \end{cases} $$

(2.32)

By virtue of Lemma 2.6, we get

$$ \limsup_{t\rightarrow\infty}\max_{\bar{\Omega}}v( \cdot,t)\leq b(\sigma+\epsilon). $$

(2.33)

So there exists $T_{1}>T_{0}$ such that, for any $(x,t)\in\bar{\Omega}\times[T_{1},\infty)$, $v(x,t)\leq b(\sigma+\epsilon)+\epsilon$. Since $(u(x,t),v(x,t))\in\mathcal{R}$, we obtain

$$ u(x,t)\leq\alpha_{1} \bigl[b(\sigma+\epsilon)+\epsilon \bigr]:=\phi(\epsilon ),\quad \forall(x,t)\in\bar{\Omega}\times[T_{1}, \infty). $$

(2.34)

Let $\eta=(1+\alpha_{1}b)/2$, then $\eta<1$. Since $\phi(0)=\alpha _{1}b\sigma<\eta\sigma$, there exists $\epsilon\ll1$ such that $\phi (\epsilon)<\eta\sigma$. So, we get

$$ u(x,t)\leq\phi(\epsilon)< \eta\sigma,\quad \forall(x,t)\in\bar{\Omega}\times[T_{1},\infty). $$

(2.35)

Then v satisfies

$$ \textstyle\begin{cases} v_{t}-d\Delta v\leq v\frac{b\eta\sigma-v}{\eta\sigma}&\mbox{in } \Omega\times(T_{1},\infty),\\ \partial_{\nu}v=0&\mbox{on } \partial\Omega\times (T_{1},\infty),\\ v(x,T_{1})>0,&\mbox{in } \bar{\Omega}, \end{cases} $$

(2.36)

and so

$$ \limsup_{t\rightarrow\infty}\max_{\bar{\Omega}}v( \cdot,t)\leq b\eta\sigma. $$

(2.37)

Then there exists $T_{2}>T_{1}$ such that when $0<\epsilon\leq(\eta \sigma/\alpha_{1})(\eta-\alpha_{1}b)$ (notice $\eta-\alpha_{1}b>0$ if $\alpha_{1}b<1$)

$$ v(x,t)\leq b\eta\sigma+\epsilon,\qquad u(x,t)\leq \alpha_{1}(b\eta\sigma +\epsilon)\leq\eta^{2}\sigma,\quad \forall(x,t)\in\bar{\Omega}\times [T_{2},\infty). $$

(2.38)

By induction, we obtain there exists an increasing sequence $\{T_{n}\} _{n=0}^{\infty}$ satisfying $T_{n}\rightarrow\infty$ as $n\rightarrow \infty$ such that

$$ u(x,t)\leq\eta^{n}\sigma,\quad \forall(x,t)\in\bar{\Omega}\times [T_{n},\infty). $$

(2.39)

Since $0<\eta<1$, we have $\lim_{t\rightarrow\infty}u(x,t)=0$ uniformly on Ω̄, and similar to the case of $r(\alpha _{1}+m)\leq\beta$, we can prove $\lim_{t\rightarrow\infty}v(x,t)=0$ uniformly on Ω̄. The proof is completed. □

3 A priori estimates for positive solutions of (1.2)

In this section, we shall give a priori estimates for the positive solutions of (1.2). In the following, we shall write Λ instead of the collective of constants $r,\alpha,\beta ,b,m$ for convenience and our main result is the following theorem.

Theorem 3.1

Let d be a fixed positive number such that $d_{1},d_{2}\geq d$ and $d_{3}\geq0$. Then there exist two positive constants $C_{1}(\Lambda ,d,\Omega)$ and $C_{2}(\Lambda,d,\Omega)$ such that the positive solution $(u,v)$ of (1.2) satisfies

$$ C_{1}< u,\qquad v< C_{2}. $$

(3.1)

In order to prove the above theorem we first give two lemmas. The first one is the maximum principle, which was given in [42].

Lemma 3.2

(Maximum principle)

Let $g\in C(\bar{\Omega}\times\mathbb{R})$ and $b_{j}(x)\in C(\bar{\Omega})$, $j=1,2,\ldots,N$. Then we have

• (i) If $w\in C^{2}(\Omega)\cap C^{1}(\bar{\Omega})$ satisfies

  $$ \textstyle\begin{cases} \Delta w+\sum_{j=1}^{N}b_{j}(x)w_{x_{j}}+g(x,w(x))\geq0& \textit{in } \Omega,\\ \frac{\partial w}{\partial\nu}\leq0& \textit{on } \partial\Omega, \end{cases} $$

  (3.2)

  and $w(x_{0})=\max_{x\in\bar{\Omega}}w(x)$, then $g(x_{0},w(x_{0}))\geq0$.

• (ii) If $w\in C^{2}(\Omega)\cap C^{1}(\bar{\Omega})$ satisfies

  $$ \textstyle\begin{cases} \Delta w+\sum_{j=1}^{N}b_{j}(x)w_{x_{j}}+g(x,w(x))\leq0& \textit{in } \Omega,\\ \frac{\partial w}{\partial\nu}\geq0& \textit{on } \partial\Omega, \end{cases} $$

  (3.3)

  and $w(x_{0})=\min_{x\in\bar{\Omega}}w(x)$, then $g(x_{0},w(x_{0}))\leq0$.

The second one is the following Harnack inequality, which was given in [48].

Lemma 3.3

(Harnack inequality)

Let $w\in C^{2}(\Omega)\cap C^{1}(\bar{\Omega})$ be a positive solution to $\Delta w(x)+c(x)w(x)=0$ subject to homogeneous Neumann boundary condition, where $c(x)\in C(\bar{\Omega})$. Then there exists a positive constant $C_{*}$ depending only on $\|c\| _{\infty}$ such that $\max_{x\in\bar{\Omega}}w(x)\leq C_{*}\min_{x\in \bar{\Omega}}w(x)$.

Proof of Theorem 3.1

In the following, we shall denote by C a generic constant independent of $d_{3}$ that may changes between lines. Also notice that C will depend on the domain Ω. However, as Ω is fixed, we will not mention the dependence explicitly. Furthermore, we will denote $\max_{\bar{\Omega}}$ and $\min_{\bar{\Omega}}$ by max and min, respectively. Since

$$ -d_{1}\Delta u=ru(1-u)-\frac{\beta uv}{u+mv}, $$

(3.4)

we can easily get $\max u\leq1$ by the maximum principle. Then we have

$$ \biggl\Vert r(1-u)-\frac{\beta v}{u+mv} \biggr\Vert _{\infty}\leq2r+\frac {\beta}{m}, $$

(3.5)

so there exists positive constant C such that $\min u(x)\geq C\max u(x)$ from Lemma 3.3. Let $x_{0}\in\bar{\Omega}$ such that $v(x_{0})(1+d_{3}/(1+\alpha u(x_{0})))=\max v(x)(1+d_{3}/(1+\alpha u(x)))$, then we get $b-v(x_{0})/u(x_{0})\geq0$ by Lemma 3.2, which means $v(x_{0})\leq bu(x_{0})\leq b$ and

$$ \begin{aligned}[b] \frac{v(x)}{v(x_{0})}&=\frac{v(x)(1+d_{3}/(1+\alpha u(x)))}{v(x_{0})(1+d_{3}/(1+\alpha u(x_{0})))} \times\frac{1+d_{3}/(1+\alpha u(x_{0}))}{1+d_{3}/(1+\alpha u(x))} \\ &\leq\frac{1+d_{3}/(1+\alpha\min u(x))}{1+d_{3}/(1+\alpha\max u(x))}\leq\frac{d_{3}/(1+\alpha\min u(x))}{d_{3}/(1+\alpha\max u(x))} \\ &\leq\frac{1+\alpha\max u(x)}{1+\alpha\min u(x)}\leq\frac{\max u(x)}{\min u(x)}\leq C. \end{aligned} $$

(3.6)

Equation (3.6) means $v(x)\leq Cv(x_{0})\leq C$. So, the right-hand side of (3.1) holds.

In order to prove the left-hand side of (3.1), we must prove $\min v(x)\geq C\max v(x)$. Let $\phi(x)=d_{2}v(x)(1+d_{3}/(1+\alpha u(x)))$, then the second equation of (1.2) becomes

$$ \textstyle\begin{cases} \Delta\phi+\frac{b-v/u}{d_{2}(1+d_{3}/(1+\alpha u(x)))}\phi=0& \mbox{in } \Omega,\\ \partial_{\nu}\phi=0&\mbox{on } \partial\Omega. \end{cases} $$

(3.7)

Since

$$ \begin{aligned}[b] \biggl\Vert \frac{b-v/u}{d_{2}(1+d_{3}/(1+\alpha u(x)))} \biggr\Vert _{\infty}&\leq \frac{b+\max v(x)/\min u(x)}{d_{2}\min(1+d_{3}/(1+\alpha u(x)))} \\ &\leq C+\frac{\max\phi(x)}{d_{2}^{2}\min u(x)\min(1+d_{3}/(1+\alpha u(x)))} \\ &=C+\frac{v(x_{0})(1+d_{3}/(1+\alpha u(x_{0})))}{d_{2}\min u(x)\min (1+d_{3}/(1+\alpha u(x)))} \\ &\leq C+\frac{bu(x_{0})(1+d_{3}/(1+\alpha u(x_{0})))}{d_{2}\min u(x)\min (1+d_{3}/(1+\alpha u(x)))} \\ &\leq C+\frac{b}{d}\frac{\max u(x)}{\min u(x)}\frac{\max (1+d_{3}/(1+\alpha u(x)))}{\min(1+d_{3}/(1+\alpha u(x)))} \\ &\leq C+\frac{b}{d}\frac{\max u(x)}{\min u(x)}\frac{\max u(x)}{\min u(x)} \\ &\leq C, \end{aligned} $$

(3.8)

we have $\min\phi(x)\geq C\max\phi(x)$ from Lemma 3.3. Hence, we obtain

$$ \begin{aligned}[b] \frac{\max v(x)}{\min v(x)}&\leq \frac{\max\phi(x)}{\min\phi (x)}\times\frac{\max(1+d_{3}/(1+\alpha u(x)))}{\min(1+d_{3}/(1+\alpha u(x)))} \\ &\leq C\frac{\max u(x)}{\min u(x)}\leq C. \end{aligned} $$

(3.9)

By way of contradiction, we suppose that $(u,v)$ does not have a positive lower bound, then there is a sequence $\{ (d_{1,i},d_{2,i},d_{3,i})\}_{i=0}^{\infty}$, $d_{1,i},d_{2,i}\geq d$, $d_{3,i}\geq0$ such that the positive solution of (1.2) corresponding $(d_{1},d_{2},d_{3})=(d_{1,i},d_{2,i},d_{3,i})$ satisfies

$$ \min u_{i}(x)\rightarrow0\quad \mbox{or}\quad \min v_{i}(x)\rightarrow0\quad \mbox{as } i\rightarrow\infty, $$

(3.10)

where $(u_{i},v_{i})$ solves the following equation:

$$ \textstyle\begin{cases} -d_{1,i}\Delta u_{i}=ru_{i}(1-u_{i})-\frac{\beta u_{i}v_{i}}{u_{i}+mv_{i}}& \mbox{in } \Omega,\\ -d_{2,i}\Delta [ (1+\frac{d_{3,i}}{1+\alpha u_{i}} )v_{i} ]=v_{i} (b-\frac{v_{i}}{u_{i}} )&\mbox{in } \Omega,\\ \partial_{\nu}u_{i}=\partial_{\nu}v_{i}=0&\mbox{on } \partial \Omega. \end{cases} $$

(3.11)

Integrating over Ω by parts for (3.11) yields

$$ \textstyle\begin{cases} \int_{\Omega}u_{i} [r(1-u_{i})-\frac{\beta v_{i}}{u_{i}+mv_{i}} ]\,dx=0,\\ \int_{\Omega}v_{i} (b-\frac{v_{i}}{u_{i}} )\,dx=0. \end{cases} $$

(3.12)

If $\min u_{i}(x)\rightarrow0$ as $i\rightarrow\infty$, one can obtain $u_{i}(x)\rightarrow0$ uniformly from $\min u_{i}(x)\geq C\max u_{i}(x)$. By the second equation of (3.12), we know that there exists a $x_{i}\in\bar{\Omega}$ such that $v_{i}(x_{i})=bu_{i}(x_{i})$ for each i. Hence, we can conclude that $v_{i}(x)\rightarrow0$ uniformly as $i\rightarrow\infty$ from $\min v_{i}(x)\geq C\max v_{i}(x)$. From the first equation of (3.12) we know that there exists $\hat{x}_{i}\in \bar{\Omega}$ for each i such that $r=ru_{i}(\hat{x}_{i})+\beta v_{i}(\hat{x}_{i})/(u_{i}(\hat{x}_{i})+mv_{i}(\hat{x}_{i}))$. One can deduce the conflict as $i\rightarrow\infty$. Hence, there exists a positive constant C such that $\min u(x)>C$.

If $\min v_{i}(x)\rightarrow0$ as $i\rightarrow\infty$, one can deduce conflict similarly. The proof is completed. □

4 Non-existence of non-constant positive solutions of problem (1.2)

In Theorem 2.3, the global stability of the positive constant solution implies the non-existence of non-constant positive solution of (1.2) regardless of diffusions. Several non-existence results of non-constant positive solutions to (1.2) will be presented in this section, and in these results, the diffusion and cross-diffusion coefficients do play important roles. Throughout this section, we let $0=\mu_{0}<\mu_{1}<\mu_{2}<\cdots$ represent the equivalent of the operator −Δ in Ω with homogeneous Neumann boundary condition. As in Sect. 3, we shall write Λ instead of the collective of constants $r,\alpha,\beta,b,m$ for convenience. The main results of this section are the following two theorems.

Theorem 4.1

Let d be a fixed positive number such that $d_{1},d_{2}\geq d$ and $d_{3}\geq0$. Then, for $\epsilon>0$ small enough, there exists a positive constant $\tilde{C}(\Lambda,d,\epsilon,\Omega)$ such that (1.2) has no non-constant positive solutions provided that

$$ d_{1}>\tilde{C} \bigl(1+d_{2}^{2}d_{3}^{2} \bigr). $$

(4.1)

Theorem 4.2

Let d be a fixed positive number such that $d_{1},d_{2}\geq d$ and $d_{3}\geq0$. Then, for any $\epsilon>0$, there exists a positive constant $\overline{C}(\Lambda,d,\epsilon,\Omega)$ such that (1.2) has no non-constant positive solutions provided that

$$ d_{2}>\frac{2(b+\epsilon)}{\mu_{1}},\qquad d_{1}> \overline{C} \bigl(1+d_{2}d_{3}^{2} \bigr). $$

(4.2)

Remark 4.3

From the above theorems, it is easy to see that if $d_{3}=0$ and $d_{1},d_{2}$ are large enough, then (1.2) does not have non-constant positive solutions.

Proof of Theorems 4.1 and 4.2

Let $(u,v)$ be a positive solution of (1.2) and denote $\bar{u}=|\Omega |^{-1}\int_{\Omega}u(x)\,dx$ and $\bar{v}=|\Omega|^{-1}\int_{\Omega}v(x)\,dx$. From Theorem 3.1, we obtain $C_{1}< u,v< C_{2}$ for some positive constants $C_{1},C_{2}$ depending only on $\Lambda,d$ and Ω.

Let us first prove Theorem 4.1. Multiplying $(u-\bar{u})/u$ and $(v-\bar{v})/v$ to the first and second equations in (1.2), respectively, and then integrating by parts over Ω, one can obtain

$$ \begin{aligned}[b] & \int_{\Omega}\biggl\{ \frac{d_{1}\bar{u}}{u^{2}} \vert \nabla u \vert ^{2}+\frac{d_{2}\bar{v}}{v^{2}} \biggl(1+\frac{d_{3}}{1+\alpha u} \biggr) \vert \nabla v \vert ^{2}-\frac {\alpha d_{2}d_{3}\bar{v}}{(1+\alpha u)^{2}v}\nabla u\cdot\nabla v \biggr\} \,dx \\ &\quad = \int_{\Omega}\biggl\{ \biggl[r(1-u)-\frac{\beta v}{u+mv} \biggr](u- \bar{u})+ \biggl(b-\frac{v}{u} \biggr) (v-\bar{v}) \biggr\} \,dx \\ &\quad = \int_{\Omega}\biggl\{ \biggl[-r+\frac{\beta\bar{v}}{(u+mv)(\bar{u}+m\bar{v})} \biggr](u-\bar{u})^{2} \\ &\qquad {}+ \biggl[-\frac{\beta\bar{u}}{(u+mv)(\bar{u}+m\bar{v})}+\frac{\bar{v}}{u\bar{u}} \biggr](u-\bar{u}) (v-\bar{v})-\frac {1}{u}(v-bar v)^{2} \biggr\} \,dx. \end{aligned} $$

(4.3)

By Young’s inequality and Poincaré’s inequality, we obtain

$$ \begin{aligned}[b] & \int_{\Omega}\biggl(\frac{d_{1}C_{1}}{C_{2}^{2}} \vert \nabla u \vert ^{2}+\frac {d_{2}C_{1}}{C_{2}^{2}} \vert \nabla v \vert ^{2} \biggr) \,dx \\ &\quad \leq \int_{\Omega}\biggl[\tilde{C}_{1}(\epsilon) (u-\bar{u})^{2}+ \biggl(\epsilon-\frac{1}{C_{2}} \biggr) (v-\bar{v})^{2}+\frac{\alpha ^{2}d_{2}^{2}d_{3}^{2}C_{2}^{2}}{4\epsilon C_{1}^{2}} \vert \nabla u \vert ^{2}+ \epsilon \vert \nabla v \vert ^{2} \biggr]\,dx \\ &\quad \leq \int_{\Omega}\biggl\{ \biggl[\frac{\tilde{C}_{1}(\epsilon)}{\mu _{1}}+ \frac{\alpha^{2}d_{2}^{2}d_{3}^{2}C_{2}^{2}}{4\epsilon C_{1}^{2}} \biggr] \vert \nabla u \vert ^{2}+ \biggl[ \frac{\epsilon}{\mu_{1}}-\frac{1}{C_{2}\mu_{1}}+\epsilon \biggr] \vert \nabla v \vert ^{2} \biggr\} \,dx, \end{aligned} $$

(4.4)

where $\tilde{C}_{1}(\epsilon)$ is a positive constant depends on $\Lambda,\Omega,\epsilon$. Taking ϵ small enough such that ${\epsilon}/{\mu_{1}}-{1}/{(C_{2}\mu_{1})}+\epsilon\leq0$ and denoting

$$\tilde{C}(\epsilon)=\frac{C_{2}^{2}}{C_{1}}\max \biggl\{ \frac{\tilde {C}_{1}(\epsilon)}{\mu_{1}}, \frac{\alpha^{2}C_{2}^{2}}{4\epsilon C_{1}^{2}} \biggr\} , $$

we obtain

$$ \int_{\Omega}\bigl(d_{1} \vert \nabla u \vert ^{2}+d_{2} \vert \nabla v \vert ^{2} \bigr)\,dx \leq \tilde{C}(\epsilon) \bigl(1+d_{2}^{2}d_{3}^{2} \bigr) \int_{\Omega} \vert \nabla u \vert ^{2}\,dx. $$

(4.5)

Combining (4.1) and (4.5), we can get Theorem 4.1.

Next, we prove Theorem 4.2. Multiplying $(u-\bar{u})$ and $(v-\bar{v})$ to the first and second equations in (1.2), respectively, and then integrating by parts over Ω, one can obtain

$$\begin{aligned} & \int_{\Omega}\biggl[d_{1} \vert \nabla u \vert ^{2}+d_{2} \biggl(1+\frac{d_{3}}{1+\alpha u} \biggr) \vert \nabla v \vert ^{2}-\frac{\alpha d_{2} d_{3}v}{(1+\alpha u)^{2}}\nabla u\cdot\nabla v \biggr] \,dx \\ &\quad = \int_{\Omega}\biggl[r-r(u+\bar{u})-\frac{\beta m v\bar{v}}{(u+mv)(\bar{u}+m\bar{v})} \biggr](u- \bar{u})^{2}\,dx \\ &\qquad + \int_{\Omega}\biggl[\frac{\bar{v}^{2}}{u\bar{u}} - \frac{\beta u\bar{u}}{(u + mv)(\bar{u} + m\bar{v})} \biggr](u - \bar{u}) (v - \bar{v})\,dx + \int_{\Omega}\biggl(b - \frac{v + \bar{v}}{u} \biggr) (v - \bar{v})^{2}\,dx \\ &\quad \leq \int_{\Omega}\biggl\{ r(u-\bar{u})^{2}+ \biggl[ \frac{\bar{v}^{2}}{u\bar{u}} - \frac{\beta u\bar{u}}{(u + mv)(\bar{u} + m\bar{v})} \biggr](u - \bar{u}) (v - \bar{v})+b(v- \bar{v})^{2} \biggr\} \,dx. \end{aligned}$$

(4.6)

By Young’s inequality and Poincaré’s inequality, we obtain

$$ \begin{aligned}[b] & \int_{\Omega}\bigl(d_{1} \vert \nabla u \vert ^{2}+d_{2} \vert \nabla v \vert ^{2} \bigr)\,dx \\ &\quad \leq \biggl[\overline{C}_{1}(\epsilon) (u-\bar{u})^{2}+(b+ \epsilon ) (v-\bar{v})^{2}+\frac{\alpha d_{2}d_{3}^{2}C_{2}^{2}}{2} \vert \nabla u \vert ^{2}+\frac {d_{2}}{2} \vert \nabla v \vert ^{2} \biggr] \\ &\quad \leq \int_{\Omega}\biggl\{ \biggl[\frac{\overline{C}_{1}(\epsilon )}{\mu_{1}}+ \frac{\alpha d_{2}d_{3}^{2}C_{2}^{2}}{2} \biggr] \vert \nabla u \vert ^{2}+ \biggl[ \frac{b+\epsilon}{\mu_{1}}+\frac{d_{2}}{2} \biggr] \vert \nabla v \vert ^{2} \biggr\} \,dx, \end{aligned} $$

(4.7)

where $\tilde{C}_{1}(\epsilon)$ is a positive constant depends on $\Lambda,\Omega,\epsilon$. Denoting

$$\tilde{C}(\epsilon)=\max \biggl\{ \frac{\overline{C}_{1}(\epsilon )}{\mu_{1}}, \frac{\alpha C_{2}^{2}}{2} \biggr\} , $$

we obtain

$$ \int_{\Omega}\biggl[d_{1} \vert \nabla u \vert ^{2}+ \biggl(\frac{d_{2}}{2}-\frac {b+\epsilon}{\mu_{1}} \biggr) \vert \nabla v \vert ^{2} \biggr]\,dx\leq\tilde {C}(\epsilon) \bigl(1+d_{2}d_{3}^{2} \bigr) \int_{\Omega} \vert \nabla u \vert ^{2} \,dx. $$

(4.8)

Combining (4.2) and (4.8), we can get Theorem 4.2. The proof is completed. □

5 Existence of non-constant positive solutions of problem (1.2)

This section is devoted to the existence of non-constant positive solutions of (1.2) for certain values of diffusion coefficients $d_{2}$ and $d_{3}$, respectively, while the other parameters are fixed. Our results show that, if the parameters are properly chosen, both the general stationary pattern and a more interesting Turing pattern can arise as a result of diffusion. Throughout this section, we denote

$$ \textstyle\begin{cases} f_{1}(u,v)=ru(1-u)-\frac{\beta uv}{u+mv},\\ f_{2}(u,v)=v (b-\frac{v}{u} ),\\ g_{1}(u,v)=d_{1}u,\\ g_{2}(u,v)=d_{2}v (1+\frac{d_{3}}{1+\alpha u} ), \end{cases} $$

(5.1)

and

$$ \textstyle\begin{cases} A_{11}=\frac{\partial f_{1}}{\partial u} |_{(u,v)=(u^{*},v^{*})}=r-2ru^{*}-\frac{\beta m {v^{*}}^{2}}{(u^{*}+mv^{*})^{2}},\\ A_{12}=\frac{\partial f_{1}}{\partial v} |_{(u,v)=(u^{*},v^{*})}=-\frac{\beta{u^{*}}^{2}}{(u^{*}+mv^{*})^{2}},\\ \delta_{2}=\frac{A_{11}}{d_{1}}+\frac{\alpha d_{3}v^{*}A_{12}}{d_{1}(1+\alpha u^{*}+d_{3})(1+\alpha u^{*})},\\ \delta_{3}=\frac{A_{11}}{d_{1}}+\frac{\alpha v^{*}A_{12}}{d_{1}(1+\alpha u^{*})}, \end{cases} $$

(5.2)

where $(u^{*},v^{*})$ is defined in (1.3).

The main result of this section is the following theorem.

Theorem 5.1

Let d be a fixed positive number such that $d_{1},d_{2}\geq d$ and $d_{3}\geq0$, then we have:

• 1. Suppose that $d_{1}, d_{3}$ are given such that $\delta_{2}\in(\mu _{i},\mu_{i+1})$ for some positive odd integer i. Then there exists a positive constant $d_{2}^{*}$ such that (1.2) has at least one non-constant positive solution if $d_{2}\geq d_{2}^{*}$.

• 2. Suppose that $d_{1}$ is given such that $\delta_{3}\in(\mu_{i},\mu _{i+1})$ for some positive odd integer i. Then, for any $d_{2}\geq d$, there exists a positive constant $d_{3}^{*}$ such that (1.2) has at least one non-constant positive solution if $d_{3}\geq d_{3}^{*}$.

In order to prove Theorem 5.1, we start with some preliminary results. Let $\{(\mu_{i},\varphi_{i})\}_{i=0}^{\infty}$ be a complete set of eigenpairs for the operator −Δ in Ω with homogeneous Neumann boundary condition, ordered such that $0=\mu_{0}<\mu_{1}<\mu _{2}<\cdots$ , and let $m(\mu_{i})$ be the multiplicity of $\mu_{i}$. Denote

$$ X= \bigl\{ (u,v)=C^{2}(\bar{\Omega})\times C^{2}( \bar{\Omega})|\partial_{\nu}u=\partial_{\nu}v=0 \mbox{ on } \partial\Omega \bigr\} . $$

(5.3)

We decompose X as

$$ X=\bigoplus_{i=0}^{\infty}X_{i},\quad \mbox{where } X_{i}= \bigl\{ \mathbf{c}\varphi _{i}(x)|\mathbf{c}\in \mathbb {R}^{2} \bigr\} . $$

(5.4)

In the following, we shall write $D=(d_{1},d_{2},d_{3})$, $\Lambda=(r,\alpha ,\beta,b,m)$. From Theorem 3.1, we known that there exists a positive constant C depending on $\Lambda,d,\Omega$ such that any positive solution $(u,v)$ of (1.2) satisfies $(u,v)\in \mathbf{B}(C)$, where

$$ \mathbf{B}(C)= \biggl\{ (u,v)\in X\Bigm|\frac{1}{C}< u,v< C \biggr\} . $$

(5.5)

Let $\mathbf{u}=(u,v)$, $\Phi(\mathbf{u})=(g_{1},g_{2})^{T}$ and $G(\mathbf{u})=(f_{1},f_{2})^{T}$, we can write (1.2) as

$$ \textstyle\begin{cases} -\Delta\Phi(\mathbf{u})=G(\mathbf{u})&\mbox{in } \Omega,\\ \partial_{\nu}\mathbf{u}=0&\mbox{on } \partial\Omega. \end{cases} $$

(5.6)

Then u is a positive solution of (5.6) if and only if $\mathbf{u}\in\mathbf{B}(C)$ and

$$ F(D;\mathbf{u}):=\mathbf{u}-(\mathrm{I}-\Delta)^{-1} \bigl\{ \Phi _{\mathbf{u}}^{-1}(\mathbf{u}) \bigl[G(\mathbf{u})+ \nabla\mathbf {u}\Phi_{\mathbf{u}\mathbf{u}}(\mathbf{u})\nabla\mathbf{u} \bigr]+\mathbf{u} \bigr\} =0\quad \mbox{in } X, $$

(5.7)

where $(\mathrm{I}-\Delta)^{-1}$ is the inverse of $\mathrm{I}-\Delta$ in X. It is easy to see that $\det\Phi_{\mathbf{u}}>0$ for all $\mathbf {u}\in\mathbf{B}(C)$, then $\Phi_{\mathbf{u}}^{-1}$ exists. As $F(D;\cdot)$ is a compact perturbation of an identity operator and $\mathbf{u}\in\mathbf{B}(C)$, the Leray–Schauder degree $\deg (F(D;\cdot),0,\mathbf{B}(C))$ is well defined.

We also note that

$$ D_{\mathbf{u}}F \bigl(D;\mathbf{u}^{*} \bigr)=\mathrm{I}-( \mathrm{I}-\Delta)^{-1} \bigl[\Phi_{\mathbf{u}}^{-1} \bigl(\mathbf{u}^{*} \bigr)G_{\mathbf{u}} \bigl(\mathbf{u}^{*} \bigr)+\mathrm{I} \bigr], $$

(5.8)

and recall that if $D_{\mathbf{u}}F(D;\mathbf{u}^{*})$ is invertible, the index of F at $\mathbf{u}^{*}$ is defined as index$(F(D;\cdot ), \mathbf{u}^{*})=(-1)^{\gamma}$, where γ is the multiplicity of negative eigenvalues of $D_{\mathbf{u}}F(D;\mathbf{u}^{*})$ (see [49], Theorem 2.8.1). It is easy to prove $X_{i}$ is invariant under $D_{\mathbf{u}}F(D;\mathbf{u}^{*})$ for each integer $i\geq0$ and λ is an eigenvalue of $D_{\mathbf{u}}F(D;\mathbf{u}^{*})$ in $X_{i}$ if and only if λ is an eigenvalue of the matrix

$$ \mathrm{I}-\frac{1}{1+\mu_{i}} \bigl[\Phi_{\mathbf{u}}^{-1} \bigl(\mathbf {u}^{*} \bigr)G_{\mathbf{u}} \bigl(\mathbf{u}^{*} \bigr)+ \mathrm{I} \bigr]=\frac{1}{1+\mu _{i}} \bigl[\mu_{i}\mathrm{I}- \Phi_{\mathbf{u}}^{-1} \bigl(\mathbf{u}^{*} \bigr)G_{\mathbf{u}} \bigl(\mathbf{u}^{*} \bigr) \bigr]. $$

(5.9)

So, for convenience, we denote

$$ H \bigl(D,\mathbf{u}^{*};\mu \bigr)=\det \bigl[\mu_{i} \mathrm{I}-\Phi_{\mathbf{u}}^{-1} \bigl(\mathbf{u}^{*} \bigr)G_{\mathbf{u}} \bigl(\mathbf{u}^{*} \bigr) \bigr]. $$

(5.10)

By an argument similar those in [23], it can be shown that the following lemma holds.

Lemma 5.2

Suppose that, for all $i\geq0$, $H(D,\mathbf{u}^{*};\mu_{i})\neq0$. Then

$$ \operatorname{index} \bigl(F(D;\cdot),\mathbf{u}^{*} \bigr)=(-1)^{\gamma},\quad \textit{where } \gamma =\sum _{i\geq0, H(D,\mathbf{u}^{*};\mu_{i})< 0}m(\mu_{i}). $$

(5.11)

Using (1.3), a direct computation yields

$$\begin{aligned} &\Phi_{\mathbf{u}} \bigl(\mathbf{u}^{*} \bigr)= \begin{pmatrix} \frac{\partial g_{1}}{\partial u}&\frac{\partial g_{1}}{\partial v}\\ \frac{\partial g_{2}}{\partial u}&\frac{\partial g_{2}}{\partial v} \end{pmatrix}_{(u,v)=(u^{*},v^{*})}= \begin{pmatrix} d_{1}&0\\ -\frac{\alpha d_{2}d_{3}v^{*}}{(1+\alpha u^{*})^{2}}&\frac{d_{2}(1+\alpha u^{*}+d_{3})}{1+\alpha u^{*}} \end{pmatrix}, \\ &G_{\mathbf{u}} \bigl(\mathbf{u}^{*} \bigr)= \begin{pmatrix} \frac{\partial f_{1}}{\partial u}&\frac{\partial f_{1}}{\partial v}\\ \frac{\partial f_{2}}{\partial u}&\frac{\partial f_{2}}{\partial v} \end{pmatrix}_{(u,v)=(u^{*},v^{*})}= \begin{pmatrix} A_{11}&A_{12}\\ b^{2}&-b \end{pmatrix} \quad \bigl(\mbox{see (5.2) for } A_{11},A_{12}\bigr), \\ & \begin{aligned} H \bigl(D,\mathbf{u}^{*};\mu \bigr)={}&\mu^{2}+ \biggl[ \frac{b(1+\alpha u^{*})}{d_{2}(1+\alpha u^{*}+d_{3})}-\frac{\alpha d_{3}v^{*}A_{12}}{d_{1}(1+\alpha u^{*}+d_{3})(1+\alpha u^{*})}-\frac{A_{11}}{d_{1}} \biggr]\mu \\ &{} -\frac{b(1+\alpha u^{*})A_{11}}{d_{1}d_{2}(1+\alpha u^{*}+d_{3})}-\frac{b^{2}(1+\alpha u^{*})A_{12}}{d_{1}d_{2}(1+\alpha u^{*}+d_{3})}. \end{aligned} \end{aligned}$$

Since (1.3) and $\beta b< r(1+bm)$, we obtain

$$ \begin{aligned} \det \bigl(G_{\mathbf{u}} \bigl(\mathbf{u}^{*} \bigr) \bigr)&=-br+2bru^{*}+\frac{\beta bm{v^{*}}^{2}}{(u^{*}+mv^{*})^{2}}+\frac{\beta b^{2}m{u^{*}}^{2}}{(u^{*}+mv^{*})^{2}} \\ &=br-\frac{2\beta b^{2}}{1+bm}+\frac{\beta b^{3}m}{(1+bm)^{2}}+\frac{\beta b^{2}}{(1+bm)^{2}} \\ &=\frac{b [r(1+bm)^{2}-2\beta b(1+bm)+\beta b^{2}m+\beta b ]}{(1+bm)^{2}} \\ &=\frac{b(1+bm)[r(1+bm)-\beta b]}{(1+bm)^{2}}>0. \end{aligned} $$

Then we get

$$ \begin{aligned}[b] &{-}\frac{b(1+\alpha u^{*})A_{11}}{d_{1}d_{2}(1+\alpha u^{*}+d_{3})}- \frac {b^{2}(1+\alpha u^{*})A_{12}}{d_{1}d_{2}(1+\alpha u^{*}+d_{3})} \\ &\quad =\frac{1+\alpha u^{*}}{d_{1}d_{2}(1+\alpha u^{*}+d_{3})}\det \bigl(G_{\mathbf{u}} \bigl(\mathbf{u}^{*} \bigr) \bigr)>0. \end{aligned} $$

(5.12)

Furthermore, we have

$$ \lim_{d_{j}\rightarrow\infty}H \bigl(D,\mathbf{u}^{*};\mu \bigr)= \mu^{2}-\delta _{j}\mu,\quad \forall j=2,3\quad \bigl(\mbox{see (5.2) for } \delta_{j}\bigr). $$

(5.13)

In order to prove Theorem 5.1, we shall use the following lemma.

Lemma 5.3

If $d_{3}=0$ and $d_{1}=d_{2}=d_{*}\geq d$, then there exists a positive constant $d^{*}$ such that $\operatorname{index}(F(D_{*};\cdot),\mathbf{u}^{*})=1$ if $d_{*}\geq d^{*}$, where $D_{*}=(d_{*},d_{*},0)$.

Proof

Since

$$H \bigl(D_{*},\mathbf{u}^{*};\mu \bigr)=\mu^{2}+ \biggl( \frac{b}{d_{*}}- \frac {A_{11}}{d_{*}} \biggr)\mu-\frac{bA_{11}}{d_{*}^{2}}- \frac{b^{2}A_{12}}{d_{*}^{2}}, $$

the root of equation $H(D_{*},\mathbf{u}^{*};\mu)=0$ are

$$ \mu^{+}=\frac{A_{11}-b+\sqrt{\triangle}}{2d_{*}},\qquad \mu^{-}=\frac {A_{11}-b-\sqrt{\triangle}}{2d_{*}}, $$

(5.14)

where

$$\triangle=(A_{11}-b)^{2}+4 \bigl(bA_{11}+b^{2}A_{12} \bigr). $$

It is obvious that $\operatorname{Re}\mu^{+}\rightarrow0$ as $d_{*}\rightarrow\infty$. So there exists a positive constant $d^{*}$ such that Re$\mu^{+}<\mu_{1}$ when $d_{*}\geq d^{*}$. For each $i\geq1$, we have $H(D_{*},\mathbf {u}^{*};\mu_{i})>0$. Furthermore, $H(D_{*},\mathbf{u}^{*};\mu_{0})>0$ by (5.12). So, $\operatorname{index}(F(D_{*};\cdot),\mathbf{u}^{*})=(-1)^{0}=1$ by (5.11). The proof is completed. □

Proof of Theorem 5.1

From Remark 4.3 and Lemma 5.3, we know that there exists a positive constant $d^{*}$ such that (1.2) does not have non-constant positive solution and $\operatorname{index}(F(D^{*};\cdot),\mathbf{u}^{*})=1$ when $d_{1}=d_{2}=d^{*}$ and $d_{3}=0$, where $D^{*}=(d^{*},d^{*},0)$. This means $\deg(F(D^{*};\cdot ),0,\mathbf{B}(C))=1$. For $d_{1},d_{2}\geq d$ and $d_{3}\geq0$, we define a homotopy as

$$ \textstyle\begin{cases} -\Delta \{[td_{1}+(1-t)d^{*}] u \}=ru(1-u)-\frac{\beta uv}{u+mv}&\mbox{in } \Omega,\\ -\Delta \{ [td_{2}+\frac{td_{2}d_{3}}{1+\alpha u}+(1-t)d^{*} ]v \}=v (b-\frac{v}{u} )&\mbox{in } \Omega,\\ \partial_{\nu}u=\partial_{\nu}v=0&\mbox{on } \partial\Omega, \end{cases} $$

(5.15)

where $t\in[0,1]$. u is a non-constant positive solution of (5.15) if and only if $\mathbf{u}\in\mathbf{B}(C)$ and

$$ F(t,\mathbf{u}):=\mathbf{u}-(\mathrm{I}-\Delta)^{-1} \bigl\{ \Phi _{\mathbf{u}}^{-1}(\mathbf{u},t) \bigl[G(\mathbf{u})+ \nabla\mathbf {u}\Phi_{\mathbf{u}\mathbf{u}}(\mathbf{u},t)\nabla\mathbf {u} \bigr]+ \mathbf{u} \bigr\} =0\quad \mbox{in } X, $$

(5.16)

where

$$\Phi(\mathbf{u},t)= \biggl( \bigl[td_{1}+(1-t)d^{*} \bigr] u, \biggl[td_{2}+\frac {td_{2}d_{3}}{1+\alpha u}+(1-t)d^{*} \biggr]v \biggr)^{T}. $$

Furthermore,

$$ F(D;\mathbf{u})=F(1,\mathbf{u})\quad \mbox{and}\quad F \bigl(D^{*}, \mathbf {u} \bigr)=F(0;\mathbf{u}). $$

(5.17)

By the homotopy invariance of the Leray–Schauder degree, one can obtain

$$ \deg \bigl(F(D;\cdot),0,\mathbf{B}(C) \bigr)=\deg \bigl(F \bigl(D^{*}; \cdot \bigr),0,\mathbf{B}(C) \bigr)=1. $$

(5.18)

Denote by $\mu^{+}$ and $\mu^{-}$, with Re$\mu^{-}\leq$Re$\mu^{+}$, the two roots to $H(D,\mathbf{u}^{*};\mu)=0$. From (5.13), we see that

$$ \lim_{d_{2}\rightarrow\infty}\mu^{-}=0,\qquad \lim _{d_{2}\rightarrow\infty }\mu^{+}=\delta_{2},\qquad \lim _{d_{3}\rightarrow\infty}\mu^{-}=0,\qquad \lim_{d_{3}\rightarrow\infty}\mu^{+}= \delta_{3}. $$

(5.19)

If $\delta_{2}\in(\mu_{i},\mu_{i+1})$ for some positive odd integer i, then, for $d_{2}$ large enough, one has

$$ 0=\mu_{0}< \mu^{-}< \mu_{1},\qquad \mu^{+}\in( \mu_{i},\mu_{i+1}). $$

(5.20)

Hence $H(D,\mathbf{u}^{*};\mu_{j})<0$ is equivalent to $j\in\{1,2,\ldots,i\}$. Since i is odd, by Lemma 5.2, we have

$$ \operatorname{index} \bigl(F(D;\cdot),\mathbf{u}^{*} \bigr)=(-1)^{i}=-1. $$

(5.21)

So $F(D;\mathbf{u})=0$ has at least another positive solution that is different from $\mathbf{u}^{*}$. Otherwise the degree of $F=0$ in $\mathbf{B}(C)$ should be −1, which contradicts (5.18). Hence the first assertion of the theorem is proved. The second assertion can be proved similarly. The proof is completed. □

6 Conclusion

In this paper, we deal with a strong coupled predator–prey model with modified Holling–Tanner functional response under homogeneous Neumann boundary conditions. First we study the stability of constant steady state for such model. Then we establish a priori upper and lower bounds for the positive solutions, deal with the non-existence of the non-constant positive solutions, and establish the existence of non-constant positive solutions for a range of diffusion and cross-diffusion coefficients.