Steady state for a predator–prey cross-diffusion system with the Beddington–DeAngelis and Tanner functional response

Abstract

The main goal of this paper is investigating the existence of nonconstant positive steady states of a linear prey–predator cross-diffusion system with Beddington–DeAngelis and Tanner functional response. An analytical method and fixed point index theory plays a significant role in our main proofs.

1 Introduction

We propose and study a predator–prey cross-diffusion system with the Beddington–DeAngelis and Tanner functional response

$$ \textstyle\begin{cases} \frac{\partial u}{\partial t}-d_{1}\Delta u-d_{2} \Delta v=u (a-u-\frac{v}{1+\alpha u+\beta v} ), & x\in \Omega , t>0, \\ \frac{\partial v}{\partial t}+d_{3}\Delta u-d_{4}\Delta v=v (b- \frac{v}{\gamma u} ), & x\in \Omega , t>0, \\ \frac{\partial u}{\partial \nu }=\frac{\partial v}{\partial \nu }=0, & x\in \partial \Omega , t>0, \\ u(x,0)=u_{0}(x)>0, \qquad v(x,0)=v_{0}(x)\geq (\not \equiv )\ 0, & x\in \Omega , \end{cases} $$

(1.1)

where a, b, α, β, γ are positive constants. Here u and v stand for the population densities of the prey and predator, respectively. The bounded domain $\Omega \in \mathbb{R}^{n}$ ($n\geq 1$) possesses a smooth boundary ∂Ω; ν is the outward directional derivative normal to ∂Ω. The diffusion rates are described by $d_{1}>0$, $d_{2}>0$, $d_{3}>0$ and $d_{4}>0$; a and b are the death rate of the predator and the intrinsic growth rate of the prey, respectively. The smooth functions $u_{0}$ and $v_{0}$ on Ω represent the initial population densities of prey and predator. The corresponding ordinary differential system (ODS) of (1.1) is performed as

$$ \textstyle\begin{cases} \frac{du}{dt}=u (a-u-\frac{v}{1+\alpha u+\beta v} ), \\ \frac{dv}{dt}=v (b-\frac{v}{\gamma u} ), \end{cases} $$

(1.2)

which was introduced by Luo [1] in recent years, and the globally asymptotical stability of (1.2) has been discussed by constructing a feasible Lyapunov function. Recently, with rapid development of biotechnology, we need to accurately illustrate the interaction between prey and predator in a real ecological environment such as hare and lynx, sparrow hawk and sparrow, spider mite and mite, and so on. Such interesting natural phenomena as the existence and global stability of interior periodic solutions and the global stability of reaction–diffusion system have been described by this ODE model [1] or the corresponding PDE system [2]. By utilizing the iteration method, Liu and Li [2] proved the globally asymptotic stability of the corresponding parabolic PDE system.

The impact of prey-taxis leads to the linear cross-diffusion in model (1.1), which means that the prey attraction to the predator may give rise to the migration of predator following the gradient of the density function of prey. By utilizing $-d_{3}\Delta u$ to describe this impact we will incorporate $d_{2}\Delta v$ into the first equation of system (1.1) to indicate the repelling effect from predator to prey. The message about the location of predator can guide the prey escape from them (see [3–5]).

The Beddington–DeAngelis functional response $\frac{uv}{1+\alpha u+\beta v}$ was firstly discovered by Beddington [6] and DeAngelis et al. [7] and intensively investigated by many researchers in mathematics, ecology, and biology (see [1, 2, 8, 9]). This functional response, in the denominator, includes an additional term βv compared with Holling-II functional response. This term illustrates the interaction effect among predators and provides a more general situation to describe the interaction between prey and predator.

As far as we can survey, this is the first paper studying the steady state for a predator–prey cross-diffusion system with the Beddington–DeAngelis and Tanner functional response. For notational simplicity, we introduce some notations used in the rest of this paper.

Notation

• (1) We denote the eigenvalues sequence by $0=\lambda _{0}\leq \lambda _{1}\leq \cdots \leq \lambda _{k}\leq \cdots$ for the elliptic operator Δ on Ω with homogeneous Neumann boundary condition;

• (2) The multiplicity of the eigenvalue $\lambda _{k}$ is denoted by $\mathcal{M}_{k}$;

• (3) The corresponding normalized eigenfunctions of $\lambda _{k}$ are represented by the set $\{\mathcal{E}_{kj}:1\leq j\leq \mathcal{M}_{k}\}$;

• (4) $\mathcal{A}=C_{\nu }^{1}(\overline{\Omega })\oplus C_{\nu }^{1}( \overline{\Omega })$, where $C_{\nu }^{1}(\overline{\Omega })\triangleq \{ \Phi \in C^{1}( \overline{\Omega }):\frac{\partial \Phi }{\partial \nu }=0 \mbox{ on } \partial \Omega \}$;

• (5) $\mathcal{D}=\mathcal{D}_{u}\oplus \mathcal{D}_{v}$, where $\mathcal{D}_{u}\triangleq \{ \Phi \in C_{\nu }^{1}( \overline{\Omega }):\Phi < a \mbox{ in } \overline{\Omega } \}$ and $\mathcal{D}_{v}\triangleq \{ \Phi \in C_{\nu }^{1}( \overline{\Omega }):\Phi < ab\gamma \mbox{ in } \overline{\Omega } \}$;

• (6) $\mathcal{Q}=\mathcal{S}\oplus \mathcal{S}$, where $\mathcal{S}\triangleq \{ \Phi \in C_{\nu }^{1}(\overline{\Omega }): \Phi (x)\geq 0,x\in \overline{\Omega } \}$;

• (7) $\mathcal{D}'=\mathcal{D}\cap \mathcal{Q}$.

The rest part of this paper is organized as follows. In Sect. 2, we provide the study of the local stability of semitrivial or interior equilibrium of model (1.1). In Sect. 3, by utilizing the fixed point index theory we investigate the existence of the nonconstant positive steady state of system (1.1). Finally, in Sect. 4, we offer some conclusions on our main theoretical results.

2 Local stability

It is obvious via a direct calculation that model (1.1) possesses one semitrivial equilibrium $(a,0)$ and one unique interior equilibrium $(u^{*},v^{*})$, where

$$ u^{*}= \frac{[a\alpha -1+b\gamma (a\beta -1)]+\sqrt{[a\alpha -1+b\gamma (a\beta -1)]^{2} +4a(\alpha +b\beta \gamma )}}{2(\alpha +b\beta \gamma )},\qquad v^{*}=b\gamma u^{*}. $$

We discuss the local stability of the semitrivial equilibrium $(a,0)$ and the interior equilibrium $(u^{*},v^{*})$ of system (1.1) as follows.

Lemma 2.1

• (1) When $a< b$, $(a,0)$ is unstable.

• (2) When $\frac{b\gamma u^{*}[1+(2\alpha +b\beta \gamma )u^{*}]}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}}< a+b$ and $a> \frac{b\gamma u^{*}(\alpha +b\beta \gamma )u^{*}}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}}$, then the interior equilibrium $(u^{*},v^{*})$ is locally asymptotically stable.

• (3) When $a+b< \frac{b\gamma u^{*}[1+(2\alpha +b\beta \gamma )u^{*}]}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}}$, then the interior equilibrium $(u^{*},v^{*})$ is unstable.

Proof

At a constant solution $E^{*}=(u,v)$, the linearized issue of model (1.1) can be proposed by

$$\left(\begin{array}{c}\frac{\partial u}{\partial t}\\ \frac{\partial v}{\partial t}\end{array}\right)=\mathcal{L}\left(\begin{array}{c}u(t)\\ v(t)\end{array}\right)=\left(\begin{array}{cc}{d}_1& d_2\\ -d_3& d_4\end{array}\right)\left(\begin{array}{c}\mathrm{\Delta }u\\ \mathrm{\Delta }v\end{array}\right)+\left(\begin{array}{cc}a-2u-\frac{v(1+\beta v)}{{(1+\alpha u+\beta v)}^2}& -\frac{u(1+\alpha u)}{{(1+\alpha u+\beta v)}^2}\\ \frac{v^2}{\gamma u^2}& b-\frac{2v}{\gamma u}\end{array}\right)\left(\begin{array}{c}u(t)\\ v(t)\end{array}\right).$$

For notational simplicity, we denote

$$\begin{array}{c}D=\left(\begin{array}{cc}{d}_1& d_2\\ -d_3& d_4\end{array}\right),\hfill \\ J(u,v)=\left(\begin{array}{cc}{F}_u(u,v)& F_v(u,v)\\ G_u(u,v)& G_v(u,v)\end{array}\right)=\left(\begin{array}{cc}a-2u-\frac{v(1+\beta v)}{{(1+\alpha u+\beta v)}^2}& -\frac{u(1+\alpha u)}{{(1+\alpha u+\beta v)}^2}\\ \frac{v^2}{\gamma u^2}& b-\frac{2v}{\gamma u}\end{array}\right).\hfill \end{array}$$

In view of Theorem 5.1.1 of [10], it is obvious that $E^{*}=(u,v)$ is locally asymptotically stable when all the eigenvalues of the operator $\mathcal{L}$ possess negative real parts. On the contrary, $E^{*}=(u,v)$ is unstable. Thus the stability of the equilibrium is based on the study of the characteristic equation

$$ \det (\mu I-J_{i})=\mu ^{2}-\operatorname{trace}J_{i} \mu +\det J_{i}=0. $$

Here $\operatorname{trace}J_{i}=-\lambda _{i}(d_{1}+d_{4})+F_{u}(u,v)+G_{v}(u,v)$ and $\det J_{i}=(d_{1}d_{4}+d_{2}d_{3})\lambda _{i}^{2} -[d_{1}G_{v}(u,v)- d_{2}G_{u}(u,v)+d_{3}F_{v}(u,v)+d_{4}F_{u}(u,v)] \lambda _{i}+F_{u}(u,v)G_{v}(u,v)-F_{v}(u,v)G_{u}(u,v)$.

(1) When $E^{*}=(a,0)$, we get

$$J(a,0)=\left(\begin{array}{cc}-a& -\frac{a}{1+a\alpha }\\ 0& b\end{array}\right).$$

If $b>a$, then for $i=0$, we obtain $\operatorname{trace}J_{i}=-a+b>0$. Therefore $(a,0)$ is unstable.

(2) When $E^{*}=(u^{*},v^{*})$, we have

$$J(u^{\ast },v^{\ast })=\left(\begin{array}{cc}-a+\frac{b\gamma u^{\ast }[1+(2\alpha +b\beta \gamma )u^{\ast }]}{{[1+(\alpha +b\beta \gamma )u^{\ast }]}^2}& -\frac{u^{\ast }(1+\alpha u^{\ast })}{{[1+(\alpha +b\beta \gamma )u^{\ast }]}^2}\\ b^2\gamma & -b\end{array}\right)\triangleq \left(\begin{array}{cc}{F}_u^{\ast }& F_v^{\ast }\\ G_u^{\ast }& G_v^{\ast }\end{array}\right).$$

Here $-a< F_{u}^{*}< \frac{b\gamma u^{*}[1+(2\alpha +b\beta \gamma )u^{*}]}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}}$, $-\frac{u^{*}}{1+\alpha u^{*}}< F_{v}^{*}<0$, $G_{u}^{*}>0$, $G_{v}^{*}<0$. Together with the hypothesis of (2), we can observe that

$$ F_{u}^{*}+G_{v}^{*}=-a+ \frac{b\gamma u^{*}[1+(2\alpha +b\beta \gamma )u^{*}]}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}}-d< 0, $$

and then

$$ \operatorname{trace}J_{i}=-(d_{1}+d_{2})\lambda _{i}-\bigl(F_{u}^{*}+G_{v}^{*} \bigr)< 0. $$

In view of the hypothesis of (2), we can observe that

$$ \begin{aligned} F_{u}^{*}G_{v}^{*}-F_{v}^{*}G_{u}^{*}&=ab- \frac{b^{2}\gamma u^{*}[1+(2\alpha +b\beta \gamma )u^{*}]}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}} + \frac{b^{2}\gamma u^{*}(1+\alpha u^{*})}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}} \\ &=ab- \frac{b^{2}\gamma u^{*}(\alpha +b\beta \gamma )u^{*}}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}}>0. \end{aligned} $$

We can find a sufficiently large eigenvalue $d_{2}$ to guarantee that

$$ \begin{aligned} \det J_{i}={}&{-}(d_{1}d_{4}+d_{2}d_{3}) \lambda _{i}^{2}+\bigl(-d_{1}G_{v}^{*}+d_{2}G_{u}^{*}-d_{3}G_{v}^{*}-d_{4}F_{u}^{*} \bigr) \lambda _{i} \\ &{}+F_{u}^{*}G_{v}^{*}-F_{v}^{*}G_{u}^{*}>0. \end{aligned} $$

Thus the interior equilibrium $(u^{*},v^{*})$ is locally asymptotically stable.

(3) When $a+b< \frac{b\gamma u^{*}[1+(2\alpha +b\beta \gamma )u^{*}]}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}}$, we obtain

$$ \operatorname{trace}J_{i}=-a+ \frac{b\gamma u^{*}[1+(2\alpha +b\beta \gamma )u^{*}]}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}}-d>0. $$

Therefore $(u^{*},v^{*})$ is unstable. This ends the proof of the lemma. □

3 Nonconstant positive steady states

In this section, we offer a rigorous study of interior solutions of the corresponding strongly coupled elliptic model

$$ \textstyle\begin{cases} -d_{1}\Delta u-d_{2}\Delta v=u (a-u- \frac{v}{1+\alpha u+\beta v} ), & x\in \Omega , t>0, \\ d_{3}\Delta u-d_{4}\Delta v=v (b-\frac{v}{\gamma u} ), & x\in \Omega , t>0, \\ \frac{\partial u}{\partial \nu }=\frac{\partial v}{\partial \nu }=0, & x\in \partial \Omega , t>0, \\ u(x,0)=u_{0}(x)>0, \qquad v(x,0)=v_{0}(x)\geq (\not \equiv )\ 0, & x\in \Omega . \end{cases} $$

(3.1)

Meanwhile, we will also investigate the nonconstant positive steady states of model (1.1). The existence of an interior solution of a linear cross-diffusion model has been studied by utilizing the approach proposed by [11] (upper and lower solutions) and many others.

Lemma 3.1

Any interior solution $(u,v)$ for system (1.1) fulfills $u(x),v(x)\leq \max \{a,ab\gamma \}$ in Ω̅.

Proof

In view of the equation of u for model (3.1), we obtain

$$ -\Delta (d_{1}u+d_{2}v)=u \biggl(a-u- \frac{v}{1+\alpha u+\beta v} \biggr)\leq u(a-u). $$

By the maximum principle proposed by [12] we have $u(a-u)\geq 0$. Hence $u\leq a$. Together with the equation for u and the equation for v for system (3.1), we get

$$ -\Delta (-d_{3}u+d_{4}v)=v \biggl(b- \frac{v}{\gamma u} \biggr)\leq bv-\frac{v^{2}}{a\gamma }. $$

Applying the maximum principle in [12], we have $bv-\frac{v^{2}}{a\gamma }\geq 0$. Thus we obtain $v\leq ab\gamma$. This ends the proof of the lemma. □

Next, we introduce the compact map $CM\in C^{2}(\overline{\Omega })\oplus C^{2}(\overline{\Omega }) \rightarrow C^{1}(\overline{\Omega })\oplus C^{1}(\overline{\Omega })$ as follows:

$$CM(u,v)={(-\mathcal{D}\mathrm{\Delta }+cI)}^{-1}\left(\begin{array}{c}au-u^2-\frac{uv}{1+\alpha u+\beta v}+cu\\ bv-\frac{v^2}{\gamma u}+cv\end{array}\right).$$

Here we can choose a sufficiently large constant $c>0$ guaranteeing that the functions $au-u^{2}-\frac{uv}{1+\alpha u+\beta v}+cu$ and $bv-\frac{v^{2}}{\gamma u}+cv$ are increasing for u and v, respectively. We can easily observe that system (3.1) is equivalent to $(u,v)=CM(u,v)$, which implies that there exists a nonconstant interior solution of system (3.1) representing a nonconstant interior fixed point of CM in $\mathcal{D}'$. When $(a,0)$ and $(u^{*},v^{*})$ exist, we suppose that they stand for an isolated fixed point of CM. On the contrary, there must exist a nonconstant fixed point in $\operatorname{Int}\mathcal{D}'$ (the interior of $\mathcal{D}'$). Therefore the associate indexes in $\mathcal{Q}$ are well defined. Based on the method introduced by [13], we can directly calculate the fix point index of CM with respect to $\mathcal{Q}$ over $\operatorname{Int}\mathcal{D}'$.

Lemma 3.2

If $\frac{b\gamma u^{*}[1+(2\alpha +b\beta \gamma )u^{*}]}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}}< a+b$ and $a> \frac{b\gamma u^{*}(\alpha +b\beta \gamma )u^{*}}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}}$, then $\operatorname{index}_{\mathcal{Q}}(CM,\operatorname{Int}\mathcal{D}')=1$.

Proof

We introduce the homotopic map, for $\eta \in (0,1)$, $CM_{\eta }\triangleq HM\in C^{2}(\overline{\Omega })\oplus C^{2}( \overline{\Omega })\rightarrow C^{1}(\overline{\Omega })\oplus C^{1}( \overline{\Omega })$ by

$$HM(u,v)={(-\mathcal{D}\mathrm{\Delta }+cI)}^{-1}\left(\begin{array}{c}au-u^2-\frac{\eta uv}{1+\alpha u+\beta v}+cu\\ bv-\frac{\eta v^2}{\gamma u}+cv\end{array}\right).$$

Utilizing a similar discussion as in Lemma 3.1, we can easily obtain that any interior fixed point $(u,v)$ for HM fulfills $u(x),v(x)\leq \max \{a,ab\gamma \}$ in Ω̅. Thus we can easily draw a conclusion that each fixed point for HM belongs to $\operatorname{Int}\mathcal{D}'$. In addition, the corresponding index of $CM_{\eta }$ over the interior of $\mathcal{D}'$ regarding $\mathcal{Q}$ is well defined. In view of the homotopy invariance theorem, we can obtain that $\operatorname{index}_{\mathcal{Q}}(CM_{0},\operatorname{Int}\mathcal{D}')= \operatorname{index}_{\mathcal{Q}}(CM_{1},\operatorname{Int}\mathcal{D}') = \operatorname{index}_{\mathcal{S}}(B,\operatorname{Int}\mathcal{D})$, where

$$B(u,v)={(-\mathcal{D}\mathrm{\Delta }+cI)}^{-1}\left(\begin{array}{c}au-u^2+cu\\ bv+cv\end{array}\right).$$

Based on the above discussion, we argue that $\operatorname{index}_{\mathcal{S}}(B,\operatorname{Int}\mathcal{D})=1$. To prove this, we propose the homotopic map

$$B_{\eta }(u,v)={(-\mathcal{D}\mathrm{\Delta }+cI)}^{-1}\left(\begin{array}{c}\eta (au-u^2)+cu\\ \eta bv+cv\end{array}\right)$$

for $\eta \in [0,1]$. Then we can directly obtain that $\operatorname{index}_{\mathcal{S}}(B_{0},\operatorname{Int}\mathcal{D})= \operatorname{index}_{\mathcal{S}}(B_{1},\operatorname{Int}\mathcal{D})$. Our concern is the eigenvalue issue

$$ B_{0}'(0,0) (\Phi ,\Psi )^{T}=\rho (\Phi ,\Psi )^{T},\quad (\Phi ,\Psi )\neq (0,0), $$

(3.2)

for $\rho >0$, where

$$B_0(0,0)={(-\mathcal{D}\mathrm{\Delta }+cI)}^{-1}\left(\begin{array}{cc}c& 0\\ 0& c\end{array}\right).$$

The corresponding eigenfunction formulas for Φ and Ψ can be performed as follows:

$$ \Phi =\sum_{k=0}^{\infty }\sum _{j=1}^{\mathcal{M}_{k}} \phi _{kj} \mathcal{E}_{kj} \quad \mbox{and}\quad \Psi =\sum_{k=0}^{ \infty } \sum_{j=1}^{\mathcal{M}_{k}}\psi _{kj} \mathcal{E}_{kj}, $$

where $\phi _{kj},\psi _{kj}\in \mathbb{R}$. We easily rewrite system (3.2) as follows:

$$ \textstyle\begin{cases} \sum_{k=0}^{\infty }\sum_{j=1}^{\mathcal{M}_{k}}[(d_{1} \lambda _{k}\rho +c\rho -c)\phi _{kj}+d_{2}\lambda _{k}\rho \psi _{kj}] \mathcal{E}_{kj}=0, \\ \sum_{k=0}^{\infty }\sum_{j=1}^{\mathcal{M}_{k}}[-d_{3}\lambda _{k} \rho \phi _{kj}+(d_{4}\lambda _{k}\rho +c\rho -c)\psi _{kj}] \mathcal{E}_{kj}=0. \end{cases} $$

(3.3)

As we know, $\{\mathcal{E}_{kj}:1\leq j\leq \mathcal{M}_{k}\}$ in $L^{2}(\Omega )$ stands for a complete orthonormal basis. By multiplying the above two equations by $\varphi _{kj}$ and integrating over Ω we get the equation

$$\left(\begin{array}{cc}{d}_1{\lambda }_k\rho +c\rho -c& d_2{\lambda }_k\rho \\ -d_3{\lambda }_k\rho & d_4{\lambda }_k\rho +c\rho -c\end{array}\right)\left(\begin{array}{c}{\varphi }_{kj}\\ {\psi }_{kj}\end{array}\right)=0.$$

Therefore

$$\begin{array}{rl}& det\left(\begin{array}{cc}{d}_1{\lambda }_k\rho +c\rho -c& d_2{\lambda }_k\rho \\ -d_3{\lambda }_k\rho & d_4{\lambda }_k\rho +c\rho -c\end{array}\right)\\ & =[(d_1{\lambda }_k+c)(d_4{\lambda }_k+c)+d_2{d}_3{\lambda }_k^2]{\rho }^2-c(d_1{\lambda }_k+d_4{\lambda }_k+2c)\rho +c^2\\ & =0.\end{array}$$

Denoting

$$ \rho _{\pm }= \frac{\overline{\Lambda }\pm \sqrt{\overline{\Lambda }^{2}-4[(d_{1} \lambda _{k}+c)(d_{4}\lambda _{k}+c)+d_{2}d_{3}\lambda _{k}^{2}]c^{2}}}{2[(d_{1} \lambda _{k}+c)(d_{4}\lambda _{k}+c)+d_{2}d_{3}\lambda _{k}^{2}]}, $$

where $\overline{\Lambda }=c(d_{1}\lambda _{k}+d_{4}\lambda _{k}+2c)$, we have

$$ \begin{aligned} \rho _{+}&< \frac{\overline{\Lambda }+\sqrt{\overline{\Lambda }^{2}-4 (d_{1}\lambda _{k}+c)(d_{4}\lambda _{k}+c)c^{2}}}{2(d_{1}\lambda _{k}+c)(d_{4}\lambda _{k}+c)} \\ &=\frac{c[(d_{1}\lambda _{k}+c)+(d_{4}\lambda _{k}+c)+ \vert (d_{1}\lambda _{k}+c)- (d_{4}\lambda _{k}+c) \vert ]}{2(d_{1}\lambda _{k}+c)(d_{4}\lambda _{k}+c)} \leq 1. \end{aligned} $$

Based on Lemma 13.1 of [14], we get $\operatorname{index}_{\mathcal{S}}(B_{0},\operatorname{Int}\mathcal{D})=1$. Hence $\operatorname{index}_{\mathcal{S}}(B,\operatorname{Int}\mathcal{D})= \operatorname{index}_{\mathcal{S}}(B_{1},\operatorname{Int}\mathcal{D})=1$. This finishes the proof of the lemma. □

Lemma 3.3

$\operatorname{index}_{\mathcal{Q}}(CM,(a,0))=0$.

Proof

Denote $\overline{\mathcal{Q}}_{(a,0)}=C_{\nu }^{1}(\overline{\Omega }) \oplus \mathcal{S}$, $\mathcal{W}_{(a,0)}=C_{\nu }^{1}(\overline{\Omega })\oplus \{ 0 \}$, and

$$C{M}^{\prime }(a,0)={(-\mathcal{D}\mathrm{\Delta }+cI)}^{-1}\left(\begin{array}{cc}c-a& -\frac{a}{1+a\alpha }\\ 0& b+c\end{array}\right).$$

Set $CM'(a,0)(\Phi ,\Psi )^{T}=(\Phi ,\Psi )^{T}\in \overline{\mathcal{Q}}_{(a,0)}$. Then we get

$$ \textstyle\begin{cases} d_{1}\Delta \Phi +d_{2}\Delta \Psi =a\Phi + \frac{a}{1+a\alpha }\Psi & \mbox{in } \Omega , \\ d_{3}\Delta \Phi -d_{4}\Delta \Psi =b\Psi & \mbox{in } \Omega , \\ \frac{\partial \Phi }{\partial \nu }= \frac{\partial \Psi }{\partial \nu }=0 & \mbox{on } \partial \Omega . \end{cases} $$

(3.4)

From the first equation of system (3.4) we have $\Phi =\Psi =0$ in Ω. Thus $I-CM'(a,0)$ represents an invertible matrix on $\overline{\mathcal{Q}}_{(a,0)}$. On the other hand, we can easily check that $CM'(a,0)$ possesses property γ, which means that for $(\Phi ,\Psi )\equiv (0,1)\in \overline{\mathcal{Q}}_{(a,0)} \setminus \mathcal{W}_{(a,0)}$ and $s=\frac{c}{b+c}\in (0,1)$, $[(\Phi ,\Psi )^{T}-sCM'(a,0)(\Phi ,\Psi )^{T}]\in \mathcal{W}_{(a,0)}$. Hence, applying Lemma 4.1(i) in [13], we obtain a precise result. This finishes the proof of the lemma. □

Denote

$$\begin{array}{rl}Det(\lambda )& :=det\left(\begin{array}{cc}{d}_1\lambda -F_u^{\ast }& d_2\lambda -F_v^{\ast }\\ -d_3\lambda -G_u^{\ast }& d_4\lambda -G_v^{\ast }\end{array}\right)\\ & =(d_1{d}_4+d_2{d}_3){\lambda }^2-[d_1{G}_v^{\ast }-d_2{G}_u^{\ast }+d_3{F}_v^{\ast }+d_4{F}_u^{\ast }]\lambda +F_u^{\ast }{G}_v^{\ast }-F_v^{\ast }{G}_u^{\ast }.\end{array}$$

In the rest of the paper, we consider $\operatorname{index}_{\mathcal{Q}}(CM,(u^{*},v^{*}))$ in the following three situations: (1) For all $\lambda >0$, $\operatorname{Det}(\lambda )>0$; (2) There exists, with multiplicity one, precisely a simple interior solution for $\operatorname{Det}(\lambda )=0$; (3) There exist, with multiplicity one, two interior solutions for $\operatorname{Det}(\lambda )=0$. Now we present our main theoretical result.

Theorem 3.1

• (1) If $\operatorname{Det}(\lambda )>0$ for all $\lambda >0$, then $\operatorname{index}_{\mathcal{Q}}(CM,(u^{*},v^{*}))=1$;

• (2) If $\operatorname{Det}(\lambda )=0$ has exactly a multiplicity-one simple positive solution $\lambda ^{*}$ in an open interval $(\lambda _{k^{*}},\lambda _{k^{*}+1})$ for some nonnegative integer $k^{*}$, then

  $$ \operatorname{index}_{\mathcal{Q}}\bigl(CM,\bigl(u^{*},v^{*} \bigr)\bigr)= \textstyle\begin{cases} -1, & \sum_{k=0}^{k^{*}}\mathcal{M}_{k} \textit{ is odd}, \\ 1, & \sum_{k=0}^{k^{*}}\mathcal{M}_{k} \textit{ is even}. \end{cases} $$

  In addition, when $\sum_{k=0}^{k^{*}}\mathcal{M}_{k}$ is odd, system (3.1) possesses at least one nonconstant interior solution;

• (3) Suppose that $\operatorname{Det}(\lambda )=0$ has two interior solutions $\lambda _{+}^{*}$ and $\lambda _{-}^{*}$ in two open intervals $(\lambda _{k_{1}^{*}},\lambda _{k_{1}^{*}+1})$ and $(\lambda _{k_{2}^{*}},\lambda _{k_{2}^{*}+1})$, respectively, where $k_{1}^{*}>k_{2}^{*}\geq 0$. Then

  $$ \operatorname{index}_{\mathcal{Q}}\bigl(CM,\bigl(u^{*},v^{*} \bigr)\bigr)= \textstyle\begin{cases} -1, & \sum_{k=k_{1}^{*}+1}^{k_{2}^{*}}\mathcal{M}_{k} \textit{ is odd}, \\ 1, & \sum_{k=k_{1}^{*}+1}^{k_{2}^{*}}\mathcal{M}_{k} \textit{ is even}. \end{cases} $$

  In addition, when $\sum_{k=k_{1}^{*}+1}^{k_{2}^{*}}\mathcal{M}_{k}$ is odd, system (3.1) possesses at least one nonconstant interior solution;

• (4) Suppose that $a< \frac{b\gamma u^{*}[1+(2\alpha +b\beta \gamma )u^{*}]}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}}$ and $\frac{F_{u}^{*}}{d_{1}}\in (\lambda _{k^{*}},\lambda _{k^{*}+1})$ for some nonnegative integer $k^{*}$. Then there is a constant $d_{4}'>0$ such that for $d_{4}'< d_{4}$,

  $$ \operatorname{index}_{\mathcal{Q}}\bigl(CM,\bigl(u^{*},v^{*} \bigr)\bigr)= \textstyle\begin{cases} -1, & \sum_{k=0}^{k^{*}}\mathcal{M}_{k} \textit{ is odd}, \\ 1, & \sum_{k=0}^{k^{*}}\mathcal{M}_{k} \textit{ is even}. \end{cases} $$

  In addition, when $\sum_{k=0}^{k^{*}}\mathcal{M}_{k}$ is odd, system (3.1) possesses at least one nonconstant interior solution.

Proof

By applying the elliptic PDE theory and the eigenfunction theory, we prove that $\operatorname{index}_{\mathcal{Q}}(CM,(u^{*},v^{*}))=1$.

(1) First, note that $\overline{\mathcal{Q}}_{(u^{*},v^{*})}=\mathcal{W}_{(u^{*},v^{*})}= \mathcal{A}$ and

$$C{M}^{\prime }(u^{\ast },v^{\ast })={(-\mathcal{D}\mathrm{\Delta }+cI)}^{-1}\left(\begin{array}{cc}c+F_u^{\ast }& F_v^{\ast }\\ G_u^{\ast }& c+G_v^{\ast }\end{array}\right).$$

Set $CM'(u^{*},v^{*})(\Phi ,\Psi )^{T}=(\Phi ,\Psi )^{T}\in \overline{\mathcal{Q}}_{(u^{*},v^{*})}$. Using the corresponding eigenfunction formulas in Lemma 3.2 for Φ and Ψ, we get

$$\left(\begin{array}{cc}{d}_1{\lambda }_k-F_u^{\ast }& -d_2{\lambda }_k-F_v^{\ast }\\ d_3{\lambda }_k-G_u^{\ast }& d_4{\lambda }_k-G_v^{\ast }\end{array}\right)\left(\begin{array}{c}{\varphi }_{kj}\\ {\psi }_{kj}\end{array}\right)=0.$$

Hence $\operatorname{Det}(\lambda _{k})=(d_{1}d_{4}+d_{2}d_{3})\lambda _{k}^{2} -[d_{1}G_{v}^{*}-d_{2}G_{u}^{*}+d_{3}F_{v}^{*}+d_{4}F_{u}^{*}] \lambda _{k} +F_{u}^{*}G_{v}^{*}-F_{v}^{*}G_{u}^{*}>0$ since $\operatorname{Det}(\lambda )>0$. This implies that $I-CM'(u^{*},v^{*})$ is an invertible matrix on $\overline{\mathcal{Q}}_{(u^{*},v^{*})}$. Therefore $CM'(u^{*},v^{*})$ does not possess property γ on $\overline{\mathcal{Q}}_{(u^{*},v^{*})}$.

Now we offer a rigorous proof to guarantee that $\delta =0$. Here δ is described in Lemma 4.1 of [13]. For $\rho >0$, we study the eigenvalue issue $(CM'(u^{*},v^{*})-I)(\Phi ,\Psi )^{T}=\rho (\Phi ,\Psi )^{T}$, $(\Phi ,\Psi )\neq (0,0)$, which means that for some positive constant ρ, the model

$$ \textstyle\begin{cases} (\rho +1)(-d_{1}\Delta \Phi -d_{2}\Delta \Psi )=(F_{u}^{*}- \rho c)\Phi +F_{v}^{*}\Psi & \mbox{in } \Omega , \\ (\rho +1)(d_{3}\Delta \Phi -d_{4}\Delta \Psi )=G_{u}^{*}\Phi +(G_{v}^{*}- \rho c)\Psi & \mbox{in } \Omega , \\ \frac{\partial \Phi }{\partial \nu }= \frac{\partial \Psi }{\partial \nu }=0 & \mbox{on } \partial \Omega , \\ \Phi \Psi \neq 0 & \mbox{in } \Omega , \end{cases} $$

(3.5)

has a nontrivial solution if and only if ρ satisfies the quadratic equation $\operatorname{Det}(Q(\rho ,\lambda _{k}))=0$, where

$$\begin{array}{rl}Det(Q(\rho ,{\lambda }_k)):=& det\left(\begin{array}{cc}(\rho +1)d_1{\lambda }_k+\rho c-F_u^{\ast }& (\rho +1)d_2{\lambda }_k-F_v^{\ast }\\ -(\rho +1)d_3{\lambda }_k-G_u^{\ast }& (\rho +1)d_4{\lambda }_k+\rho c-G_v^{\ast }\end{array}\right)\\ =& [(d_1{\lambda }_k+c)(d_4{\lambda }_k+c)+d_2{d}_3{\lambda }_k^2]{\rho }^2+2(d_1{d}_4+d_2{d}_3){\lambda }_k^2\rho \\ & +[(-d_1{G}_v^{\ast }-d_2{G}_u^{\ast }+d_3{F}_v^{\ast }-d_4{F}_u^{\ast }){\lambda }_k+(d_1{\lambda }_k-F_u^{\ast })c\\ & +(d_4{\lambda }_k-G_v^{\ast })c]\rho +Det({\lambda }_k).\end{array}$$

For $\lambda _{k}>0$, we obtain $\operatorname{Det}(\lambda _{k})>0$. This implies that $\operatorname{Det}(Q(\rho ,\lambda _{k}))=0$ may have either precisely a multiplicity-two simple interior solution, or two multiplicity-one interior solutions, or no interior solution. Thus we can conclude that for $k\geq 0$, the total algebraic multiplicity for $\operatorname{Det}(Q(\rho ,\lambda _{k}))=0$ equals zero or two, which means that $\delta =0$. Finally, we obtain that $\operatorname{index}_{\mathcal{Q}}(CM,(u^{*},v^{*}))=1$.

(2) In view of the assumption introduced by (2), we can observe that for a nonnegative integer k, $\operatorname{Det}(\lambda _{k})\neq 0$. Clearly, $I-CM'(u^{*},v^{*})$ is an invertible matrix on $\overline{\mathcal{Q}}_{(u^{*},v^{*})}$. Meanwhile, we can conclude that $CM'(u^{*},v^{*})$ cannot possess property γ on $\overline{\mathcal{Q}}_{(u^{*},v^{*})}$. Next, we focus on the total algebraic multiplicity δ for each eigenvalue of $CM'(u^{*},v^{*})-I$, which is greater than zero. When $\lambda _{0}=0$ ($k=0$), we get $\operatorname{Det}(Q(\rho ,0))=c^{2}\rho ^{2}-c(F_{u}^{*}+G_{v}^{*})\rho +F_{u}^{*}G_{v}^{*}-F_{v}^{*}G_{u}^{*}=0$. By the hypothesis of (2) we get $F_{u}^{*}G_{v}^{*}-F_{v}^{*}G_{u}^{*}\leq 0$. Obviously, $\operatorname{Det}(Q(\rho ,0))=0$ has exactly a simple interior solution. Together with $1\leq k\leq k^{*}$ and the hypothesis in (2), wee get $\operatorname{Det}(\lambda _{k})<0$, and hence $\operatorname{Det}(Q(\rho ,\lambda _{k}))=0$ possesses precisely one positive simple solution. If $k\geq k^{*}+1$, then $\operatorname{Det}(\lambda _{k})>0$, and hence $\operatorname{Det}(Q(\rho ,\lambda _{k}))=0$ may have either precisely one multiplicity-two simple interior solution, or two multiplicity-one interior solutions, or no interior solution. This discussion yields that $\sum_{k=0}^{k^{*}}\mathcal{M}_{k}+t=\delta$, where t is an even number or 0. In addition, suppose on the contrary that model (3.1) possesses no nonconstant interior solution. By Lemma 3.2, Lemma 3.3, and the last discussion we get

$$ \begin{aligned} 1&=\operatorname{index}_{\mathcal{Q}}\bigl(CM, \operatorname{Int} \mathcal{D}'\bigr)=\operatorname{index}_{\mathcal{Q}} \bigl(CM,(a,0)\bigr)+\operatorname{index}_{\mathcal{Q}}\bigl(CM, \bigl(u^{*},v^{*}\bigr)\bigr) \\ &=0+(-1)=-1. \end{aligned} $$

This contradiction tells us that system (3.1) possesses at least one nonconstant interior solution. The above argument derives the ideal outcome.

(3) The proof of this part is similar to that of part (2), and we omit it.

(4) Due to $a< \frac{b\gamma u^{*}[1+(2\alpha +b\beta \gamma )u^{*}]}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}}$, we get $F_{u}^{*}>0$. Let $\operatorname{Det}(\lambda )=0$, which means that

$$ \operatorname{Det}(\lambda )=(d_{1}d_{4}+d_{2}d_{3}) \lambda ^{2}- \Theta \lambda +F_{u}^{*}G_{v}^{*}-F_{v}^{*}G_{u}^{*}=0, $$

(3.6)

where

$$ \Theta =d_{1}G_{v}^{*}-d_{2}G_{u}^{*}+d_{3}F_{v}^{*}+d_{4}F_{u}^{*}. $$

(3.7)

By direct calculation we get that

$$\begin{aligned}& \lim_{d_{4}\rightarrow \infty }\lambda _{+}^{*}= \lim_{d_{1}\rightarrow \infty } \frac{\Theta +\sqrt{\Theta ^{2}-4(d_{1}d_{4}+d_{2}d_{3})(F_{u}^{*}G_{v}^{*}-F_{v}^{*}G_{u}^{*})}}{2(d_{1}d_{4}+d_{2}d_{3})}= \frac{F_{u}^{*}}{d_{1}}, \\& \lim_{d_{4}\rightarrow \infty }\lambda _{-}^{*}=\lim _{d_{1}\rightarrow \infty } \frac{\Theta -\sqrt{\Theta ^{2}-4(d_{1}d_{4}+d_{2}d_{3})(F_{u}^{*}G_{v}^{*}-F_{v}^{*}G_{u}^{*})}}{2(d_{1}d_{4}+d_{2}d_{3})}=0. \end{aligned}$$

Hence we can find a constant $d_{4}'>0$ guaranteeing $\lambda _{+}^{*}>\lambda _{k^{*}}$ and $\lambda _{-}^{*}<\lambda _{1}$ if $d_{4}'< d_{4}$ due to $\frac{F_{u}^{*}}{d_{1}}\in (\lambda _{k^{*}},\lambda _{k^{*}+1})$. Therefore, for a nonnegative integer k, $\operatorname{Det}(\lambda _{k})\neq 0$. We can observe that $I-CM'(u^{*},v^{*})$ is an invertible matrix on $\overline{\mathcal{Q}}_{(u^{*},v^{*})}$. Meanwhile, we can conclude that $CM'(u^{*},v^{*})$ cannot possess property γ on $\overline{\mathcal{Q}}_{(u^{*},v^{*})}$. On the other hand, note that $\operatorname{Det}(\lambda _{1})<0$ when

$$ d_{4}>\sigma _{1}:= \frac{d_{2}d_{3}\lambda _{1}^{2}+(-d_{1}G_{v}^{*} +d_{2}G_{u}^{*}-d_{3}F_{v}^{*})\lambda _{1}F_{u}^{*}G_{v}^{*} -F_{v}^{*}G_{u}^{*}}{F_{u}^{*}\lambda _{1}-d_{1}\lambda _{1}^{2}} $$

and $\operatorname{Det}(\lambda _{k^{*}})<0$ when

$$ d_{4}>\sigma _{2}:= \frac{d_{2}d_{3}\lambda _{k^{*}}^{2}+(-d_{1}G_{v}^{*}+d_{2}G_{u}^{*} -d_{3}F_{v}^{*})\lambda _{k^{*}}F_{u}^{*}G_{v}^{*}-F_{v}^{*}G_{u}^{*}}{F_{u}^{*}\lambda _{k^{*}}-d_{1}\lambda _{k^{*}}^{2}}. $$

We can take a constant $d_{4}'>\max \{\sigma _{1},\sigma _{2}\}$. We easily verify that for each $1\leq k\leq k^{*}$, $\operatorname{Det}(Q(\rho ,\lambda _{k}))=0$ has exactly one simple interior solution due to $\operatorname{Det}(\lambda _{k})<0$. Meanwhile, together with $k\geq 1+k^{*}$ and $\operatorname{Det}(\lambda _{k})>0$, $\operatorname{Det}(Q(\rho ,\lambda _{k}))=0$ has either precisely one multiplicity-two simple interior solution, or two multiplicity-one interior solutions, or no interior solution. According to the above discussion, we derive that $\sum_{k=0}^{k^{*}}\mathcal{M}_{k}+0=\delta$. In addition, suppose on the contrary that model (3.1) possesses no nonconstant interior solution. By Lemmas 3.2 and 3.3 and the last argument we get

$$ \begin{aligned} 1&=\operatorname{index}_{\mathcal{Q}}\bigl(CM, \operatorname{Int} \mathcal{D}'\bigr)=\operatorname{index}_{\mathcal{Q}} \bigl(CM,(a,0)\bigr)+\operatorname{index}_{\mathcal{Q}}\bigl(CM, \bigl(u^{*},v^{*}\bigr)\bigr) \\ &=0+(-1)=-1. \end{aligned} $$

This contradiction tells us that system (3.1) possesses at least one nonconstant interior solution. The above argument derives the ideal outcome. This ends the proof of Theorem 3.1. □

Example 1

Choosing $a=2$, $b=2$, $\alpha =1$, $\beta =1$, and $\gamma =2$, we obtain $(u^{*},v^{*})= (\frac{5+\sqrt{65}}{10}, \frac{10+2\sqrt{65}}{5} )\approx (1.3062,5.2249)$. Thus $\frac{b\gamma u^{*}[1+(2\alpha +b\beta \gamma )u^{*}]}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}} \approx 0.8141< a+b=4$ and $a=2> \frac{b\gamma u^{*}(\alpha +b\beta \gamma )u^{*}}{[1+(\alpha +b\beta \gamma )u^{*}]^{2}} \approx 0.6017$. Therefore system (1.1) possesses at least one nonconstant interior solution.

4 Conclusion

In this paper, we introduce a complicated predator–prey linear cross-diffusion system with the Beddington–DeAngelis and Tanner functional response and applied an effective rigorous approach to obtain the existence of nonconstant positive steady state of the corresponding system. By considering $\operatorname{Det}(\lambda )$ in three situations we can easily compute $\operatorname{index}\mathcal{Q}(CM,(u^{*},v^{*}))$. We proposed some parameter conditions to guarantee the local stability of the unique interior equilibrium.

It is worth noting that our methods in this paper can be applied to investigation of a class of nonlinear prey-taxis model with more general functional responses. By utilizing the fixed point index theory to get the existence of nonconstant interior steady states it is necessary to provide an a priori estimate and regularity of steady states, which plays an important role (see [15]).