Turing instability in a modified cross-diffusion Leslie–Gower predator–prey model with Beddington–DeAngelis functional response

Abstract

In this paper, a modified cross-diffusion Leslie–Gower predator–prey model with the Beddington–DeAngelis functional response is studied. We use the linear stability analysis on constant steady states to obtain sufficient conditions for the occurrence of Turing instability and Hopf bifurcation. We show that the Turing instability and associated patterns are induced by the variation of parameters in the cross-diffusion term. Some numerical simulations are given to illustrate our results.

1 Introduction

In this paper, we consider the modified Leslie–Gower predator–prey model

$$\begin{aligned} \textstyle\begin{cases} u_{t} - \Delta ((1+\alpha v)u) = r u (1-\frac{u}{k} ) - \frac{a u v}{m+bu+cv}, & (x,t)\in \Omega \times (0,\infty ), \\ v_{t} - \Delta ((\mu + \frac{1}{1+\beta u})v) = d v (1- \frac{ev}{m+bu}), & (x,t)\in \Omega \times (0,\infty ), \\ u(x,0) = u_{0} (x)\geq 0, \qquad v(x,0)=v_{0} (x)\geq 0,& x \in \Omega, \\ \partial _{\nu } u=\partial _{\nu } v=0, & (x,t)\in \partial \Omega \times (0,\infty ), \end{cases}\displaystyle \end{aligned}$$

(1.1)

where Ω is a bounded domain in $\mathbb{R}^{N}$ with smooth boundary ∂Ω, u and v represent the densities of the prey and the predator, Δ is the Laplacian operator, and ν is the outward unit normal vector on ∂Ω. Also r is the growth rate of the prey in the absence of the predator, k is the carrying capacity of the prey in the absence of the predator population, d is the intrinsic growth rate of the predator species, e is the maximum rate of death of the predator population, and $\mu > 0$ is the linear diffusion coefficient. The nonlinear cross diffusion term $\alpha \Delta (uv)$ describes a tendency such that the prey species keep away from high-density areas of the predator species and $\Delta (\frac{v}{1+\beta u})$ models a situation in which the population pressure of the predator species weakens in the high density place of the prey species. This model is called the modified Leslie–Gower predator–prey model. For more information about the Leslie–Gower predator-prey model, we refer the reader to [9–11]. The term

$$\begin{aligned} p(u,v) = \frac{au}{m+bu+cv} \end{aligned}$$

is called the Beddington–DeAngelis functional response. Here $a, b, c$, and m are the consumption rate, the saturation constant for an alternative prey, the predator interference, and the coefficient of environmental protection for the prey, respectively. To find more details on the background of this functional response, see [1, 4]. In this paper we assume that all constants $a, b, c, d, e, k, m, r, \alpha$, and β are positive.

As far as we know, there are few contributions on the cross-diffusion prey–predator model with the nonlinear cross diffusion term $\Delta (\frac{v}{1+\beta u})$. For example, in [3] the authors considered the following predator–prey model:

$$\begin{aligned} \textstyle\begin{cases} \frac{\partial u_{1}}{\partial t} = \Delta ((1+\alpha u_{2})u_{1}) - u_{1} (a - u_{1} - \frac{c u_{2}}{1 + mu_{1}} ), & x \in \Omega, t>0, \\ \frac{\partial u_{2}}{\partial t} = \Delta ((\mu + \frac{1}{1+\beta u_{1}})u_{2}) + u_{2} (b - u_{2} + \frac{du_{1}}{1 + mu_{1}}), & x \in \Omega, t>0, \\ u_{1}(x,0) = u_{2}(x,0)\geq 0, & x \in \Omega, \\ \frac{\partial u_{1}}{\partial t} = \frac{\partial u_{1}}{\partial t} = 0, & x\in \partial \Omega, t>0. \end{cases}\displaystyle \end{aligned}$$

(1.2)

They investigated the existence of the steady-state bifurcation and the Turing instability using the Lyapunov–Schmidt reduction. In [22], the authors proved the existence and multiplicity of positive steady state solutions for system (1.2). They also established the uniqueness of positive solution in a special domain. Yan and Guo in [20] discussed the direction and stability of the Hopf bifurcation for a Lotka–Volterra model with delay and cross-diffusion by the Lyapunov–Schmidt reduction method. Further, the effect of delay on the stability and Turing instability of equilibrium points for the Crowley–Martin predator-prey model was studied in [6]. In [12], the existence of a positive periodic solution for a nonautonomous Leslie–Gower reaction-diffusion model has been proved. The authors used the comparison theory of differential equations to get their results. Surendar et al. [17] studied the local stability of positive equilibrium and the existence of a Hopf bifurcation for a diffusive Holling–Tanner predator-prey system with the stoichiometric density dependence. The authors in [2] derived the Hopf bifurcation and its properties for a class of system of reaction-diffusion equations with two discrete time delays. In [21], Yi et al. studied the Turing instability and the Hopf bifurcation for the Lengyel–Epstein system. Many research works have been devoted to the Hopf bifurcation for ODE prey-predator models. For instance, Rihan et al. studied the stability of the Hopf bifurcation for a fractional-order delay prey-predator system in [15]. Sivasamy et al. in [16] analyzed the direction and stability of a Hopf bifurcation for a modified Leslie–Gower prey-predator model with gestation delay. In [13] the authors discussed the local and global stability of the interior equilibrium point and the Hopf bifurcation for a prey-predator model with Hassell–Varley functional response.

The main purpose of this paper is to investigate the effect of the cross-diffusion term on the stability and pattern formation in system (1.1). The existence of a periodic solution in the instability region and the Hopf bifurcation for the ODE and PDE models are also studied. We discuss the stability and direction of periodic solutions for the ODE system by the Poincare–Andronov–Hopf bifurcation theorem. For the PDE system, we use the center manifold theorem and the normal form theory. Our numerical results show that a Turing pattern is emerged by the variation of cross-diffusion parameters.

The organization of the rest of the paper is as follows: In Sect. 2, we analyze the asymptotic behavior of stationary solutions and the Hopf bifurcation for the ODE system. In Sect. 3, the Turing instability, the stability of bifurcating periodic solutions, and the bifurcation direction are investigated for system (1.1). Some numerical simulations are presented in Sect. 4.

2 Stability analysis and Hopf bifurcation for the ODE system

The ODE system corresponding to system (1.1) is as follows:

$$\begin{aligned} \textstyle\begin{cases} \frac{du}{dt} = r u (1-\frac{u}{k} ) - \frac{a u v}{m+bu+cv}, & t>0, \\ \frac{dv}{dt} = d v (1 - \frac{ev}{m+bu}), & t>0. \end{cases}\displaystyle \end{aligned}$$

(2.1)

We consider d as the bifurcation parameter. System (2.1) has the following equilibrium points:

$$\begin{aligned} &(0,0),\qquad (k,0),\qquad \biggl(0, \frac{m}{e}\biggr),\\ & U_{*}:=(u_{*},v_{*})=\biggl(k\biggl(1- \frac{a}{r(e+c)}\biggr), \frac{(m+bk)r(e+c)-abk}{er(e+c)}\biggr). \end{aligned}$$

We can rewrite $v_{*}$ as

$$\begin{aligned} v_{*}=\frac{m}{e}+\frac{bk}{e} \biggl(1- \frac{a}{r(e+c)} \biggr). \end{aligned}$$

Hence a sufficient condition for the positivity of $u_{*}$ and $v_{*}$ is

$$\begin{aligned} \frac{a}{ r(e+c)} < 1. \end{aligned}$$

(2.2)

From now on we assume that (2.2) is satisfied. The Jacobian matrix of system (2.1) at $U_{*}$ is

$$\begin{aligned} J (U_{*} ):= \begin{pmatrix} r (1- \frac{2 u_{*}}{k} ) - \frac{a v_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{2}} & - \frac{au_{*}(m+bu_{*})}{(m+bu_{*}+cv_{*})^{2}} \\ \frac{d b ev_{*}^{2}}{(m+bu_{*})^{2}} & d - \frac{2e d v_{*}}{m+bu_{*}} \end{pmatrix} , \end{aligned}$$

(2.3)

and the characteristic equation of J is given by

$$\begin{aligned} \lambda ^{2} - T_{0} \lambda + D_{0} =0, \end{aligned}$$

(2.4)

where $T_{0}=\operatorname{trac}(J(U_{*}))$ and $D_{0}=\det (J(U_{*}))$. Since $(u_{*},v_{*})$ is an equilibrium of system (2.1), we can simplify $T_{0}$ and $D_{0}$ as follows:

$$\begin{aligned} T_{0} &= r \biggl(1- \frac{2 u_{*}}{k} \biggr) - \frac{a v_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{2}} - d \\ &=-\frac{ru_{*}}{k} + \frac{bu_{*} r^{2}(k-u_{*})^{2}}{ak^{2}v_{*}} - d, \\ D_{0} &= \frac{d r u_{*}}{k}. \end{aligned}$$

Then the roots of (2.4) are $\lambda = \alpha _{0}(d) \pm i\beta _{0}(d)$ where

$$\begin{aligned} &\alpha _{0} (d ) = \frac{T_{0}}{2}, \\ &\beta _{0} (d )=\frac{\sqrt{T_{0}^{2}-4D_{0}}}{2}. \end{aligned}$$

The root of the equation $T_{0}(d) = 0$ is

$$\begin{aligned} d _{0}:= \frac{-r(re+rc-a)(r(m+bk)(e+c)^{2}-abk(2e+c))}{(e+c)^{2}((m+bk)r(e+c)-abk)}. \end{aligned}$$

Under the condition

$$\begin{aligned} kab (2e+c) > r(m+bk) (e+c)^{2}, \end{aligned}$$

(2.5)

$d_{0}$ is positive. Therefore, by the fact that $D_{0} > 0$, we conclude that $J(U_{*})$ has a pair of imaginary eigenvalues at $d = d_{0}$ as

$$\begin{aligned} \beta _{0} = \pm i \sqrt{\frac{d_{0} ru_{*}}{k}}. \end{aligned}$$

Also,

$$\begin{aligned} \alpha _{0}^{\prime }(d_{0}) = - \frac{1}{2} < 0. \end{aligned}$$

(2.6)

Then, by the Hopf theorem [8], system (2.1) has a Hopf bifurcation at $U_{*}$ when $d = d_{0}$. Furthermore, $T_{0}=d_{0}-d$, then $T_{0} > 0$ for $0 < d < d_{0}$, which implies that the characteristic equation (2.4) has a root with positive real part. So the equilibrium solution $U_{*}$ is unstable. Also $T_{0} < 0$ for $d_{0}< d$, therefore both roots of (2.4) have negative real parts. Hence, the equilibrium solution $U_{*}$ is locally asymptotically stable.

In the sequel, we detail the direction of Hopf bifurcation and the stability of bifurcating periodic solutions for system (2.1). First, we translate $U_{*}$ to the origin. After this translation, we obtain the following system:

$$\begin{aligned} \begin{pmatrix} \frac{du}{dt} \\ \frac{dv}{dt} \end{pmatrix} = J \dbinom{u}{v}+ \begin{pmatrix} f_{1}(u,v,d ) \\ f_{2}(u,v,d ) \end{pmatrix} , \end{aligned}$$

(2.7)

where

$$\begin{aligned} f_{1}(u,v,d) ={}& \frac{1}{2}\biggl(\frac{-2r}{k}+ \frac{2abv_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{3}}\biggr)u^{2} + \frac{1}{2}\biggl( \frac{2acu_{*}(m+bu_{*})}{(m+bu_{*}+cv_{*})^{3}}\biggr) v^{2} \\ & {}+ \biggl( \frac{-am(m+bu_{*}+cv_{*})-2abcu_{*}v_{*}}{(m+bu_{*}+cv_{*})^{3}}\biggr) uv \\ &{} + \frac{1}{2}\biggl( \frac{2abm(m+bu_{*}+cv_{*}) - 2abcv_{*}(m+cv_{*}) + 4ab^{2}cu_{*}v_{*}}{(m+bu_{*}+cv_{*})^{4}}\biggr) u^{2} v \\ &{}+ \frac{1}{2}\biggl( \frac{2acm(m+bu_{*}+cv_{*}) - 2abcu_{*}(m+bu_{*}) + 4abc^{2}u_{*}v_{*}}{(m+bu_{*}+cv_{*})^{4}}\biggr) v^{2} u \\ &{}+ \frac{1}{6}\biggl( \frac{-6ab^{2}v_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggr) u^{3} + \frac{1}{6}\biggl(\frac{-6ac^{2}u_{*}(m+bu_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggr) v^{3} \\ & {}+ O\bigl( \vert U \vert ^{\varrho }\bigr), \end{aligned}$$

(2.8)

$$\begin{aligned} f_{2}(u,v,d) ={}& \frac{1}{2}\biggl(\frac{-2d b^{2}}{e^{2}v_{*}}\biggr) u^{2} + \frac{1}{2}\biggl(\frac{-2 d}{v_{*}}\biggr) v^{2} + \biggl(\frac{2d b }{ev_{*}}\biggr) uv + \frac{1}{2}\biggl( \frac{-4d b^{2}}{e^{2}v_{*}^{2}}\biggr) u^{2}v \\ &{}+ \frac{1}{2}\biggl(\frac{2d b}{ev_{*}^{2}}\biggr) v^{2}u + \frac{1}{6}\biggl( \frac{6d b^{3}}{e^{3}v_{*}^{2}}\biggr) u^{3} + O\bigl( \vert u \vert ^{4}, \vert u \vert ^{3} \vert v \vert , \vert u \vert ^{2} \vert v \vert ^{2} \bigr) , \end{aligned}$$

(2.9)

where ϱ is a multi-index with $|\varrho |=4$. Set

$$\begin{aligned} P:= \begin{pmatrix} \frac{e \beta _{0}}{d b} & \frac{e(d +\alpha _{0})}{d b} \\ 0 & 1 \end{pmatrix} . \end{aligned}$$

Consider the following transformation:

$$\begin{aligned} \begin{pmatrix} u \\ v \end{pmatrix} = P \dbinom{x}{y}. \end{aligned}$$

Then

$$\begin{aligned} & u = \frac{e \beta _{0}}{d b} x + \frac{e( d +\alpha _{0})}{d b} y =: Mx+Ny, \\ & v = y. \end{aligned}$$

So the Jordan canonical form of system (2.7) is as follows:

$$\begin{aligned} \begin{pmatrix} \frac{du}{dt} \\ \frac{dv}{dt} \end{pmatrix} = J_{0}(d ) \dbinom{x}{y} + \dbinom{F^{1}}{F^{2}}, \end{aligned}$$

(2.10)

where

$$\begin{aligned} J_{0}(d):= P^{-1} J(d) P = \begin{pmatrix} \operatorname{Re}(\lambda ) & -\operatorname{Im}(\lambda ) \\ \operatorname{Im}(\lambda ) & \operatorname{Re}(\lambda ) \end{pmatrix} \end{aligned}$$

and

$$\begin{aligned} F^{1}(x,y,d ) ={}& \frac{d b}{e \beta _{0}}\biggl[ \frac{1}{2}\biggl( \frac{-2r}{k}+\frac{2abv_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{3}}\biggr) \bigl(M^{2}x^{2}+2MNxy+N^{2}y^{2} \bigr) \\ &{}+ \frac{1}{2}\biggl(\frac{2acu_{*}(m+bu_{*})}{(m+bu_{*}+cv_{*})^{3}}\biggr) y^{2} \\ &{}+ \biggl( \frac{-am(m+bu_{*}+cv_{*})-2abcu_{*}v_{*}}{(m+bu_{*}+cv_{*})^{3}}\biggr) \bigl(Mxy+Ny^{2}\bigr) \\ &{} + \frac{1}{2}\biggl( \frac{2abm(m+bu_{*}+cv_{*}) - 2abcv_{*}(m+cv_{*}) + 4ab^{2}cu_{*}v_{*}}{(m+bu_{*}+cv_{*})^{4}}\biggr) \\ &{}\times \bigl(M^{2}x^{2}y+2MNxy^{2}+N^{2}y^{3} \bigr) \\ &{}+ \frac{1}{2}\biggl( \frac{2acm(m+bu_{*}+cv_{*}) - 2abcu_{*}(m+bu_{*}) + 4abc^{2}u_{*}v_{*}}{(m+bu_{*}+cv_{*})^{4}}\biggr) \\ &{}\times \bigl(Mxy^{2}+Ny^{3}\bigr) \\ &{}+ \frac{1}{6}\biggl( \frac{-6ab^{2}v_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggr) \\ &{}\times \bigl(M^{3}x^{3}+3M^{2}Nx^{2}y+3MN^{2}xy^{2}+N^{3}y^{3} \bigr) \\ &{} + \frac{1}{6}\biggl( \frac{-6ac^{2}u_{*}(m+bu_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggr) y^{3} \biggr] \\ &{}- \frac{(d + \alpha _{0})}{\beta _{0}} \biggl[ \frac{1}{2}\biggl( \frac{-2d b^{2}}{e^{2}v_{*}} \biggr) \bigl(M^{2}x^{2}+2MNxy+N^{2}y^{2} \bigr) \\ &{} + \frac{1}{2}\biggl(\frac{-2 d}{v_{*}}\biggr) y^{2} + \biggl( \frac{2d b}{ev_{*}}\biggr) \bigl(Mxy+Ny^{2}\bigr) \\ &{}+ \frac{1}{2}\biggl(\frac{-4d b^{2}}{e^{2}v_{*}^{2}}\biggr) \bigl(M^{2}x^{2}y+2MNxy^{2}+N^{2}y^{3} \bigr) \\ &{} + \frac{1}{2}\biggl(\frac{2d b}{ev_{*}^{2}}\biggr) \bigl(Mxy^{2}+Ny^{3} \bigr) \\ &{}+ \frac{1}{6}\biggl(\frac{6d b^{3}}{e^{3}v_{*}^{2}}\biggr) \bigl(M^{3}x^{3}+3M^{2}Nx^{2}y+3MN^{2}xy^{2}+N^{3}y^{3} \bigr) \biggr], \\ F^{2}(x,y,d) ={}& \biggl(\frac{-d b^{2}}{e^{2}v_{*}}\biggr) \bigl(M^{2}x^{2}+2MNxy+N^{2}y^{2} \bigr) + \biggl(\frac{- d}{v_{*}}\biggr) y^{2} + \biggl( \frac{2d b}{ev_{*}}\biggr) \bigl(Mxy+Ny^{2}\bigr) \\ &{}+ \biggl(\frac{-2d b^{2}}{e^{2}v_{*}^{2}}\biggr) \bigl(M^{2}x^{2}y+2MNxy^{2}+N^{2}y^{3} \bigr) +\biggl(\frac{d b}{ev_{*}^{2}}\biggr) \bigl(Mxy^{2}+Ny^{3} \bigr) \\ & {}+\biggl(\frac{d b^{3}}{e^{3}v_{*}^{2}}\biggr) \bigl(M^{3}x^{3}+3M^{2}Nx^{2}y+3MN^{2}xy^{2}+N^{3}y^{3} \bigr). \end{aligned}$$

After transforming (2.10) into the normal form and rewriting it in the polar coordinates, we obtain the following equations:

$$\begin{aligned} \begin{aligned} &\dot{r} = \alpha _{0}(d) r + a(d) r^{3} + O\bigl(r^{5}\bigr), \\ &\dot{\theta } = \beta _{0}(d) + b(d) r^{2} + O \bigl(r^{2}\bigr). \end{aligned} \end{aligned}$$

(2.11)

For more details about the normal form in polar coordinates (2.11), see Sect. 19.2 of [19]. Using Taylor expansion of the coefficients in (2.11) about $d = d_{0}$, we get

$$\begin{aligned} &\dot{r} = \alpha _{0}^{\prime }(d_{0}) (d - d_{0}) r + a(d_{0}) r^{3} + O\bigl((d - d_{0})^{2} r, (d - d_{0}) r^{3}, r^{5}\bigr), \\ &\dot{\theta } = \beta _{0}(d_{0}) + \beta _{0}^{\prime }(d_{0}) (d - d_{0}) + b(d_{0}) r^{2} + O\bigl((d - d_{0})^{2}, (d - d_{0}) r^{2}, r^{4}\bigr). \end{aligned}$$

By Lemma 20.2.2 in [19], the sign of $a(d_{0})$ determines the stability of the periodic orbit. The coefficient $a(d_{0})$ is given by

$$\begin{aligned} a(d_{0} ) := {}&\frac{1}{16} \bigl(F^{1}_{xxx} + F^{1}_{xyy}+ F^{2}_{xxy} + F^{2}_{yyy} \bigr) (0,0,d_{0}) \\ &{} + \frac{1}{16\beta _{0}(d_{0})} \bigl(F^{1}_{xy}\bigl(F^{1}_{xx} +F^{1}_{yy}\bigr) - F^{2}_{xy} \bigl(F^{2}_{xx} + F^{2}_{yy}\bigr) - F^{1}_{xx}F^{2}_{xx} + F^{1}_{yy}F^{2}_{yy} \bigr) (0,0,d_{0}). \end{aligned}$$

By calculation, we have

$$\begin{aligned} &F^{1}_{xx}(0,0,d_{0} ) = \frac{e\beta _{0}}{d_{0} b } \biggl( \frac{-2r}{k}+\frac{2abv_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{3}}\biggr) + \frac{2\beta _{0}}{v_{*}}, \\ &F^{1}_{xy}(0,0,d_{0} ) = -\frac{2re}{kb} + \frac{am}{(m+bu_{*}+cv_{*})^{2}}, \\ &F^{1}_{yy}(0,0,d_{0} ) = -\frac{2red_{0}}{kb\beta _{0}}, \\ &F^{1}_{xxx}(0,0,d_{0} ) = - \frac{6av_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{4}} \biggl( \frac{e^{2}\beta _{0}^{2}}{d_{0}^{2}}\biggr) - \frac{6\beta _{0}^{2}}{d_{0} v_{*}^{2}}, \\ &F^{1}_{xyy}(0,0,d_{0} ) = - \frac{2am}{v_{*}(m+bu_{*}+cv_{*})^{2}}, \\ &F^{2}_{xy}(0,0,d_{0} ) = F^{2}_{yy}(0,0,d_{0} ) = F^{2}_{yyy}(0,0,d_{0} ) = 0, \\ &F^{2}_{xx}(0,0,d_{0} ) = -\frac{2\beta _{0}^{2}}{d_{0} v_{*}}, \\ &F^{2}_{xxy}(0,0,d_{0} ) = \frac{2\beta _{0}^{2}}{d_{0} v_{*}^{2}}. \end{aligned}$$

Then we get

$$\begin{aligned} a(d_{0}) = {}&\frac{1}{16}\biggl[- \frac{6av_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggl( \frac{e^{2}\beta _{0}^{2}}{d_{0}^{2}}\biggr) - \frac{2am}{(m+bu_{*}+cv_{*})^{2}}\biggl(\frac{re}{kbd_{0}} + \frac{red_{0}}{kb\beta _{0}^{2}}\biggr) \\ &{} - \frac{4av_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{3}} \biggl( \frac{erm}{kbd_{0}v_{*}}\biggr)+ \frac{4a^{2}mv_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{5}} \biggl( \frac{e}{d_{0}}\biggr) \\ &{}+ \frac{4r^{2}em}{k^{2}b^{2}d_{0}v_{*}} + \frac{4r^{2}emd_{0}}{k^{2}b^{2}\beta _{0}^{2}v_{*}}\biggr]. \end{aligned}$$

Set

$$\begin{aligned} X_{1} :={}& {-} \frac{2am}{(m+bu_{*}+cv_{*})^{2}}\biggl(\frac{re}{kbd_{0}} + \frac{red_{0}}{kb\beta _{0}^{2}}\biggr) + \frac{2a^{2}mv_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{5}}\biggl( \frac{e}{d_{0}}\biggr) \\ = {}&\frac{2kbme^{5}r^{3}a^{2}}{(-(bk+m)r(e+c)^{2}+abk(2e+c))(-(bk+m)r(e+c)+abk)^{3}}, \end{aligned}$$

(2.12)

$$\begin{aligned} X_{2}:={}&{ -}\frac{4av_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggl( \frac{e^{2}\beta _{0}^{2}}{d_{0}^{2}}\biggr) + \frac{4r^{2}em}{k^{2}b^{2}d_{0}v_{*}} + \frac{4r^{2}emd_{0}}{k^{2}b^{2}\beta _{0}^{2}v_{*}} \\ &{}- \frac{4av_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{3}} \biggl( \frac{erm}{kbd_{0}v_{*}}\biggr) \\ ={}& \frac{4kcbar^{2}e^{4}(a-r(e+c))}{(e+c)(-(bk+m)r(e+c)^{2}+abk(2e+c))(-(bk+m)r(e+c)+abk)^{2}}, \end{aligned}$$

(2.13)

$$\begin{aligned} X_{3}:={}&{ -}\frac{2av_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggl( \frac{e^{2}\beta _{0}^{2}}{d_{0}^{2}}\biggr) + \frac{2a^{2}mv_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{5}}\biggl( \frac{e}{d_{0}}\biggr). \end{aligned}$$

(2.14)

Then

$$\begin{aligned} a(d_{0}) = \frac{ X_{1}+X_{2}+X_{3}}{16}. \end{aligned}$$

If

$$\begin{aligned} (bk+m) \bigl(r(e+c)-a\bigr)^{2}-m a^{2} > 0, \end{aligned}$$

(2.15)

then by (2.2) and (2.5), $X_{1}$, $X_{2}$, and $X_{3}$ are negative. So $a(d_{0}) < 0$. By the above calculation and the Poincare–Andronov–Hopf bifurcation theorem [19], we obtain the following result.

Theorem 2.1

Assume that (2.2) and (2.5) are satisfied. Then

• (i) the equilibrium solution $U_{*}$ is unstable for (2.1) when $0 < d < d_{0}$ and asymptotically stable when $d_{0} < d$;

• (ii) system (2.1) has a Hopf bifurcation at $U_{*}$ when $d = d_{0}$. The direction of Hopf bifurcation is subcritical and the bifurcating periodic solutions are asymptotically orbitally stable when $a(d_{0}) < 0$, and the direction of Hopf bifurcation is supercritical and the bifurcating periodic solutions are unstable when $a(d_{0}) > 0$.

According to Theorem 2.1, system (2.1) has periodic solutions bifurcating from the Hopf bifurcation. In the next theorem, we show that system (2.1) has at least one stable periodic solution for $d < d_{0}$.

Theorem 2.2

Let $a < rc$ and (2.5) be satisfied. For $d < d_{0}$, system (2.1) has at least one stable periodic solution $U(t) = (u(t), v(t))$ such that $\varepsilon < u(t) < k$ and $\frac{m}{e} < v(t) < \frac{m+bk}{e}$ for

$$\begin{aligned} 0 < \varepsilon < \biggl(1-\frac{a}{rc}\biggr)k. \end{aligned}$$

(2.16)

Proof

We construct a trapping region satisfying the conditions of the Poincare–Bendixson theorem. To do this, let

$$\begin{aligned} &\mathcal{C}_{1}:= \biggl\lbrace (u,v): \varepsilon \leq u \leq k, v = \frac{m}{e} \biggr\rbrace , \\ &\mathcal{C}_{2}:= \biggl\lbrace (u,v): u = k, \frac{m}{e} \leq v \leq \frac{m+bk}{e} \biggr\rbrace , \\ &\mathcal{C}_{3}:= \biggl\lbrace (u,v): \varepsilon \leq u \leq k, v = \frac{m+bk}{e} \biggr\rbrace , \\ &\mathcal{C}_{4}:= \biggl\lbrace (u,v): u = \varepsilon, \frac{m}{e} \leq v \leq \frac{m+bk}{e} \biggr\rbrace . \end{aligned}$$

The segments $\mathcal{C}_{1}$, $\mathcal{C}_{2}$, $\mathcal{C}_{3}$, and $\mathcal{C}_{4}$ form a Jordan curve $\mathcal{C}$. Denote the interior of $\mathcal{C}$ by $\mathcal{D}$. One can see $U_{*} = (u_{*}, v_{*})$ is the only positive equilibrium point in $\mathcal{D}$. On the other hand, $\frac{dv}{dt}\vert _{(u,v)\in \mathcal{C}_{1}} > 0$, $\frac{du}{dt}\vert _{(u,v)\in \mathcal{C}_{2}} < 0$, and $\frac{dv}{dt}\vert _{(u,v)\in \mathcal{C}_{3}} < 0$. Also, by (2.16), for $(u,v) \in \mathcal{C}_{4}$, we have

$$\begin{aligned} 1-\frac{\varepsilon }{k} > \frac{a}{rc} > \frac{av}{rc(\frac{m}{c}+\frac{\varepsilon b}{c}+v)}. \end{aligned}$$

Hence $\frac{du}{dt}\vert _{(u,v)\in \mathcal{C}_{4}} > 0$. Therefore on the boundary of $\mathcal{D}$ the vector filed points inwards the set. This implies that a trajectory will stay in $\mathcal{D}$ for all t once it has entered the set. Since $U_{*} = (u_{*}, v_{*}) \in \mathcal{D}$ is unstable, by the Poincare–Bendixson theorem, system (2.1) has at least one stable periodic solution in $\mathcal{D}$. □

3 Turing instability of the PDE system

In this section, we obtain a condition under which the cross-diffusion system (1.1) loses stability and the stable positive equilibrium becomes Turing unstable.

Let $\lbrace \mu _{n} \rbrace _{n=0}^{\infty }$ be the eigenvalues of −△ in Ω with zero Neumann boundary conditions. The linearized system of (1.1) at $U_{*} = (u_{*},v_{*})$ is

$$\begin{aligned} \begin{pmatrix} u_{t} \\ v_{t} \end{pmatrix} = \bigl(D\Delta + J(U_{*})\bigr) \dbinom{u}{v}, \end{aligned}$$

where J is defined by (2.3) and

$$\begin{aligned} D:= \begin{pmatrix} (1+\alpha v_{*}) & \alpha u_{*} \\ -\frac{\beta v_{*}}{(1+\beta u_{*})^{2}} & (\mu + \frac{1}{1+\beta u_{*}}) \end{pmatrix} . \end{aligned}$$

(3.1)

Then λ is an eigenvalue of $L:= D\Delta + J(U_{*})$ on

$$\begin{aligned} X = \bigl\lbrace (u,v) \in \mathbb{H}^{2} (\Omega ) \times \mathbb{H}^{2} ( \Omega ): \partial _{\nu } u = \partial _{\nu } v =0 \bigr\rbrace , \end{aligned}$$

if and only if λ is an eigenvalue of the matrix $-\mu _{n} D + J(U_{*})$ for $n\geq 0$. Therefore λ satisfies the characteristic equation

$$\begin{aligned} \lambda ^{2} -T_{n}(d) \lambda +D_{n}(d) =0, \end{aligned}$$

(3.2)

where

$$\begin{aligned} T_{0}(d) ={}& {-}\frac{ru_{*}}{k} + \frac{bu_{*} r^{2}(k-u_{*})^{2}}{ak^{2}v_{*}} - d, \\ T_{n}(d ) ={}&{ -}\biggl[(1 + \alpha v_{*}) + \mu + \frac{1}{1+\beta u_{*}}\biggr] \mu _{n} + T_{0} \end{aligned}$$

(3.3)

$$\begin{aligned} ={}&{ -} \biggl(1 + \alpha v_{*} + \mu + \frac{1}{1+\beta u_{*}}\biggr)\mu _{n} + \biggl(- \frac{ru_{*}}{k} + \frac{bu_{*}r^{2}(k-u_{*})^{2}}{ak^{2}v_{*}} - d\biggr), \\ D_{0}(d) ={}& \frac{d r u_{*}}{k}, \\ D_{n}(d) ={}& \biggl[\biggl(\mu + \frac{1}{1+\beta u_{*}}\biggr) (1+\alpha v_{*})+ \frac{\alpha \beta u_{*}v_{*}}{(1+\beta u_{*})^{2}}\biggr]\mu _{n} ^{2} + \biggl[d(1+ \alpha v_{*}) \\ & {}- \biggl(\mu + \frac{1}{1+\beta u_{*}}\biggr) (T_{0} + d ) + \frac{\alpha d b u_{*}}{e} + \frac{e r^{2}\beta u_{*}(k-u_{*})^{2}}{ak^{2}(1+\beta u_{*})^{2}}\biggr] \mu _{n} + D_{0} \\ ={}& \biggl[\biggl(\mu + \frac{1}{1+\beta u_{*}}\biggr) (1+\alpha v_{*})+ \frac{\alpha \beta u_{*}v_{*}}{(1+\beta u_{*})^{2}}\biggr]\mu _{n} ^{2} + \biggl[ d(1+\alpha v_{*}) + \frac{e r^{2}\beta u_{*}(k-u_{*})^{2}}{ak^{2}(1+\beta u_{*})^{2}} \\ & {}- \biggl(\mu + \frac{1}{1+\beta u_{*}}\biggr) \biggl(-\frac{ru_{*}}{k} + \frac{bu_{*} r^{2}(k - u_{*})^{2}}{ak^{2}v_{*}}\biggr) + \frac{\alpha d b u_{*}}{e} \biggr] \mu _{n} + \frac{rdu_{*}}{k}. \end{aligned}$$

(3.4)

Let

$$\begin{aligned} M_{n} :={}& \biggl[\mu + \frac{1}{1+\beta u_{*}}\biggr]\mu _{n} ^{2} + \biggl[ d - \biggl(\mu + \frac{1}{1+\beta u_{*}}\biggr) \biggl(- \frac{ru_{*}}{k} + \frac{bu_{*} r^{2}(k-u_{*})^{2}}{ak^{2}v_{*}}\biggr) \\ &{}+\frac{e r^{2}\beta u_{*}(k-u_{*})^{2}}{ak^{2}(1+\beta u_{*})^{2}}\biggr] \mu _{n} + \frac{rdu_{*}}{k}, \\ N_{n} :={}& \biggl(\mu + \frac{1}{1+\beta u_{*}} + \frac{\beta u_{*}}{(1+\beta u_{*})^{2}} \biggr)\mu _{n} ^{2} + \biggl(d + \frac{ d b u_{*}}{ev_{*}}\biggr) \mu _{n}. \end{aligned}$$

According to (3.4),

$$\begin{aligned} D_{n}(d) = \alpha v_{*} N_{n} + M_{n}. \end{aligned}$$

In the next theorem, we summarized the asymptotic behavior of the equilibrium $U_{*}$ for system (1.1).

Theorem 3.1

Assume $d > d_{0}$ and let (2.2) and (2.5) be satisfied. Then

• (i) $U_{*}$ is unstable for system (1.1) if

  $$\begin{aligned} 0 < \alpha < \frac{-M_{1}}{v_{*}N_{1}}. \end{aligned}$$

  (3.5)

• (ii) $U_{*}$ is locally asymptotically stable for system (1.1) if

  • (1) $0 < \mu < \frac{\beta m - b}{b (1+\beta u_{*})^{2}}$,

    or

  • (2) $\alpha > \frac{-M_{1}}{v_{*}N_{1}} > 0$ and

    $$\begin{aligned} \mu _{1}^{2}\biggl(\mu + \frac{1}{1+\beta u_{*}} \biggr) + M_{1} - \frac{rdu_{*}}{k}>0. \end{aligned}$$

    (3.6)

• (iii) Under condition (1) in (ii) system (1.1) has a Hopf bifurcation at $U_{*}$ when $d=d_{0}$.

Proof

(i). According to (3.3), $T_{n}(d) > T_{n+1}(d)$ for $n \in \mathbb{N} \cup \lbrace 0\rbrace$. Since $T_{0}(d) < 0$ for $d > d_{0}$, we conclude that $T_{n}(d) < 0$ for $n \in \mathbb{N} \cup \lbrace 0\rbrace$. By (3.4) and condition (3.5), we have $D_{1}= \alpha v_{*} N_{1} +M_{1} < 0$. So equation (3.2) has at least one root with positive real part. Then $U_{*}$ is unstable for system (1.1).

(ii). Since $\frac{ev_{*}}{m+bu_{*}} = 1$, from (1) we conclude

$$\begin{aligned} \mathcal{A}&:=\mu + \frac{1}{1+\beta u_{*}} - \frac{\beta ev_{*}}{b(1+\beta u_{*})^{2}} = \mu - \frac{\beta (m+bu_{*})}{b(1+\beta u_{*})^{2}} + \frac{1}{1+\beta u_{*}} \\ & = \mu - \frac{\beta m-b}{b(1+\beta u_{*})^{2}} < 0. \end{aligned}$$

(3.7)

Furthermore, $D_{0}(d)>0$ for $d>0$, and we can rewrite $D_{n}(d)$ as

$$\begin{aligned} D_{n}(d) ={}& \alpha v_{*}\biggl[\biggl(\mu + \frac{1}{1+\beta u_{*}} + \frac{\beta u_{*}}{(1+\beta u_{*})^{2}}\biggr)\mu _{n} ^{2} + \biggl(d + \frac{bd u_{*}}{ev_{*}}\biggr)\mu _{n}\biggr] \\ &{}+ \biggl(\mu + \frac{1}{1+\beta u_{*}}\biggr)\mu _{n} ^{2} + \biggl[d + \frac{ru_{*}}{k} \biggl(\mu + \frac{1}{1+\beta u_{*}}\biggr) \\ &{} -\frac{bu_{*} r^{2}(k-u_{*})^{2}}{ak^{2}v_{*}}\mathcal{A}\biggr] \mu _{n} + \frac{d r u_{*}}{k}. \end{aligned}$$

(3.8)

By (3.7), we have $D_{n}(d) > 0$ for $n \in \mathbb{N}$ and $d>0$. Since $T_{n}(d) < 0$ for $n \in \mathbb{N} \cup \lbrace 0\rbrace$ when $d > d_{0}$, we deduce that the real part of every eigenvalue of L is negative. Therefore, $U_{*}$ is locally asymptotically stable for system (1.1).

Now, let $(2)$ be satisfied. Then $D_{1}= \alpha v_{*} N_{1} +M_{1} > 0$. Also, we can rewrite $M_{n}$ as

$$\begin{aligned} M_{n} &= \biggl(\mu + \frac{1}{1+\beta u_{*}}\biggr) \bigl(\mu _{n}^{2} - \mu _{n}\mu _{1}\bigr) + \frac{M_{1}\mu _{n}}{\mu _{1}} - \frac{rdu_{*}}{k}\biggl( \frac{\mu _{n}}{\mu _{1}} - 1\biggr) \\ & =\mu _{n} \biggl(\mu + \frac{1}{1+\beta u_{*}}\biggr) (\mu _{n} - \mu _{1}) - \frac{rdu_{*}}{k\mu _{1}}(\mu _{n} - \mu _{1}) + \frac{M_{1}\mu _{n}}{\mu _{1}}. \end{aligned}$$

Hence, for $n = 2, 3,\ldots$ , using condition (3.6) we have

$$\begin{aligned} M_{n}-M_{1}={}&\biggl(\mu + \frac{1}{1+\beta u_{*}}\biggr)\mu _{n} (\mu _{n} - \mu _{1}) + \frac{M_{1}(\mu _{n} - \mu _{1})}{\mu _{1}} - \frac{rdu_{*}}{k\mu _{1}}(\mu _{n} - \mu _{1}) \\ \geq{} &\frac{(\mu _{n} - \mu _{1})}{\mu _{1}} \biggl(\mu _{1}^{2}\biggl(\mu + \frac{1}{1+\beta u_{*}}\biggr) + M_{1} - \frac{rdu_{*}}{k} \biggr)>0. \end{aligned}$$

Since $N_{n+1}\geq N_{n}$ for $n \in \mathbb{N}$, the above inequalities imply $D_{n} (d)\geq D_{1}(d)> 0$ for $n \geq 2$ and $d>d_{0}$. Again using $T_{n}(d) < 0$ for $n \in \mathbb{N}$, we conclude that all roots of the characteristic equation (3.2) have negative real part for $n\geq 0$ when $d>d_{0}$. Therefore $U_{*}$ is locally asymptotically stable for system (1.1).

(iii) Under condition (1) in (ii), inequality (3.7) is stratified. Then from (3.8) we get $D_{n}(d_{0})>0$ for $n \in \mathbb{N}$. Also $D_{0}(d_{0})>0$. On the other hand, $T_{0}(d_{0})=0$, then by (3.3) we have $T_{n}(d_{0})<0$ for $n\in \mathbb{N}$. Therefore, all roots of the characteristic equation (3.2) have negative real part for $n\geq 1$ when $d=d_{0}$. Finally from (2.6) we conclude that system (1.1) has a Hopf bifurcation at $U_{*}$ when $d=d_{0}$. □

Now, we determine the direction and stability of the Hopf bifurcation for system (1.1) in $\Omega = (0, \mathit{l}\pi )$ for $\mathit{l} \in \mathbb{R}^{+}$, i.e., the following system:

$$\begin{aligned} \textstyle\begin{cases} u_{t} - ((1+\alpha v)u)_{xx} = r u (1-\frac{u}{k} ) - \frac{a u v}{m+bu+cv}, & x\in \Omega, t>0, \\ v_{t} - ((\mu + \frac{1}{1+\beta u})v)_{xx} = d v (1 - \frac{ev}{m+bu}), & x\in \Omega, t>0, \\ u(x,0) = u_{0} (x)\geq 0,\qquad v(x,0)=v_{0} (x)\geq 0, & x\in \Omega, \\ \partial _{\nu } u=\partial _{\nu } v=0, & x\in \partial \Omega, t>0. \end{cases}\displaystyle \end{aligned}$$

(3.9)

The eigenvalues and the corresponding eigenfunctions of the operator $u\longrightarrow - u_{xx}$ with zero Neumann boundary conditions on $(0,\mathit{l} \pi )$ are given by

$$\begin{aligned} \mu _{n} = \frac{n^{2}}{\mathit{l}^{2}}, \qquad\varphi _{n}(x) = \cos \biggl( \frac{nx}{\mathit{l}}\biggr),\quad \text{for }n = 0,1,2,\ldots. \end{aligned}$$

The linearized system of (3.9) at $U_{*}$ has the following form:

$$\begin{aligned} \begin{pmatrix} u_{t} \\ v_{t} \end{pmatrix} = L \dbinom{u}{v} = D \begin{pmatrix} u_{xx} \\ v_{xx} \end{pmatrix} + J \dbinom{u}{v}, \end{aligned}$$

(3.10)

where J and D are defined by (2.3) and (3.1) respectively and

$$\begin{aligned} L = \begin{pmatrix} - (1+\alpha v_{*})\mu _{n} + r (1- \frac{2 u_{*}}{k} ) - \frac{a v_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{2}} & -\alpha u_{*}\mu _{n} - \frac{au_{*}(m+bu_{*})}{(m+bu_{*}+cv_{*})^{2}} \\ \frac{\beta v_{*}\mu _{n}}{(1+\beta u_{*})^{2}} + \frac{bd}{e} & -( \mu + \frac{1}{1+\beta u_{*}})\mu _{n} - d \end{pmatrix} . \end{aligned}$$

(3.11)

Similarly, by transferring the equilibrium point $U_{*}$ to the origin, system (3.9) becomes

$$\begin{aligned} \begin{pmatrix} u_{t} \\ v_{t} \end{pmatrix} = D \begin{pmatrix} u_{xx} \\ v_{xx} \end{pmatrix} + J \dbinom{u}{v}+ \begin{pmatrix} f_{1}(u,v,d) \\ f_{2}(u,v,d) \end{pmatrix} , \end{aligned}$$

(3.12)

where $f_{1}$ and $f_{2}$ are defined by (2.8) and (2.9). Define the operator $L^{*}$ by

$$\begin{aligned} L^{*} \dbinom{u}{v}:= D \begin{pmatrix} u_{xx} \\ v_{xx} \end{pmatrix} + J^{*} \dbinom{u}{v}, \end{aligned}$$

(3.13)

where $J^{*}$ is the conjugate of J. Then $L^{*}$ is the adjoint operator of L. Set

$$\begin{aligned} q= \begin{pmatrix} \frac{e(d_{0}+i\beta _{0})}{bd_{0}} \\ 1 \end{pmatrix} ,\qquad q^{*} = \frac{1}{2\mathit{l}\pi } \begin{pmatrix} \frac{bd_{0}}{e\beta _{0}}i \\ - \frac{d_{0}}{\beta _{0}}i + 1 \end{pmatrix} , \end{aligned}$$

where $\beta _{0}(d_{0}) = \sqrt{\frac{ru_{*}d_{0}}{k}}$ and i is the imaginary unit. Then

$$\begin{aligned} L q =i\beta _{0} (d_{0}) q,\qquad L^{*} q^{*} = -i\beta _{0} (d_{0}) q^{*},\qquad \bigl\langle q^{*}, q \bigr\rangle =1,\qquad \bigl\langle q^{*}, \bar{q} \bigr\rangle = 0, \end{aligned}$$

where $\langle f, g \rangle = \int _{0}^{\mathit{l}\pi } \bar{f}^{T} g \,dx$ is the inner product in $\mathbb{L}^{2} (0,\mathit{l}\pi ) \times \mathbb{L}^{2} (0, \mathit{l}\pi )$. We decompose X as $X = X^{c} \oplus X^{s}$ where

$$\begin{aligned} X^{c} = \lbrace z q + \bar{z} \bar{q}: z \in \mathbb{C} \rbrace,\qquad X^{s} = \bigl\lbrace w \in X: \bigl\langle q^{*}, w \bigr\rangle = 0 \bigr\rbrace . \end{aligned}$$

According to [8], for every $U^{T} = (u,v) \in X$, we have

$$\begin{aligned} \dbinom{u}{v} = z q + \bar{z} \bar{q} + \dbinom{w_{1}}{w_{2}},\quad z= \bigl\langle q^{*}, (u,v)^{T} \bigr\rangle , \end{aligned}$$

where $z q + \bar{z} \bar{q} \in X^{c}$ and $w^{T} = (w_{1}, w_{2}) \in X^{s}$. Then

$$\begin{aligned} & z= \bigl\langle q^{*}, U \bigr\rangle , \\ & w = U - \bigl\langle q^{*}, U \bigr\rangle q - \bigl\langle \bar{q^{*} }, U \bigr\rangle \bar{q}. \end{aligned}$$

Thus system (3.12) in $(z, w)$ coordinates is as follows:

$$\begin{aligned} \textstyle\begin{cases} \frac{dz}{dt} = i \beta _{0} (d_{0}) z + \langle q^{*},f^{\vee } \rangle, \\ \frac{dw}{dt} = Lw+ H(z, \bar{z},w ), \end{cases}\displaystyle \end{aligned}$$

(3.14)

where $f^{\vee } = (f_{1},f_{2} )^{T}$ is defined by (2.8)–(2.9) and

$$\begin{aligned} H(z, \bar{z},w ):= f^{\vee } - \bigl\langle q^{*},f^{\vee } \bigr\rangle q- \bigl\langle \bar{q^{*}},f^{ \vee } \bigr\rangle \bar{q}. \end{aligned}$$

By calculation, we obtain

$$\begin{aligned} &\bigl\langle q^{*},f^{\vee } \bigr\rangle q= \begin{pmatrix} \frac{e(d_{0}+i\beta _{0})}{bd_{0}}(-\frac{bd_{0}i}{2e\beta _{0}}f_{1} +\frac{d_{0}i}{2\beta _{0}}f_{2} +\frac{f_{2}}{2}) \\ -\frac{bd_{0}i}{2e\beta _{0}}f_{1} +\frac{d_{0}i}{2\beta _{0}}f_{2} + \frac{f_{2}}{2} \end{pmatrix}, \\ &\bigl\langle \bar{q^{*}},f^{\vee } \bigr\rangle \bar{q} = \begin{pmatrix} \frac{e(d_{0}-i\beta _{0})}{bd_{0}}(\frac{bd_{0}i}{2e\beta _{0}}f_{1} -\frac{d_{0}i}{2\beta _{0}}f_{2} +\frac{f_{2}}{2}) \\ \frac{bd_{0}i}{2e\beta _{0}}f_{1} - \frac{d_{0}i}{2\beta _{0}}f_{2} + \frac{f_{2}}{2} \end{pmatrix}, \\ &\bigl\langle q^{*},f^{\vee } \bigr\rangle q - \bigl\langle \bar{q^{*}},f^{\vee } \bigr\rangle \bar{q} = \begin{pmatrix} f_{1} \\ f_{2} \end{pmatrix}. \end{aligned}$$

Then $H(z, \bar{z},w ) = 0$. Besides, the center manifold of system (3.14) is illustrated as follows:

$$\begin{aligned} w =\biggl(\frac{w_{20}}{2}\biggr) z^{2} + w_{11} z \bar{z} + \biggl(\frac{w_{02}}{2}\biggr) \bar{z} ^{2} + O \bigl( \vert z \vert ^{3} \bigr). \end{aligned}$$

Then we get

$$\begin{aligned} \textstyle\begin{cases} w_{20} =(2 i \beta _{0} (d_{0})I - L )^{-1} H_{20}, \\ w_{11} = (-L)^{-1}H_{11}, \\ w_{02} = (- 2 i \beta _{0} (d_{0}) I - L )^{-1} H_{02}. \end{cases}\displaystyle \end{aligned}$$

So $w_{20} =w_{02} =w_{11} =0$. Therefore, the restriction of (3.14) to the center manifold in z and z̄ coordinates is given by

$$\begin{aligned} \frac{dz}{dt} = i \beta _{0} (d_{0}) z + \bigl\langle q^{*},f^{\vee } \bigr\rangle = i \beta _{0} (d_{0})z + \frac{h_{20}}{2}z^{2} + h_{11}z \bar{z} + \frac{h_{02}}{2}\bar{z}^{2} + O \bigl( \vert z \vert ^{3} \bigr), \end{aligned}$$

(3.15)

where

$$\begin{aligned} & h_{20} = \bigl\langle q^{*}, B(q,q) \bigr\rangle \\ &\phantom{h_{20} }= \frac{re}{kbd_{0}\beta _{0}}\bigl(d_{0}^{2}i - 2\beta _{0}d_{0} - \beta _{0}^{2}i\bigr) + \frac{\beta _{0}d_{0}i + \beta _{0}^{2}}{v_{*}d_{0}} \\ &\phantom{h_{20} = }{} + \frac{am(m+bu_{*}+cv_{*}) + abcu_{*}v_{*}}{(m+bu_{*}+cv_{*})^{3}}\biggl( \frac{\beta _{0}i}{d_{0}}\biggr) \\ &\phantom{h_{20} = }{} + \frac{am}{(m+bu_{*}+cv_{*})^{2}}, \end{aligned}$$

(3.16)

$$\begin{aligned} &h_{11} = \bigl\langle q^{*},B(q,\bar{q}) \bigr\rangle \\ &\phantom{h_{11} } = \frac{re}{kbd_{0}\beta _{0}}\bigl(d_{0}^{2}i + \beta _{0}^{2}i\bigr) - \frac{\beta _{0}d_{0}i + \beta _{0}^{2}}{v_{*}d_{0}} \\ &\phantom{h_{11} =}{}+ \frac{am(m+bu_{*}+cv_{*}) + abcu_{*}v_{*}}{(m+bu_{*}+cv_{*})^{3}}\biggl(- \frac{\beta _{0}i}{d_{0}}\biggr), \end{aligned}$$

(3.17)

$$\begin{aligned} &h_{21} = \bigl\langle q^{*},C(q,q,\bar{q}) \bigr\rangle = \frac{3ab^{2}e^{2}v_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggl( \frac{d_{0}^{3}i -\beta _{0}^{3} - \beta _{0}d_{0}^{2} + \beta _{0}^{2}d_{0}i}{b^{2}d_{0}^{2}\beta _{0}}\biggr) \\ &\phantom{h_{21} = }{}+ \biggl( \frac{abm(m+bu_{*}+cv_{*})+2ab^{2}cu_{*}v_{*}-abcv_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggr) \end{aligned}$$

(3.18)

$$\begin{aligned} &\phantom{h_{21} = }{}\times \biggl( \frac{e(-3d_{0}^{2}i - \beta _{0}^{2}i + 2\beta _{0}d_{0})}{b\beta _{0}d_{0}}\biggr) \\ &\phantom{h_{21} = }{} + \biggl( \frac{acm(m+bu_{*}+cv_{*})+2abc^{2}u_{*}v_{*}-abcu_{*}(m+bu_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggr) \biggl( \frac{-3d_{0}i + \beta _{0}}{\beta _{0}}\biggr) \\ & \phantom{h_{21} = }{}+ \frac{3ac^{2}u_{*}(m+bu_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggl( \frac{bd_{0}i}{e\beta _{0}}\biggr)+ \frac{3\beta _{0}^{3}i + \beta _{0}^{2}d_{0}}{v_{*}^{2}d_{0}^{2}} \biggl(1+ \frac{d_{0}i}{\beta _{0}}\biggr), \\ &B(u,v) = \dbinom{B_{1}(u,v)}{B_{2}(u,v)},\qquad C(q,q,\bar{q}) = \dbinom{C_{1}(q,q,\bar{q})}{C_{2}(q,q,\bar{q})}, \\ &B_{1}(q,q) ={-} \frac{2re^{2}(d_{0} + i \beta _{0})^{2}}{kb^{2}d_{0}^{2}} + \frac{2av_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{3}}\biggl( \frac{e^{2}(i\beta _{0} d_{0} - \beta _{0}^{2})}{bd_{0}^{2}}\biggr) \\ &\phantom{B_{1}(q,q) =}{} - \frac{abcu_{*}v_{*}}{(m+bu_{*}+cv_{*})^{3}}\biggl( \frac{2e\beta _{0}i}{bd_{0}}\biggr), \\ &B_{2}(q,q) = \frac{2\beta _{0}^{2}}{v_{*}d_{0}}, \\ &B_{1}(q,\bar{q}) ={ -} \frac{2re^{2}(d_{0}^{2}+\beta _{0}^{2})}{kb^{2}d_{0}^{2}} + \frac{2av_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{3}}\biggl( \frac{e^{2}\beta _{0}^{2}}{bd_{0}^{2}}\biggr), \\ &B_{2}(q,\bar{q}) ={ -} \frac{2\beta _{0}^{2}}{v_{*}d_{0}}, \\ &C_{1}(q,q,\bar{q}) = {-} \frac{6ab^{2}e^{3}v_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggl( \frac{d_{0}^{3} +\beta _{0}^{3}i + \beta _{0}d_{0}^{2}i + \beta _{0}^{2}d_{0}}{b^{3}d_{0}^{3}}\biggr) \\ &\phantom{C_{1}(q,q,\bar{q}) = }{}- \frac{6ac^{2}u_{*}(m+bu_{*})}{(m+bu_{*}+cv_{*})^{4}} \\ &\phantom{C_{1}(q,q,\bar{q}) = } {}+\biggl( \frac{2abm(m+bu_{*}+cv_{*})+4ab^{2}cu_{*}v_{*}-2abcv_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggr) \\ &\phantom{C_{1}(q,q,\bar{q}) = }{}\times \biggl( \frac{e^{2}(3d_{0}^{2} + \beta _{0}^{2} + 2\beta _{0}d_{0}i)}{b^{2}d_{0}^{2}}\biggr) \\ & \phantom{C_{1}(q,q,\bar{q}) = }{}+ \biggl( \frac{2acm(m+bu_{*}+cv_{*})+4abc^{2}u_{*}v_{*} - 2abcu_{*}(m+bu_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggr) \\ &\phantom{C_{1}(q,q,\bar{q}) = }\times \biggl(\frac{e(3d_{0} + \beta _{0}i)}{bd_{0}}\biggr), \\ &C_{2}(q,q,\bar{q}) = \frac{6\beta _{0}^{3}i + 2\beta _{0}^{2}d_{0}}{v_{*}^{2}d_{0}^{2}}. \end{aligned}$$

(3.19)

According to [8],

$$\begin{aligned} \operatorname{Re} \bigl(c_{1}(0)\bigr) &= \operatorname{Re} \biggl( \frac{i}{2\beta _{0} (d_{0})}\biggl(h_{20}h_{11}-2 \vert h_{11} \vert ^{2} -\frac{1}{3} \vert h_{02} \vert ^{2} \biggr)+ \frac{h_{21}}{2}\biggr) \\ & = - \frac{1}{2 \beta _{0} (d_{0})}\operatorname{Im} (h_{20}h_{11}) + \frac{1}{2} \operatorname{Re} (h_{21}). \end{aligned}$$

Then from (3.16)–(3.18), we have

$$\begin{aligned} \operatorname{Re} c_{1}(0) = {}&\frac{1}{4}\biggl[- \frac{6av_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{4}}\biggl( \frac{e^{2}\beta _{0}^{2}}{d_{0}^{2}}\biggr) - \frac{2am}{(m+bu_{*}+cv_{*})^{2}}\biggl( \frac{re}{kbd_{0}} + \frac{red_{0}}{kb\beta _{0}^{2}}\biggr) \\ &{} - \frac{4av_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{3}} \biggl( \frac{erm}{kbd_{0}v_{*}}\biggr)+ \frac{4a^{2}mv_{*}(m+cv_{*})}{(m+bu_{*}+cv_{*})^{5}} \biggl( \frac{e}{d_{0}}\biggr) + \frac{4r^{2}em}{k^{2}b^{2}d_{0}v_{*}} \\ &{} + \frac{4r^{2}emd_{0}}{k^{2}b^{2}\beta _{0}^{2}v_{*}}\biggr]. \end{aligned}$$

Now we summarized the above results as follows.

Theorem 3.2

Assume that (2.2) and (2.5) are satisfied and let $d = d_{0}$. Then system (3.9) has a Hopf bifurcation at $U_{*}$. The direction of Hopf bifurcation is subcritical and the bifurcating periodic solutions are asymptotically orbitally stable when $\operatorname{Re} c_{1}(0) < 0$. The direction of Hopf bifurcation is supercritical and the bifurcating periodic solutions are unstable when $\operatorname{Re} c_{1}(0) > 0$.

4 Numerical simulations

In this section, we provide some numerical examples to illustrate our results. We use FlexPDE [5] to solve our PDE models. FlexPDE is a software for solving PDEs via finite elements methods.

Since the distributions of prey and predator are of the same type, we only give the numerical simulation of the prey.

Example 4.1

Consider the ODE model (2.1) with

$$\begin{aligned} \begin{aligned}& r=4,\qquad e=1.9,\qquad c=1.1,\qquad a=11,\\ & k=0.3,\qquad b=63,\qquad m=0.8. \end{aligned} \end{aligned}$$

(4.1)

Then the positive equilibrium and the critical value of bifurcation parameter are

$$\begin{aligned} (u_{*},v_{*}) = (0.025, 1.25),\qquad d_{0} = 1.2066. \end{aligned}$$

By calculation, we obtain $a(d_{0}) = -0.0153$. According to Theorem 2.1, system (2.1) with the mentioned parameters has a Hopf bifurcation at $(u_{*},v_{*})$. The direction of bifurcation is subcritical and the bifurcating periodic solutions are asymptotically stable. Besides, for $d = 1.3 > d_{0}$, the equilibrium $(u_{*},v_{*})$ is stable and the solution goes to $(u_{*},v_{*})$. We can see this result in Fig. 1(a) where the initial condition is considered as $(u_{0},v_{0}) = (0.03,1.3)$. In addition, for $d = 1.18 < d_{0}$, by Theorem 2.1, $(u_{*},v_{*})$ is unstable and by Theorem 2.2, this ODE system has a stable periodic orbit. In this case the solution goes to a limit cycle. One can see the mentioned result in Fig. 1(b) where the initial conditions are taken at $(0.03, 1.3)$ and $(0.05, 1.4)$. In [7], we can see the description of periodic orbits by using the fractal-like behavior of prime numbers (also, see [14, 18]).

Figure 1

Phase portrait of system (2.1) with parameters in (4.1): (a) the equilibrium is asymptotically stable where $d = 1.3 > d_{0}$; (b) the equilibriumis unstable and the periodic solution is stable where $d = 1.18 < d_{0}$

Now, consider system (1.1) in $\Omega = (0, 16\pi )\times (0, 16\pi )$ with parameters defined by (4.1) and

$$\begin{aligned} \mu = 103,\qquad \beta = 87. \end{aligned}$$

(4.2)

If we choose $d = 1.3 > d_{0} = 1.2066$, then $\frac{-M_{1}}{N_{1}} = 4.5$. Let $\alpha = 0.5$ and

$$\begin{aligned} (u_{0},v_{0}) = \bigl(0.025+10^{-4}\bigl(\sin (2x) + \cos (2y)\bigr), 1.25+10^{-2}\bigl( \sin (x) + \cos (y)\bigr) \bigr). \end{aligned}$$

Using Theorem 3.1(i), system (1.1) with the mentioned parameters has a Turing instability at $(u_{*}, v_{*}) = (0.025, 1.25)$. This instability is induced by the cross-diffusion term. Figure 2 presents the process of formation of Turing pattern for system (1.1) at times $t=5, 20, 100, 200$, where the involved constants are considered as $d = 1.3$, $\alpha = 0.5$, (4.1), and (4.2).

Figure 2

The process of Turing pattern for system (1.1) with parametersin (4.1), (4.2) and $d = 1.3 > d_{0} = 1.2066$

Example 4.2

For the ODE model (2.1), we choose

$$\begin{aligned} \begin{aligned} &r=0.7,\qquad e=2.3,\qquad c=0.1,\qquad a=1.5,\\ & k=4,\qquad b=27,\qquad m=0.9. \end{aligned} \end{aligned}$$

(4.3)

Then the positive equilibrium and the critical value of bifurcation parameter are

$$\begin{aligned} (u_{*},v_{*}) = (0.4285, 5.4223),\qquad d_{0} = 0.4807. \end{aligned}$$

Also, we derive $a(d_{0}) =- 0.0002$. By Theorem 2.1, the positive equilibrium $(u_{*},v_{*})$ is asymptotically stable when $d > d_{0}$ and unstable when $d < d_{0}$. Moreover, a Hopf bifurcation occurs at $d = d_{0}$. The direction of Hopf bifurcation is subcritical and the bifurcating periodic solutions are asymptotically orbitally stable. For $d = 0.5 > d_{0}$, the equilibrium $(u_{*},v_{*})$ is asymptotically stable and the solution goes to $(u_{*},v_{*})$. This result is shown in Fig. 3(a) where the initial condition is taken at $(0.4, 5.5)$. In Fig. 3(b), we choose $d = 0.45 < d_{0}$, and the initial conditions are taken at $(0.4, 5.5)$ and $(0.2, 6)$. As we can see in this figure, the equilibrium $(u_{*},v_{*})$ is unstable and the solution goes to a stable limit cycle.

Figure 3

Phase portrait of system (2.1) with parameters in (4.3): (a) the equilibrium is asymptotically stable where $d = 0.5 > d_{0}$; (b)the equilibriumis unstable and the periodic solution is stable where $d = 0.45 < d_{0}$

Now consider the PDE model (1.1) in $\Omega = (0, 32\pi )\times (0, 32\pi )$ with (4.3) and

$$\begin{aligned} &\mu = 8\times 10^{-4},\qquad \beta = 45,\qquad \alpha = 0.2, \end{aligned}$$

(4.4)

$$\begin{aligned} &\mu = 90,\qquad \beta = 45,\qquad \alpha = 0.2 . \end{aligned}$$

(4.5)

By Theorem 3.1(ii), system (1.1) with (4.4) and $d = 0.5 > d_{0}$ has a stable solution at $(u_{*}, v_{*}) = (0.4285, 5.4223)$. In Fig. 4, we can see that the effect of the initial condition disappears over time and the solution returns to $(u_{*}, v_{*})$. In this figure the initial condition is taken at

$$\begin{aligned} (u_{0},v_{0}) = \bigl(0.4285+10^{-4}\sin (4x), 5.4223+10^{-2}\sin (4x)\bigr). \end{aligned}$$

On the other hand, under (4.3) and (4.5), system (1.1) has a Turing instability at $( u_{*}, v_{*}) = (0.4285, 5.4223)$. Figure 5, illustrates the process of formation of Turing pattern for system (1.1) with $d = 0.5$, (4.3), and (4.5).

Figure 4

Numerical simulation of system (1.1) with parameters in (4.3), (4.4) and $d = 0.5 > d_{0} = 0.4807$

Figure 5

The process of Turing pattern for system (1.1) with parameters in (4.3), (4.4) and $d = 0.5 > d_{0} = 0.4807$