Bifurcation analysis for a free-boundary tumor model with angiogenesis and inhibitor

Abstract

This paper is concerned with the bifurcation phenomenon of a free-boundary problem modeling the tumor growth under the action of angiogenesis and inhibitor. Taking the surface tension coefficient γ as a bifurcation parameter, we prove that there exist a positive integer $m^{**}$ and a sequence of $\gamma_{m}$ such that, for every $\gamma_{m}$ ($m>m^{**}$), symmetry-breaking stationary solutions bifurcate from the radially symmetric stationary solutions.

1 Introduction

Over the past 40 years, an increasing number of mathematical models describing tumor growth have been developed and studied, and many theoretical and numerical results have been established; see the review papers [1–6], the recent papers [7–15], and the references therein.

In this paper, we study a mathematical model of tumor growth under the action of angiogenesis and inhibitor. Angiogenesis is an essential process in wound healing and new birth. Tumor-induced angiogenesis is a process that tumor cells secrete cytokines that stimulate the vascular system to grow toward the tumor. As a result of angiogenesis, the tumor possesses its own vasculature. Then the nutrient and the inhibitor may be supplied to tumor via the capillary network. Assume that the nutrient and the inhibitor are single species consumed by tumor cell through its own vasculature and diffusion from the boundary. We denote the concentration of nutrient and inhibitor by σ and β, respectively. Using non-dimensional scales (see [16–18]), the problem we considered can be written as follows:

$$\begin{aligned}& \Delta \sigma =\sigma , \quad x\in \Omega , \end{aligned}$$

(1.1)

$$\begin{aligned}& \Delta \beta =\lambda \beta , \quad x\in \Omega , \end{aligned}$$

(1.2)

$$\begin{aligned}& -\Delta p =\mu (\sigma -\tilde{\sigma })-\nu \beta , \quad x\in \Omega , \end{aligned}$$

(1.3)

$$\begin{aligned}& \frac{\partial \sigma }{\partial \bf n} +\alpha (\sigma -\bar{ \sigma })=0, \quad x\in \partial \Omega , \end{aligned}$$

(1.4)

$$\begin{aligned}& \frac{\partial \beta }{\partial \bf n} +\tau (\beta -\bar{\beta })=0, \quad x \in \partial \Omega , \end{aligned}$$

(1.5)

$$\begin{aligned}& p =\gamma \kappa , \quad x\in \partial \Omega , \end{aligned}$$

(1.6)

$$\begin{aligned}& \frac{\partial p}{\partial \bf n} =0, \quad x\in \partial \Omega , \end{aligned}$$

(1.7)

where Ω is the region occupied by the tumor, which is a bounded domain in $\mathbb{R}^{3}$ with boundary ∂Ω and may vary with the time t; p is the pressure within the tumor resulting from the proliferation of the tumor cells; the right term of (1.3) denotes the cell proliferation rate within the tumor; μ is a measure of mitosis; σ̃ is the threshold value of nutrient concentration for apoptosis at which tumor cells’ birth and death meet the balance; ν is a positive constant reflecting the negative functions of inhibitor on tumor-cell proliferation; $\bf n$ is the unit outward normal; σ̄, β̄ are the external concentrations of the nutrient and inhibitor, respectively; α, τ are positive constants denoting the rates of nutrient supply and inhibitor supply through a developed network of capillary vessels to the tumor, respectively; γ is the surface tension coefficient; κ is the mean curvature of the tumor surface. The parameters σ̄, σ̃, β̄, ν, α, μ are assumed to satisfy $\mu (\frac{\alpha }{ \alpha +1}\bar{\sigma }-\tilde{\sigma })-\nu \bar{\beta }>0$.

Before we turn to the point of most interest, we recall some relevant works. If the inhibitor is absent ($\beta =0$) and the boundary condition (1.4) is replaced by $\sigma =\bar{\sigma }$ (which is formally the case $\alpha =\infty$), it was proved in [19, 20] that under the assumption $\tilde{\sigma }<\bar{ \sigma }$, there exists a unique radially symmetric stationary solution $\sigma_{s}(r)$ with radius $R_{s}$, and for each $\gamma_{m}(R_{s})$ ($m\ge 2$), a branch of symmetry-breaking stationary solutions bifurcates from the above radially symmetric stationary solution with free boundary

$$ r=R_{s}+\varepsilon Y_{m,n}(\theta ,\varphi ) +O \bigl( \varepsilon^{2} \bigr), $$

where $Y_{m,n}$ is the spherical harmonic of order $(m,n)$. For the two-dimensional case, we refer the reader to [21]. The asymptotic stability of stationary solutions was studied in [22–24]. Later, for problem (1.1)–(1.7) in the absence of inhibitors ($\beta =0$), Friedman and Lam [8] proved that, for any $\alpha >0$, $0<\tilde{\sigma }<\bar{ \sigma }$, there exists a unique radially symmetric solution $\sigma_{*}(r)$ with radius $R_{*}$; furthermore, Hu et al. [10] showed that for each $\mu_{m}$ ($m\ge 2$), there exist symmetry-breaking solutions bifurcating from the radially symmetric solution. While, if the inhibitor is present ($\beta \neq 0$) and the boundary conditions (1.4) and (1.5) are replaced by the boundary conditions $\sigma =\bar{\sigma }$, $\beta =\bar{\beta }$ (which is formally the case $\alpha =\infty$, $\tau =\infty$), the existence and asymptotic stability of radially symmetric solutions were studied in [17, 25, 26]. In the sequel, for each $\mu_{m}$ ($m>m^{**}$), Wang [11] established the existence of a sequence of symmetry-breaking solutions bifurcating from the radially symmetric solution.

Motivated by the above works, in this paper, we work with the boundary conditions (1.4) and (1.5) stemming from angiogenesis. The appearance of inhibitor and the Robin boundary conditions (1.4)–(1.5) bring some difficulties to the analysis of the radially symmetric solution and the corresponding linearized problem. By using the properties of the modified Bessel functions and the spherical harmonic functions, we first establish the existence of radially symmetric solution of problem (1.1)–(1.7) for all $\gamma >0$, and then prove that there exist a positive integer $m^{**}$ and a sequence $\{\gamma_{m}\}$ such that, for each $\gamma_{m}$ ($m>m^{**}$), there exists a branch of symmetry-breaking solutions bifurcating from the above radially symmetric solution.

The rest of this paper is arranged as follows. In Sect. 2, we establish the existence of radially symmetric solutions to problem (1.1)–(1.7); in Sect. 3, we study the linearized problem of (1.1)–(1.7) at the radially symmetric solution, and then we obtain the existence of symmetry-breaking solutions by the bifurcation theorem in Sect. 4. We give our conclusion in Sect. 5.

2 Radially symmetric solutions

In this section, we study the radially symmetric solutions of problem (1.1)–(1.7). We call $(\sigma (x),\beta (x),p(x))$ a radially symmetric solution of (1.1)–(1.7) in some domain $B_{R}$ if $\sigma (x)$, $\beta (x)$, $p(x)\in C^{2}(B_{R})\cap C^{1}(\overline{B_{R}})$ are radially symmetric and satisfy (1.1)–(1.7). Let $r=\vert x \vert$. Then in the radial case, (1.1)–(1.7) reduces to the following system:

$$\begin{aligned} &\sigma ''(r)+\frac{2}{r}\sigma '(r)=\sigma (r)\quad \mbox{for } 0< r< R, \end{aligned}$$

(2.1)

$$\begin{aligned} &\beta ''(r)+\frac{2}{r}\beta '(r)= \lambda \beta (r)\quad \mbox{for } 0< r< R, \end{aligned}$$

(2.2)

$$\begin{aligned} &p''(r)+\frac{2}{r}p'(r)=-\mu \bigl(\sigma (r)-\tilde{\sigma } \bigr)+\nu \beta (r)\quad \mbox{for } 0< r< R, \end{aligned}$$

(2.3)

$$\begin{aligned} &\sigma '(0)=0,\qquad \beta '(0)=0,\qquad p'(0)=0, \end{aligned}$$

(2.4)

$$\begin{aligned} &\sigma '(R)+\alpha \bigl(\sigma (R)-\bar{\sigma } \bigr)=0,\qquad \beta '(R)+\tau \bigl( \beta (R)-\bar{\beta } \bigr)=0, \qquad p(R)= \frac{\gamma }{R}, \end{aligned}$$

(2.5)

$$\begin{aligned} &p'(R)=0. \end{aligned}$$

(2.6)

For any fixed $R_{s}>0$, solving problem (2.1)–(2.5), we obtain

$$\begin{aligned}& \sigma_{s}(r) =\frac{\alpha \bar{\sigma }}{\alpha +\coth R_{s}-\frac{1}{R _{s}}}\frac{R_{s}\sinh r}{r\sinh R_{s}} \quad \mbox{for } 0< r< R _{s}, \end{aligned}$$

(2.7)

$$\begin{aligned}& \beta_{s}(r) =\frac{\tau \bar{\beta }}{\tau +\sqrt{\lambda } ( \coth (\sqrt{ \lambda }R_{s})-\frac{1}{\sqrt{\lambda }R_{s}} ) }\frac{R_{s} \sinh (\sqrt{\lambda }r)}{r\sinh (\sqrt{\lambda }R_{s})} \quad \mbox{for } 0< r< R_{s}, \end{aligned}$$

(2.8)

$$\begin{aligned}& p_{s}(r) =\frac{1}{6}\mu \tilde{\sigma }r^{2}-\mu \sigma_{s} (r)+\frac{ \nu }{\lambda }\beta_{s}(r)+C_{p} \quad \mbox{for } 0< r< R_{s}, \end{aligned}$$

(2.9)

where

$$ C_{p}=\frac{\gamma }{R_{s}}-\frac{1}{6}\mu \tilde{\sigma }R^{2}_{s}+\frac{ \mu \alpha \bar{\sigma }}{\alpha +\coth R_{s}-\frac{1}{R_{s}}}-\frac{ \nu }{\lambda } \frac{\tau \bar{\beta }}{\tau +\sqrt{\lambda } ( \coth (\sqrt{ \lambda }R_{s})-\frac{1}{\sqrt{\lambda }R_{s}} ) }. $$

Then $(\sigma_{s}, \beta_{s} , p_{s} )$ given by (2.7)–(2.9) is a radially symmetric solution of system (1.1)–(1.7) provided that $R_{s}$ is a positive solution of $p'_{s}(R)=0$, i.e.,

$$ \frac{1}{3} \mu \tilde{\sigma }R_{s}-\mu \sigma '_{s}(R_{s})+\frac{ \nu }{\lambda }\beta '_{s}(R_{s})=0. $$

(2.10)

The next theorem shows that $(\sigma_{s}(r), \beta_{s}(r), p_{s}(r))$ given by (2.7)–(2.9) is indeed a radially symmetric solution of system (1.1)–(1.7).

Theorem 2.1

For any given positive constants λ, σ̄, β̄, σ̃, μ, ν, α, τ, γ, satisfying $\mu (\bar{\sigma }-\tilde{\sigma })- \nu \bar{\beta }>0$, there exists at least one radially symmetric solution to system (1.1)–(1.7).

Proof

Obviously, we only need to show the existence of solutions to problem (2.1)–(2.6). It is seen that $(\sigma_{s}, \beta_{s}, p _{s})$ given by (2.7)–(2.9) satisfies problem (2.1)–(2.5). Thus it suffices to prove that equation (2.10) has at least one positive solution.

Define

$$ g(s)=\coth s-\frac{1}{s}. $$

Then it follows from [8] that for $s>0$, $g'(s)>0$, $(g(s)/s)'<0$, and $g(s)$ satisfy

$$ \begin{aligned} &\lim_{s\rightarrow 0^{+}}g(s)=0, \qquad \lim_{s\rightarrow +\infty }g(s)=1, \\ &\lim_{s\rightarrow 0^{+}} \frac{g(s)}{s}=\frac{1}{3}, \qquad \lim_{s\rightarrow +\infty } \frac{g(s)}{s}=0. \end{aligned} $$

(2.11)

Moreover, $\sigma_{s}(r)$ and $\beta_{s}(r)$ can be rewritten as

$$\begin{aligned}& \sigma_{s}(r) =\frac{\alpha \bar{\sigma }}{\alpha +g(R_{s})}\frac{R _{s}\sinh r}{r\sinh R_{s}}, \\& \beta_{s}(r) =\frac{\tau \bar{\beta }}{\tau +\sqrt{\lambda }g(\sqrt{ \lambda }R_{s})}\frac{R_{s}\sinh (\sqrt{\lambda }r)}{r\sinh (\sqrt{ \lambda }R_{s})}. \end{aligned}$$

A direct computation shows that

$$\begin{aligned}& \sigma '_{s}(R_{s})= -\alpha \bigl( \sigma_{s}(R_{s})-\bar{\sigma } \bigr) =- \alpha \biggl( \frac{\alpha \bar{\sigma }}{\alpha +g(R_{s})}-\bar{ \sigma } \biggr) , \end{aligned}$$

(2.12)

$$\begin{aligned}& \beta '_{s}(R_{s})= -\tau \bigl( \beta_{s}(R_{s})-\bar{\beta } \bigr) =-\tau \biggl( \frac{\tau \bar{\beta }}{\tau +\sqrt{\lambda }g(\sqrt{ \lambda }R_{s})}-\bar{\beta } \biggr) . \end{aligned}$$

(2.13)

Substituting (2.12) and (2.13) into (2.10) yields

$$ \frac{1}{3}\mu \tilde{\sigma }R_{s}+\mu \alpha \biggl( \frac{\alpha \bar{ \sigma }}{\alpha +g(R_{s})}-\bar{\sigma } \biggr) -\frac{\nu }{\lambda }\tau \biggl( \frac{\tau \bar{\beta }}{\tau +\sqrt{\lambda }g(\sqrt{ \lambda }R_{s})}-\bar{\beta } \biggr) =0, $$

which is equivalent to the following:

$$\begin{aligned} &\mu \alpha \bar{\sigma } \frac{g(R_{s})}{R_{s}} \bigl(\tau +\sqrt{\lambda }g(\sqrt{\lambda }R_{s}) \bigr) \\ &\quad {}- \nu \tau \bar{\beta }\frac{g(\sqrt{ \lambda }R_{s})}{\sqrt{\lambda }R_{s}} \bigl(\alpha +g(R_{s}) \bigr) - \frac{1}{3}\mu \tilde{\sigma } \bigl(\alpha +g(R_{s}) \bigr) \bigl(\tau +\sqrt{ \lambda }g(\sqrt{\lambda }R_{s}) \bigr)=0. \end{aligned}$$

Let

$$\begin{aligned} T(s)&=\mu \alpha \bar{\sigma }\frac{g(s)}{s} \bigl(\tau +\sqrt{\lambda }g( \sqrt{ \lambda }s) \bigr)-\nu \tau \bar{\beta }\frac{g(\sqrt{\lambda }s)}{\sqrt{ \lambda }s} \bigl(\alpha +g(s) \bigr) \\ &\quad {} -\frac{1}{3}\mu \tilde{\sigma } \bigl(\alpha +g(s) \bigr) \bigl( \tau +\sqrt{\lambda }g(\sqrt{\lambda }s) \bigr). \end{aligned}$$

Then from (2.11) we see that

$$\begin{aligned}& \lim_{s\rightarrow 0^{+}}T(s) =\frac{1}{3}\mu \alpha \bar{\sigma } \tau -\frac{1}{3}\nu \bar{\beta }\alpha \tau -\frac{1}{3}\mu \tilde{ \sigma }\alpha \tau =\frac{\alpha \tau }{3}(\mu \bar{\sigma }- \mu \tilde{\sigma }- \nu \bar{\beta }) \end{aligned}$$

and

$$\begin{aligned}& \lim_{s\rightarrow +\infty }T(s) =-\frac{1}{3}\mu \tilde{\sigma }( \alpha +1) (\tau +\sqrt{\lambda }). \end{aligned}$$

Since $\mu (\bar{\sigma }-\tilde{\sigma })-\nu \bar{\beta }>0$, $\alpha >0$, $\tilde{\sigma }>0$, $\tau >0$, $\mu >0$, $\lambda >0$, we conclude that $T(s)=0$ has at least one positive root on $(0, \infty )$ by using the continuity of $T(s)$ and the intermediate value theorem. The proof is complete. □

3 Linearized problem

In this section, we consider the linearization of problem (1.1)–(1.7) at a radially symmetric solution $(\sigma_{s}, \beta_{s}, p_{s})$ with radius $R_{s}$, and solve it by employing spherical harmonics and modified Bessel functions.

Let $(\sigma , \beta , p)$ be the solution of problem (1.1)–(1.7) on the domains with boundaries $\partial \Omega_{\varepsilon }: r=R_{s}+\tilde{R}$, where $\tilde{R}=\varepsilon S(\theta , \varphi )$. For simplicity, we denote $R_{s}$ by R. Assume that $(\sigma , \beta , p)$ has the expansion as follows:

$$\begin{aligned} &\sigma =\sigma_{s}+\varepsilon \sigma_{1}+O \bigl( \varepsilon^{2} \bigr), \end{aligned}$$

(3.1)

$$\begin{aligned} &\beta =\beta_{s}+\varepsilon \beta_{1}+O \bigl( \varepsilon^{2} \bigr), \end{aligned}$$

(3.2)

$$\begin{aligned} &p=p_{s}+\varepsilon p_{1}+O \bigl(\varepsilon^{2} \bigr), \end{aligned}$$

(3.3)

where $\sigma_{1}$, $\beta_{1}$, $p_{1}$ are the functions to be determined.

Substituting (3.1)–(3.3) into (1.1)–(1.6), using (2.1)–(2.6) and collecting all ε-order terms (see [24]), we obtain the linearized problem satisfied by $\sigma_{1}$, $\beta_{1}$, $p_{1}$:

$$\begin{aligned}& \Delta \sigma_{1}=\sigma_{1} \quad \mbox{in } B_{R}, \end{aligned}$$

(3.4)

$$\begin{aligned}& \frac{\partial \sigma_{1}}{\partial r} + \alpha \sigma_{1}=- \bigl( \sigma ''_{s} +\alpha \sigma '_{s} \bigr)S(\theta ,\varphi ) \quad \mbox{on } \partial B_{R}, \end{aligned}$$

(3.5)

$$\begin{aligned}& \Delta \beta_{1}=\lambda \beta_{1} \quad \mbox{in } B_{R}, \end{aligned}$$

(3.6)

$$\begin{aligned}& \frac{\partial \beta_{1}}{\partial r} +\tau \beta_{1}=- \bigl( \beta ''_{s} +\tau \beta '_{s} \bigr)S(\theta ,\varphi ) \quad \mbox{on } \partial B_{R}, \end{aligned}$$

(3.7)

$$\begin{aligned}& \Delta p_{1}=-\mu \sigma_{1}+\nu \beta_{1}\quad \mbox{in } B_{R}, \end{aligned}$$

(3.8)

$$\begin{aligned}& p_{1}=-\frac{\gamma }{R^{2}} \biggl( S(\theta ,\varphi )+ \frac{1}{2} \triangle_{\omega }S(\theta ,\varphi ) \biggr) \quad \mbox{on } \partial B _{R}, \end{aligned}$$

(3.9)

where we employed the relation (see [24, 27])

$$ \kappa \bigl(R+\varepsilon S(\theta ,\varphi ) \bigr)=\frac{1}{R}- \frac{\varepsilon }{R^{2}} \biggl[ S(\theta ,\varphi )+\frac{1}{2} \Delta_{\omega }S(\theta ,\varphi ) \biggr] +O \bigl(\varepsilon^{2} \bigr). $$

Here and below, $\Delta_{\omega }$ is the Laplace operator on the unit sphere Σ:

$$ \Delta_{\omega }=\frac{1}{\sin \theta }\frac{\partial }{\partial \theta } \biggl( \sin \theta \frac{\partial }{\partial \theta } \biggr) +\frac{1}{ \sin^{2}\theta }\frac{\partial^{2}}{\partial \varphi^{2}}. $$

Before solving problem (3.4)–(3.9), we recall some properties of the spherical harmonic functions and the modified Bessel functions.

Let $Y_{m,l}(\theta , \varphi )$ denote the spherical harmonic functions. Then the family $\{Y_{m,l}\}$ forms a normalized complete orthonormal basis for $L^{2}(\Sigma )$ (see [28]), and

$$ \Delta_{\omega }Y_{m,l}=-m(m+1)Y_{m,l}. $$

(3.10)

For $m>0$, $\xi >0$, let $I_{m}(\xi )$ be the modified Bessel function (see [10, 11, 20, 22, 23]):

$$ I_{m}(\xi )= \biggl(\frac{\xi }{2} \biggr)^{m} \overset{\infty }{\underset{k=0}{\sum }} \frac{1}{k!\Gamma (m+k+1)} \biggl(\frac{\xi }{2} \biggr)^{2k}. $$

(3.11)

It satisfies the ordinary differential equation

$$ I''_{m}(\xi )+ \frac{1}{\xi }I'_{m}(\xi )- \biggl( 1+ \frac{m^{2}}{\xi^{2}} \biggr) I_{m}(\xi )=0, $$

(3.12)

and the following:

$$\begin{aligned} & I'_{m}(\xi )+\frac{m}{\xi }I_{m}(\xi )=I_{m-1}(\xi ) \quad \mbox{for } m\ge 1, \end{aligned}$$

(3.13)

$$\begin{aligned} & I'_{m}(\xi )-\frac{m}{\xi }I_{m}(\xi )=I_{m+1}(\xi ) \quad \mbox{for } m\ge 0, \end{aligned}$$

(3.14)

$$\begin{aligned} &I_{m}(\xi )=\sqrt{\frac{1}{2m\pi }} \biggl( \frac{e\xi }{2m} \biggr) ^{m} \biggl( 1+O \biggl( \frac{1}{m} \biggr) \biggr)\quad \mbox{as } m \to \infty . \end{aligned}$$

(3.15)

Computations of $\sigma_{1}$, $\beta_{1}$, $p_{1}$. Set $S(\theta , \varphi )= Y_{m,l}(\theta , \varphi )$ in (3.4)–(3.9). Using separation of variables, we seek a solution of the form:

$$\begin{aligned}& \sigma_{1}(r,\theta ,\varphi )=\hat{\sigma }_{1}(r)Y_{m,l}( \theta , \varphi ), \\& \beta_{1}(r,\theta ,\varphi )=\hat{\beta }_{1}(r)Y_{m,l}( \theta , \varphi ), \\& p_{1}(r,\theta ,\varphi )=\hat{p}_{1}(r)Y_{m,l}( \theta ,\varphi ). \end{aligned}$$

It is seen from (3.4)–(3.5) and (3.10) that $\hat{\sigma }_{1}(r)$ satisfies

$$\begin{aligned} &\hat{\sigma }_{1}''(r)+\frac{2}{r} \hat{\sigma }'_{1}(r)-\frac{m(m+1)}{r ^{2}}\hat{\sigma }_{1}(r)=\hat{\sigma }_{1}(r) \quad \mbox{for } 0< r< R, \\ & \hat{\sigma }'_{1}(R) +\alpha \hat{\sigma }_{1}(R)=-\lambda_{1}, \end{aligned}$$

with

$$ \lambda_{1}= \sigma ''_{s}(R)+ \alpha \sigma '_{s}(R) = \frac{\alpha \bar{ \sigma }}{\alpha +g(R)} \biggl[1+ \alpha g(R)-\frac{2}{R} g(R) \biggr]. $$

Solving the above problem, from the formulae of modified Bessel functions (3.11)–(3.14), we derive

$$ \hat{\sigma }_{1}(r)=\frac{-\lambda_{1}R^{\frac{1}{2}}}{mR^{-1}I_{m+ \frac{1}{2}}(R)+I_{m+\frac{3}{2}}(R)+\alpha I_{m+\frac{1}{2}}(R)}\frac{I _{m+\frac{1}{2}}(r)}{r^{\frac{1}{2}}}. $$

(3.16)

Hence, the solution of problem (3.4)–(3.5) is given by

$$ \sigma_{1}(r, \theta , \varphi )=\frac{-\lambda_{1}R^{\frac{1}{2}}}{mR ^{-1}I_{m+\frac{1}{2}}(R)+I_{m+\frac{3}{2}}(R)+\alpha I_{m+ \frac{1}{2}}(R)} \frac{I_{m+\frac{1}{2}}(r)}{r^{\frac{1}{2}}}Y_{m,l}( \theta ,\varphi ). $$

Similarly, $\hat{\beta }_{1}(r)$ satisfies

$$\begin{aligned} &\hat{\beta }_{1}''(r)+\frac{2}{r} \hat{\beta }'_{1}(r)-\frac{m(m+1)}{r ^{2}}\hat{\beta }_{1}(r)=\lambda \hat{\beta }_{1}(r) \quad \mbox{for } 0< r< R, \\ &\hat{\beta }'_{1}(R)+\tau \hat{\beta }_{1}(R)=-\lambda_{2}, \end{aligned}$$

where

$$ \lambda_{2}=\frac{\tau \bar{\beta }}{\tau +\sqrt{\lambda }g(\sqrt{ \lambda }R)}\sqrt{\lambda } \biggl( \sqrt{\lambda }- \frac{2}{R}\frac{I _{\frac{3}{2}}(\sqrt{\lambda }R)}{I_{\frac{1}{2}}(\sqrt{\lambda }R)}+ \tau \frac{I_{\frac{3}{2}}(\sqrt{\lambda }R)}{I_{\frac{1}{2}}(\sqrt{ \lambda }R)} \biggr) . $$

The solution of the above problem is given explicitly by

$$ \hat{\beta }_{1}(r)=\frac{-\lambda_{2}R^{\frac{1}{2}}}{mR^{-1}I_{m+ \frac{1}{2}}(\sqrt{\lambda }R)+\sqrt{\lambda }I_{m+\frac{3}{2}}(\sqrt{ \lambda }R)+\tau I_{m+\frac{1}{2}}(\sqrt{\lambda }R)}\frac{I_{m+ \frac{1}{2}}(\sqrt{\lambda }r)}{r^{\frac{1}{2}}}. $$

(3.17)

Correspondingly, the solution of problem (3.6)–(3.7) is as follows:

$$ \beta_{1}(r, \theta , \varphi )=\frac{-\lambda_{2}R^{\frac{1}{2}}}{ \frac{m}{R}I_{m+\frac{1}{2}}(\sqrt{\lambda }R)+\sqrt{\lambda }I _{m+\frac{3}{2}}(\sqrt{\lambda }R)+\tau I_{m+\frac{1}{2}}(\sqrt{ \lambda }R)} \frac{I_{m+\frac{1}{2}}(\sqrt{\lambda }r)}{r^{ \frac{1}{2}}}Y_{m,l}(\theta ,\varphi ). $$

It remains to solve $p_{1}$. On the boundary, from (3.9), (3.10), we get

$$ \hat{p}_{1}(R)= -\frac{\gamma }{R^{2}} \biggl( 1-\frac{m(m+1)}{2} \biggr) . $$

Let $\hat{\eta }=\hat{p}_{1}+\mu \hat{\sigma }_{1}-\frac{\nu }{\lambda }\hat{\beta }_{1}$; then $\hat{\eta }(r)$ satisfies the following problem:

$$\begin{aligned} &\hat{\eta }''+\frac{2}{r}\hat{\eta }'-\frac{m(m+1)}{r^{2}} \hat{\eta }=0 \quad \mbox{for } 0< r< R, \\ &\hat{\eta }(R)=\hat{p}_{1}(R)+\mu \hat{\sigma }_{1}(R)- \frac{\nu }{ \lambda }\hat{\beta }_{1}(R). \end{aligned}$$

By the modified Bessel functions and (3.16), (3.17), it admits a solution of the form

$$ \hat{\eta }(r)=\tilde{C}_{1} r^{m} $$

with

$$ \tilde{C_{1}}=\frac{1}{R^{m}} \biggl[ -\frac{\gamma }{R^{2}} \biggl( 1- \frac{m(m+1)}{2} \biggr) -\frac{\mu \lambda_{1}}{h_{m}(R)+\alpha }+\frac{ \nu }{\lambda } \frac{\lambda_{2}}{\sqrt{\lambda }h_{m}(\sqrt{ \lambda }R)+\tau } \biggr] , $$

where

$$ h_{m}(R)=\frac{m}{R}+ \frac{I_{m+\frac{3}{2}}(R)}{ I_{m+\frac{1}{2}}(R)}. $$

Therefore, the solution of problem (3.8) and (3.9) is given by

$$\begin{aligned} &p_{1}(r,\theta ,\varphi ) \\ &\quad {}= \biggl[ -\frac{\gamma }{R^{2}} \biggl( 1-\frac{m(m+1)}{2} \biggr) - \frac{ \mu \lambda_{1}}{h_{m}(R)+\alpha }+\frac{\nu }{\lambda }\frac{\lambda _{2}}{\sqrt{\lambda }h_{m}(\sqrt{\lambda }R)+\tau } \biggr] \biggl( \frac{r}{R} \biggr) ^{m}Y_{m,l}(\theta ,\varphi ) \\ &\quad \quad {}-\mu \sigma_{1}(r,\theta ,\varphi ) +\frac{\nu }{\lambda } \beta_{1}(r, \theta ,\varphi ). \end{aligned}$$

Rigorous justification. For $S\in C^{k+\zeta }(\sum )$, $k\ge 3$, we set $\Omega_{\varepsilon }=\{r< R+\varepsilon S\}$. Note that $(\sigma , \beta , p)$ is defined only on $\Omega_{\varepsilon }$, while $(\sigma_{s}, \beta_{s}, p_{s})$ is defined on whole $\mathbb{R}^{3}$, and $(\sigma_{1}, \beta_{1}, p_{1})$ is defined on $B_{R}$. We need to transform all these functions to the same domain $\Omega_{\varepsilon }$ by the Hanzawa transformation, which is a diffeomorphism defined by

$$ (r,\theta ,\varphi )=H_{\varepsilon } \bigl(r',\theta ',\varphi ' \bigr)= \bigl(r'+ \chi \bigl(R-r' \bigr)\varepsilon S \bigl(\theta ',\varphi ' \bigr),\theta ',\varphi ' \bigr), $$

where

$$ \chi \in C^{\infty }, \qquad \chi (z)= \textstyle\begin{cases} 0,&\mbox{if } \vert z \vert \ge 3\delta_{0}/4, \\ 1,&\mbox{if } \vert z \vert < \delta_{0}/4, \end{cases}\displaystyle \qquad \biggl\vert \frac{d^{k}\chi }{d z^{k}} \biggr\vert \le \frac{C_{2}}{ \delta_{0}^{k}}. $$

Observe that $H_{\varepsilon }$ maps $B_{R}$ onto $\Omega_{\varepsilon }$, while keeping the ball $\{r< R-(3\delta_{0}/4)\}$ fixed. The inverse Hanzawa transformation $H^{-1}_{\varepsilon }$ maps $\Omega_{\varepsilon }$ onto $B_{R}$. Set

$$ \tilde{\sigma }_{1}(r,\theta ,\varphi )=\sigma_{1} \bigl(H_{\varepsilon } ^{-1}(r,\theta ,\varphi ) \bigr)\quad \mbox{in } \Omega_{\varepsilon }. $$

Then σ, $\sigma_{s}$, and $\tilde{\sigma }_{1}$ are all defined on the same domain $\Omega_{\varepsilon }$.

The Schauder estimates yield the following results (see Lemma 3.2 of [24]):

$$\begin{aligned} & \Vert \sigma -\sigma_{s}-\varepsilon \tilde{\sigma }_{1} \Vert _{C^{3+\zeta }(\overline{\Omega_{\varepsilon }})}\leq C\vert \varepsilon \vert ^{2}\Vert S \Vert _{C^{3+\zeta }(\sum )}, \\ &\Vert \beta -\beta_{s}-\varepsilon \tilde{\beta }_{1} \Vert _{C^{3+\zeta }(\overline{ \Omega_{\varepsilon }})}\leq C\vert \varepsilon \vert ^{2}\Vert S \Vert _{C^{3+\zeta }( \sum )}, \\ &\Vert p-p_{s}-\varepsilon \tilde{p}_{1} \Vert _{C^{1+\zeta }(\overline{ \Omega_{\varepsilon }})}\leq C\vert \varepsilon \vert ^{2}\Vert S \Vert _{C^{3+\zeta }( \sum )}, \end{aligned}$$

where C is independent of ε and S. This indicates that the expansions in (3.1)–(3.3) are rigorous. We shall prove that it also satisfies the boundary condition (1.7).

4 Symmetry-breaking solutions

In this section, we reduce problem (1.1)–(1.7) to a bifurcation problem by taking the aggressive parameter γ as a bifurcation parameter, then we obtain the existence of symmetry-breaking solutions by using the following Crandall–Rabinowitz bifurcation theorem (see Theorem 1.7 in [29]).

Theorem 4.1

(Crandall–Rabinowitz theorem)

Let X, Y be real Banach spaces and $F (x, \gamma )$ be a $C^{p}$ map, $p\ge 3$, of a neighborhood $(0,\gamma_{0})$ in $X \times \mathbb{R}$ into Y. Suppose

• (i) $F (0, \gamma ) = 0$ for all γ in a neighborhood of $\gamma_{0}$;

• (ii) There exists $x_{0}\in X$ such that $\operatorname{Ker}[ F _{x}(0,\gamma_{0})]$ is a one-dimensional space spanned by $x_{0}$;

• (iii) $\operatorname{Im}[ F_{x}(0,\gamma_{0})] = Y_{1}$ has codimension 1;

• (iv) $[ F_{\gamma x}](0,\gamma_{0})x_{0}\notin Y_{1}$.

Then $(0,\gamma_{0})$ is a bifurcation point of the equation $F (x, \gamma ) = 0$ in the following sense: In a neighborhood of $(0,\gamma_{0})$ the set of solutions of $F (x, \gamma ) = 0$ consists of two $C^{p-2}$ smooth curves $\Gamma_{1}$ and $\Gamma_{2}$ which intersect only at the point $(0,\gamma_{0})$; $\Gamma_{1}$ is the curve $(0, \gamma )$ and $\Gamma_{2}$ can be parameterized as follows:

$$ \Gamma_{2} : \bigl(x(\varepsilon ), \gamma (\varepsilon ) \bigr), \qquad \vert \varepsilon \vert \textit{ small }, \qquad \bigl(x(0), \gamma (0) \bigr) = (0,\gamma_{0}),\qquad x'(0) = x_{0}. $$

For each $\tilde{R}(\theta , \varphi )$, γ, define a function F by

$$ F(\tilde{R},\gamma )=\frac{\partial p}{\partial \mathbf{n}} \bigg| _{\partial \Omega_{\varepsilon }}. $$

(4.1)

Then $(\sigma , \beta , p)$ given by (3.1)–(3.3) is a solution of problem (1.1)–(1.7) if and only if $F(\tilde{R},\gamma )=0$ for some R̃ and γ. In order to apply the Crandall–Rabinowitz theorem, we need to compute the Fréchet derivative of $F(\tilde{R}, \gamma )$. An easy computation gives that

$$ \biggl[ \frac{\partial F}{\partial \tilde{R}}(0,\gamma ) \biggr] S( \theta ,\varphi ) = \frac{\partial^{2} p_{s}(R)}{\partial r^{2}}S( \theta ,\varphi ) +\frac{\partial p_{1}}{\partial r}(R, \theta , \varphi ). $$

Setting $S(\theta , \varphi )=Y_{m,l}(\theta , \varphi )$ as that in the last section, we compute the two terms on the right-hand side of the above equality:

$$\begin{aligned}& \frac{\partial^{2}p_{s}(R)}{\partial r^{2}}=-\mu \biggl( \frac{\alpha \bar{ \sigma }}{\alpha +g(R)}-\tilde{\sigma } \biggr) +\nu \frac{\tau \bar{ \beta }}{\tau +\sqrt{\lambda }g(\sqrt{\lambda }R)}, \\& \frac{\partial p_{1}}{\partial r}(R, \theta , \varphi )= -\mu \frac{ \partial \sigma_{1}}{\partial r}(R,\theta , \varphi ) +\frac{\nu }{ \lambda }\frac{\partial \beta_{1}}{\partial r}(R, \theta ,\varphi ) + \frac{m}{R} \biggl[ \biggl( \frac{m(m+1)}{2}-1 \biggr) \frac{\gamma }{R^{2}} \\& \hphantom{ \frac{\partial p_{1}}{\partial r}(R, \theta , \varphi )=}{}- \frac{\mu \lambda_{1}}{h_{m}(R)+\alpha } +\frac{\nu }{\lambda }\frac{\lambda_{2}}{\sqrt{\lambda }h_{m}(\sqrt{\lambda }R)+\tau } \biggr] Y _{m,l}(\theta , \varphi ), \end{aligned}$$

where

$$\begin{aligned} \frac{\partial \sigma_{1}(R,\theta ,\varphi )}{\partial r} &=-\alpha \sigma_{1}(R,\theta ,\varphi )- \lambda_{1}Y_{m,l}(\theta ,\varphi ) =\frac{- \lambda_{1}h_{m}(R)}{h_{m}(R)+\alpha }Y_{m,l}( \theta ,\varphi ), \\ \frac{\partial \beta_{1}(R,\theta ,\varphi )}{\partial r} &=-\tau \beta_{1}(R,\theta ,\varphi )- \lambda_{2}Y_{m,l}(\theta ,\varphi ) \\ &=\frac{-\lambda_{2} ( mR^{-1}+\sqrt{\lambda }\frac{I_{m+ \frac{3}{2}}(\sqrt{\lambda }R)}{I_{m+\frac{1}{2}}(\sqrt{\lambda }R)} ) }{mR ^{-1}+\sqrt{\lambda }\frac{I_{m+\frac{3}{2}}(\sqrt{\lambda }R)}{I _{m+\frac{1}{2}}(\sqrt{\lambda }R)}+\tau } Y_{m,l}(\theta ,\varphi ) \\ &=\frac{-\lambda_{2}\sqrt{\lambda }h_{m}(\sqrt{\lambda }R)}{\sqrt{ \lambda }h_{m}(\sqrt{\lambda }R)+\tau }Y_{m,l}(\theta ,\varphi ). \end{aligned}$$

Therefore, we have

$$\begin{aligned} \frac{\partial p_{1}(R,\theta ,\varphi )}{\partial r} &= \biggl[ \frac{m \gamma }{R^{3}} \biggl( \frac{m(m+1)}{2}-1 \biggr) +\frac{\mu \lambda_{1}}{h _{m}(R)+\alpha } \frac{I_{m+\frac{3}{2}}(R)}{I_{m+\frac{1}{2}}(R)} \\ & \quad {}-\frac{\nu }{\lambda }\frac{\lambda_{2}}{\sqrt{\lambda }h_{m}(\sqrt{ \lambda }R)+\tau }\sqrt{\lambda } \frac{I_{m+\frac{3}{2}}(\sqrt{ \lambda }R)}{I_{m+\frac{1}{2}}(\sqrt{\lambda }R)} \biggr] Y_{m,l}( \theta ,\varphi ). \end{aligned}$$

The equation

$$ \biggl[ \frac{\partial F}{\partial \tilde{R}}(0,\gamma ) \biggr] Y_{m,l}( \theta ,\varphi )=0 $$

is then reduced to $\gamma A_{m}-B_{m}=0$, where

$$\begin{aligned} &A_{m}=\frac{m}{R^{3}} \biggl( \frac{m(m+1)}{2}-1 \biggr) = \frac{m(m-1)(m+2)}{2R ^{3}}, \\ &B_{m}=\mu \biggl( \frac{\alpha \bar{\sigma }}{\alpha +g(R)}- \tilde{\sigma } \biggr) -\nu \frac{\tau \bar{\beta }}{\tau +\sqrt{ \lambda }g(\sqrt{\lambda }R)}-\frac{\mu \lambda_{1}}{h_{m}(R)+ \alpha }\frac{I_{m+\frac{3}{2}}(R)}{I_{m+\frac{1}{2}}(R)} \\ & \hphantom{B_{m}=}{} +\frac{\nu }{\sqrt{\lambda }}\frac{\lambda_{2}}{\sqrt{\lambda }h _{m}(\sqrt{\lambda }R)+\tau }\frac{I_{m+\frac{3}{2}}(\sqrt{\lambda }R)}{I_{m+\frac{1}{2}}(\sqrt{\lambda }R)}. \end{aligned}$$

Denote

$$ \gamma_{m}=\frac{B_{m}}{A_{m}}. $$

(4.2)

We now proceed to establishing our main lemma.

Lemma 4.1

If $\mu (\frac{\alpha }{\alpha +1}\bar{\sigma }-\tilde{\sigma })- \nu \bar{\beta }>0$, then there exists $m^{*}\in \mathbb{N}$ such that $\gamma_{m}$ is positive for $m>m^{*}$ and decreases with respect to m for $m>m^{*}$; moreover, $\lim_{m\to \infty }\gamma_{m}=0$.

Proof

From the property of modified Bessel functions (3.15), we get

$$\begin{aligned} \frac{I_{m+\frac{3}{2}}(r)}{I_{m+\frac{1}{2}}(r)}&= \frac{(2m+1)^{m+1}}{(2m+3)^{m+2}} \biggl[ 1+O \biggl(\frac{1}{m} \biggr) \biggr]er \\ &= \frac{r}{2m}+O \biggl(\frac{1}{m^{2}} \biggr). \end{aligned}$$

Denote

$$ q_{m}(R)=-\frac{\mu \lambda_{1}}{h_{m}(R)+\alpha }\frac{I_{m+ \frac{3}{2}}(R)}{I_{m+\frac{1}{2}}(R)} + \frac{\nu }{\sqrt{\lambda }} \frac{\lambda_{2}}{\sqrt{\lambda }h_{m}(\sqrt{ \lambda }R)+\tau }\frac{I_{m+\frac{3}{2}}(\sqrt{\lambda }R)}{I_{m+ \frac{1}{2}}(\sqrt{\lambda }R)}. $$

Then

$$ q_{m}(R)\sim \biggl[ -\frac{\mu \lambda_{1}}{\frac{m}{R}+\frac{R}{2m} + \alpha }+\nu \frac{\lambda_{2}}{\frac{m}{R}+\frac{\lambda R}{2m} + \tau } \biggr] \frac{R}{2m}+O \biggl( \frac{1}{m^{2}} \biggr) \quad \mbox{as } m \to \infty . $$

(4.3)

In view of (4.2), we can rewrite $\gamma_{m}$ as

$$\begin{aligned} \gamma_{m}&=\frac{B_{m}(R)}{A_{m}(R)} \\ &= \biggl[ \mu \biggl( \frac{\alpha \bar{ \sigma }}{\alpha +g(R)}-\tilde{\sigma } \biggr) -\nu \frac{\tau \bar{ \beta }}{\tau +\sqrt{\lambda }g(\sqrt{\lambda }R)}+q_{m}(R) \biggr] \cdot \frac{2R^{3}}{m(m-1)(m+2)}. \end{aligned}$$

(4.4)

Since $\mu (\frac{\alpha }{\alpha +1}\bar{\sigma }-\tilde{\sigma })- \nu \bar{\beta }>0$ and $0< g(R)<1$ imply that $\mu (\frac{\alpha }{ \alpha +g(R)}\bar{\sigma }-\tilde{\sigma })-\nu \bar{\beta }\frac{ \tau }{\tau +\sqrt{\lambda }g(\sqrt{\lambda }R)}>0$, the lemma follows from (4.3) and (4.4). □

Moreover, it is seen from (4.4) that the following lemma holds.

Lemma 4.2

$\gamma_{m}$ decreases with respect to β̄.

In the following, we show that $(0, \gamma_{m})$ is the bifurcation point of the equation $F(\tilde{R},\gamma )=0$ for sufficiently large m.

For $m^{*}$ given in Lemma 4.1, define $m^{**}$ as follows: if $\min \{\gamma_{2},\gamma_{3},\ldots,\gamma_{m^{*}}\}\le 0$, then $m^{**}=m^{*}+1$; otherwise,

$$ m^{**}=\inf \bigl\{ m\in \mathbb{N}^{+}: \gamma_{m}< \min \{\gamma_{2}, \gamma_{3},\ldots,\gamma_{m^{*}}\} \bigr\} . $$

Let us introduce the Banach spaces:

$$\begin{aligned}& X^{n+\zeta } = \bigl\{ \tilde{R}\in C^{n+\zeta }(\Sigma ), \tilde{R} \mbox{ is } \pi\mbox{-periodic in } \theta , 2\pi \mbox{-periodic in } \varphi \bigr\} , \\& X^{n+\zeta }_{2} = \bigl\{ \mbox{closure of the linear space spanned by } \{Y_{j, 0}, j=0, 2, 4, \dots \}\mbox{ in } X^{n+ \zeta } \bigr\} . \end{aligned}$$

Then $F(\tilde{R},\gamma )$ maps $X^{3+\zeta }$ to $X^{\zeta }$. We choose $X=X^{3+\zeta }_{2}(\sum )$, $Y=X^{\zeta }_{2}(\sum )$.

We have proved in Lemma 4.1 that $\gamma_{m}$ are all distinct for even integer $m\ge m^{**}$, so

$$\begin{aligned} &\operatorname{Ker} \bigl[ F_{\tilde{R}}(0,\gamma_{m}) \bigr]=\operatorname{span} \{Y_{m,0}\}. \end{aligned}$$

On the other hand, $F_{\tilde{R}}(0,\gamma_{m})Y_{k,0}=(\gamma_{m} A _{k}-B_{k})Y_{k,0}$, where $\gamma_{m} A_{k}-B_{k}\neq 0$ for $k\neq m$. This suggests that

$$ \begin{aligned} &\operatorname{Im} \bigl[F_{\tilde{R}}(0, \gamma_{m}) \bigr]+ \operatorname{Ker} \bigl[ F_{\tilde{R}}(0, \gamma_{m}) \bigr]=Y. \end{aligned} $$

Note that

$$ \bigl[F_{\gamma \tilde{R}}(0,\gamma_{m}) \bigr]Y_{m,0}=A_{m} Y_{m,0}\notin {\operatorname{Im}} \bigl[ F_{\tilde{R}}(0, \gamma_{m}) \bigr]. $$

By the Crandall–Rabinowitz theorem and Lemma 4.1, for even integer $m\geq m^{**}$, $(0,\gamma_{m})$ is a bifurcation point of the equation $F(\tilde{R},\gamma )=0$. From the definition of $F( \tilde{R}, \gamma )$ given in (4.1), we have shown the existence of symmetry-breaking solutions to system (1.1)–(1.7).

Now, we conclude our main result as follows.

Theorem 4.2

If $\mu (\frac{\alpha }{\alpha +1}\bar{\sigma }-\tilde{\sigma })- \nu \bar{\beta }>0$, then there is a positive integer $m^{**}$ such that, for every even $m>m^{**}$, $(0,\gamma_{m})$ is a bifurcation point of the equation $F(\tilde{R}, \gamma )=0$, then $\gamma_{m}$ is a bifurcation point of problem (1.1)–(1.7), and the corresponding branch of solutions has the expressions:

$$\begin{aligned} &\sigma_{\varepsilon }=\sigma_{s}+\varepsilon \hat{\sigma }_{1} Y_{m,0}( \theta ,\varphi )+O \bigl(\varepsilon^{2} \bigr), \\ &\beta_{\varepsilon }=\beta_{s}+\varepsilon \hat{\beta }_{1} Y_{m,0}( \theta ,\varphi )+O \bigl(\varepsilon^{2} \bigr), \\ &p_{\varepsilon }=p_{s}+\varepsilon \hat{p}_{1} Y_{m,0}(\theta , \varphi )+O \bigl(\varepsilon^{2} \bigr), \end{aligned}$$

with its free boundaries of the form $r=R_{s}+\varepsilon Y_{m,0}( \theta ,\varphi )+O(\varepsilon^{2})$.

Remark 4.1

Since $\mu (\frac{\alpha }{\alpha +g(R)}\bar{\sigma }-\tilde{\sigma })- \nu \bar{\beta }\frac{\tau }{\tau +\sqrt{\lambda }g(\sqrt{\lambda }R)}>0$ if $0<\lambda \le 1$, combining Theorem 2.1, we see that Theorem 4.2 is also valid for $\mu (\bar{\sigma }- \tilde{\sigma })-\nu \bar{\beta }>0$ and $0<\lambda \le 1$.

5 Conclusion

Our results indicate that a tumor with an angiogenesis and inhibitor is associated with the growth of protrusions. In our model, these protrusions are expressed by the shape $r=R_{s}+\varepsilon Y_{m,0}( \theta ,\varphi )+O(\varepsilon^{2})$ of the free boundary. The aggressiveness of a tumor could be measured by the surface tension coefficient γ. The smaller γ is the easier it is for the tumor to migrate. And Lemma 4.2 implies that for the same branch protrusion, more supply of inhibitors would require a smaller tumor aggressiveness parameter to develop the same protrusions. Namely, the inhibitor could reduce the tendency of the tumor’s evolution into an invasive state.