Skip to main content

A minimum principle for stochastic control of hepatitis C epidemic model


In this paper, we discussed a stochastic optimal control of hepatitis C that minimizes the side effect and reduces the viral load. The control variables represent the drug therapy used for blocking a new infection and virus production. The solution of control problem is solved using the stochastic minimum principle and a four-step scheme. The numerical simulation is carried out to justify the theoretical analysis. The result shows that using both types of drugs for therapy is much more effective.

1 Introduction

Hepatitis C is an inflammatory liver disease caused by the hepatitis C virus (HCV) that can lead to cirrhosis and/or hepatocellular cancer [1]. Hepatitis C is a bloodborne disease, which is mostly transmitted through contaminated needles used for dialysis, tattoos, piercings, and unclean blood transfusion [24]. HCV infection does not always require treatment since the immune response can clear the infection. However, when hepatitis C becomes chronic, treatment is needed to cure the disease. WHO recommends using direct-acting antiviral (DAA) therapy [5].

Certain variants are associated with a higher potential for resistance or therapeutic failure. The DAA regimen faces a risk of resistance when administered to a group of infected patients with a special variant known as the resistant associated variant (RAV). Thus, in every case of suspected resistance or failure of therapy with the DAA regimen, it is necessary to examine resistance genotypes. The results of this examination will be considered for therapy modification, either by adding the duration, other agents, especially ribavirin, or by substituting the regimen [6].

In the past 10 years, there have been two types of standard therapies for chronic hepatitis C that are often used in the treatment of hepatitis C, namely a combination of interferon and ribavirin. This therapy gave satisfactory results, i.e., in patients with HCV genotypes 2 and 3, around 80% of patients could achieve sustained virological response (SVR24), while in HCV genotype 1, only 40-50% of patients managed to achieve SVR24 [6]. Based on the basic HCV model provided by Neumann [7], Dustin et al. [8] and Hasan et al. [6], they explained that the type and duration of therapy depend on the type of HCV genotype. Thus, giving the same therapy to different patients can result in different effects. This phenomenon will be modeled as stochastic effects.

Furthermore, a combination of interferon and ribavirin also has side effects such as anemia, drowsiness, indigestion, and shortness of breath [9]. Therefore, in this paper, a stochastic optimal control problem is established to determine the optimal drug effectiveness to treat hepatitis C. The objective function is minimizing viral load and drug side effects.

Several papers have discussed the optimal control of hepatitis C. Martin et al. in [10] had determined the optimal therapy program for hepatitis C patients over 10 years, considering biomedical and economic goals. In this case, the minimum cost is determined to reduce viral load and the cost of secondary effects caused during therapy. Meanwhile, Peregrino et al. [9] considered optimal control of cellular level HCV with immune response, and Zhang et al. [11] proposed optimal control of HCV model with treatment, both of whom used the maximum Pontryagin principle. Their research focused on a deterministic model, whereas we developed a stochastic optimal control model in this study. We could investigate the effects of uncertainty in the optimal control of the hepatitis C problem. Another research that analyzes the fractional order mathematical model of hepatitis B is studied in [12].

In [1315], they discussed about an optimal control problem in deterministic and stochastic epidemic models using the Hamilton-Jacobi equation. Meanwhile, Ishikawa [16] designed the optimal control of the SIR model using the maximum stochastic principle and the four-step scheme technique. However, researchers have not discussed the optimal control problem where the control variable is contained in the diffusion coefficient. In this article, we present the new results of the stochastic optimal control where the control variable is contained in the diffusion coefficient. The optimal control problem is solved using the Hamilton-Jacobi-Bellman (HJB) equation, which is a nonlinear function and challenging to solve numerically. In this paper, the minimum stochastic principle and the four-step scheme will be applied to the stochastic control of the hepatitis C epidemic model. Based on medical literature [7], the optimal solution determines the effectiveness of drugs related to administer drug doses during therapy.

The rest of this paper is organized as follows. Section 2 presents the mathematical modeling for hepatitis C with a stochastic control and the analysis. Then, in Sect. 3, we carried out numerical simulations to verify theoretical results. Finally, in the last section, we present a conclusion.

2 HCV stochastic control model with a diffusion coefficient that contains control variables

HCV initially infects target cells, proliferates inside them, and then gets released into the extracellular space without disrupting the cell integrity. During its life cycle, HCV usurps host cell molecules, termed host factors, and various cell biological mechanisms ranging from endocytosis to the secretory pathway. The HCV life cycle can be separated into four steps: (1) virus entry; (2) genome translation and polyprotein processing; (3) genome replication and (4) particle assembly and release from the host cell. Virions are released from the cells most likely by exocytosis or transmitted to other cells via a cell-free mechanism [17].

Recall a deterministic mathematical model of HCV given in [7] as

$$\begin{aligned}& \frac{dT}{dt} = {\Lambda - {\delta _{1}}T - (1 - \eta )\beta VT}, \\& \frac{dI}{dt} = {(1 - \eta )\beta VT - {\delta _{2}}I}, \\& \frac{dV}{dt} = {(1 - \epsilon )kI - cV.} \end{aligned}$$

The cell population \(N(t)\) is categorized into three sub-populations. Let \(T(t) \) be the number of healthy or uninfected cells, \(I(t) \) be the number of infected cells, and \(V(t) \) be the number of free viruses. The uninfected cells are produced at rate Λ and die naturally at a constant rate \(\delta _{1} \). Cells become infected when they interact with the virus at a constant rate β; once infected, they will die at a constant rate \(\delta _{2} \). HCV is produced by infected cells at a constant rate k and cleared at a constant rate c. Parameter η is the effectiveness of a drug in stopping new infection and ϵ is the effectiveness of a drug in blocking virus production.

For the deterministic Model (1), there are two equilibrium points, namely the disease-free equilibrium point \(E_{0}=(\frac{\Lambda}{\delta _{1}},0,0)\) and the endemic equilibrium point \(E_{1}=(\frac{\delta _{2}c}{(1-\eta )(1-\epsilon )k\beta}, \frac{\Lambda}{\delta _{2}}- \frac{\delta _{1}c}{(1-\eta )(1-\epsilon )k\beta}, \frac{(1-\epsilon )k\Lambda}{\delta _{2}c}- \frac{\delta _{1}}{(1-\eta )\beta})\). By using the next generation matrix method [18], the basic reproduction number for the deterministic model or uncontrolled system is \(R_{0}= \frac{(1-\eta )(1-\epsilon )\Lambda \beta k}{c\delta _{1}\delta _{2}}\).

We defined two controls variables as follows

  1. 1

    \(u_{1}(t) \) represents the control variable to block new infection.

  2. 2

    \(u_{2}(t) \) represents the control variable of hepatitis C to block the virus production.

We assumed that the control variables are stochastic, i.e., \(u_{1}(t)=u_{1}+\sigma _{1}u_{1}\,dB(t)/dt \) and \(u_{2}(t)= u_{2}+\sigma _{2}u_{2}\,dB(t)/dt \), where \(B(t)\) is standard Brownian motion. The stochastic control system model is given by

$$\begin{aligned} \textstyle\begin{cases} {dT = ( {\Lambda - {\delta _{1}}T - (1 - u_{1} )\beta VT} )\,dt + \sigma _{1}u_{1} \beta VT\,dB(t)}, \\ {dI = ( {(1 - u_{1} )\beta VT - {\delta _{2}}I} )\,dt - \sigma _{1}u_{1} \beta VT\,dB(t)}, \\ {dV = ( {(1 - u_{2})kI - cV} )\,dt - \sigma _{2}u_{2}kI \,dB(t)}, \end{cases}\displaystyle \end{aligned}$$


$$\begin{aligned} U=\bigl\{ (u_{1},u_{2}):0\leq u_{1},u_{2} < 1\bigr\} , \end{aligned}$$

is the set of control variables and \(u_{1}\), \(u_{2}\) is \(\{\mathcal{F}_{t}\}\)-adapted. If \(\sigma _{1}=\sigma _{2}=0\), then System (2) becomes a deterministic model studied in [7], where \(u_{1}=\eta \) and \(u_{2}=\epsilon \).

System (2) represents the dynamics of a biological cell population. Therefore, the number of cell populations should be non-negative and bounded. For this reason, we first established the global existence, positivity, and boundedness of solutions for the proposed model in the following theorem.

Theorem 2.1

For any initial value in \({\mathbb{R}}^{3} _{+}\), the System (2) is uniformly and ultimately bounded and belongs to the closed and bounded positively invariant set for every \(t\geq 0\). The coefficients of the System (2) satisfy the Lipschitz condition. Then there exists a unique time-global solution \((T(t),I(t),V(t)) \subset {\mathbb{R}}^{3}_{+}\), \(t \ge 0 \) with probability 1.


Consider System (2), it can be seen that the coefficients of the System (2) satisfy the Lipschitz condition thus guarantee the solution is unique and local. Next, it can be proved that the explosion time is infinity (i.e., the time when the solution tends to infinity), thus the solution is global. The detailed proof is analog to [13, 1921]. □

In our control problem, objective is to reduce the viral load and to minimize the therapy cost. This objective function/cost function is defined as follows

$$\begin{aligned} J(u) =&E \biggl[{ \int _{0}^{t_{f}} L\bigl(x(t),u(t)\bigr) \,dt} \biggr] \\ =&E \biggl[{ \int _{0}^{t_{f}}\frac{1}{2}c_{1}V(t)^{2}+ \frac{1}{2}c_{2}u_{1}(t)^{2}+ \frac{1}{2}c_{3}u_{2}(t)^{2}\,dt} \biggr]. \end{aligned}$$

The first term of equation (4) represents the main biological target, i.e., reducing the viral load. The constant \(c_{1} \) is related to the cost for lowering viral load, \(c_{2} \) and \(c_{3} \) are related to the cost of therapy. Since the integral is a random variable, then we need to take the expected value. Thus, the optimal control problem is determining optimal control \(u_{1}^{*}\), \(u_{2}^{*}\) that minimizes the objective function \(J(u)\).

The existence of the optimal control pair can be obtained using a result by [22].

Theorem 2.2

There is an optimal control \((u_{1}^{*}(t),u_{2}^{*}(t))\) such that

$$\begin{aligned} J\bigl(u_{1}^{*},u_{2}^{*}\bigr)=min \bigl\{ J(u_{1},u_{2}):u=(u_{1},u_{2})\in U\bigr\} . \end{aligned}$$

The optimal control variable can then be determined in the following theorem.

Theorem 2.3

The optimal control \(u_{1}^{*}(t)\) and \(u_{2}^{*}(t)\) of the System (2), which minimize the objective function (4) are characterized by

$$\begin{aligned}& u_{1}(t)^{*}=\min \bigl\{ max \{0,\bar{u}_{1} \},1 \bigr\} , \\& u_{2}(t)^{*}=\min \bigl\{ max\{0,\bar{u}_{2}\},1 \bigr\} . \end{aligned}$$


$$\begin{aligned}& \bar{u}_{1}= \frac{(p_{2}-p_{1})\beta VT+(q_{2}-q_{1})\beta VT}{c_{2}-(\sigma _{1}^{2}\beta ^{2}\bar{V}V\bar{T}T)(\phi _{11}-\phi _{21}+\phi _{22}-\phi _{12})}, \\& \bar{u}_{2}= \frac{p_{3}kI+q_{3}\sigma _{2}kI}{c_{3}-\sigma _{2}^{2}k^{2}\phi _{33}\bar{I}I}. \end{aligned}$$


In order to construct the optimal control \((u_{1}^{*}(t),u_{2}^{*}(t))\), defined

$$\begin{aligned}& f_{1} = \Lambda - {\delta _{1}}T - (1 - u_{1} ) \beta VT, \\& f_{2} = (1 - u_{1} )\beta VT - {\delta _{2}}I, \\& f_{3} = (1 - u_{2})kI - cV, \\& g_{1} = \sigma _{1}u_{1} \beta VT,\qquad g_{2}= -\sigma _{1}u_{1} \beta VT,\qquad g_{3}= -\sigma _{2}u_{2}kI. \end{aligned}$$

and we constructed Hamiltonian function as follows

$$\begin{aligned} H(\mathbf{x},\mathbf{u},\mathbf{p},\mathbf{q})= \bigl\langle \mathbf{f}( \mathbf{x}, \mathbf{u}),\mathbf{p} \bigr\rangle + L(\mathbf{x},\mathbf{u}) + \bigl\langle \mathbf{g}( \mathbf{x}),\mathbf{q} \bigr\rangle \end{aligned}$$

where \(\langle \cdot ,\cdot \rangle \) stands for the Euclidian inner product and

$$\begin{aligned}& \mathbf{f}:\mathbb{R}^{3}\times U \rightarrow \mathbb{R}^{3},\qquad \mathbf{g}:\mathbb{R}^{3}\rightarrow \mathbb{R}^{3},\qquad \mathbf{p}: \mathbb{R}\rightarrow \mathbb{R}^{3}, \\& \mathbf{q}:\mathbb{R} \rightarrow \mathbb{R}^{3},\qquad L:\mathbb{R}^{3}\times U \rightarrow \mathbb{R}, \\& \mathbf{f}=(f_{1},f_{2},f_{3})^{t},\qquad \mathbf{p}=(p_{1},p_{2},p_{3})^{t},\qquad \mathbf{q}=(q_{1},q_{2},q_{3})^{t},\qquad \mathbf{x}=(T,I,V)^{t}. \end{aligned}$$

Vector p and q denote the adjoint vectors. Thus, we have

$$\begin{aligned} H(\mathbf{x},\mathbf{u},\mathbf{p},\mathbf{q}) =& \bigl\langle \mathbf{f}( \mathbf{x}, \mathbf{u}),\mathbf{p} \bigr\rangle + L(\mathbf{x},\mathbf{u}) + \bigl\langle \mathbf{g}( \mathbf{x}),\mathbf{q} \bigr\rangle \\ =& p_{1}\bigl[\Lambda - {\delta _{1}}T - (1 - u_{1} )\beta VT\bigr]+p_{2}\bigl[(1 - u_{1} ) \beta VT - {\delta _{2}}I\bigr] \\ &{} + p_{3}\bigl[(1 - u_{2})kI - cV\bigr] + \frac{1}{2}c_{1}V(t)^{2}+\frac{1}{2}c_{2}u_{1}(t)^{2}+ \frac{1}{2}c_{3}u_{2}(t)^{2} \\ &{} + q_{1}\sigma _{1}u_{1} \beta VT -q_{2}\sigma _{1}u_{1} \beta VT-q_{3} \sigma _{2}u_{2} kI. \end{aligned}$$

Based on the stochastic minimum principle [23], then we get the following system of differential equations

$$\begin{aligned} \textstyle\begin{cases} {dT = ( {\Lambda - {\delta _{1}}T - (1 - u_{1} )\beta VT} )\,dt + \sigma _{1}u_{1} \beta VTdB(t)}, \\ {dI = ( {(1 - u_{1} )\beta VT - {\delta _{2}}I} )\,dt - \sigma _{1}u_{1} \beta VTdB(t)}, \\ {dV = ( {(1 - u_{2})kI - cV} )\,dt - \sigma _{2}u_{2}kI \,dB(t)}, \\ dp_{1}=[p_{1}\delta _{1}+p_{1}(1-u_{1})\beta V - p_{2}(1-u_{1})\beta V - q_{1}\sigma _{1}u_{1}\beta V \\ \hphantom{dp_{1}=}{} +q_{2}\sigma _{1}u_{1}\beta V]\,dt + q_{1}\,dB(t), \\ dp_{2}= [p_{2}\delta _{2}-p_{3}(1-u_{2})k + q_{3}\sigma _{2}u_{2}k]\,dt + q_{2}\,dB(t), \\ dp_{3}= [p_{1}(1-u_{1})\beta T -p_{2}(1-u_{1})\beta T + p_{3}c -c_{1}V -q_{1}\sigma _{1} u_{1}\beta T \\ \hphantom{dp_{3}= }{} + q_{2}\sigma _{1}u_{1} \beta T]\,dt +q_{3}\,dB(t), \\ \mathbf{x}(0)=(T_{0},I_{0},V_{0})^{t}, \qquad \mathbf{p}(t_{f})=(0,0,0)^{t}. \end{cases}\displaystyle \end{aligned}$$

According to [23], the controller can balance the scale of control and the degree of uncertainty if the diffusion coefficient contains control variables in the stochastic situation. Therefore, we need to define a New Hamiltonian function since the marginal value alone may not be able to fully characterize the trade-off between the cost and control gain in an uncertain environment. The New Hamiltonian function is given as follows

$$\begin{aligned} \mathcal{H}(t,x,u) =& H(t,x,u)-\frac{1}{2} trace\bigl[g(t,x,u)^{t}Pg(t,x,u) \bigr] \\ &{}+\frac{1}{2} trace \bigl\{ \bigl[g(t,x,u)-g(t,\bar{x},\bar{u}) \bigr]^{t} P\bigl[g(t,x,u)-g(t, \bar{x},\bar{u})\bigr]\bigr\} \\ =& H(t,x,u)-\frac{1}{2}\bigl[(P_{11}-P_{21}+P_{22}-P_{12}) (\sigma _{1}u_{1} \beta VT)^{2} \\ &{}+\sigma _{2}^{2}u_{2}^{2}k^{2}I^{2}P_{33} \bigr]+\frac{1}{2}\bigl[(P_{11}-P_{21}+P_{22}-P_{12}) ( \sigma _{1}\beta )^{2} \\ &{}\times(u_{1}VT-\bar{u}_{1}\bar{V}\bar{T})^{2} + ( \sigma _{2}k)^{2}(u_{2}I- \bar{u}_{2} \bar{I})^{2}P_{33}\bigr]. \end{aligned}$$

The value of \(\mathcal{H} \) is optimal if the following conditions are satisfied

$$\begin{aligned} \frac{\partial \mathcal{H}}{\partial u_{1}}=0 \quad \text{and}\quad \frac{\partial \mathcal{H}}{\partial u_{2}}=0, \end{aligned}$$

which give

$$\begin{aligned}& \bar{u}_{1}= \frac{(p_{2}-p_{1})\beta VT+(q_{2}-q_{1})\beta VT}{c_{2}-(\sigma _{1}^{2}\beta ^{2}\bar{V}V\bar{T}T)(P_{11}-P_{21}+P_{22}-P_{12})}, \end{aligned}$$
$$\begin{aligned}& \bar{u}_{2}= \frac{p_{3}kI+q_{3}\sigma _{2}kI}{c_{3}-\sigma _{2}^{2}k^{2}P_{33}\bar{I}I}. \end{aligned}$$

Since u should be in the control space U, then we have

$$\begin{aligned}& u_{1}(t) = \min \bigl\{ max \{0,\bar{u}_{1} \},1 \bigr\} , \end{aligned}$$
$$\begin{aligned}& u_{2}(t) = \min \bigl\{ max\{0,\bar{u}_{2}\},1\bigr\} . \end{aligned}$$

Furthermore, we consider the second order adjoining equation as follows

$$\begin{aligned}& \begin{aligned} dP&=- \bigl\{ f_{x}^{t}P(t) + P(t)f_{x} + (g_{x})^{t}P(t) (g_{x})+ (g_{x})^{t} \bigl(Q(t)\bigr) \\ &\quad {} + Q(t) (g_{x})+ H_{xx} \bigr\} \,dt+Q(t)\,dB(t), \end{aligned} \end{aligned}$$
$$\begin{aligned}& P(t_{f})= 0, \end{aligned}$$

with P and Q are \(n\times n\) matrices. The equations (10) and (17) are called forward-backward stochastic differential equations(FBSDE) with terminal conditions (18). Next, we will apply the four-step scheme method to solve the problem (17).

Step 1: Assume that \(P(t)\) and \(\mathbf{x}(t)\) are related, i.e.,

$$\begin{aligned} P(t)=\Phi \bigl(t, \mathbf{x}(t)\bigr), \end{aligned}$$

where \(\Phi = ( {\phi _{ij}}) _{n\times n}\) is \(n \times n\) matrix and \(\phi _{ij}\) is vector-valued function with \(i,j=1,2,\ldots,n\)

Step 2: By using Itô’s formula, we obtain

$$\begin{aligned} d\phi _{ij}(t,x) =& \biggl\{ \frac{\partial \phi _{ij}(t,x)}{\partial t}+ \biggl\langle \frac{\partial \phi _{ij}(t,x)}{\partial x},f(x,u) \biggr\rangle \\ &{}+ \frac{1}{2} trace \biggl[\frac{\partial}{\partial x} \biggl( \frac{\partial \phi _{ji}(t,x)}{\partial x} \biggr) g(x)g(x)^{t} \biggr] \biggr\} \,dt \\ &{} + \biggl\langle \frac{\partial \phi _{ij}(t,x)}{\partial x},g(x) \biggr\rangle \,dB(t). \end{aligned}$$

Since \(P(t)=\phi (t, x(t))\) and u is a function of p, q and x, then we have

$$\begin{aligned}& \frac{\partial \phi _{ij}(t,x)}{\partial t}+ \biggl\langle \frac{\partial \phi _{ij}(t,x)}{\partial x},f(x,\phi ,q) \biggr\rangle + \Delta \\& \quad {}+\frac{1}{2} trace \biggl[\frac{\partial}{\partial x} \biggl( \frac{\partial \phi _{ij}(t,x)}{\partial x} \biggr) g(x)g(x)^{t} \biggr] = 0 \end{aligned}$$


$$\begin{aligned} Q(t)= \biggl\langle \frac{\partial \phi _{ij}(t,x)}{\partial x},g(x) \biggr\rangle \end{aligned}$$

with the terminal conditions

$$\begin{aligned} \phi _{ij}(t_{f},x)=0,\quad i,j=1,2,3 \end{aligned}$$

and Δ is the coefficient dt of (17). Furthermore, the system of partial differential equations in equation (21) is obtained as follows

$$\begin{aligned}& \frac{\partial \phi _{11}(t,x)}{\partial t}+ \frac{\partial \phi _{11}(t,x)}{\partial T}\bigl[\Lambda - {\delta _{1}}T - (1 - u_{1} )\beta VT\bigr] \\& \quad {}+\frac{\partial \phi _{11}(t,x)}{\partial I}\bigl[(1 - u_{1} )\beta VT - { \delta _{2}}I\bigr]+\frac{\partial \phi _{11}(t,x)}{\partial V}\bigl[(1 - u_{2})kI - cV \bigr] \\& \quad {}+\frac{1}{2} \biggl(\frac{\partial \phi _{11}(t,x)}{\partial T}\sigma _{1}u_{1} \beta VT -\frac{\partial \phi _{11}(t,x)}{\partial I}\sigma _{1} u_{1} \beta VT- \frac{\partial \phi _{11}(t,x)}{\partial V}\sigma _{2} u_{2}kI \biggr)^{2} \\& \quad {}+\bigl(-2\delta _{1}\phi _{11}+(\phi _{21}-\phi _{11}) (1-u_{1})\beta V+( \phi _{12}-\phi _{11}) (1-u_{1})\beta V \\& \quad {}+(\phi _{11}-\phi _{21}+\phi _{22}-\phi _{12}) (\sigma _{1}u_{1}\beta V)^{2} \bigr) \\& \quad {}+(Q_{11}-Q_{21})\sigma _{1}u_{1}\beta V +(Q_{11}-Q_{12}\sigma _{1}u_{1} \beta V)=0. \end{aligned}$$

Analog for \(\phi _{ij}\) where \(i,j=1,2,3\).

$$\begin{aligned}& \frac{\partial \phi _{33}(t,x)}{\partial t}+ \frac{\partial \phi _{33}(t,x)}{\partial T}\bigl[\Lambda - {\delta _{1}}T - (1 - u_{1} )\beta VT\bigr] \\& \quad {}+\frac{\partial \phi _{33}(t,x)}{\partial I}\bigl[(1 - u_{1} )\beta VT - { \delta _{2}}I\bigr]+\frac{\partial \phi _{33}(t,x)}{\partial V}\bigl[(1 - u_{2})kI - cV \bigr] \\& \quad {}+\frac{1}{2} \biggl(\frac{\partial \phi _{33}(t,x)}{\partial T}\sigma _{1}u_{1} \beta VT -\frac{\partial \phi _{33}(t,x)}{\partial I}\sigma _{1} u_{1} \beta VT- \frac{\partial \phi _{33}(t,x)}{\partial V}\sigma _{2} u_{2}kI \biggr)^{2} \\& \quad {}+\bigl(-2c\phi _{33}+(\phi _{23}-\phi _{13}) (1-u_{1})\beta T+(\phi _{32}- \phi _{31}) (1-u_{1})\beta T \\& \quad {}+(\phi _{11}-\phi _{21}+\phi _{22}-\phi _{12}) (\sigma _{1}u_{1}\beta T)^{2} \bigr) \\& \quad {}+(Q_{13}-Q_{23})\sigma _{1}u_{1}\beta T +(Q_{31}-Q_{32})\sigma _{1}u_{1} \beta T +c_{1} =0. \end{aligned}$$

Meanwhile, from equation (13) and (14), we have

$$\begin{aligned}& \bar{u}_{1}= \frac{(p_{2}-p_{1})\beta VT+(q_{2}-q_{1})\beta VT}{c_{2}-(\sigma _{1}^{2}\beta ^{2}\bar{V}V\bar{T}T)(\phi _{11}-\phi _{21}+\phi _{22}-\phi _{12})}, \end{aligned}$$
$$\begin{aligned}& \bar{u}_{2}= \frac{p_{3}kI+q_{3}\sigma _{2}kI}{c_{3}-\sigma _{2}^{2}k^{2}\phi _{33}\bar{I}I}. \end{aligned}$$

By using the property of the control space U in (3), we obtain

$$\begin{aligned}& u_{1}(t)^{*}=\min \bigl\{ max \{0,\bar{u}_{1} \},1 \bigr\} , \end{aligned}$$
$$\begin{aligned}& u_{2}(t)^{*}=\min \bigl\{ max\{0,\bar{u}_{2}\},1 \bigr\} . \end{aligned}$$

Analogously, for the first-order adjoint equation (10), the four-step scheme method is also applied, and then the system of partial differential equations is given as follows

$$\begin{aligned}& \frac{\partial \theta _{1}(t,x)}{\partial t}+ \frac{\partial \theta _{1}(t,x)}{\partial T}\bigl[\Lambda - {\delta _{1}}T - (1 - u_{1} )\beta VT\bigr] \\& \quad {}+\frac{\partial \theta _{1}(t,x)}{\partial I}\bigl[(1 - u_{1} )\beta VT - { \delta _{2}}I\bigr]+\frac{\partial \theta _{1}(t,x)}{\partial V}\bigl[(1 - u_{2})kI - cV \bigr] \\& \quad {}+\frac{1}{2} \biggl(\frac{\partial \theta _{1}(t,x)}{\partial T}\sigma _{1}u_{1} \beta VT-\frac{\partial \theta _{1}(t,x)}{\partial I}\sigma _{1}u_{1} \beta VT- \frac{\partial \theta _{1}(t,x)}{\partial V}\sigma _{2}u_{2}kI \biggr)^{2} \\& \quad {}-\theta _{1}(t,x)\delta _{1}-\theta _{1}(t,x) (1 - u_{1} )\beta V+ \theta _{2}(t,x) (1 - u_{1} ) \beta V \\& \quad {}+\biggl(\frac{\partial \theta _{1}}{\partial T}- \frac{\partial \theta _{2}}{\partial T} \biggr) (\sigma _{1}u_{1} \beta V)^{2}T+ \biggl( \frac{\partial \theta _{2}}{\partial I}- \frac{\partial \theta _{1}}{\partial I} \biggr) (\sigma _{1}u_{1} \beta V)^{2}T \\& \quad {}+\biggl(\frac{\partial \theta _{2}}{\partial V}- \frac{\partial \theta _{1}}{\partial V} \biggr)\sigma _{1}u_{1} \beta V \sigma _{2}u_{2}kI=0. \end{aligned}$$

Analog for \(\theta_{2}\), then for \(\theta_{3}\) as follow

$$\begin{aligned}& \frac{\partial \theta _{3}(t,x)}{\partial t}+ \frac{\partial \theta _{3}(t,x)}{\partial T}\bigl[\Lambda - {\delta _{1}}T - (1 - u_{1} )\beta VT\bigr] \\& \quad {}+\frac{\partial \theta _{3}(t,x)}{\partial I}\bigl[(1 - u_{1} )\beta VT - { \delta _{2}}I\bigr]+\frac{\partial \theta _{3}(t,x)}{\partial V}\bigl[(1 - u_{2})kI - cV \bigr] \\& \quad {}+\frac{1}{2} \biggl(\frac{\partial \theta _{3}(t,x)}{\partial T}\sigma _{1}u_{1} \beta VT-\frac{\partial \theta _{3}(t,x)}{\partial I}\sigma _{1}u_{1} \beta VT- \frac{\partial \theta _{3}(t,x)}{\partial V}\sigma _{2}u_{2}kI \biggr)^{2} \\& \quad {}-\theta _{1}(t,x) (1 - u_{1})\beta T+\theta _{2}(t,x) (1 - u_{1})\beta T- \theta _{3}(t,x)c+c_{1}V \\& \quad {}+ \biggl(\frac{\partial \theta _{1}}{\partial T}- \frac{\partial \theta _{2}}{\partial T} \biggr) (\sigma _{1}u_{1} \beta T)^{2}V+ \biggl( \frac{\partial \theta _{2}}{\partial I}- \frac{\partial \theta _{1}}{\partial I} \biggr) (\sigma _{1}u_{1} \beta T)^{2}V \\& \quad {}+\biggl(\frac{\partial \theta _{2}}{\partial V}- \frac{\partial \theta _{1}}{\partial V} \biggr)\sigma _{1}u_{1} \beta T \sigma _{2}u_{2}kI=0 \end{aligned}$$

with the terminal condition

$$\begin{aligned} \theta _{1}(t_{f},x)=\theta _{2}(t_{f},x)= \theta _{3}(t_{f},x)=0. \end{aligned}$$

Step 3: Solve the partial differential equations systems (24)-(25) with the condition (23), thus we obtained \(\phi _{ij}(t,x)\), \(i,j=1,2,3\). Next, formed \(P(t)=\Phi (t,x(t))\).

Step 4: Substituting \(P(t)\) and \(Q(t)\) into equations (15), (16), and (17). Then, the first-order adjoining equation (10) is solved using a four-step scheme method, namely solving the system of partial differential equations (30)-(31) with terminal conditions (32), resulting in an optimal control \(u_{1}(t)\) and \(u_{2}(t)\). □

3 Numerical simulations

In this section, we carried out some numerical simulations to support our obtained theoritical result of the Model (2). We use a nonstandard finite difference method (NSFD) that is explained in [2426] to produce a reliable and consistent solution to biological nature. The parameter values chosen here are consistent with studies presented in [2729]. The parameters are \(\Lambda =1\), \(\beta =0.09\), \(\delta _{1}=0.2\), \(\delta _{2}=0.3\), \(k=2\), \(c=1.2\), \(c_{1}=100\), \(c_{2}=50\), \(c_{3}=100\), and initial values \(T(0)=3\), \(I(0)=2\), \(V(0)=1\) (arbitrary unit) and time t (arbitrary unit). We assume the parameters of intensity noise are \(\sigma _{1}=0.09\) and \(\sigma _{2}=0.0009\).

Figure 1 shows the solution of the System (2) with and without control. This figure illustrates the uncontrolled system trajectory tends to an infected equilibrium point \((T,I,V)=(2,2,3.3)\) where \(R_{0}=2.5>1\) after time \(t=20\) (arbitrary unit). It means that hepatitis C still persists. Meanwhile, the blue line in Figs. 1a, 1b, and 1c describe the solution of system (2) with control \(u_{1}\) and \(u_{2}\). We can see that the number of healthy cells gradually increases with optimal control until it reaches a constant value. We also observe that infected cells tend to zero and free viruses slightly oscillate. It can be seen that therapy can decrease the infected cells and free viruses, thus the spread of hepatitis C can be controlled. Figure 1c shows that the control can minimize the viral load. Based on Fig. 1d, the effectiveness is consistent after time \(t=20\).

Figure 1
figure 1

An epidemic of HCV with and without stochastic control for the System (2).

In Fig. 2, it can be seen that the number of healthy cells and infected cells are gradually increasing and decreasing, respectively. There are oscillations in the number of free viruses as the effect of stochastic control. In addition, Fig. 2d, 2e, and 2f show the control variable of the model when using one or both therapy. The effectiveness of drugs is consistent after 20 unit time. Regardless of the endemicity, when the \(R_{0}\) of the model without control is more than one, it means that hepatitis C persists. By considering the model with the control, hepatitis C will disappear over time.

Figure 2
figure 2

An epidemic of HCV with control \(u_{1}\), \(u_{2}\), and both.

4 Conclusions

In this paper, we have investigated the stochastic optimal control problem of the hepatitis C epidemic model, where we consider ribavirin and interferon treatment as control variables. By applying the stochastic minimum principles and a four-step scheme, we obtain the stochastic optimal solution for controlling the spread of hepatitis C. The results show the importance of considering stochastic factors in the spread of hepatitis C. The higher the noise intensity, the more variation in the control variable solution. It means that there are many variations in the optimal control values for the effectiveness of therapy. In addition, there is an effect when using the therapy of both types of drugs compared to therapy using one. Our findings showed that using both types of drugs was much more effective than using one.

In the future, research on optimal control systems that consider random factors can be developed, especially for diffusion coefficients containing control variables with Brownian motion multi-dimension.

Availability of data and materials

Not applicable.


  1. Burrell, C.J., Howard, C.R., Murphy, F.A.: Viral syndromes. In: Fenner and White’s Medical Virology, pp. 537–556 (2017).

    Chapter  Google Scholar 

  2. De Oliveira, T., Pybus, O.G., Rambaut, A., Salemi, M., Cassol, S., Ciccozzi, M., Rezza, G., Gattinara, G.C., D’Arrigo, R., Amicosante, M., et al.: HIV-1 and HCV sequences from Libyan outbreak. Nature 444, 836–837 (2006)

    Article  Google Scholar 

  3. Jafari, S., Copes, R., Baharlou, S., Etminan, M., Buxton, J.: Tattooing and the risk of transmission of hepatitis C: a systematic review and meta-analysis. Int. J. Infect. Dis. 14, e928–e940 (2010).

    Article  Google Scholar 

  4. Mast, E.E., Hwang, L.-Y., Seto, D.S., Nolte, F.S., Nainan, O.V., Wurtzel, H., Alter, M.J.: Risk factors for perinatal transmission of hepatitis C virus (HCV) and the natural history of hcv infection acquired in infancy. J. Infect. Dis. 192, 1880–1889 (2005).

    Article  Google Scholar 

  5. World Health Organization: Hepatitis C. The United Nations (2021). Accessed 1 September 2021.

  6. Hasan, I., Gani, R.A., Sulaiman, A.S., Lesmana, C.R.A., Kurniawan, J., Jasirwan, C.O.M.: Konsensus nasional penatalaksanaan hepatitis C di Indonesia, (National Consensus on Hepatitis C Management in Indonesia). Perhimpunan Peneliti Hati Indonesia, Jakarta (2017)

    Google Scholar 

  7. Neumann, A.U., Lam, N.P., Dahari, H., Gretch, D.R., Wiley, T.E., Layden, T.J., Perelson, A.S.: Hepatitis C viral dynamics in vivo and the antiviral efficacy of interferon-alpha therapy. Science 282, 103–107 (1998).

    Article  Google Scholar 

  8. Dustin, L., Bartolini, B., Capobianchi, M., Pistello, M.: Hepatitis C virus: life cycle in cells, infection and host response, and analysis of molecular markers influencing the outcome of infection and response to therapy. Clin. Microbiol. Infect. 22, 826–832 (2016).

    Article  Google Scholar 

  9. Peregrino, A., Esteva, L., Ble, G.: Optimal control applied to hepatitis C therapy considering immune system. J. Pure Appl. Math. Adv. Appl. 19, 9–35 (2018).

    Article  MATH  Google Scholar 

  10. Martin, N.K., Pitcher, A.B., Vickerman, P., Vassall, A., Hickman, M.: Optimal control of hepatitis C antiviral treatment programme delivery for prevention amongst a population of injecting drug users. PLoS ONE 6, e22309 (2011).

    Article  Google Scholar 

  11. Zhang, S., Xu, X.: Dynamic analysis and optimal control for a model of hepatitis C with treatment. Commun. Nonlinear Sci. Numer. Simul. 46, 14–25 (2017).

    Article  MathSciNet  MATH  Google Scholar 

  12. Din, A., Li, Y., Khan, F.M., Khan, Z.U., Liu, P.: On analysis of fractional order mathematical model of hepatitis B using Atangana–Baleanu Caputo (ABC) derivative. Fractals 30(1), 2240017 (2022).

    Article  MATH  Google Scholar 

  13. Gani, S.R., Halawar, S.V.: Optimal control analysis of deterministic and stochastic epidemic model with media awareness programs. Int. J. Optim. Control Theor. Appl. 9, 24–35 (2019).

    Article  MathSciNet  Google Scholar 

  14. Witbooi, P.J., Muller, G.E., Schalkwyk, G.J.V.: Vaccination control in a stochastic SVIR epidemic model. Comput. Math. Methods Med. 2015, Article ID 271654 1–9 (2015).

    Article  MathSciNet  MATH  Google Scholar 

  15. Din, A., Li, Y.: Stationary distribution extinction and optimal control for the stochastic hepatitis B epidemic model with partial immunity. Phys. Scr. 96(7), 074005 (2021).

    Article  Google Scholar 

  16. Ishikawa, M.: Stochastic optimal control of an SIR epidemic model with vaccination. In: Proceedings of the ISCIE International Symposium on Stochastic Systems Theory and Its Applications, vol. 2012, pp. 57–62 (2012).

    Chapter  Google Scholar 

  17. Gerold, G., Pietschmann, T.: The HCV life cycle: in vitro tissue culture systems and therapeutic targets. Dig. Dis. 32(5), 525–537 (2014).

    Article  Google Scholar 

  18. Driessche, P.V., Watmough, J.: Reproduction numbers and sub-threshold endemic equilibria for compartmental models of disease transmission. Math. Biosci. 180(2002), 29–48 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  19. Din, A.: The stochastic bifurcation analysis and stochastic delayed optimal control for epidemic model with general incidence function. Chaos, Interdiscip. J. Nonlinear Sci. 31(12), 123101 (2021).

    Article  MathSciNet  Google Scholar 

  20. Din, A., Li, Y.: Stochastic optimal control for norovirus transmission dynamics by contaminated food and water. Chin. Phys. B 31(2), 020202 (2022).

    Article  Google Scholar 

  21. Lestari, D., Megawati, N.Y., Susyanto, N., Adi-Kusumo, F.: Qualitative behaviour of a stochastic hepatitis C epidemic model in cellular level. Math. Biosci. Eng. 19(2), 1515–1535 (2022).

    Article  MathSciNet  MATH  Google Scholar 

  22. Øksendal, B., Sulem, A., Zhang, T.: Optimal control of stochastic delay equations and time-advanced backward stochastic differential equations. Adv. Appl. Probab. 43, 572–596 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  23. Yong, J., Zhou, X.Y.: Stochastic Controls: Hamiltonian Systems and HJB Equations, vol. 43. Springer, New York (1999)

    Book  MATH  Google Scholar 

  24. Raza, A., Rafiq, M., Awrejcewicz, J., Ahmed, N., Mohsin, M.: Dynamical analysis of coronavirus disease with crowding effect, and vaccination: a study of third strain. Nonlinear Dyn. 107(4), 3963–3982 (2022)

    Article  Google Scholar 

  25. Raza, A., Awrejcewicz, J., Rafiq, M., Ahmed, N., Mohsin, M.: Stochastic analysis of nonlinear cancer disease model through virotherapy and computational methods. Mathematics 10(3), 368 (2022)

    Article  Google Scholar 

  26. Raza, A., Awrejcewicz, J., Rafiq, M., Ahmed, N., Ehsan, M.S., Mohsin, M.: Dynamical analysis and design of computational methods for nonlinear stochastic leprosy epidemic model. Alex. Eng. J. 61(10), 8097–8111 (2022)

    Article  Google Scholar 

  27. Wodarz, D.: Hepatitis C virus dynamics and pathology: the role of CTL and antibody responses. J. Gen. Virol. 84, 1743–1750 (2003).

    Article  Google Scholar 

  28. Wodarz, D., Jansen, V.A.: A dynamical perspective of CTL cross-priming and regulation: implications for cancer immunology. Immunol. Lett. 86, 213–227 (2003).

    Article  Google Scholar 

  29. Hu, X., Li, J., Feng, X.: Threshold dynamics of a HCV model with virus to cell transmission in both liver with CTL immune response and the extrahepatic tissue. J. Biol. Dyn. 15, 19–34 (2021).

    Article  MathSciNet  MATH  Google Scholar 

Download references


The first author thanks LPDP Indonesia for the financial support under the Doctoral Program Scholarship. The authors thank to Department of Mathematics, Universitas Gadjah Mada (UGM) and Directorate for the Higher Education, Ministry of Research, Technology, and Higher Education of Indonesia for the Research Grant “Penelitian Disertasi Doktor”(PDD) UGM 2022 under grant no.1738/UN1/DITLIT/Dit-Lit/PT.01.03/2022. Special thanks are also due to some colleagues for their invaluable inputs in the discussions during the research.


The paper was financially supported by Universitas Gadjah Mada’s (UGM) Department of Mathematics, and the Directorate for the Higher Education of the Ministry of Research, Technology, and Higher Education of Indonesia through a Research Grant “Penelitian Disertasi Doktor”(PDD) UGM 2022 number 1738/UN1/DITLIT/Dit-Lit/PT.01.03/2022.

Author information

Authors and Affiliations



All authors contributed equally to the manuscript, read and approved the final manuscript.

Corresponding author

Correspondence to Dwi Lestari.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lestari, D., Adi-Kusumo, F., Megawati, N.Y. et al. A minimum principle for stochastic control of hepatitis C epidemic model. Bound Value Probl 2023, 52 (2023).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI:


  • Optimal control
  • Hepatitis C virus
  • Therapy
  • The stochastic minimum principle
  • A four-step scheme