A minimum principle for stochastic control of hepatitis C epidemic model

Abstract

In this paper, we discussed a stochastic optimal control of hepatitis C that minimizes the side effect and reduces the viral load. The control variables represent the drug therapy used for blocking a new infection and virus production. The solution of control problem is solved using the stochastic minimum principle and a four-step scheme. The numerical simulation is carried out to justify the theoretical analysis. The result shows that using both types of drugs for therapy is much more effective.

1 Introduction

Hepatitis C is an inflammatory liver disease caused by the hepatitis C virus (HCV) that can lead to cirrhosis and/or hepatocellular cancer [1]. Hepatitis C is a bloodborne disease, which is mostly transmitted through contaminated needles used for dialysis, tattoos, piercings, and unclean blood transfusion [2–4]. HCV infection does not always require treatment since the immune response can clear the infection. However, when hepatitis C becomes chronic, treatment is needed to cure the disease. WHO recommends using direct-acting antiviral (DAA) therapy [5].

Certain variants are associated with a higher potential for resistance or therapeutic failure. The DAA regimen faces a risk of resistance when administered to a group of infected patients with a special variant known as the resistant associated variant (RAV). Thus, in every case of suspected resistance or failure of therapy with the DAA regimen, it is necessary to examine resistance genotypes. The results of this examination will be considered for therapy modification, either by adding the duration, other agents, especially ribavirin, or by substituting the regimen [6].

In the past 10 years, there have been two types of standard therapies for chronic hepatitis C that are often used in the treatment of hepatitis C, namely a combination of interferon and ribavirin. This therapy gave satisfactory results, i.e., in patients with HCV genotypes 2 and 3, around 80% of patients could achieve sustained virological response (SVR24), while in HCV genotype 1, only 40-50% of patients managed to achieve SVR24 [6]. Based on the basic HCV model provided by Neumann [7], Dustin et al. [8] and Hasan et al. [6], they explained that the type and duration of therapy depend on the type of HCV genotype. Thus, giving the same therapy to different patients can result in different effects. This phenomenon will be modeled as stochastic effects.

Furthermore, a combination of interferon and ribavirin also has side effects such as anemia, drowsiness, indigestion, and shortness of breath [9]. Therefore, in this paper, a stochastic optimal control problem is established to determine the optimal drug effectiveness to treat hepatitis C. The objective function is minimizing viral load and drug side effects.

Several papers have discussed the optimal control of hepatitis C. Martin et al. in [10] had determined the optimal therapy program for hepatitis C patients over 10 years, considering biomedical and economic goals. In this case, the minimum cost is determined to reduce viral load and the cost of secondary effects caused during therapy. Meanwhile, Peregrino et al. [9] considered optimal control of cellular level HCV with immune response, and Zhang et al. [11] proposed optimal control of HCV model with treatment, both of whom used the maximum Pontryagin principle. Their research focused on a deterministic model, whereas we developed a stochastic optimal control model in this study. We could investigate the effects of uncertainty in the optimal control of the hepatitis C problem. Another research that analyzes the fractional order mathematical model of hepatitis B is studied in [12].

In [13–15], they discussed about an optimal control problem in deterministic and stochastic epidemic models using the Hamilton-Jacobi equation. Meanwhile, Ishikawa [16] designed the optimal control of the SIR model using the maximum stochastic principle and the four-step scheme technique. However, researchers have not discussed the optimal control problem where the control variable is contained in the diffusion coefficient. In this article, we present the new results of the stochastic optimal control where the control variable is contained in the diffusion coefficient. The optimal control problem is solved using the Hamilton-Jacobi-Bellman (HJB) equation, which is a nonlinear function and challenging to solve numerically. In this paper, the minimum stochastic principle and the four-step scheme will be applied to the stochastic control of the hepatitis C epidemic model. Based on medical literature [7], the optimal solution determines the effectiveness of drugs related to administer drug doses during therapy.

The rest of this paper is organized as follows. Section 2 presents the mathematical modeling for hepatitis C with a stochastic control and the analysis. Then, in Sect. 3, we carried out numerical simulations to verify theoretical results. Finally, in the last section, we present a conclusion.

2 HCV stochastic control model with a diffusion coefficient that contains control variables

HCV initially infects target cells, proliferates inside them, and then gets released into the extracellular space without disrupting the cell integrity. During its life cycle, HCV usurps host cell molecules, termed host factors, and various cell biological mechanisms ranging from endocytosis to the secretory pathway. The HCV life cycle can be separated into four steps: (1) virus entry; (2) genome translation and polyprotein processing; (3) genome replication and (4) particle assembly and release from the host cell. Virions are released from the cells most likely by exocytosis or transmitted to other cells via a cell-free mechanism [17].

Recall a deterministic mathematical model of HCV given in [7] as

$$\begin{aligned}& \frac{dT}{dt} = {\Lambda - {\delta _{1}}T - (1 - \eta )\beta VT}, \\& \frac{dI}{dt} = {(1 - \eta )\beta VT - {\delta _{2}}I}, \\& \frac{dV}{dt} = {(1 - \epsilon )kI - cV.} \end{aligned}$$

(1)

The cell population $N(t)$ is categorized into three sub-populations. Let $T(t)$ be the number of healthy or uninfected cells, $I(t)$ be the number of infected cells, and $V(t)$ be the number of free viruses. The uninfected cells are produced at rate Λ and die naturally at a constant rate $\delta _{1}$. Cells become infected when they interact with the virus at a constant rate β; once infected, they will die at a constant rate $\delta _{2}$. HCV is produced by infected cells at a constant rate k and cleared at a constant rate c. Parameter η is the effectiveness of a drug in stopping new infection and ϵ is the effectiveness of a drug in blocking virus production.

For the deterministic Model (1), there are two equilibrium points, namely the disease-free equilibrium point $E_{0}=(\frac{\Lambda}{\delta _{1}},0,0)$ and the endemic equilibrium point $E_{1}=(\frac{\delta _{2}c}{(1-\eta )(1-\epsilon )k\beta}, \frac{\Lambda}{\delta _{2}}- \frac{\delta _{1}c}{(1-\eta )(1-\epsilon )k\beta}, \frac{(1-\epsilon )k\Lambda}{\delta _{2}c}- \frac{\delta _{1}}{(1-\eta )\beta})$. By using the next generation matrix method [18], the basic reproduction number for the deterministic model or uncontrolled system is $R_{0}= \frac{(1-\eta )(1-\epsilon )\Lambda \beta k}{c\delta _{1}\delta _{2}}$.

We defined two controls variables as follows

• 1 $u_{1}(t)$ represents the control variable to block new infection.

• 2 $u_{2}(t)$ represents the control variable of hepatitis C to block the virus production.

We assumed that the control variables are stochastic, i.e., $u_{1}(t)=u_{1}+\sigma _{1}u_{1}\,dB(t)/dt$ and $u_{2}(t)= u_{2}+\sigma _{2}u_{2}\,dB(t)/dt$, where $B(t)$ is standard Brownian motion. The stochastic control system model is given by

$$\begin{aligned} \textstyle\begin{cases} {dT = ( {\Lambda - {\delta _{1}}T - (1 - u_{1} )\beta VT} )\,dt + \sigma _{1}u_{1} \beta VT\,dB(t)}, \\ {dI = ( {(1 - u_{1} )\beta VT - {\delta _{2}}I} )\,dt - \sigma _{1}u_{1} \beta VT\,dB(t)}, \\ {dV = ( {(1 - u_{2})kI - cV} )\,dt - \sigma _{2}u_{2}kI \,dB(t)}, \end{cases}\displaystyle \end{aligned}$$

(2)

where

$$\begin{aligned} U=\bigl\{ (u_{1},u_{2}):0\leq u_{1},u_{2} < 1\bigr\} , \end{aligned}$$

(3)

is the set of control variables and $u_{1}$, $u_{2}$ is $\{\mathcal{F}_{t}\}$-adapted. If $\sigma _{1}=\sigma _{2}=0$, then System (2) becomes a deterministic model studied in [7], where $u_{1}=\eta$ and $u_{2}=\epsilon$.

System (2) represents the dynamics of a biological cell population. Therefore, the number of cell populations should be non-negative and bounded. For this reason, we first established the global existence, positivity, and boundedness of solutions for the proposed model in the following theorem.

Theorem 2.1

For any initial value in ${\mathbb{R}}^{3} _{+}$, the System (2) is uniformly and ultimately bounded and belongs to the closed and bounded positively invariant set for every $t\geq 0$. The coefficients of the System (2) satisfy the Lipschitz condition. Then there exists a unique time-global solution $(T(t),I(t),V(t)) \subset {\mathbb{R}}^{3}_{+}$, $t \ge 0$ with probability 1.

Proof

Consider System (2), it can be seen that the coefficients of the System (2) satisfy the Lipschitz condition thus guarantee the solution is unique and local. Next, it can be proved that the explosion time is infinity (i.e., the time when the solution tends to infinity), thus the solution is global. The detailed proof is analog to [13, 19–21]. □

In our control problem, objective is to reduce the viral load and to minimize the therapy cost. This objective function/cost function is defined as follows

$$\begin{aligned} J(u) =&E \biggl[{ \int _{0}^{t_{f}} L\bigl(x(t),u(t)\bigr) \,dt} \biggr] \\ =&E \biggl[{ \int _{0}^{t_{f}}\frac{1}{2}c_{1}V(t)^{2}+ \frac{1}{2}c_{2}u_{1}(t)^{2}+ \frac{1}{2}c_{3}u_{2}(t)^{2}\,dt} \biggr]. \end{aligned}$$

(4)

The first term of equation (4) represents the main biological target, i.e., reducing the viral load. The constant $c_{1}$ is related to the cost for lowering viral load, $c_{2}$ and $c_{3}$ are related to the cost of therapy. Since the integral is a random variable, then we need to take the expected value. Thus, the optimal control problem is determining optimal control $u_{1}^{*}$, $u_{2}^{*}$ that minimizes the objective function $J(u)$.

The existence of the optimal control pair can be obtained using a result by [22].

Theorem 2.2

There is an optimal control $(u_{1}^{*}(t),u_{2}^{*}(t))$ such that

$$\begin{aligned} J\bigl(u_{1}^{*},u_{2}^{*}\bigr)=min \bigl\{ J(u_{1},u_{2}):u=(u_{1},u_{2})\in U\bigr\} . \end{aligned}$$

(5)

The optimal control variable can then be determined in the following theorem.

Theorem 2.3

The optimal control $u_{1}^{*}(t)$ and $u_{2}^{*}(t)$ of the System (2), which minimize the objective function (4) are characterized by

$$\begin{aligned}& u_{1}(t)^{*}=\min \bigl\{ max \{0,\bar{u}_{1} \},1 \bigr\} , \\& u_{2}(t)^{*}=\min \bigl\{ max\{0,\bar{u}_{2}\},1 \bigr\} . \end{aligned}$$

where

$$\begin{aligned}& \bar{u}_{1}= \frac{(p_{2}-p_{1})\beta VT+(q_{2}-q_{1})\beta VT}{c_{2}-(\sigma _{1}^{2}\beta ^{2}\bar{V}V\bar{T}T)(\phi _{11}-\phi _{21}+\phi _{22}-\phi _{12})}, \\& \bar{u}_{2}= \frac{p_{3}kI+q_{3}\sigma _{2}kI}{c_{3}-\sigma _{2}^{2}k^{2}\phi _{33}\bar{I}I}. \end{aligned}$$

Proof

In order to construct the optimal control $(u_{1}^{*}(t),u_{2}^{*}(t))$, defined

$$\begin{aligned}& f_{1} = \Lambda - {\delta _{1}}T - (1 - u_{1} ) \beta VT, \\& f_{2} = (1 - u_{1} )\beta VT - {\delta _{2}}I, \\& f_{3} = (1 - u_{2})kI - cV, \\& g_{1} = \sigma _{1}u_{1} \beta VT,\qquad g_{2}= -\sigma _{1}u_{1} \beta VT,\qquad g_{3}= -\sigma _{2}u_{2}kI. \end{aligned}$$

(6)

and we constructed Hamiltonian function as follows

$$\begin{aligned} H(\mathbf{x},\mathbf{u},\mathbf{p},\mathbf{q})= \bigl\langle \mathbf{f}( \mathbf{x}, \mathbf{u}),\mathbf{p} \bigr\rangle + L(\mathbf{x},\mathbf{u}) + \bigl\langle \mathbf{g}( \mathbf{x}),\mathbf{q} \bigr\rangle \end{aligned}$$

(7)

where $\langle \cdot ,\cdot \rangle$ stands for the Euclidian inner product and

$$\begin{aligned}& \mathbf{f}:\mathbb{R}^{3}\times U \rightarrow \mathbb{R}^{3},\qquad \mathbf{g}:\mathbb{R}^{3}\rightarrow \mathbb{R}^{3},\qquad \mathbf{p}: \mathbb{R}\rightarrow \mathbb{R}^{3}, \\& \mathbf{q}:\mathbb{R} \rightarrow \mathbb{R}^{3},\qquad L:\mathbb{R}^{3}\times U \rightarrow \mathbb{R}, \\& \mathbf{f}=(f_{1},f_{2},f_{3})^{t},\qquad \mathbf{p}=(p_{1},p_{2},p_{3})^{t},\qquad \mathbf{q}=(q_{1},q_{2},q_{3})^{t},\qquad \mathbf{x}=(T,I,V)^{t}. \end{aligned}$$

(8)

Vector p and q denote the adjoint vectors. Thus, we have

$$\begin{aligned} H(\mathbf{x},\mathbf{u},\mathbf{p},\mathbf{q}) =& \bigl\langle \mathbf{f}( \mathbf{x}, \mathbf{u}),\mathbf{p} \bigr\rangle + L(\mathbf{x},\mathbf{u}) + \bigl\langle \mathbf{g}( \mathbf{x}),\mathbf{q} \bigr\rangle \\ =& p_{1}\bigl[\Lambda - {\delta _{1}}T - (1 - u_{1} )\beta VT\bigr]+p_{2}\bigl[(1 - u_{1} ) \beta VT - {\delta _{2}}I\bigr] \\ &{} + p_{3}\bigl[(1 - u_{2})kI - cV\bigr] + \frac{1}{2}c_{1}V(t)^{2}+\frac{1}{2}c_{2}u_{1}(t)^{2}+ \frac{1}{2}c_{3}u_{2}(t)^{2} \\ &{} + q_{1}\sigma _{1}u_{1} \beta VT -q_{2}\sigma _{1}u_{1} \beta VT-q_{3} \sigma _{2}u_{2} kI. \end{aligned}$$

(9)

Based on the stochastic minimum principle [23], then we get the following system of differential equations

$$\begin{aligned} \textstyle\begin{cases} {dT = ( {\Lambda - {\delta _{1}}T - (1 - u_{1} )\beta VT} )\,dt + \sigma _{1}u_{1} \beta VTdB(t)}, \\ {dI = ( {(1 - u_{1} )\beta VT - {\delta _{2}}I} )\,dt - \sigma _{1}u_{1} \beta VTdB(t)}, \\ {dV = ( {(1 - u_{2})kI - cV} )\,dt - \sigma _{2}u_{2}kI \,dB(t)}, \\ dp_{1}=[p_{1}\delta _{1}+p_{1}(1-u_{1})\beta V - p_{2}(1-u_{1})\beta V - q_{1}\sigma _{1}u_{1}\beta V \\ \hphantom{dp_{1}=}{} +q_{2}\sigma _{1}u_{1}\beta V]\,dt + q_{1}\,dB(t), \\ dp_{2}= [p_{2}\delta _{2}-p_{3}(1-u_{2})k + q_{3}\sigma _{2}u_{2}k]\,dt + q_{2}\,dB(t), \\ dp_{3}= [p_{1}(1-u_{1})\beta T -p_{2}(1-u_{1})\beta T + p_{3}c -c_{1}V -q_{1}\sigma _{1} u_{1}\beta T \\ \hphantom{dp_{3}= }{} + q_{2}\sigma _{1}u_{1} \beta T]\,dt +q_{3}\,dB(t), \\ \mathbf{x}(0)=(T_{0},I_{0},V_{0})^{t}, \qquad \mathbf{p}(t_{f})=(0,0,0)^{t}. \end{cases}\displaystyle \end{aligned}$$

(10)

According to [23], the controller can balance the scale of control and the degree of uncertainty if the diffusion coefficient contains control variables in the stochastic situation. Therefore, we need to define a New Hamiltonian function since the marginal value alone may not be able to fully characterize the trade-off between the cost and control gain in an uncertain environment. The New Hamiltonian function is given as follows

$$\begin{aligned} \mathcal{H}(t,x,u) =& H(t,x,u)-\frac{1}{2} trace\bigl[g(t,x,u)^{t}Pg(t,x,u) \bigr] \\ &{}+\frac{1}{2} trace \bigl\{ \bigl[g(t,x,u)-g(t,\bar{x},\bar{u}) \bigr]^{t} P\bigl[g(t,x,u)-g(t, \bar{x},\bar{u})\bigr]\bigr\} \\ =& H(t,x,u)-\frac{1}{2}\bigl[(P_{11}-P_{21}+P_{22}-P_{12}) (\sigma _{1}u_{1} \beta VT)^{2} \\ &{}+\sigma _{2}^{2}u_{2}^{2}k^{2}I^{2}P_{33} \bigr]+\frac{1}{2}\bigl[(P_{11}-P_{21}+P_{22}-P_{12}) ( \sigma _{1}\beta )^{2} \\ &{}\times(u_{1}VT-\bar{u}_{1}\bar{V}\bar{T})^{2} + ( \sigma _{2}k)^{2}(u_{2}I- \bar{u}_{2} \bar{I})^{2}P_{33}\bigr]. \end{aligned}$$

(11)

The value of $\mathcal{H}$ is optimal if the following conditions are satisfied

$$\begin{aligned} \frac{\partial \mathcal{H}}{\partial u_{1}}=0 \quad \text{and}\quad \frac{\partial \mathcal{H}}{\partial u_{2}}=0, \end{aligned}$$

(12)

which give

$$\begin{aligned}& \bar{u}_{1}= \frac{(p_{2}-p_{1})\beta VT+(q_{2}-q_{1})\beta VT}{c_{2}-(\sigma _{1}^{2}\beta ^{2}\bar{V}V\bar{T}T)(P_{11}-P_{21}+P_{22}-P_{12})}, \end{aligned}$$

(13)

$$\begin{aligned}& \bar{u}_{2}= \frac{p_{3}kI+q_{3}\sigma _{2}kI}{c_{3}-\sigma _{2}^{2}k^{2}P_{33}\bar{I}I}. \end{aligned}$$

(14)

Since u should be in the control space U, then we have

$$\begin{aligned}& u_{1}(t) = \min \bigl\{ max \{0,\bar{u}_{1} \},1 \bigr\} , \end{aligned}$$

(15)

$$\begin{aligned}& u_{2}(t) = \min \bigl\{ max\{0,\bar{u}_{2}\},1\bigr\} . \end{aligned}$$

(16)

Furthermore, we consider the second order adjoining equation as follows

$$\begin{aligned}& \begin{aligned} dP&=- \bigl\{ f_{x}^{t}P(t) + P(t)f_{x} + (g_{x})^{t}P(t) (g_{x})+ (g_{x})^{t} \bigl(Q(t)\bigr) \\ &\quad {} + Q(t) (g_{x})+ H_{xx} \bigr\} \,dt+Q(t)\,dB(t), \end{aligned} \end{aligned}$$

(17)

$$\begin{aligned}& P(t_{f})= 0, \end{aligned}$$

(18)

with P and Q are $n\times n$ matrices. The equations (10) and (17) are called forward-backward stochastic differential equations(FBSDE) with terminal conditions (18). Next, we will apply the four-step scheme method to solve the problem (17).

Step 1: Assume that $P(t)$ and $\mathbf{x}(t)$ are related, i.e.,

$$\begin{aligned} P(t)=\Phi \bigl(t, \mathbf{x}(t)\bigr), \end{aligned}$$

(19)

where $\Phi = ( {\phi _{ij}}) _{n\times n}$ is $n \times n$ matrix and $\phi _{ij}$ is vector-valued function with $i,j=1,2,\ldots,n$

Step 2: By using Itô’s formula, we obtain

$$\begin{aligned} d\phi _{ij}(t,x) =& \biggl\{ \frac{\partial \phi _{ij}(t,x)}{\partial t}+ \biggl\langle \frac{\partial \phi _{ij}(t,x)}{\partial x},f(x,u) \biggr\rangle \\ &{}+ \frac{1}{2} trace \biggl[\frac{\partial}{\partial x} \biggl( \frac{\partial \phi _{ji}(t,x)}{\partial x} \biggr) g(x)g(x)^{t} \biggr] \biggr\} \,dt \\ &{} + \biggl\langle \frac{\partial \phi _{ij}(t,x)}{\partial x},g(x) \biggr\rangle \,dB(t). \end{aligned}$$

(20)

Since $P(t)=\phi (t, x(t))$ and u is a function of p, q and x, then we have

$$\begin{aligned}& \frac{\partial \phi _{ij}(t,x)}{\partial t}+ \biggl\langle \frac{\partial \phi _{ij}(t,x)}{\partial x},f(x,\phi ,q) \biggr\rangle + \Delta \\& \quad {}+\frac{1}{2} trace \biggl[\frac{\partial}{\partial x} \biggl( \frac{\partial \phi _{ij}(t,x)}{\partial x} \biggr) g(x)g(x)^{t} \biggr] = 0 \end{aligned}$$

(21)

and

$$\begin{aligned} Q(t)= \biggl\langle \frac{\partial \phi _{ij}(t,x)}{\partial x},g(x) \biggr\rangle \end{aligned}$$

(22)

with the terminal conditions

$$\begin{aligned} \phi _{ij}(t_{f},x)=0,\quad i,j=1,2,3 \end{aligned}$$

(23)

and Δ is the coefficient dt of (17). Furthermore, the system of partial differential equations in equation (21) is obtained as follows

$$\begin{aligned}& \frac{\partial \phi _{11}(t,x)}{\partial t}+ \frac{\partial \phi _{11}(t,x)}{\partial T}\bigl[\Lambda - {\delta _{1}}T - (1 - u_{1} )\beta VT\bigr] \\& \quad {}+\frac{\partial \phi _{11}(t,x)}{\partial I}\bigl[(1 - u_{1} )\beta VT - { \delta _{2}}I\bigr]+\frac{\partial \phi _{11}(t,x)}{\partial V}\bigl[(1 - u_{2})kI - cV \bigr] \\& \quad {}+\frac{1}{2} \biggl(\frac{\partial \phi _{11}(t,x)}{\partial T}\sigma _{1}u_{1} \beta VT -\frac{\partial \phi _{11}(t,x)}{\partial I}\sigma _{1} u_{1} \beta VT- \frac{\partial \phi _{11}(t,x)}{\partial V}\sigma _{2} u_{2}kI \biggr)^{2} \\& \quad {}+\bigl(-2\delta _{1}\phi _{11}+(\phi _{21}-\phi _{11}) (1-u_{1})\beta V+( \phi _{12}-\phi _{11}) (1-u_{1})\beta V \\& \quad {}+(\phi _{11}-\phi _{21}+\phi _{22}-\phi _{12}) (\sigma _{1}u_{1}\beta V)^{2} \bigr) \\& \quad {}+(Q_{11}-Q_{21})\sigma _{1}u_{1}\beta V +(Q_{11}-Q_{12}\sigma _{1}u_{1} \beta V)=0. \end{aligned}$$

(24)

Analog for $\phi _{ij}$ where $i,j=1,2,3$.

$$\begin{aligned}& \frac{\partial \phi _{33}(t,x)}{\partial t}+ \frac{\partial \phi _{33}(t,x)}{\partial T}\bigl[\Lambda - {\delta _{1}}T - (1 - u_{1} )\beta VT\bigr] \\& \quad {}+\frac{\partial \phi _{33}(t,x)}{\partial I}\bigl[(1 - u_{1} )\beta VT - { \delta _{2}}I\bigr]+\frac{\partial \phi _{33}(t,x)}{\partial V}\bigl[(1 - u_{2})kI - cV \bigr] \\& \quad {}+\frac{1}{2} \biggl(\frac{\partial \phi _{33}(t,x)}{\partial T}\sigma _{1}u_{1} \beta VT -\frac{\partial \phi _{33}(t,x)}{\partial I}\sigma _{1} u_{1} \beta VT- \frac{\partial \phi _{33}(t,x)}{\partial V}\sigma _{2} u_{2}kI \biggr)^{2} \\& \quad {}+\bigl(-2c\phi _{33}+(\phi _{23}-\phi _{13}) (1-u_{1})\beta T+(\phi _{32}- \phi _{31}) (1-u_{1})\beta T \\& \quad {}+(\phi _{11}-\phi _{21}+\phi _{22}-\phi _{12}) (\sigma _{1}u_{1}\beta T)^{2} \bigr) \\& \quad {}+(Q_{13}-Q_{23})\sigma _{1}u_{1}\beta T +(Q_{31}-Q_{32})\sigma _{1}u_{1} \beta T +c_{1} =0. \end{aligned}$$

(25)

Meanwhile, from equation (13) and (14), we have

$$\begin{aligned}& \bar{u}_{1}= \frac{(p_{2}-p_{1})\beta VT+(q_{2}-q_{1})\beta VT}{c_{2}-(\sigma _{1}^{2}\beta ^{2}\bar{V}V\bar{T}T)(\phi _{11}-\phi _{21}+\phi _{22}-\phi _{12})}, \end{aligned}$$

(26)

$$\begin{aligned}& \bar{u}_{2}= \frac{p_{3}kI+q_{3}\sigma _{2}kI}{c_{3}-\sigma _{2}^{2}k^{2}\phi _{33}\bar{I}I}. \end{aligned}$$

(27)

By using the property of the control space U in (3), we obtain

$$\begin{aligned}& u_{1}(t)^{*}=\min \bigl\{ max \{0,\bar{u}_{1} \},1 \bigr\} , \end{aligned}$$

(28)

$$\begin{aligned}& u_{2}(t)^{*}=\min \bigl\{ max\{0,\bar{u}_{2}\},1 \bigr\} . \end{aligned}$$

(29)

Analogously, for the first-order adjoint equation (10), the four-step scheme method is also applied, and then the system of partial differential equations is given as follows

$$\begin{aligned}& \frac{\partial \theta _{1}(t,x)}{\partial t}+ \frac{\partial \theta _{1}(t,x)}{\partial T}\bigl[\Lambda - {\delta _{1}}T - (1 - u_{1} )\beta VT\bigr] \\& \quad {}+\frac{\partial \theta _{1}(t,x)}{\partial I}\bigl[(1 - u_{1} )\beta VT - { \delta _{2}}I\bigr]+\frac{\partial \theta _{1}(t,x)}{\partial V}\bigl[(1 - u_{2})kI - cV \bigr] \\& \quad {}+\frac{1}{2} \biggl(\frac{\partial \theta _{1}(t,x)}{\partial T}\sigma _{1}u_{1} \beta VT-\frac{\partial \theta _{1}(t,x)}{\partial I}\sigma _{1}u_{1} \beta VT- \frac{\partial \theta _{1}(t,x)}{\partial V}\sigma _{2}u_{2}kI \biggr)^{2} \\& \quad {}-\theta _{1}(t,x)\delta _{1}-\theta _{1}(t,x) (1 - u_{1} )\beta V+ \theta _{2}(t,x) (1 - u_{1} ) \beta V \\& \quad {}+\biggl(\frac{\partial \theta _{1}}{\partial T}- \frac{\partial \theta _{2}}{\partial T} \biggr) (\sigma _{1}u_{1} \beta V)^{2}T+ \biggl( \frac{\partial \theta _{2}}{\partial I}- \frac{\partial \theta _{1}}{\partial I} \biggr) (\sigma _{1}u_{1} \beta V)^{2}T \\& \quad {}+\biggl(\frac{\partial \theta _{2}}{\partial V}- \frac{\partial \theta _{1}}{\partial V} \biggr)\sigma _{1}u_{1} \beta V \sigma _{2}u_{2}kI=0. \end{aligned}$$

(30)

Analog for $\theta_{2}$, then for $\theta_{3}$ as follow

$$\begin{aligned}& \frac{\partial \theta _{3}(t,x)}{\partial t}+ \frac{\partial \theta _{3}(t,x)}{\partial T}\bigl[\Lambda - {\delta _{1}}T - (1 - u_{1} )\beta VT\bigr] \\& \quad {}+\frac{\partial \theta _{3}(t,x)}{\partial I}\bigl[(1 - u_{1} )\beta VT - { \delta _{2}}I\bigr]+\frac{\partial \theta _{3}(t,x)}{\partial V}\bigl[(1 - u_{2})kI - cV \bigr] \\& \quad {}+\frac{1}{2} \biggl(\frac{\partial \theta _{3}(t,x)}{\partial T}\sigma _{1}u_{1} \beta VT-\frac{\partial \theta _{3}(t,x)}{\partial I}\sigma _{1}u_{1} \beta VT- \frac{\partial \theta _{3}(t,x)}{\partial V}\sigma _{2}u_{2}kI \biggr)^{2} \\& \quad {}-\theta _{1}(t,x) (1 - u_{1})\beta T+\theta _{2}(t,x) (1 - u_{1})\beta T- \theta _{3}(t,x)c+c_{1}V \\& \quad {}+ \biggl(\frac{\partial \theta _{1}}{\partial T}- \frac{\partial \theta _{2}}{\partial T} \biggr) (\sigma _{1}u_{1} \beta T)^{2}V+ \biggl( \frac{\partial \theta _{2}}{\partial I}- \frac{\partial \theta _{1}}{\partial I} \biggr) (\sigma _{1}u_{1} \beta T)^{2}V \\& \quad {}+\biggl(\frac{\partial \theta _{2}}{\partial V}- \frac{\partial \theta _{1}}{\partial V} \biggr)\sigma _{1}u_{1} \beta T \sigma _{2}u_{2}kI=0 \end{aligned}$$

(31)

with the terminal condition

$$\begin{aligned} \theta _{1}(t_{f},x)=\theta _{2}(t_{f},x)= \theta _{3}(t_{f},x)=0. \end{aligned}$$

(32)

Step 3: Solve the partial differential equations systems (24)-(25) with the condition (23), thus we obtained $\phi _{ij}(t,x)$, $i,j=1,2,3$. Next, formed $P(t)=\Phi (t,x(t))$.

Step 4: Substituting $P(t)$ and $Q(t)$ into equations (15), (16), and (17). Then, the first-order adjoining equation (10) is solved using a four-step scheme method, namely solving the system of partial differential equations (30)-(31) with terminal conditions (32), resulting in an optimal control $u_{1}(t)$ and $u_{2}(t)$. □

3 Numerical simulations

In this section, we carried out some numerical simulations to support our obtained theoritical result of the Model (2). We use a nonstandard finite difference method (NSFD) that is explained in [24–26] to produce a reliable and consistent solution to biological nature. The parameter values chosen here are consistent with studies presented in [27–29]. The parameters are $\Lambda =1$, $\beta =0.09$, $\delta _{1}=0.2$, $\delta _{2}=0.3$, $k=2$, $c=1.2$, $c_{1}=100$, $c_{2}=50$, $c_{3}=100$, and initial values $T(0)=3$, $I(0)=2$, $V(0)=1$ (arbitrary unit) and time t (arbitrary unit). We assume the parameters of intensity noise are $\sigma _{1}=0.09$ and $\sigma _{2}=0.0009$.

Figure 1 shows the solution of the System (2) with and without control. This figure illustrates the uncontrolled system trajectory tends to an infected equilibrium point $(T,I,V)=(2,2,3.3)$ where $R_{0}=2.5>1$ after time $t=20$ (arbitrary unit). It means that hepatitis C still persists. Meanwhile, the blue line in Figs. 1a, 1b, and 1c describe the solution of system (2) with control $u_{1}$ and $u_{2}$. We can see that the number of healthy cells gradually increases with optimal control until it reaches a constant value. We also observe that infected cells tend to zero and free viruses slightly oscillate. It can be seen that therapy can decrease the infected cells and free viruses, thus the spread of hepatitis C can be controlled. Figure 1c shows that the control can minimize the viral load. Based on Fig. 1d, the effectiveness is consistent after time $t=20$.

Figure 1

An epidemic of HCV with and without stochastic control for the System (2).

In Fig. 2, it can be seen that the number of healthy cells and infected cells are gradually increasing and decreasing, respectively. There are oscillations in the number of free viruses as the effect of stochastic control. In addition, Fig. 2d, 2e, and 2f show the control variable of the model when using one or both therapy. The effectiveness of drugs is consistent after 20 unit time. Regardless of the endemicity, when the $R_{0}$ of the model without control is more than one, it means that hepatitis C persists. By considering the model with the control, hepatitis C will disappear over time.

Figure 2

An epidemic of HCV with control $u_{1}$, $u_{2}$, and both.

4 Conclusions

In this paper, we have investigated the stochastic optimal control problem of the hepatitis C epidemic model, where we consider ribavirin and interferon treatment as control variables. By applying the stochastic minimum principles and a four-step scheme, we obtain the stochastic optimal solution for controlling the spread of hepatitis C. The results show the importance of considering stochastic factors in the spread of hepatitis C. The higher the noise intensity, the more variation in the control variable solution. It means that there are many variations in the optimal control values for the effectiveness of therapy. In addition, there is an effect when using the therapy of both types of drugs compared to therapy using one. Our findings showed that using both types of drugs was much more effective than using one.

In the future, research on optimal control systems that consider random factors can be developed, especially for diffusion coefficients containing control variables with Brownian motion multi-dimension.