 Research
 Open Access
 Published:
A minimum principle for stochastic control of hepatitis C epidemic model
Boundary Value Problems volume 2023, Article number: 52 (2023)
Abstract
In this paper, we discussed a stochastic optimal control of hepatitis C that minimizes the side effect and reduces the viral load. The control variables represent the drug therapy used for blocking a new infection and virus production. The solution of control problem is solved using the stochastic minimum principle and a fourstep scheme. The numerical simulation is carried out to justify the theoretical analysis. The result shows that using both types of drugs for therapy is much more effective.
1 Introduction
Hepatitis C is an inflammatory liver disease caused by the hepatitis C virus (HCV) that can lead to cirrhosis and/or hepatocellular cancer [1]. Hepatitis C is a bloodborne disease, which is mostly transmitted through contaminated needles used for dialysis, tattoos, piercings, and unclean blood transfusion [2–4]. HCV infection does not always require treatment since the immune response can clear the infection. However, when hepatitis C becomes chronic, treatment is needed to cure the disease. WHO recommends using directacting antiviral (DAA) therapy [5].
Certain variants are associated with a higher potential for resistance or therapeutic failure. The DAA regimen faces a risk of resistance when administered to a group of infected patients with a special variant known as the resistant associated variant (RAV). Thus, in every case of suspected resistance or failure of therapy with the DAA regimen, it is necessary to examine resistance genotypes. The results of this examination will be considered for therapy modification, either by adding the duration, other agents, especially ribavirin, or by substituting the regimen [6].
In the past 10 years, there have been two types of standard therapies for chronic hepatitis C that are often used in the treatment of hepatitis C, namely a combination of interferon and ribavirin. This therapy gave satisfactory results, i.e., in patients with HCV genotypes 2 and 3, around 80% of patients could achieve sustained virological response (SVR24), while in HCV genotype 1, only 4050% of patients managed to achieve SVR24 [6]. Based on the basic HCV model provided by Neumann [7], Dustin et al. [8] and Hasan et al. [6], they explained that the type and duration of therapy depend on the type of HCV genotype. Thus, giving the same therapy to different patients can result in different effects. This phenomenon will be modeled as stochastic effects.
Furthermore, a combination of interferon and ribavirin also has side effects such as anemia, drowsiness, indigestion, and shortness of breath [9]. Therefore, in this paper, a stochastic optimal control problem is established to determine the optimal drug effectiveness to treat hepatitis C. The objective function is minimizing viral load and drug side effects.
Several papers have discussed the optimal control of hepatitis C. Martin et al. in [10] had determined the optimal therapy program for hepatitis C patients over 10 years, considering biomedical and economic goals. In this case, the minimum cost is determined to reduce viral load and the cost of secondary effects caused during therapy. Meanwhile, Peregrino et al. [9] considered optimal control of cellular level HCV with immune response, and Zhang et al. [11] proposed optimal control of HCV model with treatment, both of whom used the maximum Pontryagin principle. Their research focused on a deterministic model, whereas we developed a stochastic optimal control model in this study. We could investigate the effects of uncertainty in the optimal control of the hepatitis C problem. Another research that analyzes the fractional order mathematical model of hepatitis B is studied in [12].
In [13–15], they discussed about an optimal control problem in deterministic and stochastic epidemic models using the HamiltonJacobi equation. Meanwhile, Ishikawa [16] designed the optimal control of the SIR model using the maximum stochastic principle and the fourstep scheme technique. However, researchers have not discussed the optimal control problem where the control variable is contained in the diffusion coefficient. In this article, we present the new results of the stochastic optimal control where the control variable is contained in the diffusion coefficient. The optimal control problem is solved using the HamiltonJacobiBellman (HJB) equation, which is a nonlinear function and challenging to solve numerically. In this paper, the minimum stochastic principle and the fourstep scheme will be applied to the stochastic control of the hepatitis C epidemic model. Based on medical literature [7], the optimal solution determines the effectiveness of drugs related to administer drug doses during therapy.
The rest of this paper is organized as follows. Section 2 presents the mathematical modeling for hepatitis C with a stochastic control and the analysis. Then, in Sect. 3, we carried out numerical simulations to verify theoretical results. Finally, in the last section, we present a conclusion.
2 HCV stochastic control model with a diffusion coefficient that contains control variables
HCV initially infects target cells, proliferates inside them, and then gets released into the extracellular space without disrupting the cell integrity. During its life cycle, HCV usurps host cell molecules, termed host factors, and various cell biological mechanisms ranging from endocytosis to the secretory pathway. The HCV life cycle can be separated into four steps: (1) virus entry; (2) genome translation and polyprotein processing; (3) genome replication and (4) particle assembly and release from the host cell. Virions are released from the cells most likely by exocytosis or transmitted to other cells via a cellfree mechanism [17].
Recall a deterministic mathematical model of HCV given in [7] as
The cell population \(N(t)\) is categorized into three subpopulations. Let \(T(t) \) be the number of healthy or uninfected cells, \(I(t) \) be the number of infected cells, and \(V(t) \) be the number of free viruses. The uninfected cells are produced at rate Λ and die naturally at a constant rate \(\delta _{1} \). Cells become infected when they interact with the virus at a constant rate β; once infected, they will die at a constant rate \(\delta _{2} \). HCV is produced by infected cells at a constant rate k and cleared at a constant rate c. Parameter η is the effectiveness of a drug in stopping new infection and ϵ is the effectiveness of a drug in blocking virus production.
For the deterministic Model (1), there are two equilibrium points, namely the diseasefree equilibrium point \(E_{0}=(\frac{\Lambda}{\delta _{1}},0,0)\) and the endemic equilibrium point \(E_{1}=(\frac{\delta _{2}c}{(1\eta )(1\epsilon )k\beta}, \frac{\Lambda}{\delta _{2}} \frac{\delta _{1}c}{(1\eta )(1\epsilon )k\beta}, \frac{(1\epsilon )k\Lambda}{\delta _{2}c} \frac{\delta _{1}}{(1\eta )\beta})\). By using the next generation matrix method [18], the basic reproduction number for the deterministic model or uncontrolled system is \(R_{0}= \frac{(1\eta )(1\epsilon )\Lambda \beta k}{c\delta _{1}\delta _{2}}\).
We defined two controls variables as follows

1
\(u_{1}(t) \) represents the control variable to block new infection.

2
\(u_{2}(t) \) represents the control variable of hepatitis C to block the virus production.
We assumed that the control variables are stochastic, i.e., \(u_{1}(t)=u_{1}+\sigma _{1}u_{1}\,dB(t)/dt \) and \(u_{2}(t)= u_{2}+\sigma _{2}u_{2}\,dB(t)/dt \), where \(B(t)\) is standard Brownian motion. The stochastic control system model is given by
where
is the set of control variables and \(u_{1}\), \(u_{2}\) is \(\{\mathcal{F}_{t}\}\)adapted. If \(\sigma _{1}=\sigma _{2}=0\), then System (2) becomes a deterministic model studied in [7], where \(u_{1}=\eta \) and \(u_{2}=\epsilon \).
System (2) represents the dynamics of a biological cell population. Therefore, the number of cell populations should be nonnegative and bounded. For this reason, we first established the global existence, positivity, and boundedness of solutions for the proposed model in the following theorem.
Theorem 2.1
For any initial value in \({\mathbb{R}}^{3} _{+}\), the System (2) is uniformly and ultimately bounded and belongs to the closed and bounded positively invariant set for every \(t\geq 0\). The coefficients of the System (2) satisfy the Lipschitz condition. Then there exists a unique timeglobal solution \((T(t),I(t),V(t)) \subset {\mathbb{R}}^{3}_{+}\), \(t \ge 0 \) with probability 1.
Proof
Consider System (2), it can be seen that the coefficients of the System (2) satisfy the Lipschitz condition thus guarantee the solution is unique and local. Next, it can be proved that the explosion time is infinity (i.e., the time when the solution tends to infinity), thus the solution is global. The detailed proof is analog to [13, 19–21]. □
In our control problem, objective is to reduce the viral load and to minimize the therapy cost. This objective function/cost function is defined as follows
The first term of equation (4) represents the main biological target, i.e., reducing the viral load. The constant \(c_{1} \) is related to the cost for lowering viral load, \(c_{2} \) and \(c_{3} \) are related to the cost of therapy. Since the integral is a random variable, then we need to take the expected value. Thus, the optimal control problem is determining optimal control \(u_{1}^{*}\), \(u_{2}^{*}\) that minimizes the objective function \(J(u)\).
The existence of the optimal control pair can be obtained using a result by [22].
Theorem 2.2
There is an optimal control \((u_{1}^{*}(t),u_{2}^{*}(t))\) such that
The optimal control variable can then be determined in the following theorem.
Theorem 2.3
The optimal control \(u_{1}^{*}(t)\) and \(u_{2}^{*}(t)\) of the System (2), which minimize the objective function (4) are characterized by
where
Proof
In order to construct the optimal control \((u_{1}^{*}(t),u_{2}^{*}(t))\), defined
and we constructed Hamiltonian function as follows
where \(\langle \cdot ,\cdot \rangle \) stands for the Euclidian inner product and
Vector p and q denote the adjoint vectors. Thus, we have
Based on the stochastic minimum principle [23], then we get the following system of differential equations
According to [23], the controller can balance the scale of control and the degree of uncertainty if the diffusion coefficient contains control variables in the stochastic situation. Therefore, we need to define a New Hamiltonian function since the marginal value alone may not be able to fully characterize the tradeoff between the cost and control gain in an uncertain environment. The New Hamiltonian function is given as follows
The value of \(\mathcal{H} \) is optimal if the following conditions are satisfied
which give
Since u should be in the control space U, then we have
Furthermore, we consider the second order adjoining equation as follows
with P and Q are \(n\times n\) matrices. The equations (10) and (17) are called forwardbackward stochastic differential equations(FBSDE) with terminal conditions (18). Next, we will apply the fourstep scheme method to solve the problem (17).
Step 1: Assume that \(P(t)\) and \(\mathbf{x}(t)\) are related, i.e.,
where \(\Phi = ( {\phi _{ij}}) _{n\times n}\) is \(n \times n\) matrix and \(\phi _{ij}\) is vectorvalued function with \(i,j=1,2,\ldots,n\)
Step 2: By using Itô’s formula, we obtain
Since \(P(t)=\phi (t, x(t))\) and u is a function of p, q and x, then we have
and
with the terminal conditions
and Δ is the coefficient dt of (17). Furthermore, the system of partial differential equations in equation (21) is obtained as follows
Analog for \(\phi _{ij}\) where \(i,j=1,2,3\).
Meanwhile, from equation (13) and (14), we have
By using the property of the control space U in (3), we obtain
Analogously, for the firstorder adjoint equation (10), the fourstep scheme method is also applied, and then the system of partial differential equations is given as follows
Analog for \(\theta_{2}\), then for \(\theta_{3}\) as follow
with the terminal condition
Step 3: Solve the partial differential equations systems (24)(25) with the condition (23), thus we obtained \(\phi _{ij}(t,x)\), \(i,j=1,2,3\). Next, formed \(P(t)=\Phi (t,x(t))\).
Step 4: Substituting \(P(t)\) and \(Q(t)\) into equations (15), (16), and (17). Then, the firstorder adjoining equation (10) is solved using a fourstep scheme method, namely solving the system of partial differential equations (30)(31) with terminal conditions (32), resulting in an optimal control \(u_{1}(t)\) and \(u_{2}(t)\). □
3 Numerical simulations
In this section, we carried out some numerical simulations to support our obtained theoritical result of the Model (2). We use a nonstandard finite difference method (NSFD) that is explained in [24–26] to produce a reliable and consistent solution to biological nature. The parameter values chosen here are consistent with studies presented in [27–29]. The parameters are \(\Lambda =1\), \(\beta =0.09\), \(\delta _{1}=0.2\), \(\delta _{2}=0.3\), \(k=2\), \(c=1.2\), \(c_{1}=100\), \(c_{2}=50\), \(c_{3}=100\), and initial values \(T(0)=3\), \(I(0)=2\), \(V(0)=1\) (arbitrary unit) and time t (arbitrary unit). We assume the parameters of intensity noise are \(\sigma _{1}=0.09\) and \(\sigma _{2}=0.0009\).
Figure 1 shows the solution of the System (2) with and without control. This figure illustrates the uncontrolled system trajectory tends to an infected equilibrium point \((T,I,V)=(2,2,3.3)\) where \(R_{0}=2.5>1\) after time \(t=20\) (arbitrary unit). It means that hepatitis C still persists. Meanwhile, the blue line in Figs. 1a, 1b, and 1c describe the solution of system (2) with control \(u_{1}\) and \(u_{2}\). We can see that the number of healthy cells gradually increases with optimal control until it reaches a constant value. We also observe that infected cells tend to zero and free viruses slightly oscillate. It can be seen that therapy can decrease the infected cells and free viruses, thus the spread of hepatitis C can be controlled. Figure 1c shows that the control can minimize the viral load. Based on Fig. 1d, the effectiveness is consistent after time \(t=20\).
In Fig. 2, it can be seen that the number of healthy cells and infected cells are gradually increasing and decreasing, respectively. There are oscillations in the number of free viruses as the effect of stochastic control. In addition, Fig. 2d, 2e, and 2f show the control variable of the model when using one or both therapy. The effectiveness of drugs is consistent after 20 unit time. Regardless of the endemicity, when the \(R_{0}\) of the model without control is more than one, it means that hepatitis C persists. By considering the model with the control, hepatitis C will disappear over time.
4 Conclusions
In this paper, we have investigated the stochastic optimal control problem of the hepatitis C epidemic model, where we consider ribavirin and interferon treatment as control variables. By applying the stochastic minimum principles and a fourstep scheme, we obtain the stochastic optimal solution for controlling the spread of hepatitis C. The results show the importance of considering stochastic factors in the spread of hepatitis C. The higher the noise intensity, the more variation in the control variable solution. It means that there are many variations in the optimal control values for the effectiveness of therapy. In addition, there is an effect when using the therapy of both types of drugs compared to therapy using one. Our findings showed that using both types of drugs was much more effective than using one.
In the future, research on optimal control systems that consider random factors can be developed, especially for diffusion coefficients containing control variables with Brownian motion multidimension.
Availability of data and materials
Not applicable.
References
Burrell, C.J., Howard, C.R., Murphy, F.A.: Viral syndromes. In: Fenner and White’s Medical Virology, pp. 537–556 (2017). https://doi.org/10.1016/B9780123751560.000394
De Oliveira, T., Pybus, O.G., Rambaut, A., Salemi, M., Cassol, S., Ciccozzi, M., Rezza, G., Gattinara, G.C., D’Arrigo, R., Amicosante, M., et al.: HIV1 and HCV sequences from Libyan outbreak. Nature 444, 836–837 (2006)
Jafari, S., Copes, R., Baharlou, S., Etminan, M., Buxton, J.: Tattooing and the risk of transmission of hepatitis C: a systematic review and metaanalysis. Int. J. Infect. Dis. 14, e928–e940 (2010). https://doi.org/10.1016/j.ijid.2010.03.019
Mast, E.E., Hwang, L.Y., Seto, D.S., Nolte, F.S., Nainan, O.V., Wurtzel, H., Alter, M.J.: Risk factors for perinatal transmission of hepatitis C virus (HCV) and the natural history of hcv infection acquired in infancy. J. Infect. Dis. 192, 1880–1889 (2005). https://doi.org/10.1086/497701
World Health Organization: Hepatitis C. The United Nations (2021). Accessed 1 September 2021. https://www.who.int/newsroom/factsheets/detail/hepatitisc
Hasan, I., Gani, R.A., Sulaiman, A.S., Lesmana, C.R.A., Kurniawan, J., Jasirwan, C.O.M.: Konsensus nasional penatalaksanaan hepatitis C di Indonesia, (National Consensus on Hepatitis C Management in Indonesia). Perhimpunan Peneliti Hati Indonesia, Jakarta (2017)
Neumann, A.U., Lam, N.P., Dahari, H., Gretch, D.R., Wiley, T.E., Layden, T.J., Perelson, A.S.: Hepatitis C viral dynamics in vivo and the antiviral efficacy of interferonalpha therapy. Science 282, 103–107 (1998). https://doi.org/10.1126/science.282.5386.103
Dustin, L., Bartolini, B., Capobianchi, M., Pistello, M.: Hepatitis C virus: life cycle in cells, infection and host response, and analysis of molecular markers influencing the outcome of infection and response to therapy. Clin. Microbiol. Infect. 22, 826–832 (2016). https://doi.org/10.1016/j.cmi.2016.08.025
Peregrino, A., Esteva, L., Ble, G.: Optimal control applied to hepatitis C therapy considering immune system. J. Pure Appl. Math. Adv. Appl. 19, 9–35 (2018). https://doi.org/10.18642/jpamaa_7100121911
Martin, N.K., Pitcher, A.B., Vickerman, P., Vassall, A., Hickman, M.: Optimal control of hepatitis C antiviral treatment programme delivery for prevention amongst a population of injecting drug users. PLoS ONE 6, e22309 (2011). https://doi.org/10.1371/journal.pone.0022309
Zhang, S., Xu, X.: Dynamic analysis and optimal control for a model of hepatitis C with treatment. Commun. Nonlinear Sci. Numer. Simul. 46, 14–25 (2017). https://doi.org/10.1016/j.cnsns.2016.10.017
Din, A., Li, Y., Khan, F.M., Khan, Z.U., Liu, P.: On analysis of fractional order mathematical model of hepatitis B using Atangana–Baleanu Caputo (ABC) derivative. Fractals 30(1), 2240017 (2022). https://doi.org/10.1142/S0218348X22400175
Gani, S.R., Halawar, S.V.: Optimal control analysis of deterministic and stochastic epidemic model with media awareness programs. Int. J. Optim. Control Theor. Appl. 9, 24–35 (2019). https://doi.org/10.11121/ijocta.01.2019.00423
Witbooi, P.J., Muller, G.E., Schalkwyk, G.J.V.: Vaccination control in a stochastic SVIR epidemic model. Comput. Math. Methods Med. 2015, Article ID 271654 1–9 (2015). https://doi.org/10.1155/2015/271654
Din, A., Li, Y.: Stationary distribution extinction and optimal control for the stochastic hepatitis B epidemic model with partial immunity. Phys. Scr. 96(7), 074005 (2021). https://doi.org/10.1088/14024896/abfacc
Ishikawa, M.: Stochastic optimal control of an SIR epidemic model with vaccination. In: Proceedings of the ISCIE International Symposium on Stochastic Systems Theory and Its Applications, vol. 2012, pp. 57–62 (2012). https://doi.org/10.5687/sss.2012.57
Gerold, G., Pietschmann, T.: The HCV life cycle: in vitro tissue culture systems and therapeutic targets. Dig. Dis. 32(5), 525–537 (2014). https://doi.org/10.1159/000360830
Driessche, P.V., Watmough, J.: Reproduction numbers and subthreshold endemic equilibria for compartmental models of disease transmission. Math. Biosci. 180(2002), 29–48 (2002)
Din, A.: The stochastic bifurcation analysis and stochastic delayed optimal control for epidemic model with general incidence function. Chaos, Interdiscip. J. Nonlinear Sci. 31(12), 123101 (2021). https://doi.org/10.1063/5.0063050
Din, A., Li, Y.: Stochastic optimal control for norovirus transmission dynamics by contaminated food and water. Chin. Phys. B 31(2), 020202 (2022). https://doi.org/10.1088/16741056/ac2f32
Lestari, D., Megawati, N.Y., Susyanto, N., AdiKusumo, F.: Qualitative behaviour of a stochastic hepatitis C epidemic model in cellular level. Math. Biosci. Eng. 19(2), 1515–1535 (2022). https://doi.org/10.3934/mbe.2022070
Øksendal, B., Sulem, A., Zhang, T.: Optimal control of stochastic delay equations and timeadvanced backward stochastic differential equations. Adv. Appl. Probab. 43, 572–596 (2011)
Yong, J., Zhou, X.Y.: Stochastic Controls: Hamiltonian Systems and HJB Equations, vol. 43. Springer, New York (1999)
Raza, A., Rafiq, M., Awrejcewicz, J., Ahmed, N., Mohsin, M.: Dynamical analysis of coronavirus disease with crowding effect, and vaccination: a study of third strain. Nonlinear Dyn. 107(4), 3963–3982 (2022)
Raza, A., Awrejcewicz, J., Rafiq, M., Ahmed, N., Mohsin, M.: Stochastic analysis of nonlinear cancer disease model through virotherapy and computational methods. Mathematics 10(3), 368 (2022)
Raza, A., Awrejcewicz, J., Rafiq, M., Ahmed, N., Ehsan, M.S., Mohsin, M.: Dynamical analysis and design of computational methods for nonlinear stochastic leprosy epidemic model. Alex. Eng. J. 61(10), 8097–8111 (2022)
Wodarz, D.: Hepatitis C virus dynamics and pathology: the role of CTL and antibody responses. J. Gen. Virol. 84, 1743–1750 (2003). https://doi.org/10.1099/vir.0.191180
Wodarz, D., Jansen, V.A.: A dynamical perspective of CTL crosspriming and regulation: implications for cancer immunology. Immunol. Lett. 86, 213–227 (2003). https://doi.org/10.1016/S01652478(03)000233
Hu, X., Li, J., Feng, X.: Threshold dynamics of a HCV model with virus to cell transmission in both liver with CTL immune response and the extrahepatic tissue. J. Biol. Dyn. 15, 19–34 (2021). https://doi.org/10.1080/17513758.2020.1859632
Acknowledgements
The first author thanks LPDP Indonesia for the financial support under the Doctoral Program Scholarship. The authors thank to Department of Mathematics, Universitas Gadjah Mada (UGM) and Directorate for the Higher Education, Ministry of Research, Technology, and Higher Education of Indonesia for the Research Grant “Penelitian Disertasi Doktor”(PDD) UGM 2022 under grant no.1738/UN1/DITLIT/DitLit/PT.01.03/2022. Special thanks are also due to some colleagues for their invaluable inputs in the discussions during the research.
Funding
The paper was financially supported by Universitas Gadjah Mada’s (UGM) Department of Mathematics, and the Directorate for the Higher Education of the Ministry of Research, Technology, and Higher Education of Indonesia through a Research Grant “Penelitian Disertasi Doktor”(PDD) UGM 2022 number 1738/UN1/DITLIT/DitLit/PT.01.03/2022.
Author information
Authors and Affiliations
Contributions
All authors contributed equally to the manuscript, read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Lestari, D., AdiKusumo, F., Megawati, N.Y. et al. A minimum principle for stochastic control of hepatitis C epidemic model. Bound Value Probl 2023, 52 (2023). https://doi.org/10.1186/s13661023017403
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13661023017403
Keywords
 Optimal control
 Hepatitis C virus
 Therapy
 The stochastic minimum principle
 A fourstep scheme