[en] Civil and maritime engineering systems, among others, from bridges to offshore platforms and wind turbines, must be efficiently managed, as they are exposed to deterioration mechanisms throughout their operational life, such as fatigue and/or corrosion. Identifying optimal inspection and maintenance policies demands the solution of a complex sequential decision-making problem under uncertainty, with the main objective of efficiently controlling the risk associated with structural failures. Addressing this complexity, risk-based inspection planning methodologies, supported often by dynamic Bayesian networks, evaluate a set of pre-defined heuristic decision rules to reasonably simplify the decision problem. However, the resulting policies may be compromised by the limited space considered in the definition of the decision rules. Avoiding this limitation, Partially Observable Markov Decision Processes (POMDPs) provide a principled mathematical methodology for stochastic optimal control under uncertain action outcomes and observations, in which the optimal actions are prescribed as a function of the entire, dynamically updated, state probability distribution. In this paper, we combine dynamic Bayesian networks with POMDPs in a joint framework for optimal inspection and maintenance planning, and we provide the relevant formulation for developing both infinite and finite horizon POMDPs in a structural reliability context. The proposed methodology is implemented and tested for the case of a structural component subject to fatigue deterioration, demonstrating the capability of state-of-the-art point-based POMDP solvers of solving the underlying planning stochastic optimization problem. Within the numerical experiments, POMDP and heuristic-based policies are thoroughly compared, and results showcase that POMDPs achieve substantially lower costs as compared to their counterparts, even for traditional problem settings.
Disciplines :
Computer science Civil engineering
Author, co-author :
Morato Dominguez, Pablo Gabriel ; Université de Liège - ULiège > Département ArGEnCo > ANAST (Systèmes de transport et constructions navales)
Papakonstantinou, K.G.; The Pennsylvania State University, University Park, PA 16802, USA > Department of Civil & Environmental Engineering
Andriotis, C.P.; Pennsylvania State University, University Park, PA 16802, USA > Department of Civil & Environmental Engineering
Nielsen, J.S.; Aalborg University, 9220 Aalborg, Denmark > Department of the Built Environment
Rigo, Philippe ; Université de Liège - ULiège > Département ArGEnCo > Constructions hydrauliques et navales
Language :
English
Title :
Optimal inspection and maintenance planning for deteriorating structural components through dynamic Bayesian networks and Markov decision processes
Publication date :
January 2022
Journal title :
Structural Safety
ISSN :
0167-4730
Publisher :
Elsevier, Netherlands
Volume :
94
Issue :
102140
Peer reviewed :
Peer Reviewed verified by ORBi
Funders :
F.R.S.-FNRS - Fonds de la Recherche Scientifique NSF - National Science Foundation
Frangopol, D.M., Life-cycle performance, management, and optimisation of structural systems under uncertainty: accomplishments and challenges (2011) Struct Infrastruct Eng, 7 (6), pp. 389-413. , http://www.tandfonline.com/doi/abs/10.1080/15732471003594427
Val, D.V., Stewart, M.G., Melchers, R.E., Effect of reinforcement corrosion on reliability of highway bridges (1998) Eng Struct, 20 (11), pp. 1010-1019. , https://linkinghub.elsevier.com/retrieve/pii/S0141029697001971
Val, D.V., Melchers, R.E., Reliability of Deteriorating RC Slab Bridges (1997) J Struct Eng, 123 (12), pp. 1638-1644
Moan, T., Reliability-based management of inspection, maintenance and repair of offshore structures (2005) Struct Infrastruct Eng, 1 (1), pp. 33-62
Lotsberg, I., Sigurdsson, G., Fjeldstad, A., Moan, T., Probabilistic methods for planning of inspection for fatigue cracks in offshore structures (2016) Marine Struct, 46, pp. 167-192. , https://linkinghub.elsevier.com/retrieve/pii/S0951833916000071
Wirsching, P.H., Fatigue reliability in welded joints of offshore structures (1980) Int J Fatigue, 2 (2), pp. 77-83
Schaumann, P., Lochte-Holtgreven, S., Steppeler, S., Special fatigue aspects in support structures of offshore wind turbines (2011) Materialwissenschaft und Werkstofftechnik, 42 (12), pp. 1075-1081
Dong, W., Moan, T., Gao, Z., Fatigue reliability analysis of the jacket support structure for offshore wind turbine considering the effect of corrosion and inspection (2012) Reliab Eng Syst Saf, 106, pp. 11-27
Yeter, B., Garbatov, Y., https://doi.org/10.1016/j.engstruct.2015.07.038, Guedes Soares C. Fatigue damage assessment of fixed offshore wind turbine tripod support structures. Eng Struct 2015
101:518–28. doi:10.1016/j.engstruct.2015.07.038
Andriotis, C.P., Papakonstantinou, K.G., Chatzi, E.N., (1912), Value of structural health information in partially observable stochastic environments
2020. arXiv preprint arXiv12534
Fauriat, W., Zio, E., Optimization of an aperiodic sequential inspection and condition-based maintenance policy driven by value of information (2020) Reliab Eng Syst Saf, 204
Ellingwood, B.R., Risk-informed condition assessment of civil infrastructure: state of practice and research issues (2005) Struct Infrastruct Eng, 1 (1), pp. 7-18
Sánchez-Silva, M., Frangopol, D.M., Padgett, J., Soliman, M., Maintenance and operation of infrastructure systems (2016) J Struct Eng, 142 (9), p. F4016004
Faber, M.H., Stewart, M.G., Risk assessment for civil engineering facilities: Critical overview and discussion (2003) Reliab Eng Syst Saf, 80 (2), pp. 173-184
Kim, S., Ge, B., Frangopol, D.M., Effective optimum maintenance planning with updating based on inspection information for fatigue-sensitive structures (2019) Probab Eng Mech, 58 (August). , https://linkinghub.elsevier.com/retrieve/pii/S0266892019300700
Raiffa, H., Schlaifer, R., Applied Statistical Decision Theory, Harvard University Graduate School of Business Administration (Division of Research) (1961), Bailey & Swinfen
Russell, S.J., Artificial intelligence: a modern approach (2010), 3rd Ed. Prentice Hall Upper Saddle River, NJ
Sørensen, J.D., Rackwitz, R., Faber, M.H., Thoft-Christensen, P., (1991), Modelling in optimal inspection and repair. In: Proceedings of the 10th international conference on offshore mechanics and arctic engineering, vol. 2. United States: American Society of Mechanical Engineers p. 281–8
Goyet, J., Straub, D., Faber, M.H., Risk-based inspection planning of offshore installations (2002) Struct Eng Int, 12 (3), pp. 200-208. , https://www.tandfonline.com/doi/full/10.2749/101686602777965360
Rangel-Ramírez, J.G., Sørensen, J.D., Risk-based inspection planning optimisation of offshore wind turbines (2012) Struct Infrastruct Eng, 8 (5), pp. 473-481
Straub, D., Stochastic Modeling of Deterioration Processes through Dynamic Bayesian Networks (2009) J Eng Mech, 135 (10), pp. 1089-1099
Luque, J., Straub, D., Risk-based optimal inspection strategies for structural systems using dynamic Bayesian networks. Struct Saf 2019
Bismut, E., Straub, D., (2021), Optimal adaptive inspection and maintenance planning for deteriorating structural systems arXiv preprint arXiv:2102.06016
Yang, D.Y., Frangopol, D.M., Probabilistic optimization framework for inspection/repair planning of fatigue-critical details using dynamic bayesian networks (2018) Comput Struct, 198, pp. 40-50
Tien, I., Der Kiureghian, A., Reliability assessment of critical infrastructure using bayesian networks (2017) J Infrastruct Syst, 23 (4), p. 04017025
Puterman, M.L., Markov decision processes: discrete stochastic dynamic programming (2014), John Wiley & Sons
Shani, G., Pineau, J., Kaplow, R., A survey of point-based POMDP solvers (2013) Autonom Agents Multi-Agent Syst, 27 (1), pp. 1-51
Kurniawati, H., Hsu, D., (2008), http://www.roboticsproceedings.org/rss04/p9.pdf, Sun Lee W. SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces. In: Proceedings of Robotics: Science and Systems IV, Zurich, Switzerland doi:10.15607/RSS.2008.IV.009
Smith, T., Simmons, R., Focused real-time dynamic programming for MDPs: Squeezing more out of a heuristic (2006) Proc Natl Conf Artif Intell, 2 (January), pp. 1227-1232
Memarzadeh, M., Pozzi, M., Zico Kolter, J., Optimal Planning and Learning in Uncertain Environments for the Management of Wind Farms (2015) J Comput Civil Eng, 29 (5), p. 04014076
Memarzadeh, M., Pozzi, M., Kolter, J.Z., Hierarchical modeling of systems with similar components: A framework for adaptive monitoring and control (2016) Reliab Eng Syst Saf, 153, pp. 159-169. , http://www.sciencedirect.com/science/article/pii/S0951832016300448
Papakonstantinou, K.G., Shinozuka, M., Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part I: Theory (2014) Reliab Eng Syst Saf, 130, pp. 214-224
Corotis, R.B., http://www.tandfonline.com/doi/abs/10.1080/15732470412331289305, Hugh Ellis J, Jiang M. Modeling of risk-based inspection, maintenance and life-cycle cost with partially observable Markov decision processes. Struct Infrastruct Eng 2005
1(1):75–84. doi:10.1080/15732470412331289305
Morato, P.G., Nielsen, J.S., Mai, A.Q., Rigo, P., POMDP based maintenance optimization of offshore wind substructures including monitoring (2019) 13th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP, pp. 270-277
Faber, M.H., Risk-based inspection: The framework (2002) Struct Eng Int, 12 (3), pp. 186-194
Straub, D., (2004), Generic Approaches to Risk Based Inspection Planning for Steel Structures, Ph.D. thesis, Swiss Federal Institute of Technology Zürich (ETH)
Ditlevsen, O., Madsen, H.O., Structural reliability methods (2007), Department of Mechanical Engineering, Technical University of Denmark
Straub, D., Reliability updating with equality information (2011) Probab Eng Mech, 26 (2), pp. 254-258
Lotsberg, I., Sigurdsson, G., Fjeldstad, A., Moan, T., Probabilistic methods for planning of inspection for fatigue cracks in offshore structures (2016) Marine Struct, 46, pp. 167-192
Luque, J., Straub, D., Reliability analysis and updating of deteriorating systems with dynamic Bayesian networks (2016) Struct Saf, 62, pp. 34-46
Jensen, F.V., Introduction to Bayesian Networks (1996), Springer-Verlag Berlin, Heidelberg
Murphy, K.P., (2002), Dynamic Bayesian Networks: Representation, Inference and Learning. Ph.D. thesis, University of California, Berkeley
Nielsen, J.S., Sørensen, J.D., Computational framework for risk-based planning of inspections, maintenance and condition monitoring using discrete Bayesian networks (2018) Struct Infrastruct Eng, 14 (8), pp. 1082-1094
Papakonstantinou, K.G., Shinozuka, M., Optimum inspection and maintenance policies for corroded structures using partially observable Markov decision processes and stochastic, physically based models (2014) Probab Eng Mech, 37, pp. 93-108
Straub, D., Stochastic Modeling of Deterioration Processes through Dynamic Bayesian Networks (2009) J Eng Mech, 135 (10), pp. 1089-1099
Papakonstantinou, K.G., Shinozuka, M., Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part II: POMDP implementation (2014) Reliab Eng Syst Saf, 130, pp. 214-224
Kochenderfer, M.J., Amato, C., Chowdhary, G., How, J.P., Reynolds, H.J.D., Thornton, J.R., Torres-Carrasquillo, P.A., Vian, J., Decision Making Under Uncertainty: Theory and Application (2015), 1st Ed. The MIT Press
Nielsen, J.S., Sørensen, J.D., (2015), p. 8. , Risk-based Decision Making for Deterioration Processes Using POMDP. In: T. Haukaas (Ed.), 12th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP12, Civil Engineering Risk and Reliability Association, Vancouver, Canada doi:10.14288/1.0076132
Robelin, C.-A., Madanat, S.M., History-dependent bridge deck maintenance and replacement optimization with Markov decision processes (2007) J Infrastruct Syst, 13 (3), pp. 195-201
Kim, J.W., Choi, G.B., Lee, J.M., A POMDP framework for integrated scheduling of infrastructure maintenance and inspection (2018) Comput Chem Eng, 112, pp. 239-252
Kaelbling, L.P., Littman, M.L., Cassandra, A.R., Planning and acting in partially observable stochastic domains (1998) Artif Intell, 101 (1-2), pp. 99-134. , papers3://publication/uuid/5ECA204D-77B0-4C68-A0A6-7323A9D8893B
Spaan, M.T.J., Vlassis, N., Perseus: Randomized point-based value iteration for POMDPs (2005) J Artif Intell Res, 24, pp. 195-220
Papakonstantinou, K.G., Andriotis, C.P., Shinozuka, M., POMDP and MOMDP solutions for structural life-cycle cost minimization under partial and mixed observability (2018) Struct Infrastruct Eng, 14 (7), pp. 869-882
Pineau, J., Gordon, G., Thrun, S., (2003), Point-based value iteration: An anytime algorithm for POMDPs. In: IJCAI International Joint Conference on Artificial Intelligence p. 1025–30
Howard, R.A., Information value theory (1966) IEEE Trans Syst Sci Cybern, 2 (1), pp. 22-26
Walraven, E., Spaan, M.T., Point-based value iteration for continuous POMDPs (2019) J Artif Intell Res, 65, pp. 307-341
Andriotis, C.P., Papakonstantinou, K.G., Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints (2021) eliab Eng Syst Saf, 212
Andriotis, C.P., Papakonstantinou, K.G., Managing engineering systems with large state and action spaces through deep reinforcement learning. Reliab Eng Syst Saf 2019
