{"id":475,"date":"2024-12-14T08:00:07","date_gmt":"2024-12-14T08:00:07","guid":{"rendered":"https:\/\/christian-engelmann.de\/?page_id=475"},"modified":"2024-12-14T16:58:30","modified_gmt":"2024-12-14T16:58:30","slug":"resilience-design-patterns","status":"publish","type":"page","link":"https:\/\/www.christian-engelmann.info\/?page_id=475","title":{"rendered":"Resilience Design Patterns"},"content":{"rendered":"<table style=\"border:5pt solid grey; padding:10pt;\">\n<tr style=\"border:0pt; padding:0pt;\">\n<td style=\"border:0pt; padding:0pt; text-align:left;\"><i><b>Summary:<\/b> Resilience design patterns offer a new, structured hardware\/software design approach for improving resilience by identifying and evaluating repeatedly occurring resilience problems and coordinating corresponding solutions. They permit resilience to become an integral part of the high-performance computing hardware\/software ecosystem through co-design, such that the burden for providing resilience is on the system by design and not on the operator or user as an afterthought.<\/i><\/td>\n<\/tr>\n<\/table>\n<p>Resilience to faults, errors, and failures in extreme-scale high-performance computing (HPC) systems is a critical challenge. Resilience design patterns offer a new, structured hardware\/software design approach for improving resilience by identifying and evaluating repeatedly occurring resilience problems and coordinating corresponding solutions. Initial efforts identified and formalized these patterns and developed a proof-of-concept prototype to demonstrate portable resilience. Further work created performance, reliability, and availability models for each of the identified 15 structural resilience design patterns and a modeling tool that allows (1) exploring the performance, reliability, and availability of each pattern, and (2) investigating the trade-offs be-tween patterns and pattern combinations.<\/p>\n<p>The resilience design patterns (Figure 1) are broadly classified into state patterns and behavioral patterns. State patterns describe all aspects of the system structure that are relevant to the forward progress of the system. These patterns are further classified into stateless and stateful patterns, where the stateful pattern is further broken down into persistent, volatile and operating environment state patterns. Behavioral patterns identify common detection, containment, or mitigation actions that enable the components in a system that realize these patterns to cope with the presence of a fault, error, or failure event. These patterns are further classified into a hierarchy of strategy, architectural and structure patterns to identify different aspects of a solution. In total, 31 resilience design patterns have been specified, 5 state patterns and 26 behavioral patterns.<\/p>\n<p align=\"center\"><img decoding=\"async\" src=\"images\/rdp\/classification.png\" hspace=\"0\" vspace=\"0\" height=\"70%\" width=\"70%\"><br \/>\n<i>Figure 1: Classification of resilience design patterns<\/i><\/p>\n<p>The model for each of the 15 structural design patterns consists of a flowchart and state diagram, identifying its dynamic error\/failure-free behavior and when handling errors\/failures. It also includes mathematical models for performance (error\/failure-free execution time and under error\/failure conditions), reliability (probability of not experiencing an error\/failure) and availability (portion of time a system provides correct service). The reliability and availability models rely on exponential error\/failure distribution to make a modeling approach possible. Other distributions, such as Weibull, would require a simulation approach. The modeling tool relies on parametrized descriptions of patterns to calculate and plot performance, reliability and availability. Complex horizontal and vertical pattern combinations can be modeled to understand system behavior. For example, Figure 2 shows the results for a 2-level checkpoint\/restart (CR) solution, with fine-grain CR at the compute node or accelerator level and coarse-grain CR at the parallel job level.<\/p>\n<p align=\"center\"><img decoding=\"async\" src=\"images\/rdp\/mlrb.png\" hspace=\"0\" vspace=\"0\" height=\"70%\" width=\"70%\"><br \/>\n<i>Figure 2: Multi-level Rollback performance, reliability, and availability<\/i><\/p>\n<p>Resilience needs to become an integral part of the HPC hardware\/software ecosystem through co-design, such that the burden for providing resilience is on the system by design and not on the operator or user as an afterthought. The resilience design pattern approach offers this capability by identifying, classifying, quantifying and coordinating the detection, containment and mitigation properties of individual resilience solutions and their vertical and horizontal compositions within an extreme-scale HPC system, avoiding coverage gaps and overprotection.<\/p>\n<p><b>Latest Resilience Design Pattern specification:<\/b><\/p>\n<ul>\n<li>Christian Engelmann, Rizwan Ashraf, Saurabh Hukerikar, Mohit Kumar, and Piyush Sao. <b>Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale (Version 2.0)<\/b>. Technical Report, ORNL\/TM-2022\/2809, Oak Ridge National Laboratory, Oak Ridge, TN, USA, December 16, 2022. DOI <a href=\"http:\/\/dx.doi.org\/10.2172\/1922296\" target=\"publication\">10.2172\/1922296<\/a>. <a href=\"javascript:showAbstract('Reliability is a serious concern for future extreme-scale high-performance computing (HPC) systems. Projections based on the current generation of HPC systems and technology roadmaps suggest the prevalence of very high fault rates in future systems. The errors resulting from these faults will propagate and generate various kinds of failures, which may result in outcomes ranging from result corruptions to catastrophic application crashes. Therefore, the resilience challenge for extreme-scale HPC systems requires coordination between various hardware and software technologies that are capable of handling a broad set of fault models at accelerated fault rates. Also, due to practical limits on power consumption in future HPC systems, they are likely to embrace innovative architectures, increasing the levels of hardware and software complexities. Therefore, the techniques that seek to improve resilience must navigate the complex trade-off space between resilience and the overheads to power consumption and performance. While the HPC community has developed various resilience solutions, application-level techniques as well as system-based solutions, the solution space of HPC resilience techniques remains fragmented. There are no formal methods to integrate the various HPC resilience techniques into composite solutions, nor are there methods to holistically evaluate the adequacy and efficacy of such solutions in terms of their protection coverage, and their performance &amp; power efficiency characteristics. Additionally, few implementations of current resilience solutions are portable to newer architectures and software environments that will be deployed on future systems. We developed a new structured approach to the management of HPC resilience using the concept of resilience-based design patterns. In general, a design pattern is a repeatable solution to a commonly occurring problem. We identified the well-known solutions that are commonly used to deal with faults, errors and failures in HPC systems. In the initial design patterns specification (version 1.0), we described the various solutions, which address specific problems in the design of resilient HPC environments, in the form of patterns. Each pattern describes a problem caused by a fault, error or failure event in an HPC environment, and then describes the core of the solution of the problem in such a way that this solution may be adapted to different systems and implemented at different layers of the system stack. The catalog of these resilience design patterns provides designers with a collection of design elements. To construct complete resilience solutions using combinations of various patterns, we defined a framework that enhances HPC designers&amp;#39; understanding of the important constraints and the opportunities for the design patterns to be implemented and deployed at various layers of the system stack. The design framework is also useful for establishing interfaces and mechanisms to coordinate flexible fault management across hardware and software components, as well as to consider the trade-off between performance, resilience, and power consumption when constructing a solution. The resilience design patterns specification version 1.1 included more detailed explanations of the pattern solutions, the context in which the patterns are applicable, and the implications for hardware or software design. It also provided several additional examples and detailed case studies to demonstrate the use of patterns to build realistic solutions. In this version 1.2 of the specification document, we have improved the pattern descriptions, including graphical representations of the pattern components. These improvements are largely based on critical comments, feedback and suggestions received from pattern experts and readers of the previous versions of the specification. The pattern classification has been modified to further clarify the relationships between pattern categories. This version of the specification also introduces a pattern language for resilience design patterns. The pattern language presents the patterns in the catalog as a network, revealing the relations among the resilience patterns. The language provides designers with the means to explore alternative techniques for handling a specific fault model that may have different efficiency and complexity characteristics. Using the pattern language also enables the design and implementation of comprehensive resilience solutions as a set of interconnected resilience patterns that can be instantiated across layers of the system stack. The overall goal of this work is to provide hardware and software designers, as well as the users and operators of HPC systems, a systematic methodology for the design and evaluation of resilience technologies in HPC systems that keep scientific applications running to a correct solution in a timely and cost-efficient manner despite frequent faults, errors, and failures of various types. Version 2.0 expands the resilience design pattern classification and catalog to include self-stabilization patterns and reliability, availability and performance models for each structural pattern.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann22rdp-20.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#engelmann22rdp-20\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ul>\n<h4>Research Projects<\/h4>\n<ul>\n<li><a href=\"?page_id=148\">2015-21: Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale<\/a><\/li>\n<\/ul>\n<h4>In the News<\/h4>\n<p><b>2021-03-30:<\/b> DOE Advanced Scientific Computing Research. <a href=\"https:\/\/www.energy.gov\/science\/ascr\/articles\/new-approach-fault-tolerance-means-more-efficient-high-performance-computers\" target=\"www.energy.gov\/science\/ascr\/articles\/new-approach-fault-tolerance-means-more-efficient-high-performance-computers\">New Approach to Fault Tolerance Means More Efficient High-Performance Computers<\/a>.<br \/>\n<b>2015-07-15:<\/b> ASCR Discovery. <a href=\"https:\/\/ascr-discovery.org\/2015\/07\/mounting-a-charge\" target=\"ascr-discovery.org\/2015\/07\/mounting-a-charge\">Mounting a charge. Early-career awardees attack exascale computing on two fronts: Power and resilience<\/a>.<br \/>\n<b>2015-07-15:<\/b> HPC Wire. <a href=\"http:\/\/www.hpcwire.com\/2015\/07\/21\/tackling-power-and-resilience-at-exascale\" target=\"www.hpcwire.com\/2015\/07\/21\/tackling-power-and-resilience-at-exascale\">Tackling Power and Resilience at Exascale<\/a>.\n<\/p>\n<h4>Peer-reviewed Journal Publications<\/h4>\n<ol>\n<li>Saurabh Hukerikar and Christian Engelmann. <b>Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale<\/b>. <i><a href=\"http:\/\/superfri.org\/superfri\" target=\"superfri.org\/superfri\">Journal of Supercomputing Frontiers and Innovations (JSFI)<\/a><\/i>, volume 4, number 3, pages 4-42, October 1, 2017. <a href=\"http:\/\/www.susu.ru\/en\" target=\"www.susu.ru\/en\">South Ural State University Chelyabinsk, Russia<\/a>. ISSN 2409-6008. DOI <a href=\"http:\/\/dx.doi.org\/10.14529\/jsfi170301\" target=\"publication\">10.14529\/jsfi170301<\/a>. <a href=\"javascript:showAbstract('Reliability is a serious concern for future extreme-scale high-performance computing (HPC) systems. Projections based on the current generation of HPC systems and technology roadmaps suggest the prevalence of very high fault rates in future systems. The errors resulting from these faults will propagate and generate various kinds of failures, which may result in outcomes ranging from result corruptions to catastrophic application crashes. Therefore, the resilience challenge for extreme-scale HPC systems requires management of various hardware and software technologies that are capable of handling a broad set of fault models at accelerated fault rates. Also, due to practical limits on power consumption in HPC systems future systems are likely to embrace innovative architectures, increasing the levels of hardware and software complexities. As a result, the techniques that seek to improve resilience must navigate the complex trade-off space between resilience and the overheads to power consumption and performance. While the HPC community has developed various resilience solutions, application-level techniques as well as system-based solutions, the solution space of HPC resilience techniques remains fragmented. There are no formal methods and metrics to investigate and evaluate resilience holistically in HPC systems that consider impact scope, handling coverage, and performance &amp; power efficiency across the system stack. Additionally, few of the current approaches are portable to newer architectures and software environments that will be deployed on future systems. In this paper, we develop a structured approach to the management of HPC resilience using the concept of resilience-based design patterns. A design pattern is a general repeatable solution to a commonly occurring problem. We identify the commonly occurring problems and solutions used to deal with faults, errors and failures in HPC systems. Each established solution is described in the form of a pattern that addresses concrete problems in the design of resilient systems. The complete catalog of resilience design patterns provides designers with reusable design elements. We also define a framework that enhances a designer&amp;#39;s understanding of the important constraints and opportunities for the design patterns to be implemented and deployed at various layers of the system stack. This design framework may be used to establish mechanisms and interfaces to coordinate flexible fault management across hardware and software components. The framework also supports optimization of the cost-benefit trade-offs among performance, resilience, and power consumption. The overall goal of this work is to enable a systematic methodology for the design and evaluation of resilience technologies in extreme-scale HPC systems that keep scientific applications running to a correct solution in a timely and cost-efficient manner despite frequent faults, errors, and failures of various types.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/hukerikar17resilience.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#hukerikar17resilience\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Peer-reviewed Conference Publications<\/h4>\n<ol>\n<li>Saurabh Hukerikar and Christian Engelmann. <b>PLEXUS: A Pattern-Oriented Runtime System Architecture for Resilient Extreme-Scale High-Performance Computing Systems<\/b>. In <i>Proceedings of the <a href=\"http:\/\/prdc.dependability.org\/PRDC2020\" target=\"prdc.dependability.org\/PRDC2020\">25th IEEE Pacific Rim International Symposium on  Dependable Computing (PRDC) 2020<\/a><\/i>, pages 31-39, Perth, Australia, December 1-4, 2020. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 978-1-7281-8004-5. ISSN 1555-094X. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/PRDC50213.2020.00014\" target=\"publication\">10.1109\/PRDC50213.2020.00014<\/a>. Acceptance rate 40.9% (18\/44). <a href=\"javascript:showAbstract('For high-performance computing (HPC) system designers and users, meeting the myriad challenges of next-generation exascale supercomputing systems requires rethinking their approach to application and system software design. Among these challenges, providing resiliency and stability to the scientific applications in the presence of high fault rates requires new approaches to software architecture and design. As HPC systems become increasingly complex, they require intricate solutions for detection and mitigation for various modes of faults and errors that occur in these large-scale systems, as well as solutions for failure recovery. These resiliency solutions often interact with and affect other system properties, including application scalability, power and energy efficiency. Therefore, resilience solutions for HPC systems must be thoughtfully engineered and deployed. In previous work, we developed the concept of resilience design patterns, which consist of templated solutions based on well-established techniques for detection, mitigation and recovery. In this paper, we use these patterns as the foundation to propose new approaches to designing runtime systems for HPC systems. The instantiation of these patterns within a runtime system enables flexible and adaptable end-to-end resiliency solutions for HPC environments. The paper describes the architecture of the runtime system, named Plexus, and the strategies for dynamically composing and adapting pattern instances under runtime control. This runtime-based approach enables actively balancing the cost-benefit trade-off between performance overhead and protection coverage of the resilience solutions. Based on a prototype implementation of PLEXUS, we demonstrate the resiliency and performance gains achieved by the pattern-based runtime system for a parallel linear solver application.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/hukerikar20plexus.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#hukerikar20plexus\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Haewon Jeong, Yaoqing Yang, Christian Engelmann, Vipul Gupta, Tze Meng Low, Pulkit Grover, Viveck Cadambe, and Kannan Ramchandran. <b>3D Coded SUMMA: Communication-Efficient and Robust Parallel Matrix Multiplication<\/b>. In <i>Lecture Notes in Computer Science: Proceedings of the <a href=\"http:\/\/www.euro-par.org\" target=\"www.euro-par.org\">26th European Conference on Parallel and Distributed Computing (Euro-Par) 2020<\/a><\/i>, pages 392-407, Warsaw, Poland, August 24-28, 2020. <a href=\"http:\/\/www.springer.com\" target=\"www.springer.com\">Springer Verlag, Berlin, Germany<\/a>. ISBN 978-3-030-57674-5. DOI <a href=\"http:\/\/dx.doi.org\/10.1007\/978-3-030-57675-2_25\" target=\"publication\">10.1007\/978-3-030-57675-2_25<\/a>. Acceptance rate 24.5% (39\/159). <a href=\"javascript:showAbstract('In this paper, we propose a novel fault-tolerant parallel matrix multiplication algorithm called 3D Coded SUMMA that is communication efficient and achieves higher failure-tolerance than replication-based schemes for the same amount of redundancy. This work bridges the gap between recent developments in coded computing and fault-tolerance in high-performance computing (HPC). The core idea of coded computing is the same as algorithm-based fault-tolerance (ABFT), which is weaving redundancy in the computation using error-correcting codes. In particular, we show that MatDot codes, an innovative code construction for distributed matrix multiplications, can be integrated into three-dimensional SUMMA (Scalable Universal Matrix Multiplication Algorithm) in a communication-avoiding manner. To tolerate any two node failures, the proposed 3D Coded SUMMA requires 50% less redundancy than replication, while the overhead in execution time is only about 5-10%.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/jeong203d.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/jeong203d.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#jeong203d\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Rizwan Ashraf, Saurabh Hukerikar, and Christian Engelmann. <b>Pattern-based Modeling of Multiresilience Solutions for High-Performance Computing<\/b>. In <i>Proceedings of the <a href=\"http:\/\/icpe2018.spec.org\" target=\"icpe2018.spec.org\">9th ACM\/SPEC International Conference on Performance Engineering (ICPE) 2018<\/a><\/i>, pages 80-87, Berlin, Germany, April 9-13, 2018. <a href=\"http:\/\/www.acm.org\" target=\"www.acm.org\">ACM Press, New York, NY, USA<\/a>. ISBN 978-1-4503-5095-2. DOI <a href=\"http:\/\/dx.doi.org\/10.1145\/3184407.3184421\" target=\"publication\">10.1145\/3184407.3184421<\/a>. Acceptance rate 23.7% (14\/59). <a href=\"javascript:showAbstract('Resiliency is the ability of large-scale high-performance computing (HPC) applications to gracefully handle different types of errors, and recover from failures. In this paper, we propose a pattern-based approach to constructing multiresilience solutions. Using resilience patterns, we evaluate the performance and reliability characteristics of detection, containment and mitigation techniques for transient errors that cause silent data corruptions and techniques for fail-stop errors that result in process failures. We demonstrate the design and implementation of the resilience techniques across multiple layers of the system stack such that they are integrated to work together to achieve resiliency to different error types in a highly performance-effcient manner.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/ashraf18pattern-based.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/ashraf18pattern-based.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#ashraf18pattern-based\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Rizwan Ashraf, Saurabh Hukerikar, and Christian Engelmann. <b>Shrink or Substitute: Handling Process Failures in HPC Systems using In-situ Recovery<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.pdp2018.org\" target=\"www.pdp2018.org\">26th Euromicro International Conference on Parallel, Distributed, and network-based Processing (PDP) 2018<\/a><\/i>, pages 178-185, Cambridge, UK, March 21-23, 2018. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 978-1-5386-4975-6. ISSN 2377-5750. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/PDP2018.2018.00032\" target=\"publication\">10.1109\/PDP2018.2018.00032<\/a>. Acceptance rate 29.3% (27\/92). <a href=\"javascript:showAbstract('Efficient utilization of today&amp;#39;s high-performance computing (HPC) systems with many, complex software and hardware components requires that the HPC applications are designed to tolerate process failures at runtime. With low mean-time-to-failure (MTTF) of current and future HPC systems, long running simulations on these systems requires capabilities for gracefully handling process failures by the applications themselves. In this paper, we explore the use of fault tolerance extensions to Message Passing Interface (MPI) called user-level failure mitigation (ULFM) for handling process failures without the need to discard the progress made by the application. We explore two alternative recovery strategies, which use ULFM along with application-driven in-memory checkpointing. In the first case, the application is recovered with only the surviving processes, and in the second case, spares are used to replace the failed processes, such that the original configuration of the application is restored. Our experimental results demonstrate that graceful degradation is a viable alternative for recovery in environments where spares may not be available.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/ashraf18shrink.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/ashraf18shrink.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#ashraf18shrink\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Saurabh Hukerikar and Christian Engelmann. <b>A Pattern Language for High-Performance Computing Resilience<\/b>. In <i>Proceedings of the <a href=\"http:\/\/europlop.net\" target=\"europlop.net\">22nd European Conference on Pattern Languages of Programs (EuroPLoP) 2017<\/a><\/i>, pages 12:1-12:16, Kloster Irsee, Germany, July 12-16, 2017. <a href=\"http:\/\/www.acm.org\" target=\"www.acm.org\">ACM Press, New York, NY, USA<\/a>. ISBN 978-1-4503-4848-5. DOI <a href=\"http:\/\/dx.doi.org\/10.1145\/3147704.3147718\" target=\"publication\">10.1145\/3147704.3147718<\/a>. <a href=\"javascript:showAbstract('High-performance computing systems (HPC) provide powerful capabilities for modeling and simulation, and data analytics in a broad class of computational problems in a variety of scientific and engineering domains. HPC designs are undergoing rapid changes in the hardware architectures and the software environment as the community pursues increasingly capable HPC systems. Among the key challenges for future generations of HPC systems is the ensuring efficient and correct operation despite the occurrence of faults or defects in system components that can cause errors and failures in a HPC system. Such events affect the correctness of the scientific applications, or may lead to their untimely termination. Future generations of HPC systems will consist of millions of compute, memory and storage components and the growing complexity of these computing behemoths increases the chances that a single fault event will cascade across the machine and bring down the entire system. Design patterns capture the essential techniques that are employed to solve recurring problems in the design of resilient computing systems. However, the complexity of modern HPC systems as well as the various challenges of future generations of systems requires consideration to numerous aspects and optimization principles, such as the impact of a resilience solution on the performance and power consumption. We present a pattern language for engineering resilience solutions. The language is targeted at hardware and software designers as well as the users and operators of HPC systems. The patterns are intended to develop complete resilience solutions that have different efficiency and complexity characteristics, which may be deployed at design time or runtime to ensure that HPC systems are able to deal with various types of faults, errors and failures.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/hukerikar17pattern.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#hukerikar17pattern\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Saurabh Hukerikar and Christian Engelmann. <b>Havens: Explicit Reliable Memory Regions for HPC Applications<\/b>. In <i>Proceedings of the <a href=\"http:\/\/ieee-hpec.org\" target=\"ieee-hpec.org\">20th IEEE High Performance Extreme Computing Conference (HPEC) 2016<\/a><\/i>, pages 1-6, Waltham, MA, USA, September 13-15, 2016. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/HPEC.2016.7761593\" target=\"publication\">10.1109\/HPEC.2016.7761593<\/a>. <a href=\"javascript:showAbstract('Supporting error resilience in future exascale-class supercomputing systems is a critical challenge. Due to transistor scaling trends and increasing memory density, the scientific simulations are expected to experience more interruptions caused by soft errors in the system memory. Existing hardware-based detection and recovery techniques will be inadequate in the presence of high memory fault rates. In this paper we propose a partial memory protection scheme using region-based memory management. We define regions called havens that provide fault protection for program objects. We provide reliability for the regions through a software-based parity protection mechanism. Our approach enables critical application code and variables to be placed in these havens. The fault coverage of our approach is application agnostic unlike algorithm-based fault tolerance techniques.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/hukerikar16havens.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/hukerikar16havens.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#hukerikar16havens\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Peer-reviewed Workshop Publications<\/h4>\n<ol>\n<li>Mohit Kumar and Christian Engelmann. <b>RDPM: An Extensible Tool for Resilience Design Patterns Modeling<\/b>. In <i>Lecture Notes in Computer Science: Proceedings of the <a href=\"http:\/\/2021.euro-par.org\" target=\"2021.euro-par.org\">27th European Conference on Parallel and Distributed Computing (Euro-Par) 2021 Workshops<\/a>: <a href=\"http:\/\/www.csm.ornl.gov\/srt\/conferences\/Resilience\/2021\" target=\"www.csm.ornl.gov\/srt\/conferences\/Resilience\/2021\">14th Workshop on Resiliency in High Performance Computing (Resilience) in Clusters, Clouds, and Grids<\/a><\/i>, pages 283-297, Lisbon, Portugal, August 30, 2021. <a href=\"http:\/\/www.springer.com\" target=\"www.springer.com\">Springer Verlag, Berlin, Germany<\/a>. ISBN 978-3-031-06155-4. DOI <a href=\"http:\/\/dx.doi.org\/10.1007\/978-3-031-06156-1_23\" target=\"publication\">10.1007\/978-3-031-06156-1_23<\/a>. Acceptance rate 66.7% (4\/6). <a href=\"javascript:showAbstract('Resilience to faults, errors, and failures in extreme-scale HPC systems is a critical challenge. Resilience design patterns offer a new, structured hardware and software design approach for improving resilience. While prior work focused on developing performance, reliability, and availability models for resilience design patterns, this paper extends it by providing a Resilience Design Patterns Modeling (RDPM) tool which allows (1) exploring performance, reliability, and availability of each resilience design pattern, (2) offering customization of parameters to optimize performance, reliability, and availability, and (3) allowing investigation of trade-off models for combining multiple patterns for practical resilience solutions.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/kumar21rdpm.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#kumar21rdpm\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Mohit Kumar and Christian Engelmann. <b>Models for Resilience Design Patterns<\/b>. In <i>Proceedings of the <a href=\"http:\/\/sc20.supercomputing.org\" target=\"sc20.supercomputing.org\">33rd International Conference on High Performance Computing, Networking, Storage and Analysis (SC) Workshops 2020<\/a>: <a href=\"http:\/\/sites.google.com\/site\/ftxsworkshop\/home\/ftxs-2020\" target=\"sites.google.com\/site\/ftxsworkshop\/home\/ftxs-2020\">10th Workshop on Fault Tolerance for HPC at eXtreme Scale (FTXS) 2020<\/a><\/i>, pages 21-30, Atlanta, GA, USA, November 11, 2020. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 978-0-7381-1080-6. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/FTXS51974.2020.00008\" target=\"publication\">10.1109\/FTXS51974.2020.00008<\/a>. Acceptance rate 66.7% (6\/9). <a href=\"javascript:showAbstract('Resilience plays an important role in supercomputers by providing correct and efficient operation in case of faults, errors, and failures. Resilience design patterns offer blueprints for effectively applying resilience technologies. Prior work focused on developing initial efficiency and performance models for resilience design patterns. This paper extends it by (1) describing performance, reliability, and availability models for all structural resilience design patterns, (2) providing more detailed models that include flowcharts and state diagrams, and (3) introducing the Resilience Design Pattern Modeling (RDPM) tool that calculates and plots the performance, reliability, and availability metrics of individual patterns and pattern combinations.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/kumar20models.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/kumar20models.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#kumar20models\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Piyush Sao, Christian Engelmann, Srinivas Eswar, Oded Green, and Richard Vuduc. <b>Self-stabilizing Connected Components<\/b>. In <i>Proceedings of the <a href=\"http:\/\/sc19.supercomputing.org\" target=\"sc19.supercomputing.org\">32nd International Conference on High Performance Computing, Networking, Storage and Analysis (SC) Workshops 2019<\/a>: <a href=\"http:\/\/sites.google.com\/site\/ftxsworkshop\/home\/ftxs-2019\" target=\"sites.google.com\/site\/ftxsworkshop\/home\/ftxs-2019\">9th Workshop on Fault Tolerance for HPC at eXtreme Scale (FTXS) 2019<\/a><\/i>, pages 50-59, Denver, CO, USA, November 22, 2019. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 978-1-7281-6013-9. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/FTXS49593.2019.00011\" target=\"publication\">10.1109\/FTXS49593.2019.00011<\/a>. Acceptance rate 60.0% (6\/10). <a href=\"javascript:showAbstract('For the problem of computing the connected components of a graph, this paper considers the design of algorithms that are resilient to transient hardware faults, like bit flips. More specifically, it applies the technique of \\emphself-stabilization. A system is self-stabilizing if, when starting from a valid or invalid state, it is guaranteed to reach a valid state after a finite number of steps. Therefore on a machine subject to a transient fault, a self-stabilizing algorithm could recover if that fault caused the system to enter an invalid state. We give a comprehensive analysis of the valid and invalid states during label propagation and derive algorithms to verify and correct the invalid state. The self-stabilizing label-propagation algorithm performs &amp;#36;\\bigoV &amp;#322;og V additional computation and requires \\bigoV additional storage over its conventional counterpart (and, as such, does not increase asymptotic complexity over conventional). When run against a battery of simulated fault injection tests, the self-stabilizing label propagation algorithm exhibits more resilient behavior than a triple modular redundancy (TMR) based fault-tolerant algorithm in 80% of cases. From a performance perspective, it also outperforms TMR as it requires fewer iterations in total. Beyond the fault-tolerance properties of self-stabilizing label-propagation, we believe, they are useful from the theoretical perspective; and may have other use-cases.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/sao19self-stabilizing.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/sao19self-stabilizing.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#sao19self-stabilizing\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Rizwan Ashraf and Christian Engelmann. <b>Performance Efficient Multiresilience using Checkpoint Recovery in Iterative Algorithms<\/b>. In <i>Lecture Notes in Computer Science: Proceedings of the <a href=\"http:\/\/europar2018.org\" target=\"europar2018.org\">24th European Conference on Parallel and Distributed Computing (Euro-Par) 2018 Workshops<\/a>: <a href=\"http:\/\/www.csm.ornl.gov\/srt\/conferences\/Resilience\/2018\" target=\"www.csm.ornl.gov\/srt\/conferences\/Resilience\/2018\">11th Workshop on Resiliency in High Performance Computing (Resilience) in Clusters, Clouds, and Grids<\/a><\/i>, pages 813-825, Turin, Italy, August 28, 2018. <a href=\"http:\/\/www.springer.com\" target=\"www.springer.com\">Springer Verlag, Berlin, Germany<\/a>. ISBN 978-3-030-10549-5. DOI <a href=\"http:\/\/dx.doi.org\/10.1007\/978-3-030-10549-5_63\" target=\"publication\">10.1007\/978-3-030-10549-5_63<\/a>. Acceptance rate 50.0% (4\/8). <a href=\"javascript:showAbstract('In this paper, we address the design challenge of building multiresilient iterative high-performance computing (HPC) applications. Multiresilience in HPC applications is the ability to tolerate and maintain forward progress in the presence of both soft errors and process failures. We address the challenge by proposing performance models which are useful to design performance efficient and resilient iterative applications. The models consider the interaction between soft error and process failure resilience solutions. We experimented with a linear solver application with two distinct kinds of soft error detectors: one detector is high overhead and high accuracy, whereas the second is low overhead and low accuracy. We show how both can be leveraged for verifying the integrity of checkpointed state used to recover from both soft errors and process failures. Our results show the performance efficiency and resiliency benefit of employing the low overhead detector with high frequency within the checkpoint interval, so that timely soft error recovery can take place, resulting in less re-computed work.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/ashraf18performance.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/ashraf18performance.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#ashraf18performance\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Saurabh Hukerikar and Christian Engelmann. <b>Pattern-based Modeling of High-Performance Computing Resilience<\/b>. In <i>Lecture Notes in Computer Science: Proceedings of the <a href=\"http:\/\/europar2017.usc.es\" target=\"europar2017.usc.es\">23rd European Conference on Parallel and Distributed Computing (Euro-Par) 2017 Workshops<\/a>: <a href=\"http:\/\/www.csm.ornl.gov\/srt\/conferences\/Resilience\/2017\" target=\"www.csm.ornl.gov\/srt\/conferences\/Resilience\/2017\">10th Workshop on Resiliency in High Performance Computing (Resilience) in Clusters, Clouds, and Grids<\/a><\/i>, pages 557-568, Santiago de Compostela, Spain, August 29, 2017. <a href=\"http:\/\/www.springer.com\" target=\"www.springer.com\">Springer Verlag, Berlin, Germany<\/a>. ISBN 978-3-319-75177-1. DOI <a href=\"http:\/\/dx.doi.org\/10.1007\/978-3-319-75178-8_45\" target=\"publication\">10.1007\/978-3-319-75178-8_45<\/a>. Acceptance rate 66.7% (4\/6). <a href=\"javascript:showAbstract('The design of supercomputing systems and their applications must consider resilience, and power consumption as the key design parameters when designing to achieve higher performance. In previous work, we established a structured methodology for developing resilience solutions based on the concept of design patterns. In this paper we discuss analytical models for the design patterns to support quantitative analysis of their performance and reliability characteristics.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/hukerikar17pattern-based.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/hukerikar17pattern-based.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#hukerikar17pattern-based\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Saurabh Hukerikar, Rizwan Ashraf, and Christian Engelmann. <b>Towards New Metrics for High-Performance Computing Resilience<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.hpdc.org\/2017\" target=\"www.hpdc.org\/2017\">26th ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC) 2017<\/a>: <a href=\"http:\/\/sites.google.com\/site\/ftxsworkshop\/home\/ftxs-2017\" target=\"sites.google.com\/site\/ftxsworkshop\/home\/ftxs-2017\">7th Workshop on Fault Tolerance for HPC at eXtreme Scale (FTXS) 2017<\/a><\/i>, pages 23-30, Washington, D.C., June 26-30, 2017. <a href=\"http:\/\/www.acm.org\" target=\"www.acm.org\">ACM Press, New York, NY, USA<\/a>. ISBN 978-1-4503-5001-3. DOI <a href=\"http:\/\/dx.doi.org\/10.1145\/3086157.3086163\" target=\"publication\">10.1145\/3086157.3086163<\/a>. Acceptance rate 83.3% (5\/6). <a href=\"javascript:showAbstract('Ensuring the reliability of applications is becoming an increasingly important challenge as high-performance computing (HPC) systems experience an ever-growing number of faults, errors and failures. While the HPC community has made substantial progress in developing various resilience solutions, it continues to rely on platform-based metrics to quantify application resiliency improvements. The resilience of an HPC application is concerned with the reliability of the application outcome as well as the fault handling efficiency. To understand the scope of impact, effective coverage and performance efficiency of existing and emerging resilience solutions, there is a need for new metrics. In this paper, we develop new ways to quantify resilience that consider both the reliability and the performance characteristics of the solutions from the perspective of HPC applications. As HPC systems continue to evolve in terms of scale and complexity, it is expected that applications will experience various types of faults, errors and failures, which will require applications to apply multiple resilience solutions across the system stack. The proposed metrics are intended to be useful for understanding the combined impact of these solutions on an application&amp;#39;s ability to produce correct results and to evaluate their overall impact on an application's performance in the presence of various modes of faults.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/hukerikar17towards.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/hukerikar17towards.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#hukerikar17towards\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Saurabh Hukerikar and Christian Engelmann. <b>Language Support for Reliable Memory Regions<\/b>. In <i>Lecture Notes in Computer Science: Proceedings of the <a href=\"http:\/\/lcpc2016.wordpress.com\" target=\"lcpc2016.wordpress.com\">29th International Workshop on Languages and Compilers for Parallel Computing<\/a><\/i>, pages 73-87, Rochester, NY, USA, September 28-30, 2016. <a href=\"http:\/\/www.springer.com\" target=\"www.springer.com\">Springer Verlag, Berlin, Germany<\/a>. ISBN 978-3-319-52708-6. ISSN 0302-9743. DOI <a href=\"http:\/\/dx.doi.org\/10.1007\/978-3-319-52709-3_6\" target=\"publication\">10.1007\/978-3-319-52709-3_6<\/a>. Acceptance rate 76.9% (20\/26). <a href=\"javascript:showAbstract('The path to exascale computational capabilities in high-performance computing (HPC) systems is challenged by the evolution of the architectures of supercomputing systems. The constraints of power have driven designs that include increasingly heterogeneous architectures and complex memory hierarchies. These systems are also expected to experience in an increased rate of errors, such that the applications will no longer be able to assume correct behavior of the underlying machine. To enable the scientific community to succeed in scaling their applications and harness the capabilities of exascale systems, we need software strategies that provide mechanisms for explicit management of locality and resilience to errors in the system. In prior work, we introduced the concept of explicitly reliable memory regions, called havens. Memory management using havens supports selective reliability through a region-based approach to memory allocation. Havens enable the creation of explicit software-enabled robust memory containers for which resilient behavior is guaranteed. In this paper, we propose language support for havens through type annotations that make the structure of a program&amp;#39;s havens more explicit. We describe how the extended haven-based memory management model is implemented and the impact on the resiliency of a conjugate gradient application.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/hukerikar16language.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/hukerikar16language.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#hukerikar16language\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Peer-reviewed Conference Posters<\/h4>\n<ol>\n<li>Christian Engelmann and Mohit Kumar. <b>Resilience Design Patterns: A Structured Modeling Approach of Resilience in Computing Systems<\/b>. Poster at the <a href=\"http:\/\/www.bnl.gov\/modsim2022\" target=\"www.bnl.gov\/modsim2022\">Workshop on Modeling and Simulation of Systems and Applications (ModSim) 2022<\/a>, Seattle, WA, USA, August 10-12, 2022. <a href=\"javascript:showAbstract('Resilience to faults, errors, and failures in extreme-scale high-performance computing (HPC) systems is a critical challenge. Resilience design patterns (Figure 1) offer a new, structured hardware\/software design approach for improving resilience by identifying and evaluating repeatedly occurring resilience problems and coordinating corresponding solutions. Initial work identified and formalized these patterns and developed a proof-of-concept prototype to demonstrate portable resilience. This recent work created performance, reliability, and availability models for each of the identified 15 structural resilience design patterns and a modeling tool that allows (1) exploring the performance, reliability, and availability of each pattern, and (2) investigating the trade-offs be-tween patterns and pattern combinations.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann22resilience.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#engelmann22resilience\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann and Rizwan Ashraf. <b>Modeling and Simulation of Extreme-Scale Systems for Resilience by Design<\/b>. Poster at the <a href=\"http:\/\/www.bnl.gov\/modsim2018\" target=\"www.bnl.gov\/modsim2018\">Workshop on Modeling and Simulation of Systems and Applications<\/a>, Seattle, WA, USA, August 15-17, 2018. <a href=\"javascript:showAbstract('Resilience is a serious concern for extreme-scale high-performance computing (HPC). While the HPC community has developed various resilience solutions, the solution space remains fragmented. We created a structured approach to the design, evaluation and optimization of HPC resilience using the concept of design patterns. A design pattern describes a generalized solution to a repeatedly occurring problem. We identified the commonly occurring problems and solutions used to deal with faults, errors and failures in HPC systems. Each well-known solution that addresses a specific resilience challenge is described in the form of a design pattern. We developed a resilience design pattern specification, language and catalog, which can be used by system architects, system software and library developers, application programmers, as well as users and operators as essential building blocks when designing and deploying resilience solutions. The resilience design pattern approach provides a unique opportunity for design space exploration. As each resilience solution is abstracted as a pattern and each solution&amp;#39;s properties are defined by pattern parameters, vertical and horizontal pattern compositions can describe the resilience capabilities of an entire HPC system. This permits the investigation of beneficial or counterproductive interactions between patterns and of the performance, resilience, and power consumption trade-off between different pattern parameters and compositions. The ultimate goal is to make resilience an integral part of the HPC hardware\/software ecosystem by coordinating the various existing resilience solutions in a design space exploration process, such that the burden for providing resilience is on the system by design and not on the user as an afterthought. We are in the early stages of developing a novel design space exploration tool that enables this investigation using modeling and simulation. We developed performance and resilience models for each resilience design pattern. We also leverage results from the Catalog project, a collaborative effort between Oak Ridge National Laboratory, Argonne National Laboratory and Lawrence Livermore National Laboratory that developed models of the faults, errors and failures in today's HPC systems. We also leverage recent results from the same project by Lawrence Livermore National Laboratory in application reliability patterns. The planned research extends and combines this work to model the performance, resilience, and power consumption of an entire HPC system, initially at node-level granularity, and to simulate the dynamic interactions between deployed resilience solutions and the rest of the system. In the next iteration, finer-grain modeling and simulation, such as at the computational unit level, is used to increase accuracy. This work leverages the experience of the investigators in parallel discrete event simulation of extreme-scale systems, such as the Extreme-scale Simulator (xSim). The current state of the art in resilience modeling and simulation is fragmented as well. There is currently no such design space exploration tool. Instead, each resilience solution is typically investigated separately. There is only a small amount of work on multi-resilience solutions, including by the investigators. While there is work in investigating the performance\/resilience trade-off space, there is almost no work in including power consumption.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann18modeling2.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#engelmann18modeling2\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Onkar Patil, Saurabh Hukerikar, Frank Mueller, and Christian Engelmann. <b>Exploring Use Cases for Non-Volatile Memories in Support of HPC Resilience<\/b>. Poster at the <a href=\"http:\/\/sc11.supercomputing.org\" target=\"sc11.supercomputing.org\">30th IEEE\/ACM International Conference on High Performance  Computing, Networking, Storage and Analysis (SC) 2017<\/a>, Denver, CO, USA, November 12-17, 2017. <a href=\"javascript:showAbstract('Improving resilience and creating resilient architectures is one of the major goals of exascale computing. With the advent of Non-volatile memory technologies, memory architectures with persistent memory regions will be a significant part of future architectures. There is potential to use them in more than one way to benefit different applications. We look to take advantage of this technology to enable more fine-grained and novel methodology that will improve resilience and efficiency of exascale applications. We have developed three modes of memory usage for persistent memory to enable efficient checkpointing in HPC applications. We have developed a simple API that is evaluated with the DGEMM benchmark on a 16-node cluster with independent SSDs on every node. Our aim is to build on this work and enable static and dynamic runtime systems that will inherently make the HPC applications more fault-tolerant and resistant to errors.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/patil17exploring.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#patil17exploring\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>White Papers<\/h4>\n<ol>\n<li>Mingyan Li, Robert A. Bridges, Pablo Moriano, Christian Engelmann, Feiyi Wang, and Ryan Adamson. <b>Toward Effective Security\/Reliability Situational Awareness via Concurrent Security-or-Fault Analytics <\/b>. <i>White paper accepted at the U.S. Department of Energy&#39;s <a href=\"http:\/\/www.orau.gov\/2021ascr-cybersecurity\" target=\"www.orau.gov\/2021ascr-cybersecurity\">ASCR Workshop on Cybersecurity and Privacy for Scientific  Computing Ecosystems<\/a><\/i>, November 3-5, 2021. <a href=\"javascript:showAbstract('Modern critical infrastructures (CI) and scientific computing ecosystems (SCE) are complex and vulnerable. The complexity of CI\/SCE, such as the distributed workload found across ASCR scientific computing facilities, does not allow for easy differentiation between emerging cyber security and reliability threats. It is also not easy to correctly identify the misbehaving systems. Sometimes, system failures are just caused by unintentional user misbehavior or actual hardware\/software reliability issues, but it may take some significant amount of time and effort to develop that understanding through root-cause analysis. On the security front, CI\/SCE are vital assets. They are prime targets of, and are vulnerable to, malicious cyber-attacks. Within DoE, inter-disciplinary and cross-facility collaboration (e.g., ORNL INTERSECT initiative, next-gen supercomputing OLCF6), traditional perimeter-based defense and demarcation line between malicious cyber-attacks and non-malicious system faults are blurring. Amidst realistic reliability and security threats, the ability to effectively distinguish between non-malicious faults and malicious attacks is critical not only in root cause identification but also in countermeasures generation. ');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/li21toward.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#li21toward\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Resilience by Codesign (and not as an Afterthought)<\/b>. <i>White paper accepted at the U.S. Department of Energy&#39;s <a href=\"http:\/\/web.cvent.com\/event\/f64a4f28-b473-4808-924c-c8c3d9a2af63\/\" target=\"web.cvent.com\/event\/f64a4f28-b473-4808-924c-c8c3d9a2af63\/\">Workshop on Reimagining Codesign 2021<\/a><\/i>, March 16-18, 2021. <a href=\"javascript:showAbstract('Resilience, i.e., obtaining a correct solution in a timely and efficient manner, is one of the key challenges in extreme-scale high-performance computing (HPC). Extreme heterogeneity, i.e., using multiple, and potentially configurable, types of processors, accelerators and memory\/storage in a single computing platform, will add a significant amount of complexity to the HPC hardware\/software eco-system. Hardware\/software HPC codesign for resilience is mostly nonexistent at this point! Resilience needs to become an integral part of the HPC hardware\/software ecosystem through codesign, such that the burden for resilience is on the system by design and not on the operator or user as an afterthought. Simply put, if resilience by design is not done now, in the early stages of extreme heterogeneity, the current state of practice for HPC resilience, global application-level checkpoint\/restart, will re-main the same for decades to come due to the high costs of adoption of alternatives later on. ');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann21resilience2.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann21resilience2.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann21resilience2\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann, Rizwan Ashraf, and Saurabh Hukerikar. <b>Extreme Heterogeneity with Resilience by Design (and not as an Afterthought)<\/b>. <i>White paper accepted at the U.S. Department of Energy&#39;s <a href=\"http:\/\/orau.gov\/exheterogeneity2018\/\" target=\"orau.gov\/exheterogeneity2018\/\">Extreme Heterogeneity Virtual Workshop 2018<\/a><\/i>, January 23-24, 2018. <a href=\"javascript:showAbstract('Resilience, i.e., obtaining a correct solution in a timely and efficient manner, is one of the key challenges in extreme-scale high-performance computing (HPC). Extreme heterogeneity, i.e., using multiple, and potentially configurable, types of processors, accelerators and  memory\/storage in a single computing platform, will add a significant amount of complexity to the HPC hardware\/software ecosystem. The notion of correct computation and program state assumed by users and application developers today, which has been based on binary bit-level correctness, will no longer hold for processing elements based on quantum qubits and analog circuits that model spiking neurons in neuromorphic computing elements. The diverse set of compute and memory components in future heterogeneous systems will require novel hardware and software resilience solutions. Errors and failures reported by such heterogeneous hardware will need to be handled by the appropriate software component to enable efficient masking, recovery, and avoidance with little burden on the user. Similarly, errors and failures reported by the software running on such heterogeneous hardware need to be equally efficiently handled with little burden on the user. This requires a new approach, where resilience is holistically provided by the HPC hardware\/software ecosystem. The key challenges are to design and to operate extreme heterogeneous HPC systems with (1) wide-ranging resilience capabilities in system software, programming models, libraries, and applications, (2) interfaces and mechanisms for coordinating resilience capabilities across diverse hardware and software components, (3) appropriate metrics and tools for assessing performance, resilience, and energy, and (4) an understanding of the performance, resilience and energy trade-off that eventually results in well-informed HPC system design choices and runtime decisions.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann18extreme.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#engelmann18extreme\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Technical Reports<\/h4>\n<ol>\n<li>Christian Engelmann, Rizwan Ashraf, Saurabh Hukerikar, Mohit Kumar, and Piyush Sao. <b>Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale (Version 2.0)<\/b>. Technical Report, ORNL\/TM-2022\/2809, Oak Ridge National Laboratory, Oak Ridge, TN, USA, December 16, 2022. DOI <a href=\"http:\/\/dx.doi.org\/10.2172\/1922296\" target=\"publication\">10.2172\/1922296<\/a>. <a href=\"javascript:showAbstract('Reliability is a serious concern for future extreme-scale high-performance computing (HPC) systems. Projections based on the current generation of HPC systems and technology roadmaps suggest the prevalence of very high fault rates in future systems. The errors resulting from these faults will propagate and generate various kinds of failures, which may result in outcomes ranging from result corruptions to catastrophic application crashes. Therefore, the resilience challenge for extreme-scale HPC systems requires coordination between various hardware and software technologies that are capable of handling a broad set of fault models at accelerated fault rates. Also, due to practical limits on power consumption in future HPC systems, they are likely to embrace innovative architectures, increasing the levels of hardware and software complexities. Therefore, the techniques that seek to improve resilience must navigate the complex trade-off space between resilience and the overheads to power consumption and performance. While the HPC community has developed various resilience solutions, application-level techniques as well as system-based solutions, the solution space of HPC resilience techniques remains fragmented. There are no formal methods to integrate the various HPC resilience techniques into composite solutions, nor are there methods to holistically evaluate the adequacy and efficacy of such solutions in terms of their protection coverage, and their performance &amp; power efficiency characteristics. Additionally, few implementations of current resilience solutions are portable to newer architectures and software environments that will be deployed on future systems. We developed a new structured approach to the management of HPC resilience using the concept of resilience-based design patterns. In general, a design pattern is a repeatable solution to a commonly occurring problem. We identified the well-known solutions that are commonly used to deal with faults, errors and failures in HPC systems. In the initial design patterns specification (version 1.0), we described the various solutions, which address specific problems in the design of resilient HPC environments, in the form of patterns. Each pattern describes a problem caused by a fault, error or failure event in an HPC environment, and then describes the core of the solution of the problem in such a way that this solution may be adapted to different systems and implemented at different layers of the system stack. The catalog of these resilience design patterns provides designers with a collection of design elements. To construct complete resilience solutions using combinations of various patterns, we defined a framework that enhances HPC designers&amp;#39; understanding of the important constraints and the opportunities for the design patterns to be implemented and deployed at various layers of the system stack. The design framework is also useful for establishing interfaces and mechanisms to coordinate flexible fault management across hardware and software components, as well as to consider the trade-off between performance, resilience, and power consumption when constructing a solution. The resilience design patterns specification version 1.1 included more detailed explanations of the pattern solutions, the context in which the patterns are applicable, and the implications for hardware or software design. It also provided several additional examples and detailed case studies to demonstrate the use of patterns to build realistic solutions. In this version 1.2 of the specification document, we have improved the pattern descriptions, including graphical representations of the pattern components. These improvements are largely based on critical comments, feedback and suggestions received from pattern experts and readers of the previous versions of the specification. The pattern classification has been modified to further clarify the relationships between pattern categories. This version of the specification also introduces a pattern language for resilience design patterns. The pattern language presents the patterns in the catalog as a network, revealing the relations among the resilience patterns. The language provides designers with the means to explore alternative techniques for handling a specific fault model that may have different efficiency and complexity characteristics. Using the pattern language also enables the design and implementation of comprehensive resilience solutions as a set of interconnected resilience patterns that can be instantiated across layers of the system stack. The overall goal of this work is to provide hardware and software designers, as well as the users and operators of HPC systems, a systematic methodology for the design and evaluation of resilience technologies in HPC systems that keep scientific applications running to a correct solution in a timely and cost-efficient manner despite frequent faults, errors, and failures of various types. Version 2.0 expands the resilience design pattern classification and catalog to include self-stabilization patterns and reliability, availability and performance models for each structural pattern.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann22rdp-20.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#engelmann22rdp-20\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Saurabh Hukerikar and Christian Engelmann. <b>Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale (Version 1.2)<\/b>. Technical Report, ORNL\/TM-2017\/745, Oak Ridge National Laboratory, Oak Ridge, TN, USA, August 1, 2017. DOI <a href=\"http:\/\/dx.doi.org\/10.2172\/1436045\" target=\"publication\">10.2172\/1436045<\/a>. <a href=\"javascript:showAbstract('Reliability is a serious concern for future extreme-scale high-performance computing (HPC) systems. Projections based on the current generation of HPC systems and technology roadmaps suggest the prevalence of very high fault rates in future systems. The errors resulting from these faults will propagate and generate various kinds of failures, which may result in outcomes ranging from result corruptions to catastrophic application crashes. Therefore, the resilience challenge for extreme-scale HPC systems requires coordination between various hardware and software technologies that are capable of handling a broad set of fault models at accelerated fault rates. Also, due to practical limits on power consumption in future HPC systems, they are likely to embrace innovative architectures, increasing the levels of hardware and software complexities. Therefore, the techniques that seek to improve resilience must navigate the complex trade-off space between resilience and the overheads to power consumption and performance. While the HPC community has developed various resilience solutions, application-level techniques as well as system-based solutions, the solution space of HPC resilience techniques remains fragmented. There are no formal methods to integrate the various HPC resilience techniques into composite solutions, nor are there methods to holistically evaluate the adequacy and efficacy of such solutions in terms of their protection coverage, and their performance &amp; power efficiency characteristics. Additionally, few implementations of current resilience solutions are portable to newer architectures and software environments that will be deployed on future systems. We developed a new structured approach to the management of HPC resilience using the concept of resilience-based design patterns. In general, a design pattern is a repeatable solution to a commonly occurring problem. We identified the well-known solutions that are commonly used to deal with faults, errors and failures in HPC systems. In the initial design patterns specification (version 1.0), we described the various solutions, which address specific problems in the design of resilient HPC environments, in the form of patterns. Each pattern describes a problem caused by a fault, error or failure event in an HPC environment, and then describes the core of the solution of the problem in such a way that this solution may be adapted to different systems and implemented at different layers of the system stack. The catalog of these resilience design patterns provides designers with a collection of design elements. To construct complete resilience solutions using combinations of various patterns, we defined a framework that enhances HPC designers&amp;#39; understanding of the important constraints and the opportunities for the design patterns to be implemented and deployed at various layers of the system stack. The design framework is also useful for establishing interfaces and mechanisms to coordinate flexible fault management across hardware and software components, as well as to consider the trade-off between performance, resilience, and power consumption when constructing a solution. The resilience design patterns specification version 1.1 included more detailed explanations of the pattern solutions, the context in which the patterns are applicable, and the implications for hardware or software design. It also provided several additional examples and detailed case studies to demonstrate the use of patterns to build realistic solutions. In this version 1.2 of the specification document, we have improved the pattern descriptions, including graphical representations of the pattern components. These improvements are largely based on critical comments, feedback and suggestions received from pattern experts and readers of the previous versions of the specification. The pattern classification has been modified to further clarify the relationships between pattern categories. This version of the specification also introduces a pattern language for resilience design patterns. The pattern language presents the patterns in the catalog as a network, revealing the relations among the resilience patterns. The language provides designers with the means to explore alternative techniques for handling a specific fault model that may have different efficiency and complexity characteristics. Using the pattern language also enables the design and implementation of comprehensive resilience solutions as a set of interconnected resilience patterns that can be instantiated across layers of the system stack. The overall goal of this work is to provide hardware and software designers, as well as the users and operators of HPC systems, a systematic methodology for the design and evaluation of resilience technologies in HPC systems that keep scientific applications running to a correct solution in a timely and cost-efficient manner despite frequent faults, errors, and failures of various types.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/hukerikar17rdp-12.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#hukerikar17rdp-12\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Saurabh Hukerikar and Christian Engelmann. <b>Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale (Version 1.1)<\/b>. Technical Report, ORNL\/TM-2016\/767, Oak Ridge National Laboratory, Oak Ridge, TN, USA, December 1, 2016. DOI <a href=\"http:\/\/dx.doi.org\/10.2172\/1345793\" target=\"publication\">10.2172\/1345793<\/a>. <a href=\"javascript:showAbstract('Reliability is a serious concern for future extreme-scale high-performance computing (HPC) systems. Projections based on the current generation of HPC systems and technology roadmaps suggest the prevalence of very high fault rates in future systems. The errors resulting from these faults will propagate and generate various kinds of failures, which may result in outcomes ranging from result corruptions to catastrophic application crashes. Therefore the resilience challenge for extreme-scale HPC systems requires management of various hardware and software technologies that are capable of handling a broad set of fault models at accelerated fault rates. Also, due to practical limits on power consumption in HPC systems future systems are likely to embrace innovative architectures, increasing the levels of hardware and software complexities. As a result the techniques that seek to improve resilience must navigate the complex trade-off space between resilience and the overheads to power consumption and performance. While the HPC community has developed various resilience solutions, application-level techniques as well as system-based solutions, the solution space of HPC resilience techniques remains fragmented. There are no formal methods and metrics to investigate and evaluate resilience holistically in HPC systems that consider impact scope, handling coverage, and performance &amp; power efficiency across the system stack. Additionally, few of the current approaches are portable to newer architectures and software environments that will be deployed on future systems. In this document, we develop a structured approach to the management of HPC resilience using the concept of resilience-based design patterns. A design pattern is a general repeatable solution to a commonly occurring problem. We identify the commonly occurring problems and solutions used to deal with faults, errors and failures in HPC systems. Each established solution is described in the form of a pattern that addresses concrete problems in the design of resilient systems. The complete catalog of resilience design patterns provides designers with reusable design elements. We also define a framework that enhances a designer&amp;#39;s understanding of the important constraints and opportunities for the design patterns to be implemented and deployed at various layers of the system stack. This design framework may be used to establish mechanisms and interfaces to coordinate flexible fault management across hardware and software components. The framework also supports optimization of the cost-benefit trade-offs among performance, resilience, and power consumption. The overall goal of this work is to enable a systematic methodology for the design and evaluation of resilience technologies in extreme-scale HPC systems that keep scientific applications running to a correct solution in a timely and cost-efficient manner in spite of frequent faults, errors, and failures of various types.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/hukerikar16rdp-11.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#hukerikar16rdp-11\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Saurabh Hukerikar and Christian Engelmann. <b>Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale (Version 1.0)<\/b>. Technical Report, ORNL\/TM-2016\/687, Oak Ridge National Laboratory, Oak Ridge, TN, USA, October 1, 2016. DOI <a href=\"http:\/\/dx.doi.org\/10.2172\/1338552\" target=\"publication\">10.2172\/1338552<\/a>. <a href=\"javascript:showAbstract('Reliability is a serious concern for future extreme-scale high-performance computing (HPC) systems. Projections based on the current generation of HPC systems and technology roadmaps suggest that very high fault rates in future systems. The errors resulting from these faults will propagate and generate various kinds of failures, which may result in outcomes ranging from result corruptions to catastrophic application crashes. Practical limits on power consumption in HPC systems will require future systems to embrace innovative architectures, increasing the levels of hardware and software complexities. The resilience challenge for extreme-scale HPC systems requires management of various hardware and software technologies that are capable of handling a broad set of fault models at accelerated fault rates. These techniques must seek to improve resilience at reasonable overheads to power consumption and performance. While the HPC community has developed various solutions, application-level as well as system-based solutions, the solution space of HPC resilience techniques remains fragmented. There are no formal methods and metrics to investigate and evaluate resilience holistically in HPC systems that consider impact scope, handling coverage, and performance &amp; power efficiency across the system stack. Additionally, few of the current approaches are portable to newer architectures and software ecosystems, which are expected to be deployed on future systems. In this document, we develop a structured approach to the management of HPC resilience based on the concept of resilience-based design patterns. A design pattern is a general repeatable solution to a commonly occurring problem. We identify the commonly occurring problems and solutions used to deal with faults, errors and failures in HPC systems. The catalog of resilience design patterns provides designers with reusable design elements. We define a design framework that enhances our understanding of the important constraints and opportunities for solutions deployed at various layers of the system stack. The framework may be used to establish mechanisms and interfaces to coordinate flexible fault management across hardware and software components. The framework also enables optimization of the cost-benefit trade-offs among performance, resilience, and power consumption. The overall goal of this work is to enable a systematic methodology for the design and evaluation of resilience technologies in extreme-scale HPC systems that keep scientific applications running to a correct solution in a timely and cost-efficient manner in spite of frequent faults, errors, and failures of various types.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/hukerikar16rdp-10.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#hukerikar16rdp-10\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Talks and Lectures<\/h4>\n<ol>\n<li>Christian Engelmann. <b>Designing Smart and Resilient Extreme-Scale Systems<\/b>. Invited talk at the <a href=\"http:\/\/www.siam.org\/conferences\/cm\/conference\/pp22\" target=\"www.siam.org\/conferences\/cm\/conference\/pp22\">20th SIAM Conference on Parallel Processing for Scientific Computing (PP) 2022<\/a>, Seattle, WA, USA, February 23-26, 2022. <a href=\"javascript:showAbstract('Resilience is one of the critical challenges of extreme-scale high-performance computing (HPC) systems, as component counts increase, individual component reliability decreases, and software complexity increases. Building a reliable supercomputer that achieves the expected performance within a given cost budget and providing efficiency and correctness during operation in the presence of faults, errors, and failures requires a full understanding of the resilience problem. This talk provides an overview of recent achievements in developing a taxonomy, catalog and models that capture the observed and inferred fault, error, and failure conditions in current supercomputers and in extrapolating this knowledge to future-generation systems. It also describes the path forward in machine-in-the-loop operational intelligence for smart computing systems, leveraging operational data analytics in a loop control that maximizes productivity and minimizes costs through adaptive autonomous operation for resilience.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann22designing.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann22designing\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Smart and Resilient Extreme-Scale Systems<\/b>. Invited talk at the  <a href=\"http:\/\/www.hipeac.net\/2021\/spring-virtual\/#\/program\/sessions\/7854\/\" target=\"www.hipeac.net\/2021\/spring-virtual\/#\/program\/sessions\/7854\/\">Workshop on Resilience in High Performance Computing  (RESILIENTHPC)<\/a>, held in conjunction with the  <a href=\"http:\/\/www.hipeac.net\/2021\" target=\"www.hipeac.net\/2021\">European Network on High-performance Embedded Architecture   and Compilation (HiPEAC) Conference 2021<\/a>, Budapest, Hungary, January 19, 2021. <a href=\"javascript:showAbstract('Resilience is one of the critical challenges of extreme-scale high-performance computing (HPC) systems, as component counts increase, individual component reliability decreases, and software complexity increases. Building a reliable supercomputer that achieves the expected performance within a given cost budget and providing efficiency and correctness during operation in the presence of faults, errors, and failures requires a full understanding of the resilience problem. This talk provides an overview of recent achievements in developing a taxonomy, catalog and models that capture the observed and inferred fault, error, and failure conditions in current supercomputers and in extrapolating this knowledge to future-generation systems. It also describes the path forward in machine-in-the-loop operational intelligence for smart computing systems, leveraging operational data analytics in a loop control that maximizes productivity and minimizes costs through adaptive autonomous operation for resilience.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann21smart.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann21smart\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Resilience by Design (and not as an Afterthought)<\/b>. Invited talk at the <a href=\"http:\/\/sos23.ornl.gov\/\" target=\"sos23.ornl.gov\/\">23rd Workshop on Distributed Supercomputing (SOS) 2019<\/a>, Asheville, NC, USA, March 26-29, 2018. <a href=\"javascript:showAbstract('Resilience, i.e., obtaining a correct solution in a timely and efficient manner, is one of the key challenges in extreme-scale high-performance computing (HPC). The challenge is to build a reliable HPC system within a given cost budget that achieves the expected performance. Every generation of supercomputers deployed at Oak Ridge National Laboratory (ORNL) had to deal with expected and unexpected faults, errors and failures. While these supercomputers are designed to deal with expected issues, unexpected reliability problems can lead to severe degradation in operational capabilities. For example, ORNL&amp;#39;s Titan supercomputer experienced an unexpected increase in general-purpose graphics processing unit (GPGPU) failures between 2015 and 2017. At the peak of the problem, Titan was losing an average of 12 GPGPUs (and corresponding compute nodes) per day. Over 50% of its 18,688 GPGPUs had to be replaced. The system and the applications using it were never designed to handle such a high failure rate in an efficient manner. Other past unexpected reliability issues with supercomputers at US Department of Energy HPC centers were caused by early wear-out, dirty power, bad solder, other manufacturing issues, design errors in hardware, design errors in software and user errors. With the expected decrease in reliability due to component count increases, process technology challenges, hardware heterogeneity and software complexity, risk mitigation against unexpected issues is becoming paramount to ensure the success of future extreme-scale HPC systems. Resilience needs to be holistically provided by the HPC hardware\/software ecosystem. The key challenges are to design and to operate extreme HPC systems with (1) wide-ranging resilience capabilities in hardware, system software, programming models, libraries, and applications, (2) interfaces and mechanisms for coordinating resilience capabilities across diverse hardware and software components, (3) appropriate metrics and tools for assessing performance, resilience, and energy, and (4) an understanding of the performance, resilience and energy trade-off that eventually results in well-informed HPC system design choices and runtime decisions.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann19resilience2.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann19resilience2\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann and Rizwan Ashraf. <b>Modeling and Simulation of Extreme-Scale Systems for Resilience by Design<\/b>. Invited talk at the <a href=\"http:\/\/www.bnl.gov\/modsim2018\" target=\"www.bnl.gov\/modsim2018\">Workshop on Modeling and Simulation of Systems and Applications<\/a>, Seattle, WA, USA, August 15-17, 2018. <a href=\"javascript:showAbstract('Resilience is a serious concern for extreme-scale high-performance computing (HPC). While the HPC community has developed various resilience solutions, the solution space remains fragmented. We created a structured approach to the design, evaluation and optimization of HPC resilience using the concept of design patterns. A design pattern describes a generalized solution to a repeatedly occurring problem. We identified the commonly occurring problems and solutions used to deal with faults, errors and failures in HPC systems. Each well-known solution that addresses a specific resilience challenge is described in the form of a design pattern. We developed a resilience design pattern specification, language and catalog, which can be used by system architects, system software and library developers, application programmers, as well as users and operators as essential building blocks when designing and deploying resilience solutions. The resilience design pattern approach provides a unique opportunity for design space exploration. As each resilience solution is abstracted as a pattern and each solution&amp;#39;s properties are defined by pattern parameters, vertical and horizontal pattern compositions can describe the resilience capabilities of an entire HPC system. This permits the investigation of beneficial or counterproductive interactions between patterns and of the performance, resilience, and power consumption trade-off between different pattern parameters and compositions. The ultimate goal is to make resilience an integral part of the HPC hardware\/software ecosystem by coordinating the various existing resilience solutions in a design space exploration process, such that the burden for providing resilience is on the system by design and not on the user as an afterthought. We are in the early stages of developing a novel design space exploration tool that enables this investigation using modeling and simulation. We developed performance and resilience models for each resilience design pattern. We also leverage results from the Catalog project, a collaborative effort between Oak Ridge National Laboratory, Argonne National Laboratory and Lawrence Livermore National Laboratory that developed models of the faults, errors and failures in today's HPC systems. We also leverage recent results from the same project by Lawrence Livermore National Laboratory in application reliability patterns. The planned research extends and combines this work to model the performance, resilience, and power consumption of an entire HPC system, initially at node-level granularity, and to simulate the dynamic interactions between deployed resilience solutions and the rest of the system. In the next iteration, finer-grain modeling and simulation, such as at the computational unit level, is used to increase accuracy. This work leverages the experience of the investigators in parallel discrete event simulation of extreme-scale systems, such as the Extreme-scale Simulator (xSim). The current state of the art in resilience modeling and simulation is fragmented as well. There is currently no such design space exploration tool. Instead, each resilience solution is typically investigated separately. There is only a small amount of work on multi-resilience solutions, including by the investigators. While there is work in investigating the performance\/resilience trade-off space, there is almost no work in including power consumption.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann18modeling.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann18modeling\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Pattern-based Modeling of Fail-stop and Soft-error Resilience for Iterative Linear Solvers<\/b>. Invited talk at the <a href=\"http:\/\/www.siam.org\/meetings\/pp18\/\" target=\"www.siam.org\/meetings\/pp18\/\">18th SIAM Conference on Parallel Processing for Scientific Computing (PP) 2018<\/a>, Tokyo, Japan, March 7-10, 2018. <a href=\"javascript:showAbstract('Reliability is a serious concern for future extreme-scale high-performance computing (HPC). While the HPC community has developed various resilience solutions, the solution space remains fragmented. With this work, we develop a structured approach to the design, evaluation and optimization of HPC resilience using the concept of design patterns. We identify the problems caused by faults, errors and failures in HPC systems and the techniques used to deal with these events. Each well-known solution that addresses a specific resilience challenge is described in the form of a pattern. We develop a catalog of such resilience design patterns, which may be used by system architects, system software and tools developers, application programmers, as well as users and operators as essential building blocks when designing and deploying resilience solutions. We also develop a design framework that enhances a designer&amp;#39;s understanding the opportunities for integrating multiple patterns across layers of the system stack and the important constraints during implementation of the individual patterns. It is also useful for designing mechanisms and interfaces to coordinate flexible fault management across hardware and software components. The resilience patterns and the design framework also enable exploration and evaluation of design alternatives and support optimization of the cost-benefit trade-offs among performance, protection coverage, and power consumption of resilience solutions.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann18pattern-based.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann18pattern-based\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Resilience Design Patterns: A Structured Approach to Resilience at Extreme Scale<\/b>. Invited talk at the <a href=\"http:\/\/www.siam.org\/meetings\/pp18\/\" target=\"www.siam.org\/meetings\/pp18\/\">18th SIAM Conference on Parallel Processing for Scientific Computing (PP) 2018<\/a>, Tokyo, Japan, March 7-10, 2018. <a href=\"javascript:showAbstract('The reliability of high-performance computing (HPC) platforms is among the most critical challenges as systems continue to increase component counts, while the individual component reliability decreases and software complexity increases. While most resilience solutions are designed to address a specific fault model, HPC applications must contend with extremely high rates of faults from various sources with different levels of severity. Therefore, resilience for extreme-scale HPC systems and their applications requires an integrated approach, which leverages detection, containment and mitigation capabilities from different layers of the HPC environment. With this work, we propose an approach based on design patterns to explore a multi-level resilience solution that addresses silent data corruptions and process failures. The structured approach enables evaluation of the key components of a multi-level resilience solution using pattern performance models and systematically integrating the patterns into a complete solution by assessing the interplay between the patterns. We describe the design steps to develop a multi-level resilience solution for an iterative linear solver application that combines algorithmic resilience features of the solver with the fault tolerance primitives provided by ULFM MPI. Our results demonstrate the viability of designing HPC applications capable of surviving simultaneous injection of hard and soft errors in a performance efficient manner.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann18resilience.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann18resilience\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Toward A Fault Model And Resilience Design Patterns For Extreme Scale Systems<\/b>. Keynote talk at the  <a href=\"http:\/\/www.csm.ornl.gov\/srt\/conferences\/Resilience\/2015\" target=\"www.csm.ornl.gov\/srt\/conferences\/Resilience\/2015\">8th  Workshop on Resiliency in High Performance Computing  (Resilience) in Clusters, Clouds, and Grids<\/a>, held in  conjunction with the <a href=\"http:\/\/europar2014.dcc.fc.up.pt\" target=\"europar2014.dcc.fc.up.pt\">21st European Conference on Parallel and Distributed  Computing (Euro-Par) 2015<\/a>, Vienna, Austria, August 24-28, 2015. <a href=\"javascript:showAbstract('The path to exascale computing poses several research challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Resilience, i.e., providing efficiency and correctness in the presence of faults, is one of the most important exascale computer science challenges as systems scale up in component count (100,000-1,000,000 nodes with 1,000-10,000 cores per node by 2022) and component reliability decreases (7 nm technology with near-threshold voltage operation by 2022). This talk provides an overview of two recently funded projects. The Characterizing Faults, Errors, and Failures in Extreme-Scale Systems project identifies, categorizes and models the fault, error and failure properties of US Department of Energy high-performance computing (HPC) systems. It develops a fault taxonomy, catalog and models that capture the observed and inferred conditions in current systems and extrapolate this knowledge to exascale HPC systems. The Resilience Design Patterns project will increase the ability of scientific applications to reach accurate solutions in a timely and efficient manner. Using a novel design pattern concept, it identifies and evaluates repeatedly occurring resilience problems and coordinates solutions throughout high-performance computing hardware and software.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann15toward.ppt.pdf\" target=\"publication\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann15toward\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<p><em><small>Symbols: <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/txt.gif\" border=\"0\" alt=\"Abstract\" height=\"10pt\"> Abstract, <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/pdf.gif\" border=\"0\" alt=\"Publication\" height=\"10pt\"> Publication, <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/ppt.gif\" border=\"0\" alt=\"Presentation\" height=\"10pt\"> Presentation, <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/bib.gif\" border=\"0\" alt=\"BibTeX Citation\" height=\"10pt\"> BibTeX Citation<\/small><\/em><\/p>\n<p><script language=\"JavaScript\">\nfunction showAbstract (text) {\n  var width  = 400;\n  var height = 400;\n  var left   = (screen.width  - width ) \/ 2;\n  var top    = (screen.height - height) \/ 2;\n  var win    = window.open('',\n                           'Abstract',\n                           'width='  + width  + ', ' + \n                           'height=' + height + ', ' +\n                           'left='   + left   + ', ' +\n                           'top='    + top    + ', ' +\n                           'toolbar=no, '     +\n                           'location=no, '    +\n                           'directories=no, ' +\n                           'status=no, '      +\n                           'menubar=no, '     +\n                           'copyhistory=no, ' +\n                           'scrollbars=yes, ' +\n                           'resizable=yes')\n  win.document.write(text);\n  win.document.close();\n}\n<\/script><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Summary: Resilience design patterns offer a new, structured hardware\/software design approach for improving resilience by identifying and evaluating repeatedly occurring resilience problems and coordinating corresponding solutions. They permit resilience to become an integral part of the high-performance computing hardware\/software ecosystem through co-design, such that the burden for providing resilience is on the system by design&hellip;&nbsp;<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":430,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"neve_meta_sidebar":"","neve_meta_container":"","neve_meta_enable_content_width":"off","neve_meta_content_width":100,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"","footnotes":""},"class_list":["post-475","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/475","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=475"}],"version-history":[{"count":19,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/475\/revisions"}],"predecessor-version":[{"id":1253,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/475\/revisions\/1253"}],"up":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/430"}],"wp:attachment":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=475"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}