{"id":244,"date":"2023-02-18T08:00:30","date_gmt":"2023-02-18T08:00:30","guid":{"rendered":"https:\/\/christian-engelmann.de\/?page_id=244"},"modified":"2023-02-19T01:00:24","modified_gmt":"2023-02-19T01:00:24","slug":"2012-14-hardware-software-resilience-co-design-tools-for-extreme-scale-high-performance-computing","status":"publish","type":"page","link":"https:\/\/www.christian-engelmann.info\/?page_id=244","title":{"rendered":"2012-14: Hardware\/Software Resilience Co-Design Tools for Extreme-scale High-Performance Computing"},"content":{"rendered":"<p>The path to exascale computing poses several research challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Resilience, i.e., providing efficiency and correctness in the presence of faults, is one of the most important exascale computer science challenges as systems scale up in component count (100,000-1,000,000 nodes with 1,000-10,000 cores per node by 2020) and component reliability decreases (7 nm technology with near-threshold voltage operation by 2020). Several high-performance computing (HPC) resilience technologies have been developed. However, there are currently no tools, methods, and metrics to compare them and to identify the cost\/benefit trade-off between the key system design factors: performance, resilience, and power consumption.<\/p>\n<p>This work focuses on developing a resilience co-design toolkit with definitions, metrics, and methods to evaluate the cost\/benefit trade-off of resilience solutions, identify hardware\/software resilience properties, and coordinate interfaces\/responsibilities of individual hardware\/software components. The primary goal of this project is to provide the tools and data needed by HPC vendors to decide on future architectures and to enable direct feedback to HPC vendors on emerging resilience threats.<\/p>\n<h4>Prominent Solutions<\/h4>\n<ul>\n<li><a href=\"?page_id=433\">xSim: The Extreme-scale Simulator<\/a><\/li>\n<\/ul>\n<h4>Funding Sources<\/h4>\n<ul>\n<li>Laboratory Directed Research and Development, <a href=\"http:\/\/www.ornl.gov\" target=\"www.ornl.gov\" rel=\"noopener\">Oak Ridge National Laboratory<\/a>\n<\/li>\n<\/ul>\n<h4>Participating Institutions<\/h4>\n<ul>\n<li><a href=\"http:\/\/www.ornl.gov\" target=\"www.ornl.gov\" rel=\"noopener\">Oak Ridge National Laboratory<\/a><\/li>\n<li><a href=\"http:\/\/www.ncsu.edu\" target=\"www.ncsu.edu\" rel=\"noopener\">North Carolina State University<\/a><\/li>\n<\/ul>\n<h4>Peer-reviewed Journal Publications<\/h4>\n<ol>\n<li>Christian Engelmann and Thomas Naughton. <b>A New Deadlock Resolution Protocol and Message Matching Algorithm for the Extreme-scale Simulator<\/b>. <i><a href=\"http:\/\/onlinelibrary.wiley.com\/journal\/10.1002\/(ISSN)1532-0634\" target=\"onlinelibrary.wiley.com\/journal\/10.1002\/(ISSN)1532-0634\" rel=\"noopener\">Concurrency and Computation: Practice and Experience<\/a><\/i>, volume 28, number 12, pages 3369-3389, August 1, 2016. <a href=\"http:\/\/www.wiley.com\" target=\"www.wiley.com\" rel=\"noopener\">John Wiley &#038; Sons, Inc.<\/a>. ISSN 1532-0634. DOI <a href=\"http:\/\/dx.doi.org\/10.1002\/cpe.3805\" target=\"publication\" rel=\"noopener\">10.1002\/cpe.3805<\/a>. <a href=\"javascript:showAbstract('Investigating the performance of parallel applications at scale on future high-performance computing (HPC) architectures and the performance impact of different HPC architecture choices is an important component of HPC hardware\/software co-design. The Extreme-scale Simulator (xSim) is a simulation toolkit for investigating the performance of parallel applications at scale. xSim scales to millions of simulated Message Passing Interface (MPI) processes. The xSim toolkit strives to limit simulation overheads in order to maintain performance and productivity criteria. This paper documents two improvements to xSim: (1) a new deadlock resolution protocol to reduce the parallel discrete event simulation overhead, and (2) a new simulated MPI message matching algorithm to reduce the oversubscription management cost. These enhancements resulted in significant performance improvements. The simulation overhead for running the NAS Parallel Benchmark suite dropped from 1,020% to 238% for the conjugate gradient (CG) benchmark and 102% to 0% for the embarrassingly parallel (EP) benchmark. Additionally, the improvements were beneficial for reducing overheads in the highly accurate simulation mode of xSim, which is useful for resilience investigation studies for tracking intentional MPI process failures. In the highly accurate mode, the simulation overhead was reduced from 37,511% to 13,808% for CG and from 3,332% to 204% for EP.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann16new.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#engelmann16new\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Scaling To A Million Cores And Beyond: Using Light-Weight Simulation to Understand The Challenges Ahead On The Road To Exascale<\/b>. <i><a href=\"http:\/\/www.elsevier.com\/locate\/fgcs\" target=\"www.elsevier.com\/locate\/fgcs\" rel=\"noopener\">Future Generation Computer Systems (FGCS)<\/a><\/i>, volume 30, number 0, pages 59-65, January 1, 2014. <a href=\"http:\/\/www.elsevier.com\" target=\"www.elsevier.com\" rel=\"noopener\">Elsevier B.V, Amsterdam, The Netherlands<\/a>. ISSN 0167-739X. DOI <a href=\"http:\/\/dx.doi.org\/10.1016\/j.future.2013.04.014\" target=\"publication\" rel=\"noopener\">10.1016\/j.future.2013.04.014<\/a>. <a href=\"javascript:showAbstract('As supercomputers scale to 1,000 PFlop\/s over the next decade, investigating the performance of parallel applications at scale on future architectures and the performance impact of different architecture choices for high-performance computing (HPC) hardware\/software co-design is crucial. This paper summarizes recent efforts in designing and implementing a novel HPC hardware\/software co-design toolkit. The presented Extreme-scale Simulator (xSim) permits running an HPC application in a controlled environment with millions of concurrent execution threads while observing its performance in a simulated extreme-scale HPC system using architectural models and virtual timing. This paper demonstrates the capabilities and usefulness of the xSim performance investigation toolkit, such as its scalability to 2^27 simulated Message Passing Interface (MPI) ranks on 960 real processor cores, the capability to evaluate the performance of different MPI collective communication algorithms, and the ability to evaluate the performance of a basic Monte Carlo application with different architectural parameters.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann13scaling.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#engelmann13scaling\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Peer-reviewed Conference Publications<\/h4>\n<ol>\n<li>Mahesh Lagadapati, Frank Mueller, and Christian Engelmann. <b>Benchmark Generation and Simulation at Extreme Scale<\/b>. In <i>Proceedings of the <a href=\"http:\/\/ds-rt.com\/2016\" target=\"ds-rt.com\/2016\" rel=\"noopener\">20th IEEE\/ACM International Symposium on Distributed Simulation and Real Time Applications (DS-RT) 2016<\/a><\/i>, pages 9-18, London, UK, September 21-23, 2016. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\" rel=\"noopener\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 978-1-5090-3506-9. ISSN 1550-6525. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/DS-RT.2016.18\" target=\"publication\" rel=\"noopener\">10.1109\/DS-RT.2016.18<\/a>. Acceptance rate 42.0% (21\/50). Best paper candidate. <a href=\"javascript:showAbstract('The path to extreme scale high-performance computing (HPC) poses several challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Investigating the performance of parallel applications at scale on future architectures and the performance impact of different architectural choices is an important component of HPC hardware\/software co-design. Simulations using models of future HPC systems and communication traces from applications running on existing HPC systems can offer an insight into the performance of future architectures. This work targets technology developed for scalable application tracing of communication events. It focuses on extreme-scale simulation of HPC applications and their communication behavior via lightweight parallel discrete event simulation for performance estimation and evaluation. Instead of simply replaying a trace within a simulator, this work promotes the generation of a benchmark from traces. This benchmark is subsequently exposed to simulation using models to reflect the performance characteristics of future-generation HPC systems. This technique provides a number of benefits, such as eliminating the data intensive trace replay and enabling simulations at different scales. The presented work features novel software co-design aspects, combining the ScalaTrace tool to generate scalable trace files, the ScalaBenchGen tool to generate the benchmark, and the xSim tool to assess the benchmark characteristics within a simulator.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/lagadapati16benchmark.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/lagadapati16benchmark.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#lagadapati16benchmark\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann and Thomas Naughton. <b>Supporting the Development of Soft-Error Resilient Message Passing Applications using Simulation<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.iasted.org\/conferences\/home-795.html\" target=\"www.iasted.org\/conferences\/home-795.html\" rel=\"noopener\">13th IASTED International Conference on Parallel and Distributed Computing and Networks (PDCN) 2016<\/a><\/i>, Innsbruck, Austria, February 15-16, 2016. <a href=\"http:\/\/www.actapress.com\" target=\"www.actapress.com\" rel=\"noopener\">ACTA Press, Calgary, AB, Canada<\/a>. ISBN 978-0-88986-979-0. DOI <a href=\"http:\/\/dx.doi.org\/10.2316\/P.2016.834-005\" target=\"publication\" rel=\"noopener\">10.2316\/P.2016.834-005<\/a>. <a href=\"javascript:showAbstract('Radiation-induced bit flip faults are of particular concern in extreme-scale high-performance computing systems. This paper presents a simulation-based tool that enables the development of soft-error resilient message passing applications by permitting the investigation of their correctness and performance under various fault conditions. The documented extensions to the Extreme-scale Simulator (xSim) enable the injection of bit flip faults at specific of injection location(s) and fault activation time(s), while supporting a significant degree of configurability of the fault type. Experiments show that the simulation overhead with the new feature is ~2,325% for serial execution and ~1,730% at 128 MPI processes, both with very fine-grain fault injection. Fault injection experiments demonstrate the usefulness of the new feature by injecting bit flips in the input and output matrices of a matrix-matrix multiply application, revealing vulnerability of data structures, masking and error propagation. xSim is the very first simulation-based MPI performance tool that supports both, the injection of process failures and bit flip faults.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann16supporting.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann16supporting.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann16supporting\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann and Thomas Naughton. <b>A Network Contention Model for the Extreme-scale Simulator<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.iasted.org\/conferences\/home-826.html\" target=\"www.iasted.org\/conferences\/home-826.html\" rel=\"noopener\">34th IASTED International Conference on Modelling, Identification and Control (MIC) 2015<\/a><\/i>, Innsbruck, Austria, February 17-18, 2015. <a href=\"http:\/\/www.actapress.com\" target=\"www.actapress.com\" rel=\"noopener\">ACTA Press, Calgary, AB, Canada<\/a>. ISBN 978-0-88986-975-2. DOI <a href=\"http:\/\/dx.doi.org\/10.2316\/P.2015.826-043\" target=\"publication\" rel=\"noopener\">10.2316\/P.2015.826-043<\/a>. <a href=\"javascript:showAbstract('The Extreme-scale Simulator (xSim) is a performance investigation toolkit for high-performance computing (HPC) hardware\/software co-design. It permits running a HPC application with millions of concurrent execution threads, while observing its performance in a simulated extreme-scale system. This paper details a newly developed network modeling feature for xSim, eliminating the shortcomings of the existing network modeling capabilities. The approach takes a different path for implementing network contention and bandwidth capacity modeling using a less synchronous and accurate enough model design. With the new network modeling feature, xSim is able to simulate on-chip and on-node networks with reasonable accuracy and overheads.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann15network.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann15network.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann15network\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann and Thomas Naughton. <b>Improving the Performance of the Extreme-scale Simulator<\/b>. In <i>Proceedings of the <a href=\"http:\/\/ds-rt.com\/2014\" target=\"ds-rt.com\/2014\" rel=\"noopener\">18th IEEE\/ACM International Symposium on Distributed Simulation and Real Time Applications (DS-RT) 2014<\/a><\/i>, pages 198-207, Toulouse, France, October 1-3, 2014. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\" rel=\"noopener\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 978-1-4799-6143-6. ISSN 1550-6525. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/DS-RT.2014.32\" target=\"publication\" rel=\"noopener\">10.1109\/DS-RT.2014.32<\/a>. Best paper candidate. <a href=\"javascript:showAbstract('Investigating the performance of parallel applications at scale on future high-performance computing (HPC) architectures and the performance impact of different architecture choices is an important component of HPC hardware\/software co-design. The Extreme-scale Simulator (xSim) is a simulation-based toolkit for investigating the performance of parallel applications at scale. xSim scales to millions of simulated Message Passing Interface (MPI) processes. The overhead introduced by a simulation tool is an important performance and productivity aspect. This paper documents two improvements to xSim: (1) a new deadlock resolution protocol to reduce the parallel discrete event simulation management overhead and (2) a new simulated MPI message matching algorithm to reduce the oversubscription management overhead. The results clearly show a significant performance improvement, such as by reducing the simulation overhead for running the NAS Parallel Benchmark suite inside the simulator  from 1,020% to 238% for the conjugate gradient (CG) benchmark and from 102% to 0% for the embarrassingly parallel (EP) and benchmark, as well as, from 37,511% to 13,808% for CG and from 3,332% to 204% for EP with accurate process failure simulation.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann14improving.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann14improving.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann14improving\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Thomas Naughton, Christian Engelmann, Geoffroy Vall&eacute;e, and Swen B&ouml;hm. <b>Supporting the Development of Resilient Message Passing Applications using Simulation<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.pdp2014.org\" target=\"www.pdp2014.org\" rel=\"noopener\">22nd Euromicro International Conference on Parallel, Distributed, and network-based Processing (PDP) 2014<\/a><\/i>, pages 271-278, Turin, Italy, February 12-14, 2014. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\" rel=\"noopener\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISSN 1066-6192. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/PDP.2014.74\" target=\"publication\" rel=\"noopener\">10.1109\/PDP.2014.74<\/a>. Acceptance rate 32.6% (73\/224). <a href=\"javascript:showAbstract('An emerging aspect of high-performance computing (HPC) hardware\/software co-design is investigating performance under failure. The work in this paper extends the Extreme-scale Simulator (xSim), which was designed for evaluating the performance of message passing interface (MPI) applications on future HPC architectures, with fault-tolerant MPI extensions proposed by the MPI Fault Tolerance Working Group. xSim permits running MPI applications with millions of concurrent MPI ranks, while observing application performance in a simulated extreme-scale system using a lightweight parallel discrete event simulation. The newly added features offer user-level failure mitigation (ULFM) extensions at the simulated MPI layer to support algorithm-based fault tolerance (ABFT). The presented solution permits investigating performance under failure and failure handling of ABFT solutions. The newly enhanced xSim is the very first performance tool that supports ULFM and ABFT.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/naughton14supporting.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/naughton14supporting.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#naughton14supporting\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Peer-reviewed Workshop Publications<\/h4>\n<ol>\n<li>Christian Engelmann and Thomas Naughton. <b>Toward a Performance\/Resilience Tool for Hardware\/Software Co-Design of High-Performance Computing Systems<\/b>. In <i>Proceedings of the <a href=\"http:\/\/icpp2013.ens-lyon.fr\" target=\"icpp2013.ens-lyon.fr\" rel=\"noopener\">42nd International Conference on Parallel Processing (ICPP) 2013<\/a>: <a href=\"http:\/\/www.psti-workshop.org\" target=\"www.psti-workshop.org\" rel=\"noopener\">4th International Workshop on Parallel Software Tools and Tool Infrastructures (PSTI)<\/a><\/i>, pages 962-971, Lyon, France, October 2, 2013. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\" rel=\"noopener\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 978-0-7695-5117-3. ISSN 0190-3918. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/ICPP.2013.114\" target=\"publication\" rel=\"noopener\">10.1109\/ICPP.2013.114<\/a>. <a href=\"javascript:showAbstract('xSim is a simulation-based performance investigation toolkit that permits running high-performance computing (HPC) applications in a controlled environment with millions of concurrent execution threads, while observing application performance in a simulated extreme-scale system for hardware\/software co-design. The presented work details newly developed features for xSim that permit the injection of MPI process failures, the propagation\/detection\/notification of such failures within the simulation, and their handling using application-level checkpoint\/restart. These new capabilities enable the observation of application behavior and performance under failure within a simulated future-generation HPC system using the most common fault handling technique.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann13toward.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann13toward.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann13toward\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Mahesh Lagadapati, Frank Mueller, and Christian Engelmann. <b>Tools for Simulation and Benchmark Generation at Exascale<\/b>. In <i>Proceedings of the <a href=\"http:\/\/tools.zih.tu-dresden.de\/2013\/\" target=\"tools.zih.tu-dresden.de\/2013\/\" rel=\"noopener\">7th Parallel Tools Workshop<\/a><\/i>, pages 19-24, Dresden, Germany, September 3-4, 2013. <a href=\"http:\/\/www.springer.com\" target=\"www.springer.com\" rel=\"noopener\">Springer Verlag, Berlin, Germany<\/a>. ISBN 978-3-319-08143-4. DOI <a href=\"http:\/\/dx.doi.org\/10.1007\/978-3-319-08144-1_2\" target=\"publication\" rel=\"noopener\">10.1007\/978-3-319-08144-1_2<\/a>. <a href=\"javascript:showAbstract('The path to exascale high-performance computing (HPC) poses several challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Investigating the performance of parallel applications at scale on future architectures and the performance impact of different architecture choices is an important component of HPC hardware\/software co-design. Simulations using models of future HPC systems and communication traces from applications running on existing HPC systems can offer an insight into the performance of future architectures. This work targets technology developed for scalable application tracing of communication events and memory profiles, but can be extended to other areas, such as I\/O, control flow, and data flow. It further focuses on extreme-scale simulation of millions of Message Passing Interface (MPI) ranks using a lightweight parallel discrete event simulation (PDES) toolkit for performance evaluation. Instead of simply replaying a trace within a simulation, the approach is to generate a benchmark from it and to run this benchmark within a simulation using models to reflect the performance characteristics of future-generation HPC systems. This provides a number of benefits, such as eliminating the data intensive trace replay and enabling simulations at different scales. The presented work utilizes the ScalaTrace tool to generate scalable trace files, the ScalaBenchGen tool to generate the benchmark, and the xSim tool to run the benchmark within a simulation.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/lagadapati13tools.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/lagadapati13tools.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#lagadapati13tools\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>White Papers<\/h4>\n<ol>\n<li>Christian Engelmann and Thomas Naughton. <b>A Hardware\/Software Performance\/Resilience\/Power Co-Design Tool for Extreme-scale Computing<\/b>. <i>White paper accepted at the U.S. Department of Energy&#39;s <a href=\"http:\/\/hpc.pnl.gov\/modsim\/2013\" target=\"hpc.pnl.gov\/modsim\/2013\" rel=\"noopener\">Workshop on Modeling &#038; Simulation of Exascale Systems &#038; Applications (ModSim) 2013<\/a><\/i>, September 18-19, 2013. <a href=\"javascript:showAbstract('xSim is a simulation-based performance investigation toolkit that permits running high-performance computing (HPC) applications in a controlled environment with millions of concurrent execution threads, while observing application performance in a simulated extreme-scale system for hardware\/software co-design. The presented work details newly developed features for xSim that permit the injection of MPI process failures, the propagation\/detection\/notification of such failures within the simulation, and their handling using application-level checkpoint\/restart. The newly added features also offer user-level failure mitigation (ULFM) extensions at the simulated MPI layer to support algorithm-based fault tolerance (ABFT). The presented solution permits investigating performance under failure and failure handling of checkpoint\/restart and ABFT solutions. The newly enhanced xSim is the very first performance tool that supports these capabilities.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann13hardware.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann13hardware.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann13hardware\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann and Thomas Naughton. <b>A Performance\/Resilience\/Power Co-design Tool for Extreme-scale High-Performance Computing<\/b>. <i>White paper accepted at the U.S. Department of Energy&#39;s <a href=\"http:\/\/hpc.pnl.gov\/modsim\/2012\" target=\"hpc.pnl.gov\/modsim\/2012\" rel=\"noopener\">Workshop on Modeling &#038; Simulation of Exascale Systems &#038; Applications (ModSim) 2012<\/a><\/i>, August 9-10, 2012. <a href=\"javascript:showAbstract('Performance, resilience and power consumption are key HPC system design factors that are highly interde-pendent. To enable extreme-scale computing it is essential to perform HPC hardware\/software co-design that identifies the cost\/benefit trade-off between these design factors for potential future architecture choices. The proposed research and development aims at developing an HPC hardware\/software co-design toolkit for evaluating the resilience\/power\/performance cost\/benefit trade-off of future architecture choices. The approach focuses on extending a simulation-based performance investigation toolkit with advanced resilience and power modeling and simulation features, such as (i) fault injection mechanisms, (ii) fault propagation, isolation, and detection models, (i) fault avoidance, masking, and recovery simulation, and (iv) power consumption models.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann12performance.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#engelmann12performance\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Talks and Lectures<\/h4>\n<ol>\n<li>Christian Engelmann. <b>Resilience Challenges and Solutions for Extreme-Scale Supercomputing<\/b>. Invited talk at the <a href=\"http:\/\/www.usna.edu\" target=\"www.usna.edu\" rel=\"noopener\">United  States Naval Academy<\/a>, Annapolis, MD, USA, February 18, 2016. <a href=\"javascript:showAbstract('The path to exascale computing poses several research challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Resilience, i.e., providing efficiency and correctness in the presence of faults, is one of the most important exascale computer science challenges as systems scale up in component count (100,000-1,000,000 nodes with 1,000-10,000 cores per node by 2022) and component reliability decreases (7 nm technology with near-threshold voltage operation by 2022). This talk provides an overview of recent and ongoing resilience research and development activities at Oak Ridge National Laboratory in advanced checkpoint storage architectures, process-level incremental checkpoint\/restart, proactive fault tolerance using prediction-triggered process or virtual machine migration, MPI process-level software redundancy, and soft-error injection tools to study the vulnerability of science applications.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann16resilience2.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann16resilience2\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Resilience Challenges and Solutions for Extreme-Scale Supercomputing<\/b>. Invited talk at the  19th Workshop on Distributed Supercomputing (SOS)   2015, Park City, UT, USA, March 2-5, 2015. <a href=\"javascript:showAbstract('The path to exascale computing poses several research challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Resilience, i.e., providing efficiency and correctness in the presence of faults, is one of the most important exascale computer science challenges as systems scale up in component count (100,000-1,000,000 nodes with 1,000-10,000 cores per node by 2022) and component reliability decreases (7 nm technology with near-threshold voltage operation by 2022). This talk provides an overview of recent and ongoing resilience research and development activities at Oak Ridge National Laboratory in advanced checkpoint storage architectures, process-level incremental checkpoint\/restart, proactive fault tolerance using prediction-triggered process or virtual machine migration, MPI process-level software redundancy, and soft-error injection tools to study the vulnerability of science applications.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann15resilience.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann15resilience\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>xSim: The Extreme-scale Simulator<\/b>. Seminar at the <a href=\"http:\/\/www.lrz-muenchen.de\" target=\"www.lrz-muenchen.de\" rel=\"noopener\">Leibniz  Rechenzentrum (LRZ)<\/a>, Garching, Germany, February 23, 2015. <a href=\"javascript:showAbstract('The path to exascale high-performance computing (HPC) poses several challenges related to power, performance, and resilience. Investigating the performance and resilience of parallel applications at scale on future architectures and the performance and resilience impact of different architecture choices is an important component of HPC hardware\/software co-design. Without having access to future architectures at scale, simulation provides an alternative. The Extreme-scale Simulator (xSim) is a performance investigation toolkit that permits running applications in a controlled environment with millions of concurrent execution threads, while observing performance and resilience in a simulated extreme-scale system. Using a lightweight parallel discrete event simulation, xSim executes a Message Passing Interface (MPI) application on a much smaller system in a highly oversubscribed fashion with a virtual wall clock time, such that performance data can be extracted based on a processor and a network model. xSim is designed like a traditional performance tool, as an interposition library that sits between the MPI application and the MPI library, using the MPI profiling interface. It has been run up to 134,217,728 (2^27) MPI ranks using a 960-core Linux cluster. xSim also permits the injection of MPI process failures, the propagation\/detection\/notification of such failures within the simulation, and their handling within the simulation using application-level checkpoint\/restart. Another feature provides user-level failure mitigation (ULFM) extensions at the simulated MPI layer to support algorithm-based fault tolerance (ABFT). xSim is the very first performance tool that supports ULFM and ABFT.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann15xsim.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann15xsim\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Supporting the Development of Resilient Message Passing Applications using Simulation<\/b>. Invited talk at the <a href=\"http:\/\/www.dagstuhl.de\/en\/program\/calendar\/semhp\/?semnr=14402\" target=\"www.dagstuhl.de\/en\/program\/calendar\/semhp\/?semnr=14402\" rel=\"noopener\">Dagstuhl Seminar on Resilience in Exascale Computing<\/a>, Schloss Dagstuhl, Wadern, Germany, September 28 &#8211; October 1, 2014. <a href=\"javascript:showAbstract('An emerging aspect of high-performance computing (HPC) hardware\/software co-design is investigating performance under failure. The presented work extends the Extreme-scale Simulator (xSim), which was designed for evaluating the performance of message passing interface (MPI) applications on future HPC architectures, with fault-tolerant MPI extensions proposed by the MPI Fault Tolerance Working Group. xSim permits running MPI applications with millions of concurrent MPI ranks, while observing application performance in a simulated extreme-scale system using a lightweight parallel discrete event simulation. The newly added features offer user-level failure mitigation (ULFM) extensions at the simulated MPI layer to support algorithm-based fault tolerance (ABFT). The presented solution permits investigating performance under failure and failure handling of ABFT solutions. The newly enhanced xSim is the very first performance tool that supports ULFM and ABFT.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann14supporting.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann14supporting\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Resilience Challenges and Solutions for Extreme-Scale Supercomputing<\/b>. Invited talk at the Technical University of Dresden,  Dresden, Germany, September 3, 2013. <a href=\"javascript:showAbstract('With the recent deployment of the 18 PFlop\/s Titan supercomputer and the exascale roadmap targeting 100, 300, and eventually 1,000 PFlop\/s by 2022, Oak Ridge National Laboratory is at the forefront of scientific capability computing. The path to exascale computing poses several research challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Resilience, i.e., providing efficiency and correctness in the presence of faults, is one of the most important exascale computer science challenges as systems scale up in component count (100,000-1,000,000 nodes with 1,000-10,000 cores per node by 2022) and component reliability decreases (7 nm technology with near-threshold voltage operation by 2022). This talk provides an overview of recent and ongoing resilience research and development activities at Oak Ridge National Laboratory in advanced checkpoint storage architectures, process-level incremental checkpoint\/restart, proactive fault tolerance using prediction-triggered process or virtual machine migration, MPI process-level software redundancy, and soft-error injection tools to study the vulnerability of science applications and of CMOS logic in processors and memory.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann13resilience.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann13resilience\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Fault Tolerance Session<\/b>. Invited talk at the  <a href=\"http:\/\/www.aanmelder.nl\/exachallenge\" target=\"www.aanmelder.nl\/exachallenge\" rel=\"noopener\">The ExaChallenge Symposium<\/a>, Dublin, Ireland, October 16-17, 2012. <a href=\"publications\/engelmann12fault.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann12fault\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<p><em><small>Symbols: <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/txt.gif\" border=\"0\" alt=\"Abstract\" height=\"10pt\"> Abstract, <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/pdf.gif\" border=\"0\" alt=\"Publication\" height=\"10pt\"> Publication, <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/ppt.gif\" border=\"0\" alt=\"Presentation\" height=\"10pt\"> Presentation, <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/bib.gif\" border=\"0\" alt=\"BibTeX Citation\" height=\"10pt\"> BibTeX Citation<\/small><\/em><\/p>\n<p><script language=\"JavaScript\">\nfunction showAbstract (text) {\n  var width  = 400;\n  var height = 400;\n  var left   = (screen.width  - width ) \/ 2;\n  var top    = (screen.height - height) \/ 2;\n  var win    = window.open('',\n                           'Abstract',\n                           'width='  + width  + ', ' + \n                           'height=' + height + ', ' +\n                           'left='   + left   + ', ' +\n                           'top='    + top    + ', ' +\n                           'toolbar=no, '     +\n                           'location=no, '    +\n                           'directories=no, ' +\n                           'status=no, '      +\n                           'menubar=no, '     +\n                           'copyhistory=no, ' +\n                           'scrollbars=yes, ' +\n                           'resizable=yes')\n  win.document.write(text);\n  win.document.close();\n}\n<\/script><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The path to exascale computing poses several research challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Resilience, i.e., providing efficiency and correctness in the presence of faults, is one of the most important exascale computer science challenges as systems scale up in component count (100,000-1,000,000 nodes with 1,000-10,000 cores per&hellip;&nbsp;<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":145,"menu_order":23,"comment_status":"closed","ping_status":"closed","template":"","meta":{"neve_meta_sidebar":"","neve_meta_container":"","neve_meta_enable_content_width":"off","neve_meta_content_width":100,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"","footnotes":""},"class_list":["post-244","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/244","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=244"}],"version-history":[{"count":9,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/244\/revisions"}],"predecessor-version":[{"id":495,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/244\/revisions\/495"}],"up":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/145"}],"wp:attachment":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=244"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}