{"id":234,"date":"2023-02-18T08:00:11","date_gmt":"2023-02-18T08:00:11","guid":{"rendered":"https:\/\/christian-engelmann.de\/?page_id=234"},"modified":"2023-02-19T01:01:26","modified_gmt":"2023-02-19T01:01:26","slug":"2013-16-mcrex-monte-carlo-resilient-exascale-solvers","status":"publish","type":"page","link":"https:\/\/www.christian-engelmann.info\/?page_id=234","title":{"rendered":"2013-16: MCREX: Monte Carlo Resilient Exascale Solvers"},"content":{"rendered":"<p>The next generation of computational science applications require numerical solvers that are capable of high performance on proposed exascale platforms. In order to meet this goal, solvers must be resilient to soft and hard failures, provide high concurrency on heterogeneous hardware configurations, and retain numerical accuracy and efficiency. In light of these requirements, a natural avenue of inquiry would be to adapt the current stable of numerically efficient solvers to this new high-performance computing (HPC) regime. However, an alternative approach is to investigate different classes of algorithms that can address issues of resiliency naturally.<\/p>\n<p>This project investigates new stochastic methods for solving linear systems, otherwise termed Monte Carlo Resilient Exascale (MCREX) solvers. The family of methods builds on the sequential Monte Carlo work of Halton, 1962. While showing significant promise, this class of solvers has not made inroads into the broader computational science community. Our initially developed methods use Monte Carlo to accelerate a fixed-point iteration. Therefore, they are called Monte Carlo Synthetic Acceleration (MCSA). Preliminary work using MCSA has demonstrated that they are at least as efficient as Jacobi-preconditioned Conjugate Gradient (PCG) on sparse, symmetric positive definite (SPD) systems. These initial results demonstrate that, because MCSA does not require symmetry or positive definiteness, very good efficiency could be attained on non-symmetric systems, thus making MCSA an ideal solver in non-linear Newton schemes. Furthermore, Monte Carlo methods have the benefit of addressing resilience in a natural way; soft errors can be treated as high variance samples and lost histories from processor failures can be easily discarded without affecting the quality of the solution.<\/p>\n<p>The developed MCREX solver is evaluated using the Extreme-scale Simulator (xSim). xSim is a performance\/resilience investigation toolkit that permits running native HPC applications or proxy applications in a controlled environment with millions of concurrent execution threads, while observing application performance and resilience in a simulated extreme-scale system for hardware\/software co-design.<\/p>\n<h4>Prominent Solutions<\/h4>\n<ul>\n<li><a href=\"?page_id=433\">xSim: The Extreme-scale Simulator<\/a><\/li>\n<\/ul>\n<h4>Funding Sources<\/h4>\n<ul>\n<li>Resilient Extreme-Scale Solvers (RX-Solvers) Program, <a href=\"http:\/\/science.energy.gov\/ascr\" target=\"science.energy.gov_ascr\" rel=\"noopener\">Office of Advanced Scientific Computing Research<\/a>, Office of Science, U.S. Department of Energy\n  <\/li>\n<\/ul>\n<h4>Participating Institutions<\/h4>\n<ul>\n<li><a href=\"http:\/\/www.ornl.gov\" target=\"www.ornl.gov\" rel=\"noopener\">Oak Ridge National Laboratory<\/a><\/li>\n<li><a href=\"http:\/\/www.mathcs.emory.edu\" target=\"www.mathcs.emory.edu\" rel=\"noopener\">Emory University<\/a><\/li>\n<\/ul>\n<h4>Peer-reviewed Journal Publications<\/h4>\n<ol>\n<li>Christian Engelmann and Thomas Naughton. <b>A New Deadlock Resolution Protocol and Message Matching Algorithm for the Extreme-scale Simulator<\/b>. <i><a href=\"http:\/\/onlinelibrary.wiley.com\/journal\/10.1002\/(ISSN)1532-0634\" target=\"onlinelibrary.wiley.com\/journal\/10.1002\/(ISSN)1532-0634\" rel=\"noopener\">Concurrency and Computation: Practice and Experience<\/a><\/i>, volume 28, number 12, pages 3369-3389, August 1, 2016. <a href=\"http:\/\/www.wiley.com\" target=\"www.wiley.com\" rel=\"noopener\">John Wiley &#038; Sons, Inc.<\/a>. ISSN 1532-0634. DOI <a href=\"http:\/\/dx.doi.org\/10.1002\/cpe.3805\" target=\"publication\" rel=\"noopener\">10.1002\/cpe.3805<\/a>. <a href=\"javascript:showAbstract('Investigating the performance of parallel applications at scale on future high-performance computing (HPC) architectures and the performance impact of different HPC architecture choices is an important component of HPC hardware\/software co-design. The Extreme-scale Simulator (xSim) is a simulation toolkit for investigating the performance of parallel applications at scale. xSim scales to millions of simulated Message Passing Interface (MPI) processes. The xSim toolkit strives to limit simulation overheads in order to maintain performance and productivity criteria. This paper documents two improvements to xSim: (1) a new deadlock resolution protocol to reduce the parallel discrete event simulation overhead, and (2) a new simulated MPI message matching algorithm to reduce the oversubscription management cost. These enhancements resulted in significant performance improvements. The simulation overhead for running the NAS Parallel Benchmark suite dropped from 1,020% to 238% for the conjugate gradient (CG) benchmark and 102% to 0% for the embarrassingly parallel (EP) benchmark. Additionally, the improvements were beneficial for reducing overheads in the highly accurate simulation mode of xSim, which is useful for resilience investigation studies for tracking intentional MPI process failures. In the highly accurate mode, the simulation overhead was reduced from 37,511% to 13,808% for CG and from 3,332% to 204% for EP.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann16new.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#engelmann16new\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Peer-reviewed Conference Publications<\/h4>\n<ol>\n<li>Christian Engelmann and Thomas Naughton. <b>Supporting the Development of Soft-Error Resilient Message Passing Applications using Simulation<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.iasted.org\/conferences\/home-795.html\" target=\"www.iasted.org\/conferences\/home-795.html\" rel=\"noopener\">13th IASTED International Conference on Parallel and Distributed Computing and Networks (PDCN) 2016<\/a><\/i>, Innsbruck, Austria, February 15-16, 2016. <a href=\"http:\/\/www.actapress.com\" target=\"www.actapress.com\" rel=\"noopener\">ACTA Press, Calgary, AB, Canada<\/a>. ISBN 978-0-88986-979-0. DOI <a href=\"http:\/\/dx.doi.org\/10.2316\/P.2016.834-005\" target=\"publication\" rel=\"noopener\">10.2316\/P.2016.834-005<\/a>. <a href=\"javascript:showAbstract('Radiation-induced bit flip faults are of particular concern in extreme-scale high-performance computing systems. This paper presents a simulation-based tool that enables the development of soft-error resilient message passing applications by permitting the investigation of their correctness and performance under various fault conditions. The documented extensions to the Extreme-scale Simulator (xSim) enable the injection of bit flip faults at specific of injection location(s) and fault activation time(s), while supporting a significant degree of configurability of the fault type. Experiments show that the simulation overhead with the new feature is ~2,325% for serial execution and ~1,730% at 128 MPI processes, both with very fine-grain fault injection. Fault injection experiments demonstrate the usefulness of the new feature by injecting bit flips in the input and output matrices of a matrix-matrix multiply application, revealing vulnerability of data structures, masking and error propagation. xSim is the very first simulation-based MPI performance tool that supports both, the injection of process failures and bit flip faults.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann16supporting.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann16supporting.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann16supporting\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann and Thomas Naughton. <b>A Network Contention Model for the Extreme-scale Simulator<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.iasted.org\/conferences\/home-826.html\" target=\"www.iasted.org\/conferences\/home-826.html\" rel=\"noopener\">34th IASTED International Conference on Modelling, Identification and Control (MIC) 2015<\/a><\/i>, Innsbruck, Austria, February 17-18, 2015. <a href=\"http:\/\/www.actapress.com\" target=\"www.actapress.com\" rel=\"noopener\">ACTA Press, Calgary, AB, Canada<\/a>. ISBN 978-0-88986-975-2. DOI <a href=\"http:\/\/dx.doi.org\/10.2316\/P.2015.826-043\" target=\"publication\" rel=\"noopener\">10.2316\/P.2015.826-043<\/a>. <a href=\"javascript:showAbstract('The Extreme-scale Simulator (xSim) is a performance investigation toolkit for high-performance computing (HPC) hardware\/software co-design. It permits running a HPC application with millions of concurrent execution threads, while observing its performance in a simulated extreme-scale system. This paper details a newly developed network modeling feature for xSim, eliminating the shortcomings of the existing network modeling capabilities. The approach takes a different path for implementing network contention and bandwidth capacity modeling using a less synchronous and accurate enough model design. With the new network modeling feature, xSim is able to simulate on-chip and on-node networks with reasonable accuracy and overheads.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann15network.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann15network.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann15network\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann and Thomas Naughton. <b>Improving the Performance of the Extreme-scale Simulator<\/b>. In <i>Proceedings of the <a href=\"http:\/\/ds-rt.com\/2014\" target=\"ds-rt.com\/2014\" rel=\"noopener\">18th IEEE\/ACM International Symposium on Distributed Simulation and Real Time Applications (DS-RT) 2014<\/a><\/i>, pages 198-207, Toulouse, France, October 1-3, 2014. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\" rel=\"noopener\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 978-1-4799-6143-6. ISSN 1550-6525. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/DS-RT.2014.32\" target=\"publication\" rel=\"noopener\">10.1109\/DS-RT.2014.32<\/a>. Best paper candidate. <a href=\"javascript:showAbstract('Investigating the performance of parallel applications at scale on future high-performance computing (HPC) architectures and the performance impact of different architecture choices is an important component of HPC hardware\/software co-design. The Extreme-scale Simulator (xSim) is a simulation-based toolkit for investigating the performance of parallel applications at scale. xSim scales to millions of simulated Message Passing Interface (MPI) processes. The overhead introduced by a simulation tool is an important performance and productivity aspect. This paper documents two improvements to xSim: (1) a new deadlock resolution protocol to reduce the parallel discrete event simulation management overhead and (2) a new simulated MPI message matching algorithm to reduce the oversubscription management overhead. The results clearly show a significant performance improvement, such as by reducing the simulation overhead for running the NAS Parallel Benchmark suite inside the simulator  from 1,020% to 238% for the conjugate gradient (CG) benchmark and from 102% to 0% for the embarrassingly parallel (EP) and benchmark, as well as, from 37,511% to 13,808% for CG and from 3,332% to 204% for EP with accurate process failure simulation.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann14improving.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann14improving.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann14improving\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Thomas Naughton, Christian Engelmann, Geoffroy Vall&eacute;e, and Swen B&ouml;hm. <b>Supporting the Development of Resilient Message Passing Applications using Simulation<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.pdp2014.org\" target=\"www.pdp2014.org\" rel=\"noopener\">22nd Euromicro International Conference on Parallel, Distributed, and network-based Processing (PDP) 2014<\/a><\/i>, pages 271-278, Turin, Italy, February 12-14, 2014. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\" rel=\"noopener\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISSN 1066-6192. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/PDP.2014.74\" target=\"publication\" rel=\"noopener\">10.1109\/PDP.2014.74<\/a>. Acceptance rate 32.6% (73\/224). <a href=\"javascript:showAbstract('An emerging aspect of high-performance computing (HPC) hardware\/software co-design is investigating performance under failure. The work in this paper extends the Extreme-scale Simulator (xSim), which was designed for evaluating the performance of message passing interface (MPI) applications on future HPC architectures, with fault-tolerant MPI extensions proposed by the MPI Fault Tolerance Working Group. xSim permits running MPI applications with millions of concurrent MPI ranks, while observing application performance in a simulated extreme-scale system using a lightweight parallel discrete event simulation. The newly added features offer user-level failure mitigation (ULFM) extensions at the simulated MPI layer to support algorithm-based fault tolerance (ABFT). The presented solution permits investigating performance under failure and failure handling of ABFT solutions. The newly enhanced xSim is the very first performance tool that supports ULFM and ABFT.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/naughton14supporting.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/naughton14supporting.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#naughton14supporting\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<p><em><small>Symbols: <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/txt.gif\" border=\"0\" alt=\"Abstract\" height=\"10pt\"> Abstract, <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/pdf.gif\" border=\"0\" alt=\"Publication\" height=\"10pt\"> Publication, <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/ppt.gif\" border=\"0\" alt=\"Presentation\" height=\"10pt\"> Presentation, <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/bib.gif\" border=\"0\" alt=\"BibTeX Citation\" height=\"10pt\"> BibTeX Citation<\/small><\/em><\/p>\n<p><script language=\"JavaScript\">\nfunction showAbstract (text) {\n  var width  = 400;\n  var height = 400;\n  var left   = (screen.width  - width ) \/ 2;\n  var top    = (screen.height - height) \/ 2;\n  var win    = window.open('',\n                           'Abstract',\n                           'width='  + width  + ', ' + \n                           'height=' + height + ', ' +\n                           'left='   + left   + ', ' +\n                           'top='    + top    + ', ' +\n                           'toolbar=no, '     +\n                           'location=no, '    +\n                           'directories=no, ' +\n                           'status=no, '      +\n                           'menubar=no, '     +\n                           'copyhistory=no, ' +\n                           'scrollbars=yes, ' +\n                           'resizable=yes')\n  win.document.write(text);\n  win.document.close();\n}\n<\/script><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The next generation of computational science applications require numerical solvers that are capable of high performance on proposed exascale platforms. In order to meet this goal, solvers must be resilient to soft and hard failures, provide high concurrency on heterogeneous hardware configurations, and retain numerical accuracy and efficiency. In light of these requirements, a natural&hellip;&nbsp;<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":145,"menu_order":22,"comment_status":"closed","ping_status":"closed","template":"","meta":{"neve_meta_sidebar":"","neve_meta_container":"","neve_meta_enable_content_width":"off","neve_meta_content_width":100,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"","footnotes":""},"class_list":["post-234","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/234","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=234"}],"version-history":[{"count":7,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/234\/revisions"}],"predecessor-version":[{"id":497,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/234\/revisions\/497"}],"up":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/145"}],"wp:attachment":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=234"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}