{"id":55,"date":"2026-09-04T08:00:27","date_gmt":"2026-09-04T08:00:27","guid":{"rendered":"https:\/\/christian-engelmann.de\/?page_id=55"},"modified":"2026-09-05T01:07:25","modified_gmt":"2026-09-05T01:07:25","slug":"bibtex-citations","status":"publish","type":"page","link":"https:\/\/www.christian-engelmann.info\/?page_id=55","title":{"rendered":"BibTex Citations"},"content":{"rendered":"
<\/a>@article{agullo22resiliency,\r\n  author        = \"Emmanuel Agullo\r\n                   and Mirco Altenbernd\r\n                   and Hartwig Anzt\r\n                   and Leonardo Bautista-Gomez\r\n                   and Tommaso Benacchio\r\n                   and Luca Bonaventura\r\n                   and Hans-Joachim Bungartz\r\n                   and Sanjay Chatterjee\r\n                   and Florina M. Ciorba\r\n                   and Nathan DeBardeleben\r\n                   and Daniel Drzisga\r\n                   and Sebastian Eibl\r\n                   and Christian Engelmann\r\n                   and Wilfried N. Gansterer\r\n                   and Luc Giraud\r\n                   and Dominik G{\\\"o}ddeke\r\n                   and Marco Heisig\r\n                   and Fabienne J{\\'e}z{\\'e}quel\r\n                   and Nils Kohl\r\n                   and Xiaoye Sherry Li\r\n                   and Romain Lion\r\n                   and Miriam Mehl\r\n                   and Paul Mycek\r\n                   and Michael Obersteiner\r\n                   and Enrique S. Quintana-Ort{\\'i}\r\n                   and Francesco Rizzi\r\n                   and Ulrich R{\\\"u}de\r\n                   and Martin Schulz\r\n                   and Fred Fung\r\n                   and Robert Speck\r\n                   and Linda Stals\r\n                   and Keita Teranishi\r\n                   and Samuel Thibault\r\n                   and Dominik Th{\\\"o}nnes\r\n                   and Andreas Wagner\r\n                   and Barbara Wohlmuth\",\r\n  title         = \"Resiliency in Numerical Algorithm Design for Extreme Scale Simulations\",\r\n  journal       = \"\\href{http:\/\/hpc.sagepub.com}{International Journal of High\r\n                   Performance Computing Applications (IJHPCA)}\",\r\n  volume        = \"36\",\r\n  number        = \"2\",\r\n  pages         = \"251--285\",\r\n  month         = mar,\r\n  year          = \"2022\",\r\n  publisher     = \"\\href{http:\/\/www.sagepub.com}{SAGE Publications}\",\r\n  issn          = \"1094-3420\",\r\n  doi           = \"10.1177\/10943420211055188\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/agullo22resiliency.pdf\",\r\n  abstract      = \"This work is based on the seminar titled 'Resiliency in\r\n                   Numerical Algorithm Design for Extreme Scale Simulations'\r\n                   held March 1-6, 2020 at Schloss Dagstuhl, that was attended\r\n                   by all the writers. Advanced supercomputing is characterized\r\n                   by very high computation speeds at the cost of involving an\r\n                   enormous amount of resources and costs. A typical large-scale\r\n                   computation running for 48 hours on a system consuming 20 MW,\r\n                   as predicted for exascale systems, would consume a million\r\n                   kWh, corresponding to about 100k Euro in energy cost for\r\n                   executing 1023 floating- point operations. It is clearly\r\n                   unacceptable to lose the whole computation if any of the\r\n                   several million parallel processes fails during the execution.\r\n                   Moreover, if a single operation suffers from a bit-flip error,\r\n                   should the whole computation be declared invalid? What about\r\n                   the notion of reproducibility itself: should this core\r\n                   paradigm of science be revised and refined for results that\r\n                   are obtained by large scale simulation? Naive versions of\r\n                   conventional resilience techniques will not scale to the\r\n                   exascale regime: with a main memory footprint of tens of\r\n                   Petabytes, synchronously writing checkpoint data all the way\r\n                   to background storage at frequent intervals will create\r\n                   intolerable overheads in runtime and energy consumption.\r\n                   Forecasts show that the mean time between failures could be\r\n                   lower than the time to recover from such a checkpoint, so\r\n                   that large calculations at scale might not make any progress\r\n                   if robust alternatives are not investigated.\r\n                   More advanced resilience techniques must be devised. The key\r\n                   may lie in exploiting both advanced system features as well\r\n                   as specific application knowledge. Research will face two\r\n                   essential questions: (1) what are the reliability\r\n                   requirements for a particular computation and (2) how do we\r\n                   best design the algorithms and software to meet these\r\n                   requirements? While the analysis of use cases can help\r\n                   understand the particular reliability requirements, the\r\n                   construction of remedies is currently wide open. One avenue\r\n                   would be to refine and improve on system- or application-level\r\n                   checkpointing and rollback strategies in the case an error\r\n                   is detected. Developers might use fault notification\r\n                   interfaces and flexible runtime systems to respond to node\r\n                   failures in an application-dependent fashion. Novel numerical\r\n                   algorithms or more stochastic computational approaches may\r\n                   be required to meet accuracy requirements in the face of\r\n                   undetectable soft errors. These ideas constituted an\r\n                   essential topic of the seminar.\r\n                   The goal of this Dagstuhl Seminar was to bring together a\r\n                   diverse group of scientists with expertise in exascale\r\n                   computing to discuss novel ways to make applications\r\n                   resilient against detected and undetected faults. In\r\n                   particular, participants explored the role that algorithms\r\n                   and applications play in the holistic approach needed to\r\n                   tackle this challenge. This article gathers a broad range\r\n                   of perspectives on the role of algorithms, applications,\r\n                   and systems in achieving resilience for extreme scale\r\n                   simulations. The ultimate goal is to spark novel ideas and\r\n                   encourage the development of concrete solutions for achieving\r\n                   such resilience holistically.\",\r\n  pts           = \"169116\"\r\n}<\/pre>\n
<\/a>@article{kumar21study,\r\n  author        = \"Mohit Kumar\r\n                   and Saurabh Gupta\r\n                   and Tirthak Patel\r\n                   and Michael Wilder\r\n                   and Weisong Shi\r\n                   and Song Fu\r\n                   and Christian Engelmann\r\n                   and Devesh Tiwari\",\r\n  title         = \"Study of Interconnect Errors, Network Congestion, and\r\n                   Applications Characteristics for Throttle Prediction on a\r\n                   Large Scale {HPC} System\",\r\n  journal       = \"\\href{http:\/\/www.elsevier.com\/locate\/jpdc}{Journal of\r\n                   Parallel and Distributed Computing (JPDC)}\",\r\n  volume        = \"153\",\r\n  pages         = \"29--43\",\r\n  month         = jul,\r\n  year          = \"2021\",\r\n  publisher     = \"\\href{http:\/\/www.elsevier.com}{Elsevier B.V, Amsterdam, The\r\n                   Netherlands}\",\r\n  issn          = \"0743-7315\",\r\n  doi           = \"10.1016\/j.jpdc.2021.03.001\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/kumar21study.pdf\",\r\n  abstract      = \"Today's High Performance Computing (HPC) systems contain\r\n                   thousand of nodes which work together to provide performance\r\n                   in the order of peta ops. The performance of these systems\r\n                   depends on various components like processors, memory, and\r\n                   interconnect. Among  all, interconnect plays a major role as\r\n                   it glues together all the hardware components in an HPC\r\n                   system. A slow interconnect can impact a scientific\r\n                   application running on multiple processes severely as they\r\n                   rely on fast network messages to communicate and synchronize\r\n                   frequently. Unfortunately, the HPC community lacks a study\r\n                   that explores different interconnect errors, congestion\r\n                   events and applications characteristics on a large-scale HPC\r\n                   system. In our previous work, we process and analyze\r\n                   interconnect data of the Titan supercomputer to develop a\r\n                   thorough understanding of interconnects faults, errors,\r\n                   and congestion events. In this work, we first show how\r\n                   congestion events can impact application performance. We\r\n                   then investigate application characteristics interaction\r\n                   with interconnect errors and network congestion to predict\r\n                   applications encountering congestion with more than 90\\%\r\n                   accuracy\",\r\n  pts           = \"153615\"\r\n}<\/pre>\n
<\/a>@article{katti18epidemic,\r\n  author        = \"Amogh Katti\r\n                   and Giuseppe Di Fatta\r\n                   and Thomas Naughton\r\n                   and Christian Engelmann\",\r\n  title         = \"Epidemic Failure Detection and Consensus for Extreme\r\n                   Parallelism\",\r\n  journal       = \"\\href{http:\/\/hpc.sagepub.com}{International Journal of High\r\n                   Performance Computing Applications (IJHPCA)}\",\r\n  volume        = \"32\",\r\n  number        = \"5\",\r\n  pages         = \"729--743\",\r\n  month         = sep,\r\n  year          = \"2018\",\r\n  publisher     = \"\\href{http:\/\/www.sagepub.com}{SAGE Publications}\",\r\n  issn          = \"1094-3420\",\r\n  doi           = \"10.1177\/1094342017690910\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/katti17epidemic.pdf\",\r\n  abstract      = \"Future extreme-scale high-performance computing systems will\r\n                   be required to work under frequent component failures. The\r\n                   MPI Forum's User Level Failure Mitigation proposal has\r\n                   introduced an operation, MPI Comm shrink, to synchronize the\r\n                   alive processes on the list of failed processes, so that\r\n                   applications can continue to execute even in the presence of\r\n                   failures by adopting algorithm-based fault tolerance\r\n                   techniques. This MPI Comm shrink operation requires a failure\r\n                   detection and consensus algorithm. This paper presents three\r\n                   novel failure detection and consensus algorithms using\r\n                   Gossiping. The proposed algorithms were implemented and\r\n                   tested using the Extreme-scale Simulator. The results show\r\n                   that in all algorithms the number of Gossip cycles to achieve\r\n                   global consensus scales logarithmically with system size. The\r\n                   second algorithm also shows better scalability in terms of\r\n                   memory and network bandwidth usage and a perfect\r\n                   synchronization in achieving global consensus. The third\r\n                   approach is a three-phase distributed failure detection and\r\n                   consensus algorithm and provides consistency guarantees even\r\n                   in very large and extreme-scale systems while at the same\r\n                   time being memory and bandwidth efficient.\",\r\n  pts           = \"72175\"\r\n}<\/pre>\n
<\/a>@article{hukerikar17resilience,\r\n  author        = \"Saurabh Hukerikar\r\n                   and Christian Engelmann\",\r\n  title         = \"Resilience Design Patterns: {A} Structured Approach to\r\n                   Resilience at Extreme Scale\",\r\n  journal       = \"\\href{http:\/\/superfri.org\/superfri}{Journal of\r\n                   Supercomputing Frontiers and Innovations (JSFI)}\",\r\n  volume        = \"4\",\r\n  number        = \"3\",\r\n  pages         = \"4--42\",\r\n  month         = oct,\r\n  year          = \"2017\",\r\n  publisher     = \"\\href{http:\/\/www.susu.ru\/en}{South Ural State University\r\n                   Chelyabinsk, Russia}\",\r\n  issn          = \"2409-6008\",\r\n  doi           = \"10.14529\/jsfi170301\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/hukerikar17resilience.pdf\",\r\n  abstract      = \"Reliability is a serious concern for future extreme-scale\r\n                   high-performance computing (HPC) systems. Projections based\r\n                   on the current generation of HPC systems and technology\r\n                   roadmaps suggest the prevalence of very high fault rates in\r\n                   future systems. The errors resulting from these faults will\r\n                   propagate and generate various kinds of failures, which may\r\n                   result in outcomes ranging from result corruptions to\r\n                   catastrophic application crashes. Therefore, the resilience\r\n                   challenge for extreme-scale HPC systems requires management\r\n                   of various hardware and software technologies that are\r\n                   capable of handling a broad set of fault models at\r\n                   accelerated fault rates. Also, due to practical limits on\r\n                   power consumption in HPC systems future systems are likely\r\n                   to embrace innovative architectures, increasing the levels\r\n                   of hardware and software complexities. As a result, the\r\n                   techniques that seek to improve resilience must navigate\r\n                   the complex trade-off space between resilience and the\r\n                   overheads to power consumption and performance. While the\r\n                   HPC community has developed various resilience solutions,\r\n                   application-level techniques as well as system-based\r\n                   solutions, the solution space of HPC resilience\r\n                   techniques remains fragmented. There are no formal methods\r\n                   and metrics to investigate and evaluate resilience\r\n                   holistically in HPC systems that consider impact scope,\r\n                   handling coverage, and performance & power efficiency\r\n                   across the system stack. Additionally, few of the current\r\n                   approaches are portable to newer architectures and software\r\n                   environments that will be deployed on future systems.\r\n                   In this paper, we develop a structured approach to the\r\n                   management of HPC resilience using the concept of\r\n                   resilience-based design patterns. A design pattern is a\r\n                   general repeatable solution to a commonly occurring problem.\r\n                   We identify the commonly occurring problems and solutions\r\n                   used to deal with faults, errors and failures in HPC systems.\r\n                   Each established solution is described in the form of a\r\n                   pattern that addresses concrete problems in the design of\r\n                   resilient systems. The complete catalog of resilience design\r\n                   patterns provides designers with reusable design elements.\r\n                   We also define a framework that enhances a designer's\r\n                   understanding of the important constraints and opportunities\r\n                   for the design patterns to be implemented and deployed at\r\n                   various layers of the system stack. This design framework\r\n                   may be used to establish mechanisms and interfaces to\r\n                   coordinate flexible fault management across hardware and\r\n                   software components. The framework also supports\r\n                   optimization of the cost-benefit trade-offs among\r\n                   performance, resilience, and power consumption. The overall\r\n                   goal of this work is to enable a systematic methodology for\r\n                   the design and evaluation of resilience technologies in\r\n                   extreme-scale HPC systems that keep scientific applications\r\n                   running to a correct solution in a timely and cost-efficient\r\n                   manner despite frequent faults, errors, and failures of\r\n                   various types.\",\r\n  pts           = \"102201\"\r\n}<\/pre>\n
<\/a>@article{engelmann16new,\r\n  author        = \"Christian Engelmann\r\n                   and Thomas Naughton\",\r\n  title         = \"A New Deadlock Resolution Protocol and Message Matching\r\n                   Algorithm for the Extreme-scale Simulator\",\r\n  journal       = \"\\href{http:\/\/onlinelibrary.wiley.com\/journal\/10.1002\/(ISSN)1532-0634}\r\n                   {Concurrency and Computation: Practice and Experience}\",\r\n  volume        = \"28\",\r\n  number        = \"12\",\r\n  pages         = \"3369--3389\",\r\n  month         = aug,\r\n  year          = \"2016\",\r\n  publisher     = \"\\href{http:\/\/www.wiley.com}{John Wiley & Sons, Inc.}\",\r\n  issn          = \"1532-0634\",\r\n  doi           = \"10.1002\/cpe.3805\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann16new.pdf\",\r\n  abstract      = \"Investigating the performance of parallel applications at\r\n                   scale on future high-performance computing~(HPC) architectures\r\n                   and the performance impact of different HPC architecture\r\n                   choices is an important component of HPC hardware\/software\r\n                   co-design. The Extreme-scale Simulator (xSim) is a simulation\r\n                   toolkit for investigating the performance of parallel\r\n                   applications at scale. xSim scales to millions of simulated\r\n                   Message Passing Interface (MPI) processes. The xSim toolkit\r\n                   strives to limit simulation overheads in order to maintain\r\n                   performance and productivity criteria. This paper documents\r\n                   two improvements to xSim: (1)~a new deadlock resolution\r\n                   protocol to reduce the parallel discrete event simulation\r\n                   overhead, and (2)~a new simulated MPI message matching\r\n                   algorithm to reduce the oversubscription management cost.\r\n                   These enhancements resulted in significant performance\r\n                   improvements. The simulation overhead for running the NAS\r\n                   Parallel Benchmark suite dropped from 1,020\\% to 238\\% for\r\n                   the conjugate gradient (CG) benchmark and 102\\% to 0\\% for\r\n                   the embarrassingly parallel~(EP) benchmark. Additionally, the\r\n                   improvements were beneficial for reducing overheads in the\r\n                   highly accurate simulation mode of xSim, which is useful for\r\n                   resilience investigation studies for tracking intentional MPI\r\n                   process failures. In the highly accurate mode, the simulation\r\n                   overhead was reduced from 37,511\\% to 13,808\\% for CG and\r\n                   from 3,332\\% to 204\\% for EP.\",\r\n  pts           = \"58541\"\r\n}<\/pre>\n
<\/a>@article{snir14addressing,\r\n  author        = \"Marc Snir\r\n                   and Robert W. Wisniewski\r\n                   and Jacob A. Abraham\r\n                   and Sarita V. Adve\r\n                   and Saurabh Bagchi\r\n                   and Pavan Balaji\r\n                   and Jim Belak\r\n                   and Pradip Bose\r\n                   and Franck Cappello\r\n                   and Bill Carlson\r\n                   and Andrew A. Chien\r\n                   and Paul Coteus\r\n                   and Nathan A. Debardeleben\r\n                   and Pedro Diniz\r\n                   and Christian Engelmann\r\n                   and Mattan Erez\r\n                   and Saverio Fazzari\r\n                   and Al Geist\r\n                   and Rinku Gupta\r\n                   and Fred Johnson\r\n                   and Sriram Krishnamoorthy\r\n                   and Sven Leyffer\r\n                   and Dean Liberty\r\n                   and Subhasish Mitra\r\n                   and Todd Munson\r\n                   and Rob Schreiber\r\n                   and Jon Stearley\r\n                   and Eric Van Hensbergen\",\r\n  title         = \"Addressing Failures in Exascale Computing\",\r\n  journal       = \"\\href{http:\/\/hpc.sagepub.com}{International Journal of High\r\n                   Performance Computing Applications (IJHPCA)}\",\r\n  volume        = \"28\",\r\n  number        = \"2\",\r\n  pages         = \"127--171\",\r\n  month         = may,\r\n  year          = \"2014\",\r\n  publisher     = \"\\href{http:\/\/www.sagepub.com}{SAGE Publications}\",\r\n  issn          = \"1094-3420\",\r\n  doi           = \"10.1177\/1094342014522573\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/snir14addressing.pdf\",\r\n  abstract      = \"We present here a report produced by a workshop on \r\n                   Addressing failures in exascale computing' held in Park City, \r\n                   Utah, 4-11 August 2012. The charter of this workshop was to \r\n                   establish a common taxonomy about resilience across all the \r\n                   levels in a computing system, discuss existing knowledge on \r\n                   resilience across the various hardware and software layers \r\n                   of an exascale system, and build on those results, examining \r\n                   potential solutions from both a hardware and software \r\n                   perspective and focusing on a combined approach.\r\n                   The workshop brought together participants with expertise in \r\n                   applications, system software, and hardware; they came from \r\n                   industry, government, and academia, and their interests ranged \r\n                   from theory to implementation. The combination allowed broad \r\n                   and comprehensive discussions and led to this document, which \r\n                   summarizes and builds on those discussions.\",\r\n  pts           = \"49208\"\r\n}<\/pre>\n
<\/a>@article{engelmann13scaling,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Scaling To A Million Cores And Beyond: {Using} Light-Weight\r\n                   Simulation to Understand The Challenges Ahead On The Road To\r\n                   Exascale\",\r\n  journal       = \"\\href{http:\/\/www.elsevier.com\/locate\/fgcs}{Future Generation\r\n                   Computer Systems (FGCS)}\",\r\n  volume        = \"30\",\r\n  number        = \"0\",\r\n  pages         = \"59--65\",\r\n  month         = jan,\r\n  year          = \"2014\",\r\n  publisher     = \"\\href{http:\/\/www.elsevier.com}{Elsevier B.V, Amsterdam, The\r\n                   Netherlands}\",\r\n  issn          = \"0167-739X\",\r\n  doi           = \"10.1016\/j.future.2013.04.014\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann13scaling.pdf\",\r\n  abstract      = \"As supercomputers scale to 1,000 PFlop\/s over the next\r\n                   decade, investigating the performance of parallel\r\n                   applications at scale on future architectures and the\r\n                   performance impact of different architecture choices for\r\n                   high-performance computing (HPC) hardware\/software co-design\r\n                   is crucial. This paper summarizes recent efforts in designing\r\n                   and implementing a novel HPC hardware\/software co-design\r\n                   toolkit. The presented Extreme-scale Simulator (xSim) permits\r\n                   running an HPC application in a controlled environment with\r\n                   millions of concurrent execution threads while observing its\r\n                   performance in a simulated extreme-scale HPC system using\r\n                   architectural models and virtual timing. This paper\r\n                   demonstrates the capabilities and usefulness of the xSim\r\n                   performance investigation toolkit, such as its scalability\r\n                   to $2^{27}$ simulated Message Passing Interface (MPI) ranks\r\n                   on 960 real processor cores, the capability to evaluate the\r\n                   performance of different MPI collective communication\r\n                   algorithms, and the ability to evaluate the performance of\r\n                   a basic Monte Carlo application with different architectural\r\n                   parameters.\",\r\n  pts           = \"42452\"\r\n}<\/pre>\n
<\/a>@article{wang12proactive,\r\n  author        = \"Chao Wang\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"Proactive Process-Level Live Migration and Back Migration in\r\n                   {HPC} Environments\",\r\n  journal       = \"\\href{http:\/\/www.elsevier.com\/locate\/jpdc}{Journal of\r\n                   Parallel and Distributed Computing (JPDC)}\",\r\n  volume        = \"72\",\r\n  number        = \"2\",\r\n  pages         = \"254--267\",\r\n  month         = feb,\r\n  year          = \"2012\",\r\n  publisher     = \"\\href{http:\/\/www.elsevier.com}{Elsevier B.V, Amsterdam, The\r\n                   Netherlands}\",\r\n  issn          = \"0743-7315\",\r\n  doi           = \"10.1016\/j.jpdc.2011.10.009\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/wang12proactive.pdf\",\r\n  abstract      = \"As the number of nodes in high-performance computing\r\n                   environments keeps increasing, faults are becoming common\r\n                   place. Reactive fault tolerance (FT) often does not scale\r\n                   due to massive I\/O requirements and relies on manual job\r\n                   resubmission.\r\n                   This work complements reactive with proactive FT at the\r\n                   process level. Through health monitoring, a subset of node\r\n                   failures can be anticipated when one's health deteriorates.\r\n                   A novel process-level live migration mechanism supports\r\n                   continued execution of applications during much of process\r\n                   migration. This scheme is integrated into an MPI execution\r\n                   environment to transparently sustain health-inflicted node\r\n                   failures, which eradicates the need to restart and requeue\r\n                   MPI jobs. Experiments indicate that 1-6.5 s of prior warning\r\n                   are required to successfully trigger live process migration\r\n                   while similar operating system virtualization mechanisms\r\n                   require 13-24 s. This self-healing approach complements\r\n                   reactive FT by nearly cutting the number of checkpoints in\r\n                   half when 70\\% of the faults are handled proactively. The\r\n                   work also provides a novel back migration approach to\r\n                   eliminate load imbalance or bottlenecks caused by migrated\r\n                   tasks. Experiments indicate the larger the amount of\r\n                   outstanding execution, the higher the benefit due to back\r\n                   migration.\",\r\n  pts           = \"35627\"\r\n}<\/pre>\n
<\/a>@article{scott10system,\r\n  author        = \"Stephen L. Scott\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Anand Tikotekar\r\n                   and Christian Engelmann\r\n                   and Hong H. Ong\",\r\n  title         = \"System-Level Virtualization Research at {Oak Ridge National\r\n                   Laboratory}\",\r\n  journal       = \"\\href{http:\/\/www.elsevier.com\/locate\/fgcs}{Future Generation\r\n                   Computer Systems (FGCS)}\",\r\n  volume        = \"26\",\r\n  number        = \"3\",\r\n  pages         = \"304--307\",\r\n  month         = mar,\r\n  year          = \"2010\",\r\n  publisher     = \"\\href{http:\/\/www.elsevier.com}{Elsevier B.V, Amsterdam, The\r\n                   Netherlands}\",\r\n  issn          = \"0167-739X\",\r\n  doi           = \"10.1016\/j.future.2009.07.001\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/scott09system.pdf\",\r\n  abstract      = \"System-level virtualization is today enjoying a rebirth as a\r\n                   technique to effectively share what were then considered\r\n                   large computing resources to subsequently fade from the\r\n                   spotlight as individual workstations gained in popularity\r\n                   with a one machine -- one user approach. One reason for\r\n                   this resurgence is that the simple workstation has grown in\r\n                   capability to rival that of anything available in the past.\r\n                   Thus, computing centers are again looking at the\r\n                   price\/performance benefit of sharing that single computing\r\n                   box via server consolidation. However, industry is only\r\n                   concentrating on the benefits of using virtualization for\r\n                   server consolidation (enterprise computing) whereas our\r\n                   interest is in leveraging virtualization to advance\r\n                   high-performance computing (HPC). While these two interests\r\n                   may appear to be orthogonal, one consolidating multiple\r\n                   applications and users on a single machine while the other\r\n                   requires all the power from many machines to be dedicated\r\n                   solely to its purpose, we propose that virtualization does\r\n                   provide attractive capabilities that may be exploited to the\r\n                   benefit of HPC interests. This does raise the two fundamental\r\n                   questions of: is the concept of virtualization (a machine\r\n                   sharing technology) really suitable for HPC and if so,\r\n                   how does one go about leveraging these virtualization\r\n                   capabilities for the benefit of HPC. To address these\r\n                   questions, this document presents ongoing studies on the\r\n                   usage of system-level virtualization in a HPC context. These\r\n                   studies include an analysis of the benefits of system-level\r\n                   virtualization for HPC, a presentation of research efforts\r\n                   based on virtualization for system availability, and a\r\n                   presentation of research efforts for the management of\r\n                   virtual systems. The basis for this document was material\r\n                   presented by Stephen L. Scott at the Collaborative and Grid\r\n                   Computing Technologies meeting held in Cancun, Mexico on\r\n                   April 12-14, 2007.\",\r\n  pts           = \"35628\"\r\n}<\/pre>\n
<\/a>@article{he09symmetric,\r\n  author        = \"Xubin (Ben) He\r\n                   and Li Ou\r\n                   and Christian Engelmann\r\n                   and Xin Chen\r\n                   and Stephen L. Scott\",\r\n  title         = \"Symmetric Active\/Active Metadata Service for High\r\n                   Availability Parallel File Systems\",\r\n  journal       = \"\\href{http:\/\/www.elsevier.com\/locate\/jpdc}{Journal of\r\n                   Parallel and Distributed Computing (JPDC)}\",\r\n  volume        = \"69\",\r\n  number        = \"12\",\r\n  pages         = \"961-973\",\r\n  month         = dec,\r\n  year          = \"2009\",\r\n  publisher     = \"\\href{http:\/\/www.elsevier.com}{Elsevier B.V, Amsterdam, The\r\n                   Netherlands}\",\r\n  issn          = \"0743-7315\",\r\n  doi           = \"10.1016\/j.jpdc.2009.08.004\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/he09symmetric.pdf\",\r\n  abstract      = \"High availability data storage systems are critical for many\r\n                   applications as research and business become more\r\n                   data-driven. Since metadata management is essential to\r\n                   system availability, multiple metadata services are used to\r\n                   improve the availability of distributed storage systems.\r\n                   Past research focused on the active\/standby model, where\r\n                   each active service has at least one redundant idle backup.\r\n                   However, interruption of service and even some loss of\r\n                   service state may occur during a fail-over depending on the\r\n                   used replication technique. In addition, the replication\r\n                   overhead for multiple metadata services can be very high.\r\n                   The research in this paper targets the symmetric\r\n                   active\/active replication model, which uses multiple\r\n                   redundant service nodes running in virtual synchrony. In\r\n                   this model, service node failures do not cause a fail-over\r\n                   to a backup and there is no disruption of service or loss\r\n                   of service state. We further discuss a fast delivery\r\n                   protocol to reduce the latency of the needed total order\r\n                   broadcast. Our prototype implementation shows that\r\n                   metadata service high availability can be achieved with\r\n                   an acceptable performance trade-off using our symmetric\r\n                   active\/active metadata service solution.\",\r\n  pts           = \"21240\"\r\n}<\/pre>\n
<\/a>@article{he07unified,\r\n  author        = \"Xubin (Ben) He\r\n                   and Li Ou\r\n                   and Martha J. Kosa\r\n                   and Stephen L. Scott\r\n                   and Christian Engelmann\",\r\n  title         = \"A Unified Multiple-Level Cache for High Performance Cluster\r\n                   Storage Systems\",\r\n  journal       = \"\\href{http:\/\/www.inderscience.com\/browse\/index.php?journalcode=ijhpcn}\r\n                   {International Journal of High Performance Computing and\r\n                   Networking (IJHPCN)}\",\r\n  volume        = \"5\",\r\n  number        = \"1-2\",\r\n  pages         = \"97--109\",\r\n  month         = nov # \"~14, \",\r\n  year          = \"2007\",\r\n  publisher     = \"\\href{http:\/\/www.inderscience.com}{Inderscience Publishers,\r\n                   Geneve, Switzerland}\",\r\n  issn          = \"1740-0562\",\r\n  doi           = \"10.1504\/IJHPCN.2007.015768\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/he07unified.pdf\",\r\n  abstract      = \"Highly available data storage for high-performance computing\r\n                   is becoming increasingly more critical as high-end computing\r\n                   systems scale up in size and storage systems are developed\r\n                   around network-centered architectures. A promising solution\r\n                   is to harness the collective storage potential of individual\r\n                   workstations much as we harness idle CPU cycles due to the\r\n                   excellent price\/performance ratio and low storage usage of\r\n                   most commodity workstations. For such a storage system,\r\n                   metadata consistency is a key issue assuring storage system\r\n                   availability as well as data reliability. In this paper, we\r\n                   present a decentralized metadata management scheme that\r\n                   improves storage availability without sacrificing\r\n                   performance.\",\r\n  pts           = \"1907\"\r\n}<\/pre>\n
<\/a>@article{engelmann06symmetric,\r\n  author        = \"Christian Engelmann\r\n                   and Stephen L. Scott\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and Xubin (Ben) He\",\r\n  title         = \"Symmetric Active\/Active High Availability for\r\n                   High-Performance Computing System Services\",\r\n  journal       = \"\\href{http:\/\/www.jcomputers.us}{Journal of Computers (JCP)}\",\r\n  volume        = \"1\",\r\n  number        = \"8\",\r\n  pages         = \"43--54\",\r\n  month         = dec,\r\n  year          = \"2006\",\r\n  publisher     = \"\\href{http:\/\/www.jcomputers.us}{Academy Publisher,\r\n                   Oulu, Finland}\",\r\n  issn          = \"1796-203X\",\r\n  doi           = \"10.4304\/jcp.1.8.43-54\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann06symmetric.pdf\",\r\n  abstract      = \"This work aims to pave the way for high availability in\r\n                   high-performance computing (HPC) by focusing on efficient\r\n                   redundancy strategies for head and service nodes. These nodes\r\n                   represent single points of failure and control for an entire\r\n                   HPC system as they render it inaccessible and unmanageable in\r\n                   case of a failure until repair. The presented approach\r\n                   introduces two distinct replication methods, internal and\r\n                   external, for providing symmetric active\/active high\r\n                   availability for multiple redundant head and service nodes\r\n                   running in virtual synchrony utilizing an existing process\r\n                   group communication system for service group membership\r\n                   management and reliable, totally ordered message delivery.\r\n                   Resented results of a prototype implementation that offers\r\n                   symmetric active\/active replication for HPC job and resource\r\n                   management using external replication show that the highest\r\n                   level of availability can be provided with an acceptable\r\n                   performance trade-off.\",\r\n  pts           = \"4583\"\r\n}<\/pre>\n
<\/a>@article{engelmann06molar,\r\n  author        = \"Christian Engelmann\r\n                   and Stephen L. Scott\r\n                   and David E. Bernholdt\r\n                   and Narasimha R. Gottumukkala\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and Jyothish Varma\r\n                   and Chao Wang\r\n                   and Frank Mueller\r\n                   and Aniruddha G. Shet\r\n                   and Ponnuswamy (Saday) Sadayappan\",\r\n  title         = \"{MOLAR}: {A}daptive Runtime Support for High-End Computing\r\n                   Operating and Runtime Systems\",\r\n  journal       = \"\\href{http:\/\/www.sigops.org\/osr.html}{ACM SIGOPS Operating\r\n                   Systems Review (OSR)}\",\r\n  volume        = \"40\",\r\n  number        = \"2\",\r\n  pages         = \"63--72\",\r\n  month         = apr,\r\n  year          = \"2006\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  issn          = \"0163-5980\",\r\n  doi           = \"10.1145\/1131322.1131337\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann06molar.pdf\",\r\n  abstract      = \"MOLAR is a multi-institutional research effort that\r\n                   concentrates on adaptive, reliable, and efficient operating\r\n                   and runtime system (OS\/R) solutions for ultra-scale,\r\n                   high-end scientific computing on the next generation of\r\n                   supercomputers. This research addresses the challenges\r\n                   outlined in FAST-OS (forum to address scalable technology for\r\n                   runtime and operating systems) and HECRTF (high-end computing\r\n                   revitalization task force) activities by exploring the use of\r\n                   advanced monitoring and adaptation to improve application\r\n                   performance and predictability of system interruptions, and\r\n                   by advancing computer reliability, availability and\r\n                   serviceability (RAS) management systems to work cooperatively\r\n                   with the OS\/R to identify and preemptively resolve system\r\n                   issues. This paper describes recent research of the MOLAR\r\n                   team in advancing RAS for high-end computing OS\/Rs.\",\r\n  pts           = \"1905\"\r\n}<\/pre>\n
<\/a>@conference{valero-lara26chatmpi,\r\n  author        = \"Pedro Valero-Lara\r\n                   and Aaron Young\r\n                   and Thomas Naughton\r\n                   and Christian Engelmann\r\n                   and Al Geist\r\n                   and Jeffrey S. Vetter\r\n                   and Keita Teranishi\r\n                   and William F. Godoy\",\r\n  title         = \"{ChatMPI}: {LLM}-Driven {MPI} Code Generation for {HPC}\r\n                   Workloads\",\r\n  booktitle     = \"Proceedings of the \\href{https:\/\/www.sca-hpcasia2026.jp}\r\n                   {Supercomputing Asia \/ International Conference on High\r\n                   Performance Computing in the Asia-Pacific Region\r\n                   (SCA\/HPCAsia) 2026}\",\r\n  pages         = \"19--30\",\r\n  month         = jan # \"~26-29, \",\r\n  year          = \"2026\",\r\n  address       = \"Osaka, Japan\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"979-8-4007-2067-3\",\r\n  doi           = \"10.1145\/3773656.3773659\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/valero-lara26chatmpi.pdf\",\r\n  url2          = \"\",\r\n  abstract      = \"The Message Passing Interface (MPI) standard plays a crucial\r\n                   role in enabling scientific applications for parallel\r\n                   computing and is an essential component in high-performance\r\n                   computing (HPC). However, implementing MPI code\r\n                   manually---especially applying a proper domain decomposition\r\n                   and communication pattern---is a challenging and error-prone\r\n                   task. We present ChatMPI, an AI assistant for MPI\r\n                   parallelization of sequential C codes. In our analysis, we\r\n                   focus on testing six essential HPC workloads, which are\r\n                   based on Basic Linear Algebra Subprograms levels 1, 2, and\r\n                   3 as well as sparse, stencil, and iterative operations. We\r\n                   analyze the process of creating ChatMPI by using the ChatHPC\r\n                   library. This lightweight large language model (LLM)--based\r\n                   infrastructure enables HPC experts to efficiently create and\r\n                   supervise trustworthy AI capabilities for critical HPC\r\n                   software tasks. We study the data required for training\r\n                   (fine-tuning) ChatMPI to generate parallel codes that not\r\n                   only use MPI syntax correctly but also apply HPC techniques\r\n                   to reduce memory communication and maximize performance by\r\n                   using proper work decomposition. With a relatively small\r\n                   training dataset composed of a few dozen prompts and fewer\r\n                   than 15 minutes of fine-tuning on one node equipped with two\r\n                   NVIDIA H100 GPUs, ChatMPI elevates trustworthiness for MPI\r\n                   code generation of current LLMs (e.g., Code Llama, ChatGPT-4o\r\n                   and ChatGPT 5). Additionally, we evaluate the performance of\r\n                   the MPI codes generated by ChatMPI in comparison with the\r\n                   ones generated by ChatGPT-4o and ChatGPT-5. The codes\r\n                   generated by ChatMPI provide up to a 4x boost in performance\r\n                   by using better problem decomposition, communication\r\n                   patterns, and HPC techniques (e.g., communication\r\n                   avoiding).\",\r\n  pts           = \"251590\"\r\n}<\/pre>\n
<\/a>@conference{kim24privacy,\r\n  author        = \"Kibaek Kim\r\n                   and Krishnan Raghavan\r\n                   and Olivera Kotevska\r\n                   and Matthieu Dorier\r\n                   and Ravi Madduri\r\n                   and Minseok Ryu\r\n                   and Todd Munson\r\n                   and Rob Ross\r\n                   and Thomas Flynn\r\n                   and Ai Kagawa\r\n                   and Byung-Jun Yoon\r\n                   and Christian Engelmann\r\n                   and Farzad Yousefian\",\r\n  title         = \"Privacy-Preserving Federated Learning for Science: Challenges\r\n                   and Research Directions\",\r\n  booktitle     = \"Proceedings of the \\href{https:\/\/ieeebigdata2024.github.io}\r\n                   {$12^{th}$ IEEE International Conference on Big Data\r\n                    (BigData) 2024}\",\r\n  pages         = \"7849-7853\",\r\n  month         = dec # \"~15-18, \",\r\n  year          = \"2024\",\r\n  address       = \"Washington, DC, USA\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  issn          = \"2639-1589\",\r\n  isbn          = \"979-8-3503-6249-7\",\r\n  doi           = \"10.1109\/BigData62323.2024.10825853\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/kim24privacy.pdf\",\r\n  abstract      = \"This paper discusses the key challenges and future research\r\n                   directions for privacy-preserving federated learning (PPFL),\r\n                   with a focus on its application to large-scale scientific\r\n                   artificial intelligence models, in particular, foundation\r\n                   models (FMs). PPFL enables collaborative model training\r\n                   across distributed datasets while preserving privacy an\r\n                   important collaborative approach for science. We discuss the\r\n                   need for efficient and scalable algorithms to address the\r\n                   increasing complexity of FMs, particularly when dealing with\r\n                   heterogeneous clients. In addition, we underscore the need\r\n                   for developing advance privacy-preserving techniques, such as\r\n                   differential privacy, to balance privacy and utility in\r\n                   large FMs emphasizing fairness and incentive mechanisms to\r\n                   ensure equitable participation among heterogeneous clients.\r\n                   Finally, we emphasize the need for a robust software stack\r\n                   supporting scalable and secure PPFL deployments across\r\n                   multiple high-performance computing facilities. We envision\r\n                   that PPFL would play a crucial role to advance scientific\r\n                   discovery and enable large-scale, privacy-aware\r\n                   collaborations across science domains.\",\r\n  pts           = \"225891\"\r\n}<\/pre>\n
<\/a>@conference{oles24understanding,\r\n  author        = \"Vladyslav Oles\r\n                   and Anna Schmedding\r\n                   and George Ostrouchov\r\n                   and Woong Shi\r\n                   and Evgenia Smirni\r\n                   and Christian Engelmann\",\r\n  title         = \"Understanding {GPU} Memory Corruption at Extreme Scale: The\r\n                   Summit Case Study\",\r\n  booktitle     = \"Proceedings of the \\href{https:\/\/ics2024.github.io}\r\n                   {$38^{th}$ ACM International Conference on Supercomputing\r\n                    (ICS) 2024}\",\r\n  pages         = \"188-200\",\r\n  month         = jun # \"~4-7, \",\r\n  year          = \"2024\",\r\n  address       = \"Kyoto, Japan\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"979-8-4007-0610-3\",\r\n  doi           = \"10.1145\/3650200.3656615\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/oles24understanding.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/oles24understanding.ppt.pdf\",\r\n  abstract      = \"GPU memory corruption and in particular double- bit errors\r\n                  (DBEs) remain one of the least understood aspects of HPC\r\n                  system reliability. Albeit rare, their occurrences always\r\n                  lead to job termination and can potentially cost thousands of\r\n                  node-hours, either from wasted computations or as the\r\n                  overhead from regular checkpointing needed to minimize the\r\n                  losses. As supercomputers and their components simultaneously\r\n                  grow in scale, density, failure rates, and environmental\r\n                  footprint, the efficiency of HPC operations becomes both an\r\n                  imperative and a challenge.\r\n                  We examine DBEs using system telemetry data and logs collected\r\n                  from the Summit supercomputer, equipped with 27,648 Tesla V100\r\n                  GPUs with 2nd-generation high-bandwidth memory (HBM2). Using\r\n                  exploratory data analysis and statistical learning, we extract\r\n                  several insights about memory reliability in such GPUs. We\r\n                  find that GPUs with prior DBE occurrences are prone to\r\n                  experience them again due to otherwise harmless factors,\r\n                  correlate this phenomenon with GPU placement, and suggest\r\n                  manufacturing variability as a factor. On the general\r\n                  population of GPUs, we link DBEs to short- and long-term high\r\n                  power consumption modes while finding no significant\r\n                  correlation with higher temperatures. We also show that\r\n                  workload type can be a factor in GPU memory\u2019s propensity to\r\n                  corruption.\",\r\n  pts           = \"212442\"\r\n}<\/pre>\n
<\/a>@conference{engelmann23science,\r\n  author        = \"Christian Engelmann\r\n                   and Suhas Somnath\",\r\n  title         = \"Science Use Case Design Patterns for Autonomous Experiments\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/europlop.net}\r\n                   {$28^{th}$ European Conference on Pattern Languages of\r\n                   Programs (EuroPLoP) 2023}\",\r\n  pages         = \"1-14\",\r\n  month         = jul # \"~5-9, \",\r\n  year          = \"2023\",\r\n  address       = \"Kloster Irsee, Germany\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"979-8-4007-0040-8\",\r\n  doi           = \"10.1145\/3628034.3628060\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann23science.pdf\",\r\n  abstract      = \"Connecting scientific instruments and robot-controlled\r\n                   laboratories with computing and data resources at the edge,\r\n                   the Cloud or the high-performance computing (HPC) center\r\n                   enables autonomous experiments, self-driving laboratories,\r\n                   smart manufacturing, and artificial intelligence (AI)-driven\r\n                   design, discovery and evaluation. The Self-driven Experiments\r\n                   for Science \/ Interconnected Science Ecosystem (INTERSECT)\r\n                   Open Architecture enables science break- throughs using\r\n                   intelligent networked systems, instruments and facilities\r\n                   with a federated hardware\/software architecture for the\r\n                   laboratory of the future. It relies on a novel approach,\r\n                   consisting of (1) science use case design patterns, (2) a\r\n                   system of systems architecture, and (3) a microservice\r\n                   architecture. This paper introduces the science use case\r\n                   design patterns of the INTERSECT Architecture. It describes\r\n                   the overall background, the involved terminology and concepts,\r\n                   and the pattern format and classification. It further offers\r\n                   an overview of the 12 defined patterns and 4 examples of\r\n                   patterns of 2 different pattern classes. It also provides\r\n                   insight into building solutions from these patterns. The\r\n                   target audience are computer, computational, instrument and\r\n                   domain science experts working in the field of autonomous\r\n                   experiments.\",\r\n  pts           = \"200749\"\r\n}<\/pre>\n
<\/a>@conference{engelmann22intersect,\r\n  author        = \"Christian Engelmann\r\n                   and Olga Kuchar\r\n                   and Swen Boehm\r\n                   and Michael J. Brim\r\n                   and Thomas Naughton\r\n                   and Suhas Somnath\r\n                   and Scott Atchley\r\n                   and Jack Lange\r\n                   and Ben Mintz\r\n                   and Elke Arenholz\",\r\n  title         = \"The {INTERSECT} Open Federated Architecture for the\r\n                   Laboratory of the Future\",\r\n  booktitle     = \"Communications in Computer and Information Science (CCIS):\r\n                   Accelerating Science and Engineering Discoveries Through\r\n                   Integrated Research Infrastructure for Experiment, Big Data,\r\n                   Modeling and Simulation.\r\n                   \\href{https:\/\/smc.ornl.gov}{$18^{th}$ Smoky Mountains\r\n                   Computational Sciences & Engineering Conference (SMC)\r\n                   2022}\",\r\n  volume        = \"1690\",\r\n  pages         = \"173--190\",\r\n  month         = aug # \"~24-25, \",\r\n  year          = \"2022\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer, Cham}\",\r\n  isbn          = \"978-3-031-23605-1\",\r\n  doi           = \"10.1007\/978-3-031-23606-8_11\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann22intersect.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann22intersect.ppt.pdf\",\r\n  abstract      = \"A federated instrument-to-edge-to-center architecture is\r\n                   needed to autonomously collect, transfer, store, process,\r\n                   curate, and archive scientific data and reduce\r\n                   human-in-the-loop needs with (a) common interfaces to\r\n                   leverage community and custom software, (b) pluggability to\r\n                   permit adaptable solutions, reuse, and digital twins, and (c)\r\n                   an open standard to enable adoption by science facilities\r\n                   world-wide. The INTERSECT Open Architecture enables science\r\n                   breakthroughs using intelligent networked systems,\r\n                   instruments and facilities with autonomous experiments,\r\n                   ``self-driving'' laboratories, smart manufacturing and\r\n                   \\gls{AI} driven design, discovery and evaluation. It creates\r\n                   an open federated architecture for the laboratory of the\r\n                   future using a novel approach, consisting of (1) science use\r\n                   case design patterns, (2) a system of systems architecture,\r\n                   and (3) a microservice architecture.\",\r\n  pts           = \"182854\"\r\n}<\/pre>\n
<\/a>@conference{hukerikar20plexus,\r\n  author        = \"Saurabh Hukerikar\r\n                   and Christian Engelmann\",\r\n  title         = \"{PLEXUS}: {A} Pattern-Oriented Runtime System Architecture\r\n                   for Resilient Extreme-Scale High-Performance Computing\r\n                   Systems\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/prdc.dependability.org\/PRDC2020}\r\n                   {$25^{th}$ IEEE Pacific Rim International Symposium on\r\n                    Dependable Computing (PRDC) 2020}\",\r\n  pages         = \"31--39\",\r\n  month         = dec # \"~1-4, \",\r\n  year          = \"2020\",\r\n  address       = \"Perth, Australia\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  issn          = \"1555-094X\",\r\n  isbn          = \"978-1-7281-8004-5\",\r\n  doi           = \"10.1109\/PRDC50213.2020.00014\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/hukerikar20plexus.pdf\",\r\n  abstract      = \"For high-performance computing (HPC) system designers and\r\n                   users, meeting the myriad challenges of next-generation\r\n                   exascale supercomputing systems requires rethinking their\r\n                   approach to application and system software design. Among\r\n                   these challenges, providing resiliency and stability to the\r\n                   scientific applications in the presence of high fault rates\r\n                   requires new approaches to software architecture and design.\r\n                   As HPC systems become increasingly complex, they require\r\n                   intricate solutions for detection and mitigation for various\r\n                   modes of faults and errors that occur in these large-scale\r\n                   systems, as well as solutions for failure recovery. These\r\n                   resiliency solutions often interact with and affect other\r\n                   system properties, including application scalability, power\r\n                   and energy efficiency. Therefore, resilience solutions for\r\n                   HPC systems must be thoughtfully engineered and deployed.\r\n                   In previous work, we developed the concept of resilience\r\n                   design patterns, which consist of templated solutions based\r\n                   on well-established techniques for detection, mitigation\r\n                   and recovery. In this paper, we use these patterns as the\r\n                   foundation to propose new approaches to designing runtime\r\n                   systems for HPC systems. The instantiation of these\r\n                   patterns within a runtime system enables flexible and\r\n                   adaptable end-to-end resiliency solutions for HPC\r\n                   environments. The paper describes the architecture of the\r\n                   runtime system, named Plexus, and the strategies for\r\n                   dynamically composing and adapting pattern instances under\r\n                   runtime control. This runtime-based approach enables\r\n                   actively balancing the cost-benefit trade-off between\r\n                   performance overhead and protection coverage of the\r\n                   resilience solutions. Based on a prototype implementation\r\n                   of PLEXUS, we demonstrate the resiliency and performance\r\n                   gains achieved by the pattern-based runtime system for a\r\n                   parallel linear solver application.\",\r\n  pts           = \"147029\"\r\n}<\/pre>\n
<\/a>@conference{ostrouchov20gpu,\r\n  author        = \"George Ostrouchov\r\n                   and Don Maxwell\r\n                   and Rizwan Ashraf\r\n                   and Christian Engelmann\r\n                   and Mallikarjun Shankar\r\n                   and James Rogers\",\r\n  title         = \"{GPU} Lifetimes on {Titan} Supercomputer: {Survival} Analysis\r\n                   and Reliability\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/sc20.supercomputing.org}{$33^{rd}$ IEEE\/ACM\r\n                   International Conference on High Performance Computing,\r\n                   Networking, Storage and Analysis (SC) 2020}\",\r\n  pages         = \"41:1--14\",\r\n  month         = nov # \"~15-20, \",\r\n  year          = \"2020\",\r\n  address       = \"Atlanta, GA, USA\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"9781728199986\",\r\n  doi           = \"10.1109\/SC41405.2020.00045\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/ostrouchov20gpu.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/ostrouchov20gpu.ppt.pdf\",\r\n  abstract      = \"The Cray XK7 Titan was the top supercomputer system in the\r\n                   world for a very long time and remained critically important\r\n                   throughout its nearly seven year life. It was also a very\r\n                   interesting machine from a reliability viewpoint as most of\r\n                   its power came from 18,688 GPUs whose operation was forced to\r\n                   execute three very significant rework cycles, two on the GPU\r\n                   mechanical assembly and one on the GPU circuitboards. We\r\n                   write about the last rework cycle and a reliability analysis\r\n                   of over 100,000 operation years in the GPU lifetimes, which\r\n                   correspond to Titan's 6 year long productive period after an\r\n                   initial break-in period. Using time between failures analysis\r\n                   and statistical survival analysis techniques, we find that\r\n                   GPU reliability is dependent on heat dissipation to an extent\r\n                   that strongly correlates with detailed nuances of the system\r\n                   cooling architecture and job scheduling. In addition to\r\n                   describing some of the system history, the data collection,\r\n                   data cleaning, and our analysis of the data, we provide\r\n                   reliability recommendations for designing future state of the\r\n                   art supercomputing systems and their operation. We make the\r\n                   data and our analysis codes publicly available.\",\r\n  pts           = \"144470\"\r\n}<\/pre>\n
<\/a>@conference{jeong203d,\r\n  author        = \"Haewon Jeong\r\n                   and Yaoqing Yang\r\n                   and Christian Engelmann\r\n                   and Vipul Gupta\r\n                   and Tze Meng Low\r\n                   and Pulkit Grover\r\n                   and Viveck Cadambe\r\n                   and Kannan Ramchandran\",\r\n  title         = \"{3D} Coded {SUMMA}: {C}ommunication-Efficient and Robust\r\n                   Parallel Matrix Multiplication\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{https:\/\/www.euro-par.org}{$26^{th}$\r\n                   European Conference on Parallel and Distributed Computing\r\n                   (Euro-Par) 2020}\",\r\n  volume        = \"12247\",\r\n  pages         = \"392--407\",\r\n  month         = aug # \"~24-28, \",\r\n  year          = \"2020\",\r\n  address       = \"Warsaw, Poland\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-030-57674-5\",\r\n  doi           = \"10.1007\/978-3-030-57675-2_25\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/jeong203d.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/jeong203d.ppt.pdf\",\r\n  abstract      = \"In this paper, we propose a novel fault-tolerant parallel\r\n                   matrix multiplication algorithm called 3D Coded SUMMA that is\r\n                   communication efficient and achieves higher failure-tolerance\r\n                   than replication-based schemes for the same amount of\r\n                   redundancy. This work bridges the gap between recent\r\n                   developments in coded computing and fault-tolerance in\r\n                   high-performance computing (HPC). The core idea of coded\r\n                   computing is the same as algorithm-based fault-tolerance\r\n                   (ABFT), which is weaving redundancy in the computation using\r\n                   error-correcting codes. In particular, we show that MatDot\r\n                   codes, an innovative code construction for distributed matrix\r\n                   multiplications, can be integrated into three-dimensional\r\n                   SUMMA (Scalable Universal Matrix Multiplication Algorithm) in\r\n                   a communication-avoiding manner. To tolerate any two node\r\n                   failures, the proposed 3D Coded SUMMA requires 50\\% less\r\n                   redundancy than replication, while the overhead in execution\r\n                   time is only about 5-10\\%.\",\r\n  pts           = \"140756\"\r\n}<\/pre>\n
<\/a>@conference{kumar18understanding,\r\n  author        = \"Mohit Kumar\r\n                   and Saurabh Gupta\r\n                   and Tirthak Patel\r\n                   and Michael Wilder\r\n                   and Weisong Shi\r\n                   and Song Fu\r\n                   and Christian Engelmann\r\n                   and Devesh Tiwari\",\r\n  title         = \"Understanding and Analyzing Interconnect Errors and Network\r\n                   Congestion on a Large Scale {HPC} System\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.dsn.org}\r\n                   {$48^{th}$ IEEE\/IFIP International Conference on Dependable\r\n                    Systems and Networks (DSN) 2018}\",\r\n  pages         = \"107--114\",\r\n  month         = jun # \"~25-28, \",\r\n  year          = \"2018\",\r\n  address       = \"Luxembourg City, Luxembourg\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  issn          = \"2158-3927\",\r\n  isbn          = \"978-1-5386-5596-2\",\r\n  doi           = \"10.1109\/DSN.2018.00023\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/kumar18understanding.pdf\",\r\n  abstract      = \"Today's High Performance Computing (HPC) systems are capable\r\n                   of delivering performance in the order of petaflops due to\r\n                   the fast computing devices, network interconnect, and\r\n                   back-end storage systems. In particular, interconnect\r\n                   resilience and congestion resolution methods have a major\r\n                   impact on the overall interconnect and application\r\n                   performance. This is especially true for scientific\r\n                   applications running multiple processes on different compute\r\n                   nodes as they rely on fast network messages to communicate\r\n                   and synchronize frequently. Unfortunately, the HPC community\r\n                   lacks state-of-practice experience reports that detail how\r\n                   different interconnect errors and congestion events occur\r\n                   on large-scale HPC systems. Therefore, in this paper, we\r\n                   process and analyze interconnect data of the Titan\r\n                   supercomputer to develop a thorough understanding of\r\n                   interconnects faults, errors and congestion events. We also\r\n                   study the interaction between interconnect, errors, network\r\n                   congestion and application characteristics.\",\r\n  pts           = \"110648\"\r\n}<\/pre>\n
<\/a>@conference{nie18machine,\r\n  author        = \"Bin Nie\r\n                   and Ji Xue\r\n                   and Saurabh Gupta\r\n                   and Tirthak Patel\r\n                   and Christian Engelmann\r\n                   and Evgenia Smirni\r\n                   and Devesh Tiwari\",\r\n  title         = \"Machine Learning Models for {GPU} Error Prediction in a Large\r\n                   Scale {HPC} System\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.dsn.org}\r\n                   {$48^{th}$ IEEE\/IFIP International Conference on Dependable\r\n                    Systems and Networks (DSN) 2018}\",\r\n  pages         = \"95--106\",\r\n  month         = jun # \"~25-28, \",\r\n  year          = \"2018\",\r\n  address       = \"Luxembourg City, Luxembourg\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  issn          = \"2158-3927\",\r\n  isbn          = \"978-1-5386-5596-2\",\r\n  doi           = \"10.1109\/DSN.2018.00022\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/nie18machine.pdf\",\r\n  abstract      = \"Recently, GPUs have been widely deployed on large-scale HPC\r\n                   systems to provide powerful computational capability for\r\n                   scientific applications from various domains. As those\r\n                   applications are normally long-running, investigating the\r\n                   characteristics of GPU errors becomes imperative. Therefore,\r\n                   in this paper, we firstly study the conditions that trigger\r\n                   GPU errors with six-month trace data collected from a\r\n                   large-scale operational HPC system. Then, we resort to\r\n                   machine learning techniques to predict the occurrence of\r\n                   GPU errors, by taking advantage of the temporal and spatial\r\n                   dependency of the collected data. As discussed in the\r\n                   evaluation section, the prediction framework is robust and\r\n                   accurate under different workloads.\",\r\n  pts           = \"110650\"\r\n}<\/pre>\n
<\/a>@conference{ashraf18pattern-based,\r\n  author        = \"Rizwan Ashraf\r\n                   and Saurabh Hukerikar\r\n                   and Christian Engelmann\",\r\n  title         = \"Pattern-based Modeling of Multiresilience Solutions for\r\n                   High-Performance Computing\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/icpe2018.spec.org}{$9^{th}$\r\n                   ACM\/SPEC International Conference on Performance Engineering\r\n                   (ICPE) 2018}\",\r\n  pages         = \"80--87\",\r\n  month         = apr # \"~9-13, \",\r\n  year          = \"2018\",\r\n  address       = \"Berlin, Germany\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-4503-5095-2\",\r\n  doi           = \"10.1145\/3184407.3184421\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/ashraf18pattern-based.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/ashraf18pattern-based.ppt.pdf\",\r\n  abstract      = \"Resiliency is the ability of large-scale high-performance\r\n                   computing (HPC) applications to gracefully handle different\r\n                   types of errors, and recover from failures. In this paper, we\r\n                   propose a pattern-based approach to constructing\r\n                   multiresilience solutions. Using resilience patterns, we\r\n                   evaluate the performance and reliability characteristics of\r\n                   detection, containment and mitigation techniques for transient\r\n                   errors that cause silent data corruptions and techniques for\r\n                   fail-stop errors that result in process failures. We\r\n                   demonstrate the design and implementation of the resilience\r\n                   techniques across multiple layers of the system stack such\r\n                   that they are integrated to work together to achieve\r\n                   resiliency to different error types in a highly\r\n                   performance-effcient manner.\",\r\n  pts           = \"109667\"\r\n}<\/pre>\n
<\/a>@conference{ashraf18shrink,\r\n  author        = \"Rizwan Ashraf\r\n                   and Saurabh Hukerikar\r\n                   and Christian Engelmann\",\r\n  title         = \"Shrink or Substitute: {H}andling Process Failures in {HPC}\r\n                   Systems using In-situ Recovery\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.pdp2018.org}{$26^{th}$\r\n                   Euromicro International Conference on Parallel, Distributed,\r\n                   and network-based Processing (PDP) 2018}\",\r\n  pages         = \"178--185\",\r\n  month         = mar # \"~21-23, \",\r\n  year          = \"2018\",\r\n  address       = \"Cambridge, UK\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  issn          = \"2377-5750\",\r\n  isbn          = \"978-1-5386-4975-6\",\r\n  doi           = \"10.1109\/PDP2018.2018.00032\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/ashraf18shrink.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/ashraf18shrink.ppt.pdf\",\r\n  abstract      = \"Efficient utilization of today's high-performance computing\r\n                   (HPC) systems with many, complex software and hardware\r\n                   components requires that the HPC applications are designed to\r\n                   tolerate process failures at runtime. With low\r\n                   mean-time-to-failure (MTTF) of current and future HPC\r\n                   systems, long running simulations on these systems requires\r\n                   capabilities for gracefully handling process failures by the\r\n                   applications themselves. In this paper, we explore the use of\r\n                   fault tolerance extensions to Message Passing Interface (MPI)\r\n                   called user-level failure mitigation (ULFM) for handling\r\n                   process failures without the need to discard the progress\r\n                   made by the application. We explore two alternative recovery\r\n                   strategies, which use ULFM along with application-driven\r\n                   in-memory checkpointing. In the first case, the application\r\n                   is recovered with only the surviving processes, and in the\r\n                   second case, spares are used to replace the failed processes,\r\n                   such that the original configuration of the application is\r\n                   restored. Our experimental results demonstrate that graceful\r\n                   degradation is a viable alternative for recovery in\r\n                   environments where spares may not be available.\",\r\n  pts           = \"107422\"\r\n}<\/pre>\n
<\/a>@conference{gupta17failures,\r\n  author        = \"Saurabh Gupta\r\n                   and Tirthak Patel\r\n                   and Christian Engelmann\r\n                   and Devesh Tiwari\",\r\n  title         = \"Failures in Large Scale Systems: {L}ong-term Measurement,\r\n                   Analysis, and Implications\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/sc17.supercomputing.org}{$30^{th}$ IEEE\/ACM\r\n                   International Conference on High Performance Computing,\r\n                   Networking, Storage and Analysis (SC) 2017}\",\r\n  pages         = \"44:1--44:12\",\r\n  month         = nov # \"~12-17, \",\r\n  year          = \"2017\",\r\n  address       = \"Denver, CO, USA\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-4503-5114-0\",\r\n  doi           = \"10.1145\/3126908.3126937\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/gupta17failures.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/gupta17failures.ppt.pdf\",\r\n  abstract      = \"Resilience is one of the key challenges in maintaining high\r\n                   efficiency of future extreme scale supercomputers.\r\n                   Unfortunately, field-data based reliability studies are far\r\n                   in between and not exhaustive. Most HPC researchers and\r\n                   system practitioners still rely on outdated studies to\r\n                   understand HPC reliability characteristics and plan for\r\n                   future HPC systems. While the complexity of managing system\r\n                   reliability has increased, the public knowledge sharing about\r\n                   lessons learned from HPC centers has not increased in the\r\n                   same proportion. To bridge this gap, in this work, we compare\r\n                   and contrast the reliability characteristics of multiple\r\n                   large-scale HPC production systems, and discuss new\r\n                   take-aways and con rm previous findings which continue to be\r\n                   valid.\",\r\n  pts           = \"100355\"\r\n}<\/pre>\n
<\/a>@conference{nie17characterizing,\r\n  author        = \"Bin Nie\r\n                   and Ji Xue\r\n                   and Saurabh Gupta\r\n                   and Christian Engelmann\r\n                   and Evgenia Smirni\r\n                   and Devesh Tiwari\",\r\n  title         = \"Characterizing Temperature, Power, and Soft-Error Behaviors\r\n                   in Data Center Systems: {I}nsights, Challenges, and\r\n                   Opportunities\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/mascots2017.cs.ucalgary.ca}\r\n                   {$25^{th}$ IEEE International Symposium on the Modeling,\r\n                   Analysis, and Simulation of Computer and Telecommunication\r\n                   Systems (MASCOTS) 2017}\",\r\n  pages         = \"22--31\",\r\n  month         = sep # \"~20-22, \",\r\n  year          = \"2017\",\r\n  address       = \"Banff, AB, Canada\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  issn          = \"2375-0227\",\r\n  isbn          = \"978-1-5386-2764-8\",\r\n  doi           = \"10.1109\/MASCOTS.2017.12\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/nie17characterizing.pdf\",\r\n  url2          = \"\",\r\n  abstract      = \"GPUs have become part of the mainstream high performance\r\n                   computing facilities that increasingly require more\r\n                   computational power to simulate physical phenomena quickly\r\n                   and accurately. However, GPU nodes also consume significantly\r\n                   more power than traditional CPU nodes, and high power\r\n                   consumption introduces new system operation challenges,\r\n                   including increased temperature, power\/cooling cost, and\r\n                   lower system reliability. This paper explores how power\r\n                   consumption and temperature characteristics affect\r\n                   reliability, provides insights into what are the implications\r\n                   of such understanding, and how to exploit these insights\r\n                   toward predicting GPU errors using neural networks.\",\r\n  pts           = \"100351\"\r\n}<\/pre>\n
<\/a>@conference{hukerikar17pattern,\r\n  author        = \"Saurabh Hukerikar\r\n                   and Christian Engelmann\",\r\n  title         = \"A Pattern Language for High-Performance Computing\r\n                   Resilience\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/europlop.net}\r\n                   {$22^{nd}$ European Conference on Pattern Languages of\r\n                   Programs (EuroPLoP) 2017}\",\r\n  pages         = \"12:1--12:16\",\r\n  month         = jul # \"~12-16, \",\r\n  year          = \"2017\",\r\n  address       = \"Kloster Irsee, Germany\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-4503-4848-5\",\r\n  doi           = \"10.1145\/3147704.3147718\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/hukerikar17pattern.pdf\",\r\n  abstract      = \"High-performance computing systems (HPC) provide powerful\r\n                   capabilities for modeling and simulation, and data analytics\r\n                   in a broad class of computational problems in a variety of\r\n                   scientific and engineering domains. HPC designs are\r\n                   undergoing rapid changes in the hardware architectures and\r\n                   the software environment as the community pursues\r\n                   increasingly capable HPC systems. Among the key challenges\r\n                   for future generations of HPC systems is the ensuring\r\n                   efficient and correct operation despite the occurrence of\r\n                   faults or defects in system components that can cause errors\r\n                   and failures in a HPC system. Such events affect the\r\n                   correctness of the scientific applications, or may lead to\r\n                   their untimely termination. Future generations of HPC systems\r\n                   will consist of millions of compute, memory and storage\r\n                   components and the growing complexity of these computing\r\n                   behemoths increases the chances that a single fault event will\r\n                   cascade across the machine and bring down the entire system.\r\n                   Design patterns capture the essential techniques that are\r\n                   employed to solve recurring problems in the design of\r\n                   resilient computing systems. However, the complexity of\r\n                   modern HPC systems as well as the various challenges of\r\n                   future generations of systems requires consideration to\r\n                   numerous aspects and optimization principles, such as the\r\n                   impact of a resilience solution on the performance and\r\n                   power consumption. We present a pattern language for\r\n                   engineering resilience solutions. The language is targeted\r\n                   at hardware and software designers as well as the users and\r\n                   operators of HPC systems. The patterns are intended to\r\n                   develop complete resilience solutions that have different\r\n                   efficiency and complexity characteristics, which may be\r\n                   deployed at design time or runtime to ensure that HPC systems\r\n                   are able to deal with various types of faults, errors and\r\n                   failures.\",\r\n  pts           = \"102869\"\r\n}<\/pre>\n
<\/a>@conference{lagadapati16benchmark,\r\n  author        = \"Mahesh Lagadapati\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\",\r\n  title         = \"Benchmark Generation and Simulation at Extreme Scale\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/ds-rt.com\/2016}{$20^{th}$\r\n                   IEEE\/ACM International Symposium on Distributed Simulation\r\n                   and Real Time Applications (DS-RT) 2016}\",\r\n  pages         = \"9--18\",\r\n  month         = sep # \"~21-23, \",\r\n  year          = \"2016\",\r\n  address       = \"London, UK\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  issn          = \"1550-6525\",\r\n  isbn          = \"978-1-5090-3506-9\",\r\n  doi           = \"10.1109\/DS-RT.2016.18\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/lagadapati16benchmark.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/lagadapati16benchmark.ppt.pdf\",\r\n  abstract      = \"The path to extreme scale high-performance computing (HPC)\r\n                   poses several challenges related to power, performance,\r\n                   resilience, productivity, programmability, data movement, and\r\n                   data management. Investigating the performance of parallel\r\n                   applications at scale on future architectures and the\r\n                   performance impact of different architectural choices is an\r\n                   important component of HPC hardware\/software co-design.\r\n                   Simulations using models of future HPC systems and\r\n                   communication traces from applications running on existing\r\n                   HPC systems can offer an insight into the performance of\r\n                   future architectures. This work targets technology developed\r\n                   for scalable application tracing of communication events. It\r\n                   focuses on extreme-scale simulation of HPC applications and\r\n                   their communication behavior via lightweight parallel\r\n                   discrete event simulation for performance estimation and\r\n                   evaluation. Instead of simply replaying a trace within a\r\n                   simulator, this work promotes the generation of a benchmark\r\n                   from traces. This benchmark is subsequently exposed to\r\n                   simulation using models to reflect the performance\r\n                   characteristics of future-generation HPC systems. This\r\n                   technique provides a number of benefits, such as eliminating\r\n                   the data intensive trace replay and enabling simulations at\r\n                   different scales. The presented work features novel software\r\n                   co-design aspects, combining the ScalaTrace tool to generate\r\n                   scalable trace files, the ScalaBenchGen tool to generate the\r\n                   benchmark, and the xSim tool to assess the benchmark\r\n                   characteristics within a simulator.\",\r\n  pts           = \"68383\"\r\n}<\/pre>\n
<\/a>@conference{hukerikar16havens,\r\n  author        = \"Saurabh Hukerikar\r\n                   and Christian Engelmann\",\r\n  title         = \"{Havens}: {Explicit} Reliable Memory Regions for {HPC}\r\n                   Applications\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/ieee-hpec.org}\r\n                   {$20^{th}$ IEEE High Performance Extreme Computing\r\n                   Conference (HPEC) 2016}\",\r\n  pages         = \"1--6\",\r\n  month         = sep # \"~13-15, \",\r\n  year          = \"2016\",\r\n  address       = \"Waltham, MA, USA\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  doi           = \"10.1109\/HPEC.2016.7761593\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/hukerikar16havens.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/hukerikar16havens.ppt.pdf\",\r\n  abstract      = \"Supporting error resilience in future exascale-class\r\n                   supercomputing systems is a critical challenge. Due to\r\n                   transistor scaling trends and increasing memory density,\r\n                   the scientific simulations are expected to experience more\r\n                   interruptions caused by soft errors in the system memory.\r\n                   Existing hardware-based detection and recovery techniques\r\n                   will be inadequate in the presence of high memory fault\r\n                   rates.\r\n                   In this paper we propose a partial memory protection scheme\r\n                   using region-based memory management. We define regions\r\n                   called havens that provide fault protection for program\r\n                   objects. We provide reliability for the regions through a\r\n                   software-based parity protection mechanism. Our approach\r\n                   enables critical application code and variables to be placed\r\n                   in these havens. The fault coverage of our approach is\r\n                   application agnostic unlike algorithm-based fault tolerance\r\n                   techniques.\",\r\n  pts           = \"69230\"\r\n}<\/pre>\n
<\/a>@conference{tang16power-capping,\r\n  author        = \"Kun Tang\r\n                   and Devesh Tiwari\r\n                   and Saurabh Gupta\r\n                   and Ping Huang\r\n                   and QiQi Lu\r\n                   and Christian Engelmann\r\n                   and Xubin He\",\r\n  title         = \"Power-Capping Aware Checkpointing: {On} the Interplay Among\r\n                   Power-Capping, Temperature, Reliability, Performance, and\r\n                   Energy\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.dsn.org}\r\n                   {$46^{th}$ IEEE\/IFIP International Conference on Dependable\r\n                    Systems and Networks (DSN) 2016}\",\r\n  pages         = \"311--322\",\r\n  month         = jun # \"~28 - \" # jul # \"~1, \",\r\n  year          = \"2016\",\r\n  address       = \"Toulouse, France\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  issn          = \"2158-3927\",\r\n  doi           = \"10.1109\/DSN.2016.36\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/tang16power-aware.pdf\",\r\n  url2          = \"\",\r\n  abstract      = \"Checkpoint and restart mechanisms have been widely used in\r\n                   large scientific simulation applications to make forward\r\n                   progress in case of failures. However, none of the prior\r\n                   works have considered the interaction of power-constraint\r\n                   with temperature, reliability, performance, and checkpointing\r\n                   interval. It is not clear how power-capping may affect\r\n                   optimal checkpointing interval. What are the involved\r\n                   reliability, performance, and energy trade-offs? In this\r\n                   paper, we develop a deep understanding about the interaction\r\n                   between power-capping and scientific applications using\r\n                   checkpoint\/restart as resilience mechanism, and propose a\r\n                   new model for the optimal checkpointing interval (OCI) under\r\n                   power-capping. Our study reveals several interesting, and\r\n                   previously unknown, insights about how power-capping affects\r\n                   the reliability, energy consumption, performance.\",\r\n  pts           = \"62738\"\r\n}<\/pre>\n
<\/a>@conference{fiala16mini-ckpts,\r\n  author        = \"David Fiala\r\n                   and Frank Mueller\r\n                   and Kurt Ferreira\r\n                   and Christian Engelmann\",\r\n  title         = \"{Mini-Ckpts}: Surviving {OS} Failures in Persistent Memory\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/ics16.bilkent.edu.tr}\r\n                   {$30^{th}$ ACM International Conference on Supercomputing\r\n                    (ICS) 2016}\",\r\n  pages         = \"7:1--7:14\",\r\n  month         = jun # \"~1-3, \",\r\n  year          = \"2016\",\r\n  address       = \"Istanbul, Turkey\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-4503-4361-9\",\r\n  doi           = \"10.1145\/2925426.2926295\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/fiala16mini-ckpts.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/fiala16mini-ckpts.ppt.pdf\",\r\n  abstract      = \"Concern is growing in the high-performance computing (HPC)\r\n                   community on the reliability of future extreme-scale systems.\r\n                   Current efforts have focused on application fault-tolerance\r\n                   rather than the operating system (OS), despite the fact that\r\n                   recent studies have suggested that failures in OS memory are\r\n                   more likely. The OS is critical to a system's correct and\r\n                   efficient operation of the node and processes it governs ---\r\n                   and in HPC also for any other nodes a parallelized\r\n                   application runs on and communicates with: Any single node\r\n                   failure generally forces all processes of this application\r\n                   to terminate due to tight communication in HPC. Therefore,\r\n                   the OS itself must be capable of tolerating failures. In\r\n                   this work, we introduce mini-ckpts, a framework which\r\n                   enables application survival despite the occurrence of a\r\n                   fatal OS failure or crash. Mini-ckpts achieves this\r\n                   tolerance by ensuring that the critical data describing a\r\n                   process is preserved in persistent memory prior to the\r\n                   failure. Following the failure, the OS is rejuvenated via\r\n                   a warm reboot and the application continues execution\r\n                   effectively making the failure and restart transparent. The\r\n                   mini-ckpts rejuvenation and recovery process is measured to\r\n                   take between three to six seconds and has a failure-free\r\n                   overhead of between 3-5\\% for a number of key HPC workloads.\r\n                   In contrast to current fault-tolerance methods, this work\r\n                   ensures that the operating and runtime system can continue in\r\n                   the presence of faults. This is a much finer-grained and\r\n                   dynamic method of fault-tolerance than the current,\r\n                   coarse-grained, application-centric methods. Handling faults\r\n                   at this level has the potential to greatly reduce overheads\r\n                   and enables mitigation of additional fault scenarios.\",\r\n  pts           = \"67816\"\r\n}<\/pre>\n
<\/a>@conference{bautista-gomez16reducing,\r\n  author        = \"Leonardo Bautista-Gomez\r\n                   and Ana Gainaru\r\n                   and Swann Perarnau\r\n                   and Devesh Tiwari\r\n                   and Saurabh Gupta\r\n                   and Franck Cappello\r\n                   and Christian Engelmann\r\n                   and Marc Snir\",\r\n  title         = \"Reducing Waste in Extreme Scale Systems Through Introspective\r\n                   Analysis\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.ipdps.org}\r\n                   {$30^{th}$ IEEE International Parallel and Distributed\r\n                   Processing Symposium (IPDPS) 2016}\",\r\n  pages         = \"212--221\",\r\n  month         = may # \"~23-27, \",\r\n  year          = \"2016\",\r\n  address       = \"Chicago, IL, USA\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  issn          = \"1530-2075\",\r\n  doi           = \"10.1109\/IPDPS.2016.100\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/bautista-gomez16reducing.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/bautista-gomez16reducing.ppt.pdf\",\r\n  abstract      = \"Resilience is an important challenge for extreme-scale \r\n                   supercomputers. Today, failures in supercomputers are \r\n                   assumed to be uniformly distributed in time. However, recent \r\n                   studies show that failures in high-performance computing \r\n                   systems are partially correlated in time, generating periods \r\n                   of higher failure density. Our study of the failure logs of \r\n                   multiple supercomputers show that periods of higher failure \r\n                   density occur with up to three times more than the average. \r\n                   We design a monitoring system that listens to hardware \r\n                   events and forwards important events to the runtime to \r\n                   detect those regime changes. We implement a runtime capable \r\n                   of receiving notifications and adapt dynamically. In \r\n                   addition, we build an analytical model to predict the gains \r\n                   that such dynamic approach could achieve. We demonstrate that \r\n                   in some systems, our approach can reduce the wasted time.\",\r\n  pts           = \"62159\"\r\n}<\/pre>\n
<\/a>@conference{engelmann16supporting,\r\n  author        = \"Christian Engelmann\r\n                   and Thomas Naughton\",\r\n  title         = \"Supporting the Development of Soft-Error Resilient Message\r\n                   Passing Applications using Simulation\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.iasted.org\/conferences\/home-795.html}\r\n                   {$13^{th}$ IASTED International Conference on Parallel and\r\n                   Distributed Computing and Networks (PDCN) 2016}\",\r\n  month         = feb # \"~15-16, \",\r\n  year          = \"2016\",\r\n  address       = \"Innsbruck, Austria\",\r\n  publisher     = \"\\href{http:\/\/www.actapress.com}{ACTA Press, Calgary, AB,\r\n                   Canada}\",\r\n  isbn          = \"978-0-88986-979-0\",\r\n  doi           = \"10.2316\/P.2016.834-005\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann16supporting.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann16supporting.ppt.pdf\",\r\n  abstract      = \"Radiation-induced bit flip faults are of particular concern\r\n                   in extreme-scale high-performance computing systems. This\r\n                   paper presents a simulation-based tool that enables the\r\n                   development of soft-error resilient message passing\r\n                   applications by permitting the investigation of their\r\n                   correctness and performance under various fault conditions.\r\n                   The documented extensions to the Extreme-scale Simulator\r\n                   (xSim) enable the injection of bit flip faults at specific\r\n                   of injection location(s) and fault activation time(s),\r\n                   while supporting a significant degree of configurability of\r\n                   the fault type. Experiments show that the simulation\r\n                   overhead with the new feature is $\\sim$2,325\\% for serial\r\n                   execution and $\\sim$1,730\\% at 128 MPI processes, both with\r\n                   very fine-grain fault injection. Fault injection experiments\r\n                   demonstrate the usefulness of the new feature by injecting\r\n                   bit flips in the input and output matrices of a matrix-matrix\r\n                   multiply application, revealing vulnerability of data\r\n                   structures, masking and error propagation. xSim is the very\r\n                   first simulation-based MPI performance tool that supports\r\n                   both, the injection of process failures and bit flip faults.\",\r\n  pts           = \"60888\"\r\n}<\/pre>\n
<\/a>@conference{katti15scalable,\r\n  author        = \"Amogh Katti\r\n                   and Giuseppe Di Fatta\r\n                   and Thomas Naughton\r\n                   and Christian Engelmann\",\r\n  title         = \"Scalable and Fault Tolerant Failure Detection and Consensus\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{https:\/\/eurompi2015.bordeaux.inria.fr}{$22^{nd}$\r\n                   European MPI Users` Group Meeting (EuroMPI) 2015}\",\r\n  pages         = \"13:1--13:9\",\r\n  month         = sep # \"~21-24, \",\r\n  year          = \"2015\",\r\n  address       = \"Bordeaux, France\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-4503-3795-3\",\r\n  doi           = \"10.1145\/2802658.2802660\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/katti15scalable.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/katti15scalable.ppt.pdf\",\r\n  abstract      = \"Future extreme-scale high-performance computing systems will\r\n                   be required to work under frequent component failures. The\r\n                   MPI Forum's User Level Failure Mitigation proposal has\r\n                   introduced an operation (MPI\\_Comm\\_shrink) to synchronize\r\n                   the alive processes on the list of failed processes, so that\r\n                   applications can continue to execute even in the presence of\r\n                   failures by adopting algorithm-based fault tolerance\r\n                   techniques. The MPI\\_Comm\\_shrink operation requires a\r\n                   fault tolerant failure detection and consensus algorithm.\r\n                   This paper presents and compares two novel failure detection\r\n                   and consensus algorithms to support this operation. The\r\n                   proposed algorithms are based on Gossip protocols and are\r\n                   inherently fault-tolerant and scalable. The proposed\r\n                   algorithms were implemented and tested using the\r\n                   Extreme-scale Simulator. The results show that in both\r\n                   algorithms the number of Gossip cycles to achieve global\r\n                   consensus scales logarithmically with system size. The second\r\n                   algorithm also shows better scalability in terms of memory\r\n                   usage and network bandwidth costs and a perfect\r\n                   synchronization in achieving global consensus.\",\r\n  pts           = \"57940\"\r\n}<\/pre>\n
<\/a>@conference{engelmann15network,\r\n  author        = \"Christian Engelmann\r\n                   and Thomas Naughton\",\r\n  title         = \"A Network Contention Model for the Extreme-scale Simulator\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.iasted.org\/conferences\/home-826.html}\r\n                   {$34^{th}$ IASTED International Conference on Modelling,\r\n                   Identification and Control (MIC) 2015}\",\r\n  month         = feb # \"~17-18, \",\r\n  year          = \"2015\",\r\n  address       = \"Innsbruck, Austria\",\r\n  publisher     = \"\\href{http:\/\/www.actapress.com}{ACTA Press, Calgary, AB,\r\n                   Canada}\",\r\n  isbn          = \"978-0-88986-975-2\",\r\n  doi           = \"10.2316\/P.2015.826-043\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann15network.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann15network.ppt.pdf\",\r\n  abstract      = \"The Extreme-scale Simulator (xSim) is a performance\r\n                   investigation toolkit for high-performance computing (HPC)\r\n                   hardware\/software co-design. It permits running a HPC\r\n                   application with millions of concurrent execution threads,\r\n                   while observing its performance in a simulated extreme-scale\r\n                   system. This paper details a newly developed network modeling\r\n                   feature for xSim, eliminating the shortcomings of the\r\n                   existing network modeling capabilities. The approach takes a\r\n                   different path for implementing network contention and\r\n                   bandwidth capacity modeling using a less synchronous and\r\n                   accurate enough model design. With the new network modeling\r\n                   feature, xSim is able to simulate on-chip and on-node\r\n                   networks with reasonable accuracy and overheads.\",\r\n  pts           = \"53873\"\r\n}<\/pre>\n
<\/a>@conference{engelmann14improving,\r\n  author        = \"Christian Engelmann\r\n                   and Thomas Naughton\",\r\n  title         = \"Improving the Performance of the Extreme-scale Simulator\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/ds-rt.com\/2014}{$18^{th}$\r\n                   IEEE\/ACM International Symposium on Distributed Simulation\r\n                   and Real Time Applications (DS-RT) 2014}\",\r\n  pages         = \"198--207\",\r\n  month         = oct # \"~1-3, \",\r\n  year          = \"2014\",\r\n  address       = \"Toulouse, France\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  issn          = \"1550-6525\",\r\n  isbn          = \"978-1-4799-6143-6\",\r\n  doi           = \"10.1109\/DS-RT.2014.32\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann14improving.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann14improving.ppt.pdf\",\r\n  abstract      = \"Investigating the performance of parallel applications\r\n                   at scale on future high-performance computing (HPC)\r\n                   architectures and the performance impact of different\r\n                   architecture choices is an important component of HPC\r\n                   hardware\/software co-design.\r\n                   The Extreme-scale Simulator (xSim) is a simulation-based\r\n                   toolkit for investigating the performance of parallel\r\n                   applications at scale. xSim scales to millions of simulated\r\n                   Message Passing Interface (MPI) processes. The overhead\r\n                   introduced by a simulation tool is an important performance\r\n                   and productivity aspect. This paper documents two\r\n                   improvements to xSim: (1) a new deadlock resolution protocol\r\n                   to reduce the parallel discrete event simulation management\r\n                   overhead and (2) a new simulated MPI message matching\r\n                   algorithm to reduce the oversubscription management overhead.\r\n                   The results clearly show a significant performance\r\n                   improvement, such as by reducing the simulation overhead for\r\n                   running the NAS Parallel Benchmark suite inside the simulator \r\n                   from 1,020\\% to 238\\% for the conjugate gradient (CG)\r\n                   benchmark and from 102\\% to 0\\% for the embarrassingly\r\n                   parallel (EP) and benchmark, as well as, from 37,511\\% to\r\n                   13,808\\% for CG and from 3,332\\% to 204\\% for EP with\r\n                   accurate process failure simulation.\",\r\n  pts           = \"50654\"\r\n}<\/pre>\n
<\/a>@conference{naughton14supporting,\r\n  author        = \"Thomas Naughton\r\n                   and Christian Engelmann\r\n                   and Geoffroy Vall{\\'e}e\r\n                   and Swen B{\\\"o}hm\",\r\n  title         = \"Supporting the Development of Resilient Message Passing\r\n                   Applications using Simulation\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.pdp2014.org}{$22^{nd}$\r\n                   Euromicro International Conference on Parallel, Distributed,\r\n                   and network-based Processing (PDP) 2014}\",\r\n  pages         = \"271--278\",\r\n  month         = feb # \"~12-14, \",\r\n  year          = \"2014\",\r\n  address       = \"Turin, Italy\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  issn          = \"1066-6192\",\r\n  doi           = \"10.1109\/PDP.2014.74\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/naughton14supporting.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/naughton14supporting.ppt.pdf\",\r\n  abstract      = \"An emerging aspect of high-performance computing (HPC)\r\n                   hardware\/software co-design is investigating performance\r\n                   under failure. The work in this paper extends the\r\n                   Extreme-scale Simulator (xSim), which was designed for\r\n                   evaluating the performance of message passing interface\r\n                   (MPI) applications on future HPC architectures, with\r\n                   fault-tolerant MPI extensions proposed by the MPI Fault\r\n                   Tolerance Working Group. xSim permits running MPI\r\n                   applications with millions of concurrent MPI ranks, while\r\n                   observing application performance in a simulated\r\n                   extreme-scale system using a lightweight parallel discrete\r\n                   event simulation. The newly added features offer user-level\r\n                   failure mitigation (ULFM) extensions at the simulated MPI\r\n                   layer to support algorithm-based fault tolerance (ABFT).\r\n                   The presented solution permits investigating performance\r\n                   under failure and failure handling of ABFT solutions.\r\n                   The newly enhanced xSim is the very first performance tool\r\n                   that supports ULFM and ABFT.\",\r\n  pts           = \"49204\"\r\n}<\/pre>\n
<\/a>@conference{vallee13runtime,\r\n  author        = \"Geoffroy Vall{\\'e}e\r\n                   and Thomas Naughton\r\n                   and Swen B{\\\"o}hm\r\n                   and Christian Engelmann\",\r\n  title         = \"A Runtime Environment for Supporting Research in Resilient\r\n                   {HPC} System Software & Tools\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/is-candar.org}\r\n                   {$1^{st}$ International Symposium on Computing and\r\n                   Networking - Across Practical Development and Theoretical\r\n                   Research - (CANDAR) 2013}\",\r\n  pages         = \"213--219\",\r\n  month         = dec # \"~4-6, \",\r\n  year          = \"2013\",\r\n  address       = \"Matsuyama, Japan\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-1-4799-2795-1\",\r\n  doi           = \"10.1109\/CANDAR.2013.38\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/vallee13runtime.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/vallee13runtime.ppt.pdf\",\r\n  abstract      = \"The high-performance computing~(HPC) community continues to\r\n                   increase the size and complexity of hardware platforms that\r\n                   support advanced scientific workloads. The runtime\r\n                   environment (RTE) is a crucial layer in the software\r\n                   stack for these large-scale systems. The RTE manages the\r\n                   interface between the operating system and the application\r\n                   running in parallel on the machine. The deployment of\r\n                   applications and tools on large-scale HPC computing systems\r\n                   requires the RTE to manage process creation in a scalable\r\n                   manner, support sparse connectivity, and provide fault\r\n                   tolerance. We have developed a new RTE that provides a basis\r\n                   for building distributed execution environments and\r\n                   developing tools for HPC to aid research in system software\r\n                   and resilience. This paper describes the software\r\n                   architecture of the Scalable runTime Component\r\n                   Infrastructure~(STCI), which is intended to provide a\r\n                   complete infrastructure for scalable start-up and\r\n                   management of many processes in large-scale HPC systems. We\r\n                   highlight features of the current implementation, which is\r\n                   provided as a system library that allows developers to easily\r\n                   use and integrate STCI in their tools and\/or applications.\r\n                   The motivation for this work has been to support ongoing\r\n                   research activities in fault-tolerance for large-scale\r\n                   systems. We discuss the advantages of the modular framework\r\n                   employed and describe two use cases that demonstrate its\r\n                   capabilities: (i) an alternate runtime for a Message Passing\r\n                   Interface (MPI) stack, and (ii) a distributed control and\r\n                   communication substrate for a fault-injection tool.\",\r\n  pts           = \"45674\"\r\n}<\/pre>\n
<\/a>@conference{engelmann13investigating,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Investigating Operating System Noise in Extreme-Scale\r\n                   High-Performance Computing Systems using Simulation\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.iasted.org\/conferences\/home-795.html}\r\n                   {$11^{th}$ IASTED International Conference on Parallel and\r\n                   Distributed Computing and Networks (PDCN) 2013}\",\r\n  month         = feb # \"~11-13, \",\r\n  year          = \"2013\",\r\n  address       = \"Innsbruck, Austria\",\r\n  publisher     = \"\\href{http:\/\/www.actapress.com}{ACTA Press, Calgary, AB,\r\n                   Canada}\",\r\n  isbn          = \"978-0-88986-943-1\",\r\n  doi           = \"10.2316\/P.2013.795-010\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann12investigating.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann12investigating.ppt.pdf\",\r\n  abstract      = \"Hardware\/software co-design for future-generation\r\n                   high-performance computing (HPC) systems aims at closing\r\n                   the gap between the peak capabilities of the hardware\r\n                   and the performance realized by applications\r\n                   (application-architecture performance gap). Performance\r\n                   profiling of architectures and applications is a crucial\r\n                   part of this iterative process. The work in this paper\r\n                   focuses on operating system (OS) noise as an additional\r\n                   factor to be considered for co-design. It represents the\r\n                   first step in including OS noise in HPC hardware\/software\r\n                   co-design by adding a noise injection feature to an existing\r\n                   simulation-based co-design toolkit. It reuses an existing\r\n                   abstraction for OS noise with frequency (periodic recurrence)\r\n                   and period (duration of each occurrence) to enhance the\r\n                   processor model of the Extreme-scale Simulator (xSim) with\r\n                   synchronized and random OS noise simulation. The results\r\n                   demonstrate this capability by evaluating the impact of OS\r\n                   noise on MPI\\_Bcast() and MPI\\_Reduce() in a simulated\r\n                   future-generation HPC system with 2,097,152 compute nodes.\",\r\n  pts           = \"40576\"\r\n}<\/pre>\n
<\/a>@conference{fiala12detection2,\r\n  author        = \"David Fiala\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\r\n                   and Kurt Ferreira\r\n                   and Ron Brightwell\r\n                   and Rolf Riesen\",\r\n  title         = \"Detection and Correction of Silent Data Corruption for\r\n                   Large-Scale High-Performance Computing\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/sc12.supercomputing.org}{$25^{th}$ IEEE\/ACM\r\n                   International Conference on High Performance Computing,\r\n                   Networking, Storage and Analysis (SC) 2012}\",\r\n  pages         = \"78:1--78:12\",\r\n  month         = nov # \"~10-16, \",\r\n  year          = \"2012\",\r\n  address       = \"Salt Lake City, UT, USA\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-4673-0804-5\",\r\n  doi           = \"10.1109\/SC.2012.49\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/fiala12detection2.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/fiala12detection2.ppt.pdf\",\r\n  abstract      = \"Faults have become the norm rather than the exception for\r\n                   high-end computing on clusters with 10s\/100s of thousands of\r\n                   cores. Exacerbating this situation, some of these faults\r\n                   remain undetected, manifesting themselves as silent errors\r\n                   that corrupt memory while applications continue to operate\r\n                   and report incorrect results.\r\n                   This paper studies the potential for redundancy to both\r\n                   detect and correct soft errors in MPI message-passing\r\n                   applications. Our study investigates the challenges inherent\r\n                   to detecting soft errors within MPI application while\r\n                   providing transparent MPI redundancy. By assuming a model\r\n                   wherein corruption in application data manifests itself by\r\n                   producing differing MPI message data between replicas, we\r\n                   study the best suited protocols for detecting and correcting\r\n                   MPI data that is the result of corruption.\r\n                   To experimentally validate our proposed detection and\r\n                   correction protocols, we introduce RedMPI, an MPI library\r\n                   which resides in the MPI profiling layer. RedMPI is capable\r\n                   of both online detection and correction of soft errors that\r\n                   occur in MPI applications without requiring any\r\n                   modifications to the application source by utilizing either\r\n                   double or triple redundancy.\r\n                   Our results indicate that our most efficient consistency\r\n                   protocol can successfully protect applications experiencing\r\n                   even high rates of silent data corruption with runtime\r\n                   overheads between 0\\% and 30\\% as compared to unprotected\r\n                   applications without redundancy.\r\n                   Using our fault injector within RedMPI, we observe that even\r\n                   a single soft error can have profound effects on running\r\n                   applications, causing a cascading pattern of corruption in\r\n                   most cases causes that spreads to all other processes.\r\n                   RedMPI's protection has been shown to successfully mitigate\r\n                   the effects of soft errors while allowing applications to\r\n                   complete with correct results even in the face of errors.\",\r\n  pts           = \"38306\"\r\n}<\/pre>\n
<\/a>@conference{elliott12combining,\r\n  author        = \"James Elliott\r\n                   and Kishor Kharbas\r\n                   and David Fiala\r\n                   and Frank Mueller\r\n                   and Kurt Ferreira\r\n                   and Christian Engelmann\",\r\n  title         = \"Combining Partial Redundancy and Checkpointing for {HPC}\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/icdcs-2012.org\/}\r\n                   {$32^{nd}$ International Conference on Distributed\r\n                   Computing Systems (ICDCS) 2012}\",\r\n  pages         = \"615--626\",\r\n  month         = jun # \"~18-21, \",\r\n  year          = \"2012\",\r\n  address       = \"Macau, SAR, China\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7695-4685-8\",\r\n  issn          = \"1063-6927\",\r\n  doi           = \"10.1109\/ICDCS.2012.56\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/elliott12combining.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/elliott12combining.ppt.pdf\",\r\n  abstract      = \"Today's largest High Performance Computing (HPC) systems\r\n                   exceed one Petaflops (10^15 floating point operations per\r\n                   second) and exascale systems are projected within seven\r\n                   years. But reliability is becoming one of the major\r\n                   challenges faced by exascale computing. With billion-core\r\n                   parallelism, the mean time to failure is projected to be in\r\n                   the range of minutes or hours instead of days. Failures are\r\n                   becoming the norm rather than the exception during execution\r\n                   of HPC applications. Current fault tolerance techniques in\r\n                   HPC focus on reactive ways to mitigate faults, namely via\r\n                   checkpoint and restart (C\/R). Apart from storage overheads,\r\n                   C\/R-based fault recovery comes at an additional cost in\r\n                   terms of application performance because normal execution\r\n                   is disrupted when checkpoints are taken. Studies have shown\r\n                   that applications running at a large scale spend more than\r\n                   50\\% of their total time saving checkpoints, restarting and\r\n                   redoing lost work. Redundancy is another fault tolerance\r\n                   technique, which employs redundant processes performing the\r\n                   same task. If a process fails, a replica of it can take over\r\n                   its execution. Thus, redundant copies can decrease the\r\n                   overall failure rate. The downside of redundancy is that\r\n                   extra resources are required and there is an additional\r\n                   overhead on communication and synchronization. This work\r\n                   contributes a model and analyzes the benefit of C\/R in\r\n                   coordination with redundancy at different degrees to\r\n                   minimize the total wallclock time and resources utilization\r\n                   of HPC applications. We further conduct experiments with an\r\n                   implementation of redundancy within the MPI layer on a\r\n                   cluster. Our experimental results confirm the benefit of dual\r\n                   and triple redundancy - but not for partial redundancy - and\r\n                   show a close fit to the model. At 80,000 processes, dual\r\n                   redundancy requires twice the number of processing resources\r\n                   for an application but allows two jobs of 128 hours wallclock\r\n                   time to finish within the time of just one job without\r\n                   redundancy. For narrow ranges of processor counts, partial\r\n                   redundancy results in the lowest time. Once the count exceeds\r\n                   770, 000, triple redundancy has the lowest overall cost.\r\n                   Thus, redundancy allows one to trade-off additional resource\r\n                   requirements against wallclock time, which provides a tuning\r\n                   knob for users to adapt to resource availabilities.\",\r\n  pts           = \"35629\"\r\n}<\/pre>\n
<\/a>@conference{wang12nvmalloc,\r\n  author        = \"Chao Wang\r\n                   and Sudharshan S. Vazhkudai\r\n                   and Xiaosong Ma\r\n                   and Fei Meng\r\n                   and Youngjae Kim\r\n                   and Christian Engelmann\",\r\n  title         = \"{NVMalloc}: Exposing an Aggregate {SSD} Store as a Memory\r\n                   Partition in Extreme-Scale Machines\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.ipdps.org}\r\n                   {$26^{th}$ IEEE International Parallel and Distributed\r\n                   Processing Symposium (IPDPS) 2012}\",\r\n  pages         = \"957--968\",\r\n  month         = may # \"~21-25, \",\r\n  year          = \"2012\",\r\n  address       = \"Shanghai, China\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7695-4675-9\",\r\n  doi           = \"10.1109\/IPDPS.2012.90\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/wang12nvmalloc.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/wang12nvmalloc.ppt.pdf\",\r\n  abstract      = \"DRAM is a precious resource in extreme-scale machines and is\r\n                   increasingly becoming scarce, mainly due to the growing\r\n                   number of cores per node. On future multi-petaflop and\r\n                   exaflop machines, the memory pressure is likely to be so\r\n                   severe that we need to rethink our memory usage models.\r\n                   Fortunately, the advent of non-volatile memory (NVM) offers\r\n                   a unique opportunity in this space. Current NVM offerings\r\n                   possess several desirable properties, such as low cost and\r\n                   power efficiency, but also suffer from high latency and\r\n                   lifetime issues. We need rich techniques to be able to use\r\n                   them alongside DRAM. In this paper, we propose a novel\r\n                   approach to exploiting NVM as a secondary memory partition\r\n                   so that applications can explicitly allocate and manipulate\r\n                   memory regions therein. More specifically, we propose an\r\n                   NVMalloc library with a suite of services that enables\r\n                   applications to access a distributed NVM storage system.\r\n                   We have devised ways within NVMalloc so that the storage\r\n                   system, built from compute node-local NVM devices, can be\r\n                   accessed in a byte-addressable fashion using the memory\r\n                   mapped I\/O interface. Our approach has the potential to\r\n                   re-energize out-of-core computations on large-scale machines\r\n                   by having applications allocate certain variables through\r\n                   NVMalloc, thereby increasing the overall memory available\r\n                   for the application. Our evaluation on a 128-core cluster\r\n                   shows that NVMalloc enables applications to compute problem\r\n                   sizes larger than the physical memory in a cost-effective\r\n                   manner. It can achieve better performance with increased\r\n                   computation time between NVM memory accesses or increased\r\n                   data access locality. In addition, our results suggest that\r\n                   while NVMalloc enables transparent access to NVM-resident\r\n                   variables, the explicit control it provides is crucial to\r\n                   optimize application performance.\",\r\n  pts           = \"35603\"\r\n}<\/pre>\n
<\/a>@conference{boehm12file,\r\n  author        = \"Swen B{\\\"o}hm and\r\n                   Christian Engelmann\",\r\n  title         = \"File {I\/O} for {MPI} Applications in Redundant Execution\r\n                   Scenarios\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.pdp2012.org}{$20^{th}$\r\n                   Euromicro International Conference on Parallel, Distributed,\r\n                   and network-based Processing (PDP) 2012}\",\r\n  pages         = \"112-119\",\r\n  month         = feb # \"~15-17, \",\r\n  year          = \"2012\",\r\n  address       = \"Garching, Germany\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7695-4633-9\",\r\n  issn          = \"1066-6192\",\r\n  doi           = \"10.1109\/PDP.2012.22\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/boehm12file.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/boehm12file.ppt.pdf\",\r\n  abstract      = \"As multi-petascale and exa-scale high-performance computing\r\n                   (HPC) systems inevitably have to deal with a number of\r\n                   resilience challenges, such as a significant growth in\r\n                   component count and smaller circuit sizes with lower circuit\r\n                   voltages, redundancy may offer an acceptable level of\r\n                   resilience that traditional fault tolerance techniques, such\r\n                   as checkpoint\/restart, do not. Although redundancy in HPC is\r\n                   quite controversial due to the associated cost for redundant\r\n                   components,  the constantly increasing number of\r\n                   cores-per-processor is tilting this cost calculation toward\r\n                   a system design where computation, such as for redundancy,\r\n                   is much cheaper and communication, needed for\r\n                   checkpoint\/restart, is much more expensive. Recent research\r\n                   and development activities in redundancy for Message Passing\r\n                   Interface (MPI) applications focused on\r\n                   availability\/reliability models and replication algorithms.\r\n                   This paper takes a first step toward solving an open research\r\n                   problem associated with running a parallel application\r\n                   redundantly, which is file I\/O under redundancy. The\r\n                   approach intercepts file I\/O calls made by a redundant\r\n                   application to employ coordination protocols that execute\r\n                   file I\/O operations in a redundancy-oblivious fashion when\r\n                   accessing a node-local file system, or in a redundancy-aware\r\n                   fashion when accessing a shared networked file system.\r\n                   A proof-of concept prototype is presented and a number of\r\n                   coordination protocols are described and evaluated. The\r\n                   results show the performance impact for redundantly accessing\r\n                   a shared networked file system, but also demonstrate the\r\n                   capability to regain performance by utilizing MPI\r\n                   communication between replicas and parallel file I\/O.\",\r\n  pts           = \"33577\"\r\n}<\/pre>\n
<\/a>@conference{boehm11xsim,\r\n  author        = \"Swen B{\\\"o}hm\r\n                   and Christian Engelmann\",\r\n  title         = \"{xSim}: {The} Extreme-Scale Simulator\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/hpcs11.cisedu.info}{International Conference on\r\n                   High Performance Computing and Simulation (HPCS) 2011}\",\r\n  pages         = \"280-286\",\r\n  month         = jul # \"~4-8, \",\r\n  year          = \"2011\",\r\n  address       = \"Istanbul, Turkey\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-1-61284-383-4\",\r\n  doi           = \"10.1109\/HPCSim.2011.5999835\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/boehm11xsim.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/boehm11xsim.ppt.pdf\",\r\n  abstract      = \"Investigating parallel application performance properties at\r\n                   scale is becoming an important part of high-performance\r\n                   computing (HPC) application development and deployment. The\r\n                   Extreme-scale Simulator (xSim) is a performance investigation\r\n                   toolkit that permits running an application in a controlled\r\n                   environment at extreme scale without the need for a\r\n                   respective extreme-scale HPC system. Using a lightweight\r\n                   parallel discrete event simulation, xSim executes a parallel\r\n                   application with a virtual wall clock time, such that\r\n                   performance data can be extracted based on a processor model\r\n                   and a network model. This paper presents significant\r\n                   enhancements to the xSim toolkit prototype that provide a\r\n                   more complete Message Passing Interface (MPI) support and\r\n                   improve its versatility. These enhancements include full\r\n                   virtual MPI group, communicator and collective communication\r\n                   support, and global variables support. The new capabilities\r\n                   are demonstrated by executing the entire NAS Parallel\r\n                   Benchmark suite in a simulated HPC environment.\",\r\n  pts           = \"29960\"\r\n}<\/pre>\n
<\/a>@conference{engelmann11redundant,\r\n  author        = \"Christian Engelmann\r\n                   and Swen B{\\\"o}hm\",\r\n  title         = \"Redundant Execution of {HPC} Applications with {MR-MPI}\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.iasted.org\/conferences\/home-719.html}\r\n                   {$10^{th}$ IASTED International Conference on Parallel and\r\n                   Distributed Computing and Networks (PDCN) 2011}\",\r\n  pages         = \"31--38\",\r\n  month         = feb # \"~15-17, \",\r\n  year          = \"2011\",\r\n  address       = \"Innsbruck, Austria\",\r\n  publisher     = \"\\href{http:\/\/www.actapress.com}{ACTA Press, Calgary, AB,\r\n                   Canada}\",\r\n  isbn          = \"978-0-88986-864-9\",\r\n  doi           = \"10.2316\/P.2011.719-031\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann11redundant.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann11redundant.ppt.pdf\",\r\n  abstract      = \"This paper presents a modular-redundant Message Passing\r\n                   Interface (MPI) solution, MR-MPI, for transparently executing \r\n                   high-performance computing (HPC) applications in a redundant\r\n                   fashion. The presented work addresses the deficiencies of\r\n                   recovery-oriented HPC, i.e., checkpoint\/restart to\/from a\r\n                   parallel file system, at extreme scale by adding the\r\n                   redundancy approach to the HPC resilience portfolio. It\r\n                   utilizes the MPI performance tool interface, PMPI, to\r\n                   transparently intercept MPI calls from an application and to\r\n                   hide all redundancy-related mechanisms. A redundantly\r\n                   executed application runs with $r*m$ native MPI processes,\r\n                   where $r$ is the number of MPI ranks visible to the\r\n                   application and $m$ is the replication degree. Messages\r\n                   between redundant nodes are replicated. Partial replication\r\n                   for tunable resilience is supported. The performance results\r\n                   clearly show the negative impact of the O(m^2) messages\r\n                   between replicas. For low-level, point-to-point benchmarks,\r\n                   the impact can be as high as the replication degree. For\r\n                   applications, performance highly depends on the actual\r\n                   communication types and counts. On single-core systems, the\r\n                   overhead can be 0\\% for embarrassingly parallel applications\r\n                   independent of the employed redundancy configuration or up\r\n                   to 70-90\\% for communication-intensive applications in a\r\n                   dual-redundant configuration. On multi-core systems, the\r\n                   overhead can be significantly higher due to the additional\r\n                   communication contention.\",\r\n  pts           = \"27623\"\r\n}<\/pre>\n
<\/a>@conference{wang10hybrid2,\r\n  author        = \"Chao Wang\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"Hybrid Checkpointing for {MPI} Jobs in {HPC} Environments\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/grid.sjtu.edu.cn\/icpads10}{$16^{th}$ IEEE\r\n                   International Conference on Parallel and Distributed Systems\r\n                   (ICPADS) 2010}\",\r\n  pages         = \"524--533\",\r\n  month         = dec # \"~8-10, \",\r\n  year          = \"2010\",\r\n  address       = \"Shanghai, China\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7695-4307-9\",\r\n  doi           = \"10.1109\/ICPADS.2010.48\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/wang10hybrid2.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/wang10hybrid2.ppt.pdf\",\r\n  abstract      = \"As the core count in high-performance computing systems keeps\r\n                   increasing, faults are becoming common place. Check pointing\r\n                   addresses such faults but captures full process images even\r\n                   though only a subset of the process image changes between\r\n                   checkpoints. We have designed a hybrid check pointing\r\n                   technique for MPI tasks of high-performance applications.\r\n                   This technique alternates between full and incremental\r\n                   checkpoints: At incremental checkpoints, only data changed\r\n                   since the last checkpoint is captured. Our implementation\r\n                   integrates new BLCR and LAM\/MPI features that complement\r\n                   traditional full checkpoints. This results in significantly\r\n                   reduced checkpoint sizes and overheads with only moderate\r\n                   increases in restart overhead. After accounting for cost and\r\n                   savings, benefits due to incremental checkpoints are an order\r\n                   of magnitude larger than overheads on restarts. We further\r\n                   derive qualitative results indicating an optimal balance\r\n                   between full\/incremental checkpoints of our novel approach at\r\n                   a ratio of 1:9, which outperforms both always-full and\r\n                   always-incremental check pointing.\",\r\n  pts           = \"25447\"\r\n}<\/pre>\n
<\/a>@conference{li10functional,\r\n  author        = \"Min Li\r\n                   and Sudharshan S. Vazhkudai\r\n                   and Ali R. Butt\r\n                   and Fei Meng\r\n                   and Xiaosong Ma\r\n                   and Youngjae Kim\r\n                   and Christian Engelmann\r\n                   and Galen Shipman\",\r\n  title         = \"Functional Partitioning to Optimize End-to-End Performance on\r\n                   Many-Core Architectures\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/sc10.supercomputing.org}{$23^{rd}$ IEEE\/ACM\r\n                   International Conference on High Performance Computing,\r\n                   Networking, Storage and Analysis (SC) 2010}\",\r\n  pages         = \"1-12\",\r\n  month         = nov # \"~13-19, \",\r\n  year          = \"2010\",\r\n  address       = \"New Orleans, LA, USA\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-4244-7559-9\",\r\n  doi           = \"10.1109\/SC.2010.28\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/li10functional.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/li10functional.ppt.pdf\",\r\n  abstract      = \"Scaling computations on emerging massive-core supercomputers\r\n                   is a daunting task, which coupled with the significantly\r\n                   lagging system I\/O capabilities exacerbates applications'\r\n                   end-to-end performance. The I\/O bottleneck often negates\r\n                   potential performance benefits of assigning additional\r\n                   compute cores to an application. In this paper, we address\r\n                   this issue via a novel functional partitioning (FP) runtime\r\n                   environment that allocates cores to specific application\r\n                   tasks - checkpointing, de-duplication, and scientific data\r\n                   format transformation - so that the deluge of cores can be\r\n                   brought to bear on the entire gamut of application\r\n                   activities. The focus is on utilizing the extra cores to\r\n                   support HPC application I\/O activities and also leverage\r\n                   solid-state disks in this context. For example, our\r\n                   evaluation shows that dedicating 1 core on an oct-core\r\n                   machine for checkpointing and its assist tasks using FP can\r\n                   improve overall execution time of a FLASH benchmark on 80 and \r\n                   160 cores by 43.95\\% and 41.34\\%, respectively.\",\r\n  pts           = \"24996\"\r\n}<\/pre>\n
<\/a>@conference{boehm10aggregation,\r\n  author        = \"Swen B{\\\"o}hm\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"Aggregation of Real-Time System Monitoring Data for Analyzing\r\n                   Large-Scale Parallel and Distributed Computing Environments\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.anss.org.au\/hpcc2010}\r\n                   {$12^{th}$ IEEE International Conference on High Performance\r\n                   Computing and Communications (HPCC) 2010}\",\r\n  pages         = \"72--78\",\r\n  month         = sep # \"~1-3, \",\r\n  year          = \"2010\",\r\n  address       = \"Melbourne, Australia\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7695-4214-0\",\r\n  doi           = \"10.1109\/HPCC.2010.32\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/boehm10aggregation.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/boehm10aggregation.ppt.pdf\",\r\n  abstract      = \"We present a monitoring system for large-scale parallel and\r\n                   distributed computing environments that allows to trade-off\r\n                   accuracy in a tunable fashion to gain scalability without\r\n                   compromising fidelity. The approach relies on classifying\r\n                   each gathered monitoring metric based on individual needs\r\n                   and on aggregating messages containing classes of individual\r\n                   monitoring metrics using a tree-based overlay network. The\r\n                   MRNet-based prototype is able to significantly reduce the\r\n                   amount of gathered and stored monitoring data, e.g., by a\r\n                   factor of ~56 in comparison to the Ganglia distributed\r\n                   monitoring system. A simple scaling study reveals, however,\r\n                   that further efforts are needed in reducing the amount of\r\n                   data to monitor future-generation extreme-scale systems with\r\n                   up to 1,000,000 nodes. The implemented solution did not had\r\n                   a measurable performance impact as the 32-node test system\r\n                   did not produce enough monitoring data to interfere with\r\n                   running applications.\",\r\n  pts           = \"24907\"\r\n}<\/pre>\n
<\/a>@conference{litvinova10proactive,\r\n  author        = \"Antonina Litvinova\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"A Proactive Fault Tolerance Framework for High-Performance\r\n                   Computing\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.iasted.org\/conferences\/home-676.html}\r\n                   {$9^{th}$ IASTED International Conference on Parallel and\r\n                   Distributed Computing and Networks (PDCN) 2010}\",\r\n  pages         = \"\",\r\n  month         = feb # \"~16-18, \",\r\n  year          = \"2010\",\r\n  address       = \"Innsbruck, Austria\",\r\n  publisher     = \"\\href{http:\/\/www.actapress.com}{ACTA Press, Calgary, AB,\r\n                   Canada}\",\r\n  isbn          = \"978-0-88986-783-3\",\r\n  doi           = \"10.2316\/P.2010.676-024\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/litvinova10proactive.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/litvinova10proactive.ppt.pdf\",\r\n  abstract      = \"As high-performance computing (HPC) systems continue to\r\n                   increase in scale, their mean-time to interrupt decreases\r\n                   respectively. The current state of practice for fault\r\n                   tolerance (FT) is checkpoint\/restart. However, with\r\n                   increasing error rates, increasing aggregate memory and not\r\n                   proportionally increasing I\/O capabilities, it is becoming\r\n                   less efficient. Proactive FT avoids experiencing failures\r\n                   through preventative measures, such as by migrating\r\n                   application parts away from nodes that are about to fail.\r\n                   This paper presents a proactive FT framework that performs\r\n                   environmental monitoring, event logging, parallel job\r\n                   monitoring and resource monitoring to analyze HPC system\r\n                   reliability and to perform FT through such preventative\r\n                   actions.\",\r\n  pts           = \"13674\"\r\n}<\/pre>\n
<\/a>@conference{taerat09blue,\r\n  author        = \"Narate Taerat\r\n                   and Nichamon Naksinehaboon\r\n                   and Clayton Chandler\r\n                   and James Elliott\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and George Ostrouchov\r\n                   and Stephen L. Scott\r\n                   and Christian Engelmann\",\r\n  title         = \"{Blue Gene\/L} Log Analysis and Time to Interrupt Estimation\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.ares-conference.eu\/ares2009}{$4^{th}$\r\n                   International Conference on Availability, Reliability and\r\n                   Security (ARES) 2009}\",\r\n  pages         = \"173--180\",\r\n  month         = mar # \"~16-19, \",\r\n  year          = \"2009\",\r\n  address       = \"Fukuoka, Japan\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-1-4244-3572-2\",\r\n  doi           = \"10.1109\/ARES.2009.105\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/taerat09blue.pdf\",\r\n  url2          = \"\",\r\n  abstract      = \"System- and application-level failures could be characterized\r\n                   by analyzing relevant log files. The resulting data might\r\n                   then be used in numerous studies on and future developments\r\n                   for the mission-critical and large scale computational\r\n                   architecture, including fields such as failure prediction,\r\n                   reliability modeling, performance modeling and power\r\n                   awareness. In this paper, system logs covering a six month\r\n                   period of the Blue Gene\/L supercomputer were obtained and\r\n                   subsequently analyzed. Temporal filtering was applied to\r\n                   remove duplicated log messages. Optimistic and pessimistic\r\n                   perspectives were exerted on filtered log information to\r\n                   observe failure behavior within the system. Further, various\r\n                   time to repair factors were applied to obtain application\r\n                   time to interrupt, which will be exploited in further\r\n                   resilience modeling research.\"\r\n}<\/pre>\n
<\/a>@conference{engelmann09evaluating,\r\n  author        = \"Christian Engelmann\r\n                   and Hong H. Ong\r\n                   and Stephen L. Scott\",\r\n  title         = \"Evaluating the Shared Root File System Approach for Diskless\r\n                   High-Performance Computing Systems\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.linuxclustersinstitute.org\/conferences}\r\n                   {$10^{th}$ LCI International Conference on High-Performance\r\n                   Clustered Computing (LCI) 2009}\",\r\n  month         = mar # \"~9-12, \",\r\n  year          = \"2009\",\r\n  address       = \"Boulder, CO, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann09evaluating.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann09evaluating.ppt.pdf\",\r\n  abstract      = \"Diskless high-performance computing (HPC) systems utilizing\r\n                   networked storage have become popular in the last several\r\n                   years. Removing disk drives significantly increases compute\r\n                   node reliability as they are known to be a major source of\r\n                   failures. Furthermore, networked storage solutions utilizing\r\n                   parallel I\/O and replication are able to provide increased\r\n                   scalability and availability. Reducing a compute node to\r\n                   processor(s), memory and network interface(s) greatly reduces\r\n                   its physical size, which in turn allows for large-scale dense\r\n                   HPC solutions. However, one major obstacle is the requirement\r\n                   by certain operating systems (OSs), such as Linux, for a root\r\n                   file system. While one solution is to remove this requirement\r\n                   from the OS, another is to share the root file system over\r\n                   the networked storage. This paper evaluates three networked\r\n                   file system solutions, NFSv4, Lustre and PVFS2, with respect\r\n                   to their performance, scalability, and availability features\r\n                   for servicing a common root file system in a diskless HPC\r\n                   configuration. Our findings indicate that Lustre is a viable\r\n                   solution as it meets both, scaling and performance\r\n                   requirements. However, certain availability issues regarding\r\n                   single points of failure and control need to be considered.\",\r\n  pts           = \"14025\"\r\n}<\/pre>\n
<\/a>@conference{engelmann09proactive,\r\n  author        = \"Christian Engelmann\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Stephen L. Scott\",\r\n  title         = \"Proactive Fault Tolerance Using Preemptive Migration\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.pdp2009.org}{$17^{th}$\r\n                   Euromicro International Conference on Parallel, Distributed,\r\n                   and network-based Processing (PDP) 2009}\",\r\n  pages         = \"252--257\",\r\n  month         = feb # \"~18-20, \",\r\n  year          = \"2009\",\r\n  address       = \"Weimar, Germany\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7695-3544-9\",\r\n  issn          = \"1066-6192\",\r\n  doi           = \"10.1109\/PDP.2009.31\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann09proactive.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann09proactive.ppt.pdf\",\r\n  abstract      = \"Proactive fault tolerance (FT) in high-performance computing\r\n                   is a concept that prevents compute node failures from\r\n                   impacting running parallel applications by preemptively\r\n                   migrating application parts away from nodes that are about\r\n                   to fail. This paper provides a foundation for proactive FT by\r\n                   defining its architecture and classifying implementation\r\n                   options. This paper further relates prior work to the\r\n                   presented architecture and classification, and discusses the\r\n                   challenges ahead for needed supporting technologies.\",\r\n  pts           = \"13674\"\r\n}<\/pre>\n
<\/a>@conference{valentini09high,\r\n  author        = \"Alessandro Valentini\r\n                   and Christian Di Biagio\r\n                   and Fabrizio Batino\r\n                   and Guido Pennella\r\n                   and Fabrizio Palma\r\n                   and Christian Engelmann\",\r\n  title         = \"High Performance Computing with {Harness} over {InfiniBand}\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.pdp2009.org}{$17^{th}$\r\n                   Euromicro International Conference on Parallel, Distributed,\r\n                   and network-based Processing (PDP) 2009}\",\r\n  pages         = \"151--154\",\r\n  month         = feb # \"~18-20, \",\r\n  year          = \"2009\",\r\n  address       = \"Weimar, Germany\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7695-3544-9\",\r\n  issn          = \"1066-6192\",\r\n  doi           = \"10.1109\/PDP.2009.64\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/valentini09high.pdf\",\r\n  abstract      = \"Harness is an adaptable and plug-in-based middleware\r\n                   framework able to support distributed parallel computing. By\r\n                   now, it is based on the Ethernet protocol which cannot\r\n                   guarantee high performance throughput and Real Time\r\n                   (determinism) performance. During last years, both the\r\n                   research and industry environments have developed both new\r\n                   network architectures (InfiniBand, Myrinet, iWARP, etc.) to\r\n                   avoid those limits. This paper concerns the integration\r\n                   between Harness and InfiniBand focusing on two solutions: IP\r\n                   over InfiniBand (IPoIB) and Socket Direct Protocol (SDP)\r\n                   technology. Those allow Harness middleware to take advantage\r\n                   of the enhanced features provided by InfiniBand.\",\r\n  pts           = \"14107\"\r\n}<\/pre>\n
<\/a>@conference{engelmann09case,\r\n  author        = \"Christian Engelmann\r\n                   and Hong H. Ong\r\n                   and Stephen L. Scott\",\r\n  title         = \"The Case for Modular Redundancy in Large-Scale High\r\n                   Performance Computing Systems\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.iasted.org\/conferences\/home-641.html}\r\n                   {$8^{th}$ IASTED International Conference on Parallel and\r\n                   Distributed Computing and Networks (PDCN) 2009}\",\r\n  pages         = \"189--194\",\r\n  month         = feb # \"~16-18, \",\r\n  year          = \"2009\",\r\n  address       = \"Innsbruck, Austria\",\r\n  publisher     = \"\\href{http:\/\/www.actapress.com}{ACTA Press, Calgary, AB,\r\n                   Canada}\",\r\n  isbn          = \"978-0-88986-784-0\",\r\n  doi           = \"\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann09case.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann09case.ppt.pdf\",\r\n  abstract      = \"Recent investigations into resilience of large-scale\r\n                   high-performance computing (HPC) systems showed a continuous\r\n                   trend of decreasing reliability and availability. Newly\r\n                   installed systems have a lower mean-time to failure (MTTF)\r\n                   and a higher mean-time to recover (MTTR) than their\r\n                   predecessors. Modular redundancy is being used in many\r\n                   mission critical systems today to provide for resilience,\r\n                   such as for aerospace and command & control systems. The\r\n                   primary argument against modular redundancy for resilience\r\n                   in HPC has always been that the capability of a HPC system,\r\n                   and respective return on investment, would be significantly\r\n                   reduced. We argue that modular redundancy can significantly\r\n                   increase compute node availability as it removes the impact\r\n                   of scale from single compute node MTTR. We further argue that\r\n                   single compute nodes can be much less reliable, and therefore\r\n                   less expensive, and still be highly available, if their\r\n                   MTTR\/MTTF ratio is maintained.\",\r\n  pts           = \"13981\"\r\n}<\/pre>\n
<\/a>@conference{wang08proactive,\r\n  author        = \"Chao Wang\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"Proactive Process-Level Live Migration in {HPC}\r\n                   Environments\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/sc08.supercomputing.org}\r\n                   {$21^{st}$ IEEE\/ACM International Conference on High\r\n                   Performance Computing, Networking, Storage and Analysis (SC)\r\n                   2008}\",\r\n  pages         = \"1--12\",\r\n  month         = nov # \"~15-21, \",\r\n  year          = \"2008\",\r\n  address       = \"Austin, TX, USA\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-4244-2835-9\",\r\n  doi           = \"10.1145\/1413370.1413414\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/wang08proactive.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/wang08proactive.ppt.pdf\",\r\n  abstract      = \"As the number of nodes in high-performance computing\r\n                   environments keeps increasing, faults are becoming common\r\n                   place. Reactive fault tolerance (FT) often does not scale due\r\n                   to massive I\/O requirements and relies on manual job\r\n                   resubmission. This work complements reactive with proactive\r\n                   FT at the process level. Through health monitoring, a subset\r\n                   of node failures can be anticipated when one's health\r\n                   deteriorates. A novel process-level live migration mechanism\r\n                   supports continued execution of applications during much of\r\n                   processes migration. This scheme is integrated into an MPI\r\n                   execution environment to transparently sustain\r\n                   health-inflicted node failures, which eradicates the need to\r\n                   restart and requeue MPI jobs. Experiments indicate that 1-6.5\r\n                   seconds of prior warning are required to successfully trigger\r\n                   live process migration while similar operating system\r\n                   virtualization mechanisms require 13-24 seconds. This\r\n                   self-healing approach complements reactive FT by nearly\r\n                   cutting the number of checkpoints in half when 70\\% of the\r\n                   faults are handled proactively.\",\r\n  pts           = \"12052\"\r\n}<\/pre>\n
<\/a>@conference{engelmann08symmetric,\r\n  author        = \"Christian Engelmann\r\n                   and Stephen L. Scott\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and Xubin (Ben) He\",\r\n  title         = \"Symmetric Active\/Active Replication for Dependent Services\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.ares-conference.eu\/ares2008}{$3^{rd}$\r\n                   International Conference on Availability, Reliability and\r\n                   Security (ARES) 2008}\",\r\n  pages         = \"260--267\",\r\n  month         = mar # \"~4-7, \",\r\n  year          = \"2008\",\r\n  address       = \"Barcelona, Spain\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7695-3102-1\",\r\n  doi           = \"10.1109\/ARES.2008.64\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann08symmetric.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann08symmetric.ppt.pdf\",\r\n  abstract      = \"During the last several years, we have established the\r\n                   symmetric active\/active replication model for service-level\r\n                   high availability and implemented several proof-of-concept\r\n                   prototypes. One major deficiency of our model is its\r\n                   inability to deal with dependent services, since its original\r\n                   architecture is based on the client-service model. This paper\r\n                   extends our model to dependent services using its already\r\n                   existing mechanisms and features. The presented concept is\r\n                   based on the idea that a service may also be a client of\r\n                   another service, and multiple services may be clients of each\r\n                   other. A high-level abstraction is used to illustrate\r\n                   dependencies between clients and services, and to decompose\r\n                   dependencies between services into respective client-service\r\n                   dependencies. This abstraction may be used for providing\r\n                   high availability in distributed computing systems with\r\n                   complex service-oriented architectures.\",\r\n  pts           = \"9456\"\r\n}<\/pre>\n
<\/a>@conference{vallee08framework,\r\n  author        = \"Geoffroy R. Vall\\'ee\r\n                   and Kulathep Charoenpornwattana\r\n                   and Christian Engelmann\r\n                   and Anand Tikotekar\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and Thomas Naughton\r\n                   and Stephen L. Scott\",\r\n  title         = \"A Framework For Proactive Fault Tolerance\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.ares-conference.eu\/ares2008}{$3^{rd}$\r\n                   International Conference on Availability, Reliability and\r\n                   Security (ARES) 2008}\",\r\n  pages         = \"659--664\",\r\n  month         = mar # \"~4-7, \",\r\n  year          = \"2008\",\r\n  address       = \"Barcelona, Spain\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7695-3102-1\",\r\n  doi           = \"10.1109\/ARES.2008.171\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/vallee08framework.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/vallee08framework.ppt.pdf\",\r\n  abstract      = \"Fault tolerance is a major concern to guarantee availability\r\n                   of critical services as well as application execution.\r\n                   Traditional approaches for fault tolerance include\r\n                   checkpoint\/restart or duplication. However it is also\r\n                   possible to anticipate failures and proactively take action\r\n                   before failures occur in order to minimize failure impact on\r\n                   the system and application execution. This document presents\r\n                   a proactive fault tolerance framework. This framework can use\r\n                   different proactive fault tolerance mechanisms, i.e.\r\n                   migration and pause\/unpause. The framework also allows the\r\n                   implementation of new proactive fault tolerance policies\r\n                   thanks to a modular architecture. A first proactive fault\r\n                   tolerance policy has been implemented and preliminary\r\n                   experimentations have been done based on system-level\r\n                   virtualization and compared with results obtained by\r\n                   simulation.\"\r\n}<\/pre>\n
<\/a>@conference{koenning08virtualized,\r\n  author        = \"Bj{\\\"o}rn K{\\\"o}nning\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\r\n                   and George A. (Al) Geist\",\r\n  title         = \"Virtualized Environments for the {Harness} High Performance\r\n                   Computing Workbench\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.pdp2008.org}{$16^{th}$\r\n                   Euromicro International Conference on Parallel, Distributed,\r\n                   and network-based Processing (PDP) 2008}\",\r\n  pages         = \"133--140\",\r\n  month         = feb # \"~13-15, \",\r\n  year          = \"2008\",\r\n  address       = \"Toulouse, France\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7695-3089-5\",\r\n  doi           = \"10.1109\/PDP.2008.14\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/koenning08virtualized.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/koenning08virtualized.ppt.pdf\",\r\n  abstract      = \"This paper describes recent accomplishments in providing a\r\n                   virtualized environment concept and prototype for scientific\r\n                   application development and deployment as part of the Harness\r\n                   High Performance Computing (HPC) Workbench research effort.\r\n                   The presented work focuses on tools and mechanisms that\r\n                   simplify scientific application development and deployment\r\n                   tasks, such that only minimal adaptation is needed when\r\n                   moving from one HPC system to another or after HPC system\r\n                   upgrades. The overall technical approach focuses on the\r\n                   concept of adapting the HPC system environment to the actual\r\n                   needs of individual scientific applications instead of the\r\n                   traditional scheme of adapting scientific applications to\r\n                   individual HPC system environment properties. The presented\r\n                   prototype implementation is based on the mature and\r\n                   lightweight chroot virtualization approach for Unix-type\r\n                   systems with a focus on virtualized file system structure\r\n                   and virtualized shell environment variables utilizing\r\n                   virtualized environment configuration descriptions in\r\n                   Extensible Markup Language (XML) format. The presented work\r\n                   can be easily extended to other virtualization technologies,\r\n                   such as system-level virtualization solutions using\r\n                   hypervisors.\",\r\n  pts           = \"11532\"\r\n}<\/pre>\n
<\/a>@conference{vallee08system,\r\n  author        = \"Geoffroy R. Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Christian Engelmann\r\n                   and Hong H. Ong\r\n                   and Stephen L. Scott\",\r\n  title         = \"System-level Virtualization for High Performance Computing\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.pdp2008.org}{$16^{th}$\r\n                   Euromicro International Conference on Parallel, Distributed,\r\n                   and network-based Processing (PDP) 2008}\",\r\n  pages         = \"636--643\",\r\n  month         = feb # \"~13-15, \",\r\n  year          = \"2008\",\r\n  address       = \"Toulouse, France\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7695-3089-5\",\r\n  doi           = \"10.1109\/PDP.2008.85\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/vallee08system.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/vallee08system.ppt.pdf\",\r\n  abstract      = \"System-level virtualization has been a research topic since\r\n                   the 70`s but regained popularity during the past few years\r\n                   because of the availability of efficient solution such as Xen\r\n                   and the implementation of hardware support in commodity\r\n                   processors (e.g. Intel-VT, AMD-V). However, a majority of\r\n                   system-level virtualization projects is guided by the server\r\n                   consolidation market. As a result, current virtualization\r\n                   solutions appear to not be suitable for high performance\r\n                   computing (HPC) which is typically based on large-scale\r\n                   systems. On another hand there is significant interest in\r\n                   exploiting virtual machines (VMs) within HPC for a number of\r\n                   other reasons. By virtualizing the machine, one is able to\r\n                   run a variety of operating systems and environments as needed\r\n                   by the applications. Virtualization allows users to isolate\r\n                   workloads, improving security and reliability. It is also\r\n                   possible to support non-native environments and\/or legacy\r\n                   operating environments through virtualization. In addition,\r\n                   it is possible to balance work loads, use migration\r\n                   techniques to relocate applications from failing machines,\r\n                   and isolate fault systems for repair. This document presents\r\n                   the challenges for the implementation of a system-level\r\n                   virtualization solution for HPC. It also presents a brief\r\n                   survey of the different approaches and techniques to address\r\n                   these challenges.\",\r\n  pts           = \"11137\"\r\n}<\/pre>\n
<\/a>@conference{ou07symmetric,\r\n  author        = \"Li Ou\r\n                   and Christian Engelmann\r\n                   and Xubin (Ben) He\r\n                   and Xin Chen\r\n                   and Stephen L. Scott\",\r\n  title         = \"Symmetric Active\/Active Metadata Service for Highly Available\r\n                   Cluster Storage Systems\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.iasted.org\/conferences\/home-590.html}\r\n                   {$19^{th}$ IASTED International Conference on Parallel and\r\n                   Distributed Computing and Systems (PDCS) 2007}\",\r\n  pages         = \"\",\r\n  month         = nov # \"~19-21, \",\r\n  year          = \"2007\",\r\n  address       = \"Cambridge, MA, USA\",\r\n  publisher     = \"\\href{http:\/\/www.actapress.com}{ACTA Press, Calgary, AB,\r\n                   Canada}\",\r\n  isbn          = \"978-0-88986-703-1\",\r\n  doi           = \"\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/ou07symmetric.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/ou07symmetric.ppt.pdf\",\r\n  abstract      = \"In a typical distributed storage system, metadata is stored\r\n                   and managed by dedicated metadata servers. One way to improve\r\n                   the availability of distributed storage systems is to deploy\r\n                   multiple metadata servers. Past research focused on the\r\n                   active\/standby model, where each active server has at least\r\n                   one redundant idle backup. However, interruption of service\r\n                   and loss of service state may occur during a fail-over\r\n                   depending on the used replication technique. The research in\r\n                   this paper targets the symmetric active\/active replication\r\n                   model using multiple redundant service nodes running in\r\n                   virtual synchrony. In this model, service node failures do\r\n                   not cause a fail-over to a backup and there is no disruption\r\n                   of service or loss of service state. We propose a fast\r\n                   delivery protocol to reduce the latency of total order\r\n                   broadcast. Our prototype implementation shows that high\r\n                   availability of metadata servers can be achieved with an\r\n                   acceptable performance trade-off using the active\/active\r\n                   metadata server solution.\",\r\n  pts           = \"8335\"\r\n}<\/pre>\n
<\/a>@conference{disaverio07distributed,\r\n  author        = \"Emanuele Di Saverio\r\n                   and Marco Cesati\r\n                   and Christian Di Biagio\r\n                   and Guido Pennella\r\n                   and Christian Engelmann\",\r\n  title         = \"Distributed Real-Time Computing with {Harness}\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{http:\/\/pvmmpi07.lri.fr}{$14^{th}$ European PVM\/MPI\r\n                   Users` Group Meeting (EuroPVM\/MPI) 2007}\",\r\n  pages         = \"281--288\",\r\n  volume        = \"4757\",\r\n  month         = sep # \"~30 - \" # oct # \"~3, \",\r\n  year          = \"2007\",\r\n  address       = \"Paris, France\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-540-75415-2\",\r\n  issn          = \"0302-9743\",\r\n  doi           = \"10.1007\/978-3-540-75416-9_39\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/disaverio07distributed.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/disaverio07distributed.ppt.pdf\",\r\n  abstract      = \"Modern parallel and distributed computing solutions are often\r\n                   built onto a middleware software layer providing a higher\r\n                   and common level of service between computational nodes.\r\n                   Harness is an adaptable, plugin-based middleware framework\r\n                   for parallel and distributed computing. This paper reports\r\n                   recent research and development results of using Harness for\r\n                   real-time distributed computing applications in the context\r\n                   of an industrial environment with the needs to perform\r\n                   several safety critical tasks. The presented work exploits\r\n                   the modular architecture of Harness in conjunction with a\r\n                   lightweight threaded implementation to resolve several\r\n                   real-time issues by adding three new Harness plug-ins to\r\n                   provide a prioritized lightweight execution environment, low\r\n                   latency communication facilities, and local timestamped event\r\n                   logging.\",\r\n  pts           = \"7023\"\r\n}<\/pre>\n
<\/a>@conference{ou07fast,\r\n  author        = \"Li Ou\r\n                   and Xubin (Ben) He\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"A Fast Delivery Protocol for Total Order Broadcasting\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.icccn.org\/icccn07}\r\n                   {$16^{th}$ IEEE International Conference on Computer\r\n                   Communications and Networks (ICCCN) 2007}\",\r\n  pages         = \"730--734\",\r\n  month         = aug # \"~13-16, \",\r\n  year          = \"2007\",\r\n  address       = \"Honolulu, HI, USA\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-1-42441-251-8\",\r\n  issn          = \"1095-2055\",\r\n  doi           = \"10.1109\/ICCCN.2007.4317904\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/ou07fast.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/ou07fast.ppt.pdf\",\r\n  abstract      = \"Sequencer, privilege-based, and communication history\r\n                   algorithms are popular approaches to implement total\r\n                   ordering, where communication history algorithms are most\r\n                   suitable for parallel computing systems, because they provide\r\n                   best performance under heavy work load. Unfortunately,\r\n                   post-transmission delay of communication history algorithms\r\n                   is most apparent when a system is idle. In this paper, we\r\n                   propose a fast delivery protocol to reduce the latency of\r\n                   message ordering. The protocol optimizes the total ordering\r\n                   process by waiting for messages only from a subset of the\r\n                   machines in the group, and by fast acknowledging messages on\r\n                   behalf of other machines. Our test results indicate that the\r\n                   fast delivery protocol is suitable for both idle and heavy\r\n                   load systems, while reducing the latency of message\r\n                   ordering.\",\r\n  pts           = \"6926\"\r\n}<\/pre>\n
<\/a>@conference{nagarajan07proactive,\r\n  author        = \"Arun B. Nagarajan\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"Proactive Fault Tolerance for {HPC} with {Xen}\r\n                   Virtualization\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/ics07.ac.upc.edu}{$21^{st}$\r\n                   ACM International Conference on Supercomputing (ICS) 2007}\",\r\n  pages         = \"23--32\",\r\n  month         = jun # \"~16-20, \",\r\n  year          = \"2007\",\r\n  address       = \"Seattle, WA, USA\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-59593-768-1\",\r\n  doi           = \"10.1145\/1274971.1274978\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/nagarajan07proactive.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/nagarajan07proactive.ppt.pdf\",\r\n  abstract      = \"Large-scale parallel computing is relying increasingly on\r\n                   clusters with thousands of processors. At such large counts\r\n                   of compute nodes, faults are becoming common place. Current\r\n                   techniques to tolerate faults focus on reactive schemes to\r\n                   recover from faults and generally rely on a\r\n                   checkpoint\/restart mechanism. Yet, in today`s systems, node\r\n                   failures can often be anticipated by detecting a\r\n                   deteriorating health status. Instead of a reactive scheme for\r\n                   fault tolerance (FT), we are promoting a proactive one where\r\n                   processes automatically migrate from unhealthy nodes to\r\n                   healthy ones. Our approach relies on operating system\r\n                   virtualization techniques exemplified by but not limited to\r\n                   Xen. This paper contributes an automatic and transparent\r\n                   mechanism for proactive FT for arbitrary MPI applications.\r\n                   It leverages virtualization techniques combined with health\r\n                   monitoring and load-based migration. We exploit Xen`s live\r\n                   migration mechanism for a guest operating system (OS) to\r\n                   migrate an MPI task from a health-deteriorating node to a\r\n                   healthy one without stopping the MPI task during most of the\r\n                   migration. Our proactive FT daemon orchestrates the tasks of\r\n                   health monitoring, load determination and initiation of guest\r\n                   OS migration. Experimental results demonstrate that live\r\n                   migration hides migration costs and limits the overhead to\r\n                   only a few seconds making it an attractive approach to\r\n                   realize FT in HPC systems. Overall, our enhancements make\r\n                   proactive FT a valuable asset for long-running MPI\r\n                   application that is complementary to reactive FT using full\r\n                   checkpoint\/restart schemes since checkpoint frequencies can\r\n                   be reduced as fewer unanticipated failures are encountered.\r\n                   In the context of OS virtualization, we believe that this is\r\n                   the first comprehensive study of proactive fault tolerance\r\n                   where live migration is actually triggered by health\r\n                   monitoring.\",\r\n  pts           = \"6489\"\r\n}<\/pre>\n
<\/a>@conference{engelmann07programming,\r\n  author        = \"Christian Engelmann\r\n                   and Stephen L. Scott\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and Xubin (Ben) He\",\r\n  title         = \"On Programming Models for Service-Level High Availability\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.ares-conference.eu\/ares2007}{$2^{nd}$\r\n                   International Conference on Availability, Reliability and\r\n                   Security (ARES) 2007}\",\r\n  pages         = \"999--1006\",\r\n  month         = apr # \"~10-13, \",\r\n  year          = \"2007\",\r\n  address       = \"Vienna, Austria\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"0-7695-2775-2\",\r\n  doi           = \"10.1109\/ARES.2007.109\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann07programming.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann07programming.ppt.pdf\",\r\n  abstract      = \"This paper provides an overview of existing programming\r\n                   models for service-level high availability and investigates\r\n                   their differences, similarities, advantages, and\r\n                   disadvantages. Its goal is to help to improve reuse of code\r\n                   and to allow adaptation to quality of service requirements by\r\n                   using a uniform programming model description. It further\r\n                   aims at encouraging a discussion about these programming\r\n                   models and their provided quality of service, such as\r\n                   availability, performance, serviceability, usability, and\r\n                   applicability. Within this context, the presented research\r\n                   focuses on providing high availability for services running\r\n                   on head and service nodes of high-performance computing\r\n                   systems.\",\r\n  pts           = \"5078\"\r\n}<\/pre>\n
<\/a>@conference{wang07job,\r\n  author        = \"Chao Wang\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"A Job Pause Service under {LAM\/MPI+BLCR} for Transparent\r\n                   Fault Tolerance\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.ipdps.org\/ipdps2007}\r\n                   {$21^{st}$ IEEE International Parallel and Distributed\r\n                   Processing Symposium (IPDPS) 2007}\",\r\n  pages         = \"1-10\",\r\n  month         = mar # \"~26-30, \",\r\n  year          = \"2007\",\r\n  address       = \"Long Beach, CA, USA\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-59593-768-1\",\r\n  doi           = \"10.1109\/IPDPS.2007.370307\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/wang07job.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/wang07job.ppt.pdf\",\r\n  abstract      = \"Checkpoint\/restart (C\/R) has become a requirement for\r\n                   long-running jobs in large-scale clusters due to a\r\n                   mean-time-to-failure (MTTF) in the order of hours. After a\r\n                   failure, C\/R mechanisms generally require a complete restart\r\n                   of an MPI job from the last checkpoint. A complete restart,\r\n                   however, is unnecessary since all but one node are typically\r\n                   still alive. Furthermore, a restart may result in lengthy job\r\n                   requeuing even though the original job had not exceeded its\r\n                   time quantum. In this paper, we overcome these shortcomings.\r\n                   Instead of job restart, we have developed a transparent\r\n                   mechanism for job pause within LAM\/MPI+BLCR. This mechanism\r\n                   allows live nodes to remain active and roll back to the last\r\n                   checkpoint while failed nodes are dynamically replaced by\r\n                   spares before resuming from the last checkpoint. Our\r\n                   methodology includes LAM\/MPI enhancements in support of\r\n                   scalable group communication with fluctuating number of\r\n                   nodes, reuse of network connections, transparent coordinated\r\n                   checkpoint scheduling and a BLCR enhancement for job pause.\r\n                   Experiments in a cluster with the NAS Parallel Benchmark\r\n                   suite show that our overhead for job pause is comparable to\r\n                   that of a complete job restart. A minimal overhead of 5.6\\%\r\n                   is only incurred in case migration takes place while the\r\n                   regular checkpoint overhead remains unchanged. Yet, our\r\n                   approach alleviates the need to reboot the LAM run-time\r\n                   environment, which accounts for considerable overhead\r\n                   resulting in net savings of our scheme in the experiments.\r\n                   Our solution further provides full transparency and\r\n                   automation with the additional benefit of reusing existing\r\n                   resources. Executing continues after failures within the\r\n                   scheduled job, {\\em \\textit{i.e.}}, the application staging\r\n                   overhead is not incurred again in contrast to a restart.\r\n                   Our scheme offers additional potential for savings through\r\n                   incremental checkpointing and proactive diskless live\r\n                   migration, which we are currently working on.\",\r\n  pts           = \"4944\"\r\n}<\/pre>\n
<\/a>@conference{uhlemann06joshua,\r\n  author        = \"Kai Uhlemann\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"{JOSHUA}: {S}ymmetric Active\/Active Replication for Highly\r\n                   Available {HPC} Job and Resource Management\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/cluster2006.org}{$8^{th}$\r\n                   IEEE International Conference on Cluster Computing (Cluster)\r\n                   2006}\",\r\n  pages         = \"1-10\",\r\n  month         = sep # \"~25-28, \",\r\n  year          = \"2006\",\r\n  address       = \"Barcelona, Spain\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"1-4244-0328-6\",\r\n  issn          = \"1552-5244\",\r\n  doi           = \"10.1109\/CLUSTR.2006.311855\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/uhlemann06joshua.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/uhlemann06joshua.ppt.pdf\",\r\n  abstract      = \"Most of today`s HPC systems employ a single head node for\r\n                   control, which represents a single point of failure as it\r\n                   interrupts an entire HPC system upon failure. Furthermore, it\r\n                   is also a single point of control as it disables an entire\r\n                   HPC system until repair. One of the most important HPC system\r\n                   service running on the head node is the job and resource\r\n                   management. If it goes down, all currently running jobs loose\r\n                   the service they report back to. They have to be restarted\r\n                   once the head node is up and running again. With this paper,\r\n                   we present a generic approach for providing symmetric\r\n                   active\/active replication for highly available HPC job and\r\n                   resource management. The JOSHUA solution provides a virtually\r\n                   synchronous environment for continuous availability without\r\n                   any interruption of service and without any loss of state.\r\n                   Replication is performed externally via the PBS service\r\n                   interface without the need to modify any service code. Test\r\n                   results as well as availability analysis of our\r\n                   proof-of-concept prototype implementation show that\r\n                   continuous availability can be provided by JOSHUA with an\r\n                   acceptable performance trade-off.\",\r\n  pts           = \"2631\"\r\n}<\/pre>\n
<\/a>@conference{baumann06parallel,\r\n  author        = \"Ronald Baumann\r\n                   and Christian Engelmann\r\n                   and George A. (Al) Geist\",\r\n  title         = \"A Parallel Plug-in Programming Paradigm\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{http:\/\/hpcc06.lrr.in.tum.de}{$7^{th}$ International\r\n                   Conference on High Performance Computing and Communications\r\n                   (HPCC) 2006}\",\r\n  volume        = \"4208\",\r\n  pages         = \"823--832\",\r\n  month         = sep # \"~13-15, \",\r\n  year          = \"2006\",\r\n  address       = \"Munich, Germany\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-540-39368-9\",\r\n  issn          = \"0302-9743\",\r\n  doi           = \"10.1007\/11847366_85\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/baumann06parallel.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/baumann06parallel.ppt.pdf\",\r\n  abstract      = \"Software component architectures allow assembly of\r\n                   applications from individual software modules based on\r\n                   clearly defined programming interfaces, thus improving the\r\n                   reuse of existing solutions and simplifying application\r\n                   development. Furthermore, the plug-in programming paradigm\r\n                   additionally enables runtime reconfigurability, making it\r\n                   possible to adapt to changing application needs, such as\r\n                   different application phases, and system properties, like\r\n                   resource availability, by loading\/unloading appropriate\r\n                   software modules. Similar to parallel programs, parallel\r\n                   plug-ins are an abstraction for a set of cooperating\r\n                   individual plug-ins within a parallel application utilizing\r\n                   a software component architecture. Parallel programming\r\n                   paradigms apply to parallel plug-ins in the same way they\r\n                   apply to parallel programs. The research presented in this\r\n                   paper targets the clear definition of parallel plug-ins and\r\n                   the development of a parallel plug-in programming paradigm.\",\r\n  pts           = \"2413\"\r\n}<\/pre>\n
<\/a>@conference{varma06scalable,\r\n  author        = \"Jyothish Varma\r\n                   and Chao Wang\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"Scalable, Fault-Tolerant Membership for {MPI} Tasks on {HPC}\r\n                   Systems\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.ics-conference.org\/2006}\r\n                   {$20^{th}$ ACM International Conference on Supercomputing\r\n                   (ICS) 2006}\",\r\n  pages         = \"219--228\",\r\n  month         = jun # \"~28-30, \",\r\n  year          = \"2006\",\r\n  address       = \"Cairns, Australia\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  doi           = \"10.1145\/1183401.1183433\",\r\n  isbn          = \"1-59593-282-8\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/varma06scalable.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/varma06scalable.ppt.pdf\",\r\n  abstract      = \"Reliability is increasingly becoming a challenge for\r\n                   high-performance computing (HPC) systems with thousands of\r\n                   nodes, such as IBM`s Blue Gene\/L. A shorter\r\n                   mean-time-to-failure can be addressed by adding fault\r\n                   tolerance to reconfigure working nodes to ensure that\r\n                   communication and computation can progress. However, existing\r\n                   approaches fall short in providing scalability and small\r\n                   reconfiguration overhead within the fault-tolerant layer.\r\n                   This paper contributes a scalable approach to reconfigure the\r\n                   communication infrastructure after node failures. We propose\r\n                   a decentralized (peer-to-peer) protocol that maintains a\r\n                   consistent view of active nodes in the presence of faults.\r\n                   Our protocol shows response times in the order of hundreds of\r\n                   microseconds and single-digit milliseconds for \r\n                   reconfiguration using MPI over Blue Gene\/L and TCP over \r\n                   Gigabit, respectively. The protocol can be adapted to match\r\n                   the network topology to further increase performance. We also\r\n                   verify experimental results against a performance model,\r\n                   which demonstrates the scalability of the approach. Hence,\r\n                   the membership service is suitable for deployment in the\r\n                   communication layer of MPI runtime systems, and we have\r\n                   integrated an early version into LAM\/MPI.\",\r\n  pts           = \"2105\"\r\n}<\/pre>\n
<\/a>@conference{okunbor06exploring,\r\n  author        = \"Daniel I. Okunbor\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"Exploring Process Groups for Reliability, Availability and\r\n                   Serviceability of Terascale Computing Systems\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.atiner.gr\/docs\/2006AAAPROGRAM_COMP.htm}\r\n                   {$2^{nd}$ International Conference on Computer Science and\r\n                   Information Systems 2006}\",\r\n  month         = jun # \"~19-21, \",\r\n  year          = \"2006\",\r\n  address       = \"Athens, Greece\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/okunbor06exploring.pdf\",\r\n  abstract      = \"This paper presents various aspects of reliability,\r\n                   availability and serviceability (RAS) systems as they relate\r\n                   to group communication service, including reliable and total\r\n                   order multicast\/broadcast, virtual synchrony, and failure\r\n                   detection. While the issue of availability, particularly\r\n                   high availability using replication-based architectures has\r\n                   recently received upsurge research interests, much still have\r\n                   to be done in understanding the basic underlying concepts for\r\n                   achieving RAS systems, especially in high-end and high\r\n                   performance computing (HPC) communities. Various attributes\r\n                   of group communication service and the prototype of symmetric\r\n                   active replication following ideas utilized in the Newtop\r\n                   protocol will be discussed. We explore the application of\r\n                   group communication service for RAS HPC, laying the\r\n                   groundwork for its integrated model.\",\r\n  pts           = \"3778\"\r\n}<\/pre>\n
<\/a>@conference{limaye05jobsite,\r\n  author        = \"Kshitij Limaye\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and Zeno Greenwood\r\n                   and Stephen L. Scott\r\n                   and Christian Engelmann\r\n                   and Richard M. Libby\r\n                   and Kasidit Chanchio\",\r\n  title         = \"Job-Site Level Fault Tolerance for Cluster and {Grid}\r\n                   Environments\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/cluster2005.org}{$7^{th}$\r\n                   IEEE International Conference on Cluster Computing (Cluster)\r\n                   2005}\",\r\n  pages         = \"1--9\",\r\n  month         = sep # \"~26-30, \",\r\n  year          = \"2005\",\r\n  address       = \"Boston, MA, USA\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"0-7803-9486-0\",\r\n  issn          = \"1552-5244\",\r\n  doi           = \"10.1109\/CLUSTR.2005.347043\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/limaye05job-site.pdf\",\r\n  abstract      = \"In order to adopt high performance clusters and Grid\r\n                   computing for mission critical applications, fault tolerance\r\n                   is a necessity. Common fault tolerance techniques in\r\n                   distributed systems are normally achieved with\r\n                   checkpoint-recovery and job replication on alternative\r\n                   resources, in cases of a system outage. The first approach\r\n                   depends on the system`s MTTR while the latter approach\r\n                   depends on the availability of alternative sites to run\r\n                   replicas. There is a need for complementing these approaches\r\n                   by proactively handling failures at a job-site level,\r\n                   ensuring the system high availability with no loss of user\r\n                   submitted jobs. This paper discusses a novel fault tolerance\r\n                   technique  that enables the job-site recovery in Beowulf\r\n                   cluster-based grid environments, whereas existing techniques\r\n                   give up a failed system by seeking alternative resources.\r\n                   Our results suggest sizable aggregate performance improvement\r\n                   during an implementation of our method in Globus-enabled\r\n                   HA-OSCAR. The technique called Smart Failover provides a\r\n                   transparent and graceful recovery mechanism that saves job\r\n                   states in a local job-manager queue and transfers those\r\n                   states to the backup server periodically, and in critical\r\n                   system events. Thus whenever a failover occurs, the backup\r\n                   server is able to restart the jobs from their last saved\r\n                   state.\"\r\n}<\/pre>\n
<\/a>@conference{song05umlbased,\r\n  author        = \"Hertong Song\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and Raja Nassar\r\n                   and Yudan Liu\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"{UML-based} {Beowulf} Cluster Availability Modeling\",\r\n  booktitle     = \"\\href{http:\/\/www.world-academy-of-science.org\/IMCSE2005\/ws\/SERP}\r\n                   {International Conference on Software Engineering Research\r\n                   and Practice (SERP) 2005}\",\r\n  pages         = \"161--167\",\r\n  month         = jun # \"~27-30, \",\r\n  year          = \"2005\",\r\n  address       = \"Las Vegas, NV, USA\",\r\n  publisher     = \"CSREA Press\",\r\n  isbn          = \"1-932415-49-1\"\r\n}<\/pre>\n
<\/a>@conference{engelmann05superscalable,\r\n  author        = \"Christian Engelmann\r\n                   and George A. (Al) Geist\",\r\n  title         = \"Super-Scalable Algorithms for Computing on 100,000\r\n                   Processors\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{http:\/\/www.iccs-meeting.org\/iccs2005}{$5^{th}$\r\n                   International Conference on Computational Science (ICCS)\r\n                   2005}, Part I\",\r\n  volume        = \"3514\",\r\n  pages         = \"313--320\",\r\n  month         = may # \"~22-25, \",\r\n  year          = \"2005\",\r\n  address       = \"Atlanta, GA, USA\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-540-26032-5\",\r\n  issn          = \"0302-9743\",\r\n  doi           = \"10.1007\/11428831_39\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann05superscalable.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann05superscalable.ppt.pdf\",\r\n  abstract      = \"In the next five years, the number of processors in high-end\r\n                   systems for scientific computing is expected to rise to tens\r\n                   and even hundreds of thousands. For example, the IBM Blue\r\n                   Gene\/L can have up to 128,000 processors and the delivery of\r\n                   the first system is scheduled for 2005. Existing deficiencies\r\n                   in scalability and fault-tolerance of scientific applications\r\n                   need to be addressed soon. If the number of processors grows\r\n                   by a magnitude and efficiency drops by a magnitude, the\r\n                   overall effective computing performance stays the same.\r\n                   Furthermore, the mean time to interrupt of high-end computer\r\n                   systems decreases with scale and complexity. In a\r\n                   100,000-processor system, failures may occur every couple of\r\n                   minutes and traditional checkpointing may no longer be\r\n                   feasible. With this paper, we summarize our recent research\r\n                   in super-scalable algorithms for computing on 100,000\r\n                   processors. We introduce the algorithm properties of scale\r\n                   invariance and natural fault tolerance, and discuss how they\r\n                   can be applied to two different classes of algorithms. We\r\n                   also describe a super-scalable diskless checkpointing\r\n                   algorithm for problems that can`t be transformed into a\r\n                   super-scalable variant, or where other solutions are more\r\n                   efficient. Finally, a 100,000-processor simulator is\r\n                   presented as a platform for testing and experimentation.\"\r\n}<\/pre>\n
<\/a>@conference{engelmann26building,\r\n  author        = \"Christian Engelmann\r\n                   and Andrew Ayres\r\n                   and Stephen DeWitt\r\n                   and Michael J. Brim\r\n                   and Brett Eiffert\",\r\n  title         = \"Building Resilient Self-Driving Laboratories with the\r\n                   {INTERSECT} Federated Ecosystem\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/sc26.supercomputing.org}\r\n                   {$39^{th}$ International Conference on High Performance\r\n                   Computing, Networking, Storage and Analysis (SC) Workshops\r\n                   2026}: \\href{https:\/\/wordpress.cels.anl.gov\/xloop-2026\/}\r\n                   {$8^{th}$ Annual Workshop on Extreme-Scale\r\n                   Experiment-in-the-Loop Computing (XLOOP) 2026}\",\r\n  pages         = \"\",\r\n  month         = nov # \"~15, \",\r\n  year          = \"2026\",\r\n  address       = \"Chicago, IL, USA\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"\",\r\n  doi           = \"\",\r\n  url           = \"\",\r\n  url2          = \"\",\r\n  abstract      = \"Failure resilience in federated ecosystems for instrument\r\n                   science presents a critical challenge. Failures disrupt\r\n                   experiments and make them potentially useless, wasting\r\n                   valuable resources and creating setbacks. Oak Ridge National\r\n                   Laboratory's Self-driven Experiments for Science \/\r\n                   Interconnected Science Ecosystem (INTERSECT) offers a\r\n                   federated ecosystem for instrument science, enabling\r\n                   autonomous experiments, self-driving laboratories, smart\r\n                   manufacturing, and AI-driven design, discovery, and\r\n                   evaluation. This paper documents the recent advances in\r\n                   creating a resilient INTERSECT ecosystem. The proposed\r\n                   solution includes a resilient architecture with resilience\r\n                   design patterns, a resilient system of systems (SoS)\r\n                   architecture, and a resilient microservices architecture;\r\n                   and a resilient software development kit with reliable\r\n                   service communication and asynchronous and synchronous\r\n                   failure detection and notification. The resilience\r\n                   capabilities are demonstrated for an autonomous additive\r\n                   manufacturing process with a real-time feedback loop.\",\r\n  pts           = \"263041\",\r\n  note          = \"To appear\"\r\n}<\/pre>\n
<\/a>@conference{boehm28intersect,\r\n  author        = \"Swen Boehm\r\n                   and Craig A. Bridges\r\n                   and Patrick Widener\r\n                   and Terry Jones\r\n                   and Sheikh Ghafoor\r\n                   and Christian Engelmann\r\n                   and Olga Kuchar\",\r\n  title         = \"The INTERSECT Scientific Data Layer: An Ontological Framework\r\n                   for Data Provenance for Complex Scientific Workflows\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{https:\/\/2026.euro-par.org}{$32^{nd}$ European Conference\r\n                   on Parallel and Distributed Computing (Euro-Par) 2026\r\n                   Workshops}: \\href{https:\/\/www.hipes-workshop.org\/}\r\n                   {$3^{rd}$ Workshop on High-Performance eScience Tools and\r\n                   Applications (HiPES)}\",\r\n  volume        = \"\",\r\n  pages         = \"\",\r\n  month         = aug # \"~25, \",\r\n  year          = \"2026\",\r\n  address       = \"Pisa, Italy\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"\",\r\n  doi           = \"\",\r\n  url           = \"\",\r\n  url2          = \"\",\r\n  abstract      = \"Complex scientific workflows have multiple stages, including\r\n                   experiments, simulations, data analyses, and visualization,\r\n                   generating data and metadata stored across heterogeneous\r\n                   storage infrastructures and used in downstream stages or\r\n                   future experimental campaigns. This paper presents the design\r\n                   and implementation of a comprehensive ontological framework\r\n                   for managing scientific data generated within such workflows,\r\n                   addressing the key challenges of data interoperability,\r\n                   provenance capture, and adherence to Findable, Accessible,\r\n                   Interoperable, and Reusable (FAIR) data principles. The\r\n                   Autonomous Chemistry Laboratory (ACL) at Oak Ridge National\r\n                   Laboratory (ORNL) enables automated liquid phase and solid\r\n                   state synthesis and related chemical analysis. Our framework\r\n                   has been deployed within the ACL for a native and\r\n                   machine-interpretable semantic representation of its\r\n                   ecosystem, including instrument capabilities, synthesis\r\n                   workflows, analytical observations, and experimental results.\r\n                   The proposed approach enables end-to-end provenance tracking,\r\n                   supports heterogeneous data formats, and establishes a\r\n                   foundation for Artificial Intelligence (AI)-ready scientific\r\n                   discovery. We validate the framework through concrete\r\n                   modeling examples.\",\r\n  pts           = \"\",\r\n  note          = \"To appear\"\r\n}<\/pre>\n
<\/a>@conference{kotevska26scalable,\r\n  author        = \"Olivera Kotevska\r\n                   and Trong Nguyen\r\n                   and Rafael Ferreira da Silva\r\n                   and Christian Engelmann\r\n                   and Prasanna Balaprakash\",\r\n  title         = \"Scalable Federated Learning for Scientific Foundation Models\r\n                   on Leadership-Class Systems\",\r\n  booktitle     = \"Proceedings of the \\href{https:\/\/2026.eurosys.org\/}{$21^{st}$\r\n                   European Conference on Computer Systems (EuroSyS)}:\r\n                   \\href{https:\/\/euromlsys.eu\/}{$6^{th}$ European Workshop on\r\n                   Machine Learning and Systems (EuroMLSys)}\",\r\n  pages         = \"439--446\",\r\n  month         = apr # \"~27, \",\r\n  year          = \"2026\",\r\n  address       = \"Edinburgh, United Kingdom\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"979-8-4007-2605-7\",\r\n  doi           = \"10.1145\/3805621.3807639\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/kotevska26scalable.pdf\",\r\n  url2          = \"\",\r\n  abstract      = \"Federated learning (FL) at leadership-class HPC systems\r\n                   remains largely unexplored, despite growing interest in\r\n                   deploying federated workflows on modern HPC systems. This\r\n                   paper provides the first system-level empirical\r\n                   characterization of federated fine-tuning of pretrained\r\n                   foundation models on an exascale supercomputer under a\r\n                   multi-node deployment. Using up to 96 concurrent FL clients\r\n                   deployed across Frontier nodes, we study the impact of client\r\n                   scale, model size, data heterogeneity, partial participation,\r\n                   and differential privacy on runtime, communication overhead,\r\n                   and convergence stability.\r\n                   Our results show that pretrained transformer models remain\r\n                   robust to heterogeneity, client dropout, and privacy noise,\r\n                   while system efficiency degrades rapidly with scale as\r\n                   synchronization and orchestration dominate runtime. We\r\n                   further demonstrate that system-aware execution strategies,\r\n                   including intra-node aggregation and early aggregation,\r\n                   significantly reduce wall-clock time without degrading model\r\n                   quality. These findings establish a practical performance\r\n                   baseline and inform the design of communication-efficient FL\r\n                   systems on leadership-class HPC platforms.\",\r\n  pts           = \"250411\"\r\n}<\/pre>\n
<\/a>@conference{brim24microservices,\r\n  author        = \"Michael J. Brim\r\n                   and Lance Drane\r\n                   and Marshall McDonnell\r\n                   and Christian Engelmann\r\n                   and Addi Malviya Thakur\",\r\n  title         = \"A Microservices Architecture Toolkit for Interconnected\r\n                   Science Ecosystems\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/sc24.supercomputing.org}\r\n                   {$37^{th}$ International Conference on High Performance\r\n                   Computing, Networking, Storage and Analysis (SC) Workshops\r\n                   2024}: \\href{https:\/\/works-workshop.org\/}\r\n                   {$19^{th}$ Workshop on Workflows in Support of Large-Scale\r\n                    Science (WORKS) 2024}\",\r\n  pages         = \"2072--2079\",\r\n  month         = nov # \"~18, \",\r\n  year          = \"2024\",\r\n  address       = \"Atlanta, GA, USA\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"979-8-3503-5554-3\",\r\n  doi           = \"10.1109\/SCW63240.2024.00259\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/brim24microservices.pdf\",\r\n  url2          = \"\",\r\n  abstract      = \"Microservices architecture is a promising approach for\r\n                   developing reusable scientific workflow capabilities for\r\n                   integrating diverse resources, such as experimental and\r\n                   observational instruments and advanced computational and data\r\n                   management systems, across many distributed organizations and\r\n                   facilities. In this paper, we describe how the INTERSECT Open\r\n                   Architecture leverages federated systems of microservices to\r\n                   construct interconnected science ecosystems, review how the\r\n                   INTERSECT software development kit eases microservice\r\n                   capability development, and demonstrate the use of such\r\n                   capabilities for deploying an example multi-facility\r\n                   INTERSECT ecosystem.\",\r\n  pts           = \"223539\"\r\n}<\/pre>\n
<\/a>@conference{kumar21rdpm,\r\n  author        = \"Mohit Kumar\r\n                   and Christian Engelmann\",\r\n  title         = \"{RDPM}: An Extensible Tool for Resilience Design Patterns\r\n                   Modeling\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{https:\/\/2021.euro-par.org}{$27^{th}$\r\n                   European Conference on Parallel and Distributed Computing\r\n                   (Euro-Par) 2021 Workshops}:\r\n                   \\href{http:\/\/www.csm.ornl.gov\/srt\/conferences\/Resilience\/2021}\r\n                   {$14^{th}$ Workshop on Resiliency in High Performance\r\n                   Computing (Resilience) in Clusters, Clouds, and Grids}\",\r\n  volume        = \"13098\",\r\n  pages         = \"283--297\",\r\n  month         = aug # \"~30, \",\r\n  year          = \"2021\",\r\n  address       = \"Lisbon, Portugal\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-031-06155-4\",\r\n  doi           = \"10.1007\/978-3-031-06156-1_23\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/kumar21rdpm.pdf\",\r\n  url2          = \"\",\r\n  abstract      = \"Resilience to faults, errors, and failures in extreme-scale\r\n                   HPC systems is a critical challenge. Resilience design\r\n                   patterns offer a new, structured hardware and software design\r\n                   approach for improving resilience. While prior work focused\r\n                   on developing performance, reliability, and availability\r\n                   models for resilience design patterns, this paper extends it\r\n                   by providing a Resilience Design Patterns Modeling (RDPM)\r\n                   tool which allows (1) exploring performance, reliability,\r\n                   and availability of each resilience design pattern, (2)\r\n                   offering customization of parameters to optimize performance,\r\n                   reliability, and availability, and (3) allowing\r\n                   investigation of trade-off models for combining multiple\r\n                   patterns for practical resilience solutions.\",\r\n  pts           = \"161085\"\r\n}<\/pre>\n
<\/a>@conference{kumar20models,\r\n  author        = \"Mohit Kumar\r\n                   and Christian Engelmann\",\r\n  title         = \"Models for Resilience Design Patterns\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/sc20.supercomputing.org}\r\n                   {$33^{rd}$ International Conference on High Performance\r\n                   Computing, Networking, Storage and Analysis (SC) Workshops\r\n                   2020}: \\href{https:\/\/sites.google.com\/site\/ftxsworkshop\/home\/ftxs-2020}\r\n                   {$10^{th}$ Workshop on Fault Tolerance for HPC at eXtreme\r\n                   Scale (FTXS) 2020}\",\r\n  pages         = \"21-30\",\r\n  month         = nov # \"~11, \",\r\n  year          = \"2020\",\r\n  address       = \"Atlanta, GA, USA\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7381-1080-6\",\r\n  doi           = \"10.1109\/FTXS51974.2020.00008\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/kumar20models.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/kumar20models.ppt.pdf\",\r\n  abstract      = \"Resilience plays an important role in supercomputers by\r\n                   providing correct and efficient operation in case of faults,\r\n                   errors, and failures. Resilience design patterns offer\r\n                   blueprints for effectively applying resilience technologies.\r\n                   Prior work focused on developing initial efficiency and\r\n                   performance models for resilience design patterns. This paper\r\n                   extends it by (1) describing performance, reliability, and\r\n                   availability models for all structural resilience design\r\n                   patterns, (2) providing more detailed models that include\r\n                   flowcharts and state diagrams, and (3) introducing the\r\n                   Resilience Design Pattern Modeling (RDPM) tool that\r\n                   calculates and plots the performance, reliability, and\r\n                   availability metrics of individual patterns and pattern\r\n                   combinations.\",\r\n  pts           = \"148010\"\r\n}<\/pre>\n
<\/a>@conference{sao19self-stabilizing,\r\n  author        = \"Piyush Sao\r\n                   and Christian Engelmann\r\n                   and Srinivas Eswar\r\n                   and Oded Green\r\n                   and Richard Vuduc\",\r\n  title         = \"Self-stabilizing Connected Components\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/sc19.supercomputing.org}\r\n                   {$32^{nd}$ International Conference on High Performance\r\n                   Computing, Networking, Storage and Analysis (SC) Workshops\r\n                   2019}: \\href{https:\/\/sites.google.com\/site\/ftxsworkshop\/home\/ftxs-2019}\r\n                   {$9^{th}$ Workshop on Fault Tolerance for HPC at eXtreme\r\n                   Scale (FTXS) 2019}\",\r\n  pages         = \"50--59\",\r\n  month         = nov # \"~22, \",\r\n  year          = \"2019\",\r\n  address       = \"Denver, CO, USA\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-1-7281-6013-9\",\r\n  doi           = \"10.1109\/FTXS49593.2019.00011\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/sao19self-stabilizing.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/sao19self-stabilizing.ppt.pdf\",\r\n  abstract      = \"For the problem of computing the connected components of a\r\n                   graph, this paper considers the design of algorithms that are\r\n                   resilient to transient hardware faults, like bit flips. More\r\n                   specifically, it applies the technique of\r\n                   \\emph{self-stabilization}. A system is self-stabilizing if,\r\n                   when starting from a valid or invalid state, it is guaranteed\r\n                   to reach a valid state after a finite number of steps.\r\n                   Therefore on a machine subject to a transient fault, a\r\n                   self-stabilizing algorithm could recover if that fault caused\r\n                   the system to enter an invalid state.\r\n                   We give a comprehensive analysis of the valid and invalid\r\n                   states during label propagation and derive algorithms to\r\n                   verify and correct the invalid state. The self-stabilizing\r\n                   label-propagation algorithm performs $\\bigo{V \\log V}$\r\n                   additional computation and requires $\\bigo{V}$ additional\r\n                   storage over its conventional counterpart (and, as such,\r\n                   does not increase asymptotic complexity over conventional).\r\n                   When run against a battery of simulated fault injection\r\n                   tests, the self-stabilizing label propagation algorithm\r\n                   exhibits more resilient behavior than a triple modular\r\n                   redundancy (TMR) based fault-tolerant algorithm in 80\\% of\r\n                   cases. From a performance perspective, it also outperforms\r\n                   TMR as it requires fewer iterations in total. Beyond the\r\n                   fault-tolerance properties of self-stabilizing\r\n                   label-propagation, we believe, they are useful from the\r\n                   theoretical perspective; and may have other use-cases.\",\r\n  pts           = \"135067\"\r\n}<\/pre>\n
<\/a>@conference{engelmann19concepts,\r\n  author        = \"Christian Engelmann\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Swaroop Pophale\",\r\n  title         = \"Concepts for {OpenMP} Target Offload Resilience\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{http:\/\/parallel.auckland.ac.nz\/iwomp2019}\r\n                   {$15^{th}$ International Workshop on OpenMP (IWOMP) 2019}\",\r\n  volume        = \"11718\",\r\n  pages         = \"78--93\",\r\n  month         = sep # \"~11-13, \",\r\n  year          = \"2019\",\r\n  address       = \"Auckland, New Zealand\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-030-28595-1\",\r\n  doi           = \"10.1007\/978-3-030-28596-8_6\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann19concepts.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann19concepts.ppt.pdf\",\r\n  abstract      = \"Recent reliability issues with one of the fastest\r\n                   supercomputers in the world, Titan at Oak Ridge National\r\n                   Laboratory, demonstrated the need for resilience in\r\n                   large-scale heterogeneous computing. OpenMP currently does\r\n                   not address error and failure behavior. This paper takes a\r\n                   first step toward resilience for heterogeneous systems by\r\n                   providing the concepts for resilient OpenMP offload to\r\n                   devices. Using real-world error and failure observations,\r\n                   the paper describes the concepts and terminology for\r\n                   resilient OpenMP target offload, including error and failure\r\n                   classes and resilience strategies. It details the\r\n                   experienced general-purpose computing on graphics processing\r\n                   units errors and failures in Titan. It further proposes\r\n                   improvements in OpenMP, including a preliminary prototype\r\n                   design, to support resilient offload to devices for\r\n                   efficient handling of errors and failures in heterogeneous\r\n                   high-performance computing systems\",\r\n  pts           = \"127338\"\r\n}<\/pre>\n
<\/a>@conference{hui18comprehensive2,\r\n  author        = \"Yawei Hui\r\n                   and Byung Hoon (Hoony) Park\r\n                   and Christian Engelmann\",\r\n  title         = \"A Comprehensive Informative Metric for Analyzing {HPC} System\r\n                   Status using the {LogSCAN} Platform\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/sc18.supercomputing.org}\r\n                   {$31^{st}$ International Conference on High Performance\r\n                   Computing, Networking, Storage and Analysis (SC) Workshops\r\n                   2018}: \\href{https:\/\/sites.google.com\/site\/ftxsworkshop\/home\/ftxs-2018}\r\n                   {$8^{th}$ Workshop on Fault Tolerance for HPC at eXtreme\r\n                   Scale (FTXS) 2018}\",\r\n  pages         = \"29--38\",\r\n  month         = nov # \"~16, \",\r\n  year          = \"2018\",\r\n  address       = \"Dallas, TX, USA\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-1-7281-0222-1\",\r\n  doi           = \"10.1109\/FTXS.2018.00007\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/hui18comprehensive2.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/hui18comprehensive2.ppt.pdf\",\r\n  abstract      = \"Log processing by Spark and Cassandra-based ANalytics\r\n                   (LogSCAN) is a newly developed analytical platform that\r\n                   provides flexible and scalable data gathering, transformation\r\n                   and computation. One major challenge is to effectively\r\n                   summarize the status of a complex computer system, such as\r\n                   the Titan supercomputer at the Oak Ridge Leadership Computing\r\n                   Facility (OLCF). Although there is plenty of operational and\r\n                   maintenance information collected and stored in real time,\r\n                   which may yield insights about short- and long-term system\r\n                   status, it is difficult to present this information in a\r\n                   comprehensive form. In this work, we present system\r\n                   information entropy (SIE), a newly developed metric that\r\n                   leverages the powers of traditional machine learning\r\n                   techniques and information theory. By compressing the\r\n                   multi-variant multi-dimensional event information recorded\r\n                   during the operation of the targeted system into a single\r\n                   time series of SIE, we demonstrate that the historical\r\n                   system status can be sensitively represented concisely and\r\n                   comprehensively. Given a sharp indicator as SIE, we argue\r\n                   that follow-up analytics based on SIE will reveal in-depth\r\n                   knowledge about system status using other sophisticated\r\n                   approaches, such as pattern recognition in the temporal\r\n                   domain or causality analysis incorporating extra\r\n                   independent metrics of the system.\",\r\n  pts           = \"119248\"\r\n}<\/pre>\n
<\/a>@conference{ashraf18analyzing,\r\n  author        = \"Rizwan Ashraf\r\n                   and Christian Engelmann\",\r\n  title         = \"Analyzing the Impact of System Reliability Events on\r\n                   Applications in the {Titan} Supercomputer\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/sc18.supercomputing.org}\r\n                   {$31^{st}$ International Conference on High Performance\r\n                   Computing, Networking, Storage and Analysis (SC) Workshops\r\n                   2018}: \\href{https:\/\/sites.google.com\/site\/ftxsworkshop\/home\/ftxs-2018}\r\n                   {$8^{th}$ Workshop on Fault Tolerance for HPC at eXtreme\r\n                   Scale (FTXS) 2018}\",\r\n  pages         = \"39--48\",\r\n  month         = nov # \"~16, \",\r\n  year          = \"2018\",\r\n  address       = \"Dallas, TX, USA\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-1-7281-0222-1\",\r\n  doi           = \"10.1109\/FTXS.2018.00008\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/ashraf18analyzing.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/ashraf18analyzing.ppt.pdf\",\r\n  abstract      = \"Extreme-scale computing systems employ Reliability,\r\n                   Availability and Serviceability (RAS) mechanisms and\r\n                   infrastructure to log events from multiple system components.\r\n                   In this paper, we analyze RAS logs in conjunction with the \r\n                   application placement and scheduling database, in order to \r\n                   understand the impact of common RAS events on application\r\n                   performance. This study conducted on the records of about 2\r\n                   million applications executed on Titan supercomputer\r\n                   provides important insights for system users, operators and\r\n                   computer science researchers. In this paper, we investigate\r\n                   the impact of RAS events on application performance and its\r\n                   variability by comparing cases where events are recorded\r\n                   with corresponding cases where no events are recorded. Such\r\n                   a statistical investigation is possible since we observed\r\n                   that system users tend to execute their applications\r\n                   multiple times. Our analysis reveals that most RAS events\r\n                   do impact application performance, although not always. We\r\n                   also find that different system components affect\r\n                   application performance differently. In particular, our\r\n                   investigation includes the following components: parallel\r\n                   file system, processor, memory, graphics processing units,\r\n                   system and user software issues. Our work establishes the\r\n                   importance of providing feedback to system users for\r\n                   increasing operational efficiency of extreme-scale systems.\",\r\n  pts           = \"119070\"\r\n}<\/pre>\n
<\/a>@conference{park18big,\r\n  author        = \"Byung Hoon (Hoony) Park\r\n                   and Yawei Hui\r\n                   and Swen Boehm\r\n                   and Rizwan Ashraf\r\n                   and Christian Engelmann\r\n                   and Christopher Layton\",\r\n  title         = \"A {Big Data} Analytics Framework for {HPC} Log Data: {Three}\r\n                   Case Studies Using the {Titan} Supercomputer Log\",\r\n  booktitle     = \"Proceedings of the \\href{https:\/\/cluster2018.github.io}\r\n                   {$19^{th}$ IEEE International Conference on Cluster Computing\r\n                   (Cluster) 2018}:\r\n                   \\href{https:\/\/sites.google.com\/site\/hpcmaspa2018}\r\n                   {$5^{th}$ Workshop on Monitoring and Analysis for High\r\n                   Performance Systems Plus Applications (HPCMASPA) 2018}\",\r\n  pages         = \"571--579\",\r\n  month         = sep # \"~10, \",\r\n  year          = \"2018\",\r\n  address       = \"Belfast, UK\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-1-5386-8319-4\",\r\n  issn          = \"2168-9253\",\r\n  doi           = \"10.1109\/CLUSTER.2018.00073\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/park18big.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/park18big.ppt.pdf\",\r\n  abstract      = \"Reliability, availability and serviceability (RAS) logs of\r\n                   high performance computing (HPC) resources, when closely\r\n                   investigated in spatial and temporal dimensions, can provide\r\n                   invaluable information regarding system status, performance,\r\n                   and resource utilization. These data are often generated from\r\n                   multiple logging systems and sensors that cover many\r\n                   components of the system. The analysis of these data for\r\n                   finding persistent temporal and spatial insights faces two\r\n                   main difficulties: the volume of RAS logs makes manual\r\n                   inspection difficult and the unstructured nature and unique\r\n                   properties of log data produced by each subsystem adds\r\n                   another dimension of difficulty in identifying implicit\r\n                   correlation among recorded events. To address these issues,\r\n                   we recently developed a multi-user Big Data analytics\r\n                   framework for HPC log data at Oak Ridge National Laboratory\r\n                   (ORNL). This paper introduces three in-progress data\r\n                   analytics projects that leverage this framework to assess\r\n                   system status, mine event patterns, and study correlations\r\n                   between user applications and system events. We describe the\r\n                   motivation of each project and detail their workflows using\r\n                   three years of log data collected from ORNL's Titan\r\n                   supercomputer.\",\r\n  pts           = \"112964\"\r\n}<\/pre>\n
<\/a>@conference{ashraf18performance,\r\n  author        = \"Rizwan Ashraf\r\n                   and Christian Engelmann\",\r\n  title         = \"Performance Efficient Multiresilience using Checkpoint\r\n                   Recovery in Iterative Algorithms\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{https:\/\/europar2018.org}{$24^{th}$\r\n                   European Conference on Parallel and Distributed Computing\r\n                   (Euro-Par) 2018 Workshops}:\r\n                   \\href{http:\/\/www.csm.ornl.gov\/srt\/conferences\/Resilience\/2018}\r\n                   {$11^{th}$ Workshop on Resiliency in High Performance\r\n                   Computing (Resilience) in Clusters, Clouds, and Grids}\",\r\n  volume        = \"11339\",\r\n  pages         = \"813--825\",\r\n  month         = aug # \"~28, \",\r\n  year          = \"2018\",\r\n  address       = \"Turin, Italy\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-030-10549-5\",\r\n  doi           = \"10.1007\/978-3-030-10549-5_63\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/ashraf18performance.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/ashraf18performance.ppt.pdf\",\r\n  abstract      = \"In this paper, we address the design challenge of building\r\n                   multiresilient iterative high-performance computing (HPC)\r\n                   applications. Multiresilience in HPC applications is the\r\n                   ability to tolerate and maintain forward progress in the\r\n                   presence of both soft errors and process failures. We address\r\n                   the challenge by proposing performance models which are\r\n                   useful to design performance efficient and resilient\r\n                   iterative applications. The models consider the interaction\r\n                   between soft error and process failure resilience solutions.\r\n                   We experimented with a linear solver application with two\r\n                   distinct kinds of soft error detectors: one detector is high\r\n                   overhead and high accuracy, whereas the second is low\r\n                   overhead and low accuracy. We show how both can be leveraged\r\n                   for verifying the integrity of checkpointed state used to\r\n                   recover from both soft errors and process failures. Our\r\n                   results show the performance efficiency and resiliency\r\n                   benefit of employing the low overhead detector with high\r\n                   frequency within the checkpoint interval, so that timely\r\n                   soft error recovery can take place, resulting in less\r\n                   re-computed work.\",\r\n  pts           = \"112980\"\r\n}<\/pre>\n
<\/a>@conference{park17big,\r\n  author        = \"Byung Hoon (Hoony) Park\r\n                   and Saurabh Hukerikar\r\n                   and Christian Engelmann\r\n                   and Ryan Adamson\",\r\n  title         = \"Big Data Meets {HPC} Log Analytics: {Scalable} Approach to\r\n                   Understanding Systems at Extreme Scale\",\r\n  booktitle     = \"Proceedings of the \\href{https:\/\/cluster17.github.io}\r\n                   {$18^{th}$ IEEE International Conference on Cluster Computing\r\n                   (Cluster) 2017}:\r\n                   \\href{https:\/\/sites.google.com\/site\/hpcmaspa2017}\r\n                   {$4^{th}$ Workshop on Monitoring and Analysis for High\r\n                   Performance Systems Plus Applications (HPCMASPA) 2017}\",\r\n  pages         = \"758--765\",\r\n  month         = sep # \"~5, \",\r\n  year          = \"2017\",\r\n  address       = \"Honolulu, HI, USA\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-1-5386-2327-5\",\r\n  issn          = \"2168-9253\",\r\n  doi           = \"10.1109\/CLUSTER.2017.113\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/park17big.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/park17big.ppt.pdf\",\r\n  abstract      = \"Today's high-performance computing (HPC) systems are heavily\r\n                   instrumented generating logs containing information about\r\n                   abnormal events, such as critical conditions, faults, errors\r\n                   and failures, system resource utilization, and about the\r\n                   resource usage of user applications. These logs, once fully\r\n                   analyzed and correlated, can produce detailed information\r\n                   about the system health, root causes of failures, and\r\n                   analyze an application's interactions with the system,\r\n                   providing invaluable insights to domain scientists and\r\n                   system administrators. However, processing HPC logs\r\n                   requires deep understanding of hardware and software\r\n                   components at multiple layers of the system stack.\r\n                   Moreover, most log data is unstructured and voluminous,\r\n                   making it more difficult for scientists and engineers to\r\n                   analyze the data. With rapid increases in the scale and\r\n                   complexity of HPC systems, log data processing is becoming\r\n                   a big data challenge. This paper introduces a HPC log data\r\n                   analytics framework that is based on a distributed NoSQL\r\n                   database technology, which provides scalability and high\r\n                   availability, and the Apache Spark for rapid in-memory\r\n                   processing of log data. The framework enables the\r\n                   extraction of a range of information about the system so\r\n                   that system administrators and end users alike can obtain\r\n                   necessary insights for their specific needs. We describe\r\n                   our experience with using this framework to glean insights\r\n                   from the log data derived from the Titan supercomputer at\r\n                   the Oak Ridge National Laboratory.\",\r\n  pts           = \"100681\"\r\n}<\/pre>\n
<\/a>@conference{hukerikar17pattern-based,\r\n  author        = \"Saurabh Hukerikar\r\n                   and Christian Engelmann\",\r\n  title         = \"Pattern-based Modeling of High-Performance Computing\r\n                   Resilience\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{http:\/\/europar2017.usc.es}{$23^{rd}$\r\n                   European Conference on Parallel and Distributed Computing\r\n                   (Euro-Par) 2017 Workshops}:\r\n                   \\href{http:\/\/www.csm.ornl.gov\/srt\/conferences\/Resilience\/2017}\r\n                   {$10^{th}$ Workshop on Resiliency in High Performance\r\n                   Computing (Resilience) in Clusters, Clouds, and Grids}\",\r\n  volume        = \"10659\",\r\n  pages         = \"557--568\",\r\n  month         = aug # \"~29, \",\r\n  year          = \"2017\",\r\n  address       = \"Santiago de Compostela, Spain\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-319-75177-1\",\r\n  doi           = \"10.1007\/978-3-319-75178-8_45\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/hukerikar17pattern-based.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/hukerikar17pattern-based.ppt.pdf\",\r\n  abstract      = \"The design of supercomputing systems and their applications\r\n                   must consider resilience, and power consumption as the key\r\n                   design parameters when designing to achieve higher\r\n                   performance. In previous work, we established a structured\r\n                   methodology for developing resilience solutions based on the\r\n                   concept of design patterns. In this paper we discuss\r\n                   analytical models for the design patterns to support\r\n                   quantitative analysis of their performance and reliability\r\n                   characteristics.\",\r\n  pts           = \"102871\"\r\n}<\/pre>\n
<\/a>@conference{hukerikar17towards,\r\n  author        = \"Saurabh Hukerikar\r\n                   and Rizwan Ashraf\r\n                   and Christian Engelmann\",\r\n  title         = \"Towards New Metrics for High-Performance Computing\r\n                   Resilience\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.hpdc.org\/2017}\r\n                   {$26^{th}$ ACM International Symposium on High-Performance\r\n                   Parallel and Distributed Computing (HPDC) 2017}:\r\n                   \\href{https:\/\/sites.google.com\/site\/ftxsworkshop\/home\/ftxs-2017}\r\n                   {$7^{th}$ Workshop on Fault Tolerance for HPC at eXtreme\r\n                   Scale (FTXS) 2017}\",\r\n  pages         = \"23--30\",\r\n  month         = jun # \"~26-30, \",\r\n  year          = \"2017\",\r\n  address       = \"Washington, D.C.\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-4503-5001-3\",\r\n  doi           = \"10.1145\/3086157.3086163\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/hukerikar17towards.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/hukerikar17towards.ppt.pdf\",\r\n  abstract      = \"Ensuring the reliability of applications is becoming an\r\n                   increasingly important challenge as high-performance\r\n                   computing (HPC) systems experience an ever-growing number of\r\n                   faults, errors and failures. While the HPC community has made\r\n                   substantial progress in developing various resilience\r\n                   solutions, it continues to rely on platform-based metrics to\r\n                   quantify application resiliency improvements. The resilience\r\n                   of an HPC application is concerned with the reliability of\r\n                   the application outcome as well as the fault handling\r\n                   efficiency. To understand the scope of impact, effective\r\n                   coverage and performance efficiency of existing and emerging\r\n                   resilience solutions, there is a need for new metrics. In\r\n                   this paper, we develop new ways to quantify resilience that\r\n                   consider both the reliability and the performance\r\n                   characteristics of the solutions from the perspective of HPC\r\n                   applications. As HPC systems continue to evolve in terms of\r\n                   scale and complexity, it is expected that applications will\r\n                   experience various types of faults, errors and failures,\r\n                   which will require applications to apply multiple resilience\r\n                   solutions across the system stack. The proposed metrics are\r\n                   intended to be useful for understanding the combined impact\r\n                   of these solutions on an application's ability to produce\r\n                   correct results and to evaluate their overall impact on an\r\n                   application's performance in the presence of various modes\r\n                   of faults.\",\r\n  pts           = \"74843\"\r\n}<\/pre>\n
<\/a>@conference{hukerikar16language,\r\n  author        = \"Saurabh Hukerikar\r\n                   and Christian Engelmann\",\r\n  title         = \"Language Support for Reliable Memory Regions\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{https:\/\/lcpc2016.wordpress.com}{$29^{th}$\r\n                   International Workshop on Languages and Compilers for\r\n                   Parallel Computing}\",\r\n  volume        = \"10136\",\r\n  pages         = \"73--87\",\r\n  month         = sep # \"~28-30, \",\r\n  year          = \"2016\",\r\n  address       = \"Rochester, NY, USA\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-319-52708-6\",\r\n  issn          = \"0302-9743\",\r\n  doi           = \"10.1007\/978-3-319-52709-3_6\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/hukerikar16language.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/hukerikar16language.ppt.pdf\",\r\n  abstract      = \"The path to exascale computational capabilities in\r\n                   high-performance computing (HPC) systems is challenged by the\r\n                   evolution of the architectures of supercomputing systems. The\r\n                   constraints of power have driven designs that include\r\n                   increasingly heterogeneous architectures and complex memory\r\n                   hierarchies. These systems are also expected to experience in\r\n                   an increased rate of errors, such that the applications will\r\n                   no longer be able to assume correct behavior of the\r\n                   underlying machine. To enable the scientific community to\r\n                   succeed in scaling their applications and harness the\r\n                   capabilities of exascale systems, we need software strategies\r\n                   that provide mechanisms for explicit management of locality\r\n                   and resilience to errors in the system.\r\n                   In prior work, we introduced the concept of explicitly\r\n                   reliable memory regions, called havens. Memory management\r\n                   using havens supports selective reliability through a\r\n                   region-based approach to memory allocation. Havens enable the\r\n                   creation of explicit software-enabled robust memory\r\n                   containers for which resilient behavior is guaranteed. In\r\n                   this paper, we propose language support for havens through\r\n                   type annotations that make the structure of a program's\r\n                   havens more explicit. We describe how the extended\r\n                   haven-based memory management model is implemented and the\r\n                   impact on the resiliency of a conjugate gradient\r\n                   application.\",\r\n  pts           = \"69644\"\r\n}<\/pre>\n
<\/a>@conference{naughton16cooperative,\r\n  author        = \"Thomas Naughton\r\n                   and Christian Engelmann\r\n                   and Geoffroy Vall{\\'e}e\r\n                   and Ferrol Aderholdt\r\n                   and Stephen L. Scott\",\r\n  title         = \"A Cooperative Approach to Virtual Machine Based Fault\r\n                   Injection\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{https:\/\/europar2016.inria.fr}{$22^{nd}$\r\n                   European Conference on Parallel and Distributed Computing\r\n                   (Euro-Par) 2016 Workshops}:\r\n                   \\href{http:\/\/www.csm.ornl.gov\/srt\/conferences\/Resilience\/2016}\r\n                   {$9^{th}$ Workshop on Resiliency in High Performance\r\n                   Computing (Resilience) in Clusters, Clouds, and Grids}\",\r\n  volume        = \"10104\",\r\n  pages         = \"671--682\",\r\n  month         = aug # \"~23, \",\r\n  year          = \"2016\",\r\n  address       = \"Grenoble, France\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-319-58943-5\",\r\n  issn          = \"0302-9743\",\r\n  doi           = \"10.1007\/978-3-319-58943-5_54\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/naughton16cooperative.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/naughton16cooperative.ppt.pdf\",\r\n  abstract      = \"Resilience investigations often employ fault injection (FI)\r\n                   tools to study the effects of simulated errors on a target\r\n                   system. It is important to keep the target system under test\r\n                   (SUT) isolated from the controlling environment in order to\r\n                   maintain control of the experiment. Virtual machines (VMs)\r\n                   have been used to aid these investigations due to the strong\r\n                   isolation properties of system-level virtualization. A key\r\n                   challenge in fault injection tools is to gain proper insight\r\n                   and context about the SUT. In VM-based FI tools, this\r\n                   challenge of target con- text is increased due to the\r\n                   separation between host and guest (VM). We discuss an\r\n                   approach to VM-based FI that leverages virtual machine\r\n                   introspection (VMI) methods to gain insight into the target's\r\n                   context running within the VM. The key to this environment is\r\n                   the ability to provide basic information to the FI system\r\n                   that can be used to create a map of the target environment.\r\n                   We describe a proof- of-concept implementation and a\r\n                   demonstration of its use to introduce simulated soft errors\r\n                   into an iterative solver benchmark running in user-space of\r\n                   a guest VM.\",\r\n  pts           = \"69232\"\r\n}<\/pre>\n
<\/a>@conference{parchman16adding,\r\n  author        = \"Zachary Parchman\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Christian Engelmann\r\n                   and David E. Bernholdt\",\r\n  title         = \"Adding Fault Tolerance to {NPB} Benchmarks Using {ULFM}\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.hpdc.org\/2016}\r\n                   {$25^{th}$ ACM International Symposium on High-Performance\r\n                   Parallel and Distributed Computing (HPDC) 2016}:\r\n                   \\href{https:\/\/sites.google.com\/site\/ftxsworkshop\/home\/ftxs-2016}\r\n                   {$6^{th}$ Workshop on Fault Tolerance for HPC at eXtreme\r\n                   Scale (FTXS) 2016}\",\r\n  pages         = \"19--26\",\r\n  month         = may # \"~31 - \" # jun # \"~4, \",\r\n  year          = \"2016\",\r\n  address       = \"Kyoto, Japan\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-4503-4349-7\",\r\n  doi           = \"10.1145\/2909428.2909429\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/parchman16adding.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/parchman16adding.ppt.pdf\",\r\n  abstract      = \"In the world of high-performance computing, fault tolerance\r\n                   and application resilience are becoming some of the primary\r\n                   concerns because of increasing hardware failures and memory\r\n                   corruptions. While the research community has been\r\n                   investigating various options, from system-level solutions to\r\n                   application-level solutions, standards such as the Message\r\n                   Passing Interface (MPI) are also starting at including such\r\n                   capabilities. The current proposal for MPI fault tolerant is\r\n                   centered around the User-Level Failure Mitigation (ULFM)\r\n                   concept, which provides means for fault detection and\r\n                   recovery of the MPI layer. This approach does not address\r\n                   application-level recovery, which is current left to\r\n                   application developers. In this work, we present a\r\n                   modification of some of the benchmarks of the NAS parallel\r\n                   benchmark (NPB) to include support of the ULFM capabilities\r\n                   as well as application- level strategies and mechanisms for\r\n                   application-level failure recovery. As such, we present:\r\n                   (i) an application-level library to ``checkpoint'' data,\r\n                   (ii) extensions of NPB benchmarks for fault tolerance based\r\n                   on different strategies, (iii) a fault injection tool, and\r\n                   (iv) some preliminary experiments that shows the impact of\r\n                   such fault tolerant strategies on the application\r\n                   execution.\",\r\n  pts           = \"62557\"\r\n}<\/pre>\n
<\/a>@conference{naughton14what,\r\n  author        = \"Thomas Naughton\r\n                   and Garry Smith\r\n                   and Christian Engelmann\r\n                   and Geoffroy Vall{\\'e}e\r\n                   and Ferrol Aderholdt\r\n                   and Stephen L. Scott\",\r\n  title         = \"What is the right balance for performance and isolation with\r\n                   virtualization in {HPC}?\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{http:\/\/europar2014.dcc.fc.up.pt}{$20^{th}$\r\n                   European Conference on Parallel and Distributed Computing\r\n                   (Euro-Par) 2014 Workshops}:\r\n                   \\href{http:\/\/www.csm.ornl.gov\/srt\/conferences\/Resilience\/2014}\r\n                   {$7^{th}$ Workshop on Resiliency in High Performance\r\n                   Computing (Resilience) in Clusters, Clouds, and Grids}\",\r\n  volume        = \"8805\",\r\n  pages         = \"570--581\",\r\n  month         = aug # \"~25, \",\r\n  year          = \"2014\",\r\n  address       = \"Porto, Portugal\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-319-14325-5\",\r\n  issn          = \"0302-9743\",\r\n  doi           = \"10.1007\/978-3-319-14325-5_49\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/naughton14what.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/naughton14what.ppt.pdf\",\r\n  abstract      = \"The use of virtualization in high-performance computing (HPC)\r\n                   has been suggested as a means to provide tailored services\r\n                   and added functionality that many users expect from\r\n                   full-featured Linux cluster environments. While the use of\r\n                   virtual machines in HPC can offer several benefits,\r\n                   maintaining performance is a crucial factor. In some\r\n                   instances performance criteria are placed above isolation\r\n                   properties and selective relaxation of isolation for\r\n                   performance is an important characteristic when considering\r\n                   resilience for HPC environments employing virtualization.\r\n                   In this paper we consider some of the factors associated with\r\n                   balancing performance and isolation in configurations that\r\n                   employ virtual machines. In this context, we propose a\r\n                   classification of errors based on the concept of ``error\r\n                   zones'', as well as a detailed analysis of the trade-offs\r\n                   between resilience and performance based on the level of\r\n                   isolation provided by virtualization solutions. Finally, the\r\n                   results from a set of experiments are presented, that use\r\n                   different virtualization solutions, and in doing so allow\r\n                   further elucidation of the topic.\",\r\n  pts           = \"51548\"\r\n}<\/pre>\n
<\/a>@conference{engelmann13toward,\r\n  author        = \"Christian Engelmann\r\n                   and Thomas Naughton\",\r\n  title         = \"Toward a Performance\/Resilience Tool for Hardware\/Software\r\n                   Co-Design of High-Performance Computing Systems\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/icpp2013.ens-lyon.fr}{$42^{nd}$ International\r\n                   Conference on Parallel Processing (ICPP) 2013}:\r\n                   \\href{http:\/\/www.psti-workshop.org} {$4^{th}$ International\r\n                   Workshop on Parallel Software Tools and Tool Infrastructures\r\n                   (PSTI)}\",\r\n  pages         = \"962-971\",\r\n  month         = oct # \"~2, \",\r\n  year          = \"2013\",\r\n  address       = \"Lyon, France\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7695-5117-3\",\r\n  issn          = \"0190-3918\",\r\n  doi           = \"10.1109\/ICPP.2013.114\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann13toward.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann13toward.ppt.pdf\",\r\n  abstract      = \"xSim is a simulation-based performance investigation toolkit\r\n                   that permits running high-performance computing (HPC)\r\n                   applications in a controlled environment with millions of\r\n                   concurrent execution threads, while observing application\r\n                   performance in a simulated extreme-scale system for\r\n                   hardware\/software co-design. The presented work details newly\r\n                   developed features for xSim that permit the injection of MPI\r\n                   process failures, the propagation\/detection\/notification of\r\n                   such failures within the simulation, and their handling using\r\n                   application-level checkpoint\/restart. These new capabilities\r\n                   enable the observation of application behavior and\r\n                   performance under failure within a simulated\r\n                   future-generation HPC system using the most common fault\r\n                   handling technique.\",\r\n  pts           = \"44445\"\r\n}<\/pre>\n
<\/a>@conference{lagadapati13tools,\r\n  author        = \"Mahesh Lagadapati\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\",\r\n  title         = \"Tools for Simulation and Benchmark Generation at Exascale\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/tools.zih.tu-dresden.de\/2013\/}\r\n                   {$7^{th}$ Parallel Tools Workshop}\",\r\n  pages         = \"19--24\",\r\n  month         = sep # \"~3-4, \",\r\n  year          = \"2013\",\r\n  address       = \"Dresden, Germany\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-319-08143-4\",\r\n  doi           = \"10.1007\/978-3-319-08144-1_2\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/lagadapati13tools.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/lagadapati13tools.ppt.pdf\",\r\n  abstract      = \"The path to exascale high-performance computing (HPC) poses several\r\n                   challenges related to power, performance, resilience, productivity,\r\n                   programmability, data movement, and data management. Investigating the\r\n                   performance of parallel applications at scale on future architectures\r\n                   and the performance impact of different architecture choices is an\r\n                   important component of HPC hardware\/software co-design. Simulations\r\n                   using models of future HPC systems and communication traces from\r\n                   applications running on existing HPC systems can offer an insight into\r\n                   the performance of future architectures. This work targets technology\r\n                   developed for scalable application tracing of communication events and\r\n                   memory profiles, but can be extended to other areas, such as I\/O,\r\n                   control flow, and data flow. It further focuses on extreme-scale\r\n                   simulation of millions of Message Passing Interface (MPI) ranks using\r\n                   a lightweight parallel discrete event simulation (PDES) toolkit for\r\n                   performance evaluation. Instead of simply replaying a trace within a\r\n                   simulation, the approach is to generate a benchmark from it and to run\r\n                   this benchmark within a simulation using models to reflect the\r\n                   performance characteristics of future-generation HPC systems. This\r\n                   provides a number of benefits, such as eliminating the data intensive\r\n                   trace replay and enabling simulations at different scales. The\r\n                   presented work utilizes the ScalaTrace tool to generate scalable trace\r\n                   files, the ScalaBenchGen tool to generate the benchmark, and the xSim\r\n                   tool to run the benchmark within a simulation.\",\r\n  pts           = \"48783\"\r\n}<\/pre>\n
<\/a>@conference{naughton13using,\r\n  author        = \"Thomas Naughton\r\n                   and Swen B{\\\"o}hm\r\n                   and Christian Engelmann\r\n                   and Geoffroy Vall{\\'e}e\",\r\n  title         = \"Using Performance Tools to Support Experiments in {HPC}\r\n                   Resilience\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{http:\/\/www.europar2013.org\/}{$19^{th}$\r\n                   European Conference on Parallel and Distributed Computing\r\n                   (Euro-Par) 2013 Workshops}:\r\n                   \\href{http:\/\/xcr.cenit.latech.edu\/resilience2013}{$6^{th}$\r\n                   Workshop on Resiliency in High Performance Computing\r\n                   (Resilience) in Clusters, Clouds, and Grids}\",\r\n  volume        = \"8374\",\r\n  pages         = \"727--736\",\r\n  month         = aug # \"~26, \",\r\n  year          = \"2013\",\r\n  address       = \"Aachen, Germany\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-642-54419-4\",\r\n  issn          = \"0302-9743\",\r\n  doi           = \"10.1007\/978-3-642-54420-0_71\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/naughton13using.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/naughton13using.ppt.pdf\",\r\n  abstract      = \"The high performance computing~(HPC) community is working to\r\n                   address fault tolerance and resilience concerns for current\r\n                   and future large scale computing platforms. This is driving\r\n                   enhancements in the programming environments, specifically\r\n                   research on enhancing message passing libraries to support\r\n                   fault tolerant computing capabilities. The community has\r\n                   also recognized that tools for resilience experimentation\r\n                   are greatly lacking. However, we argue that there are\r\n                   several parallels between ``performance tools'' and\r\n                   ``resilience tools''. As such, we believe the rich set of\r\n                   HPC performance-focused tools can be extended (repurposed)\r\n                   to benefit the resilience community. In this paper, we\r\n                   describe the initial motivation to leverage standard HPC\r\n                   performance analysis techniques to aid in developing\r\n                   diagnostic tools to assist fault tolerance experiments for\r\n                   HPC applications. These diagnosis procedures help to provide\r\n                   context for the system when the errors (failures) occurred.\r\n                   We describe our initial work in leveraging an MPI\r\n                   performance trace tool to assist in providing global context\r\n                   during fault injection experiments. Such tools will assist\r\n                   the HPC resilience community as they extend existing and new\r\n                   application codes to support fault tolerances.\",\r\n  pts           = \"45676\"\r\n}<\/pre>\n
<\/a>@conference{jones11simulation,\r\n  author        = \"Ian S. Jones\r\n                   and Christian Engelmann\",\r\n  title         = \"Simulation of Large-Scale {HPC} Architectures\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/icpp2011.org}{$40^{th}$ International Conference\r\n                   on Parallel Processing (ICPP) 2011}:\r\n                   \\href{http:\/\/www.psti-workshop.org} {$2^{nd}$ International\r\n                   Workshop on Parallel Software Tools and Tool Infrastructures\r\n                   (PSTI)}\",\r\n  pages         = \"447-456\",\r\n  month         = sep # \"~13-19, \",\r\n  year          = \"2011\",\r\n  address       = \"Taipei, Taiwan\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7695-4511-0\",\r\n  issn          = \"1530-2016\",\r\n  doi           = \"10.1109\/ICPPW.2011.44\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/jones11simulation.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/jones11simulation.ppt.pdf\",\r\n  abstract      = \"The Extreme-scale Simulator (xSim) is a recently developed\r\n                   performance investigation toolkit that permits running\r\n                   high-performance computing (HPC) applications in a controlled\r\n                   environment with millions of concurrent execution threads. It\r\n                   allows observing parallel application performance properties\r\n                   in a simulated extreme-scale HPC system to further assist in\r\n                   HPC hardware and application software co-design on the road\r\n                   toward multi-petascale and exascale computing. This paper\r\n                   presents a newly implemented network model for the xSim\r\n                   performance investigation toolkit that is capable of\r\n                   providing simulation support for a variety of HPC network\r\n                   architectures with the appropriate trade-off between\r\n                   simulation scalability and accuracy. The taken approach\r\n                   focuses on a scalable distributed solution with latency and\r\n                   bandwidth restrictions for the simulated network. Different\r\n                   network architectures, such as star, ring, mesh, torus,\r\n                   twisted torus and tree, as well as hierarchical combinations,\r\n                   such as to simulate network-on-chip and network-on-node, are\r\n                   supported. Network traffic congestion modeling is omitted to\r\n                   gain simulation scalability by reducing simulation accuracy.\",\r\n  pts           = \"31901\"\r\n}<\/pre>\n
<\/a>@conference{fiala11tunable,\r\n  author        = \"David Fiala\r\n                   and Kurt Ferreira\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\",\r\n  title         = \"A Tunable, Software-based {DRAM} Error Detection and\r\n                   Correction Library for {HPC}\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{http:\/\/europar2011.bordeaux.inria.fr\/}{$17^{th}$\r\n                   European Conference on Parallel and Distributed Computing\r\n                   (Euro-Par) 2011 Workshops, Part II}:\r\n                   \\href{http:\/\/xcr.cenit.latech.edu\/resilience2011}{$4^{th}$\r\n                   Workshop on Resiliency in High Performance Computing\r\n                   (Resilience) in Clusters, Clouds, and Grids}\",\r\n  volume        = \"7156\",\r\n  pages         = \"251-261\",\r\n  month         = aug # \"~29 - \" # sep # \"~2, \",\r\n  year          = \"2011\",\r\n  address       = \"Bordeaux, France\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-642-29740-3\",\r\n  doi           = \"10.1007\/978-3-642-29740-3_29\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/fiala11tunable.pdf\",\r\n  url2          = \"\",\r\n  abstract      = \"Proposed exascale systems will present a number of\r\n                   considerable resiliency challenges. In particular, DRAM\r\n                   soft-errors, or bit-flips, are expected to greatly increase\r\n                   due to the increased memory density of these systems.\r\n                   Current hardware-based fault-tolerance methods will be\r\n                   unsuitable for addressing the expected soft error frequency\r\n                   rate. As a result, additional software will be needed to\r\n                   address this challenge. In this paper we introduce LIBSDC,\r\n                   a tunable, transparent silent data corruption detection and\r\n                   correction library for HPC applications. LIBSDC provides\r\n                   comprehensive SDC protection for program memory by\r\n                   implementing on-demand page integrity verification.\r\n                   Experimental benchmarks with Mantevo HPCCG show that once\r\n                   tuned, LIBSDC is able to achieve SDC protection with 50\\%\r\n                   overhead of resources, less than the 100\\% needed for double\r\n                   modular redundancy.\",\r\n  pts           = \"35631\"\r\n}<\/pre>\n
<\/a>@conference{naughton11case,\r\n  author        = \"Thomas Naughton\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"A Case for Virtual Machine based Fault Injection in a\r\n                   High-Performance Computing Environment\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{http:\/\/europar2011.bordeaux.inria.fr\/}{$17^{th}$\r\n                   European Conference on Parallel and Distributed Computing\r\n                   (Euro-Par) 2011}:\r\n                   \\href{http:\/\/www.csm.ornl.gov\/srt\/conferences\/hpcvirt2011}\r\n                   {$5^{th}$ Workshop on System-level Virtualization for High\r\n                   Performance Computing (HPCVirt)}\",\r\n  volume        = \"7155\",\r\n  pages         = \"234-243\",\r\n  month         = aug # \"~29 - \" # sep # \"~2, \",\r\n  year          = \"2011\",\r\n  address       = \"Bordeaux, France\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-642-29737\",\r\n  doi           = \"10.1007\/978-3-642-29737-3_27\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/naughton11case.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/naughton11case.ppt.pdf\",\r\n  abstract      = \"Large-scale computing platforms provide tremendous\r\n                   capabilities for scientific discovery. These systems have\r\n                   hundreds of thousands of computing cores, hundreds of\r\n                   terabytes of memory, and enormous high-performance\r\n                   interconnection networks. These systems are facing enormous\r\n                   challenges to achieve performance at such scale. Failures\r\n                   are an Achilles heel of these enormous systems. As\r\n                   applications and system software scale up to multi-petaflop\r\n                   and beyond to exascale platforms, the occurrence of failure\r\n                   will be much more common. This has given rise to a push in\r\n                   fault-tolerance and resilience research for HPC systems.\r\n                   This includes work on log analysis to identify types of\r\n                   failures, enhancements to the Message Passing Interface\r\n                   (MPI) to incorporate fault awareness, and a variety of\r\n                   fault tolerance mechanisms that span redundant computation,\r\n                   algorithm based fault tolerance, and advanced checkpoint\/\r\n                   restart techniques. While there is much work to be done on\r\n                   the FT\/Resilience mechanisms for such large-scale systems,\r\n                   there is also a profound gap in the tools for\r\n                   experimentation. This gap is compounded by the fact that HPC\r\n                   environments have stringent performance requirements and are\r\n                   often highly customized. The tool chain for these systems are\r\n                   often tailored for the platform and while the majority of\r\n                   systems on the Top500 Supercomputer list run Linux, these\r\n                   operating environments typically contain many site\/machine\r\n                   specific enhancements. Therefore, it is desirable to maintain\r\n                   a consistent execution environment to minimize end-user\r\n                   (scientist) interruption. The work on system-level\r\n                   virtualization for HPC system offers a unique opportunity to\r\n                   maintain a consistent execution environment via a virtual\r\n                   machine (VM). Recent work on virtualization for HPC has shown\r\n                   that low-overhead, high performance systems can be realized\r\n                   [1, 2] Virtualization also provides a clean abstraction for\r\n                   building experimental tools for investigation into the\r\n                   effects of failures in HPC and the related research on FT\/\r\n                   Resilience mechanisms and policies. In this paper we discuss\r\n                   the motivation for tools to perform fault injection in an HPC\r\n                   context, and outline an approach that can leverage\r\n                   virtualization.\",\r\n  pts           = \"32309\"\r\n}<\/pre>\n
<\/a>@conference{engelmann10facilitating,\r\n  author        = \"Christian Engelmann\r\n                   and Frank Lauer\",\r\n  title         = \"Facilitating Co-Design for Extreme-Scale Systems Through\r\n                   Lightweight Simulation\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.cluster2010.org}{$12^{th}$ IEEE\r\n                   International Conference on Cluster Computing (Cluster)\r\n                   2010}: \\href{http:\/\/www2.wmin.ac.uk\/getovv\/aacec10.html}\r\n                   {$1^{st}$ Workshop on Application\/Architecture Co-design for\r\n                   Extreme-scale Computing (AACEC)}\",\r\n  pages         = \"1-8\",\r\n  month         = sep # \"~20-24, \",\r\n  year          = \"2010\",\r\n  address       = \"Hersonissos, Crete, Greece\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-1-4244-8395-2\",\r\n  doi           = \"10.1109\/CLUSTERWKSP.2010.5613113\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann10facilitating.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann10facilitating.ppt.pdf\",\r\n  abstract      = \"This work focuses on tools for investigating algorithm\r\n                   performance at extreme scale with millions of concurrent\r\n                   threads and for evaluating the impact of future architecture\r\n                   choices to facilitate the co-design of high-performance\r\n                   computing (HPC) architectures and applications. The approach\r\n                   focuses on lightweight simulation of extreme-scale HPC\r\n                   systems with the needed amount of accuracy. The prototype\r\n                   presented in this paper is able to provide this capability\r\n                   using a parallel discrete event simulation (PDES), such that\r\n                   a Message Passing Interface (MPI) application can be executed\r\n                   at extreme scale, and its performance properties can be\r\n                   evaluated. The results of an initial prototype are\r\n                   encouraging as a simple hello world MPI program could be\r\n                   scaled up to 1,048,576 virtual MPI processes on a four-node\r\n                   cluster, and the performance properties of two MPI programs\r\n                   could be evaluated at up to 1,024 and 16,384 virtual MPI\r\n                   processes on the same system.\",\r\n  pts           = \"25331\"\r\n}<\/pre>\n
<\/a>@conference{ostrouchov09nonparametric,\r\n  author        = \"George Ostrouchov\r\n                   and Thomas Naughton\r\n                   and Christian Engelmann\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Stephen L. Scott\",\r\n  title         = \"Nonparametric Multivariate Anomaly Analysis in Support of\r\n                   {HPC} Resilience\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/www.oerc.ox.ac.uk\/ieee}\r\n                   {$5^{th}$ IEEE International Conference on e-Science\r\n                   (e-Science) 2009}:\r\n                   \\href{http:\/\/www.oerc.ox.ac.uk\/ieee\/workshops\/workshops\/computational-science}\r\n                   {Workshop on Computational Science}\",\r\n  pages         = \"80-85\",\r\n  month         = dec # \"~9-11, \",\r\n  year          = \"2009\",\r\n  address       = \"Oxford, UK\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-1-4244-5946-9\",\r\n  doi           = \"10.1109\/ESCIW.2009.5407992\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/ostrouchov09nonparametric.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/ostrouchov09nonparametric.ppt.pdf\",\r\n  abstract      = \"Large-scale computing systems provide great potential for\r\n                   scientific exploration. However, the complexity that\r\n                   accompanies these enormous machines raises challeges for\r\n                   both, users and operators. The effective use of such systems\r\n                   is often hampered by failures encountered when running\r\n                   applications on systems containing tens-of-thousands of nodes\r\n                   and hundreds-of-thousands of compute cores capable of\r\n                   yielding petaflops of performance. In systems of this size\r\n                   failure detection is complicated and root-cause diagnosis\r\n                   difficult. This paper describes our recent work in the\r\n                   identification of anomalies in monitoring data and system\r\n                   logs to provide further insights into machine status, runtime\r\n                   behavior, failure modes and failure root causes. It discusses\r\n                   the details of an initial prototype that gathers the data and\r\n                   uses statistical techniques for analysis.\",\r\n  pts           = \"26081\"\r\n}<\/pre>\n
<\/a>@conference{naughton09fault,\r\n  author        = \"Thomas Naughton\r\n                   and Wesley Bland\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"Fault Injection Framework for System Resilience Evaluation --\r\n                   {F}ake Faults for Finding Future Failures\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.lrz-muenchen.de\/hpdc2009}{$18^{th}$\r\n                   International Symposium on High Performance Distributed\r\n                   Computing (HPDC) 2009}:\r\n                   \\href{http:\/\/xcr.cenit.latech.edu\/resilience2009}{$2^{nd}$\r\n                   Workshop on Resiliency in High Performance Computing\r\n                   (Resilience) 2009}\",\r\n  pages         = \"23--28\",\r\n  month         = jun # \"~9, \",\r\n  year          = \"2009\",\r\n  address       = \"Munich, Germany\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-60558-587-1\",\r\n  doi           = \"10.1145\/1552526.1552530\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/naughton09fault.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/naughton09fault.ppt.pdf\",\r\n  abstract      = \"As high-performance computing (HPC) systems increase in size\r\n                   and complexity they become more difficult to manage. The\r\n                   enormous component counts associated with these large systems\r\n                   lead to significant challenges in system reliability and\r\n                   availability. This in turn is driving research into the\r\n                   resilience of large scale systems, which seeks to curb the\r\n                   effects of increased failures at large scales by masking the\r\n                   inevitable faults in these systems. The basic premise being\r\n                   that failure must be accepted as a reality of large scale\r\n                   system and coped with accordingly through system resilience.\r\n                   A key component in the development and evaluation of system\r\n                   resilience techniques is having a means to conduct controlled\r\n                   experiments. A common method for performing such experiments\r\n                   is to generate synthetic faults and study the resulting\r\n                   effects. In this paper we discuss the motivation and our\r\n                   initial use of software fault injection to support the\r\n                   evaluation of resilience for HPC systems. We mention\r\n                   background and related work in the area and discuss the\r\n                   design of a tool to aid in fault injection experiments for\r\n                   both user-space (application-level) and system-level\r\n                   failures.\"\r\n}<\/pre>\n
<\/a>@conference{tikotekar09performance,\r\n  author        = \"Anand Tikotekar\r\n                   and Hong H. Ong\r\n                   and Sadaf Alam\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"Performance Comparison of Two Virtual Machine Scenarios Using\r\n                   an {HPC} Application -- {A} Case study Using Molecular\r\n                   Dynamics Simulations\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.csm.ornl.gov\/srt\/hpcvirt09}{$3^{rd}$\r\n                   Workshop on System-level Virtualization for High Performance\r\n                   Computing (HPCVirt) 2009}, in conjunction with the\r\n                   \\href{http:\/\/www.eurosys.org\/2009}{$4^{th}$ ACM SIGOPS\r\n                   European Conference on Computer Systems (EuroSys) 2009}\",\r\n  pages         = \"33--40\",\r\n  month         = mar # \"~30, \",\r\n  year          = \"2009\",\r\n  address       = \"Nuremberg, Germany\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-60558-465-2\",\r\n  doi           = \"10.1145\/1519138.1519143\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/tikotekar09performance.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/tikotekar09performance.ppt.pdf\",\r\n  abstract      = \"Obtaining high flexibility to performance-loss ratio is a\r\n                   key challenge of today's HPC virtual environment landscape.\r\n                   And while extensive research has been targeted at extracting\r\n                   more performance from virtual machines, the idea that whether\r\n                   novel virtual machine usage scenarios could lead to high\r\n                   flexibility Vs performance trade-off has received less\r\n                   attention. We, in this paper, take a step forward by studying\r\n                   and comparing the performance implications of running the\r\n                   Large-scale Atomic\/Molecular Massively Parallel Simulator\r\n                   (LAMMPS) application on two virtual machine configurations.\r\n                   First configuration consists of two virtual machines per node\r\n                   with 1 application process per virtual machine. The second\r\n                   configuration consists of 1 virtual machine per node with 2\r\n                   processes per virtual machine. Xen has been used as an\r\n                   hypervisor and standard Linux as a guest virtual machine. Our\r\n                   results show that the difference in overall performance\r\n                   impact on LAMMPS between the two virtual machine\r\n                   configurations described above is around 3\\%. We also study\r\n                   the difference in performance impact in terms of each\r\n                   configuration's individual metrics such as CPU, I\/O, Memory,\r\n                   and interrupt\/context switches.\"\r\n}<\/pre>\n
<\/a>@conference{vallee08virtual,\r\n  author        = \"Geoffroy R. Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Hong H. Ong\r\n                   and Anand Tikotekar\r\n                   and Christian Engelmann\r\n                   and Wesley Bland\r\n                   and Ferrol Aderholt\r\n                   and Stephen L. Scott\",\r\n  title         = \"Virtual System Environments\",\r\n  booktitle     = \"Communications in Computer and Information Science:\r\n                   Proceedings of the \\href{http:\/\/www.dmtf.org\/svm08}{$2^{nd}$\r\n                   DMTF Academic Alliance Workshop on Systems and Virtualization\r\n                   Management: Standards and New Technologies (SVM) 2008}\",\r\n  volume        = \"18\",\r\n  pages         = \"72--83\",\r\n  month         = oct # \"~21-22, \",\r\n  year          = \"2008\",\r\n  address       = \"Munich, Germany\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-540-88707-2\",\r\n  issn          = \"1865-0929\",\r\n  doi           = \"10.1007\/978-3-540-88708-9_7\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/vallee08virtual.pdf\",\r\n  url2          = \"\",\r\n  abstract      = \"Distributed and parallel systems are typically managed with\r\n                   static settings: the operating system (OS) and the runtime\r\n                   environment (RTE) are specified at a given time and cannot be\r\n                   changed to fit an application`s needs. This means that every\r\n                   time application developers want to use their application on\r\n                   a new execution platform, the application has to be ported to\r\n                   this new environment, which may be expensive in terms of\r\n                   application modifications and developer time. However, the\r\n                   science resides in the applications and not in the OS or the\r\n                   RTE. Therefore, it should be beneficial to adapt the OS and\r\n                   the RTE to the application instead of adapting the\r\n                   applications to the OS and the RTE. This document presents\r\n                   the concept of Virtual System Environments (VSE), which\r\n                   enables application developers to specify and create a\r\n                   virtual environment that properly fits their application`s\r\n                   needs. For that four challenges have to be addressed: (i)\r\n                   definition of the VSE itself by the application developers,\r\n                   (ii) deployment of the VSE, (iii) system administration for\r\n                   the platform, and (iv) protection of the platform from the\r\n                   running VSE. We therefore present an integrated tool for the\r\n                   definition and deployment of VSEs on top of traditional and\r\n                   virtual (i.e., using system-level virtualization) execution\r\n                   platforms. This tool provides the capability to choose the\r\n                   degree of delegation for system administration tasks and the\r\n                   degree of protection from the application (e.g., using\r\n                   virtual machines). To summarize, the VSE concept enables the\r\n                   customization of the OS\/RTE used for the execution of\r\n                   application by users without compromising local system\r\n                   administration rules and execution platform protection\r\n                   constraints.\",\r\n  pts           = \"28239\"\r\n\r\n}<\/pre>\n
<\/a>@conference{tikotekar08analysis,\r\n  author        = \"Anand Tikotekar\r\n                   and Geoffroy Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Hong H. Ong\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"An Analysis of {HPC} Benchmark Applications in Virtual\r\n                   Machine Environments\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{http:\/\/europar2008.caos.uab.es}{$14^{th}$ European\r\n                   Conference on Parallel and Distributed Computing (Euro-Par)\r\n                   2008}: \\href{http:\/\/scilytics.com\/vhpc}{$3^{rd}$ Workshop on\r\n                   Virtualization in High-Performance Cluster and Grid Computing\r\n                   (VHPC) 2008}\",\r\n  volume        = \"5415\",\r\n  pages         = \"63--71\",\r\n  month         = aug # \"~26-29, \",\r\n  year          = \"2008\",\r\n  address       = \"Las Palmas de Gran Canaria, Spain\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"978-3-642-00954-9\",\r\n  doi           = \"10.1007\/978-3-642-00955-6\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/tikotekar08analysis.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/tikotekar08analysis.ppt.pdf\",\r\n  abstract      = \"Virtualization technology has been gaining acceptance in the\r\n                   scientific community due to its overall flexibility in\r\n                   running HPC applications. It has been reported that a\r\n                   specific class of applications is better suited to a\r\n                   particular type of virtualization scheme or implementation.\r\n                   For example, Xen has been shown to perform with little\r\n                   overhead for compute-bound applications. Such a study,\r\n                   although useful, does not allow us to generalize conclusions\r\n                   beyond the performance analysis of that application which is\r\n                   explicitly executed. An explanation of why the generalization\r\n                   described above is difficult, may be due to the versatility\r\n                   in applications, which leads to different overheads in\r\n                   virtual environments. For example, two similar applications\r\n                   may spend disproportionate amount of time in their respective\r\n                   library code when run in virtual environments. In this paper,\r\n                   we aim to study such potential causes by investigating the\r\n                   behavior and identifying patterns of various overheads for\r\n                   HPC benchmark applications. Based on the investigation of the\r\n                   overhead profiles for different benchmarks, we aim to address\r\n                   questions such as: Are the overhead profiles for a particular\r\n                   type of benchmarks (such as compute-bound) similar or are\r\n                   there grounds to conclude otherwise?\"\r\n}<\/pre>\n
<\/a>@conference{engelmann08symmetric2,\r\n  author        = \"Christian Engelmann\r\n                   and Stephen L. Scott\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and Xubin (Ben) He\",\r\n  title         = \"Symmetric Active\/Active High Availability for\r\n                   High-Performance Computing System Services: Accomplishments\r\n                   and Limitations\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.ens-lyon.fr\/LIP\/RESO\/ccgrid2008}{$8^{th}$\r\n                   IEEE International Symposium on Cluster Computing and the\r\n                   Grid (CCGrid) 2008}:\r\n                   \\href{http:\/\/xcr.cenit.latech.edu\/resilience2008}{Workshop on\r\n                   Resiliency in High Performance Computing (Resilience) 2008}\",\r\n  pages         = \"813--818\",\r\n  month         = may # \"~19-22, \",\r\n  year          = \"2008\",\r\n  address       = \"Lyon, France\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"978-0-7695-3156-4\",\r\n  doi           = \"10.1109\/CCGRID.2008.78\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann08symmetric2.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann08symmetric2.pdf\",\r\n  abstract      = \"This paper summarizes our efforts over the last 3-4 years in\r\n                   providing symmetric active\/active high availability for\r\n                   high-performance computing (HPC) system services. This work\r\n                   paves the way for high-level reliability, availability and\r\n                   serviceability in extreme-scale HPC systems by focusing on\r\n                   the most critical components, head and service nodes, and by\r\n                   reinforcing them with appropriate high availability\r\n                   solutions. This paper presents our accomplishments in the\r\n                   form of concepts and respective prototypes, discusses\r\n                   existing limitations, outlines possible future work, and\r\n                   describes the relevance of this research to other, planned\r\n                   efforts.\",\r\n  pts           = \"9996\"\r\n}<\/pre>\n
<\/a>@conference{chen08online,\r\n  author        = \"Xin Chen\r\n                   and Benjamin Eckart\r\n                   and Xubin (Ben) He\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"An Online Controller Towards Self-Adaptive File System\r\n                   Availability and Performance\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/xcr.cenit.latech.edu\/hapcw2008}{$5^{th}$ High\r\n                   Availability and Performance Workshop (HAPCW) 2008}, in\r\n                   conjunction with the \\href{http:\/\/www.hpcsw.org}{$1^{st}$\r\n                   High-Performance Computer Science Week (HPCSW) 2008}\",\r\n  month         = apr # \"~3-4, \",\r\n  year          = \"2008\",\r\n  address       = \"Denver, CO, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/chen08online.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/chen08online.ppt.pdf\",\r\n  abstract      = \"At the present time, it can be a significant challenge to\r\n                   build a large-scale distributed file system that\r\n                   simultaneously maintains both high availability and high\r\n                   performance. Although many fault tolerance technologies have\r\n                   been proposed and used in both commercial and academic\r\n                   distributed file systems to achieve high availability, most\r\n                   of them typically sacrifice performance for higher system\r\n                   availability. Additionally, recent studies show that system\r\n                   availability and performance are related to the system\r\n                   workload. In this paper, we analyze the correlations among\r\n                   availability, performance, and workloads based on a\r\n                   replication strategy, and we discuss the trade off between\r\n                   availability and performance with different workloads. Our\r\n                   analysis leads to the design of an online controller that can\r\n                   dynamically achieve optimal performance and availability by\r\n                   tuning the system replication policy.\"\r\n}<\/pre>\n
<\/a>@conference{tikotekar08effects,\r\n  author        = \"Anand Tikotekar\r\n                   and Geoffroy Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Hong H. Ong\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\r\n                   and Anthony M. Filippi\",\r\n  title         = \"Effects of Virtualization on a Scientific Application --\r\n                   {R}unning a Hyperspectral Radiative Transfer Code on Virtual\r\n                   Machines\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.csm.ornl.gov\/srt\/hpcvirt08}{$2^{nd}$\r\n                   Workshop on System-level Virtualization for High Performance\r\n                   Computing (HPCVirt) 2008}, in conjunction with the\r\n                   \\href{http:\/\/www.eurosys.org\/2008}{$3^{rd}$ ACM SIGOPS\r\n                   European Conference on Computer Systems (EuroSys) 2008}\",\r\n  pages         = \"16--23\",\r\n  month         = mar # \"~31, \",\r\n  year          = \"2008\",\r\n  address       = \"Glasgow, UK\",\r\n  publisher     = \"\\href{http:\/\/www.acm.org}{ACM Press, New York, NY, USA}\",\r\n  isbn          = \"978-1-60558-120-0\",\r\n  doi           = \"10.1145\/1435452.1435455\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/tikotekar08effects.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/tikotekar08effects.ppt.pdf\",\r\n  abstract      = \"The topic of system-level virtualization has recently begun\r\n                   to receive interest for high performance computing (HPC).\r\n                   This is in part due to the isolation and encapsulation\r\n                   offered by the virtual machine. These traits enable\r\n                   applications to customize their environments and maintain\r\n                   consistent software configurations in their virtual domains.\r\n                   Additionally, there are mechanisms that can be used for fault\r\n                   tolerance like live virtual machine migration. Given these\r\n                   attractive benefits to virtualization, a fundamental question\r\n                   arises, how does this effect my scientific application? We\r\n                   use this as the premise for our paper and observe a\r\n                   real-world scientific code running on a Xen virtual machine.\r\n                   We studied the effects of running a radiative transfer\r\n                   simulation, Hydrolight, on a virtual machine. We discuss our\r\n                   methodology and report observations regarding the usage of\r\n                   virtualization with this application.\"\r\n}<\/pre>\n
<\/a>@conference{engelmann07middleware,\r\n  author        = \"Christian Engelmann\r\n                   and Hong H. Ong\r\n                   and Stephen L. Scott\",\r\n  title         = \"Middleware in Modern High Performance Computing System\r\n                   Architectures\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{http:\/\/www.iccs-meeting.org\/iccs2007}{$7^{th}$\r\n                   International Conference on Computational Science (ICCS)\r\n                   2007}, Part II: \\href{http:\/\/www.gup.uni-linz.ac.at\/cce2007}\r\n                   {$4^{th}$ Special Session on Collaborative and Cooperative\r\n                   Environments (CCE) 2007}\",\r\n  volume        = \"4488\",\r\n  pages         = \"784--791\",\r\n  month         = may # \"~27-30, \",\r\n  year          = \"2007\",\r\n  address       = \"Beijing, China\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"3-5407-2585-5\",\r\n  issn          = \"0302-9743\",\r\n  doi           = \"10.1007\/978-3-540-72586-2_111\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann07middleware.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann07middleware.ppt.pdf\",\r\n  abstract      = \"A recent trend in modern high performance computing (HPC)\r\n                   system architectures employs lean compute nodes running a\r\n                   lightweight operating system (OS). Certain parts of the OS a\r\n                   well as other system software services are moved to service\r\n                   nodes in order to increase performance and scalability. This\r\n                   paper examines the impact of this HPC system architecture\r\n                   trend on HPC middleware software solutions, which\r\n                   traditionally equip HPC systems with advanced features, such\r\n                   as parallel and distributed programming models, appropriate\r\n                   system resource management mechanisms, remote application\r\n                   steering and user interaction techniques. Since the approach\r\n                   of keeping the compute node software stack small and simple\r\n                   is orthogonal to the middleware concept of adding missing OS\r\n                   features between OS and application, the role and\r\n                   architecture of middleware in modern HPC systems needs to be\r\n                   revisited. The result is a paradigm shift in HPC middleware\r\n                   design, where single middleware services are moved to service\r\n                   nodes, while runtime environments (RTEs) continue to reside\r\n                   on compute nodes.\",\r\n  pts           = \"5260\"\r\n}<\/pre>\n
<\/a>@conference{engelmann07transparent,\r\n  author        = \"Christian Engelmann\r\n                   and Stephen L. Scott\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and Xubin (Ben) He\",\r\n  title         = \"Transparent Symmetric Active\/Active Replication for\r\n                   Service-Level High Availability\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/ccgrid07.lncc.br}{$7^{th}$\r\n                   IEEE International Symposium on Cluster Computing and the\r\n                   Grid (CCGrid) 2007}: \\href{http:\/\/www.lri.fr\/~fedak\/gp2pc-07}\r\n                   {$7^{th}$ International Workshop on Global and Peer-to-Peer\r\n                   Computing (GP2PC) 2007}\",\r\n  pages         = \"755--760\",\r\n  month         = may # \"~14-17, \",\r\n  year          = \"2007\",\r\n  address       = \"Rio de Janeiro, Brazil\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"0-7695-2833-3\",\r\n  doi           = \"10.1109\/CCGRID.2007.116\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann07transparent.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann07transparent.ppt.pdf\",\r\n  abstract      = \"As service-oriented architectures become more important in\r\n                   parallel and distributed computing systems, individual\r\n                   service instance reliability as well as appropriate service\r\n                   redundancy becomes an essential necessity in order to\r\n                   increase overall system availability. This paper focuses on\r\n                   providing redundancy strategies using service-level\r\n                   replication techniques. Based on previous research using\r\n                   symmetric active\/active replication, this paper proposes a\r\n                   transparent symmetric active\/active replication approach that\r\n                   allows for more reuse of code between individual\r\n                   service-level replication implementations by using a virtual\r\n                   communication layer. Service- and client-side interceptors\r\n                   are utilized in order to provide total transparency. Clients\r\n                   and servers are unaware of the replication infrastructure as\r\n                   it provides all necessary mechanisms internally.\",\r\n  pts           = \"5259\"\r\n}<\/pre>\n
<\/a>@conference{engelmann07configurable,\r\n  author        = \"Christian Engelmann\r\n                   and Stephen L. Scott\r\n                   and Hong H. Ong\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Thomas Naughton\",\r\n  title         = \"Configurable Virtualized System Environments for High\r\n                   Performance Computing\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.csm.ornl.gov\/srt\/hpcvirt07}{$1^{st}$\r\n                   Workshop on System-level Virtualization for High Performance\r\n                   Computing (HPCVirt) 2007}, in conjunction with the\r\n                   \\href{http:\/\/www.eurosys.org\/2008}{$2^{nd}$ ACM SIGOPS\r\n                   European Conference on Computer Systems (EuroSys) 2007}\",\r\n  month         = mar # \"~20, \",\r\n  year          = \"2007\",\r\n  address       = \"Lisbon, Portugal\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann07configurable.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann07configurable.ppt.pdf\",\r\n  abstract      = \"Existing challenges for current terascale high performance\r\n                   computing (HPC) systems are increasingly hampering the\r\n                   development and deployment efforts of system software and\r\n                   scientific applications for next-generation petascale\r\n                   systems. The expected rapid system upgrade interval toward\r\n                   petascale scientific computing demands an incremental\r\n                   strategy for the development and deployment of legacy and new\r\n                   large-scale scientific applications that avoids excessive\r\n                   porting. Furthermore, system software developers as well as\r\n                   scientific application developers require access to\r\n                   large-scale testbed environments in order to test individual\r\n                   solutions at scale. This paper proposes to address these\r\n                   issues at the system software level through the development\r\n                   of a virtualized system environment (VSE) for scientific\r\n                   computing. The proposed VSE approach enables\r\n                   plug-and-play supercomputing through\r\n                   desktop-to-cluster-to-petaflop computer system-level\r\n                   virtualization based on recent advances in hypervisor\r\n                   virtualization technologies. This paper describes the VSE\r\n                   system architecture in detail, discusses needed tools for\r\n                   VSE system management and configuration, and presents\r\n                   respective VSE use case scenarios.\",\r\n  pts           = \"5703\"\r\n}<\/pre>\n
<\/a>@conference{engelmann06towards,\r\n  author        = \"Christian Engelmann\r\n                   and Stephen L. Scott\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and Xubin (Ben) He\",\r\n  title         = \"Towards High Availability for High-Performance Computing\r\n                   System Services: {A}ccomplishments and Limitations\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/xcr.cenit.latech.edu\/hapcw2006}{$4^{th}$ High\r\n                   Availability and Performance Workshop (HAPCW) 2006}, in\r\n                   conjunction with the \\href{http:\/\/lacsi.krellinst.org}\r\n                   {$7^{th}$ Los Alamos Computer Science Institute (LACSI)\r\n                   Symposium 2006}\",\r\n  month         = oct # \"~17, \",\r\n  year          = \"2006\",\r\n  address       = \"Santa Fe, NM, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann06towards.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann06towards.ppt.pdf\",\r\n  abstract      = \"During the last several years, our teams at Oak Ridge\r\n                   National Laboratory, Louisiana Tech University, and Tennessee\r\n                   Technological University focused on efficient redundancy\r\n                   strategies for head and service nodes of high-performance\r\n                   computing (HPC) systems in order to pave the way for high\r\n                   availability (HA) in HPC. These nodes typically run critical\r\n                   HPC system services, like job and resource management, and\r\n                   represent single points of failure and control for an entire\r\n                   HPC system. The overarching goal of our research is to\r\n                   provide high-level reliability, availability, and\r\n                   serviceability (RAS) for HPC systems by combining HA and HPC\r\n                   technology. This paper summarizes our accomplishments, such\r\n                   as developed concepts and implemented proof-of-concept\r\n                   prototypes, and describes existing limitations, such as\r\n                   performance issues, which need to be dealt with for\r\n                   production-type deployment.\",\r\n  pts           = \"3736\"\r\n}<\/pre>\n
<\/a>@conference{ou06achieving,\r\n  author        = \"Li Ou\r\n                   and Xin Chen\r\n                   and Xubin (Ben) He\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"Achieving Computational {I\/O} Effciency in a High Performance\r\n                   Cluster Using Multicore Processors\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/xcr.cenit.latech.edu\/hapcw2006}{$4^{th}$ High\r\n                   Availability and Performance Workshop (HAPCW) 2006}, in\r\n                   conjunction with the \\href{http:\/\/lacsi.krellinst.org}\r\n                   {$7^{th}$ Los Alamos Computer Science Institute (LACSI)\r\n                   Symposium 2006}\",\r\n  month         = oct # \"~17, \",\r\n  year          = \"2006\",\r\n  address       = \"Santa Fe, NM, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/ou06achieving.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/ou06achieving.ppt.pdf\",\r\n  abstract      = \"Cluster computing has become one of the most popular\r\n                   platforms for high-performance computing today. The recent\r\n                   popularity of multicore processors provides a flexible way to\r\n                   increase the computational capability of clusters. Although\r\n                   the system performance may improve with multicore processors\r\n                   in a cluster, I\/O requests initiated by multiple cores may\r\n                   saturate the I\/O bus, and furthermore increase the latency by\r\n                   issuing  multiple non-contiguous disk accesses. In this\r\n                   paper, we propose an asymmetric collective I\/O for multicore\r\n                   processors to improve multiple non-contiguous accesses. In\r\n                   our configuration, one core in each multicore processor is\r\n                   designated as the coordinator, and others serve as computing\r\n                   cores. The coordinator is responsible for aggregating I\/O\r\n                   operations from computing cores and submitting a contiguous\r\n                   request. The coordinator allocates contiguous memory buffers\r\n                   on behalf of other cores to avoid redundant data copies.\",\r\n  pts           = \"4222\"\r\n}<\/pre>\n
<\/a>@conference{engelmann06rmix,\r\n  author        = \"Christian Engelmann\r\n                   and George A. (Al) Geist\",\r\n  title         = \"{RMIX}: {A} Dynamic, Heterogeneous, Reconfigurable\r\n                   Communication Framework\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{http:\/\/www.iccs-meeting.org\/iccs2006}{$6^{th}$\r\n                   International Conference on Computational Science (ICCS)\r\n                   2006}, Part II: \\href{http:\/\/www.gup.uni-linz.ac.at\/cce2006}\r\n                   {$3^{rd}$ Special Session on Collaborative and Cooperative\r\n                   Environments (CCE) 2006}\",\r\n  volume        = \"3992\",\r\n  pages         = \"573--580\",\r\n  month         = may # \"~28-31, \",\r\n  year          = \"2006\",\r\n  address       = \"Reading, UK\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"3-540-34381-4\",\r\n  issn          = \"0302-9743\",\r\n  doi           = \"10.1007\/11758525_77\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann06rmix.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann06rmix.ppt.pdf\",\r\n  abstract      = \"RMIX is a dynamic, heterogeneous, reconfigurable\r\n                   communication framework that allows software components to\r\n                   communicate using various RMI\/RPC protocols, such as ONC RPC,\r\n                   Java RMI and SOAP, by facilitating dynamically loadable\r\n                   provider plug-ins to supply different protocol stacks. With\r\n                   this paper, we present a native (C-based), flexible,\r\n                   adaptable, multi-protocol RMI\/RPC communication framework\r\n                   that complements the Java-based RMIX variant previously\r\n                   developed by our partner team at Emory University. Our\r\n                   approach offers the same multi-protocol RMI\/RPC services\r\n                   and advanced invocation semantics via a C-based interface\r\n                   that does not require an object-oriented programming\r\n                   language. This paper provides a detailed description of our\r\n                   RMIX framework architecture and some of its features. It\r\n                   describes the general use case of the RMIX framework and its\r\n                   integration into the Harness metacomputing environment in the\r\n                   form of a plug-in.\",\r\n  pts           = \"1490\"\r\n}<\/pre>\n
<\/a>@conference{engelmann06active,\r\n  author        = \"Christian Engelmann\r\n                   and Stephen L. Scott\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and Xubin (Ben) He\",\r\n  title         = \"Active\/Active Replication for Highly Available {HPC} System\r\n                   Services\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.ares-conference.eu\/ares2006}{$1^{st}$\r\n                   International Conference on Availability, Reliability and\r\n                   Security (ARES) 2006}: $1^{st}$ International Workshop on\r\n                   Frontiers in Availability, Reliability and Security (FARES)\r\n                   2006\",\r\n  pages         = \"639-645\",\r\n  month         = apr # \"~20-22, \",\r\n  year          = \"2006\",\r\n  address       = \"Vienna, Austria\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"0-7695-2567-9\",\r\n  doi           = \"10.1109\/ARES.2006.23\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann06active.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann06active.ppt.pdf\",\r\n  abstract      = \"Today`s high performance computing systems have several\r\n                   reliability deficiencies resulting in availability and\r\n                   serviceability issues. Head and service nodes represent a\r\n                   single point of failure and control for an entire system as\r\n                   they render it inaccessible and unmanageable in case of a\r\n                   failure until repair, causing a significant downtime. This\r\n                   paper introduces two distinct replication methods (internal\r\n                   and external) for providing symmetric active\/active high\r\n                   availability for multiple head and service nodes running in\r\n                   virtual synchrony. It presents a comparison of both methods\r\n                   in terms of expected correctness, ease-of-use and performance\r\n                   based on early results from ongoing work in providing\r\n                   symmetric active\/active high availability for two HPC system\r\n                   services (TORQUE and PVFS metadata server). It continues with\r\n                   a short description of a distributed mutual exclusion\r\n                   algorithm and a brief statement regarding the handling of\r\n                   Byzantine failures. This paper concludes with an overview of\r\n                   past and ongoing work, and a short summary of the presented\r\n                   research.\",\r\n  pts           = \"1485\"\r\n}<\/pre>\n
<\/a>@conference{engelmann05concepts,\r\n  author        = \"Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"Concepts for High Availability in Scientific High-End\r\n                   Computing\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/xcr.cenit.latech.edu\/hapcw2005}{$3^{rd}$ High\r\n                   Availability and Performance Workshop (HAPCW) 2005}, in\r\n                   conjunction with the\r\n                   \\href{http:\/\/lacsi.rice.edu\/symposium\/agenda_2005}{$6^{th}$\r\n                   Los Alamos Computer Science Institute (LACSI) Symposium\r\n                   2005}\",\r\n  month         = oct # \"~11, \",\r\n  year          = \"2005\",\r\n  address       = \"Santa Fe, NM, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann05concepts.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann05concepts.ppt.pdf\",\r\n  abstract      = \"Scientific high-end computing (HEC) has become an important\r\n                   tool for scientists world-wide to understand problems, such\r\n                   as in nuclear fusion, human genomics and nanotechnology.\r\n                   Every year, new HEC systems emerge on the market with better\r\n                   performance and higher scale. With only very few exceptions,\r\n                   the overall availability of recently installed systems has\r\n                   been lower in comparison to the same deployment phase of\r\n                   their predecessors. In contrast to the experienced loss of\r\n                   availability, the demand for continuous availability has\r\n                   risen dramatically due to the recent trend towards capability\r\n                   computing. In this paper, we analyze the existing\r\n                   deficiencies of current HEC systems and present several high\r\n                   availability concepts to counter the experienced loss of\r\n                   availability and to alleviate the expected impact on\r\n                   next-generation systems. We explain the application of these\r\n                   concepts to current and future HEC systems and list past and\r\n                   ongoing related research. This paper closes with a short\r\n                   summary of the presented work and a brief discussion of\r\n                   future efforts.\",\r\n  pts           = \"3777\"\r\n}<\/pre>\n
<\/a>@conference{engelmann05high,\r\n  author        = \"Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"High Availability for Ultra-Scale High-End Scientific\r\n                   Computing\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/coset.irisa.fr}{$2^{nd}$\r\n                   International Workshop on Operating Systems, Programming\r\n                   Environments and Management Tools for High-Performance\r\n                   Computing on Clusters (COSET-2) 2005}, in conjunction with\r\n                   the \\href{http:\/\/ics05.csail.mit.edu}{$19^{th}$ ACM\r\n                   International Conference on Supercomputing (ICS) 2005}\",\r\n  month         = jun # \"~19, \",\r\n  year          = \"2005\",\r\n  address       = \"Cambridge, MA, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann05high.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann05high.ppt.pdf\",\r\n  abstract      = \"Ultra-scale architectures for scientific high-end computing\r\n                   with tens to hundreds of thousands of processors, such as the\r\n                   IBM Blue Gene\/L and the Cray X1, suffer from availability\r\n                   deficiencies, which impact the efficiency of running\r\n                   computational jobs by forcing frequent checkpointing of\r\n                   applications. Most systems are unable to handle runtime\r\n                   system configuration changes caused by failures and require\r\n                   a complete restart of essential system services, such as the\r\n                   job scheduler or MPI, or even of the entire machine. In this\r\n                   paper, we present a flexible, pluggable and component-based\r\n                   high availability framework that expands today`s effort in\r\n                   high availability computing of keeping a single server alive\r\n                   to include all machines cooperating in a high-end scientific\r\n                   computing environment, while allowing adaptation to system\r\n                   properties and application needs.\"\r\n}<\/pre>\n
<\/a>@conference{leangsuksun05asymmetric,\r\n  author        = \"Chokchai (Box) Leangsuksun\r\n                   and Venkata K. Munganuru\r\n                   and Tong Liu\r\n                   and Stephen L. Scott\r\n                   and Christian Engelmann\",\r\n  title         = \"Asymmetric Active-Active High Availability for High-end\r\n                   Computing\",\r\n  booktitle     = \"Proceedings of the \\href{http:\/\/coset.irisa.fr}{$2^{nd}$\r\n                   International Workshop on Operating Systems, Programming\r\n                   Environments and Management Tools for High-Performance\r\n                   Computing on Clusters (COSET-2) 2005}, in conjunction with\r\n                   the \\href{http:\/\/ics05.csail.mit.edu}{$19^{th}$ ACM\r\n                   International Conference on Supercomputing (ICS) 2005}\",\r\n  month         = jun # \"~19, \",\r\n  year          = \"2005\",\r\n  address       = \"Cambridge, MA, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/leangsuksun05asymmetric.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/leangsuksun05asymmetric.ppt.pdf\",\r\n  abstract      = \"Linux clusters have become very popular for scientific\r\n                   computing at research institutions world-wide, because they\r\n                   can be easily deployed at a fairly low cost. However, the\r\n                   most pressing issues of today`s cluster solutions are\r\n                   availability and serviceability. The conventional Beowulf\r\n                   cluster architecture has a single head node connected to a\r\n                   group of compute nodes. This head node is a typical single\r\n                   point of failure and control, which severely limits\r\n                   availability and serviceability by effectively cutting off\r\n                   healthy compute nodes from the outside world upon overload\r\n                   or failure. In this paper, we describe a paradigm that\r\n                   addresses this issue using asymmetric active-active high\r\n                   availability. Our framework comprises of n + 1 head nodes,\r\n                   where n head nodes are active in the sense that they provide\r\n                   services to simultaneously incoming user requests. One\r\n                   standby server monitors all active servers and performs a\r\n                   fail-over in case of a detected outage. We present a\r\n                   prototype implementation based on a 2 + 1 solution and\r\n                   discuss initial results.\"\r\n}<\/pre>\n
<\/a>@conference{engelmann05lightweight,\r\n  author        = \"Christian Engelmann\r\n                   and George A. (Al) Geist\",\r\n  title         = \"A Lightweight Kernel for the Harness Metacomputing\r\n                   Framework\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.ipdps.org\/ipdps2005}{$19^{th}$ IEEE\r\n                   International Parallel and Distributed Processing Symposium\r\n                   (IPDPS) 2005}: \\href{http:\/\/www.cs.umass.edu\/~rsnbrg\/hcw2005}\r\n                   {$14^{th}$ Heterogeneous Computing Workshop (HCW) 2005}\",\r\n  month         = apr # \"~4, \",\r\n  year          = \"2005\",\r\n  address       = \"Denver, CO, USA\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"0-7695-2312-9\",\r\n  issn          = \"1530-2075\",\r\n  doi           = \"10.1109\/IPDPS.2005.34\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann05lightweight.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann05lightweight.ppt.pdf\",\r\n  abstract      = \"Harness is a pluggable heterogeneous Distributed Virtual\r\n                   Machine (DVM) environment for parallel and distributed\r\n                   scientific computing. This paper describes recent\r\n                   improvements in the Harness kernel design. By using a\r\n                   lightweight approach and moving previously integrated system\r\n                   services into software modules, the software becomes more\r\n                   versatile and adaptable. This paper outlines these changes\r\n                   and explains the major Harness kernel components in more\r\n                   detail. A short overview is given of ongoing efforts in\r\n                   integrating RMIX, a dynamic heterogeneous reconfigurable\r\n                   communication framework, into the Harness environment as a\r\n                   new plug-in software module. We describe the overall impact\r\n                   of these changes and how they relate to other ongoing work.\"\r\n}<\/pre>\n
<\/a>@conference{engelmann04high,\r\n  author        = \"Christian Engelmann\r\n                   and Stephen L. Scott\r\n                   and George A. (Al) Geist\",\r\n  title         = \"High Availability through Distributed Control\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/xcr.cenit.latech.edu\/hapcw2004}{$2^{nd}$ High\r\n                   Availability and Performance Workshop (HAPCW) 2004}, in\r\n                   conjunction with the\r\n                   \\href{http:\/\/lacsi.rice.edu\/symposium\/agenda_2004}{$5^{th}$\r\n                   Los Alamos Computer Science Institute (LACSI) Symposium\r\n                   2004}\",\r\n  month         = oct # \"~12, \",\r\n  year          = \"2004\",\r\n  address       = \"Santa Fe, NM, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann04high.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann04high.ppt.pdf\",\r\n  abstract      = \"Cost-effective, flexible and efficient scientific simulations\r\n                   in cutting-edge research areas utilize huge high-end\r\n                   computing resources with thousands of processors. In the next\r\n                   five to ten years the number of processors in such computer\r\n                   systems will rise to tens of thousands, while scientific\r\n                   application running times are expected to increase further\r\n                   beyond the Mean-Time-To-Interrupt (MTTI) of hardware and\r\n                   system software components. This paper describes the ongoing\r\n                   research in heterogeneous adaptable reconfigurable networked\r\n                   systems (Harness) and its recent achievements in the area of\r\n                   high availability distributed virtual machine environments\r\n                   for parallel and distributed scientific computing. It shows\r\n                   how a distributed control algorithm is able to steer a\r\n                   distributed virtual machine process in virtual synchrony\r\n                   while maintaining consistent replication for high\r\n                   availability. It briefly illustrates ongoing work in\r\n                   heterogeneous reconfigurable communication frameworks and\r\n                   security mechanisms. The paper continues with a short\r\n                   overview of similar research in reliable group communication\r\n                   frameworks, fault-tolerant process groups and highly\r\n                   available distributed virtual processes. It closes with a\r\n                   brief discussion of possible future research directions.\"\r\n}<\/pre>\n
<\/a>@conference{he04highly,\r\n  author        = \"Xubin (Ben) He\r\n                   and Li Ou\r\n                   and Stephen L. Scott\r\n                   and Christian Engelmann\",\r\n  title         = \"A Highly Available Cluster Storage System using Scavenging\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/xcr.cenit.latech.edu\/hapcw2004}{$2^{nd}$ High\r\n                   Availability and Performance Workshop (HAPCW) 2004}, in\r\n                   conjunction with the\r\n                   \\href{http:\/\/lacsi.rice.edu\/symposium\/agenda_2004}{$5^{th}$\r\n                   Los Alamos Computer Science Institute (LACSI) Symposium\r\n                   2004}\",\r\n  month         = oct # \"~12, \",\r\n  year          = \"2004\",\r\n  address       = \"Santa Fe, NM, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/he04highly.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/he04highly.ppt.pdf\",\r\n  abstract      = \"Highly available data storage for high-performance computing\r\n                   is becoming increasingly more critical as high-end computing\r\n                   systems scale up in size and storage systems are developed\r\n                   around network-centered architectures. A promising solution\r\n                   is to harness the collective storage potential of individual\r\n                   workstations much as we harness idle CPU cycles due to the\r\n                   excellent price\/performance ratio and low storage usage of\r\n                   most commodity workstations. For such a storage system,\r\n                   metadata consistency is a key issue assuring storage system\r\n                   availability as well as data reliability. In this paper, we\r\n                   present a decentralized metadata management scheme that\r\n                   improves storage availability without sacrificing\r\n                   performance.\"\r\n}<\/pre>\n
<\/a>@conference{engelmann03diskless,\r\n  author        = \"Christian Engelmann\r\n                   and George A. (Al) Geist\",\r\n  title         = \"A Diskless Checkpointing Algorithm for Super-scale\r\n                   Architectures Applied to the Fast Fourier Transform\",\r\n  booktitle     = \"Proceedings of the\r\n                   \\href{http:\/\/www.cs.msstate.edu\/~clade2003}{Challenges of\r\n                   Large Applications in Distributed Environments Workshop\r\n                   (CLADE) 2003}, in conjunction with the\r\n                   \\href{http:\/\/csag.ucsd.edu\/HPDC-12}{$12^{th}$ IEEE\r\n                   International Symposium on High Performance Distributed\r\n                   Computing (HPDC) 2003}\",\r\n  pages         = \"47\",\r\n  month         = jun # \"~21, \",\r\n  year          = \"2003\",\r\n  address       = \"Seattle, WA, USA\",\r\n  publisher     = \"\\href{http:\/\/www.computer.org}{IEEE Computer Society, Los\r\n                   Alamitos, CA, USA}\",\r\n  isbn          = \"0-7695-1984-9\",\r\n  doi           = \"xpls\/abs_all.jsp?arnumber=4159902\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann03diskless.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann03diskless.ppt.pdf\",\r\n  abstract      = \"This paper discusses the issue of fault-tolerance in\r\n                   distributed computer systems with tens or hundreds of\r\n                   thousands of diskless processor units. Such systems, like the\r\n                   IBM Blue Gene\/L, are predicted to be deployed in the next\r\n                   five to ten years. Since a 100,000-processor system is going\r\n                   to be less reliable, scientific applications need to be able\r\n                   to recover from occurring failures more efficiently. In this\r\n                   paper, we adapt the present technique of diskless\r\n                   checkpointing to such huge distributed systems in order to\r\n                   equip existing scientific algorithms with super-scalable\r\n                   fault-tolerance. First, we discuss the method of diskless\r\n                   checkpointing, then we adapt this technique to super-scale\r\n                   architectures and finally we present results from an\r\n                   implementation of the Fast Fourier Transform that uses the\r\n                   adapted technique to achieve super-scale fault-tolerance.\"\r\n}<\/pre>\n
<\/a>@conference{engelmann02distributed,\r\n  author        = \"Christian Engelmann\r\n                   and Stephen L. Scott\r\n                   and George A. (Al) Geist\",\r\n  title         = \"Distributed Peer-to-Peer Control in {Harness}\",\r\n  booktitle     = \"Lecture Notes in Computer Science: Proceedings of the\r\n                   \\href{http:\/\/www.science.uva.nl\/events\/ICCS2002}{$2^{nd}$\r\n                   International Conference on Computational Science (ICCS)\r\n                   2002}, Part II: Workshop on Global and Collaborative\r\n                   Computing\",\r\n  volume        = \"2330\",\r\n  pages         = \"720--727\",\r\n  month         = apr # \"~21-24, \",\r\n  year          = \"2002\",\r\n  address       = \"Amsterdam, The Netherlands\",\r\n  publisher     = \"\\href{http:\/\/www.springer.com}{Springer Verlag, Berlin,\r\n                   Germany}\",\r\n  isbn          = \"3-540-43593-X\",\r\n  issn          = \"0302-9743\",\r\n  doi           = \"content\/l537ujfwt8yta2dp\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann02distributed.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann02distributed.ppt.pdf\",\r\n  abstract      = \"Harness is an adaptable fault-tolerant virtual machine\r\n                   environment for next-generation heterogeneous distributed\r\n                   computing developed as a follow on to PVM. It additionally\r\n                   enables the assembly of applications from plug-ins and\r\n                   provides fault-tolerance. This work describes the distributed\r\n                   control, which manages global state replication to ensure a\r\n                   high-availability of service. Group communication services\r\n                   achieve an agreement on an initial global state and a linear\r\n                   history of global state changes at all members of the\r\n                   distributed virtual machine. This global state is replicated\r\n                   to all members to easily recover from single, multiple and\r\n                   cascaded faults. A peer-to-peer ring network architecture and\r\n                   tunable multi-point failure conditions provide heterogeneity\r\n                   and scalability. Finally, the integration of the distributed\r\n                   control into the multi-threaded kernel architecture of\r\n                   Harness offers a fault-tolerant global state database service\r\n                   for plug-ins and applications.\"\r\n}<\/pre>\n
<\/a>@misc{boehm26automated,\r\n  author        = \"Swen Boehm\r\n                   and Craig A. Bridges\r\n                   and Patrick Widener\r\n                   and Terry Jones\r\n                   and Sheikh Ghafoor\r\n                   and Christian Engelmann\r\n                   and Olga Kuchar\",\r\n  title         = \"From Automated Experiments and Simulations to Reusable\r\n                   Scientific Evidence\",\r\n  month         = aug # \"~24-26, \",\r\n  year          = \"2026\",\r\n  howpublished  = \"{Poster at the \\href{https:\/\/www.montereydataconference.org}\r\n                   {Monterey Data Conference}, Monterey, CA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/boehm26automated.ppt.pdf\",\r\n  abstract      = \"Scientific research increasingly depends on automated\r\n                   platforms and heterogeneous instruments that generate rich,\r\n                   multi-modal data. Yet fragmentation across domain-specific\r\n                   systems, proprietary formats, and siloed repositories impedes\r\n                   data discovery, integration, and reuse. Without systematic\r\n                   semantic representation and comprehensive provenance,\r\n                   high-quality experimental data loses much of its scientific\r\n                   value. The INTERSECT Scientific data layer (SDL) provides an\r\n                   integrated, ontology-driven ecosystem that connects\r\n                   scientific platforms, workflows, and data management\r\n                   services into a coherent whole. Built on a system-of-systems\r\n                   architecture and grounded in Linked Data Platform (LDP)\r\n                   principles, the SDL enables modular integration of diverse\r\n                   services while preserving interoperability across scientific\r\n                   domains.\"\r\n}<\/pre>\n
<\/a>@misc{boehm26intersect,\r\n  author        = \"Swen Boehm\r\n                   and Terry Jones\r\n                   and Patrick Widener\r\n                   and Christian Engelman\r\n                   and Olga Kuchar\",\r\n  title         = \"INTERSECT Scientific Data Layer: A Federated, Modular\r\n                   Framework for Scientific Data Management\",\r\n  month         = mar # \"~3-5, \",\r\n  year          = \"2026\",\r\n  howpublished  = \"{Poster at the \\href{https:\/\/data-science.llnl.gov\/d3}\r\n                   {Department of Energy (DOE) Data Days (D3) Workshop},\r\n                   Chantilly, VA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/boehm26intersect.ppt.pdf\",\r\n  abstract      = \"Scientific research increasingly depends on automated\r\n                   platforms and heterogeneous instruments that generate rich,\r\n                   multi-modal data. Yet fragmentation across domain-specific\r\n                   systems, proprietary formats, and siloed repositories\r\n                   impedes data discovery, integration, and reuse. Without\r\n                   systematic semantic representation and comprehensive\r\n                   provenance, high-quality experimental data loses much of its\r\n                   scientific value.\r\n                   \r\n                   The INTERSECT Scientific data layer (SDL) provides an\r\n                   integrated, ontology-driven ecosystem that connects\r\n                   scientific platforms, workflows, and data management services\r\n                   into a coherent whole. Built on a system-of-systems\r\n                   architecture and grounded in Linked Data Platform (LDP)\r\n                   principles, the SDL enables modular integration of diverse\r\n                   services while preserving interoperability across scientific\r\n                   domains. Core ontologies such as SSN\/SOSA for sensor and\r\n                   observation modeling, DCAT for resource cataloging, and\r\n                   PROV-O for provenance tracking provide a semantic backbone\r\n                   that ensures all entities -- data, instruments, workflows,\r\n                   and results -- are described in a machine-actionable,\r\n                   reusable way.\r\n                   \r\n                   The SDL offers semantic-first design, a microservices\r\n                   foundation, separation of concerns, and content\r\n                   negotiation:\r\n                   \r\n                   - Semantic-First Design: RDF is the native data model, not\r\n                     an auxiliary export format. Semantic richness is preserved\r\n                     throughout the data lifecycle, from instrumental observations\r\n                     through processing pipelines to publication, enabling FAIR\r\n                     data by design.\r\n                   - Microservices Foundation: Modular, independently deployable\r\n                     services (Catalog Service, Storage Service, Repository\r\n                     Service, Registry Service) coordinate through shared\r\n                     semantic libraries and standard ontologies, solving the\r\n                     distributed consistency challenge inherent in semantic\r\n                     systems.\r\n                   - Separation of Concerns: Semantic metadata (RDF triples in\r\n                     triple stores) is decoupled from data artifacts (files in\r\n                     object storage) with URIs providing semantic linking. This\r\n                     enables independent scaling of metadata management and\r\n                     storage infrastructure while maintaining coherent provenance\r\n                     relationships.\r\n                   - Content Negotiation: Services accept and return data in\r\n                     multiple RDF serializations (Turtle, JSON-LD, RDF\/XML) and\r\n                     domain-specific formats (CSV, HDF5, instrument formats),\r\n                     supporting diverse tools and workflows while maintaining \r\n                     semantic consistency.\r\n                   \r\n                   The SDL natively implements FAIR principles through\r\n                   semantic-first architecture. Persistent URIs and SPARQL\r\n                   endpoints enable discovery via machine-readable metadata\r\n                   (findable). Standard HTTP protocols and LDP containers\r\n                   support predictable REST-like access patterns (accessible).\r\n                   Composed W3C ontologies ensure semantic compatibility across\r\n                   domains (interoperable). Comprehensive end-to-end provenance\r\n                   and structured metadata make datasets suitable for both human\r\n                   researchers and AI systems (reusable).\"\r\n}<\/pre>\n
<\/a>@misc{engelmann23intersect,\r\n  author        = \"Christian Engelmann\r\n                   and Swen Boehm\r\n                   and Michael Brim\r\n                   and Jack Lange\r\n                   and Thomas Naughton\r\n                   and Patrick Widener\r\n                   and Ben Mintz\r\n                   and Rohit Srivastava\",\r\n  title         = \"INTERSECT: The Open Federated Architecture for the\r\n                   Laboratory of the Future\",\r\n  month         = aug # \"~7-10, \",\r\n  year          = \"2023\",\r\n  howpublished  = \"{Poster at the \\href{https:\/\/icpp23.sci.utah.edu\/}\r\n                   {52nd International Conference on Parallel Processing (ICPP)\r\n                   2023}, Salt Lake City, UT, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann23intersect.ppt.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann23intersect.pdf\",\r\n  abstract      = \"The open Self-driven Experiments for Science \/ Interconnected\r\n                   Science Ecosystem (INTERSECT) architecture connects\r\n                   scientific instruments and robot-controlled laboratories with\r\n                   computing and data resources at the edge, the Cloud or the\r\n                   high-performance computing center to enable autonomous\r\n                   experiments, self-driving laboratories, smart manufacturing,\r\n                   and artificial intelligence driven design, discovery and\r\n                   evaluation. Its a novel approach consists of science use case\r\n                   design patterns, a system of systems architecture, and a\r\n                   microservice architecture.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann22resilience,\r\n  author        = \"Christian Engelmann and Mohit Kumar\",\r\n  title         = \"Resilience Design Patterns: A Structured Modeling Approach of\r\n                   Resilience in Computing Systems\",\r\n  month         = aug # \"~10-12, \",\r\n  year          = \"2022\",\r\n  howpublished  = \"{Poster at the \\href{https:\/\/www.bnl.gov\/modsim2022}\r\n                   {Workshop on Modeling and Simulation of Systems and\r\n                   Applications (ModSim) 2022}, Seattle, WA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann22resilience.ppt.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann22resilience.pdf\",\r\n  abstract      = \"Resilience to faults, errors, and failures in extreme-scale\r\n                   high-performance computing (HPC) systems is a critical\r\n                   challenge. Resilience design patterns (Figure 1) offer a new,\r\n                   structured hardware\/software design approach for improving\r\n                   resilience by identifying and evaluating repeatedly occurring\r\n                   resilience problems and coordinating corresponding solutions.\r\n                   Initial work identified and formalized these patterns and\r\n                   developed a proof-of-concept prototype to demonstrate\r\n                   portable resilience. This recent work created performance,\r\n                   reliability, and availability models for each of the\r\n                   identified 15 structural resilience design patterns and a\r\n                   modeling tool that allows (1) exploring the performance,\r\n                   reliability, and availability of each pattern, and (2)\r\n                   investigating the trade-offs be-tween patterns and pattern\r\n                   combinations.\"\r\n}<\/pre>\n
<\/a>@misc{hui18realtime,\r\n  author        = \"Yawei Hui\r\n                   and Rizwan Ashraf\r\n                   and Byung Hoon (Hoony) Park\r\n                   and Christian Engelmann\",\r\n  title         = \"Real-Time Assessment of Supercomputer Status by a\r\n                   Comprehensive Informative Metric through Streaming\r\n                   Processing\",\r\n  month         = dec # \"~10-13, \",\r\n  year          = \"2018\",\r\n  howpublished  = \"{Poster at the\r\n                    \\href{http:\/\/cci.drexel.edu\/bigdata\/bigdata2018}\r\n                    {$6^{th}$ IEEE International Conference on Big Data (BigData) 2018},\r\n                    Seattle, WA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/hui18realtime.pdf\",\r\n  abstract      = \"Supercomputers are complex systems used to simulate,\r\n                   understand and solve real-world problems. In order to operate\r\n                   these systems efficiently and for the purpose of their\r\n                   maintainability, an accurate, concise, and timely\r\n                   determination of system status is crucial for its users and\r\n                   operators. However, this determination is challenging due to\r\n                   intricately connected heterogeneous software and hardware\r\n                   components, and due to sheer scale of such machines. In this\r\n                   poster, we demonstrate work-in-progress towards realization\r\n                   of a real-time monitoring framework for the 18,688-node Titan\r\n                   supercomputer at Oak Ridge Leadership Computing Facility\r\n                   (OLCF). Toward this end, we discuss the use of metrics which\r\n                   present a one-dimensional view of the system generating\r\n                   various types of information from 1000s of components and\r\n                   utilization statistics from 100s of user applications in near\r\n                   real-time. We demonstrate the efficacy of these metrics to\r\n                   understand and visualize raw log data generated by the\r\n                   system which otherwise may compose of 1000s of dimensions.\r\n                   We also demonstrate the architecture of proposed real-time\r\n                   stream processing framework which integrates, processes,\r\n                   analyzes, visualizes and stores system log data from an array\r\n                   of system components..\"\r\n}<\/pre>\n
<\/a>@misc{hui18comprehensive,\r\n  author        = \"Yawei Hui\r\n                   and Byung Hoon (Hoony) Park\r\n                   and Christian Engelmann\",\r\n  title         = \"A Comprehensive Informative Metric for Summarizing {HPC}\r\n                   System Status\",\r\n  month         = oct # \"~21, \",\r\n  year          = \"2018\",\r\n  howpublished  = \"{Poster at the \\href{http:\/\/ldav.org}\r\n                    {$8^{th}$ IEEE Symposium on Large Data Analysis and\r\n                     Visualization} in conjunction with the \r\n                    \\href{http:\/\/ieeevis.org\/year\/2018}{$8^{th}$ IEEE Vis 2018},\r\n                    Berlin, Germany}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/hui18comprehensive.pdf\",\r\n  abstract      = \"It remains a major challenge to effectively summarize and\r\n                   visualize in a comprehensive form the status of a complex\r\n                   computer system, such as the Titan supercomputer at the Oak\r\n                   Ridge Leadership Computing Facility (OLCF). In the ongoing\r\n                   research highlighted in this poster, we present system\r\n                   information entropy (SIE), a newly developed system metric\r\n                   that leverages the powers of traditional machine learning\r\n                   techniques and information theory. By compressing the\r\n                   multi-variant multi-dimensional event information recorded\r\n                   during the operation of the targeted system into a single\r\n                   time series of SIE, we demonstrate that the historical\r\n                   system status can be sensitively summarized in form of SIE\r\n                   and visualized concisely and comprehensively.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann18modeling2,\r\n  author        = \"Christian Engelmann and Rizwan Ashraf\",\r\n  title         = \"Modeling and Simulation of Extreme-Scale Systems for\r\n                   Resilience by Design\",\r\n  month         = aug # \"~15-17, \",\r\n  year          = \"2018\",\r\n  howpublished  = \"{Poster at the \\href{https:\/\/www.bnl.gov\/modsim2018}\r\n                   {Workshop on Modeling and Simulation of Systems and\r\n                   Applications}, Seattle, WA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann18modeling2.pdf\",\r\n  abstract      = \"Resilience is a serious concern for extreme-scale\r\n                   high-performance computing (HPC). While the HPC community has\r\n                   developed various resilience solutions, the solution space\r\n                   remains fragmented. We created a structured approach to the\r\n                   design, evaluation and optimization of HPC resilience using\r\n                   the concept of design patterns. A design pattern describes a\r\n                   generalized solution to a repeatedly occurring problem. We\r\n                   identified the commonly occurring problems and solutions used\r\n                   to deal with faults, errors and failures in HPC systems. Each\r\n                   well-known solution that addresses a specific resilience\r\n                   challenge is described in the form of a design pattern. We\r\n                   developed a resilience design pattern specification, language\r\n                   and catalog, which can be used by system architects, system\r\n                   software and library developers, application programmers, as\r\n                   well as users and operators as essential building blocks when\r\n                   designing and deploying resilience solutions.\r\n                   The resilience design pattern approach provides a unique\r\n                   opportunity for design space exploration. As each resilience\r\n                   solution is abstracted as a pattern and each solution's\r\n                   properties are defined by pattern parameters, vertical and\r\n                   horizontal pattern compositions can describe the resilience\r\n                   capabilities of an entire HPC system. This permits the\r\n                   investigation of beneficial or counterproductive interactions\r\n                   between patterns and of the performance, resilience, and\r\n                   power consumption trade-off between different pattern\r\n                   parameters and compositions. The ultimate goal is to make\r\n                   resilience an integral part of the HPC hardware\/software\r\n                   ecosystem by coordinating the various existing resilience\r\n                   solutions in a design space exploration process, such that\r\n                   the burden for providing resilience is on the system by\r\n                   design and not on the user as an afterthought.\r\n                   We are in the early stages of developing a novel design space\r\n                   exploration tool that enables this investigation using\r\n                   modeling and simulation. We developed performance and\r\n                   resilience models for each resilience design pattern. We also\r\n                   leverage results from the Catalog project, a collaborative\r\n                   effort between Oak Ridge National Laboratory, Argonne\r\n                   National Laboratory and Lawrence Livermore National\r\n                   Laboratory that developed models of the faults, errors and\r\n                   failures in today's HPC systems. We also leverage recent\r\n                   results from the same project by Lawrence Livermore National\r\n                   Laboratory in application reliability patterns. The planned\r\n                   research extends and combines this work to model the\r\n                   performance, resilience, and power consumption of an entire\r\n                   HPC system, initially at node-level granularity, and to\r\n                   simulate the dynamic interactions between deployed\r\n                   resilience solutions and the rest of the system. In the next\r\n                   iteration, finer-grain modeling and simulation, such as at\r\n                   the computational unit level, is used to increase accuracy.\r\n                   This work leverages the experience of the investigators in\r\n                   parallel discrete event simulation of extreme-scale systems,\r\n                   such as the Extreme-scale Simulator (xSim).\r\n                   The current state of the art in resilience modeling and\r\n                   simulation is fragmented as well. There is currently no such\r\n                   design space exploration tool. Instead, each resilience\r\n                   solution is typically investigated separately. There is only\r\n                   a small amount of work on multi-resilience solutions,\r\n                   including by the investigators. While there is work in\r\n                   investigating the performance\/resilience trade-off space,\r\n                   there is almost no work in including power consumption.\"\r\n}<\/pre>\n
<\/a>@misc{patil17exploring,\r\n  author        = \"Onkar Patil\r\n                   and Saurabh Hukerikar\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\",\r\n  title         = \"Exploring Use Cases for Non-Volatile Memories in Support of\r\n                   HPC Resilience\",\r\n  month         = nov # \"~12-17, \",\r\n  year          = \"2017\",\r\n  howpublished  = \"{Poster at the \\href{http:\/\/sc11.supercomputing.org}\r\n                   {30th IEEE\/ACM International Conference on High Performance\r\n                    Computing, Networking, Storage and Analysis (SC) 2017},\r\n                   Denver, CO, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/patil17exploring.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/patil17exploring.ppt.pdf\",\r\n  abstract      = \"Improving resilience and creating resilient architectures is\r\n                   one of the major goals of exascale computing. With the advent\r\n                   of Non-volatile memory technologies, memory architectures\r\n                   with persistent memory regions will be a significant part of\r\n                   future architectures. There is potential to use them in more\r\n                   than one way to benefit different applications. We look to\r\n                   take advantage of this technology to enable more fine-grained\r\n                   and novel methodology that will improve resilience and\r\n                   efficiency of exascale applications. We have developed three\r\n                   modes of memory usage for persistent memory to enable\r\n                   efficient checkpointing in HPC applications. We have\r\n                   developed a simple API that is evaluated with the DGEMM\r\n                   benchmark on a 16-node cluster with independent SSDs on every\r\n                   node. Our aim is to build on this work and enable static and\r\n                   dynamic runtime systems that will inherently make the HPC\r\n                   applications more fault-tolerant and resistant to errors.\"\r\n}<\/pre>\n
<\/a>@misc{fiala11detection,\r\n  author        = \"David Fiala\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\r\n                   and Rolf Riesen\r\n                   and Kurt Ferreira\",\r\n  title         = \"Detection and Correction of Silent Data Corruption for\r\n                   Large-Scale High-Performance Computing\",\r\n  month         = nov # \"~12-18, \",\r\n  year          = \"2011\",\r\n  howpublished  = \"{Poster at the \\href{http:\/\/sc11.supercomputing.org}\r\n                   {24th IEEE\/ACM International Conference on High Performance\r\n                    Computing, Networking, Storage and Analysis (SC) 2011},\r\n                   Seattle, WA, USA}\",\r\n  url           = \"\",\r\n  abstract      = \"Faults have become the norm rather than the exception for\r\n                   high-end computing on clusters with 10s\/100s of thousands of\r\n                   cores. Exacerbating this situation, some of these faults will\r\n                   not be detected, manifesting themselves as silent errors that\r\n                   will corrupt memory while applications continue to operate and\r\n                   report incorrect results. This poster introduces RedMPI, an\r\n                   MPI library which resides in the MPI profiling layer. RedMPI\r\n                   is capable of both online detection and correction of soft\r\n                   errors that occur in MPI applications without requiring any\r\n                   modifications to the application source. By providing\r\n                   redundancy, RedMPI is capable of transparently detecting\r\n                   corrupt messages from MPI processes that become faulted during\r\n                   execution. Furthermore, with triple redundancy RedMPI\r\n                   additionally ``votes'' out MPI messages of a faulted process\r\n                   by replacing corrupted results with corrected results from\r\n                   unfaulted processes. We present an experimental evaluation of\r\n                   RedMPI on an assortment of applications to demonstrate the\r\n                   effectiveness of this approach.\"\r\n}<\/pre>\n
<\/a>@misc{fiala11tunable2,\r\n  author        = \"David Fiala\r\n                   and Kurt Ferreira\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\",\r\n  title         = \"A Tunable, Software-based {DRAM} Error Detection and Correction\r\n                   Library for {HPC}\",\r\n  month         = nov # \"~12-18, \",\r\n  year          = \"2011\",\r\n  howpublished  = \"{Poster at the \\href{http:\/\/sc11.supercomputing.org}\r\n                   {24th IEEE\/ACM International Conference on High Performance\r\n                    Computing, Networking, Storage and Analysis (SC) 2011},\r\n                   Seattle, WA, USA}\",\r\n  url           = \"\",\r\n  abstract      = \"Proposed exascale systems will present a number of\r\n                   considerable resiliency challenges. In particular, DRAM\r\n                   soft-errors, or bit-flips, are expected to greatly increase\r\n                   due to the increased memory density of these systems. Current\r\n                   hardware-based fault-tolerance methods will be unsuitable for\r\n                   addressing the expected soft error frequency rate. As a\r\n                   result, additional software will be needed to address this\r\n                   challenge. In this paper we introduce LIBSDC, a tunable,\r\n                   transparent silent data corruption detection and correction\r\n                   library for HPC applications. LIBSDC provides comprehensive\r\n                   SDC protection for program memory by implementing on-demand\r\n                   page integrity verification by utilizing the MMU. Experimental \r\n                   benchmarks with Mantevo HPCCG show that once tuned, LIBSDC is\r\n                   able to achieve SDC protection with less than 100\\% overhead\r\n                   of resources.\"\r\n}<\/pre>\n
<\/a>@misc{scott09tunable2,\r\n  author        = \"Stephen L. Scott\r\n                   and Christian Engelmann\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Anand Tikotekar\r\n                   and George Ostrouchov\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and Nichamon Naksinehaboon\r\n                   and Raja Nassar\r\n                   and Mihaela Paun\r\n                   and Frank Mueller\r\n                   and Chao Wang\r\n                   and Arun B. Nagarajan\r\n                   and Jyothish Varma\",\r\n  title         = \"A Tunable Holistic Resiliency Approach for High-Performance\r\n                   Computing Systems\",\r\n  month         = aug # \"~12-14, \",\r\n  year          = \"2009\",\r\n  howpublished  = \"{Poster at the\r\n                   \\href{http:\/\/institute.lanl.gov\/resilience\/conferences\/2009}\r\n                   {National HPC Workshop on Resilience 2009}, Arlington, VA,\r\n                   USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/scott09tunable2.pdf\",\r\n  abstract      = \"In order to address anticipated high failure rates,\r\n                   resiliency characteristics have become an urgent priority for\r\n                   next-generation extreme-scale high-performance computing\r\n                   (HPC) systems. This poster describes our past and ongoing\r\n                   efforts in novel fault resilience technologies for HPC.\r\n                   Presented work includes proactive fault resilience\r\n                   techniques, system and application reliability models and\r\n                   analyses, failure prediction, transparent process- and\r\n                   virtual-machine-level migration, and trade-off models for\r\n                   combining preemptive migration with checkpoint\/restart. This\r\n                   poster summarizes our work and puts all individual\r\n                   technologies into context with a proposed holistic fault\r\n                   resilience framework.\"\r\n}<\/pre>\n
<\/a>@misc{scott09systemlevel,\r\n  author        = \"Stephen L. Scott\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Anand Tikotekar\r\n                   and Christian Engelmann\r\n                   and Hong H. Ong\",\r\n  title         = \"System-level Virtualization for for High-Performance\r\n                   Computing\",\r\n  month         = aug # \"~12-14, \",\r\n  year          = \"2009\",\r\n  howpublished  = \"{Poster at the\r\n                   \\href{http:\/\/institute.lanl.gov\/resilience\/conferences\/2009}\r\n                   {National HPC Workshop on Resilience 2009}, Arlington, VA,\r\n                   USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/scott09systemlevel.pdf\",\r\n  abstract      = \"This poster summarizes our past and ongoing research and\r\n                   development efforts in novel system software solutions for\r\n                   providing a virtual system environment (VSE) for\r\n                   next-generation extreme-scale high-performance computing\r\n                   (HPC) systems and beyond. The poster showcases results of\r\n                   developed proof-of-concept implementations and performed\r\n                   theoretical analyses, outlines planned research and\r\n                   development activities, and presents respective initial\r\n                   results.\"\r\n}<\/pre>\n
<\/a>@misc{scott09tunable,\r\n  author        = \"Stephen L. Scott\r\n                   and Christian Engelmann\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Anand Tikotekar\r\n                   and George Ostrouchov\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and Nichamon Naksinehaboon\r\n                   and Raja Nassar\r\n                   and Mihaela Paun\r\n                   and Frank Mueller\r\n                   and Chao Wang\r\n                   and Arun B. Nagarajan\r\n                   and Jyothish Varma\",\r\n  title         = \"A Tunable Holistic Resiliency Approach for High-Performance\r\n                   Computing Systems\",\r\n  month         = feb # \"~14-18, \",\r\n  year          = \"2009\",\r\n  howpublished  = \"{Poster at the \\href{http:\/\/ppopp09.rice.edu}{$14^{th}$ ACM\r\n                   SIGPLAN Symposium on Principles and Practice of Parallel\r\n                   Programming (PPoPP) 2009}, Raleigh, NC, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/scott09tunable.pdf\",\r\n  abstract      = \"In order to address anticipated high failure rates,\r\n                   resiliency characteristics have become an urgent priority for\r\n                   next-generation extreme-scale high-performance computing\r\n                   (HPC) systems. This poster describes our past and ongoing\r\n                   efforts in novel fault resilience technologies for HPC.\r\n                   Presented work includes proactive fault resilience\r\n                   techniques, system and application reliability models and\r\n                   analyses, failure prediction, transparent process- and\r\n                   virtual-machine-level migration, and trade-off models for\r\n                   combining preemptive migration with checkpoint\/restart. This\r\n                   poster summarizes our work and puts all individual\r\n                   technologies into context with a proposed holistic fault\r\n                   resilience framework.\"\r\n}<\/pre>\n
<\/a>@misc{geist08harness,\r\n  author        = \"George A. (Al) Geist\r\n                   and Christian Engelmann\r\n                   and Jack J. Dongarra\r\n                   and George Bosilca\r\n                   and Magdalena M. S\\l{}awi\\'nska\r\n                   and Jaros\\l{}aw K. S\\l{}awi\\'nski\",\r\n  title         = \"The {Harness} Workbench: {U}nified and Adaptive Access to\r\n                   Diverse High-Performance Computing Platforms\",\r\n  month         = mar # \"~30 - \" # apr # \"~5, \",\r\n  year          = \"2008\",\r\n  howpublished  = \"{Poster at the \\href{http:\/\/www.hpcsw.org}{$1^{st}$\r\n                   High-Performance Computer Science Week (HPCSW) 2008}, Denver,\r\n                   CO, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/geist08harness.pdf\",\r\n  abstract      = \"This poster summarizes our past and ongoing research and\r\n                   development efforts in novel software solutions for providing\r\n                   unified and adaptive access to diverse high-performance\r\n                   computing (HPC) platforms. The poster showcases developed\r\n                   proof-of-concept implementations of tools and mechanisms that\r\n                   simplify scientific application development and deployment\r\n                   tasks, such that only minimal adaptation is needed when\r\n                   moving from one HPC system to another or after HPC system\r\n                   upgrades.\"\r\n}<\/pre>\n
<\/a>@misc{scott08resiliency,\r\n  author        = \"Stephen L. Scott\r\n                   and Christian Engelmann\r\n                   and Hong H. Ong\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Anand Tikotekar\r\n                   and George Ostrouchov\r\n                   and Chokchai (Box) Leangsuksun\r\n                   and Nichamon Naksinehaboon\r\n                   and Raja Nassar\r\n                   and Mihaela Paun\r\n                   and Frank Mueller\r\n                   and Chao Wang\r\n                   and Arun B. Nagarajan\r\n                   and Jyothish Varma\r\n                   and Xubin (Ben) He\r\n                   and Li Ou\r\n                   and Xin Chen\",\r\n  title         = \"Resiliency for High-Performance Computing Systems\",\r\n  month         = mar # \"~30 - \" # apr # \"~5, \",\r\n  year          = \"2008\",\r\n  howpublished  = \"{Poster at the \\href{http:\/\/www.hpcsw.org}{$1^{st}$\r\n                   High-Performance Computer Science Week (HPCSW) 2008}, Denver,\r\n                   CO, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/scott08resiliency.pdf\",\r\n  abstract      = \"This poster summarizes our past and ongoing research and\r\n                   development efforts in novel system software solutions for\r\n                   providing high-level reliability, availability and\r\n                   serviceability (RAS) for next-generation extreme-scale\r\n                   high-performance computing (HPC) systems and beyond. The\r\n                   poster showcases results of developed proof-of-concept\r\n                   implementations and performed theoretical analyses, outlines\r\n                   planned research and development activities, and presents\r\n                   respective initial results.\"\r\n}<\/pre>\n
<\/a>@misc{scott08systemlevel,\r\n  author        = \"Stephen L. Scott\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Anand Tikotekar\r\n                   and Christian Engelmann\r\n                   and Hong H. Ong\",\r\n  title         = \"System-level Virtualization for for High-Performance\r\n                   Computing\",\r\n  month         = mar # \"~30 - \" # apr # \"~5, \",\r\n  year          = \"2008\",\r\n  howpublished  = \"{Poster at the \\href{http:\/\/www.hpcsw.org}{$1^{st}$\r\n                   High-Performance Computer Science Week (HPCSW) 2008}, Denver,\r\n                   CO, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/scott08systemlevel.pdf\",\r\n  abstract      = \"This poster summarizes our past and ongoing research and\r\n                   development efforts in novel system software solutions for\r\n                   providing a virtual system environment (VSE) for\r\n                   next-generation extreme-scale high-performance computing\r\n                   (HPC) systems and beyond. The poster showcases results of\r\n                   developed proof-of-concept implementations and performed\r\n                   theoretical analyses, outlines planned research and\r\n                   development activities, and presents respective initial\r\n                   results.\"\r\n}<\/pre>\n
<\/a>@misc{adamson21cybersecurity,\r\n  author        = \"Ryan Adamson\r\n                   and Christian Engelmann\",\r\n  title         = \"Cybersecurity and Privacy for Instrument-to-Edge-to-Center\r\n                   Scientific Computing Ecosystems\",\r\n  howpublished  = \"White paper accepted at the U.S. Department of Energy's\r\n                   \\href{https:\/\/www.orau.gov\/2021ascr-cybersecurity}\r\n                   {ASCR Workshop on Cybersecurity and Privacy for Scientific\r\n                    Computing Ecosystems}\",\r\n  month         = nov # \"~3-5, \",\r\n  year          = \"2021\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/adamson21cybersecurity.pdf\",\r\n  abstract      = \"The DOE's Artificial Intelligence (AI) for Science report\r\n                   outlines the need for intelligent systems, instruments, and\r\n                   facilities to enable science breakthroughs with autonomous\r\n                   experiments, 'self-driving' laboratories, smart manufacturing,\r\n                   and AI-driven design, discovery and evaluation. The DOE's\r\n                   Computational Facilities Research Workshop report identifies\r\n                   intelligent systems\/facilities as a challenge with enabling\r\n                   automation and eliminating human-in-the-loop needs as a\r\n                   cross-cutting theme. Autonomous experiments, 'self-driving'\r\n                   laboratories and smart manufacturing employ\r\n                   machine-in-the-loop intelligence for decision-making.\r\n                   Human-in-the-loop needs are reduced by an autonomous online\r\n                   control that collects experiment data, analyzes it, and\r\n                   takes appropriate operational actions in real time to steer\r\n                   an ongoing or plan the next experiment. DOE laboratories are\r\n                   currently in the process of developing and deploying\r\n                   federated hardware\/software architectures for connecting\r\n                   instruments with edge and center computing resources to\r\n                   autonomously collect, transfer, store, process, curate, and\r\n                   archive scientific data. These new\r\n                   instrument-to-edge-to-center scientific ecosystems face\r\n                   several cybersecurity and privacy challenges.\"\r\n}<\/pre>\n
<\/a>@misc{li21toward,\r\n  author        = \"Mingyan Li\r\n                   and Robert A. Bridges\r\n                   and Pablo Moriano\r\n                   and Christian Engelmann\r\n                   and Feiyi Wang\r\n                   and Ryan Adamson\",\r\n  title         = \"Toward Effective Security\/Reliability Situational Awareness\r\n                   via Concurrent Security-or-Fault Analytics \",\r\n  howpublished  = \"White paper accepted at the U.S. Department of Energy's\r\n                   \\href{https:\/\/www.orau.gov\/2021ascr-cybersecurity}\r\n                   {ASCR Workshop on Cybersecurity and Privacy for Scientific\r\n                    Computing Ecosystems}\",\r\n  month         = nov # \"~3-5, \",\r\n  year          = \"2021\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/li21toward.pdf\",\r\n  abstract      = \"Modern critical infrastructures (CI) and scientific computing\r\n                   ecosystems (SCE) are complex and vulnerable. The complexity\r\n                   of CI\/SCE, such as the distributed workload found across ASCR\r\n                   scientific computing facilities, does not allow for easy\r\n                   differentiation between emerging cyber security and\r\n                   reliability threats. It is also not easy to correctly\r\n                   identify the misbehaving systems. Sometimes, system failures\r\n                   are just caused by unintentional user misbehavior or actual\r\n                   hardware\/software reliability issues, but it may take some\r\n                   significant amount of time and effort to develop that\r\n                   understanding through root-cause analysis. On the security\r\n                   front, CI\/SCE are vital assets. They are prime targets of,\r\n                   and are vulnerable to, malicious cyber-attacks. Within DoE,\r\n                   inter-disciplinary and cross-facility collaboration (e.g.,\r\n                   ORNL INTERSECT initiative, next-gen supercomputing OLCF6),\r\n                   traditional perimeter-based defense and demarcation line\r\n                   between malicious cyber-attacks and non-malicious system\r\n                   faults are blurring. Amidst realistic reliability and\r\n                   security threats, the ability to effectively distinguish\r\n                   between non-malicious faults and malicious attacks is\r\n                   critical not only in root cause identification but also in\r\n                   countermeasures generation. \"\r\n}<\/pre>\n
<\/a>@misc{finkel21research2,\r\n  author        = \"Hal Finkel\r\n                   and Pete Beckman\r\n                   and Christian Engelmann\r\n                   and Shantenu Jha\r\n                   and Jack Lange\",\r\n  title         = \"Research Opportunities in Operating Systems for Scientific Edge Computing\",\r\n  howpublished  = \"White paper by the U.S. Department of Energy's\r\n                   \\href{https:\/\/www.orau.gov\/OSRoundtable2021}\r\n                   {ASCR Roundtable Discussions on Operating-Systems Research 2021}\",\r\n  month         = jan # \"~25, \",\r\n  year          = \"2021\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/finkel21research2.pdf\",\r\n  abstract      = \"As scientific experiments generate ever-increasing amounts of\r\n                   data, and grow in operational complexity, modern experimental\r\n                   science demands unprecedented computational capabilities at\r\n                   the edge -- physically proximate to each experiment. While\r\n                   some requirements on these computational capabilities are\r\n                   shared with high-performance-computing (HPC) systems,\r\n                   scientific edge computing has a number of unique challenges.\r\n                   In the following, we survey current trends in system\r\n                   software and edge systems for scientific computing,\r\n                   associated research challenges and open questions,\r\n                   infrastructure requirements for operating-systems research,\r\n                   communities who should be involved in that research, and the\r\n                   anticipated benefits of success.\"\r\n}<\/pre>\n
<\/a>@misc{finkel21research,\r\n  author        = \"Hal Finkel\r\n                   and Pete Beckman\r\n                   and Ron Brightwell\r\n                   and Rudi Eigenmann\r\n                   and Christian Engelmann\r\n                   and Roberto Gioiosa\r\n                   and Kamil Iskra\r\n                   and Shantenu Jha\r\n                   and Jack Lange\r\n                   and Tapasya Patki\r\n                   and Kevin Pedretti\",\r\n  title         = \"Research Opportunities in Operating Systems for High-Performance Scientific Computing\",\r\n  howpublished  = \"White paper by the U.S. Department of Energy's\r\n                   \\href{https:\/\/www.orau.gov\/OSRoundtable2021}\r\n                   {ASCR Roundtable Discussions on Operating-Systems Research 2021}\",\r\n  month         = jan # \"~25, \",\r\n  year          = \"2021\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/finkel21research.pdf\",\r\n  abstract      = \"As high-performance-computing (HPC) systems continue to\r\n                   evolve, with increasingly diverse and heterogeneous hardware,\r\n                   increasingly-complex requirements for security and\r\n                   multi-tenancy, and increasingly-demanding requirements for\r\n                   resiliency and monitoring, research in operating systems must\r\n                   continue to seed innovation to meet future needs. In the\r\n                   following, we survey current trends in system software and\r\n                   HPC systems for scientific computing, associated research\r\n                   challenges and open questions, infrastructure requirements\r\n                   for operating-systems research, communities who should be\r\n                   involved in that research, and the anticipated benefits of\r\n                   success.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann21resilience2,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Resilience by Codesign (and not as an Afterthought)\",\r\n  howpublished  = \"White paper accepted at the U.S. Department of Energy's\r\n                   \\href{https:\/\/web.cvent.com\/event\/f64a4f28-b473-4808-924c-c8c3d9a2af63\/}\r\n                   {Workshop on Reimagining Codesign 2021}\",\r\n  month         = mar # \"~16-18, \",\r\n  year          = \"2021\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann21resilience2.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann21resilience2.ppt.pdf\",\r\n  abstract      = \"Resilience, i.e., obtaining a correct solution in a timely\r\n                   and efficient manner, is one of the key challenges in\r\n                   extreme-scale high-performance computing (HPC). Extreme\r\n                   heterogeneity, i.e., using multiple, and potentially\r\n                   configurable, types of processors, accelerators and\r\n                   memory\/storage in a single computing platform, will add a\r\n                   significant amount of complexity to the HPC hardware\/software\r\n                   eco-system. Hardware\/software HPC codesign for resilience is\r\n                   mostly nonexistent at this point! Resilience needs to become\r\n                   an integral part of the HPC hardware\/software ecosystem\r\n                   through codesign, such that the burden for resilience is on\r\n                   the system by design and not on the operator or user as an\r\n                   afterthought. Simply put, if resilience by design is not\r\n                   done now, in the early stages of extreme heterogeneity, the\r\n                   current state of practice for HPC resilience, global\r\n                   application-level checkpoint\/restart, will re-main the same\r\n                   for decades to come due to the high costs of adoption of\r\n                   alternatives later on. \"\r\n}<\/pre>\n
<\/a>@misc{radojkovic20towards,\r\n  author        = \"Petar Radojkovic\r\n                   and Manolis Marazakis\r\n                   and Paul Carpenter\r\n                   and Reiley Jeyapaul\r\n                   and Dimitris Gizopoulos\r\n                   and Martin Schulz\r\n                   and Adria Armejach\r\n                   and Eduard Ayguade\r\n                   and Fran\\c{c}ois Bodin\r\n                   and Ramon Canal\r\n                   and Franck Cappello\r\n                   and Fabien Chaix\r\n                   and Guillaume Colin de Verdiere\r\n                   and Said Derradji\r\n                   and Stefano Di Carlo\r\n                   and Christian Engelmann\r\n                   and Ignacio Laguna\r\n                   and Miquel Moreto\r\n                   and Onur Mutlu\r\n                   and Lazaros Papadopoulos\r\n                   and Olly Perks\r\n                   and Manolis Ploumidis\r\n                   and Bezhad Salami\r\n                   and Yanos Sazeides\r\n                   and Dimitrios Soudris\r\n                   and Yiannis Sourdis\r\n                   and Per Stenstrom\r\n                   and Samuel Thibault\r\n                   and Will Toms\r\n                   and Osman Unsal\",\r\n  title         = \"Towards Resilient {EU} {HPC} Systems: {A} Blueprint\",\r\n  howpublished  = \"White paper by the\r\n                   \\href{https:\/\/resilienthpc.eu}\r\n                   {European HPC resilience initiative}\",\r\n  month         = apr # \"~9, \",\r\n  year          = \"2020\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/radojkovic20towards.pdf\",\r\n  abstract      = \"This document aims to spearhead a Europe-wide discussion on\r\n                   HPC system resilience and to help the European HPC community\r\n                   define best practices for resilience. We analyse a wide range\r\n                   of state-of-the-art resilience mechanisms and recommend the\r\n                   most effective approaches to employ in large-scale HPC\r\n                   systems. Our guidelines will be useful in the allocation of\r\n                   available resources, as well as guiding researchers and\r\n                   research funding towards the enhancement of resilience\r\n                   approaches with the highest priority and utility. Although\r\n                   our work is focussed on the needs of next generation HPC\r\n                   systems in Europe, the principles and evaluations are\r\n                   applicable globally.\r\n                   This document is the first output of the ongoing European HPC\r\n                   resilience initiative and it covers individual nodes in HPC\r\n                   systems, encompassing CPU, memory, intra-node interconnect\r\n                   and emerging FPGA-based hardware accelerators. With community\r\n                   support and feedback on this initial document, we will update\r\n                   the analysis and expand the scope to include other types of\r\n                   accelerators, as well as networks and storage.\",\r\n  pts           = \"140761\"\r\n}<\/pre>\n
<\/a>@misc{engelmann18extreme,\r\n  author        = \"Christian Engelmann\r\n                   and Rizwan Ashraf\r\n                   and Saurabh Hukerikar\",\r\n  title         = \"Extreme Heterogeneity with Resilience by Design (and not as\r\n                   an Afterthought)\",\r\n  howpublished  = \"White paper accepted at the U.S. Department of Energy's\r\n                   \\href{https:\/\/orau.gov\/exheterogeneity2018\/}{Extreme\r\n                   Heterogeneity Virtual Workshop 2018}\",\r\n  month         = jan # \"~23-24, \",\r\n  year          = \"2018\",\r\n  address       = \"Washington, DC, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann18extreme.pdf\",\r\n  abstract      = \"Resilience, i.e., obtaining a correct solution in a timely\r\n                   and efficient manner, is one of the key challenges in\r\n                   extreme-scale high-performance computing (HPC). Extreme\r\n                   heterogeneity, i.e., using multiple, and potentially\r\n                   configurable, types of processors, accelerators and \r\n                   memory\/storage in a single computing platform, will add a\r\n                   significant amount of complexity to the HPC hardware\/software\r\n                   ecosystem. The notion of correct computation and program\r\n                   state assumed by users and application developers today,\r\n                   which has been based on binary bit-level correctness, will\r\n                   no longer hold for processing elements based on quantum\r\n                   qubits and analog circuits that model spiking neurons in\r\n                   neuromorphic computing elements. The diverse set of compute\r\n                   and memory components in future heterogeneous systems will\r\n                   require novel hardware and software resilience solutions.\r\n                   Errors and failures reported by such heterogeneous hardware\r\n                   will need to be handled by the appropriate software\r\n                   component to enable efficient masking, recovery, and\r\n                   avoidance with little burden on the user. Similarly, errors\r\n                   and failures reported by the software running on such\r\n                   heterogeneous hardware need to be equally efficiently\r\n                   handled with little burden on the user. This requires a new\r\n                   approach, where resilience is holistically provided by the\r\n                   HPC hardware\/software ecosystem. The key challenges are to\r\n                   design and to operate extreme heterogeneous HPC systems\r\n                   with (1) wide-ranging resilience capabilities in system\r\n                   software, programming models, libraries, and applications,\r\n                   (2) interfaces and mechanisms for coordinating resilience\r\n                   capabilities across diverse hardware and software\r\n                   components, (3) appropriate metrics and tools for assessing\r\n                   performance, resilience, and energy, and (4) an\r\n                   understanding of the performance, resilience and energy\r\n                   trade-off that eventually results in well-informed HPC\r\n                   system design choices and runtime decisions.\"\r\n}<\/pre>\n
<\/a>@misc{tiwari16lightweight,\r\n  author        = \"Devesh Tiwari\r\n                   and Saurabh Gupta\r\n                   and Christian Engelmann\",\r\n  title         = \"Lightweight, Actionable Analytical Tools Based on Statistical\r\n                   Learning for Efficient System Operations\",\r\n  howpublished  = \"White paper accepted at the U.S. Department of Energy's\r\n                   \\href{http:\/\/hpc.pnl.gov\/modsim\/2016}{Workshop on Modeling\r\n                   & Simulation of Systems & Applications (ModSim)\r\n                   2016}\",\r\n  month         = aug # \"~10-12, \",\r\n  year          = \"2016\",\r\n  address       = \"Seattle, WA, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/tiwari16lightweight.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/tiwari16lightweight.ppt.pdf\",\r\n  abstract      = \"Modeling and simulation community has always relied on\r\n                   accurate and meaningful system data and parameters to drive\r\n                   analytical models and simulators. HPC systems continuously\r\n                   generate huge amount system event related data (e.g., system\r\n                   log, resource consumption log, RAS logs, power consumption\r\n                   logs), but meaningful interpretation and accuracy verification\r\n                   of such data is quite challenging. This talk offers a unique\r\n                   perspective and experience in demonstrating how modeling and\r\n                   simulation based research can actually be translated into\r\n                   production systems. We will discuss the short-term\r\n                   opportunities for modeling and simulation community to\r\n                   increase the impact and effectiveness of our analytical\r\n                   tools, ``dos and don'ts'', long-term challenges and\r\n                   opportunities.\",\r\n  pts           = \"69458\"\r\n}<\/pre>\n
<\/a>@misc{engelmann13hardware,\r\n  author        = \"Christian Engelmann\r\n                   and Thomas Naughton\",\r\n  title         = \"A Hardware\/Software Performance\/Resilience\/Power Co-Design\r\n                   Tool for Extreme-scale Computing\",\r\n  howpublished  = \"White paper accepted at the U.S. Department of Energy's\r\n                   \\href{http:\/\/hpc.pnl.gov\/modsim\/2013}{Workshop on Modeling\r\n                   & Simulation of Exascale Systems & Applications (ModSim)\r\n                   2013}\",\r\n  month         = sep # \"~18-19, \",\r\n  year          = \"2013\",\r\n  address       = \"Seattle, WA, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann13hardware.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann13hardware.ppt.pdf\",\r\n  abstract      = \"xSim is a simulation-based performance investigation toolkit\r\n                   that permits running high-performance computing (HPC)\r\n                   applications in a controlled environment with millions of\r\n                   concurrent execution threads, while observing application\r\n                   performance in a simulated extreme-scale system for\r\n                   hardware\/software co-design. The presented work details newly\r\n                   developed features for xSim that permit the injection of MPI\r\n                   process failures, the propagation\/detection\/notification of\r\n                   such failures within the simulation, and their handling using\r\n                   application-level checkpoint\/restart. The newly added features\r\n                   also offer user-level failure mitigation (ULFM) extensions\r\n                   at the simulated MPI layer to support algorithm-based fault\r\n                   tolerance (ABFT). The presented solution permits investigating\r\n                   performance under failure and failure handling of\r\n                   checkpoint\/restart and ABFT solutions. The newly enhanced xSim\r\n                   is the very first performance tool that supports these\r\n                   capabilities.\"\r\n}<\/pre>\n
<\/a>@misc{snir13addressing,\r\n  author        = \"Marc Snir and\r\n                   and Robert W. Wisniewski\r\n                   and Jacob A. Abraham\r\n                   and Sarita V. Adve\r\n                   and Saurabh Bagchi\r\n                   and Pavan Balaji\r\n                   and Bill Carlson\r\n                   and Andrew A. Chien\r\n                   and Pedro Diniz\r\n                   and Christian Engelmann\r\n                   and Rinku Gupta\r\n                   and Fred Johnson\r\n                   and Jim Belak\r\n                   and Pradip Bose\r\n                   and Franck Cappello\r\n                   and Paul Coteus\r\n                   and Nathan A. Debardeleben\r\n                   and Mattan Erez\r\n                   and Saverio Fazzari\r\n                   and Al Geist\r\n                   and Sriram Krishnamoorthy\r\n                   and Sven Leyffer\r\n                   and Dean Liberty\r\n                   and Subhasish Mitra\r\n                   and Todd Munson\r\n                   and Rob Schreiber\r\n                   and Jon Stearley\r\n                   and Eric Van Hensbergen\",\r\n  title         = \"Addressing Failures in Exascale Computing\",\r\n  howpublished  = \"Workshop report\",\r\n  month         = aug # \"~4-11, \",\r\n  year          = \"2013\",\r\n  address       = \"Park City, UT, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/snir13addressing.pdf\"\r\n}<\/pre>\n
<\/a>@misc{geist12department,\r\n  author        = \"Al Geist\r\n                   and Bob Lucas\r\n                   and Marc Snir\r\n                   and Shekhar Borkar\r\n                   and Eric Roman\r\n                   and Mootaz Elnozahy\r\n                   and Bert Still\r\n                   and Andrew Chien\r\n                   and Robert Clay\r\n                   and John Wu\r\n                   and Christian Engelmann\r\n                   and Nathan DeBardeleben\r\n                   and Rob Ross\r\n                   and Larry Kaplan\r\n                   and Martin Schulz\r\n                   and Mike Heroux\r\n                   and Sriram Krishnamoorthy\r\n                   and Lucy Nowell\r\n                   and Abhinav Vishnu\r\n                   and Lee-Ann Talley\",\r\n  title         = \"{U.S. Department of Energy} Fault Management Workshop\",\r\n  howpublished  = \"Workshop report for the U.S. Department of Energy\",\r\n  month         = jun # \"~6, \",\r\n  year          = \"2012\",\r\n  address       = \"Baltimore, MA, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/geist12department.pdf\",\r\n  abstract      = \"A Department of Energy (DOE) Fault Management Workshop was\r\n                   held on June 6, 2012 at the BWI Airport Marriot hotel in\r\n                   Maryland. The goals of this workshop were to: 1. Describe\r\n                   the required HPC resilience for critical DOE mission needs;\r\n                   2. Detail what HPC resilience research is already being done\r\n                   at the DOE national laboratories and is expected to be done\r\n                   by industry or other groups; 3. Determine what fault\r\n                   management research is a priority for DOE's Office of\r\n                   Science and National Nuclear Security Administration\r\n                   (NNSA) over the next five years; 4. Develop a roadmap for\r\n                   getting the necessary research accomplished in the timeframe\r\n                   when it will be needed by the large computing facilities\r\n                   across DOE.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann12performance,\r\n  author        = \"Christian Engelmann\r\n                   and Thomas Naughton\",\r\n  title         = \"A Performance\/Resilience\/Power Co-design Tool for\r\n                   Extreme-scale High-Performance Computing\",\r\n  howpublished  = \"White paper accepted at the U.S. Department of Energy's\r\n                   \\href{http:\/\/hpc.pnl.gov\/modsim\/2012}{Workshop on Modeling\r\n                   & Simulation of Exascale Systems & Applications (ModSim)\r\n                   2012}\",\r\n  month         = aug # \"~9-10, \",\r\n  year          = \"2012\",\r\n  address       = \"Seattle, WA, USA\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann12performance.pdf\",\r\n  abstract      = \"Performance, resilience and power consumption are key HPC\r\n                   system design factors that are highly interde-pendent. To\r\n                   enable extreme-scale computing it is essential to perform\r\n                   HPC hardware\/software co-design that identifies the\r\n                   cost\/benefit trade-off between these design factors for\r\n                   potential future architecture choices. The proposed research\r\n                   and development aims at developing an HPC hardware\/software\r\n                   co-design toolkit for evaluating the\r\n                   resilience\/power\/performance cost\/benefit trade-off of\r\n                   future architecture choices. The approach focuses on\r\n                   extending a simulation-based performance investigation\r\n                   toolkit with advanced resilience and power modeling and\r\n                   simulation features, such as (i) fault injection mechanisms,\r\n                   (ii) fault propagation, isolation, and detection models, (i)\r\n                   fault avoidance, masking, and recovery simulation, and (iv)\r\n                   power consumption models.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann12dynamic,\r\n  author        = \"Christian Engelmann\r\n                   and Geoffroy R. Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Frank Mueller\",\r\n  title         = \"Dynamic Self-Aware Runtime Software for Exascale Systems\",\r\n  howpublished  = \"White paper for the U.S. Department of Energy's\r\n                   \\href{https:\/\/collab.cels.anl.gov\/display\/exaosr\/Position+Papers}\r\n                   {Exascale Operating Systems and Runtime Technical Council}\",\r\n  month         = jul,\r\n  year          = \"2012\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann12dynamic.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann12dynamic.ppt.pdf\",\r\n  abstract      = \"At exascale, the power consumption, resilience, and load\r\n                   balancing constraints, especially their dynamic nature and\r\n                   interdependence, and the scale of the system require a\r\n                   radical change in future high-performance computing (HPC)\r\n                   operating systems and runtimes (OS\/Rs). In contrast to the\r\n                   existing static OS\/R solutions, an exascale OS\/R is needed\r\n                   that is aware of the dynamically changing resources,\r\n                   constraints, and application needs, and that is able to\r\n                   autonomously coordinate (sometimes conflicting) responses\r\n                   to different changes in the system, simultaneously and at\r\n                   scale. To provide awareness and autonomic management, a\r\n                   novel, scalable and self-aware OS\/R is needed that becomes\r\n                   the brains of the entire X-stack. It dynamically analyzes\r\n                   past, current, and future system status and application\r\n                   needs. It optimizes system usage by scheduling, migrating,\r\n                   and restarting tasks within and across nodes as needed to\r\n                   deal with multi-dimensional constraints, such as power\r\n                   consumption, permanent and transient faults, resource\r\n                   degradation, heterogeneity, data locality, and load balance.\"\r\n}<\/pre>\n
<\/a>@misc{vallee12unified,\r\n  author        = \"Geoffroy R. Vall\\'ee\r\n                   and Thomas Naughton\r\n                   and Christian Engelmann\r\n                   and David E. Bernholdt\",\r\n  title         = \"Unified Execution Environment\",\r\n  howpublished  = \"White paper for the U.S. Department of Energy's\r\n                   \\href{https:\/\/collab.cels.anl.gov\/display\/exaosr\/Position+Papers}\r\n                   {Exascale Operating Systems and Runtime Technical Council}\",\r\n  month         = jul,\r\n  year          = \"2012\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/vallee12unified.pdf\",\r\n  abstract      = \"The design and development of new system software for HPC\r\n                   (both operating systems and run-times) face multiple\r\n                   challenges, including scalability (high level of parallelism),\r\n                   efficiency, resiliency, and dynamicity. Guided by these\r\n                  fundamental design principles, we advocate for a unified\r\n                  execution environment, which aims at being scalable,\r\n                  asynchronous, dynamic, resource efficient, and reusable. The\r\n                  proposed solution is based on the following core building\r\n                  blocks, (i) events, (ii) agents, and (iii) enclaves. We use\r\n                  these building blocks to support composable environments that\r\n                  may be tailored to combine appropriate system services as well\r\n                  as user jobs.  Additionally,  for resilience and scalability\r\n                  the proposed design encourages localized or regional\r\n                  operations to foster autonomy of execution contexts. We\r\n                  advocate this approach for exascale systems, which include a\r\n                  massive number of heterogeneous computing resources, since it\r\n                  enables architecturally informed structures (topologies) as\r\n                  well as encouraging efficient grouping of\r\n                  functionality\/services.\"\r\n}<\/pre>\n
<\/a>@misc{debardeleben09high-end,\r\n  author        = \"Nathan DeBardeleben\r\n                   and James Laros\r\n                   and John T. Daly\r\n                   and Stephen L. Scott\r\n                   and Christian Engelmann\r\n                   and Bill Harrod\",\r\n  title         = \"High-End Computing Resilience: {Analysis} of Issues\r\n                   Facing the {HEC} Community and Path-Forward for\r\n                   Research and Development\",\r\n  howpublished  = \"White paper for the U.S. National Science Foundation's High-end Computing Program\",\r\n  month         = dec,\r\n  year          = \"2009\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/debardeleben09high-end.pdf\"\r\n}<\/pre>\n
<\/a>@techreport{etz25olcf,\r\n  author        = \"Brian Etz\r\n                   and Oral, Sarp\r\n                   and Rafael Ferreira Da Silva\r\n                   and Ryan Adamson\r\n                   and Anees Alnajjar\r\n                   and Tom Beck\r\n                   and Ashley Barker\r\n                   and Michael Brim\r\n                   and Paul Bryant\r\n                   and Christian Engelmann\r\n                   and Anjus George\r\n                   and Samuel Herts\r\n                   and Gustav Jansen\r\n                   and Rajesh Kalyanam\r\n                   and Ahmad Maroof Karimi\r\n                   and Jack Lange\r\n                   and Kellen Leland\r\n                   and Ketan Maheshwari\r\n                   and Marshall McDonnell\r\n                   and Bronson Messer II\r\n                   and Ross Miller\r\n                   and Daniel S. Pelfrey\r\n                   and Suzanne Prentice\r\n                   and Bran Radovanovic\r\n                   and David Rogers\r\n                   and Daniel Rosendo\r\n                   and A.J. Ruckman\r\n                   and Mallikarjun (Arjun) Shankar\r\n                   and Amir Shehata\r\n                   and Tyler Skluzacek\r\n                   and Renan Santos Souza\r\n                   and Veronica Melesse Vergar\r\n                   and Feiyi Wang\r\n                   and Jordan Webb\r\n                   and Patrick Widener\r\n                   and Christopher Zimmer\",\r\n  title         = \"{OLCF}'s Advanced Computing Ecosystem ({ACE}): {FY25} Update for Ongoing Efforts\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2025\/4050\",\r\n  month         = nov # \"~30, \",\r\n  year          = \"2025\",\r\n  doi           = \"10.2172\/3006499\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/etz25olcf.pdf\",\r\n  pts           = \"241439\"\r\n}<\/pre>\n
<\/a>@techreport{dasilva24shaping,\r\n  author        = \"Rafael Ferreira da Silva\r\n                   and Robert Moore\r\n                   and Benjamin Mintz\r\n                   and Rigoberto Advincula\r\n                   and Anees Alnajjar\r\n                   and Luke Baldwin\r\n                   and Craig Bridges\r\n                   and Ryan Coffee\r\n                   and Ewa Deelman\r\n                   and Christian Engelmann\r\n                   and Brian Etz\r\n                   and Millie Firestone\r\n                   and Ian Foster\r\n                   and Panchapakesan Ganesh\r\n                   and Leslie Hamilton\r\n                   and Dale Huber\r\n                   and Ilia Ivanov\r\n                   and Shantenu Jha\r\n                   and Ying Li\r\n                   and Yongtao Liu\r\n                   and Jay Lofstead\r\n                   and Anirban Mandal\r\n                   and Hector Martin\r\n                   and Theresa Mayer\r\n                   and Marshall McDonnell\r\n                   and Vijayakumar Murugesan\r\n                   and Sal Nimer\r\n                   and Nageswara Rao\r\n                   and Martin Seifrid\r\n                   and Mitra Taheri\r\n                   and Michela Taufer\r\n                   and Konstantinos Vogiatzis\",\r\n  title         = \"Shaping the Future of Self-Driving Autonomous Laboratories Workshop\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2024\/3714\",\r\n  month         = jan # \"~2, \",\r\n  year          = \"2024\",\r\n  publisher     = \"Zenodo\",\r\n  doi           = \"10.5281\/zenodo.14430232\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/dasilva24shaping.pdf\",\r\n  abstract      = \"The Shaping the Future of Self-Driving Autonomous\r\n                   Laboratories workshop, held in Denver on November 7-8, 2024,\r\n                   brought together leading experts from materials science and\r\n                   computing to address the growing need to revolutionize\r\n                   scientific research through AI-driven autonomous laboratories.\r\n                   The workshop identified critical challenges, including the\r\n                   integration of heterogeneous data, development of AI systems\r\n                   that understand fundamental physical principles, and\r\n                   comprehensive safety protocols. Key recommendations emerged\r\n                   around developing universal laboratory equipment interfaces,\r\n                   implementing automated metadata collection systems, and\r\n                   creating hybrid AI approaches that combine data-driven\r\n                   learning with scientific principles. The workshop\r\n                   emphasized maintaining human oversight while leveraging\r\n                   automation, transforming scientific education to prepare the\r\n                   next generation of researchers, and establishing a national\r\n                   consortium leveraging DOE facilities as anchors for broader\r\n                   collaboration with academia and industry. Participants\r\n                   stressed the urgency of addressing the growing disconnect\r\n                   between human decision-making timescales and modern\r\n                   instrumentation capabilities, highlighting the need for\r\n                   strategic automation while preserving essential human insight\r\n                   and oversight in the research process.\",\r\n  pts           = \"227078\"\r\n}<\/pre>\n
<\/a>@techreport{brim23microservice,\r\n  author        = \"Michael Brim\r\n                   and Christian Engelmann\",\r\n  title         = \"INTERSECT Architecture Specification:\r\n                   Microservice Architecture (Version 0.9)\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2023\/3171\",\r\n  address       = \"Oak Ridge, TN, USA\",\r\n  month         = sep # \"~30, \",\r\n  year          = \"2023\",\r\n  doi           = \"10.2172\/2333815\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/brim23microservice.pdf\",\r\n  abstract      = \"Oak Ridge National Laboratory (ORNL)'s Self-driven\r\n                   Experiments for Science \/ Interconnected Science Ecosystem\r\n                   (INTERSECT) architecture project, titled ``An Open Federated\r\n                   Architecture for the Laboratory of the Future'', creates an\r\n                   open federated hardware\/software architecture for the\r\n                   laboratory of the future using a novel system of systems\r\n                   (SoS) and microservice architecture approach, connecting\r\n                   scientific instruments, robot-controlled laboratories and\r\n                   edge\/center computing\/data resources to enable autonomous\r\n                   experiments, ``self-driving'' laboratories, smart\r\n                   manufacturing, and artificial intelligence (AI)-driven\r\n                   design, discovery and evaluation. The project describes\r\n                   science use cases as design patterns that identify and\r\n                   abstract the involved hardware\/software components and their\r\n                   interactions in terms of control, work and data flow. It\r\n                   creates a SoS architecture of the federated\r\n                   hardware\/software ecosystem that clarifies terms,\r\n                   architectural elements, the interactions between them and\r\n                   compliance. It further designs a federated microservice\r\n                   architecture, mapping science use case design patterns to the\r\n                   SoS architecture with loosely coupled microservices,\r\n                   standardized interfaces and multi programming language\r\n                   support. The primary deliverable of this project is an\r\n                   INTERSECT Open Architecture Specification, containing the\r\n                   science use case design pattern catalog, the federated SoS\r\n                   architecture specification and the federated microservice\r\n                   architecture specification. This document represents the\r\n                   microservice architecture of the INTERSECT Open Architecture\r\n                   Specification.\",\r\n  pts           = \"204232\"\r\n}<\/pre>\n
<\/a>@techreport{engelmann23use,\r\n  author        = \"Christian Engelmann\r\n                   and Suhas Somnath\",\r\n  title         = \"INTERSECT Architecture Specification: Use Case Design\r\n                   Patterns (Version 0.9)\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2023\/3133\",\r\n  address       = \"Oak Ridge, TN, USA\",\r\n  month         = sep # \"~30, \",\r\n  year          = \"2023\",\r\n  doi           = \"10.2172\/2229218\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann23use.pdf\",\r\n  abstract      = \"Connecting scientific instruments and robot-controlled\r\n                   laboratories with computing and data resources at the edge,\r\n                   the Cloud or the high-performance computing (HPC) center\r\n                   enables autonomous experiments, self-driving laboratories,\r\n                   smart manufacturing, and artificial intelligence (AI)-driven\r\n                   design, discovery and evaluation. The Self-driven Experiments\r\n                   for Science \/ Interconnected Science Ecosystem (INTERSECT)\r\n                   Open Architecture enables science breakthroughs using\r\n                   intelligent networked systems, instruments and facilities\r\n                   with a federated hardware\/software architecture for the\r\n                   laboratory of the future. It relies on a novel approach,\r\n                   consisting of (1) science use case design patterns, (2) a\r\n                   system of systems architecture, and (3) a microservice\r\n                   architecture. This document introduces the science use case\r\n                   design patterns of the INTERSECT Architecture. It describes\r\n                   the overall background, the involved terminology and\r\n                   concepts, and the pattern format and classification. It\r\n                   further details the 12 defined patterns and provides insight\r\n                   into building solutions from these patterns. The document\r\n                   also describes the application of these patterns in the\r\n                   context of several INTERSECT autonomous laboratories. The\r\n                   target audience are computer, computational, instrument and\r\n                   domain science experts working in the field of autonomous\r\n                   experiments.\",\r\n  pts           = \"203995\"\r\n}<\/pre>\n
<\/a>@techreport{engelmann22rdp-20,\r\n  author        = \"Christian Engelmann\r\n                   and Rizwan Ashraf\r\n                   and Saurabh Hukerikar\r\n                   and Mohit Kumar\r\n                   and Piyush Sao\",\r\n  title         = \"Resilience Design Patterns: {A} Structured Approach to\r\n                   Resilience at Extreme Scale (Version 2.0)\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2022\/2809\",\r\n  address       = \"Oak Ridge, TN, USA\",\r\n  month         = dec # \"~16, \",\r\n  year          = \"2022\",\r\n  doi           = \"10.2172\/1922296\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann22rdp-20.pdf\",\r\n  abstract      = \"Reliability is a serious concern for future extreme-scale\r\n                   high-performance computing (HPC) systems. Projections based\r\n                   on the current generation of HPC systems and technology\r\n                   roadmaps suggest the prevalence of very high fault rates in\r\n                   future systems. The errors resulting from these faults will\r\n                   propagate and generate various kinds of failures, which may\r\n                   result in outcomes ranging from result corruptions to\r\n                   catastrophic application crashes. Therefore, the resilience\r\n                   challenge for extreme-scale HPC systems requires coordination\r\n                   between various hardware and software technologies that are\r\n                   capable of handling a broad set of fault models at\r\n                   accelerated fault rates. Also, due to practical limits on\r\n                   power consumption in future HPC systems, they are likely to\r\n                   embrace innovative architectures, increasing the levels of\r\n                   hardware and software complexities. Therefore, the\r\n                   techniques that seek to improve resilience must navigate the\r\n                   complex trade-off space between resilience and the overheads\r\n                   to power consumption and performance. While the HPC community\r\n                   has developed various resilience solutions, application-level\r\n                   techniques as well as system-based solutions, the solution\r\n                   space of HPC resilience techniques remains fragmented. There\r\n                   are no formal methods to integrate the various HPC resilience\r\n                   techniques into composite solutions, nor are there methods to\r\n                   holistically evaluate the adequacy and efficacy of such\r\n                   solutions in terms of their protection coverage, and their\r\n                   performance & power efficiency characteristics. Additionally,\r\n                   few implementations of current resilience solutions are\r\n                   portable to newer architectures and software environments that\r\n                   will be deployed on future systems.\r\n                   We developed a new structured approach to the management of\r\n                   HPC resilience using the concept of resilience-based design\r\n                   patterns. In general, a design pattern is a repeatable\r\n                   solution to a commonly occurring problem. We identified the\r\n                   well-known solutions that are commonly used to deal with\r\n                   faults, errors and failures in HPC systems. In the initial\r\n                   design patterns specification (version 1.0), we described\r\n                   the various solutions, which address specific problems in\r\n                   the design of resilient HPC environments, in the form of\r\n                   patterns. Each pattern describes a problem caused by a fault,\r\n                   error or failure event in an HPC environment, and then\r\n                   describes the core of the solution of the problem in such a\r\n                   way that this solution may be adapted to different systems\r\n                   and implemented at different layers of the system stack. The\r\n                   catalog of these resilience design patterns provides\r\n                   designers with a collection of design elements. To construct\r\n                   complete resilience solutions using combinations of various\r\n                   patterns, we defined a framework that enhances HPC designers'\r\n                   understanding of the important constraints and the\r\n                   opportunities for the design patterns to be implemented and\r\n                   deployed at various layers of the system stack. The design\r\n                   framework is also useful for establishing interfaces and\r\n                   mechanisms to coordinate flexible fault management across\r\n                   hardware and software components, as well as to consider the\r\n                   trade-off between performance, resilience, and power\r\n                   consumption when constructing a solution. The resilience\r\n                   design patterns specification version 1.1 included more\r\n                   detailed explanations of the pattern solutions, the context\r\n                   in which the patterns are applicable, and the implications\r\n                   for hardware or software design. It also provided several\r\n                   additional examples and detailed case studies to demonstrate\r\n                   the use of patterns to build realistic solutions.\r\n                   In this version 1.2 of the specification document, we have\r\n                   improved the pattern descriptions, including graphical\r\n                   representations of the pattern components. These\r\n                   improvements are largely based on critical comments,\r\n                   feedback and suggestions received from pattern experts and\r\n                   readers of the previous versions of the specification. The\r\n                   pattern classification has been modified to further clarify\r\n                   the relationships between pattern categories. This version\r\n                   of the specification also introduces a pattern language for\r\n                   resilience design patterns. The pattern language presents\r\n                   the patterns in the catalog as a network, revealing the\r\n                   relations among the resilience patterns. The language\r\n                   provides designers with the means to explore alternative\r\n                   techniques for handling a specific fault model that may have\r\n                   different efficiency and complexity characteristics. Using\r\n                   the pattern language also enables the design and\r\n                   implementation of comprehensive resilience solutions as a\r\n                   set of interconnected resilience patterns that can be\r\n                   instantiated across layers of the system stack. The overall\r\n                   goal of this work is to provide hardware and software\r\n                   designers, as well as the users and operators of HPC systems,\r\n                   a systematic methodology for the design and evaluation of\r\n                   resilience technologies in HPC systems that keep scientific\r\n                   applications running to a correct solution in a timely and\r\n                   cost-efficient manner despite frequent faults, errors, and\r\n                   failures of various types.\r\n                   Version 2.0 expands the resilience design pattern\r\n                   classification and catalog to include self-stabilization\r\n                   patterns and reliability, availability and performance models\r\n                   for each structural pattern.\",\r\n  pts           = \"189180\"\r\n}<\/pre>\n
<\/a>@techreport{brim22microservice,\r\n  author        = \"Michael Brim\r\n                   and Christian Engelmann\",\r\n  title         = \"INTERSECT Architecture Specification:\r\n                   Microservice Architecture (Version 0.5)\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2022\/2715\",\r\n  address       = \"Oak Ridge, TN, USA\",\r\n  month         = sep # \"~30, \",\r\n  year          = \"2022\",\r\n  doi           = \"10.2172\/1902805\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/brim22microservice.pdf\",\r\n  abstract      = \"Oak Ridge National Laboratory (ORNL)'s Self-driven\r\n                   Experiments for Science \/ Interconnected Science Ecosystem\r\n                   (INTERSECT) architecture project, titled ``An Open Federated\r\n                   Architecture for the Laboratory of the Future'', creates an\r\n                   open federated hardware\/software architecture for the\r\n                   laboratory of the future using a novel system of systems\r\n                   (SoS) and microservice architecture approach, connecting\r\n                   scientific instruments, robot-controlled laboratories and\r\n                   edge\/center computing\/data resources to enable autonomous\r\n                   experiments, ``self-driving'' laboratories, smart\r\n                   manufacturing, and artificial intelligence (AI)-driven\r\n                   design, discovery and evaluation. The project describes\r\n                   science use cases as design patterns that identify and\r\n                   abstract the involved hardware\/software components and their\r\n                   interactions in terms of control, work and data flow. It\r\n                   creates a SoS architecture of the federated\r\n                   hardware\/software ecosystem that clarifies terms,\r\n                   architectural elements, the interactions between them and\r\n                   compliance. It further designs a federated microservice\r\n                   architecture, mapping science use case design patterns to the\r\n                   SoS architecture with loosely coupled microservices,\r\n                   standardized interfaces and multi programming language\r\n                   support. The primary deliverable of this project is an\r\n                   INTERSECT Open Architecture Specification, containing the\r\n                   science use case design pattern catalog, the federated SoS\r\n                   architecture specification and the federated microservice\r\n                   architecture specification. This document represents the\r\n                   microservice architecture of the INTERSECT Open Architecture\r\n                   Specification.\",\r\n  pts           = \"186195\"\r\n}<\/pre>\n
<\/a>@techreport{engelmann22use,\r\n  author        = \"Christian Engelmann\r\n                   and Suhas Somnath\",\r\n  title         = \"INTERSECT Architecture Specification: Use Case Design\r\n                   Patterns (Version 0.5)\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2022\/2681\",\r\n  address       = \"Oak Ridge, TN, USA\",\r\n  month         = sep # \"~30, \",\r\n  year          = \"2022\",\r\n  doi           = \"10.2172\/1896984\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann22use.pdf\",\r\n  abstract      = \"Oak Ridge National Laboratory (ORNL)'s Self-driven\r\n                   Experiments for Science \/ Interconnected Science Ecosystem\r\n                   (INTERSECT) architecture project, titled ``An Open Federated\r\n                   Architecture for the Laboratory of the Future'', creates an\r\n                   open federated hardware\/software architecture for the\r\n                   laboratory of the future using a novel system of systems\r\n                   (SoS) and microservice architecture approach, connecting\r\n                   scientific instruments, robot-controlled laboratories and\r\n                   edge\/center computing\/data resources to enable autonomous\r\n                   experiments, ``self-driving'' laboratories, smart\r\n                   manufacturing, and artificial intelligence (AI)-driven\r\n                   design, discovery and evaluation. The project describes\r\n                   science use cases as design patterns that identify and\r\n                   abstract the involved hardware\/software components and their\r\n                   interactions in terms of control, work and data flow. It\r\n                   creates a SoS architecture of the federated\r\n                   hardware\/software ecosystem that clarifies terms,\r\n                   architectural elements, the interactions between them and\r\n                   compliance. It further designs a federated microservice\r\n                   architecture, mapping science use case design patterns to the\r\n                   SoS architecture with loosely coupled microservices,\r\n                   standardized interfaces and multi programming language\r\n                   support. The primary deliverable of this project is an\r\n                   INTERSECT Open Architecture Specification, containing the\r\n                   science use case design pattern catalog, the federated SoS\r\n                   architecture specification and the federated microservice\r\n                   architecture specification. This document represents the\r\n                   science use case design pattern catalog of the INTERSECT Open\r\n                   Architecture Specification.\",\r\n  pts           = \"185612\"\r\n}<\/pre>\n
<\/a>@techreport{hukerikar17rdp-12,\r\n  author        = \"Saurabh Hukerikar\r\n                   and Christian Engelmann\",\r\n  title         = \"Resilience Design Patterns: {A} Structured Approach to\r\n                   Resilience at Extreme Scale (Version 1.2)\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2017\/745\",\r\n  address       = \"Oak Ridge, TN, USA\",\r\n  month         = aug # \"~1, \",\r\n  year          = \"2017\",\r\n  doi           = \"10.2172\/1436045\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/hukerikar17rdp-12.pdf\",\r\n  abstract      = \"Reliability is a serious concern for future extreme-scale\r\n                   high-performance computing (HPC) systems. Projections based\r\n                   on the current generation of HPC systems and technology\r\n                   roadmaps suggest the prevalence of very high fault rates in\r\n                   future systems. The errors resulting from these faults will\r\n                   propagate and generate various kinds of failures, which may\r\n                   result in outcomes ranging from result corruptions to\r\n                   catastrophic application crashes. Therefore, the resilience\r\n                   challenge for extreme-scale HPC systems requires coordination\r\n                   between various hardware and software technologies that are\r\n                   capable of handling a broad set of fault models at\r\n                   accelerated fault rates. Also, due to practical limits on\r\n                   power consumption in future HPC systems, they are likely to\r\n                   embrace innovative architectures, increasing the levels of\r\n                   hardware and software complexities. Therefore, the\r\n                   techniques that seek to improve resilience must navigate the\r\n                   complex trade-off space between resilience and the overheads\r\n                   to power consumption and performance. While the HPC community\r\n                   has developed various resilience solutions, application-level\r\n                   techniques as well as system-based solutions, the solution\r\n                   space of HPC resilience techniques remains fragmented. There\r\n                   are no formal methods to integrate the various HPC resilience\r\n                   techniques into composite solutions, nor are there methods to\r\n                   holistically evaluate the adequacy and efficacy of such\r\n                   solutions in terms of their protection coverage, and their\r\n                   performance & power efficiency characteristics. Additionally,\r\n                   few implementations of current resilience solutions are\r\n                   portable to newer architectures and software environments that\r\n                   will be deployed on future systems.\r\n                   We developed a new structured approach to the management of\r\n                   HPC resilience using the concept of resilience-based design\r\n                   patterns. In general, a design pattern is a repeatable\r\n                   solution to a commonly occurring problem. We identified the\r\n                   well-known solutions that are commonly used to deal with\r\n                   faults, errors and failures in HPC systems. In the initial\r\n                   design patterns specification (version 1.0), we described\r\n                   the various solutions, which address specific problems in\r\n                   the design of resilient HPC environments, in the form of\r\n                   patterns. Each pattern describes a problem caused by a fault,\r\n                   error or failure event in an HPC environment, and then\r\n                   describes the core of the solution of the problem in such a\r\n                   way that this solution may be adapted to different systems\r\n                   and implemented at different layers of the system stack. The\r\n                   catalog of these resilience design patterns provides\r\n                   designers with a collection of design elements. To construct\r\n                   complete resilience solutions using combinations of various\r\n                   patterns, we defined a framework that enhances HPC designers'\r\n                   understanding of the important constraints and the\r\n                   opportunities for the design patterns to be implemented and\r\n                   deployed at various layers of the system stack. The design\r\n                   framework is also useful for establishing interfaces and\r\n                   mechanisms to coordinate flexible fault management across\r\n                   hardware and software components, as well as to consider the\r\n                   trade-off between performance, resilience, and power\r\n                   consumption when constructing a solution. The resilience\r\n                   design patterns specification version 1.1 included more\r\n                   detailed explanations of the pattern solutions, the context\r\n                   in which the patterns are applicable, and the implications\r\n                   for hardware or software design. It also provided several\r\n                   additional examples and detailed case studies to demonstrate\r\n                   the use of patterns to build realistic solutions.\r\n                   In this version 1.2 of the specification document, we have\r\n                   improved the pattern descriptions, including graphical\r\n                   representations of the pattern components. These\r\n                   improvements are largely based on critical comments,\r\n                   feedback and suggestions received from pattern experts and\r\n                   readers of the previous versions of the specification. The\r\n                   pattern classification has been modified to further clarify\r\n                   the relationships between pattern categories. This version\r\n                   of the specification also introduces a pattern language for\r\n                   resilience design patterns. The pattern language presents\r\n                   the patterns in the catalog as a network, revealing the\r\n                   relations among the resilience patterns. The language\r\n                   provides designers with the means to explore alternative\r\n                   techniques for handling a specific fault model that may have\r\n                   different efficiency and complexity characteristics. Using\r\n                   the pattern language also enables the design and\r\n                   implementation of comprehensive resilience solutions as a\r\n                   set of interconnected resilience patterns that can be\r\n                   instantiated across layers of the system stack. The overall\r\n                   goal of this work is to provide hardware and software\r\n                   designers, as well as the users and operators of HPC systems,\r\n                   a systematic methodology for the design and evaluation of\r\n                   resilience technologies in HPC systems that keep scientific\r\n                   applications running to a correct solution in a timely and\r\n                   cost-efficient manner despite frequent faults, errors, and\r\n                   failures of various types.\",\r\n  pts           = \"106427\"\r\n}<\/pre>\n
<\/a>@techreport{hukerikar16rdp-11,\r\n  author        = \"Saurabh Hukerikar\r\n                   and Christian Engelmann\",\r\n  title         = \"Resilience Design Patterns: {A} Structured Approach to\r\n                   Resilience at Extreme Scale (Version 1.1)\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2016\/767\",\r\n  address       = \"Oak Ridge, TN, USA\",\r\n  month         = dec # \"~1, \",\r\n  year          = \"2016\",\r\n  doi           = \"10.2172\/1345793\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/hukerikar16rdp-11.pdf\",\r\n  abstract      = \"Reliability is a serious concern for future extreme-scale\r\n                   high-performance computing (HPC) systems. Projections based\r\n                   on the current generation of HPC systems and technology\r\n                   roadmaps suggest the prevalence of very high fault rates in\r\n                   future systems. The errors resulting from these faults will\r\n                   propagate and generate various kinds of failures, which may\r\n                   result in outcomes ranging from result corruptions to\r\n                   catastrophic application crashes. Therefore the resilience\r\n                   challenge for extreme-scale HPC systems requires management\r\n                   of various hardware and software technologies that are\r\n                   capable of handling a broad set of fault models at\r\n                   accelerated fault rates. Also, due to practical limits on\r\n                   power consumption in HPC systems future systems are likely\r\n                   to embrace innovative architectures, increasing the levels\r\n                   of hardware and software complexities. As a result the\r\n                   techniques that seek to improve resilience must navigate\r\n                   the complex trade-off space between resilience and the\r\n                   overheads to power consumption and performance. While the\r\n                   HPC community has developed various resilience solutions,\r\n                   application-level techniques as well as system-based\r\n                   solutions, the solution space of HPC resilience techniques\r\n                   remains fragmented. There are no formal methods and metrics\r\n                   to investigate and evaluate resilience holistically in HPC\r\n                   systems that consider impact scope, handling coverage, and\r\n                   performance & power efficiency across the system stack.\r\n                   Additionally, few of the current approaches are portable to\r\n                   newer architectures and software environments that will be\r\n                   deployed on future systems.\r\n                   In this document, we develop a structured approach to the\r\n                   management of HPC resilience using the concept of\r\n                   resilience-based design patterns. A design pattern is a\r\n                   general repeatable solution to a commonly occurring problem.\r\n                   We identify the commonly occurring problems and solutions\r\n                   used to deal with faults, errors and failures in HPC systems.\r\n                   Each established solution is described in the form of a\r\n                   pattern that addresses concrete problems in the design of\r\n                   resilient systems. The complete catalog of resilience design\r\n                   patterns provides designers with reusable design elements. We\r\n                   also define a framework that enhances a designer's\r\n                   understanding of the important constraints and opportunities\r\n                   for the design patterns to be implemented and deployed at\r\n                   various layers of the system stack. This design framework may\r\n                   be used to establish mechanisms and interfaces to coordinate\r\n                   flexible fault management across hardware and software\r\n                   components. The framework also supports optimization of the\r\n                   cost-benefit trade-offs among performance, resilience, and\r\n                   power consumption. The overall goal of this work is to enable\r\n                   a systematic methodology for the design and evaluation of\r\n                   resilience technologies in extreme-scale HPC systems that\r\n                   keep scientific applications running to a correct solution\r\n                   in a timely and cost-efficient manner in spite of frequent\r\n                   faults, errors, and failures of various types.\",\r\n  pts           = \"72341\"\r\n}<\/pre>\n
<\/a>@techreport{hukerikar16rdp-10,\r\n  author        = \"Saurabh Hukerikar\r\n                   and Christian Engelmann\",\r\n  title         = \"Resilience Design Patterns: {A} Structured Approach to\r\n                   Resilience at Extreme Scale (Version 1.0)\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2016\/687\",\r\n  address       = \"Oak Ridge, TN, USA\",\r\n  month         = oct # \"~1, \",\r\n  year          = \"2016\",\r\n  doi           = \"10.2172\/1338552\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/hukerikar16rdp-10.pdf\",\r\n  abstract      = \"Reliability is a serious concern for future extreme-scale\r\n                   high-performance computing (HPC) systems. Projections based\r\n                   on the current generation of HPC systems and technology\r\n                   roadmaps suggest that very high fault rates in future\r\n                   systems. The errors resulting from these faults will\r\n                   propagate and generate various kinds of failures, which may\r\n                   result in outcomes ranging from result corruptions to\r\n                   catastrophic application crashes. Practical limits on power\r\n                   consumption in HPC systems will require future systems to\r\n                   embrace innovative architectures, increasing the levels of\r\n                   hardware and software complexities.\r\n                   The resilience challenge for extreme-scale HPC systems\r\n                   requires management of various hardware and software\r\n                   technologies that are capable of handling a broad set of\r\n                   fault models at accelerated fault rates. These techniques\r\n                   must seek to improve resilience at reasonable overheads to\r\n                   power consumption and performance. While the HPC community\r\n                   has developed various solutions, application-level as well\r\n                   as system-based solutions, the solution space of HPC\r\n                   resilience techniques remains fragmented. There are no formal\r\n                   methods and metrics to investigate and evaluate resilience\r\n                   holistically in HPC systems that consider impact scope,\r\n                   handling coverage, and performance & power efficiency across\r\n                   the system stack. Additionally, few of the current approaches\r\n                   are portable to newer architectures and software ecosystems,\r\n                   which are expected to be deployed on future systems.\r\n                   In this document, we develop a structured approach to the\r\n                   management of HPC resilience based on the concept of\r\n                   resilience-based design patterns. A design pattern is a\r\n                   general repeatable solution to a commonly occurring problem.\r\n                   We identify the commonly occurring problems and solutions\r\n                   used to deal with faults, errors and failures in HPC systems.\r\n                   The catalog of resilience design patterns provides designers\r\n                   with reusable design elements. We define a design framework\r\n                   that enhances our understanding of the important constraints\r\n                   and opportunities for solutions deployed at various layers of\r\n                   the system stack. The framework may be used to establish\r\n                   mechanisms and interfaces to coordinate flexible fault\r\n                   management across hardware and software components. The\r\n                   framework also enables optimization of the cost-benefit\r\n                   trade-offs among performance, resilience, and power\r\n                   consumption. The overall goal of this work is to enable a\r\n                   systematic methodology for the design and evaluation of\r\n                   resilience technologies in extreme-scale HPC systems that\r\n                   keep scientific applications running to a correct solution\r\n                   in a timely and cost-efficient manner in spite of frequent\r\n                   faults, errors, and failures of various types.\",\r\n  pts           = \"71756\"\r\n}<\/pre>\n
<\/a>@techreport{fiala12detection,\r\n  author        = \"David Fiala\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\r\n                   and Kurt Ferreira\r\n                   and Ron Brightwell\r\n                   and Rolf Riesen\",\r\n  title         = \"Detection and Correction of Silent Data Corruption for\r\n                   Large-Scale High-Performance Computing\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2012\/227\",\r\n  address       = \"Oak Ridge, TN, USA\",\r\n  month         = jun,\r\n  year          = \"2012\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/fiala12detection.pdf\",\r\n  abstract      = \"Faults have become the norm rather than the exception for\r\n                   high-end computing on clusters with 10s\/100s of thousands\r\n                   of cores. Exacerbating this situation, some of these faults\r\n                   remain undetected, manifesting themselves as silent errors\r\n                   that corrupt memory while applications continue to operate\r\n                   and report incorrect results.\r\n                   This paper studies the potential for redundancy to both\r\n                   detect and correct soft errors in MPI message-passing\r\n                   applications. Our study investigates the challenges inherent\r\n                   to detecting soft errors within MPI application while\r\n                   providing transparent MPI redundancy. By assuming a model\r\n                   wherein corruption in application data manifests itself by\r\n                   producing differing MPI message data between replicas, we\r\n                   study the best suited protocols for detecting and correcting\r\n                   MPI data that is the result of corruption.\r\n                   To experimentally validate our proposed detection and\r\n                   correction protocols, we introduce RedMPI, an MPI library\r\n                   which resides in the MPI profiling layer. RedMPI is capable\r\n                   of both online detection and correction of soft errors that\r\n                   occur in MPI applications without requiring any modifications\r\n                   to the application source by utilizing either double or\r\n                   triple redundancy.\r\n                   Our results indicate that our most efficient consistency\r\n                   protocol can successfully protect applications experiencing\r\n                   even high rates of silent data corruption with runtime\r\n                   overheads between 0\\% and 30\\% as compared to unprotected\r\n                   applications without redundancy.\r\n                   Using our fault injector within RedMPI, we observe that even\r\n                   a single soft error can have profound effects on running\r\n                   applications, causing a cascading pattern of corruption in\r\n                   most cases causes that spreads to all other processes.\r\n                   RedMPI's protection has been shown to successfully mitigate\r\n                   the effects of soft errors while allowing applications to\r\n                   complete with correct results even in the face of errors.\"\r\n}<\/pre>\n
<\/a>@techreport{wang10hybrid,\r\n  author        = \"Chao Wang\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"Hybrid Full\/Incremental Checkpoint\/Restart for {MPI} Jobs in\r\n                   {HPC} Environments\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2010\/162\",\r\n  address       = \"Oak Ridge, TN, USA\",\r\n  month         = aug,\r\n  year          = \"2010\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/wang10hybrid.pdf\",\r\n  abstract      = \"As the number of cores in high-performance computing\r\n                   environments keeps increasing, faults are becoming common\r\n                   place. Checkpointing addresses such faults but captures\r\n                   full process images even though only a subset of the\r\n                   process image changes between checkpoints.\r\n                   We have designed a high-performance hybrid disk-based\r\n                   full\/incremental checkpointing technique for MPI tasks\r\n                   to capture only data changed since the last checkpoint.\r\n                   Our implementation integrates new BLCR and LAM\/MPI\r\n                   features that complement traditional full checkpoints.\r\n                   This results in significantly reduced checkpoint sizes\r\n                   and overheads with only moderate increases in restart\r\n                   overhead. After accounting for cost and savings, benefits\r\n                   due to incremental checkpoints significantly outweigh the\r\n                   loss on restart operations.\r\n                   Experiments in a cluster with the NAS Parallel Benchmark\r\n                   suite and mpiBLAST indicate that savings due to replacing\r\n                   full checkpoints with incremental ones average 16.64\r\n                   seconds while restore overhead amounts to just 1.17\r\n                   seconds. These savings increase with the frequency of\r\n                   incremental checkpoints. Overall, our novel hybrid\r\n                   full\/incremental checkpointing is superior to prior\r\n                   non-hybrid techniques.\"\r\n}<\/pre>\n
<\/a>@techreport{wang10proactive,\r\n  author        = \"Chao Wang\r\n                   and Frank Mueller\r\n                   and Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"Proactive Process-Level Live Migration and Back Migration in\r\n                   {HPC} Environments\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2010\/161\",\r\n  address       = \"Oak Ridge, TN, USA\",\r\n  month         = aug,\r\n  year          = \"2010\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/wang10proactive.pdf\",\r\n  abstract      = \"As the number of nodes in high-performance computing\r\n                   environments keeps increasing, faults are becoming common\r\n                   place. Reactive fault tolerance (FT) often does not scale\r\n                   due to massive I\/O requirements and relies on manual job\r\n                   resubmission. This work complements reactive with proactive\r\n                   FT at the process level. Through health monitoring, a subset\r\n                   of node failures can be anticipated when one's health\r\n                   deteriorates. A novel process-level live migration mechanism\r\n                   suppor ts continued execution of applications during much of\r\n                   processes migration. This scheme is integrated into an MPI\r\n                   execution environment to transparently sustain\r\n                   health-inflicted node failures, which eradicates the need to\r\n                   restart and requeue MPI jobs. Experiments indicate that 1-6.5\r\n                   seconds of prior warning are required to successfully trigger\r\n                   live process migration while similar operating system\r\n                   virtualization mechanisms require 13-24 seconds. This\r\n                   self-healing approach complements reactive FT by nearly\r\n                   cutting the number of checkpoints in half when 70\\% of\r\n                   the faults are handled proactively. The work also provides\r\n                   a novel back migration approach to eliminate load imbalance\r\n                   or bottlenecks caused by migrated tasks. Experiments indicate\r\n                   the larger the amount of outstanding execution, the higher\r\n                   the benefit due to back migration will be.\"\r\n}<\/pre>\n
<\/a>@dataset{shin23olcf,\r\n  author        = \"Woong Shin\r\n                   and Vladyslav Oles\r\n                   and Anna Schmedding\r\n                   and George Ostrouchov\r\n                   and Evgenia Smirni\r\n                   and Christian Engelmann\r\n                   and Feiyi Wang\",\r\n  title         = \"{OLCF Summit} Supercomputer {GPU} Snapshots During\r\n                   Double-Bit Errors and Normal Operations\",\r\n  month         = apr # \"~20, \",\r\n  year          = \"2023\",\r\n  doi           = \"10.13139\/OLCF\/1970187\",\r\n  url           = \"https:\/\/doi.ccs.ornl.gov\/ui\/doi\/429\",\r\n  abstract      = \"As we move into the exascale era, the power and energy\r\n                   footprints of high-performance computing (HPC) systems have\r\n                   grown significantly larger. Due to the harsh power and\r\n                   thermal conditions the system, components are exposed to\r\n                   extreme operating conditions. Operation of such modern HPC\r\n                   systems requires deep insights into long term system behavior\r\n                   to maintain its efficiency as well as its longevity. To help\r\n                   the HPC community to gain such insights, we provide\r\n                   double-bit errors using system telemetry data and logs\r\n                   collected from the Summit supercomputer, equipped with 27,648\r\n                   Tesla V100 GPUs with 2nd-generation high-bandwidth memory\r\n                   (HBM2). The dataset relies on Nvidia XID records internally\r\n                   collected by GPU firmware at the time of failure occurrence,\r\n                   on the reboot-time logs of each Summit node, on node-level\r\n                   job scheduler records collected after each job termination,\r\n                   and on a 1Hz data rate from the baseboard management\r\n                   controllers (BMCs) of each Summit compute node using the\r\n                   OpenBMC event subscription protocol.\"\r\n}<\/pre>\n
<\/a>@dataset{shankar20gpu,\r\n  author        = \"Mallikarjun Shankar\r\n                   and George Ostrouchov\r\n                   and Don Maxwell\r\n                   and James Rogers\r\n                   and Rizwan Ashraf\r\n                   and Christian Engelmann\",\r\n  title         = \"{GPU} Lifetimes on {Titan} Supercomputer: {Survival} Analysis\r\n                   and Reliability\",\r\n  month         = sep # \"~2, \",\r\n  year          = \"2020\",\r\n  doi           = \"10.13139\/ORNLNCCS\/1657202\",\r\n  url           = \"https:\/\/doi.ccs.ornl.gov\/dataset\/4aa54c30-3d51-5443-b839-88a130e4c713\",\r\n  abstract      = \"George Ostrouchov, Don Maxwell, Rizwan Ashraf, Mallikarjun\r\n                   Shankar, and James Rogers. 2020. GPU Lifetimes on Titan\r\n                   Supercomputer: Survival Analysis and Reliability. In\r\n                   Proceedings of the International Conference for High\r\n                   Performance Computing, Networking, Storage and Analysis\r\n                   (SC '20). Association for Computing Machinery, New York, NY,\r\n                   USA. Data and code for SC20 paper about Titan GPU reliability\r\n                   analysis: https:\/\/github.com\/olcf\/TitanGPULife. Includes R \r\n                   code to generate graphics for paper and additional analyses. \r\n                   See code\/README for instructions. Includes original Titan GPU\r\n                   reliability data on over 100,000 collective hours of\r\n                   operation: data\/titan.gpu.history.txt - history data,\r\n                   data\/titan.service.txt - service nodes for exclusion.\r\n                   Includes output data files produced by\r\n                   code\/TitanGPUmodel.Rmd: data\/gc_full.csv - cleaned up data\r\n                   (see paper and R code); data\/gc_summary_loc.csv - one record\r\n                   per GPU (variables: SN, time, nlife, nloc, last, col, row,\r\n                   cage, slot, node, max_loc_events, time_max_loc, dbe,\r\n                   dbe_loc, otb, otb_loc, out, batch, days, years, dead,\r\n                   dead_otb, dead_dbe) (see paper and R code). Includes .Rmd\r\n                   analysis document as TitanGPUmode.html. Includes Python code\r\n                   to process data\/gc_full.csv into graphics from\r\n                   time-between-failure analyses: See code\/tbf-analyses\/README\r\n                   for instructions.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann25towards,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Towards a Strategy for Future Research Infrastructures\",\r\n  month         = jun # \"~12, \",\r\n  year          = \"2025\",\r\n  howpublished  = \"{Invited panelist at a Birds of a Feather session at the\r\n                   \\href{https:\/\/www.isc-hpc.com}{$40^{th}$ ISC High Performance\r\n                   Conference (ISC) 2025}, Hamburg, Germany}\",\r\n  url           = \"https:\/\/app.swapcard.com\/event\/isc-high-performance-2025\/planning\/UGxhbm5pbmdfMjU4NjExMg==\",\r\n  abstract      = \"The amount of data gathered, shared and analysed in frontier\r\n                   research is set to increase dramatically in the coming\r\n                   decade, leading to unprecedented data processing,\r\n                   simulation\/prediction and analysis needs. As prime examples,\r\n                   the High Energy Physics and Radio Astronomy communities are\r\n                   gearing up to operate groundbreaking instruments such as the\r\n                   High-Luminosity Large Hadron Collider (LHC) and the Square\r\n                   Kilometer Array (SKA) , which will need data and compute\r\n                   capabilities many times larger than the currently available\r\n                   resources. Given the data volumes produced by these\r\n                   instruments, the size of the associated scientific\r\n                   communities and the scale of the analysis and computation\r\n                   problems, it is clear that distributed infrastructures\r\n                   integrating Edge, Cloud and large HPC\/AI centres into a data\r\n                   and compute continuum will be required.. This BoF will bring\r\n                   together top-level domain expert representatives from the\r\n                   High Energy Physics and Radio Astronomy domains and top-tier\r\n                   High Performance Computing infrastructure representatives\r\n                   across Europe and the US. Feedback from ISC community will be\r\n                   fed into the technical blueprint of the capabilities of the\r\n                   future infrastructure together with its roadmap for research,\r\n                   innovation and deployment of the future infrastructure\"\r\n}<\/pre>\n
<\/a>@misc{engelmann25two,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Two Worlds Collide: Trustworthiness and Sustainability for\r\n                   Coupled HPC and AI Simulation\",\r\n  month         = jun # \"~12, \",\r\n  year          = \"2025\",\r\n  howpublished  = \"{Invited panelist at a Birds of a Feather session at the\r\n                   \\href{https:\/\/www.isc-hpc.com}{$40^{th}$ ISC High Performance\r\n                   Conference (ISC) 2025}, Hamburg, Germany}\",\r\n  url           = \"https:\/\/app.swapcard.com\/event\/isc-high-performance-2025\/planning\/UGxhbm5pbmdfMjU4NjE0MA==\",\r\n  abstract      = \"The ``Two Worlds Collide Birds'' of a Feather (BoF) series\r\n                   focuses on the experiences, challenges, and opportunities\r\n                   faced by laboratories and vendors in integrating deep\r\n                   learning (DL) and artificial intelligence (AI) with\r\n                   high-performance computing (HPC) for advanced simulation\r\n                   research. This fourth installment, titled ``Trustworthiness\r\n                   and Sustainability for Converged HPC and AI Simulation'' aims\r\n                   to promote a trustworthy and assured integration between\r\n                   established HPC simulation and the rapidly evolving DL\r\n                   ecosystem. Furthermore, this BoF seeks to address the\r\n                   emerging sustainability concerns associated with the\r\n                   verification and validation of converged HPC and AI\r\n                   simulations.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann25federated,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"The Federated Computing Environment for Autonomous Smart Laboratories\",\r\n  month         = mar # \"~20, \",\r\n  year          = \"2025\",\r\n  howpublished  = \"{Invited talk at the \\href{https:\/\/sos27.cscs.ch}{$27^{th}$\r\n                   Workshop on Distributed Supercomputing (SOS) 2025},\r\n                   Engelberg, Switzerland}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann25federated.ppt.pdf\",\r\n  abstract      = \"The open Interconnected Science Ecosystem (INTERSECT)\r\n                   architecture connects scientific instruments and\r\n                   robot-controlled laboratories with computing and data\r\n                   resources at the edge, the Cloud or the high-performance\r\n                   computing center to enable autonomous experiments,\r\n                   self-driving laboratories, smart manufacturing, and\r\n                   artificial intelligence driven design, discovery and\r\n                   evaluation. Its a novel approach consists of science use case\r\n                   design patterns, a system of systems architecture, and a\r\n                   microservice architecture. Failure resilience in federated\r\n                   ecosystems for instrument science is a critical challenge.\r\n                   Failures disrupt experiments and make them potentially\r\n                   useless, wasting valuable instrument, network and computing\r\n                   allocations and creating setbacks for scientists. A diverse,\r\n                   yet resilient, federated high-performance computing ecosystem\r\n                   is needed with traditional and accelerated capacity and\r\n                   capability computing resources and proper network and data\r\n                   storage resources, in part with on-demand and real-time\r\n                   features. This talk presents an overview of the resilient\r\n                   INTERSECT architecture, illustrates a resilient autonomous\r\n                   additive manufacturing use case, and discusses the future\r\n                   needs for incorporating such computational workloads into\r\n                   high-performance computing systems and facilities.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann23interconnected4,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"The Interconnected Science Ecosystem (INTERSECT)\",\r\n  month         = oct # \"~4, \",\r\n  year          = \"2023\",\r\n  howpublished  = \"{Invited talk at the \\href{https:\/\/www.hartree.stfc.ac.uk}\r\n                   {Hartree Centre, Science and Technology Facilities Council,\r\n                   Daresbury, UK}}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann23interconnected4.ppt.pdf\",\r\n  abstract      = \"The Interconnected Science Ecosystem (INTERSECT) Initiative\r\n                   at Oak Ridge National Laboratory is in the process of\r\n                   creating an open federated hardware\/software architecture for\r\n                   the laboratory of the future, connecting scientific\r\n                   instruments, robot-controlled laboratories, and edge\/center\r\n                   computing\/data resources to enable autonomous experiments,\r\n                   self-driving laboratories, smart manufacturing, and\r\n                   artificial intelligence driven design, discovery, and\r\n                   evaluation. Its novel approach describes science use cases as\r\n                   design patterns that identify and abstract the involved\r\n                   hardware\/software components and their interactions in terms\r\n                   of control, work, and data flow. It creates a\r\n                   system-of-systems architecture of the federated\r\n                   hardware\/software ecosystem that clarifies terms,\r\n                   architectural elements, the interactions between them and\r\n                   compliance. It further designs a federated microservice\r\n                   architecture, mapping science use case design patterns to the\r\n                   system-of-systems architecture with loosely coupled\r\n                   microservices and standardized interfaces. The INTERSECT Open\r\n                   Architecture Specification contains a use case design pattern\r\n                   catalog, a federated system-of-systems architecture\r\n                   specification, and a federated microservice architecture\r\n                   specification. It is currently being used to prototype and\r\n                   deploy autonomous experiments and self-driving laboratories at\r\n                   Oak Ridge National Laboratory in the following science areas:\r\n                   (1) automation for electric grid interconnected-laboratory\r\n                   emulation\/simulation, (2) autonomous additive manufacturing,\r\n                   (3) autonomous continuous flow reactor synthesis, (4)\r\n                   autonomous electron microscopy, (5) autonomous\r\n                   robotic-controlled chemistry laboratory, and (6) integrating\r\n                   an ion trap quantum computing resource.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann23interconnected3,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"The Interconnected Science Ecosystem (INTERSECT)\r\n                   Architecture\",\r\n  month         = aug # \"~21-23, \",\r\n  year          = \"2023\",\r\n  howpublished  = \"{Invited talk at the \\href{https:\/\/smc2023.ornl.gov}\r\n                   {$20^{th}$ Smoky Mountains Computational Sciences &\r\n                   Engineering Conference (SMC)}, Knoxville, TN, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann23interconnected3.ppt.pdf\",\r\n  abstract      = \"The Interconnected Science Ecosystem (INTERSECT) Initiative\r\n                   at Oak Ridge National Laboratory is in the process of\r\n                   creating an open federated hardware\/software architecture for\r\n                   the laboratory of the future, connecting scientific\r\n                   instruments, robot-controlled laboratories, and edge\/center\r\n                   computing\/data resources to enable autonomous experiments,\r\n                   self-driving laboratories, smart manufacturing, and artificial\r\n                   intelligence driven design, discovery, and evaluation. Its\r\n                   novel approach describes science use cases as design patterns\r\n                   that identify and abstract the involved hardware\/software\r\n                   components and their interactions in terms of control, work,\r\n                   and data flow. It creates a system-of-systems architecture of\r\n                   the federated hardware\/software ecosystem that clarifies\r\n                   terms, architectural elements, the interactions between them\r\n                   and compliance. It further designs a federated microservice\r\n                   architecture, mapping science use case design patterns to the\r\n                   system-of-systems architecture with loosely coupled\r\n                   microservices and standardized interfaces. The INTERSECT Open\r\n                   Architecture Specification contains a use case design pattern\r\n                   catalog, a federated system-of-systems architecture\r\n                   specification, and a federated microservice architecture\r\n                   specification. It is currently being used to prototype and\r\n                   deploy autonomous experiments and self-driving laboratories\r\n                   at Oak Ridge National Laboratory in the following science\r\n                   areas: (1) automation for electric grid\r\n                   interconnected-laboratory emulation\/simulation, (2) autonomous\r\n                   additive manufacturing, (3) autonomous continuous flow reactor\r\n                   synthesis, (4) autonomous electron microscopy, (5) autonomous\r\n                   robotic-controlled chemistry laboratory, and (6) integrating\r\n                   an ion trap quantum computing resource.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann23interconnected2,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"The Interconnected Science Ecosystem (INTERSECT)\r\n                   Architecture\",\r\n  month         = jul # \"~10, \",\r\n  year          = \"2023\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.lrz-muenchen.de}{Leibniz\r\n                   Rechenzentrum (LRZ)}, Garching, Germany}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann23interconnected2.ppt.pdf\",\r\n  abstract      = \"The Interconnected Science Ecosystem (INTERSECT) Initiative\r\n                   at Oak Ridge National Laboratory is in the process of\r\n                   creating an open federated hardware\/software architecture for\r\n                   the laboratory of the future, connecting scientific\r\n                   instruments, robot-controlled laboratories, and edge\/center\r\n                   computing\/data resources to enable autonomous experiments,\r\n                   self-driving laboratories, smart manufacturing, and artificial\r\n                   intelligence driven design, discovery, and evaluation. Its\r\n                   novel approach describes science use cases as design patterns\r\n                   that identify and abstract the involved hardware\/software\r\n                   components and their interactions in terms of control, work,\r\n                   and data flow. It creates a system-of-systems architecture of\r\n                   the federated hardware\/software ecosystem that clarifies\r\n                   terms, architectural elements, the interactions between them\r\n                   and compliance. It further designs a federated microservice\r\n                   architecture, mapping science use case design patterns to the\r\n                   system-of-systems architecture with loosely coupled\r\n                   microservices and standardized interfaces. The INTERSECT Open\r\n                   Architecture Specification contains a use case design pattern\r\n                   catalog, a federated system-of-systems architecture\r\n                   specification, and a federated microservice architecture\r\n                   specification. It is currently being used to prototype and\r\n                   deploy autonomous experiments and self-driving laboratories\r\n                   at Oak Ridge National Laboratory in the following science\r\n                   areas: (1) automation for electric grid\r\n                   interconnected-laboratory emulation\/simulation, (2) autonomous\r\n                   additive manufacturing, (3) autonomous continuous flow reactor\r\n                   synthesis, (4) autonomous electron microscopy, (5) autonomous\r\n                   robotic-controlled chemistry laboratory, and (6) integrating\r\n                   an ion trap quantum computing resource.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann23interconnected,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"The Interconnected Science Ecosystem (INTERSECT)\r\n                   Architecture\",\r\n  month         = may # \"~25, \",\r\n  year          = \"2023\",\r\n  howpublished  = \"{Invited talk at the \\href{https:\/\/esailworkshop.ornl.gov}\r\n                    {$1^{st}$ Ecosystems for Smart Autonomous Interconnected\r\n                    Labs (E-SAIL) Workshop}, held in conjunction with the\r\n                    \\href{https:\/\/www.isc-hpc.com}{$38^{th}$ ISC High\r\n                    Performance (ISC) 2023}, Hamburg, Germany}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann23interconnected.ppt.pdf\",\r\n  abstract      = \"The open Interconnected Science Ecosystem (INTERSECT)\r\n                   architecture connects scientific instruments and\r\n                   robot-controlled laboratories with computing and data\r\n                   resources at the edge, the Cloud or the high-performance\r\n                   computing center to enable autonomous experiments,\r\n                   self-driving laboratories, smart manufacturing, and\r\n                   artificial intelligence driven design, discovery and\r\n                   evaluation. Its a novel approach consists of science use\r\n                   case design patterns, a system of systems architecture, and\r\n                   a microservice architecture.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann22designing,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Designing Smart and Resilient Extreme-Scale Systems\",\r\n  month         = feb # \"~23-26, \",\r\n  year          = \"2022\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{https:\/\/www.siam.org\/conferences\/cm\/conference\/pp22}\r\n                   {$20^{th}$ SIAM Conference on Parallel Processing for\r\n                   Scientific Computing (PP) 2022}, Seattle, WA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann22designing.ppt.pdf\",\r\n  abstract      = \"Resilience is one of the critical challenges of extreme-scale\r\n                   high-performance computing (HPC) systems, as component counts\r\n                   increase, individual component reliability decreases, and\r\n                   software complexity increases. Building a reliable\r\n                   supercomputer that achieves the expected performance within a\r\n                   given cost budget and providing efficiency and correctness\r\n                   during operation in the presence of faults, errors, and\r\n                   failures requires a full understanding of the resilience\r\n                   problem. This talk provides an overview of recent achievements\r\n                   in developing a taxonomy, catalog and models that capture the\r\n                   observed and inferred fault, error, and failure conditions in\r\n                   current supercomputers and in extrapolating this knowledge to\r\n                   future-generation systems. It also describes the path forward\r\n                   in machine-in-the-loop operational intelligence for smart\r\n                   computing systems, leveraging operational data analytics in a\r\n                   loop control that maximizes productivity and minimizes costs\r\n                   through adaptive autonomous operation for resilience.\"\r\n}<\/pre>\n
<\/a>@misc{mintz21enabling,\r\n  author        = \"Ben Mintz\r\n                   and Christian Engelmann\r\n                   and Elke Arenholz\r\n                   and Ryan Coffee\",\r\n  title         = \"Enabling Self-Driven Experiments for Science through an\r\n                   Interconnected Science Ecosystem (INTERSECT)\",\r\n  month         = oct # \"~20, \",\r\n  year          = \"2021\",\r\n  howpublished  = \"{Panel at the \\href{https:\/\/smc2021.ornl.gov}{$17^{th}$ Smoky\r\n                    Mountains Computational Sciences & Engineering Conference\r\n                    (SMC)}}\",\r\n  abstract      = \"The process of operating scientific instruments, conducting\r\n                   experiments, and executing scientific workflows in general is\r\n                   time-consuming and labor-intensive. Computer control of\r\n                   instruments and the rapid rise in simulation and modeling has\r\n                   led to a significant increase in both the quantity and\r\n                   quality of data, but scientists are still contributing to\r\n                   many low-level process steps in data acquisition, processing,\r\n                   and interpretation to produce scientific results. These\r\n                   issues led to the integration of automation and autonomy to\r\n                   decreased process bottlenecks and increased efficiencies.\r\n                   While automation incorporates tools that perform\r\n                   well-defined, systematic processes with limited human\r\n                   intervention, autonomy introduces smart decision-making\r\n                   techniques, such as artificial intelligence (AI) and machine\r\n                   learning (ML). Combining these advances to automate entire\r\n                   scientific workflows and controlling them with AI\/ML will\r\n                   bring about revolutionary efficiencies and research outcomes.\r\n                   This kind of autonomous control of processes, experiments,\r\n                   and laboratories will fundamentally change the way scientists\r\n                   work, allowing us to explore high-dimensional problems\r\n                   previously considered impossible and discover new subtle\r\n                   correlations.\r\n\r\n                   To enable the interoperability of existing and future\r\n                   self-driven experiments, the scientific community needs a\r\n                   common Interconnected Science Ecosystem (INTERSECT) that\r\n                   consistently incorporates data management software, data\r\n                   analysis workflow tools, and experiment management\/steering\r\n                   software as well as AI\/ML capabilities. The development of\r\n                   INTERSECT requires tight collaboration between computer\r\n                   scientists, software engineers, data scientists, and domain\r\n                   scientists. This panel will introduce INTERSECT and discuss\r\n                   opportunities, challenges, and business goals for this type\r\n                   of ecosystem including scalability, interoperability, and\r\n                   solution\/software transferability\/reusability.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann21faults,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Faults, Errors and Failures in Extreme-Scale Supercomputers\",\r\n  month         = aug # \"~30, \",\r\n  year          = \"2021\",\r\n  howpublished  = \"{Keynote talk at the\r\n                    \\href{http:\/\/www.csm.ornl.gov\/srt\/conferences\/Resilience\/2021}{$14^{th}$\r\n                    Workshop on Resiliency in High Performance Computing\r\n                    (Resilience) in Clusters, Clouds, and Grids}, held in\r\n                    conjunction with the \\href{http:\/\/europar2014.dcc.fc.up.pt}\r\n                    {$27^{th}$ European Conference on Parallel and Distributed\r\n                    Computing (Euro-Par) 2021}, Lisbon, Portugal}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann21faults.ppt.pdf\",\r\n  abstract      = \"Resilience is one of the critical challenges of extreme-scale\r\n                   high-performance computing systems, as component counts\r\n                   increase, individual component reliability decreases, and\r\n                   software complexity increases. Building a reliable\r\n                   supercomputer that achieves the expected performance within a\r\n                   given cost budget and providing efficiency and correctness\r\n                   during operation in the presence of faults, errors, and\r\n                   failures requires a full understanding of the resilience\r\n                   problem. This talk provides an overview of reliability\r\n                   experiences with some of the largest supercomputers in the\r\n                   world and recent achievements in developing a taxonomy,\r\n                   catalog and models that capture the observed and inferred\r\n                   fault, error, and failure conditions in these systems.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann21resilience,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"The Resilience Problem in Extreme Scale Computing:\r\n                   Experiences and the Path Forward\",\r\n  month         = mar # \"~1-5, \",\r\n  year          = \"2021\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{https:\/\/www.siam.org\/conferences\/cm\/conference\/cse21}\r\n                   {SIAM Conference on Computational Science and Engineering\r\n                   (CSE) 2021}, Fort Worth, TX, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann21resilience.ppt.pdf\",\r\n  abstract      = \"Resilience is one of the critical challenges of extreme-scale\r\n                   high-performance computing systems, as component counts\r\n                   increase, individual component reliability decreases, and\r\n                   software complexity increases. Building a reliable\r\n                   supercomputer that achieves the expected performance within a\r\n                   given cost budget and providing efficiency and correctness\r\n                   during operation in the presence of faults, errors, and\r\n                   failures requires a full understanding of the resilience\r\n                   problem. This talk provides an overview of reliability\r\n                   experiences with some of the largest supercomputers in the\r\n                   world and recent achievements in developing a taxonomy,\r\n                   catalog and models that capture the observed and inferred\r\n                   fault, error, and failure conditions in these systems.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann21smart,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Smart and Resilient Extreme-Scale Systems\",\r\n  month         = jan # \"~19, \",\r\n  year          = \"2021\",\r\n  howpublished  = \"{Invited talk at the\r\n                    \\href{https:\/\/www.hipeac.net\/2021\/spring-virtual\/#\/program\/sessions\/7854\/}\r\n                    {Workshop on Resilience in High Performance Computing\r\n                    (RESILIENTHPC)}, held in conjunction with the\r\n                    \\href{https:\/\/www.hipeac.net\/2021}\r\n                    {European Network on High-performance Embedded Architecture\r\n                     and Compilation (HiPEAC) Conference 2021}, Budapest, Hungary}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann21smart.ppt.pdf\",\r\n  abstract      = \"Resilience is one of the critical challenges of extreme-scale\r\n                   high-performance computing (HPC) systems, as component counts\r\n                   increase, individual component reliability decreases, and\r\n                   software complexity increases. Building a reliable\r\n                   supercomputer that achieves the expected performance within a\r\n                   given cost budget and providing efficiency and correctness\r\n                   during operation in the presence of faults, errors, and\r\n                   failures requires a full understanding of the resilience\r\n                   problem. This talk provides an overview of recent\r\n                   achievements in developing a taxonomy, catalog and models\r\n                   that capture the observed and inferred fault, error, and\r\n                   failure conditions in current supercomputers and in\r\n                   extrapolating this knowledge to future-generation systems.\r\n                   It also describes the path forward in machine-in-the-loop\r\n                   operational intelligence for smart computing systems,\r\n                   leveraging operational data analytics in a loop control that\r\n                   maximizes productivity and minimizes costs through adaptive\r\n                   autonomous operation for resilience.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann20resilience,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"The Resilience Problem in Extreme Scale Computing\",\r\n  month         = feb # \"~12-15, \",\r\n  year          = \"2020\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{https:\/\/www.siam.org\/conferences\/cm\/conference\/pp20}\r\n                   {$19^{th}$ SIAM Conference on Parallel Processing for\r\n                   Scientific Computing (PP) 2020}, Seattle, WA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann20resilience.ppt.pdf\",\r\n  abstract      = \"Resilience is one of the critical challenges of extreme-scale\r\n                   high-performance computing (HPC) systems, as component counts\r\n                   increase, individual component reliability decreases, and\r\n                   software complexity increases. Building a reliable\r\n                   supercomputer that achieves the expected performance within\r\n                   a given cost budget and providing efficiency and correctness\r\n                   during operation in the presence of faults, errors, and\r\n                   failures requires a full understanding of the resilience\r\n                   problem. This talk provides an overview of recent\r\n                   achievements in developing a taxonomy, catalog and models\r\n                   that capture the observed and inferred fault, error, and\r\n                   failure conditions in current supercomputers and in\r\n                   extrapolating this knowledge to future-generation systems.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann19resilience3,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Resilience in Parallel Programming Environments\",\r\n  month         = oct # \"~30-31, \",\r\n  year          = \"2019\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{https:\/\/iadac.github.io\/events\/adac8}{$8^{th}$\r\n                   Accelerated Data Analytics and Computing (ADAC) Institute\r\n                   Workshop}, Tokyo, Japan}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann19resilience.ppt.pdf\",\r\n  abstract      = \"Recent reliability issues with one of the fastest\r\n                   supercomputers in the world, Titan at Oak Ridge National\r\n                   Laboratory, demonstrated the need for resilience in\r\n                   large-scale heterogeneous computing. OpenMP currently does\r\n                   not address error and failure behavior. The presented work\r\n                   takes a first step toward resilience for heterogeneous\r\n                   systems by providing the concepts for resilient OpenMP\r\n                   offload to devices. Using real-world error and failure\r\n                   observations, this work describes the concepts and\r\n                   terminology for resilient OpenMP target offload, including\r\n                   error and failure classes and resilience strategies. It\r\n                   details the experienced general-purpose computing on\r\n                   graphics processing units errors and failures in Titan. It\r\n                   further proposes improvements in OpenMP, including a\r\n                   preliminary prototype design, to support resilient offload\r\n                   to devices for efficient handling of errors and failures in\r\n                   heterogeneous high-performance computing systems.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann19resilience2,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Resilience by Design (and not as an Afterthought)\",\r\n  month         = mar # \"~26-29, \",\r\n  year          = \"2018\",\r\n  howpublished  = \"{Invited talk at the \\href{https:\/\/sos23.ornl.gov\/}{$23^{rd}$\r\n                   Workshop on Distributed Supercomputing (SOS) 2019}, Asheville,\r\n                   NC, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann19resilience2.ppt.pdf\",\r\n  abstract      = \"Resilience, i.e., obtaining a correct solution in a timely\r\n                   and efficient manner, is one of the key challenges in\r\n                   extreme-scale high-performance computing (HPC). The challenge\r\n                   is to build a reliable HPC system within a given cost budget\r\n                   that achieves the expected performance. Every generation of\r\n                   supercomputers deployed at Oak Ridge National Laboratory\r\n                   (ORNL) had to deal with expected and unexpected faults,\r\n                   errors and failures. While these supercomputers are designed\r\n                   to deal with expected issues, unexpected reliability problems\r\n                   can lead to severe degradation in operational capabilities.\r\n                   For example, ORNL's Titan supercomputer experienced an\r\n                   unexpected increase in general-purpose graphics processing\r\n                   unit (GPGPU) failures between 2015 and 2017. At the peak of\r\n                   the problem, Titan was losing an average of 12 GPGPUs (and\r\n                   corresponding compute nodes) per day. Over 50\\% of its 18,688\r\n                   GPGPUs had to be replaced. The system and the applications\r\n                   using it were never designed to handle such a high failure\r\n                   rate in an efficient manner. Other past unexpected\r\n                   reliability issues with supercomputers at US Department of\r\n                   Energy HPC centers were caused by early wear-out, dirty\r\n                   power, bad solder, other manufacturing issues, design errors\r\n                   in hardware, design errors in software and user errors. With\r\n                   the expected decrease in reliability due to component count\r\n                   increases, process technology challenges, hardware\r\n                   heterogeneity and software complexity, risk mitigation\r\n                   against unexpected issues is becoming paramount to ensure\r\n                   the success of future extreme-scale HPC systems. Resilience\r\n                   needs to be holistically provided by the HPC\r\n                   hardware\/software ecosystem. The key challenges are to\r\n                   design and to operate extreme HPC systems with (1)\r\n                   wide-ranging resilience capabilities in hardware, system\r\n                   software, programming models, libraries, and applications,\r\n                   (2) interfaces and mechanisms for coordinating resilience\r\n                   capabilities across diverse hardware and software\r\n                   components, (3) appropriate metrics and tools for assessing\r\n                   performance, resilience, and energy, and (4) an\r\n                   understanding of the performance, resilience and energy\r\n                   trade-off that eventually results in well-informed HPC\r\n                   system design choices and runtime decisions.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann19resilience,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Resilience for Extreme Scale Systems: Understanding the\r\n                   Problem\",\r\n  month         = feb # \"~25 - \" # mar # \"~1, \",\r\n  year          = \"2018\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{https:\/\/www.siam.org\/meetings\/cse19\/}{SIAM Conference\r\n                   on Computational Science and Engineering (CSE) 2019},\r\n                   Spokane, WA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann19resilience.ppt.pdf\",\r\n  abstract      = \"Resilience is one of the critical challenges of extreme-scale\r\n                   high-performance computing (HPC) systems, as component counts\r\n                   increase, individual component reliability decreases, and\r\n                   software complexity increases. Building a reliable\r\n                   supercomputer that achieves the expected performance within a\r\n                   given cost budget and providing efficiency and correctness\r\n                   during operation in the presence of faults, errors, and\r\n                   failures requires a full understanding of the resilience\r\n                   problem. This talk provides an overview of the Catalog\r\n                   project, which develops a taxonomy, catalog and models that\r\n                   capture the observed and inferred fault, error, and failure\r\n                   conditions in current supercomputers and extrapolates this\r\n                   knowledge to future-generation systems. To date, this\r\n                   project has analyzed billions of node hours of system logs\r\n                   from supercomputers at Oak Ridge National Laboratory and\r\n                   Argonne National Laboratory.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann18modeling,\r\n  author        = \"Christian Engelmann and Rizwan Ashraf\",\r\n  title         = \"Modeling and Simulation of Extreme-Scale Systems for\r\n                   Resilience by Design\",\r\n  month         = aug # \"~15-17, \",\r\n  year          = \"2018\",\r\n  howpublished  = \"{Invited talk at the \\href{https:\/\/www.bnl.gov\/modsim2018}\r\n                   {Workshop on Modeling and Simulation of Systems and\r\n                   Applications}, Seattle, WA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann18modeling.ppt.pdf\",\r\n  abstract      = \"Resilience is a serious concern for extreme-scale\r\n                   high-performance computing (HPC). While the HPC community has\r\n                   developed various resilience solutions, the solution space\r\n                   remains fragmented. We created a structured approach to the\r\n                   design, evaluation and optimization of HPC resilience using\r\n                   the concept of design patterns. A design pattern describes a\r\n                   generalized solution to a repeatedly occurring problem. We\r\n                   identified the commonly occurring problems and solutions used\r\n                   to deal with faults, errors and failures in HPC systems. Each\r\n                   well-known solution that addresses a specific resilience\r\n                   challenge is described in the form of a design pattern. We\r\n                   developed a resilience design pattern specification, language\r\n                   and catalog, which can be used by system architects, system\r\n                   software and library developers, application programmers, as\r\n                   well as users and operators as essential building blocks when\r\n                   designing and deploying resilience solutions.\r\n                   The resilience design pattern approach provides a unique\r\n                   opportunity for design space exploration. As each resilience\r\n                   solution is abstracted as a pattern and each solution's\r\n                   properties are defined by pattern parameters, vertical and\r\n                   horizontal pattern compositions can describe the resilience\r\n                   capabilities of an entire HPC system. This permits the\r\n                   investigation of beneficial or counterproductive interactions\r\n                   between patterns and of the performance, resilience, and\r\n                   power consumption trade-off between different pattern\r\n                   parameters and compositions. The ultimate goal is to make\r\n                   resilience an integral part of the HPC hardware\/software\r\n                   ecosystem by coordinating the various existing resilience\r\n                   solutions in a design space exploration process, such that\r\n                   the burden for providing resilience is on the system by\r\n                   design and not on the user as an afterthought.\r\n                   We are in the early stages of developing a novel design space\r\n                   exploration tool that enables this investigation using\r\n                   modeling and simulation. We developed performance and\r\n                   resilience models for each resilience design pattern. We also\r\n                   leverage results from the Catalog project, a collaborative\r\n                   effort between Oak Ridge National Laboratory, Argonne\r\n                   National Laboratory and Lawrence Livermore National\r\n                   Laboratory that developed models of the faults, errors and\r\n                   failures in today's HPC systems. We also leverage recent\r\n                   results from the same project by Lawrence Livermore National\r\n                   Laboratory in application reliability patterns. The planned\r\n                   research extends and combines this work to model the\r\n                   performance, resilience, and power consumption of an entire\r\n                   HPC system, initially at node-level granularity, and to\r\n                   simulate the dynamic interactions between deployed\r\n                   resilience solutions and the rest of the system. In the next\r\n                   iteration, finer-grain modeling and simulation, such as at\r\n                   the computational unit level, is used to increase accuracy.\r\n                   This work leverages the experience of the investigators in\r\n                   parallel discrete event simulation of extreme-scale systems,\r\n                   such as the Extreme-scale Simulator (xSim).\r\n                   The current state of the art in resilience modeling and\r\n                   simulation is fragmented as well. There is currently no such\r\n                   design space exploration tool. Instead, each resilience\r\n                   solution is typically investigated separately. There is only\r\n                   a small amount of work on multi-resilience solutions,\r\n                   including by the investigators. While there is work in\r\n                   investigating the performance\/resilience trade-off space,\r\n                   there is almost no work in including power consumption.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann18characterizing2,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Characterizing Faults, Errors, and Failures in Extreme-Scale\r\n                   Systems\",\r\n  month         = jul # \"~2-4, \",\r\n  year          = \"2018\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{https:\/\/pasc18.pasc-conference.org}{Platform for\r\n                   Advanced Scientific Computing (PASC) Conference 2018},\r\n                   Basel, Switzerland}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann18characterizing2.ppt.pdf\",\r\n  abstract      = \"Building a reliable supercomputer that achieves the expected\r\n                   performance within a given cost budget and providing\r\n                   efficiency and correctness during operation in the presence\r\n                   of faults, errors, and failures requires a full understanding\r\n                   of the resilience problem. The Catalog project develops a\r\n                   fault taxonomy, catalog and models that capture the observed\r\n                   and inferred conditions in current supercomputers and\r\n                   extrapolates this knowledge to future-generation systems. To\r\n                   date, the Catalog project has analyzed billions of node hours\r\n                   of system logs from supercomputers at Oak Ridge National\r\n                   Laboratory and Argonne National Laboratory. This talk\r\n                   provides an overview of our findings and lessons learned.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann18characterizing,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Characterizing Faults, Errors, and Failures in Extreme-Scale\r\n                   Systems\",\r\n  month         = jun # \"~20-21, \",\r\n  year          = \"2018\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{https:\/\/iadac.github.io\/adac6}{$6^{th}$\r\n                   Accelerated Data Analytics and Computing (ADAC) Institute\r\n                   Workshop}, Zurich, Switzerland}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann18characterizing.ppt.pdf\",\r\n  abstract      = \"Building a reliable supercomputer that achieves the expected\r\n                   performance within a given cost budget and providing\r\n                   efficiency and correctness during operation in the presence\r\n                   of faults, errors, and failures requires a full understanding\r\n                   of the resilience problem. The Catalog project develops a\r\n                   fault taxonomy, catalog and models that capture the observed\r\n                   and inferred conditions in current supercomputers and\r\n                   extrapolates this knowledge to future-generation systems. To\r\n                   date, the Catalog project has analyzed billions of node hours\r\n                   of system logs from supercomputers at Oak Ridge National\r\n                   Laboratory and Argonne National Laboratory. This talk\r\n                   provides an overview of our findings and lessons learned.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann18pattern-based,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Pattern-based Modeling of Fail-stop and Soft-error Resilience\r\n                   for Iterative Linear Solvers\",\r\n  month         = mar # \"~7-10, \",\r\n  year          = \"2018\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{https:\/\/www.siam.org\/meetings\/pp18\/}{$18^{th}$ SIAM\r\n                   Conference on Parallel Processing for Scientific Computing\r\n                   (PP) 2018}, Tokyo, Japan}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann18resilience.ppt.pdf\",\r\n  abstract      = \"Reliability is a serious concern for future extreme-scale\r\n                   high-performance computing (HPC). While the HPC community has\r\n                   developed various resilience solutions, the solution space\r\n                   remains fragmented. With this work, we develop a structured\r\n                   approach to the design, evaluation and optimization of HPC\r\n                   resilience using the concept of design patterns. We identify\r\n                   the problems caused by faults, errors and failures in HPC\r\n                   systems and the techniques used to deal with these events.\r\n                   Each well-known solution that addresses a specific resilience\r\n                   challenge is described in the form of a pattern. We develop a\r\n                   catalog of such resilience design patterns, which may be used\r\n                   by system architects, system software and tools developers,\r\n                   application programmers, as well as users and operators as\r\n                   essential building blocks when designing and deploying\r\n                   resilience solutions. We also develop a design framework that\r\n                   enhances a designer's understanding the opportunities for\r\n                   integrating multiple patterns across layers of the system\r\n                   stack and the important constraints during implementation of\r\n                   the individual patterns. It is also useful for designing\r\n                   mechanisms and interfaces to coordinate flexible fault\r\n                   management across hardware and software components. The\r\n                   resilience patterns and the design framework also enable\r\n                   exploration and evaluation of design alternatives and\r\n                   support optimization of the cost-benefit trade-offs among\r\n                   performance, protection coverage, and power consumption of\r\n                   resilience solutions.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann18resilience,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Resilience Design Patterns: A Structured Approach to\r\n                   Resilience at Extreme Scale\",\r\n  month         = mar # \"~7-10, \",\r\n  year          = \"2018\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{https:\/\/www.siam.org\/meetings\/pp18\/}{$18^{th}$ SIAM\r\n                   Conference on Parallel Processing for Scientific Computing\r\n                   (PP) 2018}, Tokyo, Japan}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/pattern-based.ppt.pdf\",\r\n  abstract      = \"The reliability of high-performance computing (HPC) platforms\r\n                   is among the most critical challenges as systems continue to\r\n                   increase component counts, while the individual component\r\n                   reliability decreases and software complexity increases.\r\n                   While most resilience solutions are designed to address a\r\n                   specific fault model, HPC applications must contend with\r\n                   extremely high rates of faults from various sources with\r\n                   different levels of severity. Therefore, resilience for\r\n                   extreme-scale HPC systems and their applications requires an\r\n                   integrated approach, which leverages detection, containment\r\n                   and mitigation capabilities from different layers of the HPC\r\n                   environment. With this work, we propose an approach based on\r\n                   design patterns to explore a multi-level resilience solution\r\n                   that addresses silent data corruptions and process failures.\r\n                   The structured approach enables evaluation of the key\r\n                   components of a multi-level resilience solution using pattern\r\n                   performance models and systematically integrating the\r\n                   patterns into a complete solution by assessing the interplay\r\n                   between the patterns. We describe the design steps to develop\r\n                   a multi-level resilience solution for an iterative linear\r\n                   solver application that combines algorithmic resilience\r\n                   features of the solver with the fault tolerance primitives\r\n                   provided by ULFM MPI. Our results demonstrate the viability\r\n                   of designing HPC applications capable of surviving\r\n                   simultaneous injection of hard and soft errors in a\r\n                   performance efficient manner.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann17catalog2,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"A Catalog of Faults, Errors, and Failures in Extreme-Scale\r\n                   Systems\",\r\n  month         = jul # \"~10-14, \",\r\n  year          = \"2017\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{http:\/\/www.siam.org\/meetings\/an17\/}{SIAM Annual\r\n                   Meeting (AM) 2017}, Pittsburgh, PA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann17catalog2.ppt.pdf\",\r\n  abstract      = \"Building a reliable supercomputer that achieves the expected\r\n                   performance within a given cost budget and providing\r\n                   efficiency and correctness during operation in the presence\r\n                   of faults, errors, and failures requires a full understanding\r\n                   of the resilience problem. The Catalog project develops a\r\n                   fault taxonomy, catalog and models that capture the observed\r\n                   and inferred conditions in current supercomputers and\r\n                   extrapolates this knowledge to future-generation systems. To\r\n                   date, the Catalog project has analyzed billions of node hours\r\n                   of system logs from supercomputers at Oak Ridge National\r\n                   Laboratory and Argonne National Laboratory. This talk\r\n                   provides an overview of our findings and lessons learned.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann17characterizing,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Characterizing Faults, Errors and Failures in Extreme-Scale\r\n                   Computing Systems\",\r\n  month         = jun # \"~16-22, \",\r\n  year          = \"2017\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{http:\/\/www.isc-hpc.com}\r\n                   {International Supercomputing Conference (ISC) 2017},\r\n                   Frankfurt am Main, Germany}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann17characterizing.ppt.pdf\",\r\n  abstract      = \"Building a reliable supercomputer that achieves the expected\r\n                   performance within a given cost budget and providing\r\n                   efficiency and correctness during operation in the presence\r\n                   of faults, errors, and failures requires a full understanding\r\n                   of the resilience problem. The Catalog project develops a\r\n                   fault taxonomy, catalog and models that capture the observed\r\n                   and inferred conditions in current supercomputers and\r\n                   extrapolates this knowledge to future-generation systems. To\r\n                   date, the Catalog project has analyzed billions of node hours\r\n                   of system logs from supercomputers at Oak Ridge National\r\n                   Laboratory and Argonne National Laboratory. This talk\r\n                   provides an overview of our findings and lessons learned.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann17catalog,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"A Catalog of Faults, Errors, and Failures in Extreme-Scale\r\n                   Systems\",\r\n  month         = may # \"~24-26, \",\r\n  year          = \"2017\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{http:\/\/icl.cs.utk.edu\/workshops\/scheduling2017\/}\r\n                   {$12^{th}$ Scheduling for Large Scale Systems Workshop\r\n                   (SLSSW) 2017}, Knoxville, TN, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann17catalog.ppt.pdf\",\r\n  abstract      = \"Building a reliable supercomputer that achieves the expected\r\n                   performance within a given cost budget and providing\r\n                   efficiency and correctness during operation in the presence\r\n                   of faults, errors, and failures requires a full understanding\r\n                   of the resilience problem. The Catalog project develops a\r\n                   fault taxonomy, catalog and models that capture the observed\r\n                   and inferred conditions in current supercomputers and\r\n                   extrapolates this knowledge to future-generation systems. To\r\n                   date, the Catalog project has analyzed billions of node hours\r\n                   of system logs from supercomputers at Oak Ridge National\r\n                   Laboratory and Argonne National Laboratory. This talk\r\n                   provides an overview of our findings and lessons learned.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann16missing,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"The Missing High-Performance Computing Fault Model\",\r\n  month         = apr # \"~12-15, \",\r\n  year          = \"2016\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{http:\/\/www.siam.org\/meetings\/pp16\/}{$17^{th}$ SIAM\r\n                   Conference on Parallel Processing for Scientific Computing\r\n                   (PP) 2016}, Paris, France}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann16missing.ppt.pdf\",\r\n  abstract      = \"The path to exascale computing poses several research\r\n                   challenges. Resilience is one of the most important\r\n                   challenges. This talk will present recent work in\r\n                   developing the missing high-performance computing (HPC)\r\n                   fault model. This effort identifies, categorizes and\r\n                   models the fault, error and failure properties of\r\n                   today's HPC systems. It develops a fault taxonomy,\r\n                   catalog and models that capture the observed and inferred\r\n                   conditions in current systems and extrapolates this\r\n                   knowledge to exascale HPC systems.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann16resilience2,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Resilience Challenges and Solutions for Extreme-Scale\r\n                   Supercomputing\",\r\n  month         = feb # \"~18, \",\r\n  year          = \"2016\",\r\n  howpublished  = \"{Invited talk at the \\href{http:\/\/www.usna.edu}{United\r\n                    States Naval Academy}, Annapolis, MD, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann16resilience2.ppt.pdf\",\r\n  abstract      = \"The path to exascale computing poses several research\r\n                   challenges related to power, performance, resilience,\r\n                   productivity, programmability, data movement, and data\r\n                   management. Resilience, i.e., providing efficiency and\r\n                   correctness in the presence of faults, is one of the most\r\n                   important exascale computer science challenges as systems\r\n                   scale up in component count (100,000-1,000,000 nodes with\r\n                   1,000-10,000 cores per node by 2022) and component\r\n                   reliability decreases (7 nm technology with near-threshold\r\n                   voltage operation by 2022). This talk provides an overview\r\n                   of recent and ongoing resilience research and development\r\n                   activities at Oak Ridge National Laboratory in advanced\r\n                   checkpoint storage architectures, process-level incremental\r\n                   checkpoint\/restart, proactive fault tolerance using\r\n                   prediction-triggered process or virtual machine migration,\r\n                   MPI process-level software redundancy, and soft-error\r\n                   injection tools to study the vulnerability of science\r\n                   applications.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann15toward,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Toward A Fault Model And Resilience Design Patterns For\r\n                   Extreme Scale Systems\",\r\n  month         = aug # \"~24-28, \",\r\n  year          = \"2015\",\r\n  howpublished  = \"{Keynote talk at the\r\n                    \\href{http:\/\/www.csm.ornl.gov\/srt\/conferences\/Resilience\/2015}{$8^{th}$\r\n                    Workshop on Resiliency in High Performance Computing\r\n                    (Resilience) in Clusters, Clouds, and Grids}, held in\r\n                    conjunction with the \\href{http:\/\/europar2014.dcc.fc.up.pt}\r\n                    {$21^{st}$ European Conference on Parallel and Distributed\r\n                    Computing (Euro-Par) 2015}, Vienna, Austria}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann15toward.ppt.pdf\",\r\n  abstract      = \"The path to exascale computing poses several research\r\n                   challenges related to power, performance, resilience,\r\n                   productivity, programmability, data movement, and data\r\n                   management. Resilience, i.e., providing efficiency and\r\n                   correctness in the presence of faults, is one of the most\r\n                   important exascale computer science challenges as systems\r\n                   scale up in component count (100,000-1,000,000 nodes with\r\n                   1,000-10,000 cores per node by 2022) and component\r\n                   reliability decreases (7 nm technology with near-threshold\r\n                   voltage operation by 2022). This talk provides an overview\r\n                   of two recently funded projects.\r\n                   The Characterizing Faults, Errors, and Failures in\r\n                   Extreme-Scale Systems project identifies, categorizes and\r\n                   models the fault, error and failure properties of US\r\n                   Department of Energy high-performance computing (HPC)\r\n                   systems. It develops a fault taxonomy, catalog and models\r\n                   that capture the observed and inferred conditions in current\r\n                   systems and extrapolate this knowledge to exascale HPC\r\n                   systems.\r\n                   The Resilience Design Patterns project will increase the\r\n                   ability of scientific applications to reach accurate\r\n                   solutions in a timely and efficient manner. Using a novel\r\n                   design pattern concept, it identifies and evaluates\r\n                   repeatedly occurring resilience problems and coordinates\r\n                   solutions throughout high-performance computing hardware\r\n                   and software.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann15resilience,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Resilience Challenges and Solutions for Extreme-Scale\r\n                   Supercomputing\",\r\n  month         = mar # \"~2-5, \",\r\n  year          = \"2015\",\r\n  howpublished  = \"{Invited talk at the\r\n                    $19^{th}$ Workshop on Distributed Supercomputing (SOS)\r\n                     2015, Park City, UT, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann15resilience.ppt.pdf\",\r\n  abstract      = \"The path to exascale computing poses several research\r\n                   challenges related to power, performance, resilience,\r\n                   productivity, programmability, data movement, and data\r\n                   management. Resilience, i.e., providing efficiency and\r\n                   correctness in the presence of faults, is one of the most\r\n                   important exascale computer science challenges as systems\r\n                   scale up in component count (100,000-1,000,000 nodes with\r\n                   1,000-10,000 cores per node by 2022) and component\r\n                   reliability decreases (7 nm technology with near-threshold\r\n                   voltage operation by 2022). This talk provides an overview\r\n                   of recent and ongoing resilience research and development\r\n                   activities at Oak Ridge National Laboratory in advanced\r\n                   checkpoint storage architectures, process-level incremental\r\n                   checkpoint\/restart, proactive fault tolerance using\r\n                   prediction-triggered process or virtual machine migration,\r\n                   MPI process-level software redundancy, and soft-error\r\n                   injection tools to study the vulnerability of science\r\n                   applications.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann15xsim,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"xSim: {T}he Extreme-scale Simulator\",\r\n  month         = feb # \"~23, \",\r\n  year          = \"2015\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.lrz-muenchen.de}{Leibniz\r\n                   Rechenzentrum (LRZ)}, Garching, Germany}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann15xsim.ppt.pdf\",\r\n  abstract      = \"The path to exascale high-performance computing (HPC) poses\r\n                   several challenges related to power, performance, and\r\n                   resilience. Investigating the performance and resilience of\r\n                   parallel applications at scale on future architectures and\r\n                   the performance and resilience impact of different\r\n                   architecture choices is an important component of HPC\r\n                   hardware\/software co-design. Without having access to future\r\n                   architectures at scale, simulation provides an alternative.\r\n                   The Extreme-scale Simulator (xSim) is a performance\r\n                   investigation toolkit that permits running applications in\r\n                   a controlled environment with millions of concurrent\r\n                   execution threads, while observing performance and\r\n                   resilience in a simulated extreme-scale system. Using a\r\n                   lightweight parallel discrete event simulation, xSim executes\r\n                   a Message Passing Interface (MPI) application on a much\r\n                   smaller system in a highly oversubscribed fashion with a\r\n                   virtual wall clock time, such that performance data can be\r\n                   extracted based on a processor and a network model. xSim is\r\n                   designed like a traditional performance tool, as an\r\n                   interposition library that sits between the MPI application\r\n                   and the MPI library, using the MPI profiling interface. It\r\n                   has been run up to 134,217,728 (2^27) MPI ranks using a\r\n                   960-core Linux cluster. xSim also permits the injection of\r\n                   MPI process failures, the propagation\/detection\/notification\r\n                   of such failures within the simulation, and their handling\r\n                   within the simulation using application-level\r\n                   checkpoint\/restart. Another feature provides user-level\r\n                   failure mitigation (ULFM) extensions at the simulated MPI\r\n                   layer to support algorithm-based fault tolerance (ABFT).\r\n                   xSim is the very first performance tool that supports ULFM\r\n                   and ABFT.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann14supporting,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Supporting the Development of Resilient Message Passing\r\n                   Applications using Simulation\",\r\n  month         = sep # \"~28 - \" # oct # \"~1, \",\r\n  year          = \"2014\",\r\n  howpublished  = \"Invited talk at the\r\n                   \\href{http:\/\/www.dagstuhl.de\/en\/program\/calendar\/semhp\/?semnr=14402}\r\n                   {Dagstuhl Seminar on Resilience in Exascale Computing},\r\n                   Schloss Dagstuhl, Wadern, Germany\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann14supporting.ppt.pdf\",\r\n  abstract      = \"An emerging aspect of high-performance computing (HPC)\r\n                   hardware\/software co-design is investigating performance\r\n                   under failure. The presented work extends the Extreme-scale\r\n                   Simulator (xSim), which was designed for evaluating the\r\n                   performance of message passing interface (MPI) applications\r\n                   on future HPC architectures, with fault-tolerant MPI\r\n                   extensions proposed by the MPI Fault Tolerance Working Group.\r\n                   xSim permits running MPI applications with millions of\r\n                   concurrent MPI ranks, while observing application performance\r\n                   in a simulated extreme-scale system using a lightweight\r\n                   parallel discrete event simulation. The newly added features\r\n                   offer user-level failure mitigation (ULFM) extensions at the\r\n                   simulated MPI layer to support algorithm-based fault tolerance\r\n                   (ABFT). The presented solution permits investigating\r\n                   performance under failure and failure handling of ABFT\r\n                   solutions. The newly enhanced xSim is the very first\r\n                   performance tool that supports ULFM and ABFT.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann13resilience,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Resilience Challenges and Solutions for Extreme-Scale\r\n                   Supercomputing\",\r\n  month         = sep # \"~3, \",\r\n  year          = \"2013\",\r\n  howpublished  = \"{Invited talk at the Technical University of Dresden,\r\n                    Dresden, Germany}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann13resilience.ppt.pdf\",\r\n  abstract      = \"With the recent deployment of the 18 PFlop\/s Titan\r\n                   supercomputer and the exascale roadmap targeting 100, 300,\r\n                   and eventually 1,000 PFlop\/s by 2022, Oak Ridge National\r\n                   Laboratory is at the forefront of scientific capability\r\n                   computing. The path to exascale computing poses several\r\n                   research challenges related to power, performance,\r\n                   resilience, productivity, programmability, data movement,\r\n                   and data management. Resilience, i.e., providing efficiency\r\n                   and correctness in the presence of faults, is one of the\r\n                   most important exascale computer science challenges as\r\n                   systems scale up in component count (100,000-1,000,000\r\n                   nodes with 1,000-10,000 cores per node by 2022) and\r\n                   component reliability decreases (7 nm technology with\r\n                   near-threshold voltage operation by 2022). This talk\r\n                   provides an overview of recent and ongoing resilience\r\n                   research and development activities at Oak Ridge National\r\n                   Laboratory in advanced checkpoint storage architectures,\r\n                   process-level incremental checkpoint\/restart, proactive\r\n                   fault tolerance using prediction-triggered process or\r\n                   virtual machine migration, MPI process-level software\r\n                   redundancy, and soft-error injection tools to study the\r\n                   vulnerability of science applications and of CMOS logic\r\n                   in processors and memory.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann12fault,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Fault Tolerance Session\",\r\n  month         = oct # \"~16-17, \",\r\n  year          = \"2012\",\r\n  howpublished  = \"{Invited talk at the\r\n                    \\href{http:\/\/www.aanmelder.nl\/exachallenge}\r\n                    {The ExaChallenge Symposium}, Dublin, Ireland}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann12fault.ppt.pdf\"\r\n}<\/pre>\n
<\/a>@misc{engelmann12high-end,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"High-End Computing Resilience: Analysis of Issues Facing the\r\n                   HEC Community and Path Forward for Research and Development\",\r\n  month         = aug # \"~4-11, \",\r\n  year          = \"2012\",\r\n  howpublished  = \"{Invited talk at the Argonne National Laboratory (ANL)\r\n                    Institute of Computing in Science (ICiS)\r\n                    \\href{http:\/\/www.icis.anl.gov\/programs\/summer2012-4b}\r\n                    {Summer Workshop Week on Addressing Failures in Exascale\r\n                     Computing}, Park City, UT, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann12high-end.ppt.pdf\",\r\n  abstract      = \"The path to exascale computing poses several research\r\n                   challenges related to power, performance, resilience,\r\n                   productivity, programmability, data movement, and data\r\n                   management. Resilience, i.e., providing efficiency and\r\n                   correctness in the presence of faults, is one of the most\r\n                   important exascale computer science challenges as systems\r\n                   scale up in component count (100,000-1,000,000 nodes with\r\n                   1,000-10,000 cores per node by 2020) and component\r\n                   reliability decreases (7 nm technology with near-threshold\r\n                   voltage operation by 2020). To provide input for a\r\n                   discussion of future needs in resilience research,\r\n                   development, and standards work, this talk gives a brief\r\n                   summary of the outcomes from the National HPC Workshop on\r\n                   Resilience, held in Arlington, VA, USA on August 12-14,\r\n                   2009.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann12resilience,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Resilience for Permanent, Transient, and Undetected Errors\",\r\n  month         = mar # \"~12-15, \",\r\n  year          = \"2012\",\r\n  howpublished  = \"{Invited talk at the\r\n                    \\href{http:\/\/www.cs.sandia.gov\/Conferences\/SOS16}\r\n                    {$16^{th}$ Workshop on Distributed Supercomputing (SOS)\r\n                     2012}, Santa Barbara, CA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann12resilience.ppt.pdf\",\r\n  abstract      = \"With the ongoing deployment of 10-20 PFlop\/s supercomputers\r\n                   and the exascale roadmap targeting 100, 300, and eventually\r\n                   1,000 PFlop\/s by 2020, the path to exascale computing poses\r\n                   several research challenges related to power, performance,\r\n                   resilience, productivity, programmability, data movement,\r\n                   and data management. Resilience, i.e., providing efficiency\r\n                   and correctness in the presence of faults, is one of the\r\n                   most important exascale computer science challenges as\r\n                   systems scale up in component count (100,000-1,000,000\r\n                   nodes with 1,000-10,000 cores per node by 2020) and\r\n                   component reliability decreases (7 nm technology with\r\n                   near-threshold voltage operation by 2020). This talk\r\n                   provides an overview of recent and ongoing resilience\r\n                   research and development activities at Oak Ridge National\r\n                   Laboratory, and of future needs in resilience research,\r\n                   development, and standards work.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann12scaling,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Scaling To A Million Cores And Beyond: A Basic Understanding\r\n                   Of The Challenges Ahead On The Road To Exascale\",\r\n  month         = jan # \"~24, \",\r\n  year          = \"2012\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{https:\/\/researcher.ibm.com\/researcher\/view_page.php?id=2580}\r\n                   {$1^{st}$ International Workshop on Extreme Scale Parallel\r\n                   Architectures and Systems (ESPAS) 2012}, in conjunction with\r\n                   the \\href{http:\/\/www.hipeac.net\/conference\/paris}{$7^{th}$\r\n                   International Conference on High-Performance and Embedded\r\n                   Architectures and Compilers (HiPEAC) 2012}, Paris France}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann12scaling.ppt.pdf\",\r\n  abstract      = \"On the road toward multi-petascale and exascale HPC, the\r\n                   trend in architecture goes clearly in only one direction.\r\n                   HPC systems will dramatically scale up in compute node and\r\n                   processor core counts. By 2020, an exascale system may have\r\n                   up to 1,000,000 compute nodes with 1,000 cores per node. The\r\n                   substantial growth in concurrency causes parallel application\r\n                   scalability issues due to sequential application parts,\r\n                   synchronizing communication, and other bottlenecks.\r\n                   Investigating parallel algorithm performance properties at\r\n                   this scale and with these architectural properties for HPC\r\n                   hardware\/software co-design is crucial to enable\r\n                   extreme-scale computing. The presented work utilizes the\r\n                   Extreme-scale Simulator (xSim) performance investigation\r\n                   toolkit to identify the scaling characteristics of a simple\r\n                   Monte Carlo algorithm from 1 to 16 million MPI processes on\r\n                   different multi-core architecture choices. The results show\r\n                   the limitations of strong scaling and the negative impact of\r\n                   employing more but less powerful cores for energy savings.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann11resilient,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Resilient Software for ExaScale Computing\",\r\n  month         = nov # \"~17, \",\r\n  year          = \"2011\",\r\n  howpublished  = \"{Invited talk at the Birds of a Feather Session on Resilient\r\n                   Software for ExaScale Computing at the\r\n                   \\href{http:\/\/sc11.supercomputing.org}\r\n                   {24th IEEE\/ACM International Conference on High Performance\r\n                    Computing, Networking, Storage and Analysis (SC) 2011},\r\n                   Seattle, WA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann11resilient.ppt.pdf\",\r\n  abstract      = \"ExaScale computing systems will likely consist of millions\r\n                   of cores executing applications with billions of threads,\r\n                   based on 14nm or less CMOS technology, according to the\r\n                   ITRS roadmap. Processing elements built on this technology,\r\n                   coupled with dynamic power management will exhibit high\r\n                   variability in performance, between cores and across\r\n                   different runs. Even worse, preliminary figures indicates\r\n                   that on average about every couple of minutes - at least -\r\n                   something in the system will break. Traditional\r\n                   checkpointing strategies are unlikely to work, given the\r\n                   time it will take to save the huge quantities of data\r\n                   combined with the fact that they will need to be restored\r\n                   frequently. This BoF wants to investigate resilient\r\n                   software: software that is able to survive failing\r\n                   hardware and continue to run, without minimal performance\r\n                   impact. Furthermore, we may also discuss tradeoffs between\r\n                   rerunning the application and the cost of instrumentation\r\n                   to deal with resilience.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann11resilience,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Resilience and Hardware\/Software Co-design for Extreme-Scale\r\n                   Supercomputing\",\r\n  month         = jul # \"~27, \",\r\n  year          = \"2011\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.bsc.es}{Barcelona\r\n                   Supercomputing Center}, Barcelona, Spain}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann11resilience.ppt.pdf\",\r\n  abstract      = \"Oak Ridge National Laboratory (ORNL) provides the most\r\n                   powerful high-performance computing (HPC) resources in the\r\n                   world for open scientific research. Jaguar, a 224,162-core\r\n                   Cray XT5 with a LINPACK performance of 1.759 PFlop\/s, for\r\n                   example, is the world's 3rd fastest supercomputer. 80\\% of\r\n                   its resources are allocated through a reviewed process to\r\n                   address the most challenging scientific problems in climate\r\n                   modeling, renewable energy, materials science, fusion and\r\n                   other areas. ORNL's Computer Science and Mathematics Division\r\n                   performs computer science and mathematics research to\r\n                   increase supercomputer efficiency and application scientist\r\n                   productivity while accelerating time to solution for\r\n                   scientific breakthroughs. This talk details recent research\r\n                   advancements at ORNL in two areas: (1) resilience and (2)\r\n                   hardware\/software co-design for extreme-scale supercomputing.\r\n                   Both are essential on the road toward exa-scale HPC systems\r\n                   with millions-to-billions of cores. Due to the expected\r\n                   drastic increase in scale, the corresponding decrease in\r\n                   system mean-time to interrupt warrants a rethinking of the\r\n                   traditional checkpoint\/restart approach for HPC resilience.\r\n                   New concepts discussed in this talk range from preventative\r\n                   measures, such as task migration based on fault prediction,\r\n                   to more aggressive fault masking, such as various levels of\r\n                   redundancy. Further, the expected drastic increase in task\r\n                   parallelism requires redesigning algorithms to avoid the\r\n                   consequences of Amdahl's law at extreme scale. As million-way\r\n                   task parallel systems don't exist yet, this talk discusses a\r\n                   lightweight system simulation approach for performance\r\n                   estimation of algorithms at scale.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann10scalable,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Scalable HPC System Monitoring\",\r\n  month         = oct # \"~13, \",\r\n  year          = \"2010\",\r\n  howpublished  = \"{Invited talk at the $3^{rd}$ HPC Resiliency Summit: Workshop\r\n                   on Resiliency for Petascale HPC 2010, in conjunction with the\r\n                   \\href{http:\/\/www.lanl.gov\/conferences\/lacss\/2010}{$3^{rd}$\r\n                   Los Alamos Computer Science Symposium (LACSS) 2010}, Santa\r\n                   Fe, NM, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann10scalable.ppt.pdf\",\r\n  abstract      = \"We present a monitoring system for large-scale parallel and\r\n                   distributed computing environments that allows to trade-off\r\n                   accuracy in a tunable fashion to gain scalability without\r\n                   compromising fidelity. The approach relies on classifying\r\n                   each gathered monitoring metric based on individual needs\r\n                   and on aggregating messages containing classes of individual\r\n                   monitoring metrics using a tree-based overlay network. The\r\n                   MRNet-based prototype is able to significantly reduce the\r\n                   amount of gathered and stored monitoring data, e.g., by a\r\n                   factor of ~56 in comparison to the Ganglia distributed\r\n                   monitoring system. A simple scaling study reveals, however,\r\n                   that further efforts are needed in reducing the amount of\r\n                   data to monitor future-generation extreme-scale systems with\r\n                   up to 1,000,000 nodes. The implemented solution did not had\r\n                   a measurable performance impact as the 32-node test system\r\n                   did not produce enough monitoring data to interfere with\r\n                   running applications.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann10beyond,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Beyond Application-Level Checkpoint\/Restart - {Advanced}\r\n                   Software Approaches for Fault Resilience\",\r\n  month         = sep # \"~6, \",\r\n  year          = \"2010\",\r\n  howpublished  = \"{Talk at the\r\n                   \\href{http:\/\/www.speedup.ch\/workshops\/w39_2010.html}\r\n                   {$39^{th}$ SPEEDUP Workshop on High Performance Computing},\r\n                   Zurich, Switzerland}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann10beyond.ppt.pdf\"\r\n}<\/pre>\n
<\/a>@misc{engelmann10reliability,\r\n  author        = \"Christian Engelmann and\r\n                   Stephen L. Scott\",\r\n  title         = \"Reliability, Availability, and Serviceability ({RAS}) for\r\n                   Petascale High-End Computing and Beyond\",\r\n  month         = jun # \"~22, \",\r\n  year          = \"2010\",\r\n  howpublished  = \"{Talk at the \\href{http:\/\/www.usenix.org\/events\/fastos10}\r\n                   {Forum to Address Scalable Technology for Runtime and\r\n                   Operating Systems (FAST-OS) Workshop}, in conjunction with\r\n                   the \\href{http:\/\/www.usenix.org\/events\/confweek10}{USENIX\r\n                   Federated Conferences Week (USENIX) 2010}, Boston MA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann10reliability.ppt.pdf\",\r\n  abstract      = \"This project aims at scalable technologies for providing\r\n                   high-level RAS for next-generation petascale scientific\r\n                   high-performance computing (HPC) resources and beyond as\r\n                   outlined by the U.S. Department of Energy (DOE) Forum to\r\n                   Address Scalable Technology for Runtime and Operating\r\n                   Systems (FAST-OS) and the U.S. National Coordination Office\r\n                   for Networking and Information Technology Research and\r\n                   Development (NCO\/NITRD) High-End Computing Revitalization\r\n                   Task Force (HECRTF) activities. Based on virtualized\r\n                   adaptation, reconfiguration, and preemptive measures, the\r\n                   ultimate goal is to provide for non-stop scientific computing\r\n                   on a 24x7 basis without interruption. The taken technical\r\n                   approach leverages system-level virtualization technology to\r\n                   enable transparent proactive and reactive fault tolerance\r\n                   mechanisms on extreme scale HPC systems. This effort targets:\r\n                   (1) reliability analysis for identifying pre-fault\r\n                   indicators, predicting failures, and modeling and monitoring\r\n                   component and system reliability, (2) proactive fault\r\n                   tolerance technology based on preemptive migration away from\r\n                   components that are about to fail, (3) reactive fault\r\n                   tolerance enhancements, such as checkpoint interval and\r\n                   placement adaptation to actual and predicted system health\r\n                   threats, and (4) holistic fault tolerance through combination\r\n                   of adaptive proactive and reactive fault tolerance.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann10resilience,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Resilience Challenges at the Exascale\",\r\n  month         = mar # \"~8-11, \",\r\n  year          = \"2010\",\r\n  howpublished  = \"{Talk at the\r\n                   \\href{http:\/\/www.csm.ornl.gov\/workshops\/SOS14}{$14^{th}$\r\n                   Workshop on Distributed Supercomputing (SOS) 2010}, Savannah,\r\n                   GA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann10resilience.ppt.pdf\",\r\n  abstract      = \"The path to exascale computing poses several research\r\n                   challenges related to power, performance, resilience,\r\n                   productivity, programmability, data movement, and data\r\n                   management. Resilience, i.e., providing efficiency and\r\n                   correctness in the presence of faults, is one of the most\r\n                   important exascale computer science challenges as systems\r\n                   scale up in component count and component reliability\r\n                   decreases. This talk discusses the future needs in\r\n                   resilience research, development, and standards work\r\n                   based on the outcomes from the National HPC Workshop on\r\n                   Resilience, held in Arlington, VA, USA on August 12-14,\r\n                   2009.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann10hpc,\r\n  author        = \"Christian Engelmann\r\n                   and Stephen L. Scott\",\r\n  title         = \"{HPC} System Software Research at {Oak Ridge National\r\n                   Laboratory}\",\r\n  month         = feb # \"~22, \",\r\n  year          = \"2010\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.lrz-muenchen.de}{Leibniz\r\n                   Rechenzentrum (LRZ)}, Garching, Germany}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann10hpc.ppt.pdf\",\r\n  abstract      = \"Oak Ridge National Laboratory (ORNL) is the largest energy\r\n                   laboratory in the United States. Its National Center for\r\n                   Computational Sciences (NCCS) provides the most powerful\r\n                   computing resources in the world for open scientific\r\n                   research. Jaguar, a Cray XT5 system at NCCS, is the fastest\r\n                   supercomputer in the world. It recently ranked #1 in the Top\r\n                   500 List of Supercomputer Sites with a maximal LINPACK\r\n                   benchmark performance of 1.759 PFlop\/s and a theoretical peak\r\n                   performance of 2.331 PFlop\/s, where 1 PFlop\/s is $10^{15}$\r\n                   Floating Point Operations Per Second. Annually, 80 percent of\r\n                   Jaguar's resources are allocated through the U.S Department\r\n                   of Energy's Innovative and Novel Computational Impact on\r\n                   Theory and Experiment (INCITE) program, a competitively\r\n                   selected, peer reviewed process open to researchers from\r\n                   universities, industry, government and non-profit\r\n                   organizations. These allocations address some of the most\r\n                   challenging scientific problems in areas such as climate\r\n                   modeling, renewable energy, materials science, fusion and\r\n                   combustion. In conjunction with NCCS, the Computer Science\r\n                   and Mathematics Division at ORNL performs basic and applied\r\n                   research in HPC, mathematics, and intelligent systems. This\r\n                   talk gives a summary of the HPC research and development in\r\n                   system software performed at ORNL, including resilience at\r\n                   extreme scale and virtualization technologies in HPC.\r\n                   Specifically, this talk will focus on advanced resilience\r\n                   technologies, such as migration of computation away from\r\n                   components that are about to fail and on management and\r\n                   customization of virtualized environments.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann09high2,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"High-Performance Computing Research Internship and Appointment\r\n                   Opportunities at {Oak Ridge National Laboratory}\",\r\n  month         = dec # \"~14, \",\r\n  year          = \"2009\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.cs.reading.ac.uk}{Department\r\n                   of Computer Science}, \\href{http:\/\/www.reading.ac.uk}\r\n                   {University of Reading}, Reading, United Kingdom}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann09high2.ppt.pdf\",\r\n  abstract      = \"Oak Ridge National Laboratory (ORNL) is the largest energy\r\n                   laboratory in the United States. Its National Center for\r\n                   Computational Sciences (NCCS) provides the most powerful\r\n                   computing resources in the world for open scientific\r\n                   research. Jaguar, a Cray XT5 system at NCCS, is the fastest\r\n                   supercomputer in the world. It recently ranked #1 in the Top\r\n                   500 List of Supercomputer Sites with a maximal LINPACK\r\n                   benchmark performance of 1.759 PFlop\/s and a theoretical peak\r\n                   performance of 2.331 PFlop\/s, where 1 PFlop\/s is $10^{15}$\r\n                   Floating Point Operations Per Second. Annually, 80 percent of\r\n                   Jaguar's resources are allocated through the U.S Department\r\n                   of Energy's Innovative and Novel Computational Impact on \r\n                   Theory and Experiment (INCITE) program, a competitively\r\n                   selected, peer reviewed process open to researchers from\r\n                   universities, industry, government and non-profit\r\n                   organizations. These allocations address some of the most\r\n                   challenging scientific problems in areas such as climate\r\n                   modeling, renewable energy, materials science, fusion and\r\n                   combustion. In conjunction with NCCS, the Computer Science\r\n                   and Mathematics Division at ORNL performs basic and applied\r\n                   research in HPC, mathematics, and intelligent systems. This\r\n                   talk gives a summary of the HPC research performed at ORNL.\r\n                   It provides details about the Jaguar peta-scale computing\r\n                   resource, an overview of the computational science research\r\n                   carried out using ORNL's computing resources, and a\r\n                   description of various computer science efforts targeting\r\n                   solutions for next-generation HPC systems. This talk also\r\n                   provides information about internship opportunities for MSc\r\n                   students and research appointment opportunities for recent\r\n                   graduates.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann09jcas,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"{JCAS} - {IAA} Simulation Efforts at {Oak Ridge National\r\n                   Laboratory}\",\r\n  month         = sep # \"~1-2, \",\r\n  year          = \"2009\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{http:\/\/www.cs.sandia.gov\/CSRI\/Workshops\/2009\/IAA}\r\n                   {IAA Workshop on HPC Architectural Simulation (HPCAS)},\r\n                   Boulder, CO, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann09jcas.ppt.pdf\"\r\n}<\/pre>\n
<\/a>@misc{engelmann09modeling,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Modeling Techniques Towards Resilience\",\r\n  month         = aug # \"~12-14, \",\r\n  year          = \"2009\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{http:\/\/institute.lanl.gov\/resilience\/conferences\/2009}\r\n                   {National HPC Workshop on Resilience 2009}, Arlington, VA,\r\n                   USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann09modeling.ppt.pdf\"\r\n}<\/pre>\n
<\/a>@misc{engelmann09system,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"System Resilience Research at {ORNL} in the Context of\r\n                   {HPC}\",\r\n  month         = may # \"~15, \",\r\n  year          = \"2009\",\r\n  howpublished  = \"{Invited talk at the \\href{http:\/\/www.inria.fr\/inria\/organigramme\/fiche_ur-ren.fr.html}\r\n                   {Institut National de Recherche en Informatique et en\r\n                   Automatique (INRIA)}, Rennes, France}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann09system.pdf\",\r\n  abstract      = \"The continuing growth in high performance computing (HPC)\r\n                   system scale poses a challenge for system software and\r\n                   scientific applications with respect to reliability,\r\n                   availability and serviceability (RAS). With only very few\r\n                   exceptions, the availability of recently installed systems\r\n                   has been lower in comparison to the same deployment phase of\r\n                   their predecessors. As a result, sites lower allowable job\r\n                   run times in order to force applications to store\r\n                   intermediate results (checkpoints) as insurance against lost\r\n                   computation time. However, checkpoints themselves waste\r\n                   valuable computation time and resources. In contrast to the\r\n                   experienced loss of availability, the demand for continuous\r\n                   availability has risen dramatically with the trend towards\r\n                   capability computing, which drives the race for scientific\r\n                   discovery by running applications on the fastest machines\r\n                   available while desiring significant amounts of time (weeks\r\n                   and months) without interruption. These machines must be able\r\n                   to run in the event of frequent interrupts in such a manner\r\n                   that the capability is not severely degraded. Thus, research\r\n                   and development of scalable RAS technologies is paramount to\r\n                   the success of future extreme-scale systems. This talk\r\n                   summarizes our accomplishments in the area of high-level RAS\r\n                   for HPC, such as developed concepts and implemented\r\n                   proof-of-concept prototypes.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann09high,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"High-Performance Computing Research and {MSc} Internship\r\n                   Opportunities at {Oak Ridge National Laboratory}\",\r\n  month         = may # \"~11, \",\r\n  year          = \"2009\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.cs.reading.ac.uk}{Department\r\n                   of Computer Science}, \\href{http:\/\/www.reading.ac.uk}\r\n                   {University of Reading}, Reading, United Kingdom}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann09high.pdf\",\r\n  abstract      = \"Oak Ridge National Laboratory (ORNL) is the largest energy\r\n                   laboratory in the United States. Its National Center for\r\n                   Computational Sciences (NCCS) provides the most powerful\r\n                   computing resources in the world for open scientific\r\n                   research. Jaguar, a Cray XT5 system at NCCS, is the second\r\n                   HPC system to exceed 1 PFlop\/s ($10^{15}$ Floating Point\r\n                   Operations Per Second), and the fastest open science\r\n                   supercomputer in the world. It recently ranked #2 in the Top\r\n                   500 List of Supercomputer Sites with a maximal LINPACK\r\n                   benchmark performance of 1.059 PFlop\/s and a theoretical peak\r\n                   performance of 1.3814 PFlop\/s. Annually, 80 percent of\r\n                   Jaguar's resources are allocated through the U.S Department\r\n                   of Energy's Innovative and Novel Computational Impact on\r\n                   Theory and Experiment (INCITE) program, a competitively\r\n                   selected, peer reviewed process open to researchers from\r\n                   universities, industry, government and non-profit\r\n                   organizations. These allocations address some of the most\r\n                   challenging scientific problems in areas such as climate\r\n                   modeling, renewable energy, materials science, fusion and\r\n                   combustion. In conjunction with NCCS, the Computer Science\r\n                   and Mathematics Division at ORNL performs basic and applied\r\n                   research in HPC, mathematics, and intelligent systems. This\r\n                   talk gives a summary of the HPC research performed at ORNL.\r\n                   It provides details about the Jaguar peta-scale computing\r\n                   resource, an overview of the computational science research\r\n                   carried out using ORNL's computing resources, and a\r\n                   description of various computer science efforts targeting\r\n                   solutions for next-generation HPC systems. This talk also\r\n                   provides information about internship opportunities for MSc\r\n                   students.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann09modular,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Modular Redundancy for Soft-Error Resilience in Large-Scale\r\n                   {HPC} Systems\",\r\n  month         = may # \"~3-8, \",\r\n  year          = \"2009\",\r\n  howpublished  = \"{Invited talk at the \\href{http:\/\/www.dagstuhl.de\/en\/program\/calendar\/semhp\/?semnr=09191}\r\n                   {Dagstuhl Seminar on Fault Tolerance in High-Performance\r\n                   Computing and Grids}, Schloss Dagstuhl, Wadern, Germany}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann09modular.pdf\",\r\n  abstract      = \"Recent investigations into resilience of large-scale\r\n                   high-performance computing (HPC) systems showed a continuous\r\n                   trend of decreasing reliability and availability. Newly\r\n                   installed systems have a lower mean-time to failure (MTTF)\r\n                   and a higher mean-time to recover (MTTR) than their\r\n                   predecessors. Modular redundancy is being used in many\r\n                   mission critical systems today to provide for resilience,\r\n                   such as for aerospace and command & control systems. The\r\n                   primary argument against modular redundancy for resilience\r\n                   in HPC has always been that the capability of a HPC system,\r\n                   and respective return on investment, would be significantly\r\n                   reduced. We argue that modular redundancy can significantly\r\n                   increase compute node availability as it removes the impact\r\n                   of scale from single compute node MTTR. We further argue that\r\n                   single compute nodes can be much less reliable, and therefore\r\n                   less expensive, and still be highly available, if their\r\n                   MTTR\/MTTF ratio is maintained.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann09proactive2,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Proactive Fault Tolerance Using Preemptive Migration\",\r\n  month         = apr # \"~22-24, \",\r\n  year          = \"2009\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{http:\/\/acet.rdg.ac.uk\/events\/details\/cancun.php}\r\n                   {$3^{rd}$ Collaborative and Grid Computing Technologies\r\n                   Workshop (CGCTW) 2009}, Cancun, Mexico}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann09proactive2.pdf\",\r\n  abstract      = \"The continuing growth in high-performance computing (HPC)\r\n                   system scale poses a challenge for system software and\r\n                   scientific applications with respect to reliability,\r\n                   availability and serviceability (RAS). In order to address\r\n                   anticipated high failure rates, resiliency characteristics\r\n                   have become an urgent priority for next-generation HPC\r\n                   systems. The concept of proactive fault tolerance prevents\r\n                   compute node failures from impacting running parallel\r\n                   applications by preemptively migrating application parts\r\n                   away from nodes that are about to fail. This talk presents\r\n                   our past and ongoing efforts in proactive fault resilience\r\n                   for HPC. Presented work includes proactive fault resilience\r\n                   techniques, transparent process- and virtual-machine-level\r\n                   migration, system and application reliability models and\r\n                   analyses, failure prediction, and trade-off models for\r\n                   combining preemptive migration with checkpoint\/restart. All\r\n                   these individual technologies are put into context with a\r\n                   proposed holistic HPC fault resilience framework.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann09resiliency,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Resiliency\",\r\n  month         = mar # \"~9-12, \",\r\n  year          = \"2009\",\r\n  howpublished  = \"{Panel at the\r\n                   \\href{http:\/\/www.cs.sandia.gov\/Conferences\/SOS13}{$13^{th}$\r\n                   Workshop on Distributed Supercomputing (SOS) 2009}, Hilton\r\n                   Head, SC, USA}\"\r\n}<\/pre>\n
<\/a>@misc{engelmann08high,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"High-Performance Computing Research at {Oak Ridge National\r\n                   Laboratory}\",\r\n  month         = dec # \"~8, \",\r\n  year          = \"2008\",\r\n  howpublished  = \"{Invited talk at the Reading Annual Computational Science\r\n                    Workshop, Reading, United Kingdom}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann08high.pdf\",\r\n  abstract      = \"Oak Ridge National Laboratory (ORNL) is the largest energy\r\n                   laboratory in the United States. Its National Center for\r\n                   Computational Sciences (NCCS) provides the most powerful\r\n                   computing resources in the world for open scientific\r\n                   research. Jaguar, a Cray XT5 system at NCCS, is the second\r\n                   HPC system to exceed 1 PFlop\/s (10^15 Floating Point\r\n                   Operations Per Second), and the fastest open science\r\n                   supercomputer in the world. It recently ranked #2 in the Top\r\n                   500 List of Supercomputer Sites with a maximal LINPACK\r\n                   benchmark performance of 1.059 PFlop\/s and a theoretical peak\r\n                   performance of 1.3814 PFlop\/s. Annually, 80 percent of\r\n                   Jaguar\u2019s resources are allocated through the U.S Department\r\n                   of Energy\u2019s Innovative and Novel Computational Impact on\r\n                   Theory and Experiment (INCITE) program, a competitively\r\n                   selected, peer reviewed process open to researchers from\r\n                   universities, industry, government and non-profit\r\n                   organizations. These allocations address some of the most\r\n                   challenging scientific problems in areas such as climate\r\n                   modeling, renewable energy, materials science, fusion and\r\n                   combustion. In conjunction with NCCS, the Computer Science\r\n                   and Mathematics Division at ORNL performs basic and applied\r\n                   research in HPC, mathematics, and intelligent systems. This\r\n                   talk gives a summary of the HPC research performed at ORNL.\r\n                   It provides details about the Jaguar peta-scale computing\r\n                   resource, an overview of the computational science research\r\n                   carried out using ORNL\u2019s computing resources, and a\r\n                   description of various computer science efforts targeting\r\n                   solutions for next-generation HPC systems.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann08modular,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Modular Redundancy in {HPC} Systems: {W}hy, Where, When and How?\",\r\n  month         = oct # \"~15, \",\r\n  year          = \"2008\",\r\n  howpublished  = \"{Invited talk at the $1^{st}$ HPC Resiliency Summit: Workshop\r\n                   on Resiliency for Petascale HPC 2008, in conjunction with the\r\n                   \\href{http:\/\/www.lanl.gov\/conferences\/lacss\/2008}{$1^{st}$\r\n                   Los Alamos Computer Science Symposium (LACSS) 2008}, Santa\r\n                   Fe, NM, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann08modular.ppt.pdf\",\r\n  abstract      = \"The continuing growth in high-performance computing (HPC)\r\n                   system scale poses a challenge for system software and\r\n                   scientific applications with respect to reliability,\r\n                   availability and serviceability (RAS). With only very few\r\n                   exceptions, the availability of recently installed systems\r\n                   has been lower in comparison to the same deployment phase of\r\n                   their predecessors. As a result, sites lower allowable job\r\n                   run times in order to force applications to store\r\n                   intermediate results (checkpoints) as insurance against lost\r\n                   computation time. However, checkpoints themselves waste\r\n                   valuable computation time and resources. In contrast to the\r\n                   experienced loss of availability, the demand for continuous\r\n                   availability has risen dramatically with the trend towards\r\n                   capability computing, which drives the race for scientific\r\n                   discovery by running applications on the fastest machines\r\n                   available while desiring significant amounts of time (weeks\r\n                   and months) without interruption. These machines must be able\r\n                   to run in the event of frequent interrupts in such a manner\r\n                   that the capability is not severely degraded. Thus, research\r\n                   and development of scalable RAS technologies is paramount to\r\n                   the success of future extreme-scale systems. This talk\r\n                   summarizes our past accomplishments, ongoing work, and future\r\n                   plans in the area of high-level RAS for HPC.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann08resiliency,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Resiliency for High-Performance Computing\",\r\n  month         = apr # \"~10-12, \",\r\n  year          = \"2008\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{http:\/\/acet.rdg.ac.uk\/events\/details\/cancun.php}\r\n                   {$2^{nd}$ Collaborative and Grid Computing Technologies\r\n                   Workshop (CGCTW) 2008}, Cancun, Mexico}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann08resiliency.ppt.pdf\",\r\n  abstract      = \"In order to address anticipated high failure rates,\r\n                   resiliency characteristics have become an urgent priority for\r\n                   next-generation high-performance computing (HPC) systems. One\r\n                   major source of concern are non-recoverable soft errors,\r\n                   i.e., bit flips in memory, cache, registers, and logic. The\r\n                   probability of such errors not only grows with system size,\r\n                   but also with increasing architectural vulnerability caused\r\n                   by employing accelerators and by shrinking nanometer\r\n                   technology. Reactive fault tolerance technologies, such as\r\n                   checkpoint\/restart, are unable to handle high failure rates\r\n                   due to associated overheads, while proactive resiliency\r\n                   technologies, such as preemptive migration, simply fail as\r\n                   random soft errors can't be predicted. This talk proposes a\r\n                   new, bold direction in resiliency for HPC as it targets\r\n                   resiliency for next-generation extreme-scale HPC systems at\r\n                   the system software level through computational redundancy\r\n                   strategies, i.e., dual- and triple-modular redundancy.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann08advanced,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Advanced Fault Tolerance Solutions for High Performance\r\n                   Computing\",\r\n  month         = feb # \"~11, \",\r\n  year          = \"2008\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.laas.fr}{Laboratoire\r\n                   d'Analyse et d'Architecture des Syst\\'emes},\r\n                   \\href{http:\/\/www.cnrs.fr}{Centre National de la Recherche\r\n                   Scientifique}, Toulouse, France}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann08advanced.ppt.pdf\",\r\n  abstract      = \"The continuing growth in high performance computing (HPC)\r\n                   system scale poses a challenge for system software and\r\n                   scientific applications with respect to reliability,\r\n                   availability and serviceability (RAS). With only very few\r\n                   exceptions, the availability of recently installed systems\r\n                   has been lower in comparison to the same deployment phase of\r\n                   their predecessors. As a result, sites lower allowable job\r\n                   run times in order to force applications to store\r\n                   intermediate results (checkpoints) as insurance against lost\r\n                   computation time. However, checkpoints themselves waste\r\n                   valuable computation time and resources. In contrast to the\r\n                   experienced loss of availability, the demand for continuous\r\n                   availability has risen dramatically with the trend towards\r\n                   capability computing, which drives the race for scientific\r\n                   discovery by running applications on the fastest machines\r\n                   available while desiring significant amounts of time (weeks\r\n                   and months) without interruption. These machines must be able\r\n                   to run in the event of frequent interrupts in such a manner\r\n                   that the capability is not severely degraded. Thus, research\r\n                   and development of scalable RAS technologies is paramount to\r\n                   the success of future extreme-scale systems. This talk\r\n                   summarizes our accomplishments in the area of high-level RAS\r\n                   for HPC, such as developed concepts and implemented\r\n                   proof-of-concept prototypes, and describes existing\r\n                   limitations, such as performance issues, which need to be\r\n                   dealt with for production-type deployment.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann07service,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Service-Level High Availability in Parallel and Distributed\r\n                   Systems\",\r\n  month         = oct # \"~10, \",\r\n  year          = \"2007\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.cs.reading.ac.uk}{Department\r\n                   of Computer Science}, \\href{http:\/\/www.reading.ac.uk}\r\n                   {University of Reading}, Reading, United Kingdom}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann07service.pdf\",\r\n  abstract      = \"As service-oriented architectures become more important in\r\n                   parallel and distributed computing systems, individual\r\n                   service instance reliability as well as appropriate service\r\n                   redundancy are essential to increase overall system\r\n                   availability. This talk focuses on redundancy strategies\r\n                   using service-level replication techniques. An overview of\r\n                   existing programming models for service-level high\r\n                   availability is presented and their differences,\r\n                   similarities, advantages, and disadvantages are discussed.\r\n                   Recent advances in providing service-level symmetric\r\n                   active\/active high availability are discussed. While the\r\n                   primary target of the presented research is high availability\r\n                   for service nodes in tightly-coupled extreme-scale\r\n                   high-performance computing (HPC) systems, it is also\r\n                   applicable to loosely-coupled distributed computing\r\n                   scenarios.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann07advanced2,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Advanced Fault Tolerance Solutions for High Performance\r\n                   Computing\",\r\n  month         = jun # \"~8, \",\r\n  year          = \"2007\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{http:\/\/www.thaigrid.or.th\/wttc2007}{Workshop on Trends,\r\n                   Technologies and Collaborative Opportunities in High\r\n                   Performance and Grid Computing (WTTC) 2007}, Khon Kean,\r\n                   Thailand}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann07advanced2.ppt.pdf\",\r\n  abstract      = \"The continuing growth in high performance computing (HPC)\r\n                   system scale poses a challenge for system software and\r\n                   scientific applications with respect to reliability,\r\n                   availability and serviceability (RAS). With only very few\r\n                   exceptions, the availability of recently installed systems\r\n                   has been lower in comparison to the same deployment phase of\r\n                   their predecessors. As a result, sites lower allowable job\r\n                   run times in order to force applications to store\r\n                   intermediate results (checkpoints) as insurance against lost\r\n                   computation time. However, checkpoints themselves waste\r\n                   valuable computation time and resources. In contrast to the\r\n                   experienced loss of availability, the demand for continuous\r\n                   availability has risen dramatically with the trend towards\r\n                   capability computing, which drives the race for scientific\r\n                   discovery by running applications on the fastest machines\r\n                   available while desiring significant amounts of time (weeks\r\n                   and months) without interruption. These machines must be able\r\n                   to run in the event of frequent interrupts in such a manner\r\n                   that the capability is not severely degraded. Thus, research\r\n                   and development of scalable RAS technologies is paramount to\r\n                   the success of future extreme-scale systems. This talk\r\n                   summarizes our accomplishments in the area of high-level RAS\r\n                   for HPC, such as developed concepts and implemented\r\n                   proof-of-concept prototypes, and describes existing\r\n                   limitations, such as performance issues, which need to be\r\n                   dealt with for production-type deployment.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann07advanced,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Advanced Fault Tolerance Solutions for High Performance\r\n                   Computing\",\r\n  month         = jun # \"~4-5, \",\r\n  year          = \"2007\",\r\n  howpublished  = \"{Invited talk at the\r\n                   \\href{http:\/\/www.thaigrid.or.th\/wttc2007}{Workshop on Trends,\r\n                   Technologies and Collaborative Opportunities in High\r\n                   Performance and Grid Computing (WTTC) 2007}, Bangkok,\r\n                   Thailand}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann07advanced.ppt.pdf\",\r\n  abstract      = \"The continuing growth in high performance computing (HPC)\r\n                   system scale poses a challenge for system software and\r\n                   scientific applications with respect to reliability,\r\n                   availability and serviceability (RAS). With only very few\r\n                   exceptions, the availability of recently installed systems\r\n                   has been lower in comparison to the same deployment phase of\r\n                   their predecessors. As a result, sites lower allowable job\r\n                   run times in order to force applications to store\r\n                   intermediate results (checkpoints) as insurance against lost\r\n                   computation time. However, checkpoints themselves waste\r\n                   valuable computation time and resources. In contrast to the\r\n                   experienced loss of availability, the demand for continuous\r\n                   availability has risen dramatically with the trend towards\r\n                   capability computing, which drives the race for scientific\r\n                   discovery by running applications on the fastest machines\r\n                   available while desiring significant amounts of time (weeks\r\n                   and months) without interruption. These machines must be\r\n                   able to run in the event of frequent interrupts in such a\r\n                   manner that the capability is not severely degraded. Thus,\r\n                   research and development of scalable RAS technologies is\r\n                   paramount to the success of future extreme-scale systems.\r\n                   This talk summarizes our accomplishments in the area of\r\n                   high-level RAS for HPC, such as developed concepts and\r\n                   implemented proof-of-concept prototypes, and describes\r\n                   existing limitations, such as performance issues, which\r\n                   need to be dealt with for production-type deployment.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann07operating,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Operating System Research at {ORNL}: {S}ystem-level\r\n                   Virtualization\",\r\n  month         = apr # \"~10, \",\r\n  year          = \"2007\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.gup.uni-linz.ac.at}\r\n                   {Institute of Graphics and Parallel Processing},\r\n                   \\href{http:\/\/www.uni-linz.ac.at}{Johannes Kepler University},\r\n                   Linz, Austria}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann07operating.ppt.pdf\",\r\n  abstract      = \"The emergence of virtualization enabled hardware, such as the\r\n                   latest generation AMD and Intel processors, has raised\r\n                   significant interest in High Performance Computing (HPC)\r\n                   community. In particular, system-level virtualization\r\n                   provides an opportunity to advance the design and development\r\n                   of operating systems, programming environments,\r\n                   administration practices, and resource management tools. This\r\n                   leads to some potential research topics for HPC, such as\r\n                   failure tolerance, system management, and solutions for\r\n                   application porting to new HPC platforms. This talk will\r\n                   present an overview of the research in System-level\r\n                   Virtualization taking place by the Systems Research Team in\r\n                   the Computer Science Research Group at Oak Ridge National\r\n                   Laboratory.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann07towards,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Towards High Availability for High-Performance Computing\r\n                   System Services: {A}ccomplishments and Limitations\",\r\n  month         = mar # \"~14, \",\r\n  year          = \"2007\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.cs.reading.ac.uk}{Department\r\n                   of Computer Science}, \\href{http:\/\/www.reading.ac.uk}\r\n                   {University of Reading}, Reading, United Kingdom}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann07towards.pdf\",\r\n  abstract      = \"During the last several years, our teams at Oak Ridge\r\n                   National Laboratory, Louisiana Tech University, and Tennessee\r\n                   Technological University focused on efficient redundancy\r\n                   strategies for head and service nodes of high-performance\r\n                   computing (HPC) systems in order to pave the way for high\r\n                   availability (HA) in HPC. These nodes typically run critical\r\n                   HPC system services, like job and resource management, and\r\n                   represent single points of failure and control for an entire\r\n                   HPC system. The overarching goal of our research is to\r\n                   provide high-level reliability, availability, and\r\n                   serviceability (RAS) for HPC systems by combining HA and HPC\r\n                   technology. This talk summarizes our accomplishments, such as\r\n                   developed concepts and implemented proof-of-concept\r\n                   prototypes, and describes existing limitations, such as\r\n                   performance issues, which need to be dealt with for\r\n                   production-type deployment.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann06high,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"High Availability for Ultra-Scale High-End Scientific\r\n                   Computing\",\r\n  month         = jun # \"~9, \",\r\n  year          = \"2006\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.cs.reading.ac.uk}{Department\r\n                   of Computer Science}, \\href{http:\/\/www.reading.ac.uk}\r\n                   {University of Reading}, Reading, United Kingdom}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann06high.ppt.pdf\",\r\n  abstract      = \"A major concern in exploiting ultra-scale architectures for\r\n                   scientific high-end computing (HEC) with tens to hundreds of\r\n                   thousands of processors, such as the IBM Blue Gene\/L and the\r\n                   Cray X1, is the potential inability to identify problems and\r\n                   take preemptive action before a failure impacts a running\r\n                   job. In fact, in systems of this scale, predictions estimate\r\n                   the mean time to interrupt in terms of hours. Current\r\n                   solutions for fault-tolerance in HEC focus on dealing with\r\n                   the result of a failure. However, most are unable to handle\r\n                   runtime system configuration changes caused by failures and\r\n                   require a complete restart of essential system services\r\n                   (e.g. MPI) or even of the entire machine. High availability\r\n                   (HA) computing strives to avoid the problems of unexpected\r\n                   failures through preemptive measures. There are various\r\n                   techniques to implement high availability. In contrast to\r\n                   active\/hot-standby high availability with its fail-over\r\n                   model, active\/active high availability with its virtual\r\n                   synchrony model is superior in many areas including\r\n                   scalability, throughput, availability and responsiveness.\r\n                   However, it is significantly more complex. The overall goal\r\n                   of our research is to expand today`s effort in HA for HEC,\r\n                   so that systems that have the ability to hot-swap hardware\r\n                   components can be kept alive by an OS runtime environment\r\n                   that understands the concept of dynamic system configuration.\r\n                   This talk will present an overview of recent research at Oak\r\n                   Ridge National Laboratory in high availability solutions for\r\n                   ultra-scale scientific high-end computing.\"\r\n}<\/pre>\n
<\/a>@misc{scott06advancing,\r\n  author        = \"Stephen L. Scott\r\n                   and Christian Engelmann\",\r\n  title         = \"Advancing Reliability, Availability and Serviceability for\r\n                   High-Performance Computing\",\r\n  month         = apr # \"~19, \",\r\n  year          = \"2006\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.gup.uni-linz.ac.at}\r\n                   {Institute of Graphics and Parallel Processing},\r\n                   \\href{http:\/\/www.uni-linz.ac.at}{Johannes Kepler University},\r\n                   Linz, Austria}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/scott06advancing.ppt.pdf\",\r\n  abstract      = \"Today\u2019s high performance computing systems have several\r\n                   reliability deficiencies resulting in noticeable availability\r\n                   and serviceability issues. For example, head and service\r\n                   nodes represent a single point of failure and control for an\r\n                   entire system as they render it inaccessible and unmanageable\r\n                   in case of a failure until repair, causing a significant\r\n                   downtime. Furthermore, current solutions for fault-tolerance\r\n                   focus on dealing with the result of a failure. However, most\r\n                   are unable to transparently mask runtime system configuration\r\n                   changes caused by failures and require a complete restart of\r\n                   essential system services, such as MPI, in case of a failure.\r\n                   High availability computing strives to avoid the problems of\r\n                   unexpected failures through preemptive measures. The overall\r\n                   goal of our research is to expand today\u2019s effort in high\r\n                   availability for high-performance computing, so that systems\r\n                   can be kept alive by an OS runtime environment that\r\n                   understands the concepts of dynamic system configuration and\r\n                   degraded operation mode. This talk will present an overview\r\n                   of recent research performed at Oak Ridge National Laboratory\r\n                   in collaboration with Louisiana Tech University, North\r\n                   Carolina State University and the University of Reading in\r\n                   developing core technologies and proof-of-concept prototypes\r\n                   that improve the overall reliability, availability and\r\n                   serviceability of high-performance computing systems.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann05high4,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"High Availability for Ultra-Scale High-End Scientific\r\n                   Computing\",\r\n  month         = oct # \"~18, \",\r\n  year          = \"2005\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.cs.reading.ac.uk}{Department\r\n                   of Computer Science}, \\href{http:\/\/www.reading.ac.uk}\r\n                   {University of Reading}, Reading, United Kingdom}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann05high4.ppt.pdf\",\r\n  abstract      = \"A major concern in exploiting ultra-scale architectures for\r\n                   scientific high-end computing (HEC) with tens to hundreds of\r\n                   thousands of processors, such as the IBM Blue Gene\/L and the\r\n                   Cray X1, is the potential inability to identify problems and\r\n                   take preemptive action before a failure impacts a running\r\n                   job. In fact, in systems of this scale, predictions estimate\r\n                   the mean time to interrupt in terms of hours. Current\r\n                   solutions for fault-tolerance in HEC focus on dealing with\r\n                   the result of a failure. However, most are unable to handle\r\n                   runtime system configuration changes caused by failures and\r\n                   require a complete restart of essential system services (e.g.\r\n                   MPI) or even of the entire machine. High availability (HA)\r\n                   computing strives to avoid the problems of unexpected\r\n                   failures through preemptive measures. There are various\r\n                   techniques to implement high availability. In contrast to\r\n                   active\/hot-standby high availability with its fail-over\r\n                   model, active\/active high availability with its virtual\r\n                   synchrony model is superior in many areas including\r\n                   scalability, throughput, availability and responsiveness.\r\n                   However, it is significantly more complex. The overall goal\r\n                   of our research is to expand today`s effort in HA for HEC, so\r\n                   that systems that have the ability to hot-swap hardware\r\n                   components can be kept alive by an OS runtime environment\r\n                   that understands the concept of dynamic system configuration.\r\n                   This talk will present an overview of recent research at Oak\r\n                   Ridge National Laboratory in high availability solutions for\r\n                   ultra-scale scientific high-end computing.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann05high3,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"High Availability for Ultra-Scale High-End Scientific\r\n                   Computing\",\r\n  month         = sep # \"~26, \",\r\n  year          = \"2005\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.uncfsu.edu\/macsc}{Department\r\n                   of Mathematics and Computer Science},\r\n                   \\href{http:\/\/www.uncfsu.edu}{Fayetteville State University},\r\n                   Fayetteville, NC, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann05high3.ppt.pdf\",\r\n  abstract      = \"A major concern in exploiting ultra-scale architectures for\r\n                   scientific high-end computing (HEC) with tens to hundreds of\r\n                   thousands of processors, such as the IBM Blue Gene\/L and the\r\n                   Cray X1, is the potential inability to identify problems and\r\n                   take preemptive action before a failure impacts a running\r\n                   job. In fact, in systems of this scale, predictions estimate\r\n                   the mean time to interrupt in terms of hours. Current\r\n                   solutions for fault-tolerance in HEC focus on dealing with\r\n                   the result of a failure. However, most are unable to handle\r\n                   runtime system configuration changes caused by failures and\r\n                   require a complete restart of essential system services (e.g.\r\n                   MPI) or even of the entire machine. High availability (HA)\r\n                   computing strives to avoid the problems of unexpected\r\n                   failures through preemptive measures. There are various\r\n                   techniques to implement high availability. In contrast to\r\n                   active\/hot-standby high availability with its fail-over\r\n                   model, active\/active high availability with its virtual\r\n                   synchrony model is superior in many areas including\r\n                   scalability, throughput, availability and responsiveness.\r\n                   However, it is significantly more complex. The overall goal\r\n                   of our research is to expand today\u2019s effort in HA for HEC, so\r\n                   that systems that have the ability to hot-swap hardware\r\n                   components can be kept alive by an OS runtime environment\r\n                   that understands the concept of dynamic system configuration.\r\n                   This talk will present an overview of recent research at Oak\r\n                   Ridge National Laboratory in fault tolerance and high\r\n                   availability solutions for ultra-scale scientific high-end\r\n                   computing.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann05high2,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"High Availability for Ultra-Scale High-End Scientific\r\n                   Computing\",\r\n  month         = may # \"~13, \",\r\n  year          = \"2005\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.cs.reading.ac.uk}{Department\r\n                   of Computer Science}, \\href{http:\/\/www.reading.ac.uk}\r\n                   {University of Reading}, Reading, United Kingdom}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann05high2.ppt.pdf\",\r\n  abstract      = \"A major concern in exploiting ultra-scale architectures for\r\n                   scientific high-end computing (HEC) with tens to hundreds of\r\n                   thousands of processors, such as the IBM Blue Gene\/L and the\r\n                   Cray X1, is the potential inability to identify problems and\r\n                   take preemptive action before a failure impacts a running\r\n                   job. In fact, in systems of this scale, predictions estimate\r\n                   the mean time to interrupt in terms of hours. Current\r\n                   solutions for fault-tolerance in HEC focus on dealing with\r\n                   the result of a failure. However, most are unable to handle\r\n                   runtime system configuration changes caused by failures and\r\n                   require a complete restart of essential system services (e.g.\r\n                   MPI) or even of the entire machine. High availability (HA)\r\n                   computing strives to avoid the problems of unexpected\r\n                   failures through preemptive measures. There are various\r\n                   techniques to implement high availability. In contrast to\r\n                   active\/hot-standby high availability with its fail-over\r\n                   model, active\/active high availability with its virtual\r\n                   synchrony model is superior in many areas including\r\n                   scalability, throughput, availability and responsiveness.\r\n                   However, it is significantly more complex. The overall goal\r\n                   of our research is to expand today\u2019s effort in HA for HEC,\r\n                   so that systems that have the ability to hot-swap hardware\r\n                   components can be kept alive by an OS runtime environment\r\n                   that understands the concept of dynamic system configuration.\r\n                   This talk will present an overview of recent research at Oak\r\n                   Ridge National Laboratory in fault-tolerant heterogeneous\r\n                   metacomputing, advanced super-scalable algorithms and high\r\n                   availability system software for ultra-scale scientific\r\n                   high-end computing.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann05high1,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"High Availability for Ultra-Scale High-End Scientific\r\n                   Computing\",\r\n  month         = apr # \"~15, \",\r\n  year          = \"2005\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/cenit.latech.edu}{Center for\r\n                   Entrepreneurship and Information Technology},\r\n                   \\href{http:\/\/www.latech.edu}{Louisiana Tech University},\r\n                   Ruston, LA, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann05high1.ppt.pdf\",\r\n  abstract      = \"A major concern in exploiting ultra-scale architectures for\r\n                   scientific high-end computing (HEC) with tens to hundreds of\r\n                   thousands of processors is the potential inability to\r\n                   identify problems and take preemptive action before a failure\r\n                   impacts a running job. In fact, in systems of this scale,\r\n                   predictions estimate the mean time to interrupt in terms of\r\n                   hours. Current solutions for fault-tolerance in HEC focus on\r\n                   dealing with the result of a failure. However, most are\r\n                   unable to handle runtime system configuration changes caused\r\n                   by failures and require a complete restart of essential\r\n                   system services (e.g. MPI) or even of the entire machine.\r\n                   High availability (HA) computing strives to avoid the\r\n                   problems of unexpected failures through preemptive measures.\r\n                   There are various techniques to implement high availability.\r\n                   In contrast to active\/hot-standby high availability with its\r\n                   fail-over model, active\/active high availability with its\r\n                   virtual synchrony model is superior in many areas including\r\n                   scalability, throughput, availability and responsiveness.\r\n                   However, it is significantly more complex. The overall goal\r\n                   of this research is to expand today\u2019s effort in HA for HEC,\r\n                   so that systems that have the ability to hot-swap hardware\r\n                   components can be kept alive by an OS runtime environment\r\n                   that understands the concept of dynamic system configuration.\r\n                   With the aim of addressing the future challenges of high\r\n                   availability in ultra-scale HEC, this project intends to\r\n                   develop a proof-of-concept implementation of an active\/active\r\n                   high availability system software framework.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann04diskless,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Diskless Checkpointing on Super-scale Architectures --\r\n                   {A}pplied to the Fast Fourier Transform\",\r\n  month         = feb # \"~25, \",\r\n  year          = \"2004\",\r\n  howpublished  = \"{Invited talk at the \\href{http:\/\/www.siam.org\/meetings\/pp04}\r\n                   {$11^{th}$ SIAM Conference on Parallel Processing for\r\n                   Scientific Computing (SIAM PP) 2004}, San Francisco, CA,\r\n                   USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann04diskless.ppt.pdf\",\r\n  abstract      = \"This talk discusses the issue of fault-tolerance in\r\n                   distributed computer systems with tens or hundreds of\r\n                   thousands of diskless processor units. Such systems, like the\r\n                   IBM Blue Gene\/L, are predicted to be deployed in the next\r\n                   five to ten years. Since a 100,000-processor system is going\r\n                   to be less reliable, scientific applications need to be able\r\n                   to recover from occurring failures more efficiently. In this\r\n                   paper, we adapt the present technique of diskless\r\n                   checkpointing to such huge distributed systems in order to\r\n                   equip existing scientific algorithms with super-scalable\r\n                   fault-tolerance. First, we discuss the method of diskless\r\n                   checkpointing, then we adapt this technique to super-scale\r\n                   architectures and finally we present results from an\r\n                   implementation of the Fast Fourier Transform that uses the\r\n                   adapted technique to achieve super-scale fault-tolerance.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann04superscalable,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Super-scalable Algorithms -- {N}ext Generation Supercomputing\r\n                   on 100,000 and more Processors\",\r\n  month         = jan # \"~29, \",\r\n  year          = \"2004\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.csm.ornl.gov}{Computer\r\n                   Science and Mathematics Division}, \\href{http:\/\/www.ornl.gov}\r\n                   {Oak Ridge National Laboratory}, Oak Ridge, TN, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann04superscalable.ppt.pdf\",\r\n  abstract      = \"This talk discusses recent research into the issues and\r\n                   potential problems of algorithm scalability and\r\n                   fault-tolerance on next-generation high-performance computer\r\n                   systems with tens and even hundreds of thousands of\r\n                   processors. Such massively parallel computers, like the IBM\r\n                   Blue Gene\/L, are going to be deployed in the next five to ten\r\n                   years and existing deficiencies in scalability and\r\n                   fault-tolerance need to be addressed soon. Scientific\r\n                   algorithms have shown poor scalability on 10,000-processor\r\n                   systems that exist today. Furthermore, future systems will be\r\n                   less reliable due to the large number of components.\r\n                   Super-scalable algorithms, which have the properties of scale\r\n                   invariance and natural fault-tolerance, are able to get the\r\n                   correct answer despite multiple task failures and without\r\n                   checkpointing. We will show that such algorithms exist for a\r\n                   wide variety of problems, such as finite difference, finite\r\n                   element, multigrid and global maximum. Despite these\r\n                   findings, traditional algorithms may still be preferred due\r\n                   to their known behavior, or simply because a super-scalable\r\n                   algorithm does not exist or is hard to find for a particular\r\n                   problem. In this case, we propose a peer-to-peer diskless\r\n                   checkpointing algorithm that can provide scale invariant\r\n                   fault-tolerance.\"\r\n}<\/pre>\n
<\/a>@misc{engelmann03distributed,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Distributed Peer-to-Peer Control for {Harness}\",\r\n  month         = feb # \"~11, \",\r\n  year          = \"2004\",\r\n  howpublished  = \"{Seminar at the \\href{http:\/\/www.csc.ncsu.edu}{Department of\r\n                   Computer Science}, \\href{http:\/\/www.ncsu.edu}{North Carolina\r\n                   State University}, Raleigh, NC, USA}\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann03distributed.ppt.pdf\",\r\n  abstract      = \"Harness is an adaptable fault-tolerant virtual machine\r\n                   environment for next-generation heterogeneous distributed\r\n                   computing developed as a follow on to PVM. It additionally\r\n                   enables the assembly of applications from plug-ins and\r\n                   provides fault-tolerance. This work describes the distributed\r\n                   control, which manages global state replication to ensure a\r\n                   high-availability of service. Group communication services\r\n                   achieve an agreement on an initial global state and a linear\r\n                   history of global state changes at all members of the\r\n                   distributed virtual machine. This global state is replicated\r\n                   to all members to easily recover from single, multiple and\r\n                   cascaded faults. A peer-to-peer ring network architecture and\r\n                   tunable multi-point failure conditions provide heterogeneity\r\n                   and scalability. Finally, the integration of the distributed\r\n                   control into the multi-threaded kernel architecture of\r\n                   Harness offers a fault-tolerant global state database service\r\n                   for plug-ins and applications.\"\r\n}<\/pre>\n
<\/a>@mastersthesis{jones10simulation,\r\n  author        = \"Ian S. Jones\",\r\n  title         = \"Simulation of Large Scale Architectures on High Performance\r\n                   Computers\",\r\n  month         = oct # \"~22, \",\r\n  year          = \"2010\",\r\n  school        = \"\\href{http:\/\/www.cs.reading.ac.uk}{Department of Computer\r\n                   Science}, \\href{http:\/\/www.reading.ac.uk}{University of\r\n                   Reading}, UK\",\r\n  note          = \"Thesis research performed at Oak Ridge National Laboratory.\r\n                   Advisors: Prof. Vassil N. Alexandrov (University of Reading);\r\n                   Christian Engelmann (Oak Ridge National Laboratory);\r\n                   George Bosilca (University of Tennessee, Knoxville)\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/jones10simulation.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/jones10simulation.ppt.pdf\",\r\n  abstract      = \"Powerful supercomputers often need to be simulated for the\r\n                   purposes of testing the scalability of various applications.\r\n                   This thesis endeavours to further develop the existing\r\n                   simulator, XSIM, and implement the functionality to simulate\r\n                   real-world networks and the latency which might be encountered\r\n                   by messages travelling through that network. The upgraded\r\n                   simulator will then be tested at the Oak Ridge National\r\n                   Laboratory. The work completed herein should provide a solid\r\n                   foundation for further improvements to XSIM; it simulates a\r\n                   variety of basic network topologies, calculating the shortest\r\n                   path for any given message and generates a transmission time.\"\r\n}<\/pre>\n
<\/a>@mastersthesis{boehm10development,\r\n  author        = \"Swen B{\\\"o}hm\",\r\n  title         = \"Development of a {RAS} Framework for {HPC} Environments:\r\n                   {Realtime} Data Reduction of Monitoring Data\",\r\n  month         = mar # \"~12, \",\r\n  year          = \"2010\",\r\n  school        = \"\\href{http:\/\/www.cs.reading.ac.uk}{Department of Computer\r\n                   Science}, \\href{http:\/\/www.reading.ac.uk}{University of\r\n                   Reading}, UK\",\r\n  note          = \"Thesis research performed at Oak Ridge National Laboratory.\r\n                   Advisors: Prof. Vassil N. Alexandrov (University of Reading);\r\n                   Christian Engelmann (Oak Ridge National Laboratory);\r\n                   George Bosilca (University of Tennessee, Knoxville)\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/boehm10development.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/boehm10development.ppt.pdf\",\r\n  abstract      = \"The advancements of high-performance computing (HPC) systems\r\n                   in the last decades lead to more and more complex systems\r\n                   containing thousands or tens-of-thousands computing systems\r\n                   that are working together. While the computational performance\r\n                   of these systems increased dramaticaly in the last years the\r\n                   I\/O subsystems have not gained such a significant improvement.\r\n                   With increasing nummbers of hardware components in the next\r\n                   generation HPC systems maintaining the relaiability of such\r\n                   systems becomes more and more difficult since the probability\r\n                   of hardware failures is increasing with the number of\r\n                   components. The capacities of traditional reactive fault\r\n                   tolerance technologies are exceeded by the development of next\r\n                   generation systems and alternatives have to be found. This\r\n                   paper discusses a monitoring system that is using data\r\n                   reduction techniques to decrease the amount of the collected\r\n                   data. The system is part of a proactive fault tolerance system\r\n                   that may challenge the reliability problems of exascale\r\n                   HPC systems.\"\r\n}<\/pre>\n
<\/a>@mastersthesis{lauer10simulation,\r\n  author        = \"Frank Lauer\",\r\n  title         = \"Simulation of Advanced Large-Scale {HPC} Architectures\",\r\n  month         = mar # \"~12, \",\r\n  year          = \"2010\",\r\n  school        = \"\\href{http:\/\/www.cs.reading.ac.uk}{Department of Computer\r\n                   Science}, \\href{http:\/\/www.reading.ac.uk}{University of\r\n                   Reading}, UK\",\r\n  note          = \"Thesis research performed at Oak Ridge National Laboratory.\r\n                   Advisors: Prof. Vassil N. Alexandrov (University of Reading);\r\n                   Christian Engelmann (Oak Ridge National Laboratory);\r\n                   George Bosilca (University of Tennessee, Knoxville)\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/lauer10simulation.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/lauer10simulation.ppt.pdf\",\r\n  abstract      = \"The rapid development of massive parallel systems in the high-\r\n                   performance computing (HPC) area requires efficient\r\n                   scalability of applications. The next generation's design of\r\n                   supercomputers is today not certain in terms of what will be\r\n                   the computational, memory and I\/O capabilities. However it is\r\n                   most certain that they become even more parallel. Getting\r\n                   the most performance from these machines in not only a matter\r\n                   of hardware, it is also an issue of programming design.\r\n                   Therefore, it has to be a co-development. However, how to test\r\n                   algorithm's on machines which are not existing today. To\r\n                   address the programming issues in terms of scalability and\r\n                   fault tolerance for the next generation, this projects aim is\r\n                   to design and develop a simulator based on parallel discrete\r\n                   event simulation (PDES) for applications using MPI\r\n                   communication. Some of the fastest supercomputers in the world\r\n                   already interconnecting $10^5$ cores together to catch up the\r\n                   simulator will be able to simulate at least $10^7$ virtual\r\n                   processes.\"\r\n}<\/pre>\n
<\/a>@mastersthesis{litvinova09ras,\r\n  author        = \"Antonina Litvinova\",\r\n  title         = \"{RAS} Framework Engine Prototype\",\r\n  month         = sep # \"~22, \",\r\n  year          = \"2009\",\r\n  school        = \"\\href{http:\/\/www.cs.reading.ac.uk}{Department of Computer\r\n                   Science}, \\href{http:\/\/www.reading.ac.uk}{University of\r\n                   Reading}, UK\",\r\n  note          = \"Thesis research performed at Oak Ridge National Laboratory.\r\n                   Advisors: Prof. Vassil N. Alexandrov (University of Reading);\r\n                   Christian Engelmann (Oak Ridge National Laboratory);\r\n                   George Bosilca (University of Tennessee, Knoxville)\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/litvinova09ras.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/litvinova09ras.ppt.pdf\",\r\n  abstract      = \"Extreme high performance computing (HPC) systems constantly\r\n                   increase in scale from a few thousands of processors cores\r\n                   to thousands of thousands of processors cores and beyond.\r\n                   However their system mean-time to interrupt decreases\r\n                   according. The current approach of fault tolerance in HPC\r\n                   is checkpoint\/restart, i.e. a method based on recovery from\r\n                   experienced failures. However checkpoint\/restart cannot deal\r\n                   with errors in the same efficient way anymore, because of\r\n                   HPC systems modification. For example, increasing error\r\n                   rates, increasing aggregate memory, and not proportionally\r\n                   increasing input\/output capabilities. The recently\r\n                   introduced concept is proactive fault tolerance which\r\n                   avoids experiencing failures through preventative measures.\r\n                   Proactive fault tolerance uses migration which is an\r\n                   emerging technology that prevents failures on HPC systems\r\n                   by migrating applications or application parts away from\r\n                   a node that is deteriorating to a spare node. This thesis\r\n                   discusses work conducted at ORNL to develop a Proactive\r\n                   Fault Tolerance Framework Engine Prototype for HPC systems\r\n                   with high reliability, availability and serviceability.\r\n                   The prototype performs environmental system monitoring,\r\n                   system event logging, parallel job monitoring and system\r\n                   resource monitoring in order to analyse HPC system\r\n                   reliability and to perform fault avoidance through a\r\n                   migration.\"\r\n}<\/pre>\n
<\/a>@mastersthesis{koenning07virtualized,\r\n  author        = \"Bj{\\\"o}rn K{\\\"o}nning\",\r\n  title         = \"Virtualized Environments for the {Harness Workbench}\",\r\n  month         = mar # \"~14, \",\r\n  year          = \"2007\",\r\n  school        = \"\\href{http:\/\/www.cs.reading.ac.uk}{Department of Computer\r\n                   Science}, \\href{http:\/\/www.reading.ac.uk}{University of\r\n                   Reading}, UK\",\r\n  note          = \"Thesis research performed at Oak Ridge National Laboratory.\r\n                   Advisors: Prof. Vassil N. Alexandrov (University of Reading);\r\n                   Christian Engelmann (Oak Ridge National Laboratory)\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/koenning07virtualized.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/koenning07virtualized.ppt.pdf\",\r\n  abstract      = \"The expanded use of computational sciences today leads to a\r\n                   significant need of high performance computing systems. High\r\n                   performance computing is currently undergoing vigorous\r\n                   revival, and multiple efforts are underway to develop much\r\n                   faster computing systems in the near future. New software\r\n                   tools are required for the efficient use of petascale\r\n                   computing systems. With the new Harness Workbench Project\r\n                   the Oak Ridge National Laboratory intends to develop an\r\n                   appropriate development and runtime environment for high\r\n                   performance computing platforms. This dissertation project\r\n                   is part of the Harness Workbench Project, and deals with the\r\n                   development of a concept for virtualised environments and\r\n                   various approaches to create and describe them. The developed\r\n                   virtualisation approach is based on the \\verb|chroot|\r\n                   mechanism and uses platform-independent environment\r\n                   descriptions. File structures and environment variables are\r\n                   emulated to provide the portability of computational software\r\n                   over diverse high performance computing platforms. Security\r\n                   measures and sandbox characteristic are integrable.\"\r\n}<\/pre>\n
<\/a>@mastersthesis{weber07high,\r\n  author        = \"Matthias Weber\",\r\n  title         = \"High Availability for the {Lustre} File System\",\r\n  month         = mar # \"~14, \",\r\n  year          = \"2007\",\r\n  school        = \"\\href{http:\/\/www.cs.reading.ac.uk}{Department of Computer\r\n                   Science}, \\href{http:\/\/www.reading.ac.uk}{University of\r\n                   Reading}, UK\",\r\n  note          = \"Thesis research performed at Oak Ridge National Laboratory.\r\n                   Double diploma in conjunction with the\r\n                   \\href{http:\/\/www.f1.fhtw-berlin.de}{Department of\r\n                   Engineering~I}, \\href{http:\/\/www.f1.fhtw-berlin.de}{Technical\r\n                   College for Engineering and Economics (FHTW) Berlin},\r\n                   Germany. Advisors: Prof. Vassil N. Alexandrov (University of\r\n                   Reading); Christian Engelmann (Oak Ridge National\r\n                   Laboratory)\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/weber07high.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/weber07high.ppt.pdf\",\r\n  abstract      = \"With the growing importance of high performance computing\r\n                   and, more importantly, the fast growing size of sophisticated\r\n                   high performance computing systems, research in the area of\r\n                   high availability is essential to meet the needs to sustain\r\n                   the current growth. This Master thesis project aims to\r\n                   improve the availability of Lustre. Major concern of this\r\n                   project is the metadata server of the file system. The\r\n                   metadata server of Lustre suffers from the last single point\r\n                   of failure in the file system. To overcome this single point\r\n                   of failure an active\/active high availability approach is\r\n                   introduced. The new file system design with multiple MDS\r\n                   nodes running in virtual synchrony leads to a significant\r\n                   increase of availability. Two prototype implementations aim\r\n                   to show how the proposed system design and its new realized\r\n                   form of symmetric active\/active high availability can be\r\n                   accomplished in practice. The results of this work point out\r\n                   the difficulties in adapting the file system to the\r\n                   active\/active high availability design. Tests identify not\r\n                   achieved functionality and show performance problems of the\r\n                   proposed solution. The findings of this dissertation may be\r\n                   used for further work on high availability for distributed\r\n                   file systems.\"\r\n}<\/pre>\n
<\/a>@mastersthesis{baumann06design,\r\n  author        = \"Ronald Baumann\",\r\n  title         = \"Design and Development of Prototype Components for the\r\n                   {Harness} High-Performance Computing Workbench\",\r\n  month         = mar # \"~6, \",\r\n  year          = \"2006\",\r\n  school        = \"\\href{http:\/\/www.cs.reading.ac.uk}{Department of Computer\r\n                   Science}, \\href{http:\/\/www.reading.ac.uk}{University of\r\n                   Reading}, UK\",\r\n  note          = \"Thesis research performed at Oak Ridge National Laboratory.\r\n                   Double diploma in conjunction with the\r\n                   \\href{http:\/\/www.f1.fhtw-berlin.de}{Department of\r\n                   Engineering~I}, \\href{http:\/\/www.f1.fhtw-berlin.de}{Technical\r\n                   College for Engineering and Economics (FHTW) Berlin},\r\n                   Germany. Advisors: Prof. Vassil N. Alexandrov (University of\r\n                   Reading); George A. (Al) Geist and Christian  Engelmann (Oak\r\n                   Ridge National Laboratory)\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/baumann06design.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/baumann06design.ppt.pdf\",\r\n  abstract      = \"This master thesis examines plug-in technology, especially\r\n                   the new field of parallel plug-ins. Plug-ins are popular\r\n                   because they extend the capabilities of software packages\r\n                   such as browsers and Photoshop, and allow an individual user\r\n                   to add new functionality. Parallel plug-ins also provide the\r\n                   above capabilities to a distributed set of resources, i.e.,\r\n                   a plug-in now becomes a set of coordinating plug-ins. Second,\r\n                   the set of plugins may be heterogeneous either in function or\r\n                   because the underlying resources are heterogeneous. This new\r\n                   dimension of complexity provides a rich research space which\r\n                   is explored in this thesis. Experiences are collected and\r\n                   presented as parallel plug-in paradigms and concepts. The\r\n                   Harness framework was used in this project, in particular the\r\n                   plugin manager and available communication capabilities.\r\n                   Plug-ins provide methods for users to extend Harness\r\n                   according to their requirements. The result of this thesis is\r\n                   a parallel plug-in paradigm and template for Harness. Users\r\n                   of the Harness environment will be able to design and\r\n                   implement their applications in the form of parallel plug-ins\r\n                   easier and faster by using the paradigm resulting from this\r\n                   project. Prototypes were implemented which handle different\r\n                   aspects of parallel plug-ins. Parallel plug-in configurations\r\n                   were tested on an appropriate number of Harness kernels,\r\n                   including available communication and error-handling\r\n                   capabilities. Furthermore, research was done in the area of\r\n                   fault tolerance while parallel plug-ins are (un)loaded, as\r\n                   well as while a task is performed.\"\r\n}<\/pre>\n
<\/a>@mastersthesis{uhlemann06high,\r\n  author        = \"Kai Uhlemann\",\r\n  title         = \"High Availability for High-End Scientific Computing\",\r\n  school        = \"\\href{http:\/\/www.cs.reading.ac.uk}{Department of Computer\r\n                   Science}, \\href{http:\/\/www.reading.ac.uk}{University of\r\n                   Reading}, UK\",\r\n  month         = mar # \"~6, \",\r\n  year          = \"2006\",\r\n  note          = \"Thesis research performed at Oak Ridge National Laboratory.\r\n                   Double diploma in conjunction with the\r\n                   \\href{http:\/\/www.f1.fhtw-berlin.de}{Department of\r\n                   Engineering~I}, \\href{http:\/\/www.f1.fhtw-berlin.de}{Technical\r\n                   College for Engineering and Economics (FHTW) Berlin},\r\n                   Germany. Advisors: Prof. Vassil N. Alexandrov (University of\r\n                   Reading); George A. (Al) Geist and  Christian Engelmann (Oak\r\n                   Ridge National Laboratory)\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/uhlemann06high.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/uhlemann06high.ppt.pdf\",\r\n  abstract      = \"With the growing interest and popularity in high performance\r\n                   cluster computing and, more importantly, the fast growing\r\n                   size of compute clusters, research in the area of high\r\n                   availability is essential to meet the needs to sustain the\r\n                   current growth. This Master thesis project introduces a new\r\n                   approach for high availability focusing on the head node of a\r\n                   cluster system. This projects focus is on providing high\r\n                   availability to the job scheduler service, which is the most\r\n                   vital part of the traditional Beowulf-style cluster\r\n                   architecture. This research seeks to add high availability to\r\n                   the job scheduler service and resource management system,\r\n                   typically running on the head node, leading to a significant\r\n                   increase of availability for cluster computing. Also, this\r\n                   software project takes advantage of the virtual synchrony\r\n                   paradigm to achieve active\/active replication, the highest\r\n                   form of high availability. A proof-of-concept implementation\r\n                   shows how high availability can be designed in software and\r\n                   what results can be expected of such a system. The results\r\n                   may be reused for future or existing projects to further\r\n                   improve and extent the high availability of compute\r\n                   clusters.\"\r\n}<\/pre>\n
<\/a>@phdthesis{engelmann08symmetric3,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Symmetric Active\/Active High Availability for\r\n                   High-Performance Computing System Services\",\r\n  month         = dec # \"~8, \",\r\n  year          = \"2008\",\r\n  school        = \"\\href{http:\/\/www.cs.reading.ac.uk}{Department of Computer\r\n                   Science}, \\href{http:\/\/www.reading.ac.uk}{University of\r\n                   Reading}, UK\",\r\n  note          = \"Thesis research performed at Oak Ridge National Laboratory.\r\n                   Advisor: Prof. Vassil N. Alexandrov (University of Reading)\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann08symmetric3.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann08symmetric3.ppt.pdf\",\r\n  abstract      = \"In order to address anticipated high failure rates,\r\n                   reliability, availability and serviceability have become an\r\n                   urgent priority for next-generation high-performance\r\n                   computing (HPC) systems. This thesis aims to pave the way for\r\n                   highly available HPC systems by focusing on their most\r\n                   critical components and by reinforcing them with appropriate\r\n                   high availability solutions. Service components, such as head\r\n                   and service nodes, are the Achilles heel of a HPC system.\r\n                   A failure typically results in a complete system-wide outage.\r\n                   This thesis targets efficient software state replication\r\n                   mechanisms for service component redundancy to achieve high\r\n                   availability as well as high performance. Its methodology\r\n                   relies on defining a modern theoretical foundation for\r\n                   providing service-level high availability, identifying\r\n                   availability deficiencies of HPC systems, and comparing\r\n                   various service-level high availability methods. This thesis\r\n                   showcases several developed proof-of-concept prototypes\r\n                   providing high availability for services running on HPC head\r\n                   and service nodes using the symmetric active\/active\r\n                   replication method, i.e., state-machine replication, to\r\n                   complement prior work in this area using active\/standby and\r\n                   asymmetric active\/active configurations. Presented\r\n                   contributions include a generic taxonomy for service high\r\n                   availability, an insight into availability deficiencies of\r\n                   HPC systems, and a unified definition of service-level high\r\n                   availability methods. Further contributions encompass a fully\r\n                   functional symmetric active\/active high availability\r\n                   prototype for a HPC job and resource management service that\r\n                   does not require modification of service, a fully functional\r\n                   symmetric active\/active high availability prototype for a HPC\r\n                   parallel file system metadata service that offers high\r\n                   performance, and two preliminary prototypes for a transparent\r\n                   symmetric active\/active replication software framework for\r\n                   client-service and dependent service scenarios that hide the\r\n                   replication infrastructure from clients and services.\r\n                   Assuming a mean-time to failure of 5,000 hours for a head or\r\n                   service node, all presented prototypes improve service\r\n                   availability from 99.285\\% to 99.995\\% in a two-node system,\r\n                   and to 99.99996\\% with three nodes.\"\r\n}<\/pre>\n
<\/a>@mastersthesis{engelmann01distributed,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Distributed Peer-to-Peer Control for {Harness}\",\r\n  month         = jul # \"~7, \",\r\n  year          = \"2001\",\r\n  school        = \"\\href{http:\/\/www.cs.reading.ac.uk}{Department of Computer\r\n                   Science}, \\href{http:\/\/www.reading.ac.uk}{University of\r\n                   Reading}, UK\",\r\n  note          = \"Thesis research performed at Oak Ridge National Laboratory.\r\n                   Double diploma in conjunction with the\r\n                   \\href{http:\/\/www.f1.fhtw-berlin.de}{Department of\r\n                   Engineering~I}, \\href{http:\/\/www.f1.fhtw-berlin.de}{Technical\r\n                   College for Engineering and Economics (FHTW) Berlin},\r\n                   Germany. Advisors: Prof. Vassil N. Alexandrov (University of\r\n                   Reading); George A. (Al) Geist (Oak Ridge National\r\n                   Laboratory)\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann01distributed.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann01distributed.ppt.pdf\",\r\n  abstract      = \"Parallel processing, the method of cutting down a large\r\n                   computational problem into many small tasks which are solved\r\n                   in parallel, is a field of increasing importance in science.\r\n                   Cost-effective, flexible and efficient simulations of\r\n                   mathematical models of physical, chemical or biological\r\n                   real-world problems are replacing the traditional\r\n                   experimental research. Current software solutions for\r\n                   parallel and scientific computation, like Parallel Virtual\r\n                   Machine and Message Passing Interface, have limitations in\r\n                   handling faults and failures, in utilizing heterogeneous and\r\n                   dynamically changing communication structures, and in\r\n                   enabling migrating or cooperative applications. The current\r\n                   research in heterogeneous adaptable reconfigurable networked\r\n                   systems (Harness) aims to produce the next generation of\r\n                   software solutions for distributed computing. A\r\n                   high-available and light-weighted distributed virtual\r\n                   machine service provides an encapsulation of a few hundred\r\n                   to a few thousand physical machines in a virtual\r\n                   heterogeneous large scale cluster. A high availability of\r\n                   a service in distributed systems can be achieved by\r\n                   replication of the service state on multiple server\r\n                   processes. If one ore more server processes fails, the\r\n                   surviving ones continue to provide the service because they\r\n                   know the state. Since every member of a distributed virtual\r\n                   machine is part of the distributed virtual machine service\r\n                   state and is able to change this state, a distributed control\r\n                   is needed to replicate the state and maintain its\r\n                   consistency. This distributed control manages state changes\r\n                   as well as the state-replication and the detection of and\r\n                   recovery from faults and failures of server processes. This\r\n                   work analyzes system architectures currently used in\r\n                   heterogeneous distributed computing by defining terms,\r\n                   conditions and assumptions. It shows that such systems are\r\n                   asynchronous and may use partially synchronous communication\r\n                   to detect and to distinguish different classes of faults and\r\n                   failures. It describes how a high availability of a large\r\n                   scale distributed service on a huge number of servers\r\n                   residing on different geographical locations can be realized.\r\n                   Asynchronous group communication services, such as Reliable\r\n                   Broadcast, Atomic Broadcast, Distributed Agreement and\r\n                   Membership, are analyzed to develop linear scalable\r\n                   algorithms in an unidirectional and in a bidirectional\r\n                   connected asynchronous peer-to-peer ring architecture.\r\n                   A Transaction Control group communication service is\r\n                   introduced as state-replication service. The system analysis\r\n                   distinguishes different types of distributed systems, where\r\n                   active transactions execute state changes using\r\n                   non-replicated data of one or more servers and inactive\r\n                   transactions report state changes using replicated data only.\r\n                   It is applicable for passive fault-tolerant distributed\r\n                   databases as well as for active fault-tolerant distributed\r\n                   control mechanisms. No control token is used and time stamps\r\n                   are avoided, so that all members of a server group have equal\r\n                   responsibilities and are independent from the system time.\r\n                   A prototype which implements the most complicated Transaction\r\n                   Control algorithm is realized due to the complexity of the\r\n                   distributed system and the early development stage of the\r\n                   introduced algorithms. The prototype is used to obtain\r\n                   practical experience with the state-replication algorithm.\"\r\n}<\/pre>\n
<\/a>@mastersthesis{engelmann01distributed2,\r\n  author        = \"Christian Engelmann\",\r\n  title         = \"Distributed Peer-to-Peer Control for {Harness}\",\r\n  month         = feb # \"~23, \",\r\n  year          = \"2001\",\r\n  school        = \"\\href{http:\/\/www.f1.fhtw-berlin.de}{Department of\r\n                   Engineering~I}, \\href{http:\/\/www.f1.fhtw-berlin.de}{Technical\r\n                   College for Engineering and Economics (FHTW) Berlin},\r\n                   Germany\",\r\n  note          = \"Thesis research performed at Oak Ridge National Laboratory.\r\n                   Double diploma in conjunction with the\r\n                   \\href{http:\/\/www.cs.reading.ac.uk}{Department of Computer\r\n                   Science}, \\href{http:\/\/www.reading.ac.uk}{University of\r\n                   Reading}, UK. Advisors: Prof. Uwe Metzler (Technical College\r\n                   for Engineering and Economics (FHTW) Berlin); George A. (Al)\r\n                   Geist (Oak Ridge National Laboratory)\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann01distributed2.pdf\",\r\n  url2          = \"http:\/\/www.christian-engelmann.info\/publications\/engelmann01distributed2.ppt.pdf\",\r\n  abstract      = \"Parallel processing, the method of cutting down a large\r\n                   computational problem into many small tasks which are solved\r\n                   in parallel, is a field of increasing importance in science.\r\n                   Cost-effective, flexible and efficient simulations of\r\n                   mathematical models of physical, chemical or biological\r\n                   real-world problems are replacing the traditional\r\n                   experimental research. Current software solutions for\r\n                   parallel and scientific computation, like Parallel Virtual\r\n                   Machine and Message Passing Interface, have limitations in\r\n                   handling faults and failures, in utilizing heterogeneous and\r\n                   dynamically changing communication structures, and in\r\n                   enabling migrating or cooperative applications. The current\r\n                   research in heterogeneous adaptable reconfigurable networked\r\n                   systems (Harness) aims to produce the next generation of\r\n                   software solutions for distributed computing. A\r\n                   high-available and light-weighted distributed virtual\r\n                   machine service provides an encapsulation of a few hundred\r\n                   to a few thousand physical machines in a virtual\r\n                   heterogeneous large scale cluster. A high availability of\r\n                   a service in distributed systems can be achieved by\r\n                   replication of the service state on multiple server\r\n                   processes. If one ore more server processes fails, the\r\n                   surviving ones continue to provide the service because they\r\n                   know the state. Since every member of a distributed virtual\r\n                   machine is part of the distributed virtual machine service\r\n                   state and is able to change this state, a distributed control\r\n                   is needed to replicate the state and maintain its\r\n                   consistency. This distributed control manages state changes\r\n                   as well as the state-replication and the detection of and\r\n                   recovery from faults and failures of server processes. This\r\n                   work analyzes system architectures currently used in\r\n                   heterogeneous distributed computing by defining terms,\r\n                   conditions and assumptions. It shows that such systems are\r\n                   asynchronous and may use partially synchronous communication\r\n                   to detect and to distinguish different classes of faults and\r\n                   failures. It describes how a high availability of a large\r\n                   scale distributed service on a huge number of servers\r\n                   residing on different geographical locations can be realized.\r\n                   Asynchronous group communication services, such as Reliable\r\n                   Broadcast, Atomic Broadcast, Distributed Agreement and\r\n                   Membership, are analyzed to develop linear scalable\r\n                   algorithms in an unidirectional and in a bidirectional\r\n                   connected asynchronous peer-to-peer ring architecture.\r\n                   A Transaction Control group communication service is\r\n                   introduced as state-replication service. The system analysis\r\n                   distinguishes different types of distributed systems, where\r\n                   active transactions execute state changes using\r\n                   non-replicated data of one or more servers and inactive\r\n                   transactions report state changes using replicated data only.\r\n                   It is applicable for passive fault-tolerant distributed\r\n                   databases as well as for active fault-tolerant distributed\r\n                   control mechanisms. No control token is used and time stamps\r\n                   are avoided, so that all members of a server group have equal\r\n                   responsibilities and are independent from the system time.\r\n                   A prototype which implements the most complicated Transaction\r\n                   Control algorithm is realized due to the complexity of the\r\n                   distributed system and the early development stage of the\r\n                   introduced algorithms. The prototype is used to obtain\r\n                   practical experience with the state-replication algorithm.\"\r\n}<\/pre>\n
<\/a>@techreport{kuchar23system,\r\n  author        = \"Olga A. Kuchar\r\n                   and Swen Boehm\r\n                   and Thomas Naughton\r\n                   and Suhas Somnath\r\n                   and Ben Mintz\r\n                   and Jack Lange\r\n                   and Scott Atchley\r\n                   and Rohit Srivastava\r\n                   and Patrick Widener\",\r\n  title         = \"{INTERSECT} Architecture Specification: {System-of-systems}\r\n                   Architecture (Version 0.9)\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2023\/3168\",\r\n  address       = \"Oak Ridge, TN, USA\",\r\n  month         = sep # \"~30, \",\r\n  year          = \"2023\",\r\n  doi           = \"10.2172\/2333813\"\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/kuchar23system.pdf\",\r\n  abstract      = \"Oak Ridge National Laboratory (ORNL)'s Self-driven\r\n                   Experiments for Science \/ Interconnected Science Ecosystem\r\n                   (INTERSECT) architecture project, titled ``An Open Federated\r\n                   Architecture for the Laboratory of the Future'', creates an\r\n                   open federated hardware\/software architecture for the\r\n                   laboratory of the future using a novel system of systems\r\n                   (SoS) and microservice architecture approach, connecting\r\n                   scientific instruments, robot-controlled laboratories and\r\n                   edge\/center computing\/data resources to enable autonomous\r\n                   experiments, ``self-driving'' laboratories, smart\r\n                   manufacturing, and artificial intelligence (AI)-driven\r\n                   design, discovery and evaluation.\",\r\n  pts           = \"204165\"\r\n}<\/pre>\n
<\/a>@techreport{kuchar22system,\r\n  author        = \"Olga A. Kuchar\r\n                   and Swen Boehm\r\n                   and Thomas Naughton\r\n                   and Suhas Somnath\r\n                   and Ben Mintz\r\n                   and Jack Lange\r\n                   and Scott Atchley\r\n                   and Rohit Srivastava\r\n                   and Patrick Widener\",\r\n  title         = \"INTERSECT Architecture Specification:\r\n                   System-of-systems Architecture (Version 0.5)\",\r\n  institution   = \"Oak Ridge National Laboratory\",\r\n  number        = \"ORNL\/TM-2022\/2717\",\r\n  address       = \"Oak Ridge, TN, USA\",\r\n  month         = sep # \"~30, \",\r\n  year          = \"2022\",\r\n  doi           = \"10.2172\/1968700\",\r\n  url           = \"http:\/\/www.christian-engelmann.info\/publications\/kuchar22system.pdf\",\r\n  abstract      = \"Oak Ridge National Laboratory (ORNL)'s Self-driven\r\n                   Experiments for Science \/ Interconnected Science Ecosystem\r\n                   (INTERSECT) architecture project, titled ``An Open Federated\r\n                   Architecture for the Laboratory of the Future'', creates an\r\n                   open federated hardware\/software architecture for the\r\n                   laboratory of the future using a novel system of systems\r\n                   (SoS) and microservice architecture approach, connecting\r\n                   scientific instruments, robot-controlled laboratories and\r\n                   edge\/center computing\/data resources to enable autonomous\r\n                   experiments, ``self-driving'' laboratories, smart\r\n                   manufacturing, and artificial intelligence (AI)-driven\r\n                   design, discovery and evaluation.\",\r\n  pts           = \"182854\"\r\n}<\/pre>\n","protected":false},"excerpt":{"rendered":"
@article{agullo22resiliency, author = “Emmanuel Agullo and Mirco Altenbernd and Hartwig Anzt and Leonardo Bautista-Gomez and Tommaso Benacchio and Luca Bonaventura and Hans-Joachim Bungartz and Sanjay Chatterjee and Florina M. Ciorba and Nathan DeBardeleben and Daniel Drzisga and Sebastian Eibl and Christian Engelmann and Wilfried N. Gansterer and Luc Giraud and Dominik G{\\”o}ddeke and Marco Heisig… <\/p>\n","protected":false},"author":1,"featured_media":0,"parent":16,"menu_order":9,"comment_status":"closed","ping_status":"closed","template":"","meta":{"neve_meta_sidebar":"","neve_meta_container":"","neve_meta_enable_content_width":"off","neve_meta_content_width":100,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"","footnotes":""},"class_list":["post-55","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/55","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=55"}],"version-history":[{"count":50,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/55\/revisions"}],"predecessor-version":[{"id":1450,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/55\/revisions\/1450"}],"up":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/16"}],"wp:attachment":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=55"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}