{"id":368,"date":"2023-02-18T08:00:48","date_gmt":"2023-02-18T08:00:48","guid":{"rendered":"https:\/\/christian-engelmann.de\/?page_id=368"},"modified":"2023-02-19T01:02:56","modified_gmt":"2023-02-19T01:02:56","slug":"2004-06-reliability-availability-and-serviceability-ras-for-terascale-computing","status":"publish","type":"page","link":"https:\/\/www.christian-engelmann.info\/?page_id=368","title":{"rendered":"2004-06: Reliability, Availability, and Serviceability (RAS) for Terascale Computing"},"content":{"rendered":"<p>This project produces proof-of-concept solutions that enables the removal of the numerous single points of failure in large systems while improving scalability and access to systems and data. Our research effort focuses on efficient redundancy strategies for head and service nodes as well as on a distributed storage infrastructure.<\/p>\n<p>We develop replication mechanisms for providing symmetric active\/active high availability for services running on head and service nodes in order to offer the highest level of availability without significantly impacting performance. The targeted prototypes for the batch job management system, TORQUE, and the parallel virtual file system (PVFS) metadata server can offer 99.9997% service uptime using just 3 redundant nodes.<\/p>\n<p>For distributed data storage, the proposed FreeLoader solution is built on a contributed desktop storage substrate. We develop parallel I\/O mechanisms to store\/access data to\/from network workstations as well as caching mechanisms to store more recently used datasets. FreeLoader can offer high retrieval rates for large datasets using novel striping strategies. It also may be utilized as a virtual cache, storing only prefixes of datasets and yet delivering the entire dataset by masking the suffix patching.<\/p>\n<h4>Prominent Solutions<\/h4>\n<ul>\n<li><a href=\"?page_id=457\">Symmetric Active\/Active High Availability for HPC System Services<\/a><\/li>\n<\/ul>\n<h4>Funding Sources<\/h4>\n<ul>\n<li>Laboratory Directed Research and Development, <a href=\"http:\/\/www.ornl.gov\" target=\"www.ornl.gov\" rel=\"noopener\">Oak Ridge National Laboratory<\/a>\n<\/li>\n<\/ul>\n<h4>Participating Institutions<\/h4>\n<ul>\n<li><a href=\"http:\/\/www.ornl.gov\" target=\"www.ornl.gov\" rel=\"noopener\">Oak Ridge National Laboratory<\/a><\/li>\n<li><a href=\"http:\/\/www.ncsu.edu\" target=\"www.ncsu.edu\" rel=\"noopener\">North Carolina State University<\/a><\/li>\n<\/ul>\n<h4>Peer-reviewed Journal Publications<\/h4>\n<ol>\n<li>Xubin (Ben) He, Li Ou, Christian Engelmann, Xin Chen, and Stephen L. Scott. <b>Symmetric Active\/Active Metadata Service for High Availability Parallel File Systems<\/b>. <i><a href=\"http:\/\/www.elsevier.com\/locate\/jpdc\" target=\"www.elsevier.com\/locate\/jpdc\" rel=\"noopener\">Journal of Parallel and Distributed Computing (JPDC)<\/a><\/i>, volume 69, number 12, pages 961-973, December 1, 2009. <a href=\"http:\/\/www.elsevier.com\" target=\"www.elsevier.com\" rel=\"noopener\">Elsevier B.V, Amsterdam, The Netherlands<\/a>. ISSN 0743-7315. DOI <a href=\"http:\/\/dx.doi.org\/10.1016\/j.jpdc.2009.08.004\" target=\"publication\" rel=\"noopener\">10.1016\/j.jpdc.2009.08.004<\/a>. <a href=\"javascript:showAbstract('High availability data storage systems are critical for many applications as research and business become more data-driven. Since metadata management is essential to system availability, multiple metadata services are used to improve the availability of distributed storage systems. Past research focused on the active\/standby model, where each active service has at least one redundant idle backup. However, interruption of service and even some loss of service state may occur during a fail-over depending on the used replication technique. In addition, the replication overhead for multiple metadata services can be very high. The research in this paper targets the symmetric active\/active replication model, which uses multiple redundant service nodes running in virtual synchrony. In this model, service node failures do not cause a fail-over to a backup and there is no disruption of service or loss of service state. We further discuss a fast delivery protocol to reduce the latency of the needed total order broadcast. Our prototype implementation shows that metadata service high availability can be achieved with an acceptable performance trade-off using our symmetric active\/active metadata service solution.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/he09symmetric.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#he09symmetric\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann, Stephen L. Scott, Chokchai (Box) Leangsuksun, and Xubin (Ben) He. <b>Symmetric Active\/Active High Availability for High-Performance Computing System Services<\/b>. <i><a href=\"http:\/\/www.jcomputers.us\" target=\"www.jcomputers.us\" rel=\"noopener\">Journal of Computers (JCP)<\/a><\/i>, volume 1, number 8, pages 43-54, December 1, 2006. <a href=\"http:\/\/www.jcomputers.us\" target=\"www.jcomputers.us\" rel=\"noopener\">Academy Publisher, Oulu, Finland<\/a>. ISSN 1796-203X. DOI <a href=\"http:\/\/dx.doi.org\/10.4304\/jcp.1.8.43-54\" target=\"publication\" rel=\"noopener\">10.4304\/jcp.1.8.43-54<\/a>. <a href=\"javascript:showAbstract('This work aims to pave the way for high availability in high-performance computing (HPC) by focusing on efficient redundancy strategies for head and service nodes. These nodes represent single points of failure and control for an entire HPC system as they render it inaccessible and unmanageable in case of a failure until repair. The presented approach introduces two distinct replication methods, internal and external, for providing symmetric active\/active high availability for multiple redundant head and service nodes running in virtual synchrony utilizing an existing process group communication system for service group membership management and reliable, totally ordered message delivery. Resented results of a prototype implementation that offers symmetric active\/active replication for HPC job and resource management using external replication show that the highest level of availability can be provided with an acceptable performance trade-off.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann06symmetric.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#engelmann06symmetric\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Peer-reviewed Conference Publications<\/h4>\n<ol>\n<li>Christian Engelmann, Stephen L. Scott, Chokchai (Box) Leangsuksun, and Xubin (Ben) He. <b>Symmetric Active\/Active Replication for Dependent Services<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.ares-conference.eu\/ares2008\" target=\"www.ares-conference.eu\/ares2008\" rel=\"noopener\">3rd International Conference on Availability, Reliability and Security (ARES) 2008<\/a><\/i>, pages 260-267, Barcelona, Spain, March 4-7, 2008. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\" rel=\"noopener\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 978-0-7695-3102-1. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/ARES.2008.64\" target=\"publication\" rel=\"noopener\">10.1109\/ARES.2008.64<\/a>. Acceptance rate 21.1% (40\/190). <a href=\"javascript:showAbstract('During the last several years, we have established the symmetric active\/active replication model for service-level high availability and implemented several proof-of-concept prototypes. One major deficiency of our model is its inability to deal with dependent services, since its original architecture is based on the client-service model. This paper extends our model to dependent services using its already existing mechanisms and features. The presented concept is based on the idea that a service may also be a client of another service, and multiple services may be clients of each other. A high-level abstraction is used to illustrate dependencies between clients and services, and to decompose dependencies between services into respective client-service dependencies. This abstraction may be used for providing high availability in distributed computing systems with complex service-oriented architectures.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann08symmetric.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann08symmetric.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann08symmetric\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Li Ou, Christian Engelmann, Xubin (Ben) He, Xin Chen, and Stephen L. Scott. <b>Symmetric Active\/Active Metadata Service for Highly Available Cluster Storage Systems<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.iasted.org\/conferences\/home-590.html\" target=\"www.iasted.org\/conferences\/home-590.html\" rel=\"noopener\">19th IASTED International Conference on Parallel and Distributed Computing and Systems (PDCS) 2007<\/a><\/i>, Cambridge, MA, USA, November 19-21, 2007. <a href=\"http:\/\/www.actapress.com\" target=\"www.actapress.com\" rel=\"noopener\">ACTA Press, Calgary, AB, Canada<\/a>. ISBN 978-0-88986-703-1. Acceptance rate 49%. <a href=\"javascript:showAbstract('In a typical distributed storage system, metadata is stored and managed by dedicated metadata servers. One way to improve the availability of distributed storage systems is to deploy multiple metadata servers. Past research focused on the active\/standby model, where each active server has at least one redundant idle backup. However, interruption of service and loss of service state may occur during a fail-over depending on the used replication technique. The research in this paper targets the symmetric active\/active replication model using multiple redundant service nodes running in virtual synchrony. In this model, service node failures do not cause a fail-over to a backup and there is no disruption of service or loss of service state. We propose a fast delivery protocol to reduce the latency of total order broadcast. Our prototype implementation shows that high availability of metadata servers can be achieved with an acceptable performance trade-off using the active\/active metadata server solution.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/ou07symmetric.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/ou07symmetric.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#ou07symmetric\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann, Stephen L. Scott, Chokchai (Box) Leangsuksun, and Xubin (Ben) He. <b>On Programming Models for Service-Level High Availability<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.ares-conference.eu\/ares2007\" target=\"www.ares-conference.eu\/ares2007\" rel=\"noopener\">2nd International Conference on Availability, Reliability and Security (ARES) 2007<\/a><\/i>, pages 999-1006, Vienna, Austria, April 10-13, 2007. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\" rel=\"noopener\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 0-7695-2775-2. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/ARES.2007.109\" target=\"publication\" rel=\"noopener\">10.1109\/ARES.2007.109<\/a>. Acceptance rate 28.3% (60\/212). <a href=\"javascript:showAbstract('This paper provides an overview of existing programming models for service-level high availability and investigates their differences, similarities, advantages, and disadvantages. Its goal is to help to improve reuse of code and to allow adaptation to quality of service requirements by using a uniform programming model description. It further aims at encouraging a discussion about these programming models and their provided quality of service, such as availability, performance, serviceability, usability, and applicability. Within this context, the presented research focuses on providing high availability for services running on head and service nodes of high-performance computing systems.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann07programming.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann07programming.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann07programming\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Kai Uhlemann, Christian Engelmann, and Stephen L. Scott. <b>JOSHUA: Symmetric Active\/Active Replication for Highly Available HPC Job and Resource Management<\/b>. In <i>Proceedings of the <a href=\"http:\/\/cluster2006.org\" target=\"cluster2006.org\" rel=\"noopener\">8th IEEE International Conference on Cluster Computing (Cluster) 2006<\/a><\/i>, pages 1-10, Barcelona, Spain, September 25-28, 2006. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\" rel=\"noopener\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 1-4244-0328-6. ISSN 1552-5244. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/CLUSTR.2006.311855\" target=\"publication\" rel=\"noopener\">10.1109\/CLUSTR.2006.311855<\/a>. Acceptance rate 33.1% (42\/127). <a href=\"javascript:showAbstract('Most of today`s HPC systems employ a single head node for control, which represents a single point of failure as it interrupts an entire HPC system upon failure. Furthermore, it is also a single point of control as it disables an entire HPC system until repair. One of the most important HPC system service running on the head node is the job and resource management. If it goes down, all currently running jobs loose the service they report back to. They have to be restarted once the head node is up and running again. With this paper, we present a generic approach for providing symmetric active\/active replication for highly available HPC job and resource management. The JOSHUA solution provides a virtually synchronous environment for continuous availability without any interruption of service and without any loss of state. Replication is performed externally via the PBS service interface without the need to modify any service code. Test results as well as availability analysis of our proof-of-concept prototype implementation show that continuous availability can be provided by JOSHUA with an acceptable performance trade-off.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/uhlemann06joshua.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/uhlemann06joshua.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#uhlemann06joshua\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Jyothish Varma, Chao Wang, Frank Mueller, Christian Engelmann, and Stephen L. Scott. <b>Scalable, Fault-Tolerant Membership for MPI Tasks on HPC Systems<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.ics-conference.org\/2006\" target=\"www.ics-conference.org\/2006\" rel=\"noopener\">20th ACM International Conference on Supercomputing (ICS) 2006<\/a><\/i>, pages 219-228, Cairns, Australia, June 28-30, 2006. <a href=\"http:\/\/www.acm.org\" target=\"www.acm.org\" rel=\"noopener\">ACM Press, New York, NY, USA<\/a>. ISBN 1-59593-282-8. DOI <a href=\"http:\/\/dx.doi.org\/10.1145\/1183401.1183433\" target=\"publication\" rel=\"noopener\">10.1145\/1183401.1183433<\/a>. Acceptance rate 26.2% (37\/141). <a href=\"javascript:showAbstract('Reliability is increasingly becoming a challenge for high-performance computing (HPC) systems with thousands of nodes, such as IBM`s Blue Gene\/L. A shorter mean-time-to-failure can be addressed by adding fault tolerance to reconfigure working nodes to ensure that communication and computation can progress. However, existing approaches fall short in providing scalability and small reconfiguration overhead within the fault-tolerant layer. This paper contributes a scalable approach to reconfigure the communication infrastructure after node failures. We propose a decentralized (peer-to-peer) protocol that maintains a consistent view of active nodes in the presence of faults. Our protocol shows response times in the order of hundreds of microseconds and single-digit milliseconds for  reconfiguration using MPI over Blue Gene\/L and TCP over  Gigabit, respectively. The protocol can be adapted to match the network topology to further increase performance. We also verify experimental results against a performance model, which demonstrates the scalability of the approach. Hence, the membership service is suitable for deployment in the communication layer of MPI runtime systems, and we have integrated an early version into LAM\/MPI.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/varma06scalable.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/varma06scalable.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#varma06scalable\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Daniel I. Okunbor, Christian Engelmann, and Stephen L. Scott. <b>Exploring Process Groups for Reliability, Availability and Serviceability of Terascale Computing Systems<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.atiner.gr\/docs\/2006AAAPROGRAM_COMP.htm\" target=\"www.atiner.gr\/docs\/2006AAAPROGRAM_COMP.htm\" rel=\"noopener\">2nd International Conference on Computer Science and Information Systems 2006<\/a><\/i>, Athens, Greece, June 19-21, 2006. <a href=\"javascript:showAbstract('This paper presents various aspects of reliability, availability and serviceability (RAS) systems as they relate to group communication service, including reliable and total order multicast\/broadcast, virtual synchrony, and failure detection. While the issue of availability, particularly high availability using replication-based architectures has recently received upsurge research interests, much still have to be done in understanding the basic underlying concepts for achieving RAS systems, especially in high-end and high performance computing (HPC) communities. Various attributes of group communication service and the prototype of symmetric active replication following ideas utilized in the Newtop protocol will be discussed. We explore the application of group communication service for RAS HPC, laying the groundwork for its integrated model.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/okunbor06exploring.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#okunbor06exploring\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Kshitij Limaye, Chokchai (Box) Leangsuksun, Zeno Greenwood, Stephen L. Scott, Christian Engelmann, Richard M. Libby, and Kasidit Chanchio. <b>Job-Site Level Fault Tolerance for Cluster and Grid Environments<\/b>. In <i>Proceedings of the <a href=\"http:\/\/cluster2005.org\" target=\"cluster2005.org\" rel=\"noopener\">7th IEEE International Conference on Cluster Computing (Cluster) 2005<\/a><\/i>, pages 1-9, Boston, MA, USA, September 26-30, 2005. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\" rel=\"noopener\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 0-7803-9486-0. ISSN 1552-5244. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/CLUSTR.2005.347043\" target=\"publication\" rel=\"noopener\">10.1109\/CLUSTR.2005.347043<\/a>. Acceptance rate 39.6% (45\/138). <a href=\"javascript:showAbstract('In order to adopt high performance clusters and Grid computing for mission critical applications, fault tolerance is a necessity. Common fault tolerance techniques in distributed systems are normally achieved with checkpoint-recovery and job replication on alternative resources, in cases of a system outage. The first approach depends on the system`s MTTR while the latter approach depends on the availability of alternative sites to run replicas. There is a need for complementing these approaches by proactively handling failures at a job-site level, ensuring the system high availability with no loss of user submitted jobs. This paper discusses a novel fault tolerance technique  that enables the job-site recovery in Beowulf cluster-based grid environments, whereas existing techniques give up a failed system by seeking alternative resources. Our results suggest sizable aggregate performance improvement during an implementation of our method in Globus-enabled HA-OSCAR. The technique called Smart Failover provides a transparent and graceful recovery mechanism that saves job states in a local job-manager queue and transfers those states to the backup server periodically, and in critical system events. Thus whenever a failover occurs, the backup server is able to restart the jobs from their last saved state.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/limaye05jobsite.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#limaye05jobsite\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Hertong Song, Chokchai (Box) Leangsuksun, Raja Nassar, Yudan Liu, Christian Engelmann, and Stephen L. Scott. <b>UML-based Beowulf Cluster Availability Modeling<\/b>. In <i><a href=\"http:\/\/www.world-academy-of-science.org\/IMCSE2005\/ws\/SERP\" target=\"www.world-academy-of-science.org\/IMCSE2005\/ws\/SERP\" rel=\"noopener\">International Conference on Software Engineering Research and Practice (SERP) 2005<\/a><\/i>, pages 161-167, Las Vegas, NV, USA, June 27-30, 2005. CSREA Press. ISBN 1-932415-49-1. <a href=\"?page_id=55#song05umlbased\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Peer-reviewed Workshop Publications<\/h4>\n<ol>\n<li>Christian Engelmann, Stephen L. Scott, Chokchai (Box) Leangsuksun, and Xubin (Ben) He. <b>Symmetric Active\/Active High Availability for High-Performance Computing System Services: Accomplishments and Limitations<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.ens-lyon.fr\/LIP\/RESO\/ccgrid2008\" target=\"www.ens-lyon.fr\/LIP\/RESO\/ccgrid2008\" rel=\"noopener\">8th IEEE International Symposium on Cluster Computing and the Grid (CCGrid) 2008<\/a>: <a href=\"http:\/\/xcr.cenit.latech.edu\/resilience2008\" target=\"xcr.cenit.latech.edu\/resilience2008\" rel=\"noopener\">Workshop on Resiliency in High Performance Computing (Resilience) 2008<\/a><\/i>, pages 813-818, Lyon, France, May 19-22, 2008. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\" rel=\"noopener\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 978-0-7695-3156-4. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/CCGRID.2008.78\" target=\"publication\" rel=\"noopener\">10.1109\/CCGRID.2008.78<\/a>. <a href=\"javascript:showAbstract('This paper summarizes our efforts over the last 3-4 years in providing symmetric active\/active high availability for high-performance computing (HPC) system services. This work paves the way for high-level reliability, availability and serviceability in extreme-scale HPC systems by focusing on the most critical components, head and service nodes, and by reinforcing them with appropriate high availability solutions. This paper presents our accomplishments in the form of concepts and respective prototypes, discusses existing limitations, outlines possible future work, and describes the relevance of this research to other, planned efforts.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann08symmetric2.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann08symmetric2.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann08symmetric2\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Xin Chen, Benjamin Eckart, Xubin (Ben) He, Christian Engelmann, and Stephen L. Scott. <b>An Online Controller Towards Self-Adaptive File System Availability and Performance<\/b>. In <i>Proceedings of the <a href=\"http:\/\/xcr.cenit.latech.edu\/hapcw2008\" target=\"xcr.cenit.latech.edu\/hapcw2008\" rel=\"noopener\">5th High Availability and Performance Workshop (HAPCW) 2008<\/a>, in conjunction with the <a href=\"http:\/\/www.hpcsw.org\" target=\"www.hpcsw.org\" rel=\"noopener\">1st High-Performance Computer Science Week (HPCSW) 2008<\/a><\/i>, Denver, CO, USA, April 3-4, 2008. <a href=\"javascript:showAbstract('At the present time, it can be a significant challenge to build a large-scale distributed file system that simultaneously maintains both high availability and high performance. Although many fault tolerance technologies have been proposed and used in both commercial and academic distributed file systems to achieve high availability, most of them typically sacrifice performance for higher system availability. Additionally, recent studies show that system availability and performance are related to the system workload. In this paper, we analyze the correlations among availability, performance, and workloads based on a replication strategy, and we discuss the trade off between availability and performance with different workloads. Our analysis leads to the design of an online controller that can dynamically achieve optimal performance and availability by tuning the system replication policy.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/chen08online.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/chen08online.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#chen08online\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann, Stephen L. Scott, Chokchai (Box) Leangsuksun, and Xubin (Ben) He. <b>Transparent Symmetric Active\/Active Replication for Service-Level High Availability<\/b>. In <i>Proceedings of the <a href=\"http:\/\/ccgrid07.lncc.br\" target=\"ccgrid07.lncc.br\" rel=\"noopener\">7th IEEE International Symposium on Cluster Computing and the Grid (CCGrid) 2007<\/a>: <a href=\"http:\/\/www.lri.fr\/ fedak\/gp2pc-07\" target=\"www.lri.fr\/ fedak\/gp2pc-07\" rel=\"noopener\">7th International Workshop on Global and Peer-to-Peer Computing (GP2PC) 2007<\/a><\/i>, pages 755-760, Rio de Janeiro, Brazil, May 14-17, 2007. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\" rel=\"noopener\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 0-7695-2833-3. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/CCGRID.2007.116\" target=\"publication\" rel=\"noopener\">10.1109\/CCGRID.2007.116<\/a>. <a href=\"javascript:showAbstract('As service-oriented architectures become more important in parallel and distributed computing systems, individual service instance reliability as well as appropriate service redundancy becomes an essential necessity in order to increase overall system availability. This paper focuses on providing redundancy strategies using service-level replication techniques. Based on previous research using symmetric active\/active replication, this paper proposes a transparent symmetric active\/active replication approach that allows for more reuse of code between individual service-level replication implementations by using a virtual communication layer. Service- and client-side interceptors are utilized in order to provide total transparency. Clients and servers are unaware of the replication infrastructure as it provides all necessary mechanisms internally.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann07transparent.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann07transparent.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann07transparent\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann, Stephen L. Scott, Chokchai (Box) Leangsuksun, and Xubin (Ben) He. <b>Towards High Availability for High-Performance Computing System Services: Accomplishments and Limitations<\/b>. In <i>Proceedings of the <a href=\"http:\/\/xcr.cenit.latech.edu\/hapcw2006\" target=\"xcr.cenit.latech.edu\/hapcw2006\" rel=\"noopener\">4th High Availability and Performance Workshop (HAPCW) 2006<\/a>, in conjunction with the <a href=\"http:\/\/lacsi.krellinst.org\" target=\"lacsi.krellinst.org\" rel=\"noopener\">7th Los Alamos Computer Science Institute (LACSI) Symposium 2006<\/a><\/i>, Santa Fe, NM, USA, October 17, 2006. <a href=\"javascript:showAbstract('During the last several years, our teams at Oak Ridge National Laboratory, Louisiana Tech University, and Tennessee Technological University focused on efficient redundancy strategies for head and service nodes of high-performance computing (HPC) systems in order to pave the way for high availability (HA) in HPC. These nodes typically run critical HPC system services, like job and resource management, and represent single points of failure and control for an entire HPC system. The overarching goal of our research is to provide high-level reliability, availability, and serviceability (RAS) for HPC systems by combining HA and HPC technology. This paper summarizes our accomplishments, such as developed concepts and implemented proof-of-concept prototypes, and describes existing limitations, such as performance issues, which need to be dealt with for production-type deployment.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann06towards.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann06towards.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann06towards\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Li Ou, Xin Chen, Xubin (Ben) He, Christian Engelmann, and Stephen L. Scott. <b>Achieving Computational I\/O Effciency in a High Performance Cluster Using Multicore Processors<\/b>. In <i>Proceedings of the <a href=\"http:\/\/xcr.cenit.latech.edu\/hapcw2006\" target=\"xcr.cenit.latech.edu\/hapcw2006\" rel=\"noopener\">4th High Availability and Performance Workshop (HAPCW) 2006<\/a>, in conjunction with the <a href=\"http:\/\/lacsi.krellinst.org\" target=\"lacsi.krellinst.org\" rel=\"noopener\">7th Los Alamos Computer Science Institute (LACSI) Symposium 2006<\/a><\/i>, Santa Fe, NM, USA, October 17, 2006. <a href=\"javascript:showAbstract('Cluster computing has become one of the most popular platforms for high-performance computing today. The recent popularity of multicore processors provides a flexible way to increase the computational capability of clusters. Although the system performance may improve with multicore processors in a cluster, I\/O requests initiated by multiple cores may saturate the I\/O bus, and furthermore increase the latency by issuing  multiple non-contiguous disk accesses. In this paper, we propose an asymmetric collective I\/O for multicore processors to improve multiple non-contiguous accesses. In our configuration, one core in each multicore processor is designated as the coordinator, and others serve as computing cores. The coordinator is responsible for aggregating I\/O operations from computing cores and submitting a contiguous request. The coordinator allocates contiguous memory buffers on behalf of other cores to avoid redundant data copies.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/ou06achieving.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/ou06achieving.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#ou06achieving\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann, Stephen L. Scott, Chokchai (Box) Leangsuksun, and Xubin (Ben) He. <b>Active\/Active Replication for Highly Available HPC System Services<\/b>. In <i>Proceedings of the <a href=\"http:\/\/www.ares-conference.eu\/ares2006\" target=\"www.ares-conference.eu\/ares2006\" rel=\"noopener\">1st International Conference on Availability, Reliability and Security (ARES) 2006<\/a>: 1st International Workshop on Frontiers in Availability, Reliability and Security (FARES) 2006<\/i>, pages 639-645, Vienna, Austria, April 20-22, 2006. <a href=\"http:\/\/www.computer.org\" target=\"www.computer.org\" rel=\"noopener\">IEEE Computer Society, Los Alamitos, CA, USA<\/a>. ISBN 0-7695-2567-9. DOI <a href=\"http:\/\/dx.doi.org\/10.1109\/ARES.2006.23\" target=\"publication\" rel=\"noopener\">10.1109\/ARES.2006.23<\/a>. <a href=\"javascript:showAbstract('Today`s high performance computing systems have several reliability deficiencies resulting in availability and serviceability issues. Head and service nodes represent a single point of failure and control for an entire system as they render it inaccessible and unmanageable in case of a failure until repair, causing a significant downtime. This paper introduces two distinct replication methods (internal and external) for providing symmetric active\/active high availability for multiple head and service nodes running in virtual synchrony. It presents a comparison of both methods in terms of expected correctness, ease-of-use and performance based on early results from ongoing work in providing symmetric active\/active high availability for two HPC system services (TORQUE and PVFS metadata server). It continues with a short description of a distributed mutual exclusion algorithm and a brief statement regarding the handling of Byzantine failures. This paper concludes with an overview of past and ongoing work, and a short summary of the presented research.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann06active.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann06active.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann06active\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann and Stephen L. Scott. <b>Concepts for High Availability in Scientific High-End Computing<\/b>. In <i>Proceedings of the <a href=\"http:\/\/xcr.cenit.latech.edu\/hapcw2005\" target=\"xcr.cenit.latech.edu\/hapcw2005\" rel=\"noopener\">3rd High Availability and Performance Workshop (HAPCW) 2005<\/a>, in conjunction with the <a href=\"http:\/\/lacsi.rice.edu\/symposium\/agenda_2005\" target=\"lacsi.rice.edu\/symposium\/agenda_2005\" rel=\"noopener\">6th Los Alamos Computer Science Institute (LACSI) Symposium 2005<\/a><\/i>, Santa Fe, NM, USA, October 11, 2005. <a href=\"javascript:showAbstract('Scientific high-end computing (HEC) has become an important tool for scientists world-wide to understand problems, such as in nuclear fusion, human genomics and nanotechnology. Every year, new HEC systems emerge on the market with better performance and higher scale. With only very few exceptions, the overall availability of recently installed systems has been lower in comparison to the same deployment phase of their predecessors. In contrast to the experienced loss of availability, the demand for continuous availability has risen dramatically due to the recent trend towards capability computing. In this paper, we analyze the existing deficiencies of current HEC systems and present several high availability concepts to counter the experienced loss of availability and to alleviate the expected impact on next-generation systems. We explain the application of these concepts to current and future HEC systems and list past and ongoing related research. This paper closes with a short summary of the presented work and a brief discussion of future efforts.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann05concepts.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann05concepts.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann05concepts\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann and Stephen L. Scott. <b>High Availability for Ultra-Scale High-End Scientific Computing<\/b>. In <i>Proceedings of the <a href=\"http:\/\/coset.irisa.fr\" target=\"coset.irisa.fr\" rel=\"noopener\">2nd International Workshop on Operating Systems, Programming Environments and Management Tools for High-Performance Computing on Clusters (COSET-2) 2005<\/a>, in conjunction with the <a href=\"http:\/\/ics05.csail.mit.edu\" target=\"ics05.csail.mit.edu\" rel=\"noopener\">19th ACM International Conference on Supercomputing (ICS) 2005<\/a><\/i>, Cambridge, MA, USA, June 19, 2005. <a href=\"javascript:showAbstract('Ultra-scale architectures for scientific high-end computing with tens to hundreds of thousands of processors, such as the IBM Blue Gene\/L and the Cray X1, suffer from availability deficiencies, which impact the efficiency of running computational jobs by forcing frequent checkpointing of applications. Most systems are unable to handle runtime system configuration changes caused by failures and require a complete restart of essential system services, such as the job scheduler or MPI, or even of the entire machine. In this paper, we present a flexible, pluggable and component-based high availability framework that expands today`s effort in high availability computing of keeping a single server alive to include all machines cooperating in a high-end scientific computing environment, while allowing adaptation to system properties and application needs.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann05high.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann05high.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann05high\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Chokchai (Box) Leangsuksun, Venkata K. Munganuru, Tong Liu, Stephen L. Scott, and Christian Engelmann. <b>Asymmetric Active-Active High Availability for High-end Computing<\/b>. In <i>Proceedings of the <a href=\"http:\/\/coset.irisa.fr\" target=\"coset.irisa.fr\" rel=\"noopener\">2nd International Workshop on Operating Systems, Programming Environments and Management Tools for High-Performance Computing on Clusters (COSET-2) 2005<\/a>, in conjunction with the <a href=\"http:\/\/ics05.csail.mit.edu\" target=\"ics05.csail.mit.edu\" rel=\"noopener\">19th ACM International Conference on Supercomputing (ICS) 2005<\/a><\/i>, Cambridge, MA, USA, June 19, 2005. <a href=\"javascript:showAbstract('Linux clusters have become very popular for scientific computing at research institutions world-wide, because they can be easily deployed at a fairly low cost. However, the most pressing issues of today`s cluster solutions are availability and serviceability. The conventional Beowulf cluster architecture has a single head node connected to a group of compute nodes. This head node is a typical single point of failure and control, which severely limits availability and serviceability by effectively cutting off healthy compute nodes from the outside world upon overload or failure. In this paper, we describe a paradigm that addresses this issue using asymmetric active-active high availability. Our framework comprises of n + 1 head nodes, where n head nodes are active in the sense that they provide services to simultaneously incoming user requests. One standby server monitors all active servers and performs a fail-over in case of a detected outage. We present a prototype implementation based on a 2 + 1 solution and discuss initial results.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/leangsuksun05asymmetric.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/leangsuksun05asymmetric.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#leangsuksun05asymmetric\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Xubin (Ben) He, Li Ou, Stephen L. Scott, and Christian Engelmann. <b>A Highly Available Cluster Storage System using Scavenging<\/b>. In <i>Proceedings of the <a href=\"http:\/\/xcr.cenit.latech.edu\/hapcw2004\" target=\"xcr.cenit.latech.edu\/hapcw2004\" rel=\"noopener\">2nd High Availability and Performance Workshop (HAPCW) 2004<\/a>, in conjunction with the <a href=\"http:\/\/lacsi.rice.edu\/symposium\/agenda_2004\" target=\"lacsi.rice.edu\/symposium\/agenda_2004\" rel=\"noopener\">5th Los Alamos Computer Science Institute (LACSI) Symposium 2004<\/a><\/i>, Santa Fe, NM, USA, October 12, 2004. <a href=\"javascript:showAbstract('Highly available data storage for high-performance computing is becoming increasingly more critical as high-end computing systems scale up in size and storage systems are developed around network-centered architectures. A promising solution is to harness the collective storage potential of individual workstations much as we harness idle CPU cycles due to the excellent price\/performance ratio and low storage usage of most commodity workstations. For such a storage system, metadata consistency is a key issue assuring storage system availability as well as data reliability. In this paper, we present a decentralized metadata management scheme that improves storage availability without sacrificing performance.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/he04highly.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/he04highly.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#he04highly\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Peer-reviewed Conference Posters<\/h4>\n<ol>\n<li>Stephen L. Scott, Christian Engelmann, Hong H. Ong, Geoffroy R. Vall&eacute;e, Thomas Naughton, Anand Tikotekar, George Ostrouchov, Chokchai (Box) Leangsuksun, Nichamon Naksinehaboon, Raja Nassar, Mihaela Paun, Frank Mueller, Chao Wang, Arun B. Nagarajan, Jyothish Varma, Xubin (Ben) He, Li Ou, and Xin Chen. <b>Resiliency for High-Performance Computing Systems<\/b>. Poster at the <a href=\"http:\/\/www.hpcsw.org\" target=\"www.hpcsw.org\" rel=\"noopener\">1st High-Performance Computer Science Week (HPCSW) 2008<\/a>, Denver, CO, USA, March 30 &#8211; April 5, 2008. <a href=\"javascript:showAbstract('This poster summarizes our past and ongoing research and development efforts in novel system software solutions for providing high-level reliability, availability and serviceability (RAS) for next-generation extreme-scale high-performance computing (HPC) systems and beyond. The poster showcases results of developed proof-of-concept implementations and performed theoretical analyses, outlines planned research and development activities, and presents respective initial results.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/scott08resiliency.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"?page_id=55#scott08resiliency\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Talks and Lectures<\/h4>\n<ol>\n<li>Christian Engelmann. <b>System Resilience Research at ORNL in the Context of HPC<\/b>. Invited talk at the <a href=\"http:\/\/www.inria.fr\/inria\/organigramme\/fiche_ur-ren.fr.html\" target=\"www.inria.fr\/inria\/organigramme\/fiche_ur-ren.fr.html\" rel=\"noopener\">Institut National de Recherche en Informatique et en Automatique (INRIA)<\/a>, Rennes, France, May 15, 2009. <a href=\"javascript:showAbstract('The continuing growth in high performance computing (HPC) system scale poses a challenge for system software and scientific applications with respect to reliability, availability and serviceability (RAS). With only very few exceptions, the availability of recently installed systems has been lower in comparison to the same deployment phase of their predecessors. As a result, sites lower allowable job run times in order to force applications to store intermediate results (checkpoints) as insurance against lost computation time. However, checkpoints themselves waste valuable computation time and resources. In contrast to the experienced loss of availability, the demand for continuous availability has risen dramatically with the trend towards capability computing, which drives the race for scientific discovery by running applications on the fastest machines available while desiring significant amounts of time (weeks and months) without interruption. These machines must be able to run in the event of frequent interrupts in such a manner that the capability is not severely degraded. Thus, research and development of scalable RAS technologies is paramount to the success of future extreme-scale systems. This talk summarizes our accomplishments in the area of high-level RAS for HPC, such as developed concepts and implemented proof-of-concept prototypes.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann09system.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann09system\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>High-Performance Computing Research at Oak Ridge National Laboratory<\/b>. Invited talk at the Reading Annual Computational Science  Workshop, Reading, United Kingdom, December 8, 2008. <a href=\"javascript:showAbstract('Oak Ridge National Laboratory (ORNL) is the largest energy laboratory in the United States. Its National Center for Computational Sciences (NCCS) provides the most powerful computing resources in the world for open scientific research. Jaguar, a Cray XT5 system at NCCS, is the second HPC system to exceed 1 PFlop\/s (10^15 Floating Point Operations Per Second), and the fastest open science supercomputer in the world. It recently ranked #2 in the Top 500 List of Supercomputer Sites with a maximal LINPACK benchmark performance of 1.059 PFlop\/s and a theoretical peak performance of 1.3814 PFlop\/s. Annually, 80 percent of Jaguar\u2019s resources are allocated through the U.S Department of Energy\u2019s Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program, a competitively selected, peer reviewed process open to researchers from universities, industry, government and non-profit organizations. These allocations address some of the most challenging scientific problems in areas such as climate modeling, renewable energy, materials science, fusion and combustion. In conjunction with NCCS, the Computer Science and Mathematics Division at ORNL performs basic and applied research in HPC, mathematics, and intelligent systems. This talk gives a summary of the HPC research performed at ORNL. It provides details about the Jaguar peta-scale computing resource, an overview of the computational science research carried out using ORNL\u2019s computing resources, and a description of various computer science efforts targeting solutions for next-generation HPC systems.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann08high.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann08high\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Modular Redundancy in HPC Systems: Why, Where, When and How?<\/b>. Invited talk at the 1st HPC Resiliency Summit: Workshop on Resiliency for Petascale HPC 2008, in conjunction with the <a href=\"http:\/\/www.lanl.gov\/conferences\/lacss\/2008\" target=\"www.lanl.gov\/conferences\/lacss\/2008\" rel=\"noopener\">1st Los Alamos Computer Science Symposium (LACSS) 2008<\/a>, Santa Fe, NM, USA, October 15, 2008. <a href=\"javascript:showAbstract('The continuing growth in high-performance computing (HPC) system scale poses a challenge for system software and scientific applications with respect to reliability, availability and serviceability (RAS). With only very few exceptions, the availability of recently installed systems has been lower in comparison to the same deployment phase of their predecessors. As a result, sites lower allowable job run times in order to force applications to store intermediate results (checkpoints) as insurance against lost computation time. However, checkpoints themselves waste valuable computation time and resources. In contrast to the experienced loss of availability, the demand for continuous availability has risen dramatically with the trend towards capability computing, which drives the race for scientific discovery by running applications on the fastest machines available while desiring significant amounts of time (weeks and months) without interruption. These machines must be able to run in the event of frequent interrupts in such a manner that the capability is not severely degraded. Thus, research and development of scalable RAS technologies is paramount to the success of future extreme-scale systems. This talk summarizes our past accomplishments, ongoing work, and future plans in the area of high-level RAS for HPC.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann08modular.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann08modular\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Resiliency for High-Performance Computing<\/b>. Invited talk at the <a href=\"http:\/\/acet.rdg.ac.uk\/events\/details\/cancun.php\" target=\"acet.rdg.ac.uk\/events\/details\/cancun.php\" rel=\"noopener\">2nd Collaborative and Grid Computing Technologies Workshop (CGCTW) 2008<\/a>, Cancun, Mexico, April 10-12, 2008. <a href=\"javascript:showAbstract('In order to address anticipated high failure rates, resiliency characteristics have become an urgent priority for next-generation high-performance computing (HPC) systems. One major source of concern are non-recoverable soft errors, i.e., bit flips in memory, cache, registers, and logic. The probability of such errors not only grows with system size, but also with increasing architectural vulnerability caused by employing accelerators and by shrinking nanometer technology. Reactive fault tolerance technologies, such as checkpoint\/restart, are unable to handle high failure rates due to associated overheads, while proactive resiliency technologies, such as preemptive migration, simply fail as random soft errors can&amp;#39;t be predicted. This talk proposes a new, bold direction in resiliency for HPC as it targets resiliency for next-generation extreme-scale HPC systems at the system software level through computational redundancy strategies, i.e., dual- and triple-modular redundancy.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann08resiliency.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann08resiliency\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Advanced Fault Tolerance Solutions for High Performance Computing<\/b>. Seminar at the <a href=\"http:\/\/www.laas.fr\" target=\"www.laas.fr\" rel=\"noopener\">Laboratoire d&#39;Analyse et d&#8217;Architecture des Syst&eacute;mes<\/a>, <a href=\"http:\/\/www.cnrs.fr\" target=\"www.cnrs.fr\" rel=\"noopener\">Centre National de la Recherche Scientifique<\/a>, Toulouse, France, February 11, 2008. <a href=\"javascript:showAbstract('The continuing growth in high performance computing (HPC) system scale poses a challenge for system software and scientific applications with respect to reliability, availability and serviceability (RAS). With only very few exceptions, the availability of recently installed systems has been lower in comparison to the same deployment phase of their predecessors. As a result, sites lower allowable job run times in order to force applications to store intermediate results (checkpoints) as insurance against lost computation time. However, checkpoints themselves waste valuable computation time and resources. In contrast to the experienced loss of availability, the demand for continuous availability has risen dramatically with the trend towards capability computing, which drives the race for scientific discovery by running applications on the fastest machines available while desiring significant amounts of time (weeks and months) without interruption. These machines must be able to run in the event of frequent interrupts in such a manner that the capability is not severely degraded. Thus, research and development of scalable RAS technologies is paramount to the success of future extreme-scale systems. This talk summarizes our accomplishments in the area of high-level RAS for HPC, such as developed concepts and implemented proof-of-concept prototypes, and describes existing limitations, such as performance issues, which need to be dealt with for production-type deployment.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann08advanced.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann08advanced\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Service-Level High Availability in Parallel and Distributed Systems<\/b>. Seminar at the <a href=\"http:\/\/www.cs.reading.ac.uk\" target=\"www.cs.reading.ac.uk\" rel=\"noopener\">Department of Computer Science<\/a>, <a href=\"http:\/\/www.reading.ac.uk\" target=\"www.reading.ac.uk\" rel=\"noopener\">University of Reading<\/a>, Reading, United Kingdom, October 10, 2007. <a href=\"javascript:showAbstract('As service-oriented architectures become more important in parallel and distributed computing systems, individual service instance reliability as well as appropriate service redundancy are essential to increase overall system availability. This talk focuses on redundancy strategies using service-level replication techniques. An overview of existing programming models for service-level high availability is presented and their differences, similarities, advantages, and disadvantages are discussed. Recent advances in providing service-level symmetric active\/active high availability are discussed. While the primary target of the presented research is high availability for service nodes in tightly-coupled extreme-scale high-performance computing (HPC) systems, it is also applicable to loosely-coupled distributed computing scenarios.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann07service.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann07service\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Advanced Fault Tolerance Solutions for High Performance Computing<\/b>. Invited talk at the <a href=\"http:\/\/www.thaigrid.or.th\/wttc2007\" target=\"www.thaigrid.or.th\/wttc2007\" rel=\"noopener\">Workshop on Trends, Technologies and Collaborative Opportunities in High Performance and Grid Computing (WTTC) 2007<\/a>, Khon Kean, Thailand, June 8, 2007. <a href=\"javascript:showAbstract('The continuing growth in high performance computing (HPC) system scale poses a challenge for system software and scientific applications with respect to reliability, availability and serviceability (RAS). With only very few exceptions, the availability of recently installed systems has been lower in comparison to the same deployment phase of their predecessors. As a result, sites lower allowable job run times in order to force applications to store intermediate results (checkpoints) as insurance against lost computation time. However, checkpoints themselves waste valuable computation time and resources. In contrast to the experienced loss of availability, the demand for continuous availability has risen dramatically with the trend towards capability computing, which drives the race for scientific discovery by running applications on the fastest machines available while desiring significant amounts of time (weeks and months) without interruption. These machines must be able to run in the event of frequent interrupts in such a manner that the capability is not severely degraded. Thus, research and development of scalable RAS technologies is paramount to the success of future extreme-scale systems. This talk summarizes our accomplishments in the area of high-level RAS for HPC, such as developed concepts and implemented proof-of-concept prototypes, and describes existing limitations, such as performance issues, which need to be dealt with for production-type deployment.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann07advanced2.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann07advanced2\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Advanced Fault Tolerance Solutions for High Performance Computing<\/b>. Invited talk at the <a href=\"http:\/\/www.thaigrid.or.th\/wttc2007\" target=\"www.thaigrid.or.th\/wttc2007\" rel=\"noopener\">Workshop on Trends, Technologies and Collaborative Opportunities in High Performance and Grid Computing (WTTC) 2007<\/a>, Bangkok, Thailand, June 4-5, 2007. <a href=\"javascript:showAbstract('The continuing growth in high performance computing (HPC) system scale poses a challenge for system software and scientific applications with respect to reliability, availability and serviceability (RAS). With only very few exceptions, the availability of recently installed systems has been lower in comparison to the same deployment phase of their predecessors. As a result, sites lower allowable job run times in order to force applications to store intermediate results (checkpoints) as insurance against lost computation time. However, checkpoints themselves waste valuable computation time and resources. In contrast to the experienced loss of availability, the demand for continuous availability has risen dramatically with the trend towards capability computing, which drives the race for scientific discovery by running applications on the fastest machines available while desiring significant amounts of time (weeks and months) without interruption. These machines must be able to run in the event of frequent interrupts in such a manner that the capability is not severely degraded. Thus, research and development of scalable RAS technologies is paramount to the success of future extreme-scale systems. This talk summarizes our accomplishments in the area of high-level RAS for HPC, such as developed concepts and implemented proof-of-concept prototypes, and describes existing limitations, such as performance issues, which need to be dealt with for production-type deployment.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann07advanced.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann07advanced\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>Towards High Availability for High-Performance Computing System Services: Accomplishments and Limitations<\/b>. Seminar at the <a href=\"http:\/\/www.cs.reading.ac.uk\" target=\"www.cs.reading.ac.uk\" rel=\"noopener\">Department of Computer Science<\/a>, <a href=\"http:\/\/www.reading.ac.uk\" target=\"www.reading.ac.uk\" rel=\"noopener\">University of Reading<\/a>, Reading, United Kingdom, March 14, 2007. <a href=\"javascript:showAbstract('During the last several years, our teams at Oak Ridge National Laboratory, Louisiana Tech University, and Tennessee Technological University focused on efficient redundancy strategies for head and service nodes of high-performance computing (HPC) systems in order to pave the way for high availability (HA) in HPC. These nodes typically run critical HPC system services, like job and resource management, and represent single points of failure and control for an entire HPC system. The overarching goal of our research is to provide high-level reliability, availability, and serviceability (RAS) for HPC systems by combining HA and HPC technology. This talk summarizes our accomplishments, such as developed concepts and implemented proof-of-concept prototypes, and describes existing limitations, such as performance issues, which need to be dealt with for production-type deployment.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann07towards.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann07towards\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>High Availability for Ultra-Scale High-End Scientific Computing<\/b>. Seminar at the <a href=\"http:\/\/www.cs.reading.ac.uk\" target=\"www.cs.reading.ac.uk\" rel=\"noopener\">Department of Computer Science<\/a>, <a href=\"http:\/\/www.reading.ac.uk\" target=\"www.reading.ac.uk\" rel=\"noopener\">University of Reading<\/a>, Reading, United Kingdom, June 9, 2006. <a href=\"javascript:showAbstract('A major concern in exploiting ultra-scale architectures for scientific high-end computing (HEC) with tens to hundreds of thousands of processors, such as the IBM Blue Gene\/L and the Cray X1, is the potential inability to identify problems and take preemptive action before a failure impacts a running job. In fact, in systems of this scale, predictions estimate the mean time to interrupt in terms of hours. Current solutions for fault-tolerance in HEC focus on dealing with the result of a failure. However, most are unable to handle runtime system configuration changes caused by failures and require a complete restart of essential system services (e.g. MPI) or even of the entire machine. High availability (HA) computing strives to avoid the problems of unexpected failures through preemptive measures. There are various techniques to implement high availability. In contrast to active\/hot-standby high availability with its fail-over model, active\/active high availability with its virtual synchrony model is superior in many areas including scalability, throughput, availability and responsiveness. However, it is significantly more complex. The overall goal of our research is to expand today`s effort in HA for HEC, so that systems that have the ability to hot-swap hardware components can be kept alive by an OS runtime environment that understands the concept of dynamic system configuration. This talk will present an overview of recent research at Oak Ridge National Laboratory in high availability solutions for ultra-scale scientific high-end computing.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann06high.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann06high\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Stephen L. Scott and Christian Engelmann. <b>Advancing Reliability, Availability and Serviceability for High-Performance Computing<\/b>. Seminar at the <a href=\"http:\/\/www.gup.uni-linz.ac.at\" target=\"www.gup.uni-linz.ac.at\" rel=\"noopener\">Institute of Graphics and Parallel Processing<\/a>, <a href=\"http:\/\/www.uni-linz.ac.at\" target=\"www.uni-linz.ac.at\" rel=\"noopener\">Johannes Kepler University<\/a>, Linz, Austria, April 19, 2006. <a href=\"javascript:showAbstract('Today\u2019s high performance computing systems have several reliability deficiencies resulting in noticeable availability and serviceability issues. For example, head and service nodes represent a single point of failure and control for an entire system as they render it inaccessible and unmanageable in case of a failure until repair, causing a significant downtime. Furthermore, current solutions for fault-tolerance focus on dealing with the result of a failure. However, most are unable to transparently mask runtime system configuration changes caused by failures and require a complete restart of essential system services, such as MPI, in case of a failure. High availability computing strives to avoid the problems of unexpected failures through preemptive measures. The overall goal of our research is to expand today\u2019s effort in high availability for high-performance computing, so that systems can be kept alive by an OS runtime environment that understands the concepts of dynamic system configuration and degraded operation mode. This talk will present an overview of recent research performed at Oak Ridge National Laboratory in collaboration with Louisiana Tech University, North Carolina State University and the University of Reading in developing core technologies and proof-of-concept prototypes that improve the overall reliability, availability and serviceability of high-performance computing systems.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/scott06advancing.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#scott06advancing\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>High Availability for Ultra-Scale High-End Scientific Computing<\/b>. Seminar at the <a href=\"http:\/\/www.cs.reading.ac.uk\" target=\"www.cs.reading.ac.uk\" rel=\"noopener\">Department of Computer Science<\/a>, <a href=\"http:\/\/www.reading.ac.uk\" target=\"www.reading.ac.uk\" rel=\"noopener\">University of Reading<\/a>, Reading, United Kingdom, October 18, 2005. <a href=\"javascript:showAbstract('A major concern in exploiting ultra-scale architectures for scientific high-end computing (HEC) with tens to hundreds of thousands of processors, such as the IBM Blue Gene\/L and the Cray X1, is the potential inability to identify problems and take preemptive action before a failure impacts a running job. In fact, in systems of this scale, predictions estimate the mean time to interrupt in terms of hours. Current solutions for fault-tolerance in HEC focus on dealing with the result of a failure. However, most are unable to handle runtime system configuration changes caused by failures and require a complete restart of essential system services (e.g. MPI) or even of the entire machine. High availability (HA) computing strives to avoid the problems of unexpected failures through preemptive measures. There are various techniques to implement high availability. In contrast to active\/hot-standby high availability with its fail-over model, active\/active high availability with its virtual synchrony model is superior in many areas including scalability, throughput, availability and responsiveness. However, it is significantly more complex. The overall goal of our research is to expand today`s effort in HA for HEC, so that systems that have the ability to hot-swap hardware components can be kept alive by an OS runtime environment that understands the concept of dynamic system configuration. This talk will present an overview of recent research at Oak Ridge National Laboratory in high availability solutions for ultra-scale scientific high-end computing.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann05high4.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann05high4\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>High Availability for Ultra-Scale High-End Scientific Computing<\/b>. Seminar at the <a href=\"http:\/\/www.uncfsu.edu\/macsc\" target=\"www.uncfsu.edu\/macsc\" rel=\"noopener\">Department of Mathematics and Computer Science<\/a>, <a href=\"http:\/\/www.uncfsu.edu\" target=\"www.uncfsu.edu\" rel=\"noopener\">Fayetteville State University<\/a>, Fayetteville, NC, USA, September 26, 2005. <a href=\"javascript:showAbstract('A major concern in exploiting ultra-scale architectures for scientific high-end computing (HEC) with tens to hundreds of thousands of processors, such as the IBM Blue Gene\/L and the Cray X1, is the potential inability to identify problems and take preemptive action before a failure impacts a running job. In fact, in systems of this scale, predictions estimate the mean time to interrupt in terms of hours. Current solutions for fault-tolerance in HEC focus on dealing with the result of a failure. However, most are unable to handle runtime system configuration changes caused by failures and require a complete restart of essential system services (e.g. MPI) or even of the entire machine. High availability (HA) computing strives to avoid the problems of unexpected failures through preemptive measures. There are various techniques to implement high availability. In contrast to active\/hot-standby high availability with its fail-over model, active\/active high availability with its virtual synchrony model is superior in many areas including scalability, throughput, availability and responsiveness. However, it is significantly more complex. The overall goal of our research is to expand today\u2019s effort in HA for HEC, so that systems that have the ability to hot-swap hardware components can be kept alive by an OS runtime environment that understands the concept of dynamic system configuration. This talk will present an overview of recent research at Oak Ridge National Laboratory in fault tolerance and high availability solutions for ultra-scale scientific high-end computing.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann05high3.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann05high3\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>High Availability for Ultra-Scale High-End Scientific Computing<\/b>. Seminar at the <a href=\"http:\/\/www.cs.reading.ac.uk\" target=\"www.cs.reading.ac.uk\" rel=\"noopener\">Department of Computer Science<\/a>, <a href=\"http:\/\/www.reading.ac.uk\" target=\"www.reading.ac.uk\" rel=\"noopener\">University of Reading<\/a>, Reading, United Kingdom, May 13, 2005. <a href=\"javascript:showAbstract('A major concern in exploiting ultra-scale architectures for scientific high-end computing (HEC) with tens to hundreds of thousands of processors, such as the IBM Blue Gene\/L and the Cray X1, is the potential inability to identify problems and take preemptive action before a failure impacts a running job. In fact, in systems of this scale, predictions estimate the mean time to interrupt in terms of hours. Current solutions for fault-tolerance in HEC focus on dealing with the result of a failure. However, most are unable to handle runtime system configuration changes caused by failures and require a complete restart of essential system services (e.g. MPI) or even of the entire machine. High availability (HA) computing strives to avoid the problems of unexpected failures through preemptive measures. There are various techniques to implement high availability. In contrast to active\/hot-standby high availability with its fail-over model, active\/active high availability with its virtual synchrony model is superior in many areas including scalability, throughput, availability and responsiveness. However, it is significantly more complex. The overall goal of our research is to expand today\u2019s effort in HA for HEC, so that systems that have the ability to hot-swap hardware components can be kept alive by an OS runtime environment that understands the concept of dynamic system configuration. This talk will present an overview of recent research at Oak Ridge National Laboratory in fault-tolerant heterogeneous metacomputing, advanced super-scalable algorithms and high availability system software for ultra-scale scientific high-end computing.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann05high2.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann05high2\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Christian Engelmann. <b>High Availability for Ultra-Scale High-End Scientific Computing<\/b>. Seminar at the <a href=\"http:\/\/cenit.latech.edu\" target=\"cenit.latech.edu\" rel=\"noopener\">Center for Entrepreneurship and Information Technology<\/a>, <a href=\"http:\/\/www.latech.edu\" target=\"www.latech.edu\" rel=\"noopener\">Louisiana Tech University<\/a>, Ruston, LA, USA, April 15, 2005. <a href=\"javascript:showAbstract('A major concern in exploiting ultra-scale architectures for scientific high-end computing (HEC) with tens to hundreds of thousands of processors is the potential inability to identify problems and take preemptive action before a failure impacts a running job. In fact, in systems of this scale, predictions estimate the mean time to interrupt in terms of hours. Current solutions for fault-tolerance in HEC focus on dealing with the result of a failure. However, most are unable to handle runtime system configuration changes caused by failures and require a complete restart of essential system services (e.g. MPI) or even of the entire machine. High availability (HA) computing strives to avoid the problems of unexpected failures through preemptive measures. There are various techniques to implement high availability. In contrast to active\/hot-standby high availability with its fail-over model, active\/active high availability with its virtual synchrony model is superior in many areas including scalability, throughput, availability and responsiveness. However, it is significantly more complex. The overall goal of this research is to expand today\u2019s effort in HA for HEC, so that systems that have the ability to hot-swap hardware components can be kept alive by an OS runtime environment that understands the concept of dynamic system configuration. With the aim of addressing the future challenges of high availability in ultra-scale HEC, this project intends to develop a proof-of-concept implementation of an active\/active high availability system software framework.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann05high1.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann05high1\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Co-advised Theses<\/h4>\n<ol>\n<li>Matthias Weber. <b>High Availability for the Lustre File System<\/b>. Master&#8217;s thesis, <a href=\"http:\/\/www.cs.reading.ac.uk\" target=\"www.cs.reading.ac.uk\" rel=\"noopener\">Department of Computer Science<\/a>, <a href=\"http:\/\/www.reading.ac.uk\" target=\"www.reading.ac.uk\" rel=\"noopener\">University of Reading<\/a>, UK, March 14, 2007. Thesis research performed at Oak Ridge National Laboratory. Double diploma in conjunction with the <a href=\"http:\/\/www.f1.fhtw-berlin.de\" target=\"www.f1.fhtw-berlin.de\" rel=\"noopener\">Department of Engineering I<\/a>, <a href=\"http:\/\/www.f1.fhtw-berlin.de\" target=\"www.f1.fhtw-berlin.de\" rel=\"noopener\">Technical College for Engineering and Economics (FHTW) Berlin<\/a>, Germany. Advisors: Prof. Vassil N. Alexandrov (University of Reading); Christian Engelmann (Oak Ridge National Laboratory). <a href=\"javascript:showAbstract('With the growing importance of high performance computing and, more importantly, the fast growing size of sophisticated high performance computing systems, research in the area of high availability is essential to meet the needs to sustain the current growth. This Master thesis project aims to improve the availability of Lustre. Major concern of this project is the metadata server of the file system. The metadata server of Lustre suffers from the last single point of failure in the file system. To overcome this single point of failure an active\/active high availability approach is introduced. The new file system design with multiple MDS nodes running in virtual synchrony leads to a significant increase of availability. Two prototype implementations aim to show how the proposed system design and its new realized form of symmetric active\/active high availability can be accomplished in practice. The results of this work point out the difficulties in adapting the file system to the active\/active high availability design. Tests identify not achieved functionality and show performance problems of the proposed solution. The findings of this dissertation may be used for further work on high availability for distributed file systems.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/weber07high.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/weber07high.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#weber07high\" ><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<li>Kai Uhlemann. <b>High Availability for High-End Scientific Computing<\/b>. Master&#8217;s thesis, <a href=\"http:\/\/www.cs.reading.ac.uk\" target=\"www.cs.reading.ac.uk\" rel=\"noopener\">Department of Computer Science<\/a>, <a href=\"http:\/\/www.reading.ac.uk\" target=\"www.reading.ac.uk\" rel=\"noopener\">University of Reading<\/a>, UK, March 6, 2006. Thesis research performed at Oak Ridge National Laboratory. Double diploma in conjunction with the <a href=\"http:\/\/www.f1.fhtw-berlin.de\" target=\"www.f1.fhtw-berlin.de\" rel=\"noopener\">Department of Engineering I<\/a>, <a href=\"http:\/\/www.f1.fhtw-berlin.de\" target=\"www.f1.fhtw-berlin.de\" rel=\"noopener\">Technical College for Engineering and Economics (FHTW) Berlin<\/a>, Germany. Advisors: Prof. Vassil N. Alexandrov (University of Reading); George A. (Al) Geist and  Christian Engelmann (Oak Ridge National Laboratory). <a href=\"javascript:showAbstract('With the growing interest and popularity in high performance cluster computing and, more importantly, the fast growing size of compute clusters, research in the area of high availability is essential to meet the needs to sustain the current growth. This Master thesis project introduces a new approach for high availability focusing on the head node of a cluster system. This projects focus is on providing high availability to the job scheduler service, which is the most vital part of the traditional Beowulf-style cluster architecture. This research seeks to add high availability to the job scheduler service and resource management system, typically running on the head node, leading to a significant increase of availability for cluster computing. Also, this software project takes advantage of the virtual synchrony paradigm to achieve active\/active replication, the highest form of high availability. A proof-of-concept implementation shows how high availability can be designed in software and what results can be expected of such a system. The results may be reused for future or existing projects to further improve and extent the high availability of compute clusters.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/uhlemann06high.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/uhlemann06high.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#uhlemann06high\" ><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<h4>Theses<\/h4>\n<ol>\n<li>Christian Engelmann. <b>Symmetric Active\/Active High Availability for High-Performance Computing System Services<\/b>. PhD thesis, <a href=\"http:\/\/www.cs.reading.ac.uk\" target=\"www.cs.reading.ac.uk\" rel=\"noopener\">Department of Computer Science<\/a>, <a href=\"http:\/\/www.reading.ac.uk\" target=\"www.reading.ac.uk\" rel=\"noopener\">University of Reading<\/a>, UK, December 8, 2008. Thesis research performed at Oak Ridge National Laboratory. Advisor: Prof. Vassil N. Alexandrov (University of Reading). <a href=\"javascript:showAbstract('In order to address anticipated high failure rates, reliability, availability and serviceability have become an urgent priority for next-generation high-performance computing (HPC) systems. This thesis aims to pave the way for highly available HPC systems by focusing on their most critical components and by reinforcing them with appropriate high availability solutions. Service components, such as head and service nodes, are the Achilles heel of a HPC system. A failure typically results in a complete system-wide outage. This thesis targets efficient software state replication mechanisms for service component redundancy to achieve high availability as well as high performance. Its methodology relies on defining a modern theoretical foundation for providing service-level high availability, identifying availability deficiencies of HPC systems, and comparing various service-level high availability methods. This thesis showcases several developed proof-of-concept prototypes providing high availability for services running on HPC head and service nodes using the symmetric active\/active replication method, i.e., state-machine replication, to complement prior work in this area using active\/standby and asymmetric active\/active configurations. Presented contributions include a generic taxonomy for service high availability, an insight into availability deficiencies of HPC systems, and a unified definition of service-level high availability methods. Further contributions encompass a fully functional symmetric active\/active high availability prototype for a HPC job and resource management service that does not require modification of service, a fully functional symmetric active\/active high availability prototype for a HPC parallel file system metadata service that offers high performance, and two preliminary prototypes for a transparent symmetric active\/active replication software framework for client-service and dependent service scenarios that hide the replication infrastructure from clients and services. Assuming a mean-time to failure of 5,000 hours for a head or service node, all presented prototypes improve service availability from 99.285% to 99.995% in a two-node system, and to 99.99996% with three nodes.');\"><img decoding=\"async\" src=\"images\/txt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Abstract\"><\/a> <a href=\"publications\/engelmann08symmetric3.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/pdf.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Publication\"><\/a> <a href=\"publications\/engelmann08symmetric3.ppt.pdf\" target=\"publication\" rel=\"noopener\"><img decoding=\"async\" src=\"images\/ppt.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"Presentation\"><\/a> <a href=\"?page_id=55#engelmann08symmetric3\"><img decoding=\"async\" src=\"images\/bib.gif\" border=\"0\" style=\"border-style:none\" height=\"10pt\" alt=\"BibTeX Citation\"><\/a><\/li>\n<\/ol>\n<p><em><small>Symbols: <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/txt.gif\" border=\"0\" alt=\"Abstract\" height=\"10pt\"> Abstract, <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/pdf.gif\" border=\"0\" alt=\"Publication\" height=\"10pt\"> Publication, <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/ppt.gif\" border=\"0\" alt=\"Presentation\" height=\"10pt\"> Presentation, <img decoding=\"async\" style=\"border-style: none;\" src=\"images\/bib.gif\" border=\"0\" alt=\"BibTeX Citation\" height=\"10pt\"> BibTeX Citation<\/small><\/em><\/p>\n<p><script language=\"JavaScript\">\nfunction showAbstract (text) {\n  var width  = 400;\n  var height = 400;\n  var left   = (screen.width  - width ) \/ 2;\n  var top    = (screen.height - height) \/ 2;\n  var win    = window.open('',\n                           'Abstract',\n                           'width='  + width  + ', ' + \n                           'height=' + height + ', ' +\n                           'left='   + left   + ', ' +\n                           'top='    + top    + ', ' +\n                           'toolbar=no, '     +\n                           'location=no, '    +\n                           'directories=no, ' +\n                           'status=no, '      +\n                           'menubar=no, '     +\n                           'copyhistory=no, ' +\n                           'scrollbars=yes, ' +\n                           'resizable=yes')\n  win.document.write(text);\n  win.document.close();\n}\n<\/script><\/p>\n","protected":false},"excerpt":{"rendered":"<p>This project produces proof-of-concept solutions that enables the removal of the numerous single points of failure in large systems while improving scalability and access to systems and data. Our research effort focuses on efficient redundancy strategies for head and service nodes as well as on a distributed storage infrastructure. We develop replication mechanisms for providing&hellip;&nbsp;<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":145,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"neve_meta_sidebar":"","neve_meta_container":"","neve_meta_enable_content_width":"off","neve_meta_content_width":100,"neve_meta_title_alignment":"","neve_meta_author_avatar":"","neve_post_elements_order":"","neve_meta_disable_header":"","neve_meta_disable_footer":"","neve_meta_disable_title":"","footnotes":""},"class_list":["post-368","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/368","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=368"}],"version-history":[{"count":7,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/368\/revisions"}],"predecessor-version":[{"id":500,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/368\/revisions\/500"}],"up":[{"embeddable":true,"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=\/wp\/v2\/pages\/145"}],"wp:attachment":[{"href":"https:\/\/www.christian-engelmann.info\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=368"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}