Chapter 17: Superfacility: The Convergence of Data, Compute, Networking, Analytics and Software

The Department of Energy’s Office of Science operates experimental and observational facilities such as light sources, telescopes, sequencers, and distributed sensor networks. These facilities have revolutionized our understanding of the physical world, and has also produced an exponentially growing body of data. Across many fields of observational and experimental science, data-rich discovery environments are emerging. Assembling these environments requires computational and networking resources for data processing, transfer, storage, and analysis; scalable and flexible software tools and applications; and highly skilled experts, including mathematicians, engineers, and computational and domain scientists. Our vision for the Superfacility model is to transform science through a network of connected facilities, software, and expertise to enable new modes of discovery. We envision an experimental facility, one or more data management and processing facilities, and the network fabric and software infrastructure to bind them all together, integrated to provide seamless, real-time access to network, data analysis, and data management capabilities transparent to the end user. While progress has been made toward this vision, many challenges remain. This chapter describes our vision and plans for creating a superfacility framework between DOE facilities. It includes a description of use cases, current and next generation infrastructure, as well as software and analysis capabilities.