N-DISE: NDN-Based Data Distribution for Large-Scale Data-Intensive Science

Published

September 6, 2022

Author(s)

Edmund Yeh, Harvey Newman, Lixia Zhang, Jason Cong, Susmit Shannigrahi, Yuanhao Wu, Catalin Iordache, Volkan Mutlu, Sankalpa Timilsina, Sichen Song, Michael Lo, Ran Liu, Chengyu Fan, Raimondas Sirvinskas, Justas Balcas, Yuezhou Liu, Davide Pesavento, Junxiao Shi, Lotfi Benmohamed

Abstract

To meet unprecedented challenges faced by the world's largest data- and network-intensive science programs, we design and implement a new, highly efficient and field-tested data distribution, caching, access and analysis system for the Large Hadron Collider (LHC) high energy physics (HEP) network and other major science programs. We develop a hierarchical Named Data Networking (NDN) naming scheme for HEP data, implement new consumer and producer applications to interface with the high-performance NDN-DPDK forwarder, and build on recently developed high-throughput NDN caching and forwarding methods. We integrate NDN systems concepts and algorithms with the mainstream data distribution, processing, and management system of the Compact Muon Solenoid (CMS) experiment. We design and prototype stable, high-performance virtual LANs (VLANs) over a continental-scale wide area network testbed. In extensive experiments, our proposed integrated system, named NDN for Data-Intensive Science Experiments (N-DISE), is shown to deliver LHC data over the wide area network (WAN) testbed at throughputs exceeding 31 Gbps between Caltech and StarLight, with dramatically reduced download time.

Proceedings Title

Proceedings of the 9th ACM Conference on Information-Centric Networking

Conference Dates

September 19-21, 2022

Conference Location

Osaka, JP

Conference Title

9th ACM Conference on Information Centric Networking (ICN 2022)

Pub Type

Conferences

Download Paper

https://doi.org/10.1145/3517212.3558087

Local Download

Keywords

named data networking, information centric networking, naming, caching, forwarding, high energy physics, large hadron collider

Citation

Yeh, E. , Newman, H. , Zhang, L. , Cong, J. , Shannigrahi, S. , Wu, Y. , Iordache, C. , Mutlu, V. , Timilsina, S. , Song, S. , Lo, M. , Liu, R. , Fan, C. , Sirvinskas, R. , Balcas, J. , Liu, Y. , Pesavento, D. , Shi, J. and Benmohamed, L. (2022), N-DISE: NDN-Based Data Distribution for Large-Scale Data-Intensive Science, Proceedings of the 9th ACM Conference on Information-Centric Networking, Osaka, JP, [online], https://doi.org/10.1145/3517212.3558087, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=935223 (Accessed April 25, 2024)