Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

N-DISE: NDN-Based Data Distribution for Large-Scale Data-Intensive Science

Published

Author(s)

Edmund Yeh, Harvey Newman, Lixia Zhang, Jason Cong, Susmit Shannigrahi, Yuanhao Wu, Catalin Iordache, Volkan Mutlu, Sankalpa Timilsina, Sichen Song, Michael Lo, Ran Liu, Chengyu Fan, Raimondas Sirvinskas, Justas Balcas, Yuezhou Liu, Davide Pesavento, Junxiao Shi, Lotfi Benmohamed

Abstract

To meet unprecedented challenges faced by the world's largest data- and network-intensive science programs, we design and implement a new, highly efficient and field-tested data distribution, caching, access and analysis system for the Large Hadron Collider (LHC) high energy physics (HEP) network and other major science programs. We develop a hierarchical Named Data Networking (NDN) naming scheme for HEP data, implement new consumer and producer applications to interface with the high-performance NDN-DPDK forwarder, and build on recently developed high-throughput NDN caching and forwarding methods. We integrate NDN systems concepts and algorithms with the mainstream data distribution, processing, and management system of the Compact Muon Solenoid (CMS) experiment. We design and prototype stable, high-performance virtual LANs (VLANs) over a continental-scale wide area network testbed. In extensive experiments, our proposed integrated system, named NDN for Data-Intensive Science Experiments (N-DISE), is shown to deliver LHC data over the wide area network (WAN) testbed at throughputs exceeding 31 Gbps between Caltech and StarLight, with dramatically reduced download time.
Proceedings Title
Proceedings of the 9th ACM Conference on Information-Centric Networking
Conference Dates
September 19-21, 2022
Conference Location
Osaka, JP
Conference Title
9th ACM Conference on Information Centric Networking (ICN 2022)

Keywords

named data networking, information centric networking, naming, caching, forwarding, high energy physics, large hadron collider

Citation

Yeh, E. , Newman, H. , Zhang, L. , Cong, J. , Shannigrahi, S. , Wu, Y. , Iordache, C. , Mutlu, V. , Timilsina, S. , Song, S. , Lo, M. , Liu, R. , Fan, C. , Sirvinskas, R. , Balcas, J. , Liu, Y. , Pesavento, D. , Shi, J. and Benmohamed, L. (2022), N-DISE: NDN-Based Data Distribution for Large-Scale Data-Intensive Science, Proceedings of the 9th ACM Conference on Information-Centric Networking, Osaka, JP, [online], https://doi.org/10.1145/3517212.3558087, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=935223 (Accessed February 3, 2023)
Created September 6, 2022, Updated November 29, 2022