Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Spatial Computations over Terabyte-Sized Images on Hadoop Platforms

Published

Author(s)

Peter Bajcsy, Antoine Vandecreme, Mary C. Brady, Phuong Nguyen

Abstract

Our objective is to lower the barrier of executing spatial image computations in a computer cluster/cloud environment instead of in a desktop/laptop computing environment. We research two related problems encountered during an execution of spatial computations over terabyte-sized images using Apache Hadoop running on distributed computing resources. The two problems address (a) detection of spatial computations and their parameter estimation from a library of image processing functions, and (b) partitioning of image data for spatial image computations on Hadoop cluster/cloud computing platforms in order to minimize network data transfer. The first problem is solved by designing an iterative estimation methodology. The second problem is formulated as an optimization over three partitioning schemas (physical, logical without overlap and logical with overlap), and evaluated over several system configuration parameters. Our experimental results for the two problems demonstrate 100% accuracy in detecting spatial computations in the Java Advanced Imaging and ImageJ libraries, a speed-up of 5.36 between the default Hadoop physical partitioning and developed logical image partitioning with overlap, and 3.14 times faster execution of logical partitioning with overlap than the one without overlap. The novelty of our work is in designing an extension to Apache Hadoop to run a class of spatial image processing operations efficiently on a distributed computing resource.
Conference Dates
October 27-30, 2014
Conference Location
Washington, DC
Conference Title
2014 IEEE International Conference on Big Data

Keywords

Spatial image operations, Hadoop, Image partition, Distributed computing

Citation

Bajcsy, P. , Vandecreme, A. , Brady, M. and , P. (2014), Spatial Computations over Terabyte-Sized Images on Hadoop Platforms, 2014 IEEE International Conference on Big Data , Washington, DC, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=916282 (Accessed June 20, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created October 27, 2014, Updated March 17, 2017