Link to the Information Technology Laboratory Website Link to the Information Access Division Website Link to the NIST Website NIST, IAD Banner

Advantages of using a data flow

Using a data flow system has a lot of advantages, especially for data driven applications where processing requirements are beyond the computational power provided by a single host. The most obvious advantage is that you don't have to handle the transport of your data between client nodes. This task is delegated to the NDFS-II. To connect client nodes together through flows, you don't have to deal with the locations of your client nodes in the network but instead you specify which flows you want to connect. It allows you to focus on specific problems rather than spending time on how to transport your data.

In order to achieve the data transport and address the needs of as many people as possible, the data flow has to have some key capabilities. NDFS II is

  • Dynamic: A host can join or leave an application at any time. Client nodes can also join or leave an application at any time and in the same manner they can create or destroy flows whenever they need. The system has been designed to support this kind of behavior.

  • Decentralized: There is no central server in the NDFS network. If a server or a client node crashes, it won't make the system unstable and unusable. Of course client nodes consuming data from a producer client node, which crashes, will be affected. But they will only be affected by not receiving data.

  • Extensible: You can define your own kind of flows. These flows can transport any kind of data or metadata. Flows work as “plug-ins” meaning that you don't have to recompile the whole system to use them.

  • Cross platform: Because people have different needs where some programs are tied to a specific architecture (e.g.: a camera only supported under Windows), the NDFS-II runs on Unix/Linux, Mac OS X and Windows XP. The middleware will transport the data independently of the operating system.

  • Multi-language: The system has been developed in C++, and it provides an object oriented API to use. However, users can develop client nodes using Java thanks to our wrapper. So an application can be composed of a Java client node capturing video and feeding a face tracker client node written in C++.

  • Optimized: The transport is optimized to increase the performance. Data are transported within a host using shared memory and via the network using TCP/IP between hosts. If a producer client node sends a flow to two consumer client nodes located on the same machine, the data will only be transported once over the network.

  • Modular: It promotes the development of reusable client nodes with well-defined interfaces. Several applications may use a Word Recognizer, for example.

Created on 2008-06-18 by Antoine Fillinger - Last updated on 2008-11-23 by Antoine Fillinger