Link to the Information Technology Laboratory Website Link to the Information Access Division Website Link to the NIST Website NIST, IAD Banner

Supported flows

Besides the example clients using ‘Flow_BlockTest’ we would like to mention four more complex flows which have been developed in order to facilitate our research. These flows might be a good entry point to design and develop new flows, because they address problems of transporting signal processing type data and make use of more sophisticated techniques (metadata, serialization, stream dynamics). These flows are provided with the audio/video processing library.

  • The mentioned Flow_BlockTest is a simple buffer used in example clients.

  • Flow_Audio_SFA transports audio data and is momentarily the most complex flow. The transported type is a class hierarchy based on the SFAudio class (smart flow audio), which can transport many types of single- to multichannel data of various sampling rate and bit-depth (32, 24, 16, 8 bit integer, channels interleaved or non-interleaved). Using metadata and typecasting, this flow demonstrates how to change type and parameters of the transported data on the fly. This means that it is possible to change bit-depth, amount of channels, sample size and sample-rate while a processing pipeline is running. A source separation processing node could for example stream 2 channels of separated audio data in case there were two sources detected, but switch lateron to stream 3 channels because of the addition of a third source to the scene.

  • Flow_Video_SFI transports video and image data. Only one type of data, called SFImage (smart flow image) is transported. Using metadata the flow can be used dynamically as well (change of image size on the fly). SFI wraps OpenCVs IplImage or uses a raw BGR representation on platforms where OpenCV is not available. Based on this type we have also developed a AVCodec compression flow, which encodes and decodes a image sequence on the fly, using ffmpeg. This flow will be added soon to the repository and is at the moment available only on request.

  • Flow_Feature_SFM transports matrices and vectors (float). The transported data type is called SFMatrix (smart flow matrix). Handling of the matrix attributes (size, transpose) is done dynamically as well, so different types of matrices can be transported using the same stream.

  • Flow_Feature_SSO transports a vector of SSO (simple scene object) data of unkown size. This flow demonstrates how to apply metadata to group single data-types (in this case the SSO class) using a standard container (vector).

Please keep in mind that the flows buffer size has to be known a priori. In order to use the dynamic features of the described flows, e.g. increase the data-size on the fly, please use buffers bigger than your current data to avoid problems.

Created on 2008-06-18 by Antoine Fillinger - Last updated on 2008-11-23 by Antoine Fillinger