NML Programmer's Guide (Ada Version)

Introduction

The Real-Time Control System (RCS) library is a C++/C/Java/ada class library intended for multi-platform real-time distributed applications. It has been compiled and tested on several platforms including Linux, MacOs X, QNX, VxWorks, MS Windows and several UNIX Versions. This document describes the use of the Neutral Message Language (NML) components of the library.

The Communication Management System (CMS) provides access to a fixed-size buffer of general data to multiple reader or writer processes on the same processor, across a backplane, or over a network. Regardless of the communication method required, the interface to CMS is uniform. Methods are provided to encode all of the basic C data types in a machine independent or neutral format, and to return them to the native format. A CMS_HEADER is added to each buffer which provides information about whether the buffer has been written or read from last, whether the buffer is new to a particular process and the size of the last write to the buffer. CMS uses a configuration file so that users can change communications protocols or parameters without recompiling or relinking the applications.

The Neutral Message Language (NML) provides a higher level interface to CMS. It provides a mechanism for handling multiple types of messages in the same buffer as well as simplifying the interface for encoding and decoding buffers in neutral format and the configuration mechanism. End user applications now see NML almost exclusively while CMS only peeks through when examining some of the automatically generated code.

Terminology

The figure below illustrates the structure of a typical RCS application using NML. The application is distributed across three computers. Processes 1, 2, and 3 are able to write directly into the shared memory buffers they use because they are located in the same computer or backplane. It is for this reason that they are labeled "LOCAL". Processes 4,5 and 6 can only access the buffers through an NML Server and are therefore labeled "REMOTE". The description might need to be complicated in a system with buffers in more than one machine. Processes would then need to be described as local or remote with respect to a particular buffer.

NML servers must be run for each buffer that will be accessed by remote processes. They read and write to the buffer in the same way as local processes on the behalf of remote processes.

NML uses configuration files to store information about which processes communicate with which buffers and how. Most of the options available to NML programmers are chosen by specifying them in the configuration file. (The configuration files are ascii text files with a format described under "Writing NML Configuration Files".)Some systems use an NML Configuration server instead of or in addition to the NML Configuration files. See NML Configuration Server.

NML is message-based rather than stream-based. Each successful read operation retrieves the data sent in exactly one write operation. Unless queuing is enabled, each write operation moves one message into the buffer replacing any previous message.

More than one type of message can be sent to the same buffer so a unique type identifier is always contained in the message. After a read operation, the process must use this identifier to determine the type of message before using any of the data in the message. Each type of message implies a particular data structure. Most messages are user-defined.

Messages are called encoded if they have been translated into a machine-independent or neutral format such as the eXternal Data Representation (XDR). Buffers are called encoded if the messages in them are to be encoded which is established in the configuration file. NML servers can encode and decode messages on behalf of remote processes. An NML vocabulary defines the set of messages that may be used in an application and provides the necessary functions for encoding and decoding the messages. All buffers in an application could share a common vocabulary or each buffer could have its own associated vocabulary.

The next figure shows the structure of a single concurrent process module using NML (the memory buffer appears to be local to the application)

The applications routines initialize and use objects from class NML and NMLmsg which depend on some user-defined functions. The format function selects from a set of user defined update functions for each aggregate type the user will need to pass to the memory buffer. The update function for each aggregate type is built by updating each member inpidually using CMS routines for the basic C data types. These basic update routines write to and read from internal CMS buffers which are themselves read or written to memory buffers that are available to other concurrent processes using the CMS Communications Routines.

Notation

I like lots of examples.

Commands users are expected to enter at a command prompt will look like this. 

Computer program generated example output will look like this. 

Text files listed in line look like this.

Compiling and Linking

Currently the Ada Spec and Body files are just kept in the src/ada directory when the RCS library source code is extracted. So for example to compile and link an example that uses NML in file "example.adb" containing a main procedure "Example" on a system where the source code was extracted into "/path/to/rcslib-version/" and the code was compiled with prefix or install directory set to "/path/to/rcslib_install_dir" run:

gnatmake example -I/path/to/rcslib-version/src/ada -L/path/to/rcslib_install_dir/lib  

Check for a README file in the source code archive and http://www.isd.mel.nist.gov/projects/rcslib/getrcs.html for more up-to-date or detailed compiling instructions.

Designing an NML Application.

Because NML is configurable, programmers can choose between protocols with higher performance but which may be more restrictive or require more expensive hardware or those that are less restrictive or require less expensive more widely available hardware. By making a buffer local to a process you can improve the performance of that process. By moving processes you may be able to reduce the load on one CPU or increase the number of processes able to use the faster local protocol. Using servers to provide remote access to buffers frees local processes from being slowed down by the communications with remote processes.

Example: Robot Controller/Supervisor Design

A controller for a robot must poll a variety of inputs and perform some computations every "n" milliseconds and a remote supervisor should be able to check the status of the robot when needed.

The next figure shows one possible design for this application. Because the controller can write directly to the shared memory buffer, writing the status takes a minimum time for the controller. Using the NML server allows the supervisor to be located almost anywhere and on almost any host.

Summary of Design Suggestions.

1. Avoid overloading any CPU by assigning too many processes to it or building a single process which must do too much work.
2. Place buffers so that they may be accessed locally by the most time-critical process(es).
3. Use the "LOCAL" protocol whenever possible.
4. Only use neutrally encoded buffers when necessary.(i.e. backplane communications between different types of processors)

Programming with NML

NML applications programmers need to create a message vocabulary and associated format function, write a configuration file, create an NML object, and use the read and write member functions.

Creating an NML Vocabulary (Format Functions, Update Functions, and Message Definitions)

The message vocabulary is a set of C++ classes, derived from NMLmsg that map directly to a set of Ada record types, which can be thought of as data structures that are copied into the NML buffer during a write operation, and copied out during a read operation. Each class is associated with a unique identifier, a positive integer, that allows readers to identify which message is present in the buffer. Besides the particular data members of the class, each class also needs an update function which calls CMS methods to convert the data members to types CMS can handle. Currently, CMS provides support for the basic C language built-in types.

To enable CMS to neutrally format the data in the buffer or to allow NML servers to encode and decode the data for remote processes, a format function is required. This format function is nothing more than a switch statement, associating NML identifiers with the update functions of particular NML message classes. The format function can be manually programmed as will be described below, or it can be automatically generated using the NML Code Generator.

Variable Length Arrays

Some advanced users define messages with variable length arrays. There are several ways to do this, but the simplest and most convenient way is ussually to use the DECLARE_NML_DYNAMIC_LENGTH_ARRAY macro. The macro has special meaning to the NML Code Generator. The result is an array with a constant maximum size but where only the first name_length elements are sent across the network with each remote read or write. Local reads and writes can be forced to use the condensed version by setting the neutral configuration file flag to 1.

For your information the text of the macro is:

 #define DECLARE_NML_DYNAMIC_LENGTH_ARRAY(type, name, size) int name##_length; type name[size]; 

Example: Message Definition.

Files needed for this example include: nml_ex1.hh, nml_ex1.cc

Here is nml_ex1.hh

/* nml_ex1.hh */   #ifndef NML_EX1_HH  #define NML_EX1_HH  #include "rcs.hh"  /* Give the new structure a unique id number */ /* The name is ussually constructed by taking the class or struct name  and adding "_TYPE" to it. */ #define EXAMPLE_MSG_TYPE 101   /* The id number must be unique within a CMS buffer, i.e. the number must be different than the id of any other type that might be written to a particular buffer. For simplicity it is recommended that the id number also be unique within an application. */  /* Define the new message structure */  struct EXAMPLE_MSG: public NMLmsg {  /* The constructor needs to store the id number */  /* and the size of the new structure */ /* by passing them as arguments to the base class constructor. */  EXAMPLE_MSG():NMLmsg(EXAMPLE_MSG_TYPE, sizeof(EXAMPLE_MSG)){};  /* Each new type needs to overload the update function. */ void update(CMS *cms);  /* Data in this new message format. */ float f; char c; int i;  DECLARE_NML_DYNAMIC_LENGTH_ARRAY(int, da, 100);  };  /* Declare the NML Format function. */  int ex_format(NMLTYPE type, void *buf, CMS *cms);  #endif  /* End of NML_EMC_HH */  

This will then be used to generate the following Ada Spec and body. This is the command that was used.

java -jar ~/rcslib/plat/java/lib/CodeGenCmdLine.jar nml_ex1.hh  

Ada Spec File:

-- --New Ada Spec File starts here. --This file should be named nml_ex1_n_ada.ads --Automatically generated by NML CodeGen Java Applet. --on Wed Sep 01 14:17:00 EDT 2004 --  with Nml_Msg; use Nml_Msg; with Cms;  with Posemath_N_Ada;  use Posemath_N_Ada;   --Some standard Ada Packages we always need. with Unchecked_Deallocation; with Unchecked_Conversion; with Interfaces.C; use Interfaces.C;  package nml_ex1_n_ada is  -- Create Ada versions of the Enumeration types.  function Format(Nml_Type : in long; Msg : in NmlMsg_Access; Cms_Ptr : in Cms.Cms_Access) return int;  pragma Export(C,Format,"ada_nml_ex1_n_ada_format");   -- Redefine the C++ NML message classes as Ada Records.  EXAMPLE_MSG_TYPE : constant := 101;  type EXAMPLE_MSG is new NMLmsg with  record f : c_float; c : char; i : int; da_length : int; da : Cms.Int_Array(1..100); end record;  type EXAMPLE_MSG_Access is access all EXAMPLE_MSG; procedure Initialize(Msg : in out EXAMPLE_MSG); function NmlMsg_to_EXAMPLE_MSG is new Unchecked_Conversion(NmlMsg_Access,EXAMPLE_MSG_Access); procedure Update_Internal_EXAMPLE_MSG(Cms_Ptr : in Cms.Cms_Access; Msg : in out EXAMPLE_MSG); procedure Free is new Unchecked_Deallocation(EXAMPLE_MSG,EXAMPLE_MSG_Access); type EXAMPLE_MSG_Array is array(Integer range <>) of EXAMPLE_MSG;   end nml_ex1_n_ada;  -- End of Ada spec file  nml_ex1_n_ada.ads  

Here is the corresponding body with Format and Update functions.

-- --New Ada Body File starts here. --This file should be named nml_ex1_n_ada.adb --Automatically generated by NML CodeGen Java Applet. --on Wed Sep 01 14:22:44 EDT 2004 --  with Nml_Msg; use Nml_Msg;  with Posemath_N_Ada;  use Posemath_N_Ada;  with Cms;   --Some standard Ada Packages we always need. with Unchecked_Deallocation; with Unchecked_Conversion; with Interfaces.C; use Interfaces.C; with Interfaces.C.Strings; use Interfaces.C.Strings;  package body nml_ex1_n_ada is  -- Create some common variables  and functions needed for updating Enumeration types.  -- Every NMLmsg type needs an update and an initialize function.  procedure Initialize(Msg : in out EXAMPLE_MSG) is begin Msg.NmlType := EXAMPLE_MSG_TYPE; Msg.Size := EXAMPLE_MSG'Size/8; end Initialize;  procedure Update_EXAMPLE_MSG(Cms_Ptr : in Cms.Cms_Access; Msg : in EXAMPLE_MSG_Access) is begin Cms.Begin_Class(Cms_Ptr,"EXAMPLE_MSG",""); Msg.NmlType := EXAMPLE_MSG_TYPE; Msg.Size := EXAMPLE_MSG'Size/8; Cms.Update_C_Float(Cms_Ptr, "f", Msg.f); Cms.Update_Char(Cms_Ptr, "c", Msg.c); Cms.Update_Int(Cms_Ptr, "i", Msg.i); Cms.Update_Dla_Length(Cms_Ptr,"da_length", Msg.da_Length); Cms.Update_Int_Dla(Cms_Ptr, "da", Msg.da,Msg.da_length,100); Cms.End_Class(Cms_Ptr,"EXAMPLE_MSG",""); end Update_EXAMPLE_MSG;  procedure Update_Internal_EXAMPLE_MSG(Cms_Ptr : in Cms.Cms_Access; Msg : in out EXAMPLE_MSG) is begin Cms.Begin_Class(Cms_Ptr,"EXAMPLE_MSG",""); Msg.NmlType := EXAMPLE_MSG_TYPE; Msg.Size := EXAMPLE_MSG'Size/8; Cms.Update_C_Float(Cms_Ptr, "f", Msg.f); Cms.Update_Char(Cms_Ptr, "c", Msg.c); Cms.Update_Int(Cms_Ptr, "i", Msg.i); Cms.Update_Dla_Length(Cms_Ptr,"da_length", Msg.da_Length); Cms.Update_Int_Dla(Cms_Ptr, "da", Msg.da,Msg.da_length,100); Cms.End_Class(Cms_Ptr,"EXAMPLE_MSG",""); end Update_Internal_EXAMPLE_MSG;    NameList : constant Char_Array(1..24) := ( 'E','X','A','M','P','L','E','_','M','S','G',nul, nul,nul,nul,nul,nul,nul,nul,nul,nul,nul,nul,nul );  IdList : constant Cms.Long_Array(1..2) := ( EXAMPLE_MSG_TYPE, -- 101, 0 -1);  SizeList : constant Cms.Size_T_Array(1..2) := ( EXAMPLE_MSG'Size/8, 0); Symbol_Lookup_EXAMPLE_MSG_Name : constant Interfaces.C.Strings.Chars_Ptr := Interfaces.C.Strings.New_String("EXAMPLE_MSG");  function Symbol_Lookup(Nml_Type : in long) return Interfaces.C.Strings.Chars_Ptr; pragma Export(C,Symbol_Lookup,"ada_nml_ex1_n_ada_symbol_lookup");  function Symbol_Lookup(Nml_Type : in long) return Interfaces.C.Strings.Chars_Ptr is begin case Nml_Type is  when EXAMPLE_MSG_TYPE=>return Symbol_Lookup_EXAMPLE_MSG_Name;  when others=>return Null_Ptr; end case; end Symbol_Lookup;  function Format(Nml_Type : in long; Msg : in NmlMsg_Access; Cms_Ptr : in Cms.Cms_Access) return int is Checked_Nml_Type : long;  begin Checked_Nml_Type := Cms.Check_Type_Info(Cms_Ptr,Nml_Type, NmlMsg_Access_To_Limited_Controlled_Access(Msg), "nml_ex1_n_ada", Symbol_Lookup'Access, NameList,IdList,SizeList,2,12);  if Msg = Null then return 0; end if;  case Checked_Nml_Type is when EXAMPLE_MSG_TYPE=>Update_EXAMPLE_MSG(Cms_Ptr, NmlMsg_to_EXAMPLE_MSG(Msg)); when others=>return 0; end case; return 1; end Format;  end nml_ex1_n_ada;  -- End of Ada Body file  nml_ex1_n_ada.adb

NOTE: All the NML updates are identical except that the body should call the CMS update function for each member in the structure. The update function has been overloaded to accept references to all of the basic C data types (ints, floats, etc.) Depending on the CMS mode the update functions will either store their argument in a neutrally encoded buffer or decode the buffer and store the output in the variables passed to the update functions. Just as with the format function, the update functions can be either manually coded or generated automatically with the NML Code Generator.

Creating an NML Connection Object

The CreateConnection function prototyped in nml.ads is as follows:

   function CreateConnection(NewCallBackFunction : in Format_Callback_Func;                              BufferName,ProcessName,ConfigSource: in String)                             return NmlConnection_Access;     function Valid(Connection : in NmlConnection_Access) return Boolean;

 The parameters are: the NewCallBackFunction which is the address of the FormatFunction which is normally automatically generated specific to the types of messages, BufferName the name of the buffer to make a connection to which should match an entry in the configuration file and the name passed by other processes with which you will communicate with, a process name which identifies the calling process in the configuration file, and ConfigSource which is either a file name in the format described in nmlcfg.html or the location of an nml configuration server as described in nmlcfgsvr.html. After creating a connection one should normally check that it is valid using the Valid function.

Reading NML Data

These are the member functions used to perform read:

function Read(Connection : in NmlConnection_Access) return Integer;  function Peek(Connection : in NmlConnection_Access) return Integer;  function Blocking_Read(Connection : in NmlConnection_Access;                           Timeout : Interfaces.C.Double ) return Integer;  function Get_Address(Connection : in NmlConnection_Access) return NmlMsg_Access; function Get_Last_Error_Type(Connection : in NmlConnection_Access)                                return NML_ERROR_TYPE;

 Either the read, peek or blocking_read functions can be used to retrieve the message which is then stored in local memory and either the type of message, or zero indicatating no new data or -1 indicating an error occured. After one knows the type of data one can use Get_Address to make the data available through a variable of the appropriate type. Read will check buffer to see if a new message has been written and remove it from the queue if this is a queued buffer or mark the buffer as having been read otherwise. Peek does the same thing but leaves the message in the queue where it then might be read by another process or for this one through another connection object at does not mark the buffer as having been read. Blocking_read waits until a new message is available or until the timeout occurs and reads the buffer as read would when a new message is available. For non-queued buffers NML keeps track of the last messageread through each connection object, so if a second process reads from the same buffer it will see messages as new even if it has already been read by the first process, since it is in fact new to the second process. If an error occurs one can call Get_Last_Error_Type to determine what kind of corrective action to take, more descriptive info is also logged to where ever the rcs_print_destination is set to, which is normally stdout.

Writing NML Data

The write function is the following:

function Write(Connection : in NmlConnection_Access; Message : in NmlMsg_Access) return Integer;  function Get_Last_Error_Type(Connection : in NmlConnection_Access)                                return NML_ERROR_TYPE;

The write function copies the given message into the buffer associated whith this connection object.

with Nml; use Nml; with Mynmlmsg; use Mynmlmsg; with Ada.Text_IO; with Ada.Integer_Text_IO; with Unchecked_Conversion; with Interfaces.C;  procedure Nmltest is    Connection1 :  NmlConnection_Access;    Connection2 :  NmlConnection_Access;    Read_Msg : Mynmlmsg_Type_Access;    Msg :  Mynmlmsg_Type_Access;    Ok : Integer := 0;    Msg_Type : Integer;  begin    Msg := new Mynmlmsg_Type;    Msg.AnotherInt := 67;    Connection1 := Nml.CreateConnection(Mynmlmsg.Format'Access,"b1","proc","nmlcfgsvr::::options=neutral=1");    Connection2 := Nml.CreateConnection(Mynmlmsg.Format'Access,"b1","proc","nmlcfgsvr::::options=neutral=1");    Ok := Nml.Write(Connection1,NmlMsg_Access(Msg));    Msg_Type := Nml.Read(Connection2);    if Msg_Type = 1001 then       Read_Msg := Mynmlmsg.NmlMsg_Access_To_Mynmlmsg_Type_Access(Nml.Get_Address(Connection2));       Ada.Text_IO.Put("Read_Msg.AnotherInt=");       Ada.Integer_Text_IO.Put(Integer(Read_Msg.AnotherInt));       Ada.Text_IO.New_Line;    end if;    Nml.Free(Connection2);    Nml.Free(Connection1);  end Nmltest;

Last Modified: October 24,2004

If you have questions or comments regarding this page please contact Will Shackleford at shackle [at] cme.nist.gov (shackle[at]cme[dot]nist[dot]gov)

To be automatically sent information on updates to the RCS library, please subscribe to the "nist_rcs" mailing list on http://www.onelist.com.