NML Programmer's Guide (Java Version)

• Class Hierarachy (javadoc generated)
• See other RCS Library Documents.

Introduction

The Real-Time Control System (RCS) library is a class library intended for multi-platform real-time distributed applications. There are now 2 versions of the library, one in C++ and the other in Java. The Java version currently does not provide all of the functionality of the C++ version, but the client-side communications classes for NML have been re-written in Java. This document describes the use of the Neutral Message Language (NML) components of the library.

The Communication Management System (CMS) provides access to a fixed-size buffer of general data to multiple reader or writer processes on the same processor, across a backplane, or over a network. Regardless of the communication method required, the interface to CMS is uniform. Methods are provided to encode all of the basic C/Java data types in a machine independent or neutral format, and to return them to the native format. A CMS_HEADER is added to each buffer which provides information about whether the buffer has been written or read from last, whether the buffer is new to a particular process and the size of the last write to the buffer. CMS uses a configuration file so that users can change communications protocols or parameters without recompiling or relinking the applications.

The Neutral Message Language (NML) provides a higher level interface to CMS. It provides a mechanism for handling multiple types of messages in the same buffer as well as simplifying the interface for encoding and decoding buffers in neutral format and the configuration mechanism.

Terminology

The figure below illustrates the structure of a typical RCS application using NML. The application is distributed across three computers. Processes 1, 2, and 3 are able to write directly into the shared memory buffers they use because they are located in the same computer or backplane. It is for this reason that they are labeled "LOCAL". Processes 4,5 and 6 can only access the buffers through an NML Server and are therefore labeled "REMOTE". The description might need to be complicated in a system with buffers in more than one machine. Processes would then need to be described as local or remote with respect to a particular buffer.

NML servers must be run for each buffer that will be accessed by remote processes. They read and write to the buffer in the same way as local processes on the behalf of remote processes.

NML uses configuration files to store information about which processes communicate with which buffers and how. Most of the options available to NML programmers are chosen by specifying them in the configuration file. (The configuration files are ascii text files with a format described under "Writing NML Configuration Files".)

NML is message-based rather than stream-based. Each successful read operation retrieves the data sent in exactly one write operation. Unless queuing is enabled, each write operation moves one message into the buffer replacing any previous message.

More than one type of message can be sent to the same buffer so a unique type identifier is always contained in the message. After a read operation, the process must use this identifier to determine the type of message before using any of the data in the message. Each type of message implies a particular data structure. Most messages are user-defined.

Messages are called encoded if they have been translated into a machine-independent or neutral format such as the eXternal Data Representation (XDR). Buffers are called encoded if the messages in them are to be encoded which is established in the configuration file. NML servers can encode and decode messages on behalf of remote processes. An NML vocabulary defines the set of messages that may be used in an application and provides the necessary functions for encoding and decoding the messages.

Header File

All of the necessary header files will be included if rcs.hh is included.

Classes

The following classes provide the programming interface for CMS and NML:

NML
NMLmsg
NML_SERVER

CMS
CMS_HEADER
CMS_SERVER

CMS_USER

These classes are detailed in the following sections.

NML Application Structure

The next figure shows the structure of a single concurrent process module using NML (the memory buffer appears to be local to the application)

The applications routines initialize and use objects from class NML and NMLmsg which depend on some user-defined functions. The format function selects from a set of user defined update functions for each aggregate type the user will need to pass to the memory buffer. The update function for each aggregate type is built by updating each member individually using CMS routines for the basic C data types. These basic update routines write to and read from internal CMS buffers which are themselves read or written to memory buffers that are available to other concurrent processes using the CMS Communications Routines.

Designing an NML Application.

Because NML is configurable, programmers can choose between protocols with higher performance but which may be more restrictive or require more expensive hardware or those that are less restrictive or require less expensive more widely available hardware. By making a buffer local to a process you can improve the performance of that process. By moving processes you may be able to reduce the load on one CPU or increase the number of processes able to use the faster local protocol. Using servers to provide remote access to buffers frees local processes from being slowed down by the communications with remote processes. Currently the NML communications servers and most-likely the real-time modules need to be written in C++, however GUI's, diagnostics tools, and upper level supervisors with less-rigid time deadlines can be written in Java and use NML to communicate with C++ parts of the application.

Example: Robot Controller/Supervisor Design

A controller for a robot must poll a variety of inputs and perform some computations every "n" milliseconds and a remote supervisor should be able to check the status of the robot when needed.

The next figure shows one possible design for this application. Because the controller can write directly to the shared memory buffer, writing the status takes a minimum time for the controller. Using the NML server allows the supervisor to be located almost anywhere and on almost any host.

Summary of Design Suggestions.

1. Avoid overloading any CPU by assigning too many processes to it or building a single process which must do too much work.
2. Place buffers so that they may be accessed locally by the most time-critical process(es).
3. Use the "LOCAL" protocol whenever possible.
4. Only use neutrally encoded buffers when necessary.(i.e. backplane communications between different types of processors)

Programming with NML

NML applications programmers need to create a message vocabulary and associated format function, write a configuration file, create an NML object, and use the read and write member functions.

Creating an NML Vocabulary (Format Functions, Update Functions, and Message Definitions)

The message vocabulary is a set of C++/Java classes, derived from NMLmsg, which can be thought of as data structures that are copied into the NML buffer during a write operation, and copied out during a read operation. Each class is associated with a unique identifier, a positive integer, that allows readers to identify which message is present in the buffer. Besides the particular data members of the class, each class also needs an update function which calls CMS methods to convert the data members to types CMS can handle. Currently, CMS provides support for the basic C/Java language built-in types. However, avoid using enums or long doubles in NML messages. (See "Trouble Shooting - Insufficient Arguments Error")

To enable CMS to neutrally format the data in the buffer or to allow NML servers to encode and decode the data for remote processes, a format function is required. This format function is nothing more than a switch statement, associating NML identifiers with the update functions of particular NML message classes. The format function can be manually programmed as will be described below, or it can be automatically generated using the NML Code Generator.

Example: Message Definition.

Files needed for this example include: nml_ex1MsgDict.java, EXAMPLE_MSG.java

Both EXAMPLE_MSG.java, and nml_ex1MsgDict.java were generated from nml_ex1.hh using the NML Code Generator. nml_ex1.hh is also used in the C++ examples.

  /* *New Java File starts here. *This file should be named nml_ex1_MsgDict.java *Automatically generated by NML CodeGen Java Applet. *with command line arguments :  generate_for_java=true generate_for_cpp=false HHFile=nml_ex1.hh *RCS_VERSION=2009.12.17_1575* *.gen script : *0:load nml_ex1.hh *1:clear *2:select_from_file nml_ex1.hh *3:generate Java dict>nml_ex1_MsgDict.java *4:generate Java classes >* *5:exit * */  // Import NML classes and interfaces import rcs.nml.*;  import java.util.Hashtable; /* *Class definition for nml_ex1_MsgDict *Automatically generated by NML CodeGen Java Applet. */ public class nml_ex1_MsgDict implements NMLMessageDictionary {  // Define an object of every message class. private EXAMPLE_MSG EXAMPLE_MSG_object = null;  // ID Type Constants public static final int EXAMPLE_MSG_TYPE  = 101;  // Sizes useful for C++ compat and preallocating byte storage.  //(not used for normal NML reads/writes). public long getEstimatedSize(final int _type) { switch(_type) { case EXAMPLE_MSG_TYPE: /*101*/ return 232; default: break; } return 232; /* maximum */ }  public long getMaxEstimatedSize() { return 232; }    //Define an NML_ENUM_INFO object for the type ID's NML_ENUM_INFO nml_enum_info_for_type_names =null;   //Create a constructor to setup the NML_ENUM_INFO object. public nml_ex1_MsgDict() { nml_enum_info_for_type_names= new NML_ENUM_INFO(); nml_enum_info_for_type_names.name="nml_ex1_MsgDict"; Hashtable h1 = new Hashtable(); Hashtable h2 = new Hashtable(); Integer I = null; String  S = null; I=new Integer(EXAMPLE_MSG_TYPE); S="EXAMPLE_MSG"; h1.put(I,S); h2.put(S,I); nml_enum_info_for_type_names.int_to_string_hash=h1; nml_enum_info_for_type_names.string_to_int_hash=h2; }    // Miscellaneous Pre-Defined Values  // Enumerated Type Constants   // NML Format Function public int formatMsg(NMLFormatConverter nml_fc) { int return_val=0; nml_fc.check_type_info(nml_enum_info_for_type_names);  switch(nml_fc.msg_type) { case EXAMPLE_MSG_TYPE: /*101*/ if(null == EXAMPLE_MSG_object) {  EXAMPLE_MSG_object = new EXAMPLE_MSG(); } nml_fc.msg_to_update  = EXAMPLE_MSG_object; EXAMPLE_MSG_object.update(nml_fc); break;  default: return_val=-1; break; } return(return_val); }  } 

/* *New Java File starts here. *This file should be named EXAMPLE_MSG.java */  // Import all NML, CMS, and RCS classes and interfaces import rcs.*; import rcs.nml.*; import rcs.nml.NMLFormatConverter; import rcs.nml.NMLMessageDictionary; import rcs.nml.NMLmsg; import rcs.nml.RCS_CMD_MSG; import rcs.nml.RCS_STAT_MSG;  /* *Class definition for EXAMPLE_MSG *Automatically generated by RCS Java Diagnostics Tool. *on Thu Jun 05 13:38:12 EDT 1997 */ public class EXAMPLE_MSG extends NMLmsg { float f = 0; byte c = 0; int i = 0;   // Constructor  public EXAMPLE_MSG() { super(101); }   public void update(NMLFormatConverter nml_fc) { super.update(nml_fc); f  = nml_fc.update(f ); c  = nml_fc.update(c ); i  = nml_fc.update(i );  } } 

These two files could also be generated with the command.

java -jar CodeGenCmdLine.jar generate_for_java=true generate_for_cpp=false nml_ex1.hh 

Whether you generate the source files or download them, the should be compiled with this command.

javac -classpath rcs.jar EXAMPLE_MSG.java  nml_ex1_MsgDict.java 

It is normal to see the following notes. (We could change the code to use safer checked functions but then the code would no longer compile with older jdk's. The files rcs.jar and CodeGenCmdLine.jar are available on the NIST ftp site o r can be built with the rcslib to be found in the bin directory or plat/java/lib directory.

Note: nml_ex1_MsgDict.java uses unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. 

NOTE: All the NML updates are identical except that the body should call the NMLFormatConverter update function for each member in the class. The update function has been overloaded to accept references to all of the basic data types (ints, floats, etc.) Depending on the NMLFormatConverter mode the update functions will either store their argument in a neutrally encoded buffer or decode the buffer and store the output in the variables passed to the update functions. Just as with the format function, the update functions can be either manually coded or generated automatically with the NML Code Generator.

Creating an NML Object

NMLConnection has several constructors, but most users will use the following.

NMLConnection(NMLMessageDictionary msg_dict, String buf, String proc, String file) throws NMLException

The parameters are:
msg_dict = < message dictionary object to use>;
buf = <name of the buffer to connect to as specified in configuration file>;
proc = <name under which to access the buffer>;
file = <name of the configuration file>;

Example: Constructors

Files needed for this example include: nml_ex1MsgDict.java, EXAMPLE_MSG.java,nml_ex2.java

/* nml_ex2.java */  //Import the custom message dictionary // Normally one would have something like import package.nml_ex1_MsgDict; // but since we are not using packages we don't need the import.  // Import all NML, CMS, and RCS classes and interfaces import rcs.*; import rcs.nml.*;  public class nml_ex2 { public static void main(String args[]) throws Exception { /* NML( format function, buffer name, process name, configuration file ) */ NMLConnection example_nml = new NMLConnection(new nml_ex1_MsgDict(), "ex_buf1","ex2_proc", "ex_cfg.nml"); }  } 

Compile with:

javac -classpath rcs.jar nml_ex2.java nml_ex1_MsgDict.java EXAMPLE_MSG.java 

To run it you need to first start the server in the background. The server must be written in C++ and is example 9 in the NML C++ Programmer's Guide.

# Compile command depends on OS and compiler versions. g++ nml_ex9.cc nml_ex1.cc rcslib/lib/librcs.a -lpthread -o nml_ex9 ./nm_ex9 & 

To run the java app.

java -classpath rcs.jar nml_ex2 

This example normally has no output or effect.

Reading NML Data

These are the member functions used to perform reads:

• public NMLmsg peek() throws NMLException() 
    Read an NMLmsg but do not change the was_read flag.
• public String peekDataString() throws NMLException()
    Reads an NMLmsg using peek() and converts it to a String.
• public NMLmsg read() throws NMLException() 
    Read a NMLmsg.
• public String readDataString() throws NMLException() 
    Reads an NMLmsg and converts it to a String.

There is a difference between these functions and the C++ versions. The C++ functions NML::read() and NML::peek() return an integer indicating what type of message if any was read and the actual data is stored in the area that can be found by calling NML::get_address(). But in java these functions return references to NMLmsg objects or null if there was no message to read, and there is no get_address function. (References are as close as Java gets to pointers.) The reason is that Java has two features that are not commonly available in C++: Run-time type checking and automatic garbage collection. In Java, it is not possible to simply create one static memory area to deposit all the NML messages recieved there, on the other hand once the objects are created, casting them to the wrong class causes a ClassCastException unlike C++ which would just let the object be used in inappropriate ways as long as there was an explicit cast.

If an error occurs that makes it impossible to read from this NML buffer, an NMLException is thrown.

Peek works exactly the same as read except that the flag that lets others know when the buffer is read is not changed and if queuing is enabled the message is not removed from the queue. This could be useful if you need to monitor a buffer without letting other processes using the buffer know.

Example: Reading from an NML Channel.

Files needed in this example include: nml_ex1MsgDict.java, EXAMPLE_MSG.java, nml_ex3.java

// Import the custom message dictionary and message classes // import nml_ex1MsgDict; -- import not needed since not using packages // import EXAMPLE_MSG; -- import not needed since not using packages  // Import all NML, CMS, and RCS classes and interfaces import rcs.*; import rcs.nml.*; import rcs.utils.*;  class nml_ex3 {   public static void main(String args[])  throws Exception   {     /* NMLConnection( message dictionary, buffer name, process name, configuration file ) */     NMLConnection example_nml = new NMLConnection(new nml_ex1_MsgDict(),    "ex_buf1",   "ex3_java_proc",    "ex_cfg.nml");      EXAMPLE_MSG msg = null;     while(msg == null)     {       msg = (EXAMPLE_MSG) example_nml.read();       Thread.sleep(100);/* Sleep for 100 milliseconds. */     }     /* Print something to the console to indicate a message was recieved. */     System.out.println("Message recieved: f = "+msg.f+", c = "+msg.c+", i = "+msg.i);   }  } 

Compile with:

javac -classpath rcs.jar nml_ex3.java nml_ex1_MsgDict.java EXAMPLE_MSG.java 

To run it you need to first start the server in the background. The server must be written in C++ and is example 9 in the NML C++ Programmer's Guide.

# Compile command depends on OS and compiler versions. g++ nml_ex9.cc nml_ex1.cc rcslib/lib/librcs.a -lpthread -o nml_ex9 ./nm_ex9 & 

To run the java app.

java -classpath rcs.jar nml_ex3 

If you try to run this example, it will wait for something to be written into the buffer. To write something into the buffer, you can use the example in "Writing NML Data". You will also need an NML server. The program nml_ex9 which is described in the C++ version of this guide under "Using the run_nml_servers function." can be used as the NML server.

Writing NML Data

These are the member functions used to perform writes:

 public int write(NMLmsg msg) 

 public int writeDataString(String dataString) 

The equivalent of the write_if_read() function in C++ has not been implemented yet.

Example: Writing to an NML Channel.

Files needed in this example include: nml_ex1MsgDict.java, EXAMPLE_MSG.java, nml_ex4.java

// Import the custom message dictionary and message classes // import nml_ex1MsgDict; -- import not needed, not using packages // import EXAMPLE_MSG; -- import not needed, not using packages  // Import all NML, CMS, and RCS classes and interfaces import rcs.*; import rcs.nml.*; import rcs.utils.*;  class nml_ex4 {   public static void main(String args[]) throws Exception   {     /* NMLConnection( message dictionary, buffer name, process name, configuration file ) */     NMLConnection example_nml = new NMLConnection(new nml_ex1_MsgDict(),    "ex_buf1",   "ex2_proc",    "ex_cfg.nml");      EXAMPLE_MSG msg = new EXAMPLE_MSG();     msg.f = (float) 3.14;     msg.c = (byte) '1';     msg.i = 1024;     example_nml.write(msg);      System.out.println("Sent message: f = "+msg.f+", c = "+msg.c+", i = "+msg.i);   }  } 

Compile with:

javac -classpath rcs.jar nml_ex4.java nml_ex1_MsgDict.java EXAMPLE_MSG.java 

Before running start the NML server, see previous examples.

To run the java app.

java -classpath rcs.jar:. nml_ex4 

It should print:

Sent message: f = 3.14, c = 49, i = 1024 

This example writes a message into a buffer. To read the message use the example in Reading NML Data

Handling Errors

Errors from NML can be dealt with more rigorously by catching NMLException.

Example: Catching an NMLException.

. . .  try { /* NMLConnection( message dictionary, buffer name, process name, configuration file ) */ NMLConnection example_nml = new NMLConnection(new nml_ex1MsgDict(),    "ex_buf1",   "ex3_java_proc",    "ex_cfg.nml");  EXAMPLE_MSG msg = null; while(msg == null) {   msg = (EXAMPLE_MSG) example_nml.read();   Thread.sleep(100);/* Sleep for 100 milliseconds. */ } } catch(NMLException nml_except) { System.out.println("Sorry can't read from NML."); return(-1); } return(0); . . . 

The Section called "Writing an NML Configuration File" has been moved to it's own page at http://www.isd.mel.nist.gov/projects/rcs_lib/NMLcfg.html.

Last Modified: Feb 1,2010

• See other RCS Library Documents.
• Go to the top of this document.

If you have questions or comments regarding this page or you would like to be notified of changes to the RCS library via email, please contact Will Shackleford at shackle [at] cme.nist.gov (shackle[at]cme[dot]nist[dot]gov)