Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Version 2.0 NML Configuration File Format and Tool

The Version 2.0 NML Configuration File Format and Tool

Background

The version 1 configuration file format described in "Writing NML Configuration Files" has remained stable since 1995 except for some rarely used options that can be placed at the end of buffer and process lines . That format is also the only format that NML constructors can currently use directly and it is the format generated by the NML Configuration File Builder (a CGI script and HTML form) and by the RCS Design Tool (a Java application).

In May 1999, the configuration format described here was proposed. The goals for this new configuration format were the following:

  • Easier for multiple organizations to share configuration files.
  • Less verbose files made possible by allowing users to specify only items that need to be different than the default.
  • Fewer errors caused by novice users.
  • Eliminate some tedious tasks.

The version 2 configuration format is supported only through a tool that takes files written in the new format and produces files in the older format. These files can then be used in an application in the same way as files written by hand in the older format or generated by one of the graphical tools.

The reader for this document is expected to be familiar with either C++ or Java, the NML Programmer's Guide, C++ Version/Java Version, and with the the older configuration file format.

Configuration File Line Types

The new configuration files are ASCII text files, that contain eight different types of lines that can be idenfied by the first character or word on the line:

Starting Word or Character

Line Description.

#

Comment Lines starting with a single '#' have no effect on the output file. Comment lines starting with "##" are also placed in the output file.

##

Insert lines have the same effect as regular comment lines except that they are inserted as comments in the output file.

include

Include lines cause the contents of another file to be read as if the text were included at that point in the original file.

define

Definition lines define a variable that can be used through the rest of the file, by preceding the variable name with "$" and surrounding it with parentheses "( )" like this "$(varname)" .

define

Definition lines define a variable that can be used through the rest of the file, by preceding the variable name with "$" and surrounding it with parentheses "( )" like this "$(varname)" .

format_source_pattern

Update February-2009: Sets a pattern that will be used to determine the format_source from the format name by replacing the first "%s" in the pattern with the format_name.

format_header_pattern

Update February-2009: Sets a pattern that will be used to determine the header from the format name by replacing the first "%s" in the pattern with the format_name.

buffer_default

Buffer Default lines set defaults that affect buffer lines that occur after this line.

b

Buffer lines begin with must contain "name=" and the name of the buffer somewhere on the line. These lines are used to create and entry for a particular buffer.

process_default

Process Default lines set defaults that affect process lines that occur after this line.

p

Process lines must contain either "name=" or "bufname=" somewhere on the line and are used to create an entry linking a particular process to a particular buffer. For proper checking process lines must occur after the buffer line for the buffer the process is connecting to.

Buffer Variables

The following variables with the exception of "name" can be set either on a buffer line to modify only one buffer or on a buffer default line to affect several buffers:

name
The name used as an argument to the NML constructor in the C++ or Java code. It must be unique. There is no default value, this variable must be set on each line with a "b ".
buftype
The type of buffer to create which can be shmem,globmem,filemem,locmem or phantom. The default value is shmem.
host
The host name where a server must be run if any processes are going to connect remotely. The default value is localhost.
size
The size of the largest message that can be sent to the buffer. The amount of memory allocated will be slightly larger than this to accomodate some handshaking flags. The default value is 960.
neutral
Whether the local buffer should be neutrally encoded. Buffers should be neutrally encoded if they can be accessed by multiple CPU types by a Bit3 adaptor for example or to force messages to go through format and update functions that significantly reduces message size such as when variable length arrays are used. Messages are always neutrally encoded when sent over a network. The default value is false.
bufnumber
A unique number used to identify the buffer within a server. The default value is calculated based on the position of the buffer line in the file. If its default value is modified with a default buffer line the default value for subsequent lines will still be incremented from this starting value.
max_proc
The maximum number of processes that can connect to a buffer locally. It does not affect remote processes or shmem buffers using the default mutual exclusion mechanism. The default value is calculated based on the number of processes connecting to this buffer. Update February 2005: The number is ignored unless GLOBMEM or mutex=mao is specified.
key
A unique number used to identify the shared memory and semaphore used for mutual exclusion in a shared memory buffer. It is relavent when using the "ipcs" or "ipcrm" commands. The default value is calculated based on the position of the buffer line in the file.
bsem
A unique number used to identify the semaphore used for blocking reads. Ther default value is -1, so blocking reads are not allowed by default. However if the value is changed with a default buffer line then subsequent lines will increment this starting value.
vme_addr
The VME address used for GLOBMEM on a VME backplane. The default value is 0 which is unusable. However if the default value is changed with a default buffer line then subsequent buffers will use the sum of this values plus the size of the preceding buffers.
remotetype
The protocol that should be used by remote processes connecting to this buffer which could be tcp,stcp, or udp. The default value is tcp.
port
The TCP or UDP port used by remote processes. The default value is 30000.
enc
The nuetral encoding method which can be xdr,ascii,or disp. The default value is xdr.
queue
Whether messages in this buffer should be queued. Setting it to a value greater than 1 also multiplies the size of the buffer by this value.The default value is 0.
diag
Whether to enable supplemental timing diagnostic information to be logged to the buffer. ( See Supplementary NML Timing Diagnostics Tools.)
format_name
Update February-2009: Sets the name of the format function that should be used with the buffer. The format name should not include the final "_format" If the NML constructor should have been passed ex_format as the first argument, the tag in the nml file "format_name=ex" would accept this. If the format name does not match the process will print an error message, NML::valid() will return false, and NML::error_type will be set to NML_FORMAT_NAME_DOES_NOT_MATCH_ERROR. If format_source_pattern or format_header_pattern is used then the format_name and either format_source or header was not set explicitly the format_name will be used to determine them. If the size is not set explicitly it can be determined by searching for the comment with the line "Estimated_size MAXIMUM" followed by a size to use for the buffer in the generated source file. "format=" is also accepted as a shorter verion of "format_name=".
format_source
Update February-2009: Sets the name of the C++ file with the autogenerated format and update functions that should be used with the buffer. If the size is not set explicitly it can be determined by searching for the comment with the line "Estimated_size MAXIMUM" followed by a size to use for the buffer in the generated format source file.
header
Update February-2009: Sets the name of the C++ header file with the definitions of the message classes that should be used with the buffer. If neither the format_source nor format_name is set explicitly the format source will be assumed to be the base of the deader name plus "_n.cc". If the size is not set explicitly it can be determined by searching for the comment with the line "Estimated_size MAXIMUM" followed by a size to use for the buffer in the generated format source file.

There are some additional flags that were available in the old configuration file that the new tool will not recognize. This will produce a wargning but the old flag should simply be pasted to the end of the generated buffer line which should allow the use of the unrecognized flag.

Process Variables

The following variables can be place either on a process line to affect only one process connection to one buffer or on a default process line to affect multiple connections:

name
The name of the process that is passed to the NML constructor in the C++ or Java constructor. There is no default value, this variable must be set. Update February.2005: The special value of "default" will match any name. The paremeters on this line affect any process not specifically mentioned earlier in the file. It is different than setting a process_default parameter in that setting a process default affects all processes subsequently named while this typically affects processes not mentioned in the file at all.Setting both the process name and buffer name to default matches any combination.
bufname
The name of the buffer that this process is connecting to. There is no default value, this variable must be set. Update February.2005: The special value of "default" will match any buffer name. The paremeters on this line affect any buffer for this process not specifically mentioned earlier in the file. It is different than setting a process_default parameter in that setting a process default affects all buffers subsequently named while this typically affects buffers not mentioned in the file at all. Setting both the process name and buffer name to default matches any combination.
proctype
The type of process which can be local, remote or auto. Local processes use some form of shared memory directly, remote processes go through TCP, or UDP and require a server. Processes set to auto will attempt to determine if direct shared memory access is possible by comparing IP addresses.Update February 2005 The default is auto.
host
The name of the host this process is running on. It is currently only used to comment the output file. The default value is localhost.
ops
The operations allowed by this process on this buffer which can be r, w,or rw. (For READ_ONLY, WRITE_ONLY, and READ_WRITE respectively) The default value is rw.
timeout
The time in seconds to allow before this process should timeout waiting for a read or write specified as a double or "INF" to indicate infinity.The default value is infinity.
master
Whether this process will be the master of this buffer. The master creates and clears the buffer when it is started. The default value is false.
server
Whether this process will act as a server for this buffer. The value of 2 has the special meaning that the process will spawn a server but then continue to access the buffer locally. The default value is false or 0.
c_num
The connection number which should be unique among processes connecting to the same buffer locally. The default value is calculated based on the number of processes that connected to this buffer before. Update February 2005: The number is ignored unless GLOBMEM or mutex=mao is specified.
sub
The subscription interval in seconds for remote processes. The default value is -1 which indicates no subscription.

Tool Command Line Arguments

The program "nmlcfg" converts files from the new format to the old format which must be done because the NML constructors can not currently read the new format.

All command line not preceded by one of the following flags is considered to be input files that should be in the new format. If more than one input file is specified they are read in the order they occur and only one output file will be produced.

Here are the additional flags:

-D name=value
Defines a value on the command line just as if a define line was used.
-I directory
Adds a directory to a list of directories to be searched if a include line is found.
-o output_file
Specified the file where the old format file will be written.

Examples

The following example includes a separate file that can be modified to change the host name, 2 buffers, and 3 processes connecting to it.

The file myhosts contains the following.

# Set host aliases. define host1=localhost 

The file newnmlcfgex.cfg contains the following:

include myhosts  # Set the pattern used to find the header file by: # Replace %s with the value from format_name to find find the main # C++ header file associated with the buffer that prototypes the format function. format_header_pattern %s.hh  # Set the pattern used to find Estimated_Sizes and format  # function implementation. # Replace %s with the value from format_name to find the main # C++ file associated with the buffer with the autogenerated format function # implementation. # if the next line is not recognized you need atleast version  format_source_pattern %s_n.cc  buffer_default host=$(host1) b name=ex_cmd format_name=exCmd b name=ex_stat format_name=exStat  process_default server=1 master=1 proctype=local name=mysvr p bufname=ex_cmd p bufname=ex_stat  process_default server=0 master=0  process_default name=pl p bufname=ex_cmd p bufname=ex_stat   process_default name=ex p bufname=ex_cmd p bufname=ex_stat  ## Double pound comment get preserved in output config file. ## Special default process line that will be used by any process that does not match above. p bufname=default name=default proctype=auto server=0 master=0  

The old style/output config file generated is this newnmlcfgex.nml.

# newnmlcfgex.nml    # Buffers: #       name    type        host size neut 0 buf# max_proc . . . B     ex_cmd SHMEM         localhost 128 0 * 1 * 35001 TCP=20001  format_name=exCmd format_source=exCmd_n.cc header=exCmd.hh packed B    ex_stat SHMEM         localhost 256 0 * 2 * 35002 TCP=20002  format_name=exStat format_source=exStat_n.cc header=exStat.hh packed    # Processes:  #       Name     Buffer       type       host        ops server timeout master c_num   . . . P      mysvr     ex_cmd      LOCAL  localhost         RW 1 INF     1 0  P      mysvr    ex_stat      LOCAL  localhost         RW 1 INF     1 0    P         pl     ex_cmd      LOCAL  localhost         RW 0 INF     0 1  P         pl    ex_stat      LOCAL  localhost         RW 0 INF     0 1    P         ex     ex_cmd      LOCAL  localhost         RW 0 INF     0 2  P         ex    ex_stat      LOCAL  localhost         RW 0 INF     0 2   # Double pound comment get preserved in output config file. # Special default process line that will be used by any process that does not match above. P    default    default       AUTO  localhost         RW 0 INF     0 0  
The files exCmd.hh,exCmd_n.cc, exStat.hh and exStat_n.cc also had an effect on the output.  

Last Modified: February-2009

If you have questions or comments regarding this page or you would like to be notified of changes to the RCS library via email, please contact Will Shackleford at shackle [at] cme.nist.gov (shackle[at]cme[dot]nist[dot]gov)

Created July 11, 2014, Updated June 2, 2021