The cmdmod wrapper

cmdmod is a wrapper that simplifies the development of Pyrame modules by mutualizing the network server and client, the XML and Pyrame protocol parsing and the control of public/private functions among others.

Figure 1 summarizes the inner workings of cmdmod.

Figure 1: Example of module Y interacting with modules X and Z. Module Y is run inside cmdmod

cmdmod’s usage is:

cmdmod config_file [-c cmod_ip]

On the following the config_file and the cmod functionality are described.

The XML config_file

config_file is an XML file with a main <config> tag describing a number of parameters of the module. The required parameters are the Python implementation file and the TCP listening port :

<config>
    <file>/opt/pyrame/cmd_test.py</file>
    <listen_port>9212</listen_port>
</config>

Optionally, a ports file (via the <port_base> tag) or functions (via the <cmd> tag) can be included. The ports file is usually /opt/pyrame/ports.txt, which includes all the stock Pyrame distribution ports. When the ports file is indicated, a non-numeric port can be used:

<config>
    <file>/opt/pyrame/cmd_test.py</file>
    <port_base>/opt/pyrame/ports.txt</port_base>
    <listen_port>CMD_TEST</listen_port>
</config>

Regarding the functions, depending on the functionality of the module and its comunication needs with others, a list of script or host functions can be included:

<config>
    <file>/opt/pyrame/cmd_test.py</file>
    <listen_port>CMD_TEST</listen_port>
    <port_base>/opt/pyrame/ports.txt</port_base>
    <cmd name="test_test" type="script">
        <function>test_test</function>
    </cmd>
    <cmd name="setvar_varmod" type="host">
        <host>localhost</host>
        <port>VARMOD_PORT</port>
    </cmd>
</config>

Script functions

The script functions are those that exist in the module and are intended to be public and thus callable from other modules. The <cmd> tag must include a public name and be of type=script. Inside the <cmd> tag a <function> tag must be included with the name of function on the Python implementation file. In most cases both will coincide for simplicity.

Host functions

The host functions are those that are implemented in other Pyrame modules and that the module might need to call at some point of its execution. The <cmd> tag must include the public name of the function on the external module. Inside the <cmd> tag, <host> and <port> tags must be included with the hostname (or IP) and the port of the external module. The port can be of text-type, if a <port_base> ports file is given and the desired port name is included on it.

The submod API

cmdmod provides plenty of functions to its modules. They are presented here by theme:

Basic functions

submod.execcmd(function@module[, parameter1[, ...]])

It allows the code to call an external Pyrame function on another module. The first parameter is the function name suffixed with the module name and the rest is the parameters.

Once the Pyrame command has been sent to the external module, the function waits for a response indefinetely. The response can be of any size within the system’s memory constrains.

The return value is a tuple of return numerical code and return string.

submod_exit(exit_code)

This function make cmdmod to exit gently and emitting the exit_code

submod_getport(module_name)

Return the numerical port value of a Pyrame module known by its name

submod_setmyname(newname)

Allow a module to rename itself. This operation is typically done during init function.

Environment

Cmdmod provides an environment to its modules. It is a storage where the module can store simple values (string, integer or flotting point numbers). The values are indexed by a name and a namespace. This allow to create objects (the namespace) and to have multiple variable inside (the names)

This environment is backuped on the disk at any modification. This way, if a module crash, it can be relaunched with its previous environment and thus the system recover from this very severe problem. The environment is automatically coupled to cmod which allow to syncrhonize the values of the module and the value of the xml configuration file. The programmer does not need anymore to register the object in cmod nor update the value of its parameters.

submod_setvar(ns, name, value, type)

store a variable in namespace “ns” with the name “name” with value “value” and of type “type”. This type has to be numerical (0 for string, 1 for int and 2 for float)

submod_getvar(ns, name)

get the value of a variable in a namespace

submod_gettype(ns, name)

get the type of a variable in a namespace

submod_delvar(ns, name)

delete a variable in a namespace

submod_newns(ns)

create a namespace

submod_checkns(ns)

check if a namespace exists

submod_delns(ns)

delete a namespace

submod_dumpns(ns)

dump the values of all variables of a namespace

submod_listns()

give the list of all available namespace

Dispatcher

Cmdmod includes a data dispatching system allowing to distribute online data along all other modules. The data are made of blocks of events. The module store these blocks in a circular buffer and send them to any requesting module. It can also describes its own data through a schema.

submod_init_dispatcher(nb_blocks, name, schema_str, port)

This function initialize the data dispatcher. - nb_blocks is the number of available block slots in the circular buffer - name is the name of the data source - schema_str is a string describing the data format in saf #TODOREF - port is the numerical value of the listening port. If undef, it is allocated automatically by the system

Note that only one dispatcher is available by module. Thus this function must be called only once.

submod_flush_dispatcher()

Remove any data from the circular buffer

submod_new_block(block_id)

Create a new block with block_id as identifier and return a handler for it

submod_set_field_block(block, field_name, value)

Fill a block with a data - block is the block identifier returned by submod_new_block - field_name is the name of a field as described in the schema - value is the value of the field

submod_new_event(block)

Create an event in the block and return a handler for it

submod_set_field_event(event, field_name, value)

Fill an event with a data - event is the event identifier returned by submod_new_event - field_name is the name of a field as described in the schema - value is the value of the field

submod_finalize_block(block)

Finalize a block: the block is checked for verifying that all fields has been properly filled and then add the block to the circular buffer.

submod_set_chans(schema)

Change the schema of the dispatcher

Asynchronous execution

To avoid all risk of deadlock, the Pyrame architecture allow a module to execute only one action at a time. In some very special case, it is necessary to execute asynchronous action. These two function allow that but be careful when using them because a deadlock can appears as soon as a module calls another module that calls him in return.

submod_bgcontext()

obtain a background context. Necessary to go to asynchronous mode

submod_sendres(context, retcode, retstr)

send the result of the execution when the asynchronous part of the function is executed. - context is the background context obtained previoulsy by submod_bg_context - retcode is the numerical return code - retstr is the return message

Note that the caller has been remained waiting during the asynchronous part. Only another module can ask for more function.

Here is an example code in Python for using asynchronous calls

def asyncwait_t(bg_context,ptime):
  print("waiting for %s seconds"%(ptime))
  time.sleep(float(ptime))
  print("sleep is finished")
  submod_sendres(bg_context,1,"ok")

#pyrapi:asyncwait_test:time
def asyncwait_test(ptime):
  retcode,res=submod_bgcontext()
  t=threading.Thread(target=asyncwait_t,args=(res,ptime))
  t.start()
  return KEEPOPEN,"ok"

When the command asyncwait_test is called, a thread is launched and the Keepopen code is sent. Then, the module is ready to execute other commands but the caller of the asyncwait_test function waits. In the thread, the action goes on and when finished the sendres define the result of the call. This retcode and retstr are then sent to the original caller.

CMOD functionality

cmdmod allows to extend further the host functions functionality provided by the XML by using the cmod module. It is therefore only used when calling external functions via submod.execcmd. It is enabled by using the -c cmod_ip command-line argument, with cmod_ip being the IP or hostname where cmod runs.

When using cmod, the hostname of the module receiving the function call is determined in realtime (and cached for subsequent calls) based on the first parameter of call, which must be the device_id assigned by cmod. The module being called must coincide in name with substring of the function after the last “_” character.

Example:

A module called chip is present in two network nodes and each manages different physical devices of the same type. The module chip registers their devices on cmod with type chip. The module implements a function init_chip(dev_id).

On this situation the host function scheme of the XML config_file is no longer valid, as the same function init_chip exists on two network nodes, and the selection of it depends on the first parameter of the function call.

When a Python code running on cmdmod wants to call init_chip(3), cmdmod will detect that this function is not present on the XML config_file and, if the cmod functionality has been enabled, it will try to resolve from cmod the network node to which address that function call, based on the first parameter (3) and the type of device (“chip”). The type of device is extracted from the function name, as the substring after the last “_”. Note that this forbids this functionality to work with modules that have “_” characters on their name.