<REVISED>

ROOTS Type System

ROOTS provides a type system based on Java types to persist any Java data, and to transport Java data between heterogeneous computer platforms. Here, we give a brief introduction to this type system and the supporting bundles of ROOTS.

The document is structured as follows:

1. ROOTS Type System
   1. Introduction
      1. Motivation
      2. Summary of Requirements
      3. Discussion of Implementation Options
      4. SERS Overview
   2. Major Bundles Involved
   3. SERS in Detail
      1. Type Names and Identifiers
      2. Pre-defined Types
         1. Base Types
         2. Multi-Value Base types
      3. Extension Types
         1. Step 1: Define Extension Type
         2. Step 2: Define Application Type Factory
         3. Step 3: Provide Extension Type to SERS
      4. Miscellaneous
   4. The Extension Type SersMap
   5. HINO
2. Appendix A: Byte Value Text Representation

Introduction

Motivation

One of the design goals of ROOTS was to provide a platform for event centric processing (short: ECP). ECP is based on the publish-subscribe-pattern is the core concept for any application programming with ROOTS. Besides others, ECP introduces the features persistence of events as well as an event process concept on top of single events.

For both features persistence of data is required. We decided to do persistence transparently for any event applications. Thus ECP applications are allowed to use any Java type for events and processes. The ECP implementation persists these data at relevant internal processing points without application interference.

Finally, another requirement was on the table: support of remote events. Events may be thrown at location one and processed at location two. That forces to support a interchange of event data between heterogeneous computers in a network. Again, this should work in a transparent way for applications.

Summary of Requirements

In the section above the motivation for the ROOTS type system was given. It stated the following requirements which are relevant for an implementation of the type system:

1. use of Java types
2. transparently persist Java type values
3. transparently interchange of type values based on remote communications between heterogeneous platforms

Since ROOTS is OSGi based, it is obvious that OSGi conformance is an overall requirement.

Based on these requirements a short discussion of implementation options is given in the next section.

Discussion of Implementation Options

Regarding the requirements 1 and 2 of the previous section, support is given by Java itself through the Serialization interface and the concepts behind. Any Java object may be persisted in a transparent way by this approach. Since it uses a binary format, requirement 3 is not well supported. Furthermore, since Java object serialization is known for bad performance we skipped this approach.

Regarding requirement 3 we took a look at JSON. JSON is a good solution for data interchange. It also supports requirements 1 and 2 in general. However, it lacks of

• inherent support of type information to guarantee type safety, and
• support for binary data

Finally, we decided to use a text based format of our own for the ROOTS type system. Currently, we name it SERS which is a acronym for Serialization by Strings. A more older name found in our documentation is 'StrObjects' for String Objects.
We not only use SERS as the name for serialized format, but, also as the name for the complete ROOTS type system.

SERS Overview

In SERS there is a distinction between pre-defined types and extension types. Pre-defined types include all Java base object types, i.e. the object form, and collections and arrays of base types. SERS is able to serialize and de-serialize pre-defined types without any additional effort from application developers side.

SERS provides a service interface for introducing application-defined types. Here, any Java type (class) can be introduced together with methods for converting object to string and vice versa.

Any type, whether pre-defined or not, has a unique type id in SERS.

Major Bundles Involved

In this section an overview of bundles related to SERS is provides. Bundles are as follows (here, + is shortcut for biz.kachel.roots):

SERS in Detail

The overview section at the beginning of this document gave a brief introduction to SERS. Here, SERS is documented in more details.

Type Names and Identifiers

Each type in SERS has a unique type name which is represented by a String value. These type names are called RSOID (Roots String Object IDentifier). Type names are unique across ROOTS Server instances (ROOTS server installation). However, since it is a string representation it is too long for some storage purposes. Therefore, the Short RSOID is introduced which is represented by a Long value. Short RSOID are only unique on a ROOTS Server instance, but may differ from instance to instance.

Pre-defined Types

Base Types

The following table provides all pre-defined SERS base types, their RSOIDs, and the text representation of type values:

For the concrete String representation for value.toString() see the Java documentation.

Multi-Value Base types

Collections or arrays of any of the SERS base types, except SOUnsupported, are also part of the SERS pre-defined types. The RSOID of a collection or an array is the concatenation of either SOArray or SOCollection followed by . and the RSOID of the base type.

Example: The RSOID of String[] is SOAray.SOString.

Extension Types

Any ROOTS application may define its own SERS type, called extension type. Once declared, extension types can be used by SERS resp. applications using SERS.
The definition of an extension type consists of three steps. As an example for these steps the type SersMap is used.

Step 1: Define Extension Type

Each extension type must implement the StrSerialzed interface. In case of SersMap it looks like the following.

public class SersMap<T extends Object> extends Hashtable<String, T> implements StrSerialized

The interface is defined as follows:

/**
 * Common interface for all classes implementing serialization by string (SERS) objects.
 * For any class implementing this interface also a factory have to be provided implementing StrSerializedFactory.
 * A StrSerialized type has an type identifier, the RSOId (Roots string object type identifier).
 * The string value format of StrSeruialized type is <RSOId>:<value>.
 */
public interface StrSerialized {

	/**
	 * Provides the object value as string.
	 * @return object as string.
	 * @throws UnsupportedType in case the object is not well initialized.
	 */
	public String toStr() throws UnsupportedType;
	
	/**
	 * Reads object from string.
	 * @param objectValue string representation of the object as provided by toStr().
	 * @throws UnsupportedType in case obejctValue does not represent this type.
	 */
	public void fromStr(String objectValue) throws UnsupportedType;
	
	/**
	 * Provides the unique ROOTS SERS type identifier.
	 * @return ROOTS SERS type identifier.
	 */
	public RSOID getRSOId();
		
}

The JavaDoc of the interface definition declares the meaning of the interface methods. toStr() and fromStr() have to be implemented in the way that the operations must be inverse for each other. The sequence object.fromStr(object.toStr()) must keep the object value as is.

SERS uses the interface methods to perform string serialization operations and be aware of the type without knowing any implementation details. The RSOID provided by the interface is application defined and must be unique. It is up to conventions to define type name strategies (if nothing defined use full qualified class names for RSOIDs). However, type names are restricted in the way that they are not allowed to contain a '.'.

The RSOID provided by the interface is a plain name, e.g. MyType. Internally, SERS uses an expanded, full qualified name representation with SOSerialized as prefix. In this example it will be SOSerialized.MyType.

Step 2: Define Application Type Factory

Besides plain extension types, arrays and collections of these types are also allowed. To give SERS the ability to create objects and arrays or collections of objects on its own, the extension type factory interface is introduced as follows:

/**
 * Factory interface for SERS (Serialization by String) types.
 * @author gerd
 *
 */
public interface StrSerializedFactory {
	
	/**
	 * Service property name. 
	 */
	public static final String RSOID_SERVICE_PROPERTY_NAME = "RSOID"; 
	
	/**
	 * Creates a new object of the SERS type.
	 * The object is provided by its StrSerialized interface.
	 * @return new object created.
	 */
	public StrSerialized create();

	/**
	 * Creates a new empty object array of the SERS type.
	 * The object array is provided by its StrSerialized interface.
	 * @param dimension the dimension of the array.
	 * @return new object array created.
	 */
	public StrSerialized[] createArray(int dimension);

	/**
	 * Creates a new empty object collection of the SERS type.
	 * The object collection is provided by its StrSerialized interface.
	 * @param initialSize the initial size of the collection.
	 * @return new object collection created.
	 */
	public Collection<StrSerialized> createCollection(int initialSize);

}

For the type SersMap the factory definition is:

public class SersMapFactory implements StrSerializedFactory {

	/**
	 * Constructor.
	 */
	public SersMapFactory() {
		super();
	}

	/* (non-Javadoc)
	 * @see biz.kachel.roots.strobjects.StrSerializedFactory#create()
	 */
	@Override
	public StrSerialized create() {
		return new SersMap<>();
	}

	/* (non-Javadoc)
	 * @see biz.kachel.roots.strobjects.StrSerializedFactory#createArray(int)
	 */
	@Override
	public StrSerialized[] createArray(int dimension) {
		return new SersMap[dimension];
	}

	/* (non-Javadoc)
	 * @see biz.kachel.roots.strobjects.StrSerializedFactory#createCollection(int)
	 */
	@Override
	public Collection<StrSerialized> createCollection(int initialSize) {
		return new ArrayList<>(initialSize);
	}

}

As you see the implementation of the interface methods is very straight-forward.

The use of this factory is content of the next section.

Step 3: Provide Extension Type to SERS

The extension type is provided to SERS by defining an OSGi service which provides the factory interface StrSerializedFactory together with the service property name RSOID holding the type name as value. SERS implementation listen on the service registry and tracks defined type on the fly.

By using the OSGi service annotation and component system of ROOTS the definition of the SersMap extension type looks as follows.

For the service definition:

	@Component(type = Type.COMPONENT)
	@Provides(interfaces={StrSerializedFactory.class})
	public class SersMapFactory implements StrSerializedFactory

For the type instantiation within the bundle activator:

		Dictionary dic = new Hashtable<>();
		dic.put(StrSerializedFactory.RSOID_SERVICE_PROPERTY_NAME,
				SersMap.MY_RSOID.toString());
		this.instantiate(SersMapFactory.class, dic);

Miscellaneous

For implementing own extension types there is a set of serialization utilities provided by the package biz.kachel.roots.strobjects.util.

The Extension Type SersMap

In the previous section SersMap was used as an example for extension type definitions. SersMap is the key extension type in ROOTS. It provides a string serialized implementation of Java class Hashtable with a string key. By that it can be used in many Java places where objects of type Map or Dictionary are required. On the other hand, SersMap can be constructed using a Map or a Dictionary object. Thus, it is the a big bridge between Java and SERS.

You will find that SersMap is used in multiple ROOTS places like HINO (see below) or the remote system. For the latter, it is of value to know the string representation of SersMap to allow non Java clients to access ROOTS remotely.

The SersMap implementation uses the utility class StringSerializationByLE provided in package biz.kachel.roots.strobjects.util. The text representation of a SersMap is as follows:

1. Each property value pair is stored in one text line (ends with EOL character).
2. A property value pair is mapped to text by using a separator ':'. The mapped result is <property name>:<short RSOID>:<string serialized value>.
3. Occurrences of ':' in either names or string serialized values are masked by '/:/'.

HINO

HINO (Hierarchical Nodes) is the ROOTS database system. It is a hierarchical node database where any node consists of a name and a set of key value pairs. At the HINO interface each type is allowed which is known by SERS. Besides that there is the so called HINO Node Model (shortly HNM) which is also an extension type (with RSOID = HNM). HNM is a type to define HINO data transiently.

TODO: Add HINO reference as far as HINO documentation is available.

Appendix A: Byte Value Text Representation

Byte data is represented as text by an own calculation which performs better than known codings like BASE64. Furthermore, it keeps mostly human readable for byte data representing text itself. Calculation is done as follows.