BUTLER Device Data eXchange (DDX)

DDX stands for Device Data eXchange and works as a BUTLER SmartServer implementing or using some of the functional components defined in the BUTLER Architecture, namely the Data Management and Marketplace Portal, the Resource and Context Exposition, the Complex Event Processing engine and the Permanent Storage. One of the functionalities provided by BUTLER is the possibility of accessing exposed data generated by BUTLER SmartObjects. This exposed data can be roughly described as a continuous flow of values of a given Resource. Such functionality enables third-party developers to create new applications on top of BUTLER. BUTLER addresses the exposition and purchase of BUTLER SmartObjects data by said third-parties through a horizontal BUTLER SmartServer, Device Data eXchange, that mediates between SmartObjects and applications. This functionality aims to enable Service Developers to discover and use data from SmartObjects through a single discovery point. It also addresses the management of simple events that Service Developers can define taking SmartObject data as input. That way, Service Developers can define a set of conditions that take SmartObject data as input and trigger actions whenever the conditions are met. Such simple events are named Business Events. Two main roles are thus defined: the aforementioned Service Developer and the so-called Device Owner:
  • Device Owners are the actual owners (or administrators) of SmartObjects. They are the ones that make a decision on sharing with third-parties the information coming from their SmartObjects.
  • Said third parties, Service Developers, are in turn those wishing to use data coming from SmartObjects to create new services and applications.
This multi-sided platform:
  • Provides Organized, real-time and ready-to-use data from devices and sensors
  • Enables the creation of a M2M ecosystem
  • Brings together Data Owners and App Developers
  • Resulting in new revenue streams

Intellectual property rights (IPR)

The components will be provided a full open source.

Ericsson España S.A.U. will grant access to these components for the one-year period after the finalization of the project and provide support for non-commercial purposes. It will also offer commercial agreements to firms interested in exploiting these components for their businesses.

Introduction

DDX platform comprises two main functional modules: the Real-Time Data Marketplace –the front end of the solution composed by the Data Owner Portal (DOP), and the Dynamic Data Mall (DDM)–, and the DDX Data Broker –the data communication and processing module–.

DDX Functional Components

DDX Functional Components

Basics

Definitions

The following concepts are handled throughout the whole user guide and therefore should be known beforehand:

  1. Sensors: Basic sources of real-time information. A sensor is always attached to a device that provides an addressing mechanism and a way of communication.
  2. Devices: Hardware unit that hosts a group of sensors using a common location and communication technology.
  3. (Virtual) Entities: Representation of a physical entity in the physical world that can be usually drawn on a map, therefore containing geographical information. They can be nested. Several devices can be associated to one entity. Each entity is handled and owned by a Data Owner, who is responsible for its management (creation, device association, removal…).
  4. Packages (Offering): Sensor or group of sensors offered by the Data Owner to the Service Developers through the Dynamic Data Mall. Packages do not depend on entities. However, a set of sensors associated to a device or to a virtual entity can be offered as a package.
  5. Data stream: The set of data items continuously issued by a sensor.
  6. Business Rules: They are statements that define a set of complex event processing conditions regarding the data items sent by the sensors that, when accomplished, trigger a Business Event.
  7. Business Event: A set of actions that must be executed when some conditions (a Business Rule), defined on several sources of real-time device data, are met.
  8. Context: It is an abstraction associated to a virtual entity. It comprises a number of attributes that describe a virtual entity. Each attribute can be built by combining data from different data streams.
  9. Real-Time Data Marketplace: The “management” plane of the BUTLER DDX SmartServer. It enables Data Owners and Service Developers to interact with the BUTLER DDX SmartServer in order to define, search and purchase data offerings. It comprises two main components, the Data Owner Portal (DOP) and the Dynamic Data Mall (DDM).
  10. Data Owner Portal (DOP): The web site where Data Owners configure their real time data offerings.
  11. Dynamic Data Mall (DDM): The web site where Service Developers select and purchase real-time data offerings (packages) they wish to use to create new services.
  12. Data item: The information unit sent by the data sources. They usually contain measures.
  13. DDX Data Broker: The “traffic” part of the BUTLER DDX SmartServer. It handles actual data brokering and processing. Data Consumers are the clients of the DDX Data Broker.
  14. Data Owners: One of the main roles supported by the BUTLER DDX SmartServer. Through the Data Owner Portal, Data Owners create and activate real-time data offerings (packages) that will be later on offered to Service Developers.
  15. Service Developers: One of the main roles supported by the BUTLER DDX SmartServer. Service Developers look for and discover real-time data offerings that match their requirements, subsequently they use (and pay if needed) them so that they can be used in the applications and services they create (Data Consumers). They will be able to browse the Data Owners’ offers and purchase packages through the Dynamic Data Mall.
  16. Data Consumers: These are the software entities (services and applications) that actually consume data items provided by the BUTLER DDX SmartServer. They are created by Service Developers and inherit their permissions (that is, once a Service Developer has purchased a given Package or created a Business Event, his/her services can receive the data provided by the BUTLER DDX SmartServer).
  17. End user: Users that purchase and use the applications and services implemented by the Service Developers.

Functionalities

The BUTLER Device Data eXchange (DDX) SmartServer aims to become a horizontal enabler that bridges the gap between real-time data sources, such as those associated to devices, and the application world. Although the scenarios where DDX is applied and the players in the value chain may impose different constraints to real deployments, the main functionalities associated to DDX remain the same:

  • On the one hand: data owners (owners of M2M devices and other real-time data sources, primarily deployed to support its business needs) can expose such information and get additional revenues from it. Offering of data associated to devices is done through the Data Owner Portal (DOP). The functionalities and procedures available to data owners are described in section 3.1. 3.2.
  • On the other hand: service developers are offered a single point of access to find and use data sources, simplified API’s for exposed data, critical mass of data sources, means for the discovery of data sources (by feature and/or geographical location), and even pay-per-use procedures. It is donethrough the Dynamic Data Mall (DDM). The functionalities and procedures available to data owners are described in section 3.2.

Getting started

Data Owner Portal (DOP)

The Data Owner Portal is currently available at http://195.53.58.184:8080/dopbutler. It has been optimized for Mozilla Firefox web browser. A User ID and a password are needed to access the Data Owner Portal functionality. The home page can be seen in the figure below:

Data Owner Portal main page

Data Owner Portal main page

To access the DOP, registration is required. The following web form must be filled in to get registered (see next figure).

Data Owner registration form

Data Owner registration form

Data Owner registration and access to main menu

Once registered or authenticated, the user will access the DOP main menu (see figure about the Data Owner Portal main menu, below). Data Owners are thus enabled to manage the devices they actually “own” (or are entitled to manage). Management options comprise different levels of granularity: data sources (sensors) hosted by devices, (virtual) entities (representing physical entities in the real world) and data packages (the basic offering unit provided by Data Owners to Service Developers through the Dynamic Data Mall).

The options available on the left pane of the DOP main menu are the following:

  • Devices. This option enables the Data Owner to manage the data sources (sensors) associated to their devices.
  • Entities. This option enables the Data Owner to manage representations real-world entities (virtual entities) and link such real data sources to virtual entities.
  • Packages. The actual data offerings a Data Owner wishes to offer are managed through here.
  • Storage Management. It enables if and how storage of data from devices is achieved in the BUTLER DDX platform.
  • Overview. A description of the functionalities available to Data Owners is provided here.

In the top right corner of the window the user’s name (‘Lars’ in the example below) and a disconnection icon are shown. Additionally, a graphical view of the available entities, devices and sensors belonging to a user is shown on the right pane (see section 3.1.2.4).

Data Owner Portal main menu

Data Owner Portal main menu

Device Management

Device Management is accessing by selecting the following option in the DOP menu:

Devices

Devices

This option enables a Data Owner to manage data sources (sensors). Provided that the Data Owner has been granted appropriate permissions regarding his/her sensors or devices, they are enabled to tag their sensors, to access the current sensor features, to verify the availability of the sensor, and to manage the lifecycle of the devices.

The Device Management functionality relies on Google Maps. Once authenticated the user accesses the following window. From there s/he can access the Device Set-up window, by selecting ‘Add device’.

Device Management window

Device Management window

Registration of devices

When the ‘Add device’ option is selected, registration of devices can be carried out. It is done in the following way:

  • Step #1: Click on the option ‘Add Device’ in the left side bar
  • Step #2: Type a device name and a set of device tags (tags separated by commas)
  • Step #3: Select a Parent entity if desired (see section 3.1.3 to know how to manage entities).
  • Step #4: Set the device location in the map
  • Step #5: Select ‘Save’ in the left side bar

The device is created and shown on the map as a map pin.

Device Set-Up window

Device Set-Up window

Once created, the left side pane provides the following information (see figure below):

  • Device Name: the one provided in the previous step.
  • Device ID: an internal numerical ID created by the Data Owner Portal. It is unique among all the device identifiers handled by the BUTLER DDX SmartServer.
  • Parent: whether the device is associated to an entity or not. If associated, the name of the entity is shown.
  • Device tags: the ones provides in the previous step.
Device Information window

Device Information window

It is possible to delete a device as well by clicking on the trash bin icon next to the Device name.

Batch registration of devices

When the amount of involved devices is large, it is not practicle to handle them on a one-by-one basis. The DOP interface supports also a mechanism for batch upload of device descriptions. It is carried out by dragging and dropping, in the same Device Set-up window described in section 3.1.2.1, a file describing a set of entities, devices and sensors on the map-based pane (see the figure below).

The file must follow a specific XML format (see below for a description). The user takes a file from the files explorer and drops it on the right pane of the window (that one where the map is shown):

Batch Device Registration window

Batch Device Registration window

An example of an XML file describing a group of devices as used as input for the drag and drop functionality can be seen below:

<definition>
<entities>
<entity>
<name>Mother</name>
<color>#DF3E3E</color>
</entity>
<entity>
<name>Child</name>
<color>#535DDF</color>
<devices>
<device>
<name>First</name>
<location>
<latitude>40.474107</latitude>
<longitude>-3.632129</longitude>
</location>
<tags>
<tag>First</tag>
<tag>device</tag>
</tags>
<sensors>
<sensor>
<name>Thermometer</name>
<magnitude>Temperature</magnitude>
<units>Celsius degrees</units>
<refreshrate>30000</refreshrate>
<tags>
<tag>sensor</tag>
<tag>thermometer</tag>
<tag>celsius</tag>
<tag>temperature</tag>
</tags>
</sensor>
<sensor>
<name>Anemometer</name>
<magnitude>Wind speed</magnitude>
<units>Km-h</units>
<refreshrate>20000</refreshrate>
<tags>
<tag>anemometer</tag>
<tag>sensor</tag>
<tag>wi       nd</tag>
<tag>speed</tag>
<tag>kilometer</tag>
<tag>hour</tag>
<tag>km-h</tag>
</tags>
</sensor>
</sensors>
</device>
<device>
<name>Second</name>
<location>
<latitude>40.394709</latitude>
<longitude>-3.67460</longitude>
</location>
<tags>
<tag>Second</tag>
<tag>device</tag>
</tags>
<sensors>
<sensor>
<name>Hygrometer</name>
<magnitude>Relative humidity</magnitude>
<units>percentage</units>
<refreshrate>10000</refreshrate>
<tags>
<tag>sensor</tag>
<tag>hygrometer</tag>
<tag>percentage</tag>
<tag>relative</tag>
<tag>humidity</tag>
</tags>
</sensor>
<sensor>
<name>Photodetector</name>
<magnitude>Luminosity</magnitude>
<units>Lux</units>
<refreshrate>30000</refreshrate>
<tags>
<tag>sensor</tag>
<tag>photodetector</tag>
<tag>luminosity</tag>
<tag>lux</tag>
</tags>
</sensor>
</sensors>
</device>
</devices>
</entity>
</entities>
</definition>

Review of device information

When the user gets authenticated and accesses the Device Management option, s/he is able to review the features of already registered devices by checking the list provided on the left pane or by navigating through the map on the right side and clicking on the selected device. When any of the elements in the list or the map pins representing the devices is selected, its name, id, parents and tags are shown (in the same way as with the Device Information windos).

Registration of sensors

Once one or more devices have been registered, it is possible to attach individual sensors to any of said devices. In order to attach individual sensors to devices, the device must be selected from the list on the left-hand pane or by picking up a device from the map. Then, the option ‘Add Sensor’ on the left side bar should be selected. A menu appears on the left pane enabling the user to introduce the sensor name, magnitude, units, tags and refresh rate.

Sensor Attachment window

Sensor Attachment window

Magnitude and units are similar to the tags, as they have informational purposes (NOTE: please do not use special characters like µ, £, €, etc). Refresh rate means the estimated frequency the sensor will send a data item. This information is useful for the lifecycle color code:

  • Grey: The device is registered in the BUTLER DDX SmartServer but none of its attached sensors have sent any data yet.
  • Red: None of the sensors attached to the device is working properly (i.e, they are sending data items with a lower frequency to the one established by the refresh rate).
  • Green: At least one of the sensors attached to the device is sending data items according the refresh rate (or with an even higher frequency).

When a sensor has been sending data items for a while, it is possible to monitor its activity through a real-time chart. If the Data Owner clicks on the sensor name on the left side pane (see Graphical Device Description window in section 3.1.2.7), the Sensor Description window shows the chart, as depicted in the figure below.

Sensor Description window

Sensor Description window

Connection of devices

In order to inject the devices’ data streams into the BUTLER DDX platform, it is necessary to implement a layer that adapts the data model of the incoming data items. The BUTLER DDX platform will receive and read correctly every text stream sent through a socket connected to the address 195.53.58.184, port 54444, where data items are sent following this format:

[DeviceId]:[SensorId] [Data]

Where [DeviceId] is the device identifier (an integer that uniquely identifies each device; it is generated whenever a new device is registered by the Data Owner in the Data Owner Portal); [SensorId] is the sensor identifier (an integer that uniquely identifies each sensor; it is generated when a new sensor is registered by the Data Owner); and [Data] is the measure sent by the sensor and passed on by the data item. It can be an integer or a decimal number. It is worthy to note that there is an empty space between [SensorId] and [Data]. Both identifiers can be checked at the device window.

Further information can be found in the first example.

Sensor simulator

In the case of no real data sources availability, a Java application that plays the role of data source simulator is provided in the DOP. In the Device Management window there are two buttons below the top menu bar. One of them allows accessing the Simulator guide (see Annex A). The other one can be used to download a Java file implementing the simulator.

Graphical view

To provide an extended view of all available devices, detailed information about the magnitude and units, and to have a graphic view as well, the data owner can select at any time the following icon on the right side of the top menu bar:

Graphical View

When selected, an interface such as the one in the figure below is shown:

Graphical Device Description window

Graphical Device Description window

Below, the left pane in the map window has been zoomed in in order to show a detailed graphical view (see figure below).

Detailed Graphical View

Detailed Graphical View

The graphical view shows the relationships between entities, devices and sensors already created in previous sections.

Entity management

This option enables Data Owners to manage virtual entities. The Data Owner is expected to freely define an entity using a map-based web interface and assign it a name, an entity type, a geolocalized shape and tags for subsequent discovery. There are two ways to access this functionality:

  • By selecting the option ‘Entities’ in the top menu bar in the Device Management section.
  • By selecting the following option in the DOP main menu:

Entities

 

Creation of entities

Once inside in Entities section, the window in the figure below appears. There, the Data Owner is free to create an entity.

Entity Management window

Entity Management window

The procedure is as follows:

  • Step #1: Select the option ‘Add entity’ on the left side bar;
  • Step #2: Define a name to the entity;
  • Step #3 (optional): Select a color;
  • Step #4 (optional): Click on ‘Add’;
  • Step #5 (optional): Define the entity’s area on the map;
  • Step #6: Save the new entity;

The entity is created and shown on the map as a polygonal shape.

Entity Set-Up window

Entity Set-Up window

As seen in the figure above, it is possible to switch between the Entity and Device functionalities by picking up the appropriate option in the top menu bar.

When the new entity is saved, the following window is shown:

Entity Information window

Entity Information window

Entity hierarchy

It is possible to create entity hierarchies (that is, to make an entity a parent of other).

First of all, the parent entity must have been created before establishing the parent-son relationship. There are two ways of establishing said relationship. The first one is at the entity creation time. Once the entity has been saved, the Entity Information window appears. In the left side bar there is an ‘Edit’ option that, when selected, allows the user to set the parent of the just created entity. The Entity Set-up window appears and a drop-down list with the names of all the entities created by the Data Owner is shown. The Data Owner can select one of the options and click ‘Save’.

Additionally, it is possible to carry out the same procedure from the Entity Management window. When the user selects the ‘Entity Management’ option, the Entity Management window shows the available entities (both as a list in the left side bar and as polygonal shapes on the map, see the figure below).

Entity Management window (with existing entities)

Entity Management window (with existing entities)

From there, the user can select one of the existing entities (either from the list or by picking up one of them on the map). When an entity is selected, the option ‘Edit’ on the left side bar has to be selected. The window in the figure below appears. As with the previous possibility, a drop-down list with the names of all the entities created by the Data Owner is shown. The Data Owner can select one of the options as parent and click ‘Save’.

Entity Set-Up window (sub-entities)

Entity Set-Up window (sub-entities)

Data Package management

As mentioned before a group of sensors is called a package. This group of sensors could be provided by the Data Owner or created by the very Service Developer.

Package creation

Data Package Management is chosen by selecting the following option in the DOP main menu:

Packages

Once selected the window in the figure below appears.

Package Management window

Package Management window

 

It shows the available entities and sensors. The procedure to create a package is as follows:

  • Step#1: Select ‘Add package’
  • Step#2: Type name and package description
  • Step#3: Select the sensors the package will comprise from the right pane, by dragging and dropping into the description pane
  • Step#4: Select ‘Save’
Package Creation window

Package Creation window

Package information

Once a package has been created, all the Data Owner’s packages are listed on the left side pane. By selecting ‘+Info’ on any of them the Data Owner can review the information available about the package.

 

Package Information window

Package Information window

 

Storage Management

Storage Management enables the Data Owner to create custom rules determining storage criteria. It is accessed by selecting the following option in the DOP main menu:

Storage Management

Data sent by sensors can be saved always, never, or following a set of custom storage rules. Each rule created by the Data Owner affects only to a unique sensor. There are two configurable parameters per rule: variance and the time window (see the Storage Management window in the figure below).

Storage Management window

Storage Management window

Each data source has two text boxes available, one for setting the variance and another for setting the time window. The Storage Guide linked below the top menu bar gives a better understanding about how the rules work. It can be also found in the second example.

Dynamic Data Mall (DDM)

The Dynamic Data Mall (DDM) is offered to the Service Developer who will use the data offered by the Data Owners to build applications (data consumers). The Dynamic Data Mall is currently available at http://195.53.58.184:8080/ddmbutler. It has been optimized for Mozilla Firefox web browser. A User ID and password is needed to access the Dynamic Data Mall functionality. The home page can be seen in the figure below.

Dynamic Data Mall main page

Dynamic Data Mall main page

Service Developer registration and access to main menu

To get started in the portal, the service developer must get registered into the Dynamic Data Mall and fill in the following form as shown in the figure below.

Service Developer registration form

Service Developer registration form

Once registered or authenticated, the Service Developer can access the Dynamic Data Mall main menu (see figure below). It provides, through the available options, an overview of the general process to be followed in the DDM:

  • Firstly, and as a mandatory option, data source discovery and acquisition (‘Data sources offering selection’ option) has to be carried out. In this step the service developer decides, according to his/her criteria, which data sources (grouped in packages) suit their needs better, and purchases them (actual purchase is not currently implemented).
  • Second, and optionally, it is possible to configure the platform data processing options (‘Processing rules configuration’ option). The service developer will access this option if s/he wishes to detect events or combine data streams from different data sources to create new data streams (named context).
  • And finally, once the Service Developer has picked up the data sources s/he wants to use in his/her applications (actual purchase is not currently available), s/he can select the client download option (‘DDX client download’ option) to download the needed configuration files and samples that will enable the integration of data sources mediated through DDX into his/her applications.

The three available options are numbered (see numbers in the background of the images to access each option) to provide an overall view of the process that should be followed by Service Developers, from the selection of data sources to the client download (for his/her applications to receive the data) going through the data processing and ending with the download of the tools s/he may need to integrate data into the developed applications.

Dynamic Data Mall main menu

Dynamic Data Mall main menu

A more detailed explanation of every option is provided below:

  • Data sources offering selection (1): This option enables Service Developers to discover suitable data sources, available in packages already defined by the Data Owners. Once the Service Developer has selected (and purchased in a real deployment scenario) a set of packages s/he can proceed to DDX clients download (3) and download the needed files to start to use the real-time information delivered by the sensors, or go to Processing rules configuration (2) to configure rules to be applied to the data streams before the platform relays them to the applications.
  • DDX clients download (3): In this step the developer can download the libraries (JAR files) for ths applications to use the data being mediated by the BUTLER DDX SmartServer. There are two types of libraries, depending on the type of access (in one case, the information is continuously pushed to the applications while in the other one, the application queries selected data from the BUTLER DDX SmartServer). Sample code to learn how to manage the information is provided as well.
  • Processing rules configuration (2): If the Service Developer wishes to use the BUTLER DDX SmartServer processing capabilities, s/he should select this option and define Business Events and/or the Context rules that will be used to process the real-time information coming from data sources.

The selection order creates different paths:

  • Direct Input – Output selection: The Service Developer is only interested in raw data generated by the real-time data sources selected in option 1. In this case, no further processing is required. The Service Developer can go directly to option 3 after performing the desired subscriptions in option 1.
  • Input – Processing – Output selection: The Service Developer selects the data sources that s/he wishes to use in option 1, but in this case s/he wants to make use of the processing capabilities provided by the platform. So s/he shall choose option 2 and configure the business rules or context rules to be applied on the already selected input data.

Data sources offering selection (mandatory)

As explained in the previous section, and following a generic workflow, it is mandatory to get started with accessing the first available option. Option 1 allows Service Developers to browse information describing the available data sources in the Package Catalog (see figure below). The Package Catalog contains all the packages created by Data Owners (the offering provided by them). A search option is available in order to discover packages whose tags match the input typed in the box.

Package Catalogue window

Package Catalogue window

Once the Developer has picked up a package, if s/he clicks on it detailed information about the devices and related sensors included in the package and their geographical location is obtained (see next figure).

Package Description window

Package Description window

By clicking on a sensor name it is possible to access further details, such as the data format or the charts mentioned in section 3.1.2.4.

It is possible to add several packages to the shopping cart and then purchase them, as shown in the figures below (NOTE: options 2 −Processing rules− and 3 −DDX client download− are only available if the Service Developer purchases at least one package, otherwise s/he will not be allowed to access them).

In the Package window there is an ‘Add to cart’ green button. Clicking that button adds the package to the shopping cart. Once all the desired packages have been added to the cart, it is necessary to confirm the purchase from the ‘Cart’ window. In order to do that the ‘Purchase’ green button has to be clicked. If the Service Developer wants to discard any package before the purchase confirmation, s/he can do it by clicking the ‘X’ button surrounded with a circle at the left side of the packages’ name, that package will be deleted from the cart (see next figure).

 

Shopping Cart window (package pre-purchase) window

Shopping Cart window (package pre-purchase)

Shopping Cart window (selection confirmed)

Shopping Cart window (selection confirmed)

Once the purchase process has finished successfully, the user can check his/her purchased packages in the menu called ‘My Packages’.

My Packages window

My Packages window

It is possible to remove a package from the user subscription by selecting the package and clicking on the button ‘Unsubscribe’ in the following window (see figure below).

Package window

Package window

Processing rules configuration (optional)

The second option, Processing rules configuration, leads to a window that presents two sub-options: Events and Context (see figure below).

Processing Configuration main menu

Processing Configuration main menu

Business Events

This option provides the means to configure in an easy way the business events definition by means of rules. It is highly recommended to read the downloadable Events guide (see third example) in order to understand the rule syntax and the interface usage. Once a rule set is saved, when the data items sent by the specified data sources matches the defined conditions, the event is triggered. The reception of the events as well as raw data is explained in section 3.2.4.

Business Event Definition window

Business Event Definition window

It is possible to check the already defined events in the window My Events.

Context

This second processing sub-option provides a different approach towards the processing options of the BUTLER DDX SmartServer. It is virtual entity-oriented and instead of rules there is a default function set provided. Once again, reading the downloadable Context guide (see fourth example) is mandatory.

Context Definition window

Context Definition window

The reception of context information is explained in section 3.2.4.

DDX clients download (mandatory)

Back to main window, the third option (DDX Clients Download), Service Developers will find all the tools needed in order to allow their application to receive the push notifications coming from the sensors included in the purchased packages or query the stored ones on demand.

DDX Client Download main menu

DDX Client Download main menu

Push Client

Push client window (figure below) shows three different links.

  • The Developer guide (see fifth example) provides all needed information to develop and launch a simple application able to subscribe to the BUTLER DDX publisher capabilities (implemented by means of a MQTT broker called Mosquitto).
  • The Download Client Push button provides the JAR file that has to be integrated in the application developed by the Service Developer.
  • Download Sample Code shows how applications can receive notifications through Java code. From that basis, it is easy to develop more complex applications.

The box at the right side (‘Your information’) shows a link that allows the Service Developer to know which data streams his/her applications are entitled to receive.

Push Client Download window

Push Client Download window

The Credentials configuration window (see figure below) displays all the data sources the applications created by the Service Developer are entitled to receive. The Service Developer can choose among them and download a ‘credentials file’, needed to make the applications work. Only by using this file it is possible to subscribe to data streams pushed by the BUTLER DDX SmartServer.

Credential Configuration window

Credential Configuration window

Pull Client

In a similar way to the previous client, a Developer guide (see sixth example), a library packaged into a JAR file (Download Client Pull) and Sample Code are provided. This time no credentials are needed. A set of predefined queries are ready to be used in the sample code and the developer guide explains how to use them.

Pull Client Download window

Pull Client Download window

Summary

Service Developers have four main scenarios they can manage from the Dynamic Data Mall.

  1. Input (Data sources offering selection) – Output (DDX clients download)
  2. Input (Data sources offering selection) – Events (Processing rules configuration) – Output (DDX clients download)
  3. Input (Data sources offering selection) – Context (Processing rules configuration) – Output (DDX clients download)
  4. Input (Data sources offering selection) – Event & Context (Processing rules configuration) – Output (DDX clients download)
Input – Output

This scenario is related to Developers who want to build applications that consume directly raw data sent by the data sources. This could be the case of Applications that use data sent by sensors to generate more elaborated information. In this case the Developer has to subscribe to at least one package and then choose between the two provided clients.

Input – Events – Output

This second case is related to Developers who want to make a more intensive use of the computational capabilities of the platform and perform analysis than didn’t do in the first option. To do this the Developer shall start by selecting a package and later on s/he shall define a set of rules (Events) to determine when the DDX PoC must submit specific notifications. In this case, the right client to choose in the last option is the Push client.

Input – Context – Output

This third path is quite similar to the previous one but with the difference that the rules will be based on the computation of mathematical functions applied to selected virtual entities. This means that the Developer will select not only a set of rules to be applied to the data items sent by the devices, but their results will be useful in order to build the selected virtual entity context information. In this case the Pull client is the suitable one.

Input – Events & Context – Output

This last scenario is the result of the combination of paths depicted in 3.2.5.2 and 3.2.5.3, which are fully compatible. Once again, the Push client is the right option.

Service level

DDX is an initiative of the Technology and Innovation Unit at Ericsson Spain. Thus, it's NOT a regular Ericsson product and therefore, no support or endorsement from Ericsson can be requested.

Support will be provided on a case by case basis. Non-commercial parties can expect clarifications and know-how transfers. Any detected critical bug will be patched. Professional support will be provided in case of a commercial agreement with firms interested in exploiting these components for their businesses. Contact person is Miguel-Angel Monjas ([email protected]).

DDX has detected the need of built-in analytics tools able to provide insights over the data streams being mediated by the platform. New releases of an analytics-based DDX are being planned.

Technology Readiness Level

4 - technology validated in lab

Reuse Readiness Levels

3 - Basic reusability; the software might be reusable by skilled users at substantial effort, cost, and risk.

How to connect data sources to DDX

DDX exposes a public socket and is ready to receive every measure sent by any data source registered into the DDX platform. The information must be sent in the following stream format:

DeviceId:SensorId Data

Where DeviceId is the Device Id (an integer number that uniquely identifies each device generated when a new device is registered by the Device Owner); SensorId is the Sensor Id (an integer number that uniquely identifies each sensor generated when a new sensor is registered by the Device Owner); and Data is the integer or decimal number representing the information the sensor sends. Note that between SensorId and Data there is a space.

Device Owners are expected to implement a layer between their devices and the DDX socket in order to format the data their devices provide. They can know the device and sensor identifiers because the Data Owner Portal displays that information, in the device and sensor details page.

The current address of the socket is 195.53.58.184, port 54444.

It is possible to check if the measures are reaching their goal properly by checking the charts located at the sensor details page.

In case of no real data source availability, a software based simulator can be used. It is possible to download a Java-based sensor simulator from the Devices screen in the Data Owner Portal (Simulator.jar).

BUTLER DDX Sensor Simulator
BUTLER DDX sensor simulator interface

The simulator is implemented to send data to the ip and port mentioned above. First two fields are the aforementioned Device Id and Sensor Id. Third and fourth fields set the upper and lower limits between the simulator will generate values, and the last one sets the generation frequency.

How to configure storage rules

By default the established rules store every single data item coming from the data sources, so this interface allows Data Owners to alter this behavior.

Two fields are displayed per each data source the Service Developer has acquired: Variance and Time Window. Variance is the value that acts as threshold between a data item and its predecessor to trigger the storage rule. Time Window is the time period (milliseconds) the data items will be stored. When the default rules are on, both fields show 0 values.

In order to clarify these concepts, let’s explain all the possibilities using examples:

  • Fill in the Variance field only: if we set 5, a data item will be stored only if its value is 5 or more units higher or lower than the previous data item’s value received. This field works with absolute values, so negative numbers are not allowed.
  • Fill in the Time Window field only: if we set 10000, the rate of data item storage is 10 seconds.
    • In the case the data source update rate is 5 seconds, only half of the data items will be stored. Two data items will arrive per time window and only the first one is stored.
    • In the case the data source update rate is 15 seconds, all the data items will be stored. Only one data item arrives per time window as maximum, and some time windows even receive any.
  • Fill in both fields: Both rules will work together. If we combine both previous examples, Variance 5 and Time Window 10000, a data item will be stored if its value is 5 or more units higher or lower than the previous one or it’s the first data item received within a 10 seconds time window.

How to configure business events

DDX events allow users to define a variety of actions that are executed when certain conditions are met by the sensor values to which the user is subscribed. These conditions will also be defined by the user, as rules, whose syntax is described in this guide. A user interface has been developed in order to help the user in his/her tasks.

The definition screen is composed by the following elements:

  1. Sensor list. Here the user will find the list of data sources s/he is subscribed to. The data sources the event that is going to be defined shall use should be selected.
  2. Create Templates button. By clicking this button, a rule template with the selected data sources in the sensor list will be displayed at the Event Rules text box.
  3. Event Rules text box. A rule template for a single data source looks like the following:

( variableX : Measure (( id == deviceId:sensorId ) && (( condition_data >= 0.0 ))))

The components of the rule template are as follows:

    • deviceId:sensorId: This field will take the value of the selected data source.
    • condition_data: This field represents the attribute the rule will read. It can take three different values:
      • Latitude
      • Longitude
      • Data

All of them are available at the Operators buttons panel.

    • >=: This field hosts the comparison operator. The possible ones are:
      • <: Less than
      • >: Greater than
      • <=: Less or equals
      • >=: Greater or equals
      • ==: Equals

All of them are available at the Operators buttons panel.

    • 0.0: Numerical value. The only condition is the format: it must be double.
    • ))): More conditions. If the user wants to add more conditions to the rule, click on the  [Add Condition]  button (see the Operators buttons section below). A rule with two conditions will look like the following:

( variableX : Measure (( id == deviceId:sensorId ) && (( condition_data  >= 0.0) && (condition_data >= 0.0 ))))

    • &&: Logical operator. This field will be displayed only if the user adds more conditions to the rule. The possible values are && or ||.

An example of a real rule could be the following:

( variable0 : Measure (( id == 365:156 ) && (( Data  >= 1.0) && (Data < 5.0 ))))

The rule above will trigger an event if the measure sent by the data source 365:156 is greater or equals than 1 and less than five.

Despite everything said above, the Event Rules text box is fully editable, so the usage of the color fields or Operators buttons is not mandatory. The user can type his/her rules freely.

  1. Operators buttons. In order to use the different operator buttons, first of all it is necessary to click on a colored zone of the rule template. Then, if the user clicks on a suitable button (matching color), the selected color zone will change its text by the one on the chosen button.The only button not explained yet is the [OK?] one. It lets the user know if the typed rules are syntactically correct and the other text boxes are filled correctly before submitting them.
  2. Event Actions text box. This field hosts the text the event will contain. If someone receives an event, this is the text s/he/it will read. There is a restriction, only alphanumeric characters and spaces can be used. Example:

Alarm in sensor 365 156 Pressure too high

  1. Event Name text box. The id of the event.
  2. Save Event button. It stores the event in the Complex Event Processing engine. All the defined events can be checked, edited and deleted from My Events page.

How to define context attributes

The context feature allows the user to define attributes attached to virtual entities based on the data sources s/he is subscribed to. Those so-called attributes basically consist of basic mathematical operations applied to the measures sent by the data sources.

New context attributes is defined in two steps:

Step 1: select the virtual entity. In the first screen a list of all the existing virtual entities will be displayed.

Step 2: select the data sources and configure the mathematical operations. The second screen displays the list of the data sources the user is subscribed to. First of all, a data source should be selected. Then, the available operations have restrictions that vary from one to another.

  • SUM. It returns the sum of all the received measures from the selected data source since its definition. Example: SUM(‘459:123’). Besides, it can be combined with other data sensors’ sums. Example: SUM(‘459:123’) + SUM(‘647:465’) by using the button [Add Sensor]
  • AVG. It returns the average of all the received measures from the selected data source since its definition. Example: MED(‘647:465’). It also can be combined with other data sources’ averages. Example: MED(‘647:465’) + MED(‘459:123’) by using the button [Add Sensor]. This operation allows a different format as well. It is possible to limit the average calculation to a given number of received measures. Example: MED(‘647:465’, 8) will return the average of the last eight measures received. It is necessary to type the comma and the number in order to use this format of AVG operation.
  • MAX. It returns the maximum of the measures sent by at least two data sources. Example: MAX(‘459:123’, ‘647:465’). In order to add the second data source, use the button [Add Sensor]. In order to add a third or more data sources, type them.
  • MIN. It returns the minimum of the measures sent by at least two data sources. Example: MIN(‘459:123’, ‘647:465’). In order to add the second data source, use the button [Add Sensor]. In order to add a third or more data sources, type them.
  • Other operators. In the SUM and AVG cases, although they are atomic operations, it is possible to combine several of them. The operators between each operation are colored in grey and they can be replaced by any of the four ones available at the buttons panel (+, -, /, *). Click on the grey zone and then on the desired operator button in order to perform the replacement.

After defining the operations and giving a name to the attribute, it should be added to the attribute buffer by clicking on the Add button. The Attribute Rules screen will be empty again allowing the definition of a new attribute and the previous attribute will be shown below. When all the desired attributes are defined, click on the Save Context button in order to send them to the DDX Data Broker.

In order to check all the context attributes go to the My Context page.

 How to use the push interface client

  1. Download the file com.ericsson.tbi.ddx.client-vX.Y.jar from the 'Download Client Push' page at the DDX Marketplace (http://195.53.58.184:8080/ddm/download-jar.jsp).
  2. Download the credentials file generated from the 'Configure Credentials' page at the DDX Marketplace (http://195.53.58.184:8080/ddm/configure-credentials.jsp).
  3. Within the JAR file, the class com.ericsson.tbi.ddx.client.Subscriptor allows subscribing to all the purchased packages, defined business event rules and configured context by using the credentials file.
  4. To activate a subscription, that class has to be instantiated with the following parameters:

Subscriptor suscriptor = null;
try {
suscriptor = new Subscriptor("195.53.58.184", 1883, "credentialsFile");
} catch (ClientException e) {
e.printStackTrace();
}
suscriptor.addNotificationListener(new NotificationListener());

  1. To start receiving measures, the com.ericsson.tbi.ddx.client.NotificationListener interface must be implemented. Then, one or more instances of this class can be added to the Subscriptor object created before.

public class NotificationListener implements INotificationListener
{
@Override
public void handleNotification(Action action) {
System.out.println("[" + new Date() + "]"+ action);
}
@Override
public void handleNotification(Measure measure) {
System.out.println("[" + new Date() + "]"+ measure);
}
@Override
public void handleNotification(Attribute attribute) {
System.out.println("[" + new Date() + "]"+ attribute);
}
}

  1. When a new Action or Measure object is received, the handleNotification methods will be invoked.

How to use the pull interface client

This output interface provides a query mechanism to retrieve data stored in a NoSQL database. This database stores the measures sent by the sensors in JSON format.

Example:

{
"id" : "100:20",
"location" : {
"name" : null,
"latitude" : 40.39028,
"longitude" : 3.628211,
"altitude" : 0
},
"timestamp" : "1367319143888",
"data" : 25.8,
"magnitude" : "Temperature",
"units" : "%"
}

A Java client has to be downloaded (com.ericsson.tbi.ddx.pull.client-vX.Y.jar at http://195.53.58.184:8080/ddm/clientpull.jsp). Below the list of available functions:

To select all the stored data belonging to a single sensor/data source:

public List<String> selectAll(long id, String user, String password)

To select those stored data belonging to a single sensor/data source that matches the condition of being less than a given value.

public List<String> selectLessThan(long id, double value, String user, String password)

To select those stored data belonging to a single sensor/data source that matches the condition of being greater than a given value.

public List<String> selectGreaterThan(long id, double value)

To select those stored data belonging to a single sensor/data source that matches the condition of being less than a given value and greater than another given value.

public List<String> selectBetween(long id, double value1, double value2)

To select those stored data belonging to a single sensor/data source that matches the condition of being equals to a given value.

public List<String> selectEquals(long id, double value)

To select all the stored data from every sensor/data source located within a circle defined by a central point (coordinates) and its radius.

public List<String> selectAllCloseTo(double latitude, double longitude, double radius)

To select all the stored data belonging to a single sensor/data source received during a given time period.

public List<String> selectBetweenTime(long id,long start,long end)

Here a sample application source code is available:

import com.ericsson.tbi.ddx.pull.client.PullRestClient;
import java.util.List;

public class Sample {
/**
* @param args
*/
public static void main(String[] args) {
PullRestClient acc = new PullRestClient("195.53.58.184", 27017);
List<String> res = acc.selectAll("100:20","user","password");
//TO DO
}
}

First of all, it is necessary to add the library com.ericsson.tbi.ddx.pull.client-v1.0.jar to the project classpath. Then, a PullRestClient object must be instantiated. All the implemented functions can be called from that instance.

A more complete sample application is available for download as well.