Hardware Recap

In Part One of this demo, we took two Texas Instruments Sensortags, connected them using Bluetooth LE to a Beaglebone Black, ran a Node.js gateway to connect to DeviceHive¹, and saw it all work. This is the diagram of our hardware setup, as completed at that point:

Fantastic! Our hardware works. Now we are going to hook some data services up using the wot.io data service exchange™, and do some fun stuff with it.

Data Services

Now let's expand it to include everything we'll do with the data services. We are going to use scriptr, Circonus, bip.io, and a Nest thermostat. Here's the plan:

1. Send the data from DeviceHive to scriptr for processing
2. Using scriptr, massage our data, and make some logs
3. Send the data from scriptr to Circonus for graphing
4. Send the data from scriptr to bip.io for alerting and control of the Nest thermostat

Message Flow Graph

Below is a diagram of the message flow. All the green lines are implemented using the wot.io data service exchange™ (which I also call the bus), connecting data service sources to data service sinks.

You'll notice that some of the scripts, bips, and graphs are named temperature, and others are named color. I have a confession - to save time, I just stuck with the default setup that comes out of the box with wot.io's Ship IoT initiative which converts temperature units and maps them onto the color spectrum for use with some Philips Hue bulbs like we saw in an earlier post. I just figured that since wot.io has so many data services, and I have so little time, why not just re-use what was already done? So, let's just agree to ignore the fact that scripts named color might no longer have anything to do with color. Maybe we're just coloring our data. Ok? Onward!

Scriptr

Our data's first stop after leaving DeviceHive is scriptr, so we'll start there. The scriptr.io data service offers a very fast way to create custom back-end APIs to process your data in the cloud using JavaScript. This enables fast productivity and power for your Internet of Things (IoT) and other projects, ever more so when tied to other data services via wot.io. All the messages come into a script called transform, as defined by the wot.io bus configuration.

scriptr: transform

The first task we perform on our message stream is a data normalization step. You'd expect to see something like this in most real-world applications—a layer of abstraction that transforms incoming messages to a unified format for subsequent services to consume. This script will massage the incoming messages into this simple JSON structure, and remove bits that may no longer be relevant now that we are outside of the local network that the originating devices were using:

[ device_id, { key:value, ... } ]

for keys:  
key: "name" | "value" | "units"

for values:  
name: "temperature" | "humidity"  
value: a floating-point number  
units: "C" | "F" | "%RH"  

For example, from this input message,

{"action":"notification/insert","deviceGuid":"ca11ab1e-c0de-b007-ab1e-de71ce10ad01","notification":{"id":1558072464,"notification":"temperature","deviceGuid":"ca11ab1e-c0de-b007-ab1e-de71ce10ad01","timestamp":"2015-10-15T20:28:33.266","parameters":{"name":"temperature","value":23.7104174805,"units":"C"}},"subscriptionId":"00000000-6410-4e1a-b729-000000000000"}

...we get this output message:

["ca11ab1e-c0de-b007-ab1e-de71ce10ad00",{"name":"temperature","value":23.7104174805,"units":"C"}]

Now we are ready to sink these normalized messages back onto the bus for further processing by other data services.

As the message flow graph above illustrates, messages from transform will use the bus to fan out and sink into convert and color in scriptr, and also into bip.io and Circonus.

Here's our full transform code:

// Convert DeviceHive Notification to well known format of [<devicehive deviceId>, <devicehive parameters>]
var log = require("log"),  
    data = JSON.parse(request.rawBody).data,
    payload = data && data[0];
log.setLevel("DEBUG");  
log.debug("testraw: " + JSON.stringify(data[0]) );  
if (payload && payload["deviceGuid"] && payload["notification"]["parameters"]) {  
  response = JSON.stringify([payload["deviceGuid"], payload["notification"]["parameters"]]);
    log.debug("response: " + response);
    return response
}
log.debug("Invalid Request: " + JSON.stringify(payload))  

scriptr: convert

This is a utility set up to demonstrate data transformation and message decoration. We take messages from the incoming data source, parse out the type and units, and create a new data structure with additional information based on the incoming message. This data source will be sent in a message to whatever sink is configured.

A more complex implementation could take incoming data, perform lookups against a database, add semantic analysis, analyze for part-of-speech tagging, or do any number of other things. Complex message graphs composed of small, well-defined services let us build up behaviours from simple parts—much like the Unix philosophy when it comes to small command-line tools.

In this case, we convert Celsius to Fahrenheit, or Fahrenheit to Celsius, depending on what the incoming format is, and put both values into the resulting message. For humidity we simply pass along the value and label it as rh for relative humidity.

  switch (units) {
    case "c":
      // The incoming reading is in celsius. Convert to Fahrenheit
      response.tf = temp && (temp * 9 / 5 + 32).toFixed(1) || "N/A";
      response.tc = temp && temp.toFixed(1) || "N/A";
      break;
    case "f":
      // The incoming reading is in Fahrenheit. Convert to celsius
      response.tf = temp && temp.toFixed(1) || "N/A";
      response.tc = temp && ((temp - 32) * 5 / 9).toFixed(1) || "N/A";
      break;
    default:
      response.error = "unknown units";
  }

These demonstration messages currently sink into Scriptr's logs, and can be used in future systems. Here's the result of a temperature message, and we can see the incoming ºC data was converted to ºF and logged:

Scriptr: color

Once again, this script was originally meant to control a Philips Hue lamp, but we've co-opted it to send data along to bip.io and control our furnace. (I've left in the color calculations if you're curious). It would be trivial to expand the message graph in bip.io to do the lamp control, I just didn't have the time to set it up. Aren't I quite the model of efficiency today?

// Unpack the parameters passed in
var log = require("log"),  
    timestamp  = request.parameters["apsws.time"],
    data = JSON.parse(request.rawBody).data,
    reading = data && data[0],
    deviceId = reading && reading[0];

if (reading && (reading[1] instanceof Object) && reading[1].name == "humidity") {  
  // we just drop humidity messsages here, as this is intended to control
  // the thermostat settings later on and nothing else at this time.
  return null;
}

var celsius = reading && (reading[1] instanceof Object) && reading[1].value;

// Convert temperature in range of 0C to 30C to visible light in nm
// 440-485 blue, 485-500 cyan, 500-565 green, 565-590 yellow, 590-625 orange, 625-740 red
// 300nm range, 30C range
var temperature = celsius < 0 ? 0 : (celsius > 30 ? 30 : celsius),  
    color = 440 + (300 * (celsius / 30));

// Populate response values or default to non-value
var response = {  
  time: timestamp,
  temperature: celsius.toFixed(2),
  color: parseFloat(color.toFixed(2)),
  device: deviceId || "N/A"
};
log.setLevel("DEBUG");  
log.debug("response: " + JSON.stringify(response));  
return JSON.stringify(response);  

Circonus Graphs

Circonus is designed to collect your data into graphs, dashboards, analytics, and alerts. While it is often used for DevOps or IT Operations style monitoring, we're showcasing how well it serves as a key component of an IoT solution. Today, we'll simply use it to graph our timeseries messages and send ourselves an alert if the data stops flowing. This could indicate a problem with the battery in the Sensortag, or that we are out of range. Use your imagination, the sky is the limit, and Circonus has a powerful feature set.

Checks

You can see the four device IDs here, and the checks that were set up as part of this demonstration message flow.

Graphs

As the metrics are colleted, Circonus tracks it and can create graphs and dashboards for you. There's only a bit of data shown in the graph here because I've only had it running for a few minutes.

There are some powerful analytics tools and alerts at your fingertips here. It's hard to show with the small amount of data, but you can use anomaly detection, trend prediction, and many other functions on your data. This is a simple sliding window moving average, which we could use to smooth out spurious temperature readings and prevent the furnace from turning on needlessly.

Alerts

Circonus maks it simple to notify you with an alert if the data stops flowing. This is essential for mission-critical systems.

bip.io Workflow

We've covered the details of creating a bip.io workflows elsewhere, and many of the details like endpoints, auth tokens, etc. are already taken care of for us automatically by the wot.io integrations and tooling.

Here in the dashboard we can see the four bips that are referenced in the above message flow graph. Each has the device ID embedded into the name, and the endpoint.

We'll have a look at two of them, both for the sensor ID ending in 00 (which is from the device MAC ending in :70, way back up the chain!). First, the alert.

alert bip

Here we see the overall message flow inside the alert bip. Incoming messages from the wot.io bus are processed by a math expression, a truthyness check, and if it all passes the criteria, an email alert is sent.

Here's the expression. It's basic; we are simply checking if the temperature is too low, which could indicate some problem with the heating system:

The truthy check looks at the result of the previous Calculate expression, and will trigger the following node in the graph if it's true:

And finally, we send an email alert, going to an address we specify, with the data embedded in it via template replacements:

Simple!

Nest Temp Control bip

Now we have a simple bip set up to take the incoming temperature message, calculate an error factor, and generate an offset temperature setting for the Nest thermostat. Unfortunately, Nest doesn't have an API call that lets us send the sensor temperature in directly. Granted, that's an odd use case, but they probably haven't heard of this cool idea yet ;)

With the lack of the sensor API, we need to get creative. We'll take the value from the sensortag, and calculate an error offset:

(desired_temp - sensed_temp)

Then we'll combine the error offset with the desired temperature:

(desired_temp - sensed_temp) + desired_temp

Here it is in the Math function in the bip, with a set point of 20ºC:

This will give us a new set point for the Nest, and we send it along in the bip as pictured above. This is a basic setup, and you would want to refine this for long-term use. I'd suggest adding hysteresis to prevent the furnace from turning on and off too rapidly when close to the set point, and calibrate yourself a PID control loop to smooth things out.

Wrap-Up

This concludes our writeup of what turned out to be a rather complex message flow graph. We started with a local network of devices, built a hardware and software gateway to get those devices out to a device management platform, connected that to the wot.io bus, and wired up some powerful tools whose depths we have only started to plumb.

Yet even with all the complexity and details that we covered, you can see how simple it is to compose behaviors using the wot.io data service exchange™. And that is the whole point: to get us quickly to a working system. And since it's based on a fully scalable architecture, your solution is ready to grow with you from prototype into production.

In other words, you can focus on Shipping your IoT!

See you next time!

Link to Part One of this series