In my previous blogpost I showed how to read gas and electricity measurements from a smart meter. Although it is a nice accomplishment to be able to read these measurements, the next step is obviously to do something useful with them. So, let’s create some nice dashboards based on these measurements. In particular I want to have dashboards that show the total amount of electricity and gas usage as well as the current electricity usage. I decided to use the TIG stack for this purpose. The TIG stack consists of Telegraf, InfluxDB and Grafana. Before I describe in debt what they are and how to configure them I will first introduce the Influx line protocol.

Influx line protocol

Telegraf supports multiple data input formats, which you can find here. The influx line protocol is one of the supported formats and is a text based format for writing points to the database. Each line in the influx line protocol represents one data point in InfluxDB. And it has the following format:

The first part called measurement is the name of the measurement that you want to write the data to and is a required field. The tag_set represents the tags you want to include with the data point and are optional in Influx line protocol. Then the actual measurement(s) follow in the field_set part of the line protocol. And finally an optional timestamp. Note that Influx line protocol has a strict format and uses comma and whitespace as separator characters. You can read more about the Influx line protocol here.

Then I changed dsmr-forwarder to publish measurements in Influx line protocol format. In this case I prefer the Influx line protocol format over another format, because this way gas and electricity measurements are each represented by a single line. Thus, information that belongs together is grouped together in a single data point. The method below parses the given telegram and produces two lines in Influx line protocol, one for electricity and one for gas measurements.

In the next step I publish these lines on MQTT with each line on it’s own topic, as in:

Telegraf

Telegraf is a plugin-driven server agent for collecting and reporting metrics and has several integrations available out of the box. In this case I use Telegraf to read my smart meter measurements from MQTT and write the results to InfluxDB. You can generate a default Telegraf configuration with the following command:

Next, I adjust the default Telegraf configuration file to enable the MQTT inputs section by uncommenting [[inputs.mqtt_consumer]] and let the servers property point to my MQTT broker. The basic configuration for the MQTT consumer input is shown below:

Note that I configured Telegraf to consume all topics from MQTT starting with ‘dsmr0/p1‘ and that I set the data format to ‘influx‘. Then I adjust the Telegraf configuration file to enable InfluxDB outputs section by uncommenting [[outputs.influxdb]]. I set the target database and the username property to ‘telegraf‘. We will need to create those in InfluxDB later. See the basic configuration below:

Now the Telegraf configuration is prepared to consume our dsmr MQTT topics and write the measurements to the database. Before we start Telegraf we will first configure InfluxDB.

InfluxDB

InfluxDB is an open-source time series database and is like Telegraf developed by InfluxData. It has the ability to store large amounts of time-stamped data. First, we create a docker network such that Telegraf and InfluxDB containers can easily communicate with each other:

We can run the InfluxDB docker image with the following command:

The next step is to create the database and user for Telegraf. So, let’s connect to the InfluxDB container by invoking the following command:

Inside the running container you can run:

to launch the InfluxDB console. Let’s create the telegraf database with:

And create a database user with:

Now we have InfluxDB up and running with a dedicated database and user for Telegraf. Once InfluxDB is running we can start Telegraf with the following command:

Then we can check the logs of Telegraf to verify whether it successfully connected to the MQTT broker with:

And verify whether the measurements are successfully read from MQTT and inserted into InfluxDB. First we connect to the running InfluxDB container and launch the InfluxDB console again. If everything is working correctly then our measurements are now ingested into InfluxDB and we can verify that with the following commands:

Note that p1_elec and p1_gas should be part of the listed measurements.

Grafana

Grafana is a data visualisation and monitoring tool with support for multiple input datasources like Graphite, InfluxDB, Prometheus and Elasticsearch to name a few. Recently, Grafana added support for ARM architectures. So now you can install the official Grafana releases on your Raspberry Pi. You can download and install Grafana with the following commands:

Then you browse to http://grafana-host-ip:3000 and login in with admin/admin. In Grafana we configure a datasource for InfluxDB using the telegraf database and user created during InfluxDB setup. Then finally we can create a dashboard and plot our desired graphs. Here is the configured query for the current electricity usage graph.

And below are some graphs from my dashboard. Happy monitoring 😉