Tag Archives: Push notification

Matters relating to the use of ‘push notification’ (e.g. to a mobile device) providing alerts and update messaging

Particulates Sensing with the NOVA SDS011

Here at Cranfield University we are putting in place plans related to the new ‘Living Laboratory’ project, part of our ‘Urban Observatory’. This project sits within the wider UKCRIC initiative, across a number of universities. Of the many experiments in development, we are gathering environmental data from IoT devices and building data dashboards to show the data and related analyses. One of our projects will be to investigate air quality on the campus, in our lecture rooms and public spaces. Cranfield is a unique University in the UK for having its own airfield as part of the campus – we want to monitor any particular impacts that can arise from this. To do this, one of the tools we will use is the amazing Nova SDS011 particulates sensor (http://www.inovafitness.com/en/a/index.html).

The sensor itself, available from many outlets for instance here, is extremely cheap for what it offers, and is widely reported on with many projects on the Internet. We followed the excellent tutorial laid out on Hackernoon (https://hackernoon.com/how-to-measure-particulate-matter-with-a-raspberry-pi-75faa470ec35). We used a Raspberry Pi Zero, and we used the USB interface to speed the process of prototyping.

Rather than repeat the instructions laid out so well by Hackernoon, here we have some observations, and then some small adaptations to enable notifications and data logging.

One thing to remember in using the Raspberry Pi is that you need adapters (shown above) to connect traditional USB plugs to the micro plugs on the Pi. Also you need to remember that of the two USB ports, one is for powering the device and one is for peripherals. Plugging them in the wrong way round led to lots of unnecessary head scratching!

That said, once the instructions were followed, and the code put in place, the system was up and running and we could access the simple dashboard Hackernoon have developed using lighttpd.

This could be the end of the blog, all worked well, we have readings and a simple dashboard showing AQI. The device is incredibly sensitive – we can attest that during building the setup a late night pizza was accidentally burned (too busy hacking)! But the machine picked up the spike in particulates very well.

So the next challenge was to log the data being generated. In earlier blogs, we have used and liked ThingSpeak as a quick means to log data and build dashboards, so we decided to use this. This meant editing the Python code that hacker noon provided.

To write to ThingSpeak in Python, one can use the ‘urllib2’ library. We followed the excellent Instructables blog to do this. First, at the top of the code we import the urllib2 library and set up a variable to hold the connection string to ThingSpeak (using the API key for writing to the Channel we have created to hold the data):

<code>import urllib2 baseURL = 'http://api.thingspeak.com/update?api_key=CHANNEL_WRITE_API_KEY'</code>

Next, we located in the code where the particulate values for PM2.5 and PM10 are extracted and sent off to the web dashboard (full code used at the end). Here we inserted code to also send the same data to ThingSpeak:

<code>f = urllib2.urlopen(baseURL + '&amp;field1=' + str(values[0]) + '&amp;field2=' + str(values[1]))
f.read()
f.close()</code>

This worked well and data was transmitted to ThingSpeak and with its timestamp, this enabled a more comprehensive dashboard to be created that monitored the data values detected by the device (rather than the AQI values shown in the Hackernoon dashboard – clearly one could write that conversion in python in future if needed).

We then followed Hackernoon’s instructions to make the process start up on boot by placing the script into the crontab file. However, in doing this we realised it isn’t always possible to know when the script has started. As the script only starts on boot, if something goes wrong, the script never runs. We found that this was not a unique issue as others have found this also in other blogs. Thanks to the instructions on the Raspberry Pi website, we realised we could add a sleep command in to the crontab to ensure that the script was only started when there was a good chance the rest of the system was up and running. This solved the problem and now the crontab command was:

<code>@reboot sleep 60 &amp;&amp; cd /home/pi/ &amp;&amp; ./aqi.py</code>

The time could be extended from 60 seconds if needed. In any case, we now wanted to know it had indeed started up OK. We wanted a message sent to a mobile phone to say the process had started up OK. To do this we used the push notification approach of Prowl used in earlier blogs on this site (you need an iPhone for this although there will be equivalents for other phones. To get prowl to work in Python, we used the Python module for Prowl iPhone notification service from jacobb at https://github.com/jacobb/prowlpy. Installing this means downloading the ‘prowlpy.py’ script, and then a further adaptation in the aqi script at the start to call it appropriately, thus:

<code>import prowlpy
 apikey = 'PROWL_API_KEY'
 p = prowlpy.Prowl(apikey)
 try:
     p.add('AirQual','Starting up',"System commencing", 1, None, "http://www.prowlapp.com/")
     print('Success')
 except Exception,msg:
     print(msg)</code>

Finally, were it required, the push notification approach could also be used to inform particulate readings. The values of pm can also be intercepted, as per the ThingSpeak export, to send to the mobile phone too, code to do this would be thus:

<code>_message = "pm25: %.2f, pm10: %.2f, at %s" % (values[0], values[1], time.strftime("%d.%m.%Y %H:%M:%S"))          
print(_message) # debug line 
try:
    p.add('AirQual','Reading', _message, 1, None, "http://www.prowlapp.com/") 
except Exception,msg:
    print(msg)</code>

Although this worked perfectly, the phone was immediately overwhelmed with the number of messages, and this was quickly turned off! Notifications could be used however to message the user’s phone if important air quality thresholds were breached – reminding the operator to, for example, take the pizza out of the oven!

The final code script used for ‘aqi.py’ was:

<code>#!/usr/bin/python -u
# coding=utf-8
# "DATASHEET": http://cl.ly/ekot
# https://gist.github.com/kadamski/92653913a53baf9dd1a8
from __future__ import print_function
import serial, struct, sys, time, json, subprocess

# Customisations ######
import urllib2
baseURL = 'http://api.thingspeak.com/update?api_key=THINGSPEAK_API'

import prowlpy
apikey = 'PROWL_API_CODE'
p = prowlpy.Prowl(apikey)
try:
    p.add('AirQual','Starting up',"System commencing", 1, None, "http://www.prowlapp.com/")
    print('Success')
except Exception,msg:
    print(msg)
####################

DEBUG = 0
CMD_MODE = 2
CMD_QUERY_DATA = 4
CMD_DEVICE_ID = 5
CMD_SLEEP = 6
CMD_FIRMWARE = 7
CMD_WORKING_PERIOD = 8
MODE_ACTIVE = 0
MODE_QUERY = 1
PERIOD_CONTINUOUS = 0

JSON_FILE = '/var/www/html/aqi.json'

MQTT_HOST = ''
MQTT_TOPIC = '/weather/particulatematter'

ser = serial.Serial()
ser.port = "/dev/ttyUSB0"
ser.baudrate = 9600

ser.open()
ser.flushInput()

byte, data = 0, ""

def dump(d, prefix=''):
    print(prefix + ' '.join(x.encode('hex') for x in d))

def construct_command(cmd, data=[]):
    assert len(data) &lt;= 12
    data += [0,]*(12-len(data))
    checksum = (sum(data)+cmd-2)%256
    ret = "\xaa\xb4" + chr(cmd)
    ret += ''.join(chr(x) for x in data)
    ret += "\xff\xff" + chr(checksum) + "\xab"

    if DEBUG:
        dump(ret, '> ')
    return ret

def process_data(d):
    r = struct.unpack('&lt;HHxxBB', d[2:])
    pm25 = r[0]/10.0
    pm10 = r[1]/10.0
    checksum = sum(ord(v) for v in d[2:8])%256
    return [pm25, pm10]
    #print("PM 2.5: {} μg/m^3  PM 10: {} μg/m^3 CRC={}".format(pm25, pm10, "OK" if (checksum==r[2] and r[3]==0xab) else "NOK"))

def process_version(d):
    r = struct.unpack('&lt;BBBHBB', d[3:])
    checksum = sum(ord(v) for v in d[2:8])%256
    print("Y: {}, M: {}, D: {}, ID: {}, CRC={}".format(r[0], r[1], r[2], hex(r[3]), "OK" if (checksum==r[4] and r[5]==0xab) else "NOK"))

def read_response():
    byte = 0
    while byte != "\xaa":
        byte = ser.read(size=1)

    d = ser.read(size=9)

    if DEBUG:
        dump(d, '&lt; ')
    return byte + d

def cmd_set_mode(mode=MODE_QUERY):
    ser.write(construct_command(CMD_MODE, [0x1, mode]))
    read_response()

def cmd_query_data():
    ser.write(construct_command(CMD_QUERY_DATA))
    d = read_response()
    values = []
    if d[1] == "\xc0":
        values = process_data(d)
    return values

def cmd_set_sleep(sleep):
    mode = 0 if sleep else 1
    ser.write(construct_command(CMD_SLEEP, [0x1, mode]))
    read_response()

def cmd_set_working_period(period):
    ser.write(construct_command(CMD_WORKING_PERIOD, [0x1, period]))
    read_response()

def cmd_firmware_ver():
    ser.write(construct_command(CMD_FIRMWARE))
    d = read_response()
    process_version(d)

def cmd_set_id(id):
    id_h = (id>>8) % 256
    id_l = id % 256
    ser.write(construct_command(CMD_DEVICE_ID, [0]*10+[id_l, id_h]))
    read_response()

def pub_mqtt(jsonrow):
    cmd = ['mosquitto_pub', '-h', MQTT_HOST, '-t', MQTT_TOPIC, '-s']
    print('Publishing using:', cmd)
    with subprocess.Popen(cmd, shell=False, bufsize=0, stdin=subprocess.PIPE).stdin as f:
        json.dump(jsonrow, f)


if __name__ == "__main__":
    cmd_set_sleep(0)
    cmd_firmware_ver()
    cmd_set_working_period(PERIOD_CONTINUOUS)
    cmd_set_mode(MODE_QUERY);
    while True:
        cmd_set_sleep(0)
        for t in range(15):
            values = cmd_query_data();
            if values is not None and len(values) == 2 and values[0] != 0 and values[1] != 0:
              print("PM2.5: ", values[0], ", PM10: ", values[1])
              time.sleep(2)

	      # ThingSpeak ######
	      f = urllib2.urlopen(baseURL + '&amp;field1=' + str(values[0]) + '&amp;field2=' + str(values[1]))
	      f.read()
	      f.close()
              ###################

              # Push notifications ######
              #_message = "pm25: %.2f, pm10: %.2f, at %s" % (values[0], values[1], time.strftime("%d.%m.%Y %H:%M:%S"))
              #print(_message)
              #try:
              #	p.add('AirQual','Reading', _message, 1, None, "http://www.prowlapp.com/")
              #except Exception,msg:
              #  print(msg)
              ####################


        # open stored data
        try:
            with open(JSON_FILE) as json_data:
                data = json.load(json_data)
        except IOError as e:
            data = []

        # check if length is more than 100 and delete first element
        if len(data) > 100:
            data.pop(0)

        # append new values
        jsonrow = {'pm25': values[0], 'pm10': values[1], 'time': time.strftime("%d.%m.%Y %H:%M:%S")}
        data.append(jsonrow)

        # save it
        with open(JSON_FILE, 'w') as outfile:
            json.dump(data, outfile)

        if MQTT_HOST != '':
            pub_mqtt(jsonrow)

        print("Going to sleep for 1 min...")
        cmd_set_sleep(1)
        time.sleep(60)</code>

IOT Project – Using an ESP8266 with IOS Push Notifications

Following on from a recent post http://www.geothread.net/iot-project-using-an-esp32-device-to-check-a-web-service/, here at Cranfield University, we wanted to explore the use of ‘Push Notifications‘ to a mobile phone or tablet from an event triggered on a ESP8266 Internet of Things device. This could be useful for a range of applications – for example following a trigger from a sensor to indicate that some threshold has been exceeded (e.g. a set temperature or humidity), or from the utility previously described, testing periodically to see if a web service is running or not.

Contents:
Parts required
Push notification configuration and Prowl
Hardware
Configuring Arduino
Test Sketch
Results
Next Steps
Buttons Sketch
Web Service monitoring with push notifications

To get this all working we need a simple test rig, described here, to bring together all the parts.

Parts required:

top
IOS device – Apple iPhone (Android is OK too – comments below)
– installed with ‘Prowl’ app
ESP8266 device – We used a ‘TOOGOO ESP8266’
Arduino IDE correctly configured
– installed with ESP8266WiFi and EspProwl libraries
– suitable serial driver

Push notification configuration and Prowl:

top
If we want to be informed of Internet events, we can make devices trigger communications in. number of ways – examples being emails, tweets, and push notifications. Of these, ‘Push Notifications’ are the most immediate. They can put a note up on the home screen of a mobile device (phone or tablet) to draw immediate attention. We used a mobile phone for this test, an Apple iPhone running IOS.

The leading IOS push notifier is Prowl – https://www.prowlapp.com. The web service is free to use, so we registered a login account with Prowl. The next step is to generate a unique API keycode in Prowl. In doing this, a short description can be added to remind what it relates to, e.g. ‘IoT notifications’. One can have multiple API key codes for different applications/projects.

Once that is done, the next step is to install the Prowl app on the iPhone. The app is available on the iTunes App Store.

Note that the app is not free, but it is priced modestly, and is fair given the great service Prowl provides. Once the app is installed, the account details set above on the website can be entered.

We are now ready to receive push notifications.

Android

The Prowl app is designed for IOS devices. Android has its own equivalent utilities such as PushBullet, described in a similar tutorial blog online here.

Hardware:

top
The last post used an ESP32 device. These chips are newer than the ESP8266 devices, being slightly more expensive, and with BlueTooth as an additional feature alongside WiFi. To keep costs down, we used the TOOGOO ESP8266 device, available from a range of providers, eg. Amazon. The ESP8266 is described well in this blog article.

Configuring Arduino:

top
The ESP8266 device is designed to operate with the Arduino IDE development environment. As described fully in the earlier post, using a MacBook, one needs to install a serial driver to communicate with the hardware, and the appropriate cables. Next, as the device is an ESP8266, we need to install the appropriate device libraries.

In the Arduino ‘Preferences’ dialog, one can add references to external libraries through adding a line to the source. We added the ESP8266 source thus:

http://arduino.esp8266.com/stable/package_esp8266com_index.json

Now in the Arduino, under ‘Sketch’ -> ‘Include Library’, we can see ‘ESP8266WiFi’, which can be added to our test sketch.

For the particular TOOGOO ESP8266 device, we used the Arduino board definition ‘NodeMCU 1.0 (ESP-12e Module)’.

EPS8266 notifications

We now need a means in the Arduino code sketch to configure and initiate push notifications. Fortunately we can use the excellent EspProwl library described online at https://github.com/marcos69/EspProwl. The code here can be downloaded to the library folder of the Arduino installation folder. Now the library can be added to the Sketch code (e.g. #include <EspProwl.h>), and then used in the code.

Once this is all in place, we can develop a minimal test sketch to open a WiFi connection and initiate a push notification.

Test Sketch code:

top

#include &lt;ESP8266WiFi.h&gt;
#include &lt;EspProwl.h&gt;

// WiFi parameters
char* ssid = "MYSSID";
char* password = "MYWIFIKEY";

void setup() {
       // Start Serial
       Serial.begin(115200);
       WiFi.begin(ssid, password);
       while (WiFi.status() != WL_CONNECTED) {
          delay(500);
          Serial.print(".");
       }
       Serial.println("");
       Serial.println("WiFi now connected at address");
       // Print the IP address
       Serial.println(WiFi.localIP());
       EspProwl.begin();
       EspProwl.setApiKey("MY_PROWL_API_CODE");
       EspProwl.setApplicationName("ESP6266 Prowl test");
       int returnCode = EspProwl.push("Hello World event", "Initial notification", 0);
}

void loop() {
  // not used in this test
}

Results:

top

The result of the test sketch is that once it runs, a WiFi connection is made, and then a push notification initiated. Note that the push here is placed in the ‘Setup’ block – so it only runs once (fine for testing). Note also that one can firstly set an Application Name, and secondly, a Push Title and Message. This permits a lot of flexibility in the messaging options open. Hopefully when run the results will appear on the mobile device as shown.

The Prowl app does permit some filtering and the ability to (temporarily) filter messages and have quiet times etc. which could be useful if there are a lot of messages appearing.

Next Steps:

top
This project has shown how to create push notifications, triggered by the ESP8266 device. Note, we used the Setup block in the code example above – in reality one would use the ‘Loop’ block for event monitoring and message triggering. However, more programmatic control is needed for this to be used in a real project. For example, if one is say monitoring temperature every few minutes, the operator may not wish to receive a push notification at the same frequency, or once messaged, another trigger notification may not be wanted until the threshold is crossed once more. In either case, a secondary ‘loop’ or register could be configured in the code sketch, within the continuously running 5 minute loop, so as to restrict messaging to say an hourly basis or just when significant changes occur – this might prove especially relevant if some other action is being undertaken at the finer timestep – such as database logging of data. It depends on the application – but overall push notifications offer a very useful tool for building up control and monitoring systems.

Buttons Sketch:

top
The hardware device also has a few in-built buttons. So we created a simple sketch that triggers push notifications when they are pressed and released:

#include &lt;avdweb_Switch.h&gt;
// See https://github.com/avandalen/avdweb_Switch
#include &lt;ESP8266WiFi.h&gt;
#include &lt;EspProwl.h&gt;
// Drawing on code at https://github.com/marcos69/EspProwl

// Buttons - FLATH, RSET, D5, D6, D7
const byte buttonUp_Pin = D6;
Switch buttonUp = Switch(buttonUp_Pin);
const byte buttonDown_Pin = D7;
Switch buttonDown = Switch(buttonDown_Pin);
const byte buttonPress_Pin = D5;
Switch buttonPress = Switch(buttonPress_Pin);
int i;

// WiFi parameters
char* ssid = "MYSSID";
char* password = "MYWIFIKEY";

void setup() {
       Serial.begin(115200);
       Serial.println("Starting:");
       WiFi.begin(ssid, password);
       while (WiFi.status() != WL_CONNECTED) {
          delay(500);
          Serial.print(".");
       }
       Serial.println("");
       Serial.println("WiFi now connected at address");
       // Print the IP address
       Serial.println(WiFi.localIP());
       EspProwl.begin();
       EspProwl.setApiKey("MY_PROWL_API_CODE");
       EspProwl.setApplicationName("ESP6266 Prowl test");
       int returnCode = EspProwl.push("Button press test", "Ready", 0);
}

void loop() {
       buttonUp.poll();
       if(buttonUp.pushed()) {
           Serial.print(++i); Serial.print(" "); Serial.print("Up pushed, ");
           int returnCode = EspProwl.push("Up", "Pushed", 0);
       }
       if(buttonUp.released()) {
           Serial.println("Up released");
           int returnCode = EspProwl.push("Up", "Release", 0);
       }
       buttonDown.poll();
       if(buttonDown.pushed()) {
           Serial.print(++i); Serial.print(" "); Serial.print("Down pushed, ");
           int returnCode = EspProwl.push("Down", "Pushed", 0);
       }
       if(buttonDown.released()) {
            Serial.println("Down released");
            int returnCode = EspProwl.push("Down", "Release", 0);
       }
       buttonPress.poll();
       if(buttonPress.pushed()) {
            Serial.print(++i); Serial.print(" "); Serial.print("Press pushed, ");
            int returnCode = EspProwl.push("Press", "Pushed", 0);
       }
       if(buttonPress.released()) {
            Serial.println("Press released");
            int returnCode = EspProwl.push("Press", "Release", 0);
       }
}


Web Service monitoring with push notifications:

top
An earlier blog showed how to use an EPS device to monitor a web service, http://www.geothread.net/iot-project-using-an-esp32-device-to-check-a-web-service/. We can revisit that code now, and add in Prowl push alert code. We can also make the code a little more sophisticated too to avoid false positives. As before you will need to add in the SSID and WiFi passcode, and the website address and REST endpoint, as well as the Prowl API code. Here is the source code, edited to work on an EPS8266 device, rather than EPS32 device (its the slightly cheaper older chip).

// TTGO EPS8266_WebServiceCheck : WiFi & Bluetooth Battery ESP32 Module - webservices checker

// Import required libraries
#include <Wire.h>
#include <OLEDDisplayFonts.h>
#include <OLEDDisplay.h>
#include <OLEDDisplayUi.h>
#include <SSD1306Wire.h>
#include <SSD1306.h>
#include "images.h"
#include "fonts.h"
#include <ESP8266WiFi.h>
#include <EspProwl.h>

// The built-in OLED is a 128*64 mono pixel display
// i2c address = 0x3c
// SDA = 5
// SCL = 4
SSD1306 display(0x3c, 5, 4);

// Web service to check
const int httpPort = 80;
const char* host = "MYWEBSERVICE_HOSTNAME";

// WiFi parameters
const char* ssid = "MYSSID";
const char* password = "MYWIFIKEY";

void setup() {
       // Initialize the display
       display.init();
       //display.flipScreenVertically();
       display.setFont(Roboto_Medium_14);

       // Start Serial
       Serial.begin(115200);
       // Connect to WiFi
       display.drawString(0, 0, "Going online");
       display.drawXbm(34, 14, WiFi_Logo_width, WiFi_Logo_height, WiFi_Logo_bits);
       display.display();
       WiFi.begin(ssid, password);
       while (WiFi.status() != WL_CONNECTED) {
          delay(500);
          Serial.print(".");
       }
       Serial.println("");
       Serial.println("WiFi now connected at address");
       // Print the IP address
       Serial.println(WiFi.localIP());
       display.clear();
       EspProwl.begin();
       EspProwl.setApiKey("MY_PROWL_API_CODE");
       EspProwl.setApplicationName("Web Service Checker");
       int returnCode = EspProwl.push("Checker", "System up", 0);
}

void loop() {
       Serial.print("\r\nConnecting to ");
       Serial.println(host);
       display.clear();
       display.setTextAlignment(TEXT_ALIGN_LEFT);
       display.drawString(0, 0, "Check web service");
       display.display();
       Serial.println("Check web service");
       
       // Setup URI for GET request
       String url = "/PATH/TO/REST/ENDPOINT/";
       // if service is up ok, return will contain: 'Service running'

       WiFiClient client;
       if (!client.connect(host, httpPort)) {
         Serial.println("Connection failed");
         display.clear();
         display.drawString(0, 0, "Connection failed");
         display.display();
         return;
       }

       client.print("GET " + url + " HTTP/1.1\r\n");
       client.print("Host: " + (String)host + "\r\n");
       client.print("Authorization: Basic YWRtaW46NSs2KndralhLcVApOVd2JWokQ2o=\r\n");
       client.print("User-Agent: Arduino/1.0\r\n");
       client.print("Cache-Control: no-cache\r\n\r\n");

       // Read all the lines of the reply from server
       delay(800);
       bool running = false;
       while (client.available()) {
         String line = client.readStringUntil('\r\n');
         Serial.println(line);
         if (line == "Service running") {
           running = true;
         }
       }
       if (running == true) {
           display.drawString(0, 25, "Service up OK");
           display.display();
           delay(3000);
           ProwlAlert(running);       
       } else {
           display.drawString(0, 25, "Service DOWN");
           display.display();
           delay(3000);
           ProwlAlert(running);       
       }

       Serial.println();
       Serial.println("Closing connection");
       Serial.println("=================================================");
       Serial.println("Sleeping");
       display.clear();
       display.drawString(0, 0, "Closing connection");
       display.display();
       delay(1000);
       display.clear();
       client.stop();
       // progress bar
       for (int i=1; i<=28; i++) {
         float progress = (float) i / 28 * 100;
         delay(500); // = all adds up to delay 14000 (14 sec)
         // draw percentage as String
         display.drawProgressBar(0, 32, 120, 10, (uint8_t) progress);
         display.display();
         display.setTextAlignment(TEXT_ALIGN_CENTER);
         display.drawString(64, 15, "Sleeping " + String((int) progress) + "%");
         display.display();
         display.clear();
         Serial.print((int) progress);Serial.print(",");
       }
       delay (1000);
}

int ProwlAlert(bool running) {
        // prevent too many false positives
        static int counter = 0;
        static bool resumed = true;
        int returnCode = -999;
        Serial.print("Status: ");
        Serial.print(running);
        Serial.print(" | Run count: ");
        Serial.println(counter);
        if (running == true) {
           counter = 0;
           if (resumed == false) {
              returnCode = EspProwl.push("Checker", "Service resumed as normal", 0);
              resumed = true;
           }
        } else {
           counter = counter + 1;
           resumed = false;       
        }
        if (counter == 5) {
           returnCode = EspProwl.push("Checker", "Service detected as being down - attention required", 0);
           counter = 0;
        }
        return returnCode;
}