Voice Activated Wio Node temperature sensor

Here at Cranfield University we are interested in the ‘Internet of Things’ for environmental sensing. In this blog, we use one off SeedStudio’s Wio Node devices, with a home-made connection to a one wire temperature sensor, and finally an IOS shortcut on an iPhone to read and speak the value using Siri. In part, this blog follows this project.

Hardware

The WioNode from SeeedStudio

Microprocessor: We’ve had a few projects reported here with Arduino’s, Photons and Raspberry Pis, so it was interesting to get hold of a Wio Node device from Seeed Studio. This is an inexpensive ESP8266 based open-source Wi-Fi development board that supports Seeed’s Plug-n-Play Grove standard. There are two Grove sockets on the Node device. The node is marketed as permitting one to create a working IoT application in 5 minutes. Key to this is the Plug-n-Play physical sensor design, and the modular software tools in the app.

Sensor: To measure temperature, we used a DS18B20 Waterproof Temperature Sensor, available cheaply from a range of sources, such as here. The three wires are Red(VCC), Yellow (DATA), Black(GND).

Construction

The Grove standard is a modular, standardized connector prototyping system, permitting Plug-n-Play connectivity with scores of sensor devices for measuring a wide range of phenomena. The advantage of Seeed Studio’s Wio range is that these simply plug into the sockets on the board. However, if one has a sensor without a Grove plug, there is a Protoshield prototyping board available. This can be used to wire up a sensor such as the DS18B20 to allow it to be connected.

Unfortunately we didn’t have a Protoshield, however, we did have a pre-wired grove compatible plug that fitted the socket on the Wio Node – so a visit to the trusty soldering iron and heat-shrink cable was required. To make the DS18B20 work, a 4.7k pull-up resistor is required between the VCC and signal lines. The wiring diagram is as follows:

Wio Node wiring diagram

After wiring up the temperature sensor as shown, and plugging it in, we could turn to the software to control it.

Software controller

Wio Link Configuration

There is a Wio Link App available for smartphones, such as the Apple phone, that hugely simplifies the process of plugging in and operating sensors. We installed the app, and followed the simple instructions to add a new Wio Node device, and to enable Wi-Fi on it. Once this was working, we configured the Node to add our new sensor to the ‘S0’ port (there are two ports – S0 and S1).

REST endpoints for new sensor

The option used was ‘One Wire Temperature Sensor’ on the GPIO toolbar. Once this was working, we ‘flashed’ the new settings onto the Node via the ‘Update Firmware’ option. Next, we could view the API settings for the new configuration.

The URL REST endpoints are shown and can be used to access the temperature value from the sensor. A test option within the app allows one to access the reading directly.

The REST URL takes the form, thus:

https://us.wio.seeed.io/v1/node/GroveTemp1WireD1/temp?access_token=TOKEN_GOES_HERE

The temperature values are returned as a JSON string, appearing thus:

{"temperature":19.1800000000001}

To access the actual value from the JSON, we will need to extract the number from the data pair (see step 3 below). Once we were happy the approach was all working, we copied the URL off for use in the next stage of the project.

Voice control

The project at this point is all working well – the temperature sensor reports to the Wio Node, and an API REST URL can be used to access the value. However, we can go further. We next used an Apple iPhone X, and its newly introduced ‘Shortcuts’ capability. This involves firstly installing the Shortcuts app from the iTunes App Store.

Shortcuts allow one to build a relatively sophisticated programme very simply. We therefore set out to develop a ‘Shortcut’ that accessed the new sensor – for this we needed the URL (REST endpoint) from before. The shortcut settings are shown below, namely: (1) URL – enter in the URL from above; (2) Get Contents of URL [use Advanced options – method: GET, Header: temperature; (3) Get Dictionary Value [Key – temperature]; (4) Set Variable [Temperature] (saves value off to a new variable called Temperature); (5) Text [degrees Celsius] (creates a block of text to append); (6) Add to Variable [Temperature] (appends text to variable); (7) Combine Text [Separator – New Lines]; (8) Speak Text [Wait Until Finished]. A further elaboration is to use the ‘If‘ statement to check there was a valid reading (e.g. temperature > -20) ‘Else‘ advise of a problem.

We then added a custom icon ‘glyph’ (thermometer!) and colouring for the shortcut, and added this to the home screen of the phone. This allows the shortcut to be run easily.

As a final finishing touch, we then also added a Siri shortcut with a simple voice command ‘temperature’ – this allows one to call ‘Hey Siri’ and give the command. The device sensor temperature reading is then spoken back. Pretty amazing! See the video below for an example.

Epilogue

This approach worked well, and could be developed to work with any of the other Grove sensors plugged into the Wio Node. For more complex projects, the Wio Node, with its two connectors could be limiting. Fortunately there are a whole range of extended micro controllers, such as the Wio Link, with 6 Grove connectors – as well as lots of great projects to try.

Perl vs Python

Here at Cranfield University we work a lot with data in our GIS and data-related teaching and research. A common challenge is in transferring a complex dataset that is in one format into another format to make it useable. Many times there are tools we can use to help in that manipulation, both proprietary and open source. For the spatial datasets we often work with, we can use the range of data convertors in ArcGIS and QGIS, we can use the fantastic ‘Feature Manipulation Engine’ (FME) from Safe inc., or its manifestation in ArcGIS – the data interoperability tool, then again we can look to libraries such as the Geospatial Data Abstraction Library (gdal) for scripted functionality. As ever in computing, there are many ways of achieving our objectives.

However, sometimes there is nothing for it but to hack away in a favourite programming scripting language to make the conversion. Traditionally we used the wonderfully eclectic ‘Perl‘ language (pathologically eclectic rubbish lister – look it up!!) More recently the emphasis has perhaps shifted to Python as the language of choice. Certainly, if we are asked by our students which general purpose programming language to use for data manipulation, we advise Python is the one to have experience with on the CV.

If we have a simple data challenge, for example, we might want to convert an ASCII text file with data in one format to another format and write it out to a new file. We might want to go say from a file in this format (in ‘input.csv’):

AL1 1AG,1039499.00,0
AL1 1AG,383009.76,10251
etc......

To this format (in ‘output.csv’) …

UK,Item 1,R,AL1 1AG,,,,,,,,,,1039499.00,0,,,,
UK,Item 2,R,AL1 1AG,,,,,,,,,,383009.76,10251,,,,
etc......

For this Perl is a great solution – integration the strengths of awk and sed. Perl can produce code which quickly chomps through huge data files. One has to be careful as to how the code is developed, to ensure its readability. Sometimes, coming back to a piece of code one can struggle to remember how it works for a while – and this is especially so where the code is highly compacted.

#!/usr/bin/env perl
# Call as ‘perl script.pl <in_file> > <out_file>‘
# e.g. perl script.pl input.csv > output.csv
use Text::CSV;
my $csv = Text::CSV->new({sep_char => ',' });
$j=1;
while (<>) {
  chomp;
  if ($csv->parse($_)) {
    my @fields = $csv->fields();
    printf("UK,Item %d,R,%s,,,,,,,,,,%s,%s,,,,\n",$j++,@fields[0],@fields[1],@fields[2]);
  }       
}

The equivalent task in Python is equally simple, and perhaps a little more readable…

#!/usr/bin/env python
# python3 code
# Call as 'python3 script.py'
import csv
o = open('output.csv','w')
with open('input.csv', 'r') as f:
   reader = csv.reader(f)
   mylist = list(reader)
j = 0
for row in mylist:
   j+=1
   o.write('UK,Item {:d},R,{:s},,,,,,,,,{:s},{:s},,,,\n'.format(j, row[0], row[1], row[2]$

Note the code above is Python3 not Python2. Like Perl (with cpan), Python is extensible (with pip) – and in fact one really needs to use extensions (modules, or imported libraries) to get the most out of it (and to help prevent you needing to reinvent the wheel and introducing unnecessary errors). There is no need to write lots of code for handling CSV files for example – the csv library above does this very efficiently in Python. Likewise, if say we want to write data back out to JSON (JavaScript Object Notation format), again the json library can come to the rescue:

import csv
import json
jsonfile = open('/folderlocation/output.json', 'w')
with open('/folderlocation/input.csv', newline='', encoding='utf-8-sig') as csvfile:
    reader = csv.reader(csvfile)
    for row in reader:
        print(', '.join(row))
        json.dump(row, jsonfile)

There is probably not really a lot in the difference between the two languages – it all rather depends on ones preferences. However, for GIS professionals, Python expertise is a must as it is adopted as the scripting language of choice in ArcGIS (in fact even being shipped with ArcGIS). Other alternatives exist of course for these sorts of tasks – ‘R‘ is one that comes to mind – again being equally extensible.

Machine Vision with a Raspberry Pi

In this blog we will describe the steps needed to do some machine vision using the Raspberry Pi Zeros we described in the earlier blog. Here at Cranfield University we are building these amazing devices into our research. In this case we are interested in using the Pi as a device for counting pedestrians passing a site – trying to understand how different design choices influence people’s choice of walking routes.

Contents:
Background
Toolkits
Kerberos
   Installation of Kerberos
   Configuration of Kerberos
   Configuration of the Pi
   Output and Data Capture from Kerberos
Epilogue

Background:

top
In the earlier blog we showed how to set up the Raspberry Pi Zero W, connecting up the new v2 camera in a case and connecting power. Once we had installed Rasbian on a new microSD card all was ready to go.

A bit of research was needed to understand the various options for machine vision on a Pi. There are three levels we might want. First a simple motion detection with the camera would give a presence or absence of activity, but not much more. This could be useful when pointing the camera directly at a location. Second, we can use more sophisticated approaches to consider detecting movement passing across the camera’s view, for example left to right or vice versa. This could be useful when pointing the camera transverse to a route along which pedestrians are travelling. Thirdly, and with the ultimate sophistication, we could try and classify the image to detect what the ‘objects’ passing across the view are. Classifier models might for example detect adults, young persons, and other items such as bicycles and push buggies etc. Needless to say, we wanted to start off easy and then work up the list!

Looking at the various software tools available, it is clear that many solutions draw on OpenCV (Open Source Computer Vision Library) (https://opencv.org). OpenCV is an open source computer vision and machine learning software library, built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception. There are many other potential libraries for machine vision – for example, SOD (https://sod.pixlab.io), and other libraries such as Dlib (http://dlib.net). OpenCV can be daunting, and there are wrappers such as SimpleCV (http://simplecv.org) to try and simplify the process.

Toolkits:

top
We then looked at options for toolkits that use these basic building blocks. A useful reference is Jason Antman’s blog here https://blog.jasonantman.com/2018/05/linux-surveillance-camera-software-evaluation/. Although not Jason’s final choice, the tool that stuck out to us was Kerberos (https://kerberos.io), developed by Cedric Verstraeten and grown out of his earlier OpenCV project (https://github.com/cedricve/motion-detection).

Kerberos:

top
Kerberos has a number of key resources:
Main home website – https://kerberos.io
Documentation – https://doc.kerberos.io
Git – https://github.com/kerberos-io
Helpdesk – https://kerberosio.zendesk.com
Corporate – https://verstraeten.io

Although the full source for Kerberos is available, and also a docker implementation, what we really liked was the SD image for the Raspberry Pi Zero – so really made for the job.

Installation of Kerberos:

top
We downloaded the cross-platform installer from the Kerberos website. This is based on the Etcher tool, used to install Rasbian so familiar to any Pi user. In our case we selected the Mac installer, downloading an installer dmg file (c.80Mb). Then, ensuring the Micro SD card destined for the Pi was in a flash writer dongle attached to the Mac, we were able to easily install the image. The Etcher app asks a couple of questions on the way about the WiFi network SSID and WiFi and system passcodes, as well as a name for the device, and writes these details onto the SD card with the rest of the image. As a result, on inserting the SD card and booting the Pi with the Kerberos image, the device started up and connected correctly and without issue on the WiFi network. A check on the router on our closed network showed the device had correctly registered itself at IP address 192.168.1.24.

Management of the Pi and camera is achieved via app running on a web server on the Pi. So to access our device, we entered browsed the URL http://192.168.1.24/login.

Configuration of Kerberos:

top
The dashboard app provides complete control over the operation of the Pi and camera.The image here shows the ‘heatmap’ camera view, and statistical graphs and charts of timings of activations.
. To configure the many settings we headed over to https://doc.kerberos.io for the documentation. The concept is that the image processing is undertaken on the ‘Machinery’ configuration, and that the ‘Web’ then controls access to the results.

Selecting ‘Configuration’ we could start adjusting the settings for the Machinery as we required. There are default settings for all the options.
However, the settings you will use depend on the application for the device. We followed the settings for ‘People Counter‘ recommended both in the docs, and a subsequent blog. It seems that there the settings are very sensitive, so one has to adjust until the desired results are obtained.

Being on a Raspberry Pi, one can also ssh connect directly to the device on a terminal connection (eg from terminal on the Mac, or via Putty from a PC). Connect to the device with the command:

ssh root@192.168.1.22
cd /data/machinery/config

This takes you to the location of the configuration files, as written out by the web app. Below are the settings we used to get the People Counter working (the values here correspond to the settings in the web app).

less config.xml
<?xml version="1.0"?>
<kerberos>
    <instance>
        <name type="text">Dream</name>
        <logging type="bool">false</logging>
        <timezone type="timezone">Europe-London</timezone>
        <capture file="capture.xml">RaspiCamera</capture>
        <stream file="stream.xml">Mjpg</stream>
        <condition file="condition.xml" type="multiple">Time</condition>
        <algorithm file="algorithm.xml">BackgroundSubtraction</algorithm>
        <expositor file="expositor.xml">Hull</expositor>
        <heuristic file="heuristic.xml">Counter</heuristic>
        <io file="io.xml" type="multiple">Script,Disk</io>
        <cloud file="cloud.xml">S3</cloud>
    </instance>
</kerberos>
less capture.xml
<?xml version="1.0"?>
<captures>
    <IPCamera>
        <url type="text">rtsp://admin:888888@192.168.0.13/tcp/av0_1</url>
        <frameWidth type="number">640</frameWidth>
        <frameHeight type="number">480</frameHeight>
        <delay type="number">500</delay>
        <angle type="number">0</angle>
    </IPCamera>
    <USBCamera>
        <frameWidth type="number">640</frameWidth>
        <frameHeight type="number">480</frameHeight>
        <deviceNumber type="number">0</deviceNumber>
        <fourcc type="text">MJPG</fourcc>
        <delay type="number">500</delay>
        <angle type="number">0</angle>
    </USBCamera>
    <RaspiCamera>
        <frameWidth type="number">640</frameWidth>
        <frameHeight type="number">480</frameHeight>
        <delay type="number">0</delay>
        <angle type="number">0</angle>
        <framerate type="number">30</framerate>
        <sharpness type="number">0</sharpness>
        <saturation type="number">0</saturation>
        <contrast type="number">0</contrast>
        <brightness type="number">50</brightness>
    </RaspiCamera>
    <VideoCapture>
        <frameWidth type="number">640</frameWidth>
        <frameHeight type="number">480</frameHeight>
        <path type="text">0</path>
        <delay type="number">500</delay>
        <angle type="number">0</angle>
    </VideoCapture>
</captures>
less stream.xml
<?xml version="1.0"?>
<streams>
    <Mjpg>
    	<enabled type="bool">true</enabled>
    	<streamPort type="number">8889</streamPort>
    	<quality type="number">75</quality>
    	<fps type="number">30</fps>
    	<username type="text"></username>
    	<password type="text"></password>
    </Mjpg>
</streams>
less condition.xml
<?xml version="1.0"?>
<conditions>
    <Time>
        <times type="timeselection">7:00,18:30-7:00,18:30-7:00,18:30-7:00,18:30
-7:00,18:30-7:00,18:30-7:00,18:30</times>
        <delay type="number">10000</delay>
    </Time>
    <Enabled>
    	<active type="bool">true</active>
        <delay type="number">5000</delay>
    </Enabled>
</conditions>
less algorithm.xml
<?xml version="1.0"?>
<algorithms>
    <DifferentialCollins>
        <erode type="number">5</erode>
    	<threshold type="number">15</threshold>
    </DifferentialCollins>
	<BackgroundSubtraction>
		<shadows type="text">false</shadows>
		<history type="number">5</history>
		<nmixtures type="number">5</nmixtures>
		<ratio type="number">1</ratio>
		<erode type="number">6</erode>
		<dilate type="number">15</dilate>
    	<threshold type="number">20</threshold>
    </BackgroundSubtraction>
</algorithms>
less expositor.xml
<?xml version="1.0"?>
<expositors>	
	<Rectangle>
	    <region>
		    <x1 type="number">0</x1>
		    <y1 type="number">0</y1>
		    <x2 type="number">640</x2>
		    <y2 type="number">480</y2>
		 </region>
	</Rectangle>
	<Hull>
	    <region type="hullselection">640,286|640,243|539,223|332,226|184,22
9|188,323|188,392|210,478|640,480|640,316|640,308|640,297</region>
	</Hull>
</expositors>
less heuristic.xml
<?xml version="1.0"?>
<heuristics>
	<Sequence>
	    <minimumChanges type="number">20</minimumChanges>
	    <minimumDuration type="number">2</minimumDuration>
        <noMotionDelayTime type="number">1000</noMotionDelayTime>
	</Sequence> 
	<Counter>
	    <appearance type="number">3</appearance>
	    <maxDistance type="number">400</maxDistance>
	    <minArea type="number">25</minArea>
	    <onlyTrueWhenCounted type="bool">true</onlyTrueWhenCounted>
	    <minimumChanges type="number">15</minimumChanges>
        <noMotionDelayTime type="number">0</noMotionDelayTime>
		<markers type="twolines">211,281|213,442|396,284|397,439</marke
rs>
	</Counter>
</heuristics>

Note the settings above for the twolines markers on the video image – used for counting pedestrians passing from left to right, and from right to left, (coordinate position 0,0 is the top left corner)

less io.xml
<?xml version="1.0"?>
<ios>
    <Disk>
        <fileFormat type="text">timestamp_microseconds_instanceName_regionCoord
inates_numberOfChanges_token.jpg</fileFormat>
        <directory type="text">/etc/opt/kerberosio/capture/</directory>
        <markWithTimestamp type="bool">true</markWithTimestamp>
        <timestampColor type="text">white</timestampColor>
        <privacy type="bool">false</privacy>
        <throttler type="number">0</throttler>
    </Disk>
    <Video>
        <fps type="number">30</fps>
        <recordAfter type="number">5</recordAfter>
        <maxDuration type="number">30</maxDuration>
        <extension type="number">mp4</extension>
        <codec type="number">h264</codec>
        <fileFormat type="text">timestamp_microseconds_instanceName_regionCoord
inates_numberOfChanges_token</fileFormat>
        <directory type="text">/etc/opt/kerberosio/capture/</directory>
        <hardwareDirectory type="text">/etc/opt/kerberosio/h264/</hardwareDirec
tory>
        <enableHardwareEncoding type="bool">true</enableHardwareEncoding>
        <markWithTimestamp type="bool">true</markWithTimestamp>
        <timestampColor type="text">white</timestampColor>
        <privacy type="bool">false</privacy>
        <throttler type="number">0</throttler>
    </Video>
    <GPIO>
        <pin type="number">17</pin>
        <periods type="number">1</periods>
        <periodTime type="number">100000</periodTime>
        <throttler type="number">0</throttler>
    </GPIO>
    <TCPSocket>
        <server type="number">127.0.0.1</server>
        <port type="number">1337</port>
        <message type="text">motion-detected</message>
        <throttler type="number">0</throttler>
    </TCPSocket>
    <Webhook>
        <url type="text">http://localhost/api/v1/webhook</url>
        <throttler type="number">0</throttler>
    </Webhook>
    <Script>
        <path type="text">/data/run.sh</path>
        <throttler type="number">0</throttler>
    </Script>
    <MQTT>
        <secure type="bool">false</secure>
        <verifycn type="bool">false</verifycn>
        <server type="number">127.0.0.1</server>
        <port type="number">1883</port>
        <clientId type="text"></clientId>
        <topic type="text">kios/mqtt</topic>
        <username type="text"></username>
        <password type="text"></password>
        <throttler type="number">0</throttler>
    </MQTT>
    <Pushbullet>
        <url type="text">https://api.pushbullet.com</url>
        <token type="text">o.mC5LPVCvPCphtSsEgWZQpFM86w9ciWQ3</token>
        <throttler type="number">10</throttler> 
    </Pushbullet>
</ios>

Configuration of the Pi:

top
Another configuration required was to tun off the bright green LED on the Raspberry Pi as it draws attention when the unit is operating. To turn OFF the LEDs for Zero, we followed the instructions at https://www.jeffgeerling.com/blogs/jeff-geerling/controlling-pwr-act-leds-
raspberry-pi. Note that unlike other Raspberry Pi models, the Raspberry Pi Zero only has one LED, led0 (labeled ‘ACT’ on the board). The LED defaults to on (brightness 0), and turns off (brightness 1) to indicate disk activity.

To turn off the LEDs interactively, the following commands can be run each time the Pi boots.

# Set the Pi Zero ACT LED trigger to 'none'.
echo none | sudo tee /sys/class/leds/led0/trigger
# Turn off the Pi Zero ACT LED.
echo 1 | sudo tee /sys/class/leds/led0/brightness

To make these settings permanent, add the following lines to the Pi’s ‘/boot/config.txt’ file and reboot:

# Disable the ACT LED on the Pi Zero.
dtparam=act_led_trigger=none
dtparam=act_led_activelow=on

Note the ‘/’filesystem is made read-only by default in the Kerberos build. To temporarily fix this to force read write for the root ‘/’ filesystem, type:

mount -o remount,rw /

Now the config.txt file can be edited normally, eg in the editor nano, and then the Pi can be rebooted.

cd /boot
nano config.txt
reboot

Output and Data Capture from Kerberos:

top
To obtain data from the tool, we are using the ‘script’ setting in io.xml, which runs the script ‘/data/run.sh’ (a bash script). This script just writes the data receives (a JSON structure) out to disk.

#!/bin/bash

# -------------------------------------------
# This is an example script which illustrates
# how to use the Script IO device.
#

# --------------------------------------
# The first parameter is the JSON object
#
# e.g. {"regionCoordinates":[308,250,346,329],"numberOfChanges":194,"timestamp":"1486049622","microseconds":"6-161868","token":344,"pathToImage":"1486049622_6-161868_frontdoor_308-250-346-329_194_344.jpg","instanceName":"frontdoor"}

JSON=$1

# -------------------------------------------
# You can use python to parse the JSON object
# and get the required fields

echo $JSON >> /data/capture_data.json

coordinates=$(echo $JSON | python -c "import sys, json; print json.load(sys.stdin)['regionCoordinates']")
changes=$(echo $JSON | python -c "import sys, json; print json.load(sys.stdin)['numberOfChanges']")
incoming=$(echo $JSON | python -c "import sys, json; print json.load(sys.stdin)['incoming']")
outgoing=$(echo $JSON | python -c "import sys, json; print json.load(sys.stdin)['outgoing']")
time=$(echo $JSON | python -c "import sys, json; print json.load(sys.stdin)['timestamp']")
microseconds=$(echo $JSON | python -c "import sys, json; print json.load(sys.stdin)['microseconds']")
token=$(echo $JSON | python -c "import sys, json; print json.load(sys.stdin)['token']")
instancename=$(echo $JSON | python -c "import sys, json; print json.load(sys.stdin)['instanceName']")

printf "%(%m/%d/%Y %T)T\t%d\t%d\t%d\t%d\n" "$time" "$time" "$changes" "$incoming" "$outgoing" >> /data/results.txt

Note the use of the parameters to convert the Julian timestamp to a readable date/time.

When an event triggers the system (someone walking past the camera view) two actions follow, an image is saved to disk, and the script is run, with a parameter of the JSON structure. The script then processes the JSON. The script here both writes out the whole JSON structure to a the file ‘capture_data.json’ (this is included as a debug and could be omitted), and also extracts out the data elements we actually wanted and writes these to a CSV file called ‘results.txt’.

A sample of ‘capture_data.json’ look like this:

{"regionCoordinates":[413,323,617,406],"numberOfChanges":1496,"incoming":1,"outgoing":0,"name":"Dream","timestamp":"1539760397","microseconds":"6-928567","token":722,"instanceName":"Dream"}
{"regionCoordinates":[190,318,636,398],"numberOfChanges":2349,"incoming":1,"outgoing":0,"name":"Dream","timestamp":"1539760405","microseconds":"6-747074","token":814,"instanceName":"Dream"}
{"regionCoordinates":[185,315,279,436],"numberOfChanges":1793,"incoming":0,"outgoing":1,"name":"Dream","timestamp":"1539760569","microseconds":"6-674179","token":386,"instanceName":"Dream"}

A sample of ‘results.txt’ looks like this:

10/17/2018 08:17:08	1539760628	917	0	1
10/17/2018 08:17:18	1539760638	690	0	1
10/17/2018 08:18:56	1539760736	2937	0	1
10/17/2018 08:19:38	1539760778	3625	1	0
10/17/2018 08:22:05	1539760925	1066	1	0
10/17/2018 08:24:06	1539761046	2743	0	1
10/17/2018 08:24:45	1539761085	1043	1	0
10/17/2018 08:26:11	1539761171	322	0	1

Epilogue:

top
This blog has shown how the Kerberos toolkit has been used with an inexpensive Raspberry Pi for detecting motion and also directional movement across the camera view. The data captures a JSON data structure for each event triggered, and a script extracts from this the data required, which is saved off to disk for later use.

There are still issues to grapple with – for example reduce false positives, and perhaps more importantly not missing events as they occur. The settings of the configuration machinery are very sensitive. The best approach is to successively vary these settings (particularly the expositor and heuristic settings) until the right result is obtained. Kerberos has a verbose setting for event logging, and inspecting the log with this switched on reveals that the Counter conditions are very sensitive – so many more people may be walking past the camera than are being directly logged as such (e.g. motion activations may be greater than count events).

The commands below show how to access the log – it is also shown in the ‘System’ tab of the web dashboard. The command ‘tail -f’ is useful as it shows the log update in real time – helpful if the video live feed screen is being displayed alongside on-screen. Then you can see what is and isn’t being logged very easily.

cd /data/machinery/logs
tail -f log.stash

Ultimately, the Raspberry Pi may not have enough power to operate full classifier models, such as that developed by Joseph Redmon with the Darkweb YOLO tool he developed (‘You Only Look Once’) (https://pjreddie.com/darknet/yolo/). However, Kerberos itself has a cloud model that provides post-processing of images in the cloud on AWS servers, with classifier models available – perhaps something to try in a later blog.

Setting up a Raspberry Pi Zero W

Here on GeoThread, we have taken delivery of one of the new Raspberry Pi Zeros for some projects – so here is a blog on how we set it all up for some of the work we get up to at Cranfield University. We’ve had quite a few projects with these devices over the years – reported here on GeoThread, starting back in 2016 with the Raspberry Pi 3, with various tutorials presented. The Raspberry Pi Zero is the latest in the line of these excellent, inexpensive microcomputers, see https://www.raspberrypi.org/products/raspberry-pi-zero/. Following on from the Pi A and B series devices, the Zero is a great entry level machine for learning coding and software development. We have a Raspberry Pi Zero W – the ‘W’ meaning it also has wireless and Bluetooth connectivity.

To get us going, we bought a kit that contained not only the Pi, but also an acrylic case and a power supply. There are many such kits available – we selected the Vilros offering. We also bought the new Raspberry Pi Camera v2 to fit in the case.

Raspberry Pi Zero, Vilros Starter Kit

Note that we have also bought (shown at the bottom centre), an HDMI Mini Type C (male) to HDMI Normal Type A (female) Monitor Cable Display Adapter – the Raspberry Pi Zero only has a HDMI Mini socket. In addition, you may also need to buy a single (or dual) Micro USB Male To USB 2.0 Female OTG (USB On the Go) cable for connecting a USB keyboard (or a USB keyboard and mouse). The HDMI and USB cables/adapters are to be used to get the unit up and running in the first instance, using a keyboard, mouse and monitor. Once the configuration is complete and the WiFi is connected, we will run the Raspberry Pi remotely and so disconnect the keyboard and screen.

We also bought one of the new ‘version 2’ cameras for the Raspberry Pi. This is an 8MPixel camera made by Sony, and a huge improvement over the original camera. It can fit into the case. The case has three alternative lids – shown here is the version with the camera mounting and aperture built in.

Raspberry Pi Zero and unboxing the camera v2

One thing to note immediately is that the Raspberry Pi Zero has a different camera cable connector size to the earlier models of the Pi. This means that you will need to obtain a new ribbon cable connector to replace the one provided. You can easily buy these separately, but the Vilros kit (like other kits) thoughtfully provide this.

Raspberry Pi Zero Camera v2 and case

Raspberry Pi Zero camera socket

The camera fixing has a small plastic bar that can be gently prised outwards to allow the old connector ribbon cable to be slipped out. The new cable can then be slotted in and the plastic bar pushed back into place to secure it. Be sure to insert the cable the right way up to allow contact.This is a delicate operation, but the cable is reasonably robust.

Raspberry Pi Zero Camera v2 ribbon cables

In the image above, the old connector has been removed, and the new one inserted. Note the new connector is very short, allowing it to be fitted inside the case. We are nearly ready to assemble the parts together in the case.

Raspberry Pi Zero Case Camera v2 – ready for fitting

The camera, with its new connector, can now be fitted to the Raspberry Pi Zero. The Zero has another plastic clip for slotting in the ribbon cable. As before, prise away the locking bar and gently insert the cable. When tightly fitted (the right way up to allow contact again), the bar can be pushed back in to grip the cable in place.

Raspberry Pi Zero Case Camera v2 – fitting the camera into the case

Once fitted, the case can be put together. Note that the Vilros kit comes with a small heatsink that can be fitted to the Pi to cool the 1GHz ARM CPU chip. However, with the camera fitted in the case, there is no room for the heatsink also. Time will tell if that is an issue!

Raspberry Pi Zero Case and Camera v2, fully Assembled

The Vilros kit comes with the power supply, fitted with the customary MicroUSB plug.

Raspberry Pi Zero Power Supply

Note that the power socket is the socket to the right, as shown.

Raspberry Pi Zero sockets

Before the case can be clicked into place, there is one last thing needed – the operating system. This is installed on an SD card.
We followed the very clear instructions on the Raspberry Pi website, here https://projects.raspberrypi.org/en/projects/raspberry-pi-setting-up/3. We downloaded the latest Raspbian img file and flashed it onto a new Micro-SD card. To do this on a MacBook Pro, we used the Etcher app (v1.4.4).

SD card Flashing with Etcher

Once the Micro SD card was flashed (e.g. the img file was copied onto it with Etcher), we could insert it carefully into the SD socket on the Raspberry Pi Zero. At this point the case was all clipped together.
We are now ready for the next stage of the project!

Installing QGIS on a Macbook

QGIS (https://qgis.org) is a popular open source Geographical Information System (GIS) tool that we use a lot here at Cranfield University. It is possible to get it running on a Mac running MacOS High Sierra, but it can be a bit of a fiddle. The following instructions were found to work well.

The Mac operating system has no built in package manager, like ‘rpm’ for Linux. However, there are tools that can do the job. A popular one is Homebrew (https://brew.sh). Installing this allows one to install both command line tools which are not installed nubby default (e.g. wget), and also whole binary tools, such as QGIS itself.

Having followed the instructions to install Brew, and updated the installation as directed, the next step is to install the X-Windows window manager, Quartz. Brew can be used for achieving this, thus:

brew cask install xquartz

Next, we can turn to the OSGEO open source geospatial foundation (https://www.osgeo.org). OSGEO have a port of their suite of open source GIS tools ready for Brew, and so following the instructions on the OSGEO Github page, here (https://github.com/OSGeo/homebrew-osgeo4mac), we can run the following:

brew tap osgeo/osgeo4mac
brew install osgeo/osgeo4mac/qgis3

To then run QGis, type qgis3 in the terminal to launch, then pin the dock menu icon to simplify launching it in future.

IOT Project – Using an ESP8266 with IOS Push Notifications

Following on from a recent post http://www.geothread.net/iot-project-using-an-esp32-device-to-check-a-web-service/, here at Cranfield University, we wanted to explore the use of ‘Push Notifications‘ to a mobile phone or tablet from an event triggered on a ESP8266 Internet of Things device. This could be useful for a range of applications – for example following a trigger from a sensor to indicate that some threshold has been exceeded (e.g. a set temperature or humidity), or from the utility previously described, testing periodically to see if a web service is running or not.

Contents:
Parts required
Push notification configuration and Prowl
Hardware
Configuring Arduino
Test Sketch
Results
Next Steps
Buttons Sketch
Web Service monitoring with push notifications

To get this all working we need a simple test rig, described here, to bring together all the parts.

Parts required:

top
IOS device – Apple iPhone (Android is OK too – comments below)
– installed with ‘Prowl’ app
ESP8266 device – We used a ‘TOOGOO ESP8266’
Arduino IDE correctly configured
– installed with ESP8266WiFi and EspProwl libraries
– suitable serial driver

Push notification configuration and Prowl:

top
If we want to be informed of Internet events, we can make devices trigger communications in. number of ways – examples being emails, tweets, and push notifications. Of these, ‘Push Notifications’ are the most immediate. They can put a note up on the home screen of a mobile device (phone or tablet) to draw immediate attention. We used a mobile phone for this test, an Apple iPhone running IOS.

The leading IOS push notifier is Prowl – https://www.prowlapp.com. The web service is free to use, so we registered a login account with Prowl. The next step is to generate a unique API keycode in Prowl. In doing this, a short description can be added to remind what it relates to, e.g. ‘IoT notifications’. One can have multiple API key codes for different applications/projects.

Once that is done, the next step is to install the Prowl app on the iPhone. The app is available on the iTunes App Store.

Note that the app is not free, but it is priced modestly, and is fair given the great service Prowl provides. Once the app is installed, the account details set above on the website can be entered.

We are now ready to receive push notifications.

Android

The Prowl app is designed for IOS devices. Android has its own equivalent utilities such as PushBullet, described in a similar tutorial blog online here.

Hardware:

top
The last post used an ESP32 device. These chips are newer than the ESP8266 devices, being slightly more expensive, and with BlueTooth as an additional feature alongside WiFi. To keep costs down, we used the TOOGOO ESP8266 device, available from a range of providers, eg. Amazon. The ESP8266 is described well in this blog article.

Configuring Arduino:

top
The ESP8266 device is designed to operate with the Arduino IDE development environment. As described fully in the earlier post, using a MacBook, one needs to install a serial driver to communicate with the hardware, and the appropriate cables. Next, as the device is an ESP8266, we need to install the appropriate device libraries.

In the Arduino ‘Preferences’ dialog, one can add references to external libraries through adding a line to the source. We added the ESP8266 source thus:

http://arduino.esp8266.com/stable/package_esp8266com_index.json

Now in the Arduino, under ‘Sketch’ -> ‘Include Library’, we can see ‘ESP8266WiFi’, which can be added to our test sketch.

For the particular TOOGOO ESP8266 device, we used the Arduino board definition ‘NodeMCU 1.0 (ESP-12e Module)’.

EPS8266 notifications

We now need a means in the Arduino code sketch to configure and initiate push notifications. Fortunately we can use the excellent EspProwl library described online at https://github.com/marcos69/EspProwl. The code here can be downloaded to the library folder of the Arduino installation folder. Now the library can be added to the Sketch code (e.g. #include <EspProwl.h>), and then used in the code.

Once this is all in place, we can develop a minimal test sketch to open a WiFi connection and initiate a push notification.

Test Sketch code:

top

#include &lt;ESP8266WiFi.h&gt;
#include &lt;EspProwl.h&gt;

// WiFi parameters
char* ssid = "MYSSID";
char* password = "MYWIFIKEY";

void setup() {
       // Start Serial
       Serial.begin(115200);
       WiFi.begin(ssid, password);
       while (WiFi.status() != WL_CONNECTED) {
          delay(500);
          Serial.print(".");
       }
       Serial.println("");
       Serial.println("WiFi now connected at address");
       // Print the IP address
       Serial.println(WiFi.localIP());
       EspProwl.begin();
       EspProwl.setApiKey("MY_PROWL_API_CODE");
       EspProwl.setApplicationName("ESP6266 Prowl test");
       int returnCode = EspProwl.push("Hello World event", "Initial notification", 0);
}

void loop() {
  // not used in this test
}

Results:

top

The result of the test sketch is that once it runs, a WiFi connection is made, and then a push notification initiated. Note that the push here is placed in the ‘Setup’ block – so it only runs once (fine for testing). Note also that one can firstly set an Application Name, and secondly, a Push Title and Message. This permits a lot of flexibility in the messaging options open. Hopefully when run the results will appear on the mobile device as shown.

The Prowl app does permit some filtering and the ability to (temporarily) filter messages and have quiet times etc. which could be useful if there are a lot of messages appearing.

Next Steps:

top
This project has shown how to create push notifications, triggered by the ESP8266 device. Note, we used the Setup block in the code example above – in reality one would use the ‘Loop’ block for event monitoring and message triggering. However, more programmatic control is needed for this to be used in a real project. For example, if one is say monitoring temperature every few minutes, the operator may not wish to receive a push notification at the same frequency, or once messaged, another trigger notification may not be wanted until the threshold is crossed once more. In either case, a secondary ‘loop’ or register could be configured in the code sketch, within the continuously running 5 minute loop, so as to restrict messaging to say an hourly basis or just when significant changes occur – this might prove especially relevant if some other action is being undertaken at the finer timestep – such as database logging of data. It depends on the application – but overall push notifications offer a very useful tool for building up control and monitoring systems.

Buttons Sketch:

top
The hardware device also has a few in-built buttons. So we created a simple sketch that triggers push notifications when they are pressed and released:

#include &lt;avdweb_Switch.h&gt;
// See https://github.com/avandalen/avdweb_Switch
#include &lt;ESP8266WiFi.h&gt;
#include &lt;EspProwl.h&gt;
// Drawing on code at https://github.com/marcos69/EspProwl

// Buttons - FLATH, RSET, D5, D6, D7
const byte buttonUp_Pin = D6;
Switch buttonUp = Switch(buttonUp_Pin);
const byte buttonDown_Pin = D7;
Switch buttonDown = Switch(buttonDown_Pin);
const byte buttonPress_Pin = D5;
Switch buttonPress = Switch(buttonPress_Pin);
int i;

// WiFi parameters
char* ssid = "MYSSID";
char* password = "MYWIFIKEY";

void setup() {
       Serial.begin(115200);
       Serial.println("Starting:");
       WiFi.begin(ssid, password);
       while (WiFi.status() != WL_CONNECTED) {
          delay(500);
          Serial.print(".");
       }
       Serial.println("");
       Serial.println("WiFi now connected at address");
       // Print the IP address
       Serial.println(WiFi.localIP());
       EspProwl.begin();
       EspProwl.setApiKey("MY_PROWL_API_CODE");
       EspProwl.setApplicationName("ESP6266 Prowl test");
       int returnCode = EspProwl.push("Button press test", "Ready", 0);
}

void loop() {
       buttonUp.poll();
       if(buttonUp.pushed()) {
           Serial.print(++i); Serial.print(" "); Serial.print("Up pushed, ");
           int returnCode = EspProwl.push("Up", "Pushed", 0);
       }
       if(buttonUp.released()) {
           Serial.println("Up released");
           int returnCode = EspProwl.push("Up", "Release", 0);
       }
       buttonDown.poll();
       if(buttonDown.pushed()) {
           Serial.print(++i); Serial.print(" "); Serial.print("Down pushed, ");
           int returnCode = EspProwl.push("Down", "Pushed", 0);
       }
       if(buttonDown.released()) {
            Serial.println("Down released");
            int returnCode = EspProwl.push("Down", "Release", 0);
       }
       buttonPress.poll();
       if(buttonPress.pushed()) {
            Serial.print(++i); Serial.print(" "); Serial.print("Press pushed, ");
            int returnCode = EspProwl.push("Press", "Pushed", 0);
       }
       if(buttonPress.released()) {
            Serial.println("Press released");
            int returnCode = EspProwl.push("Press", "Release", 0);
       }
}


Web Service monitoring with push notifications:

top
An earlier blog showed how to use an EPS device to monitor a web service, http://www.geothread.net/iot-project-using-an-esp32-device-to-check-a-web-service/. We can revisit that code now, and add in Prowl push alert code. We can also make the code a little more sophisticated too to avoid false positives. As before you will need to add in the SSID and WiFi passcode, and the website address and REST endpoint, as well as the Prowl API code. Here is the source code, edited to work on an EPS8266 device, rather than EPS32 device (its the slightly cheaper older chip).

// TTGO EPS8266_WebServiceCheck : WiFi & Bluetooth Battery ESP32 Module - webservices checker

// Import required libraries
#include <Wire.h>
#include <OLEDDisplayFonts.h>
#include <OLEDDisplay.h>
#include <OLEDDisplayUi.h>
#include <SSD1306Wire.h>
#include <SSD1306.h>
#include "images.h"
#include "fonts.h"
#include <ESP8266WiFi.h>
#include <EspProwl.h>

// The built-in OLED is a 128*64 mono pixel display
// i2c address = 0x3c
// SDA = 5
// SCL = 4
SSD1306 display(0x3c, 5, 4);

// Web service to check
const int httpPort = 80;
const char* host = "MYWEBSERVICE_HOSTNAME";

// WiFi parameters
const char* ssid = "MYSSID";
const char* password = "MYWIFIKEY";

void setup() {
       // Initialize the display
       display.init();
       //display.flipScreenVertically();
       display.setFont(Roboto_Medium_14);

       // Start Serial
       Serial.begin(115200);
       // Connect to WiFi
       display.drawString(0, 0, "Going online");
       display.drawXbm(34, 14, WiFi_Logo_width, WiFi_Logo_height, WiFi_Logo_bits);
       display.display();
       WiFi.begin(ssid, password);
       while (WiFi.status() != WL_CONNECTED) {
          delay(500);
          Serial.print(".");
       }
       Serial.println("");
       Serial.println("WiFi now connected at address");
       // Print the IP address
       Serial.println(WiFi.localIP());
       display.clear();
       EspProwl.begin();
       EspProwl.setApiKey("MY_PROWL_API_CODE");
       EspProwl.setApplicationName("Web Service Checker");
       int returnCode = EspProwl.push("Checker", "System up", 0);
}

void loop() {
       Serial.print("\r\nConnecting to ");
       Serial.println(host);
       display.clear();
       display.setTextAlignment(TEXT_ALIGN_LEFT);
       display.drawString(0, 0, "Check web service");
       display.display();
       Serial.println("Check web service");
       
       // Setup URI for GET request
       String url = "/PATH/TO/REST/ENDPOINT/";
       // if service is up ok, return will contain: 'Service running'

       WiFiClient client;
       if (!client.connect(host, httpPort)) {
         Serial.println("Connection failed");
         display.clear();
         display.drawString(0, 0, "Connection failed");
         display.display();
         return;
       }

       client.print("GET " + url + " HTTP/1.1\r\n");
       client.print("Host: " + (String)host + "\r\n");
       client.print("Authorization: Basic YWRtaW46NSs2KndralhLcVApOVd2JWokQ2o=\r\n");
       client.print("User-Agent: Arduino/1.0\r\n");
       client.print("Cache-Control: no-cache\r\n\r\n");

       // Read all the lines of the reply from server
       delay(800);
       bool running = false;
       while (client.available()) {
         String line = client.readStringUntil('\r\n');
         Serial.println(line);
         if (line == "Service running") {
           running = true;
         }
       }
       if (running == true) {
           display.drawString(0, 25, "Service up OK");
           display.display();
           delay(3000);
           ProwlAlert(running);       
       } else {
           display.drawString(0, 25, "Service DOWN");
           display.display();
           delay(3000);
           ProwlAlert(running);       
       }

       Serial.println();
       Serial.println("Closing connection");
       Serial.println("=================================================");
       Serial.println("Sleeping");
       display.clear();
       display.drawString(0, 0, "Closing connection");
       display.display();
       delay(1000);
       display.clear();
       client.stop();
       // progress bar
       for (int i=1; i<=28; i++) {
         float progress = (float) i / 28 * 100;
         delay(500); // = all adds up to delay 14000 (14 sec)
         // draw percentage as String
         display.drawProgressBar(0, 32, 120, 10, (uint8_t) progress);
         display.display();
         display.setTextAlignment(TEXT_ALIGN_CENTER);
         display.drawString(64, 15, "Sleeping " + String((int) progress) + "%");
         display.display();
         display.clear();
         Serial.print((int) progress);Serial.print(",");
       }
       delay (1000);
}

int ProwlAlert(bool running) {
        // prevent too many false positives
        static int counter = 0;
        static bool resumed = true;
        int returnCode = -999;
        Serial.print("Status: ");
        Serial.print(running);
        Serial.print(" | Run count: ");
        Serial.println(counter);
        if (running == true) {
           counter = 0;
           if (resumed == false) {
              returnCode = EspProwl.push("Checker", "Service resumed as normal", 0);
              resumed = true;
           }
        } else {
           counter = counter + 1;
           resumed = false;       
        }
        if (counter == 5) {
           returnCode = EspProwl.push("Checker", "Service detected as being down - attention required", 0);
           counter = 0;
        }
        return returnCode;
}