Tag Archives: SQLite

Matters relating to the SQLite database tool

Using SQLite on a Raspberry Pi

There is a lot of interest in the amazing Raspberry Pi 3 computer here at Cranfield University. Sometimes, an application we build – for example on the Raspberry Pi – needs to store data – for example we might wish to store data from sensors. For this we need a database. However, again sometimes the overhead of having a full-blown database server up and running is too much. We need a simple, localised way to store and retrieve data. Fortunately there is a great solution to this, ‘SQLite’ (https://www.sqlite.org).

This brief tutorial shows how to setup and configure SQLite on a Raspberry Pi, and gives and example if it in use.

To start with, as with every time we use the Pi, we should update and upgrade the distribution – to ensure all the software within the Debian distribution are up-to-date. As described on the Raspberry Pi pages (https://www.raspberrypi.org/documentation/raspbian/updating.md), this is pretty straight forward.

First, update your system’s package list by entering the following command:

sudo apt-get update

Next, upgrade all your installed packages to their latest versions with the command:

sudo apt-get dist-upgrade

Once this is done, we are ready to install SQLite:

sudo apt-get install sqlite3

We will use the database to store data from within programming code – for example a software application that stores off sensor data. However, in the first instance, we can also run SQLite interactively to ‘create’, ‘read’, ‘update’ and ‘delete’ data directly in SQL tables. SQLite comes with a command line interpreter (CLI), that provides a command prompt to enter in commands. If we create a new database, we can try this out:

sqlite3 sensor_db

This starts up SQLite and will, at the same time, also create a new database – a single file, in this case called ‘sensor_db’ in your current folder.

Now we can create a database structure in our new database:

CREATE TABLE IF NOT EXISTS readings (
'temperature' REAL NOT NULL,
'humidity' REAL NOT NULL,
'datetime_int' INTEGER NOT NULL,
'sensor_id' INTEGER NOT NULL);

Note the semi-colon ending the statement. Now we can check that was created OK:

PRAGMA table_info([readings]);

Let’s place a couple of dummy data items (rows) into the table:

INSERT INTO readings (temperature, humidity, datetime_int, sensor_id) values (18.5,45.3,strftime('%s','now'),1);
INSERT INTO readings (temperature, humidity, datetime_int, sensor_id) values (19.4,42.8,strftime('%s','now'),1);

Now select the data to make sure it was stored correctly. Note we are using one of the two means of storing dates, here using integers not strings (see http://www.sqlitetutorial.net/sqlite-date/).

SELECT temperature, humidity, datetime(datetime_int, 'unixepoch'), sensor_id FROM readings;

The data should be shown…

To quit the interactive mode:

.exit

SQLite is a powerful database solution for small applications. Programming it is just like its larger server-based equivalents. It is worth spending time reading the SQLite tutorials for further information (http://www.sqlitetutorial.net). Another good tutorial is https://www.tutorialspoint.com/sqlite/index.htm.

Merry Christmas 2015

As another year draws to a close at Cranfield University, sure enough we have another Christmas map for you. As with previous years, we’ve collected a sample of tweets from Twitter that match a number of Christmas related keywords and mapped them using the same process we outlined last year.

The colour range from green to red indicates the density of Christmas related tweets from low to high in that county, relative to the normal density of twitter activity in that area (taken from a random sample of all tweets in the UK).

We’ll let you draw your own conclusions from the map. This time we thought we’d use the opportunity to focus on some of the web mapping technologies we’ve started using this year in other projects and hope to develop our use of heading into 2016. The biggest difference between the map you see above and some of the other maps we’ve published in the past is that this one doesn’t makes use of any GIS server or map hosting platform. Traditionally we’ve used either Geoserver or ArcGIS Server to publish our map tiles and other geospatial data for consumption with JavaScript web mapping APIs. Alternatively, web map hosting services can be used if one doesn’t have access to their own GIS Server, these include ArcGIS Online, Mapbox, CartoDB and others. The example here doesn’t use any of these services, but instead is running from a set of map tiles hosted on this very webserver. Interactivity (roll your mouse over the map to view the county names) is provided via a set of UTF-Grid tiles, also hosted on this webserver.

UTFGrid tiles use a combination of JSON encoding and ASCII grid files that sit alongside the map’s image tiles. For each PNG image tile there is a corresponding ASCII tile with a one pixel to one ASCII character mapping. An accompanying JSON lookup table provides the full set of attributes so that you can go beyond a simple raster map and offer full identify style interactivity.

UTFGrid functionality is available in many of the popular JavaScript web mapping APIs, either out of the box or as easily downloadable plugins. This particular map makes use of the Mapbox JS API, an extension of the Leaflet API.

The tiles used by this map are generated using the TileMill software and stored as a .mbtiles file (which is actually an SQLite database). A small PHP file, acting as a tile server, exposes this SQLite/MBTiles database to web mapping APIs as a large nested folder of image and UTFGrid files in the usual {z}/{x}/{y}.png or {z}/{x}/{y}.json fashion.

We like this approach to web mapping as it is reasonably lightweight and portable. The whole application, including web pages, JavaScript, map tiles and PHP tile server can be picked up and dropped onto any web server that supports PHP and is ready to go. It might not provide some of the advanced features you get with more heavyweight solutions, but a simple interactive map with query-able attributes is often all that’s needed for many web mapping applications. It’s also extremely fast and can be built using entirely open source software and tools.

More information on some of the packages and technologies used can be found here:

Merry Christmas and a happy New Year from all at Geothread!