When and where is the next train going?! The IoL City citizens have things to do and places to be!

Goal

  • Pull in Transport For London API data to get schedule information for a local train stop
  • Display next trains and time to arrival on an OLED screen

Technologies & Components

Overview

The OLED display project will build on the skills and technology learned in the Weather Station project. The basic concept is to utilize a Cactus Micro micro controller (Arduino clone with ESP8266 WiFi built-in) with MQTT messaging to exchange information. But instead of the device sending data, it will be receiving data to display on an OLED screen. In addition, the schedule data will be published to a PubNub channel for use with other IoL City projects.

Transport for London API

As a fun exercise, I wanted to pull in some publicly available data via an API, manipulate it and use it with an application. The TFL API was a great fit for this project. It works by simply going to a website, which responds with information in a JSON format. No need for API keys, just a simple GET request and you have data!

So how does it really work? I’m glad you asked. Basically, there are a number of URLs that you can visit that return related information. Anything from trains, buses to boats can be retrieved with their “Unified RESTful API”.

All the information can be found here: https://api-portal.tfl.gov.uk/docs

To keep things simple,I’ve selected a local train station near me which has two outbound routes. I will then use that information to determine which direction the train track should be switched to and also display the time it will arrive.

To find the station ID, I first used the following GET URL

https://api.tfl.gov.uk/StopPoint/Search/fields

Which then returns the following result snippet

{"$type":"Tfl.Api.Presentation.Entities.SearchResponse, Tfl.Api.Presentation.Entities","query":"fields","total":50,"matches":[{"$type":"Tfl.Api.Presentation.Entities.MatchedStop, Tfl.Api.Presentation.Entities","stationId":"940GZZLUNFD","icsId":"1000159","topMostParentId":"940GZZLUNFD","modes":["bus","tube"],"status":true,"id":"940GZZLUNFD","name":"Northfields Underground Station","lat":51.499324,"lon":-0.31472},{"$type":"Tfl.Api.Presentation.Entities.MatchedStop, Tfl.Api.Presentation.Entities","stationId":"940GZZLUSFS","icsId":"1000209","topMostParentId":"940GZZLUSFS","modes":["bus","tube"],"status":true,"id":"940GZZLUSFS","name":"Southfields Underground Station","lat":51.445076,"lon":-0.2066},{"$type":"Tfl.Api.Presentation.Entities.MatchedStop, Tfl.Api.Presentation.Entities","stationId":"910GLONFLDS","icsId":"1001183","topMostParentId":"910GLONFLDS","modes":["overground","national-rail"],"status":true,"id":"910GLONFLDS","name":"London Fields Rail Station","lat":51.541158,"lon":-0.057747},
…...

I then searched for “London Fields” and found that the ID is 910GLONFLDS.

Finally, I used the following URL to retrieve the local train schedule.
URL: https://api.tfl.gov.uk/StopPoint/910GLONFLDS/arrivals

Which then returns the following result snippet

[{"$type":"Tfl.Api.Presentation.Entities.Prediction, Tfl.Api.Presentation.Entities","id":"-407419356","operationType":1,"vehicleId":"6uXFyoXFIOb2cMfIGrmiLQ==","naptanId":"910GLONFLDS","stationName":"London Fields Rail Station","lineId":"london-overground","lineName":"London Overground","platformName":"2","direction":"outbound","destinationNaptanId":"910GENFLDTN","destinationName":"Enfield Town Rail Station","timestamp":"2016-01-15T18:38:32.787Z","timeToStation":1680,"currentLocation":"","towards":"","expectedArrival":"2016-01-15T19:06:32.787Z","timeToLive":"2016-01-15T19:06:32.787Z","modeName":"overground"},{"$type":"Tfl.Api.Presentation.Entities.Prediction, Tfl.Api.Presentation.Entities","id":"-422333055","operationType":1,"vehicleId":"Br9J3rKECr9QYjK0MhQCaw==","naptanId":"910GLONFLDS","stationName":"London Fields Rail Station","lineId":"london-overground","lineName":"London Overground","platformName":"1","direction":"inbound","destinationNaptanId":"910GLIVST","destinationName":"London Liverpool Street Rail Station","timestamp":"2016-01-15T18:38:32.802Z","timeToStation":2220,"currentLocation":"","towards":"","expectedArrival":"2016-01-15T19:15:32.802Z","timeToLive":"2016-01-15T19:15:32.802Z","modeName":"overground"},{"$type":"Tfl.Api.Presentation.Entities.Prediction, Tfl.Api.Presentation.Entities","id":"1667012363","operationType":1,"vehicleId":"Dk12CzBKtYQmV+mYwrJh6w==","naptanId":"910GLONFLDS","stationName":"London Fields Rail Station","lineId":"london-overground","lineName":"London
….

Cool! But that is way too much info for my purposes. I also want to automate this task and send the results to my Internet of Things LEGO

Programming

I will be using Node-RED to perform the plumbing for my two messaging systems, MQTT & PubNub. Although I wrote much of this application originally in pure NodeJS, I found that Node-RED was still more fun to experiment with. For example, I could quickly pipe this data to Twitter/Slack/Twilio to send me a message if my train is close. By just passing around objects and manipulating them with JS in function nodes, I can pretty much do what I want for IoT related tasks and rapid prototyping.

Node-RED Flow Schedule and Display

Screen Shot 2016-01-15 at 11.40.58 PM

Download flow: Gist

To get the TFL data into the system, I first started by creating a function to define the URL that will be passed to the HTTP Get request node. This was accomplished by using a function node to simply write the msg.url property to the desired station ID. I use an injection node with a repeat schedule to automate the data stream. Because it pulls every second, I placed a delay of 1 minute to not abuse the network. I also added a RESTful interface myself for future use and flexibility.

The API data is returned as a JSON string, which I then use a JSON node to convert it to an object. This is necessary so I can iterate through it and extract the important data.

With the data in hand, its time to extract it and create a new schema. Here’s an example of what a populated data set will look like.

{
  "station": "London Fields Rail Station",
  "schedule": [
    {
      "expected": "3",
      "destination": "Enfield Town Rail Station"
    },
    {
      "expected": "15",
      "destination": "Cheshunt Rail Station"
    },
    {
      "expected": "25",
      "destination": "Enfield Town Rail Station"
    },
    {
      "expected": "35",
      "destination": "Cheshunt Rail Station"
    },
    {
      "expected": "50",
      "destination": "Enfield Town Rail Station"
    }
  ]
}

But first, the data is originally sent in some arbitrary order, so it needs to be sorted. At the same time, the desired properties are saved to the new schema. This function node takes care of all that:

Screen Shot 2016-01-15 at 11.41.56 PM

var tfl = msg.payload;
var trains = {};
var station;
var expected;
var nextTrains = [];


// check for valid data first
if (msg.statusCode == 200) {
    //if(tfl[0] !== undefined){
    if(tfl[0]){
        station = tfl[0].stationName;

        // sort schedule
        tfl.sort(function(a,b){
      	    return (a.timeToStation - b.timeToStation);
    	});
        
        // iterate through all schedule records and save required data
        for (var i = 0; i < tfl.length; i++) {
          if(tfl[i].direction == 'outbound'){
          	nextTrains.push({
          	    'expected': (tfl[i].timeToStation)/60,
          	    'destination': tfl[i].destinationName
          	});
          } 
        }
    }else{
        // no schedule received
        station = msg.stationid; // use station id as default
        nextTrains.push({
            'expected': 0,
          	'destination': "Out of Service"
        })
    }
    
    trains.station = station;
    trains.schedule = nextTrains;
}

// display and send schedule data
console.log("outbound schedule new schema > ");
console.log(util.inspect(trains, false, null));
msg.payload = trains;
flow.set('trainschedule', msg.payload); // save for flow use
return msg;

 

HTTP response node

By adding an HTTP input and response node, I can toggle the train schedule routine and get a current formatted schedule. Although I am not relying on this for this project, I thought it was a handy option for the future. Maybe a UI dashboard can request that data as needed.

GET Request: http://myNodeRedServer:1880/trainschedule

REST interface for new train scheduleHTTP response node

Example of GET response

{"station":"London Fields Rail Station","schedule":[{"expected”:"0","destination":"Cheshunt Rail Station"},{"expected":"18","destination":"Enfield Town Rail Station"},{"expected":"33","destination":"Cheshunt Rail Station"},{"expected":"48","destination":"Enfield Town Rail Station"},{"expected":"63","destination":"Cheshunt Rail Station"}]}

Now that I have the data… the magic begins!

The msg object, which now contains the schedule information, will be passed to multiple nodes for future applications.

MQTT

Send the schedule to the OLED display board.

Since this data is in JSON format, it must first be converted to a simple text string. I also pre-formatted it to avoid too much code on the embedded side. This approach also allows the OLED display board to be easily used for other applications without much adjustment. The schedule was too much text, so the destination names were simplified. Instead of “Enfield Town Rail Station”, the string was split and then the first two words and a space were joined back. I also padded the expected time with a leading 0 if less than 10 so that the station names would line up. Finally, the data was sent to the MQTT topic /trainschedule/910GLONFLDS

Node-RED schedule to MQTT display

Function: Format to MQTT display string

var station = msg.payload.station;
var schedule = msg.payload.schedule;
var destination;
var split;
msg.topic = "/trainschedule/" + msg.stationid;

if (schedule[0].destination == "Out of Service"){
    destination = schedule[0].destination;
    //destination = "Out of Service";
    msg.payload = destination;
    return msg;
}else{
    // Split the string and only use the first two words of each value
    split = station.split(" ");
    station = split[0] +" "+ split[1] + "\n\n"; 
    msg.payload = station;
    for(var x = 0; x < schedule.length; x++){
        split = schedule[x].destination.split(" ");
        destination = split[0] +" "+ split[1];
        msg.payload += pad(schedule[x].expected) + "m" + "  " + destination + "\n";
    }
    return msg;
}


function pad(n) {
    return (n < 10) ? ("0" + n) : n;
}

 

Schedule Display

Cactus Micro Rev2

cactus micro rev2 pinout

An Arduino LillyPadUSB clone with built-in ESP8266 module.

Resources:

http://wiki.aprbrother.com/wiki/Cactus_Micro_Rev2

OLED 128×64

OLED screen

128x64 I2C

128×64 I2C

Specifications: 0.96” I2C IIC 128X64 OLED LCD LED

Resources: https://www.adafruit.com/products/326

Text Only Library: https://github.com/greiman/SSD1306Ascii

Circuit

The circuit is very simple. Just connect the OLED board to the VCC and ground pins. Then connect the SDA (GPIO 2) and SDC (GPIO 3) pins.

Note: This Fritzing diagram uses an Arduino Nano, so pin placement will be slightly different.

Train Schedule - OLED Fritzing

Code

Before the project code can be uploaded to the Cactus Micro, we must first flash the onboard ESP8266 with the appropriate firmware. To accomplish this, a utility sketch will need to be first uploaded to the Cactus, which will create a transparent bridge to the ESP8266.

Step 1: Configure the Arduino as an ESP programmer

Connect the Cactus Micro to your computer, and set the Arduino IDE to use the LillyPad USB for the board. Then upload the following sketch.

sketch esp8266Programmer

More info:

http://wiki.aprbrother.com/wiki/How_to_made_Cactus_Micro_R2_as_ESP8266_programmer

Step 2: Flash the ESP8266 with this firmware

The ESP8266 is now directly accessible from a serial port and can be programmed with the esp tool. Download the firmware and tool, then run the programming command. Be sure to adjust the serial port (e.g.ty.usbmodem1421) to match your configuration.

Firmware: espduino

Tool: esptool

./esptool.py -p tty.usbmodem1421 write_flash 0x00000 esp8266/release/0x00000.bin 0x40000 esp8266/release/0x40000.bin

 

Step 3: Program project code to Cactus Micro

The general code flow begins with initializing a timer, OLED screen, ESP8266 WiFi and MQTT messaging.

Since this microcontroller only has 32k of memory, space is a real challenge. I originally tried to use the nice Adafruit graphics library, but it wouldn’t fit on the device with the additional communication libraries. So instead, a text only custom library will be used. I’ve also used the MQTT library provided by April Brothers, who manufacture the Cactus Micro.

OLED Graphics Libary

MQTT Library

Once everything is initialized and the device has connected to the MQTT broker over WiFi, the system waits for an incoming train schedule message. Every few seconds, the screen is updated with the new schedule data and periodically splash a brand page “IoL City Train Network”.

Train Schedule - IoL City Train Network

/*** Internet of Lego - Train Schedule Display
* This program connects to WiFi, utilizes MQTT and displays the local train schedule on an OLED screen
* The OLED graphics library is slimmed down to only text for memory efficiency
* https://github.com/greiman/SSD1306Ascii
*
* Hardware: Cactus Micro Rev2 - http://wiki.aprbrother.com/wiki/Cactus_Micro_Rev2
* Written by: Cory Guynn with some snippets from public examples
* 2016
*/
 
// Global Variables
unsigned long time; // used to limit publish frequency
 
// OLED
#include "SSD1306Ascii.h"
#include "SSD1306AsciiAvrI2c.h"
// 0X3C+SA0 - 0x3C or 0x3D
#define I2C_ADDRESS 0x3C
SSD1306AsciiAvrI2c oled;
 
// ESP8266 WiFi
#include 
#define PIN_ENABLE_ESP 13
#define SSID  "yourSSID"
#define PASS  "yourpassword"
 
// MQTT Messaging
#include 
ESP esp(&Serial1, &Serial, PIN_ENABLE_ESP);
MQTT mqtt(&esp);
boolean wifiConnected = false;
#define mqttBroker "aws.internetoflego.com"
#define mqttSub "/trainschedule/910GLONFLDS"
String mqttMessage = "";
 
/*******************
// Functions
*******************/
 
void wifiCb(void* response)
{
  uint32_t status;
  RESPONSE res(response);
 
  if(res.getArgc() == 1) {
    res.popArgs((uint8_t*)&status, 4);
    if(status == STATION_GOT_IP) {
      wifiConnected = true;
      Serial.println("WIFI CONNECTED");
      oled.print("WiFi Online: ");
      oled.println(SSID);
      mqtt.connect(mqttBroker, 1883, false);
      //or mqtt.connect("host", 1883); /*without security ssl*/
    } else {
      wifiConnected = false;
      mqtt.disconnect();
      oled.println("WiFi OFFLINE");
    }
 
  }
}
 
void mqttConnected(void* response)
{
  Serial.println("MQTT Connected");
  oled.println("MQTT Connected");
  mqtt.subscribe(mqttSub); //or mqtt.subscribe("topic"); /*with qos = 0*/
  mqtt.publish("/news", "OLED display is online");
}
 
void mqttDisconnected(void* response)
{
  oled.clear();
  Serial.println("MQTT Disconnected");
}
 
void mqttData(void* response)
{
  RESPONSE res(response);
  Serial.println("mqttTopic:");
  String topic = res.popString();
  Serial.println(topic);
  oled.println(topic);
  Serial.println("mqttMessage:");
  mqttMessage = res.popString();
  Serial.println(mqttMessage);
}
 
void mqttPublished(void* response)
{
 
}
 
void setup() {
  // OLED initilize
  oled.begin(&Adafruit128x64, I2C_ADDRESS);
  oled.setFont(Adafruit5x7);
  oled.clear();
  oled.println("OLED online");
 
  // Hardware Serial (for ESP8266) and Serial Concole initialization
  Serial1.begin(19200);
  Serial.begin(19200);
  esp.enable();
  delay(500);
  esp.reset();
  delay(500);
  while(!esp.ready());
 
  Serial.println("Setup MQTT client");
  if(!mqtt.begin("TrainScheduleDisplay-LF", "admin", "Isb_C4OGD4c3", 120, 1)) {
    Serial.println("Failed to setup mqtt");
    while(1);
  }
 
  Serial.println("Setup MQTT lwt");
  mqtt.lwt("/lwt", "offline", 0, 0); //or mqtt.lwt("/lwt", "offline");
 
  /*setup mqtt events */
  mqtt.connectedCb.attach(&mqttConnected);
  mqtt.disconnectedCb.attach(&mqttDisconnected);
  mqtt.publishedCb.attach(&mqttPublished);
  mqtt.dataCb.attach(&mqttData);
 
  /*setup wifi*/
  Serial.println("Setup WiFi");
  esp.wifiCb.attach(&wifiCb);
  esp.wifiConnect(SSID, PASS);
 
  Serial.println("System Online");
}
 
void loop() {
 
  esp.process();
 
  if (millis() > (time + 10000)) {
    time = millis();
    oled.clear();
    oled.set2X();
    oled.println(" IoL City ");
    oled.println("");
    oled.set1X();
    oled.println("   Train Network");
    delay(2000);
    oled.clear();
    oled.println(mqttMessage);
    if(wifiConnected) {
      // publish stuff here
    }
   }
}

Download Source Code: Gist

Gallery

OLED display, powered by Arduino and Node-RED using the Transport for London API

Posted by Internet of Lego