Little Big Data: Splunk meets NXT

Splunk is an enterprise-class machine data gathering and analysis tool, capable of consuming 100s of gigabytes of machine generated data and turning it into arbitrarily searchable data and presenting it with fancy reports and graphs.  I’ve used Splunk in the past for a project and another time to do post-mortem analysis of a hacked webserver.

Since I was working on a driver for the NXT2WIFI and programming an SNMP agent in ROBOTC, I thought, “Why not have Splunk do something with this?”  So I hooked up a LEGO Light Sensor, Sound Sensor and a HiTechnic Barometer Sensor.  Now I could measure atmosphere pressure, temperature, ambient light level and background sound over WiFi.

I implemented some SNMP OIDs to return appropriate sensor values, depending on the OID queried:

• iso.3, 6, 1, 2, 1, 1, 5, 0: name of the brick
• iso.3, 6, 1, 3.1: temperature in C
• iso.3, 6, 1, 3.2: pressure in 1/1000th inch Hg
• iso.3, 6, 1, 3.3: sound
• iso.3, 6, 1, 3.4: ambient light

In order to fetch all the sensor data, I cobbled together a quick script to run on the Splunk host (a Linux VM):

#!/bin/sh 
IP=192.168.0.102 
DATE=`date` 
TEMP=`snmpget  -r 0 -c public -v 1 $IP iso.3.6.1.3.1 | sed 's/[^:]\+: //'` 
TEMP=`echo "scale=2; $TEMP/100" | bc` 
sleep 1 
PRESS=`snmpget  -r 0 -c public -v 1 $IP iso.3.6.1.3.2 | sed 's/[^:]\+: //'` 
sleep 1 
SOUND=`snmpget  -r 0 -c public -v 1 $IP iso.3.6.1.3.3 | sed 's/[^:]\+: //'` 
sleep 1 
LIGHT=`snmpget  -r 0 -c public -v 1 $IP iso.3.6.1.3.4 | sed 's/[^:]\+: //'` 
echo $DATE,$TEMP,$PRESS,$SOUND,$LIGHT

This produced a nice and easy to digest output:

Mon Jun 11 20:46:19 CEST 2012,23.5,30761,34,60

Next I setup Splunk to run this every minute or so.  After some tweaking and fiddling, I managed to get it to produce some nifty looking reports:

Overkill?  Sure, but it was pretty cool to have my little NXT being queried by enterprise-class tooling!  All the SNMP agent code will be part of the driver suite when I get around to publishing it.