RPi Data Capture

About the blogger - I'm an analytics manager for an asset management firm, and a data geek.  Professionally I'm working on developing a data set and visualization tools to help a sales and service organization understand our clients' trading behavior in our mutual funds.  Initiatives include data acquisition, data storage, data summarization, reporting, dashboards, visualization and development of data science skills in the team.

Please note that this blog was originally begun in 2013, and has been substantially rewritten after a project delay and start-over with the latest hardware and software versions.

Outside of work I'm starting out on a personal Raspberry Pi project.

The genesis of the project was a discussion at Middlebury Ski Bowl with a friend, Gary Davis, during a ski race.  Anyone who is a parent of a ski racer is familiar with the hours spent talking with friends between 53 seconds periods of anxiety. That's a lot of time to discuss random ideas.  "What could we do with a Raspberry Pi to develop insights into ski racing?"  We talked of timing devices, GPS tracking the course and other options.

For those of you who don't know what a Raspberry Pi is (from here on in I will abbreviate to RPi), it's a tiny computer about the size of two credit cards that was invented by academics at Cambridge University as a platform for students to learn about basic computer hardware.

It is designed to be easily interfaced to additional circuit boards to teach engineering students about basic electronics.  Yes, you can use it to do basic things like turning on and off LEDs.  That was the idea, but this is ambrosia of the geeks.   Throw down the gauntlet and hobbyists from around the world will buy one and think of incredible things to program it to do.  There are RPi media servers, RPi parallel computer arrays, RPi controlling cameras on private drones and weather balloons, RPi controlling robots ... you name it, and in the last year someone has probably thought of programming an RPi to do it!  The RPi runs on Linux operating system.  You can view it's operating system screen by connecting it to a monitor or TV using an HDMI cable.  It has two USB ports for connecting keyboard, mouse, wifi card and other peripherals (if you need more, use a USB hub).  It also has an ethernet port and various connectors for interfacing directly to the motherboard.

While I'm an electronic systems engineer, it's maybe 25 years since I had any hands-on experience of circuit boards (other than plugging them into IDE slots on PCs!).  I've kept up some software development skills in C, VB, and perl; but I've not programmed in Python before, and Linux is new to me.  Those aren't reasons to not do it - they're why the project seems like a good mind-expanding opportunity!

The conversation with Gary was  way in the back of my mind when I recently heard the guys at HUGE describe their passion for capturing data related to their interests and blogging about it.  They brew beer and capture data from sensors and analyze it.  They attach environmental sensors to their bikes and ride around Brooklyn capturing data, and analyze it.

So what about this for an idea ... take a RPi and add to it an MiniMu-9 board.  The board has nine on-board sensors each providing its own stream of data.  3 gyros (X,Y,Z axis), 3 accelerometers (X,Y,Z axis), 3 compass / magnetometers (X,Y,Z axis) .  

My immediate goal is to capture data and then explore what I can calculate from it and explore visualizations.  Many people are using the MiniMu-9 for robotics feedback and program their RPi to use that feedback to understand where it is and control movement.  This is different, there is no need to do onboard calculation like that.  I want to capture the data and see if I can calculate where the RPi has been.  Could I build a 3D visualization of the path it has followed?  Could I add to the visualization overlays showing speed, force etc?  Is it possible to overlay runs down the same race course and use the data to understand how turning early or late on a gate gains vital fractions of a second?  Lots of ideas to play with there.

And there are all sorts of spin-offs that spring to mind.  Heck it would be fun to take this device on a roller coaster, in a race car, over a ski jump, down a skicross course!

I'm thinking this blog will be where I track the project.  Early days will be about the RPi, Python, boards etc.  Then I can get into the practicalities of data gathering (I'm certainly going to have to stabilize the Pi both physically against the skier's body but also temp stabilization).  Once I start accumulating some usable data then I can get into visualization - many of the BI tools are available on a try-before-you-buy basis.

Text Conventions

In this blog I have tried to follow a set of (my own) conventions for formatting code examples.

Normal text is in Times New Roman.

Linux command line is in tan Courier on a brown background (credit to the OSX Terminal app's "Red Sands" theme):
sudo apt-get install python-smbus

Configuration details entered into the UI of applications, entered into Linux configuration files, or data examples are in bold black Courier on a white background:
blacklist i2c-bcm2708

Python snippets are in purple Courier on a green background:
bus = smbus.SMBus(n) 

Where I have copied final python code directly from PyCharm IDE I maintained the colors and font of the PyCharm theme that I use:

def write_data(filename, data):
    # assumes directory already set in main (avoids repeated sets)

    f = open(filename, "w")
    json.dump(data, f)
    f.close()