From OpenBeacon

BruCON 2011, day 2 - OpenBeacon RFID Tag Proximity Data was used to build social network graph for second day over all tag IDs in the lounge area (in PDF and PNG format). You can also find the live version of this graph (including source code).

Active 2.4GHz OpenBeacon Proximity Tag was used to measure social interaction between BruCON 2011 participants and to track positions

OpenBeacon Ethernet EasyReader PoE II with Aluminum Case

Live OpenBeacon Tracker API

You can see the API in action at http://api.openbeacon.net/get/brucon.json - just hit reload to get new data. The data used there loops day 2 of BruCON 2011. The JSON format used there captures the entire state of the installation in a single object - no recursive API calls are needed.

Live Demo Applications

• Phosphor Tag Tracking (source) by Jeff Katz
• Realtime Social Network (source) by Milosch Meriac
• Tags per Reader based on D3.js (source) by Milosch Meriac
• Tags positions versus Tag Proximity sightings (source) by Milosch Meriac
• Visitor Avatars Bowtie Animation by Aestetix
• Visitor Tracking based on D3.js (source) by Rob Zinkov

Installing the Tracker API under Fedora 17 x64 bit

Please make sure that your user is in the sudoers file to enable him to run sudo. You can enable sudo in System Settings/User Accounts by pressing the Unlock-button and setting Account type from Standard to Administrator.

Get the latest sources

See git documentation for more information on this topic.

# make sure that 'git' is installed
sudo yum install git
 
# go to home directory
cd ~
 
# Grab the latest sources
git clone http://github.com/meriac/openbeacon.git
 
# install missing development packets (press 'y' to confirm the packet selection shown)
make -C openbeacon/host/services/openbeacon-tracker dependencies-fedora

Compile the openbeacon-tracker API service

# go to tracker source directory
cd ~/openbeacon/host/services/openbeacon-tracker
 
# compile the openbeacon-tracker service
make clean all

Run the openbeacon-tracker API service

You need to make sure that all OpenBeacon Ethernet EasyReaders point to the IP of your linux machine.

Additionally you need to open your local machines firewall UDP port 2342 to receive tracking packets:

• go to Activities/Applications/Other/Firewall
• in Firewall Configuration, Other Ports/Add - enable User Defined"
• enter 2342 in Port / Port Range
• select Protocol/udp
• press OK to confirm your changes
• press Apply to apply your changes
• Close Firewall Configuration

# go to tracker source directory
cd ~/openbeacon/host/services/openbeacon-tracker
 
# run the tracker software
./openbeacon-tracker

If everything is correct you will see now a continuous stream of JSON objects on stdout':

{
  "id":375,
  "api":{"name":"openbeacon-tracker","ver":"1.0.3-5-gfbe1"},
  "time":1349514764,
  "packets":{
    "per_key":[0,0,0,0,0,0,0,0,0,1386],
    "rate":26,
    "unknown_reader":1386,
    "crc_ok":1386
    },
  "tag":[
    {"id":192,"key":9,"reader":1095},
    {"id":80,"key":9,"reader":1095}
    ],
  "reader":[
 
    ],
  "edge":[
    {"tag":[80,192],"power":5}
    ]
},{
  "id":376,
  "api":{"name":"openbeacon-tracker","ver":"1.0.3-5-gfbe1"},
  "time":1349514764,
  "packets":{
    "per_key":[0,0,0,0,0,0,0,0,0,1394],
    "rate":26,
    "unknown_reader":1394,
    "crc_ok":1394
    },
  "tag":[
    {"id":192,"key":9,"reader":1095},
    {"id":80,"key":9,"reader":1095}
    ],
  "reader":[
 
    ],
  "edge":[
    {"tag":[80,192],"power":5}
    ]
},

In this example you can see two tags with the ID's 192 and 80 detected by reader 1095 - and as both tags are next to each other the proximity function kicks in and detects a sighting (called edge) between both tags with the power level 5.

Connecting OpenBeacon Tracker API to an Apache web server

Please watch the video tutorial on setting up the OpenBeacon PoE Ethernet readers, configuring apache for serving the JSON API and installing the example web visualizations.

The examples below use shortened object results for performance reasons. A the BruCON 2011 page you can find a real world example of a JSON object to clarify the data format. You can nicely see the list of tag positions with the button press indicator, the list of readers and positions plus the current social interaction of visitors in the edges'-area. You can see the live API at http://api.openbeacon.net/get/brucon.json - please hit reload to get new data. The BSON object format in the MongoDB database is the same as on the JSON web API.

If you add the option -l to filter-mongodb, these object will be stored into the MongoDB databes in the collection named log.

Connecting OpenBeacon Tracker API to a MongoDB database

Let's try some offline operations first to get the hang of MongoDB. This example downloads example log files from the BruCON 2011 conference and populate the MongoDB database openbeacon' with the processed data.

# go to tracker source directory
cd ~/openbeacon/host/services/openbeacon-tracker
 
# make sure that mongodb-server is installed
sudo yum install mongodb-server
# start mongod server
sudo systemctl start mongod.service
# start mongod server on every boot
sudo chkconfig --levels 235 mongod on
 
# make sure to start from scratch 
make clean all
 
# download the latest mongo-c driver, example logfiles from BruCON 2011 (~534 MB compressed data)
make demo-mongodb

You will see a similar output as a result:

new tag      115 [key=1] seen
new tag      298 [key=1] seen
new tag      369 [key=1] seen
new tag      186 [key=1] seen
new tag      402 [key=1] seen
new tag      183 [key=1] seen
new tag      231 [key=1] seen
new tag      334 [key=1] seen

The openbeacon-tracker software chews through the entire BruCON 2011 logfile as quick as possible and stores all JSON objects into the mongodb tables openbeacon.tag and openbeacon.edge:

./openbeacon-tracker logs/brucon-2011-log.bin 0 | ./filter-mongodb > /dev/null

Multiple filters can be concatenated as stdin/stdout/stderr pipes are used for communication between openbeacon-tracker and possible filters like ./filter-mongodb or ./filter-singular-sighting

running a MongoDB live update from openbeacon-tracker

This example runs ./openbeacon-tracker and pipes the data to ./filter-mongodb where it's stored into the MongoDB database collection openbeaco.live.

# go to tracker source directory
cd ~/openbeacon/host/services/openbeacon-tracker
 
# make sure to start from scratch - recompile
make clean all
make filter-mongodb
 
# actually run the tracker and pipe the data to ./filter-mongodb
./openbeacon-tracker | ./filter-mongodb

As you can see in this example, the output of ./filter-mongodb is not redirected to >/dev/null in this example. This allows you to see the current status of the tracking system live in your console window. Please maximize the console window for the best visual experience.

Here you can see how a custom MongoDB Server IP, the database port and a database name are specified:

# Usage: ./filter-mongodb [-l] [-h db_host] [-p db_port] [-d db_name]
openbeacon-tracker | ./filter-mongodb -h 10.254.0.123 -p 1234 -d mydatabase

Default Configuration Settings:

 // can be overridden with '-h'
 mongo_host = "127.0.0.1";
 
 // can be overridden with '-p'
 mongo_port = 27017;
 
 // can be overridden with '-b'
 mongo_db = "openbeacon";

Querying the MongoDB database

Run the interactive MongoDB shell first and connect to the openbeacon collection:

mongo openbeacon

As a response the interactive shell pops up:

MongoDB shell version: 2.0.7
connecting to: openbeacon
> 

Creating database indexes for increased performance

First make sure to create meaningful indexes to speed up querying of your database.

// add geospatial index to tag position collection 
db.tag.ensureIndex ( { "loc" : "2d" , "time":1 , "button":1 } , { min:0,max:1024 } );
 
// add index for edge detection collection
db.edge.ensureIndex ( { "tag":1, "time":1, "power":1 } );
 
// ensure double documents from multiple replays of the same data
// index is only need when '-l' was enabled on the filter-mongodb command line
db.log.ensureIndex ({"id":1, "time": 1}, {unique: true});

Show time and position of a tag

Show time and position of a tag seen at the venue:

db.tag.find ( { "id" : 275 },{"_id":0,"time":1,"loc":1} );{ "time" : 1316510836 }
{ "time" : 1316510837 }
{ "time" : 1316510838 }
{ "time" : 1316510839 }
{ "time" : 1316510840 }
{ "time" : 1316510841 }
{ "time" : 1316510842 }
{ "time" : 1316510843 }
{ "time" : 1316510844 }
{ "time" : 1316510845 }
{ "time" : 1316510846 }
{ "time" : 1316510847 }
{ "time" : 1316510848 }
{ "time" : 1316510849 }
{ "loc" : [ 826, 514 ], "time" : 1316510850 }
{ "loc" : [ 825, 514 ], "time" : 1316510851 }
{ "loc" : [ 824, 515 ], "time" : 1316510852 }
{ "loc" : [ 822, 515 ], "time" : 1316510853 }
{ "loc" : [ 821, 515 ], "time" : 1316510854 }
{ "loc" : [ 820, 516 ], "time" : 1316510855 }
has more

Using the command it (for 'iterate') you can iterate through the whole dataset.

Show times when tags were within a range of two pixels near to the coordinate X=808,Y=516

db.tag.find ( { "loc" : { $near : [808, 516], $maxDistance : 2 }}, {"_id":0,"id":1,"time":1} );{ "id" : 177, "time" : 1316511449 }
{ "id" : 275, "time" : 1316511542 }
{ "id" : 389, "time" : 1316512473 }
has more

Show times when tag number 382 was 5 pixels near to the coordinate X=808,Y=516

db.tag.find ( { "loc" : { $near : [808, 516], $maxDistance : 5 }, "id": 382 }, {"_id":0,"time":1} );{ "time" : 1316531371 }
{ "time" : 1316531372 }
{ "time" : 1316531373 }
has more

Show all datasets with proximity sightings involving tag number 275

Returns tag ID tuplets of proximity sightings that include tag 275.

db.edge.find ( { "tag" : 275 },{"_id":0});{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512365 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512366 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512367 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512368 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512369 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512370 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512371 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512372 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512373 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512374 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512375 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512376 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512377 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512378 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512379 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512380 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512381 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512382 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512383 }
{ "tag" : [ 275, 375 ], "power" : 3, "time" : 1316512397 }
has more
>