OpenGraphiti: Data Visualization Engine

2015-01-13T22:57:21
ID N0WHERE:26006
Type n0where
Reporter N0where
Modified 2015-01-13T22:57:21

Description

OpenGraphiti is a free and open source 3D data visualization engine for data scientists to visualize semantic networks and to work with them. It offers an easy-to-use API with several associated libraries to create custom-made datasets. It leverages the power of GPUs to process and explore the data and sits on a homemade 3D engine.

OpenGraphiti Requirements

  • Mac OSX or Linux
  • Clang++ or g++
  • Python 2.7.x
  • Emscripten (Optional)

Installation

First, you will need to clone the repository. The OpenDNS data visualization framework references a couple of git submodules. You need to use the –recursive option to make sure they all get downloaded properly.

$ git clone https://github.com/opendns/dataviz.git --recursive

Second, you have to install some required libraries if you don’t already have them. In case the most recent versions of those libraries aren’t compatible with the current version of OpenGraphiti, we have included the ones used for our developement in the repository. They can be found in the graphiti/Lib folder.

$ pip install networkx
$ pip install pygeoip

Now let’s compile the OpenGraphiti engine :

$ cd dataviz/graphiti
$ make

You should now see a new binary named graphiti inside the your current folder.

Alternative : Emscripten

Alternatively, you can compile OpenGraphiti with emscripten to generate a Javascript file interpreting the compiled LLVM bytecode. Using this option you can integrate our data visualization engine in a web page.

First, you will need to install Emscripten . Download the emsdk-portable archive , extract it and run :

./emsdk update
./emsdk install latest
./emsdk activate latest
source emsdk_set_env.sh

_ NOTE : This last line will set up your environment variables, you may consider putting it in your ~/.profile (or equivalent). _

You are now ready to compile OpenGraphiti for the web :

$ make web

You should see a new folder name Web . It contains a couple of files you can use for web integration.

Running

OpenGraphiti comes with a couple of script packages. They are aimed to give our data scientists a couple of examples on how to use the engine and its API. The main demo package implements a variety of scripts that can be used to visualize graph datasets and manipulate them in 3D. The curious users are invited to add custom-made packages with their own specific scripts.

To list the available packages :

./graphiti

To run the demo package :

./graphiti demo [dataset.json]

Datasets

To create your custom datasets, you will need to use the SemanticNet library. We have provided several example scripts and use cases and these can be found in the dataviz/semanticnet/examples folder.

More information at https://github.com/ThibaultReuille/semanticnet

Also , make sure to check OpenDNS DEF CON 22 Presentation: Catching Malware En Masse DNS and IP Style for some interesting application of this framework:

OpenGraphiti: Data Visualization Engine