Weave Data Tutorial
Tutorial for Weave Magazine 03.10 about visualizing network data

Client:
Weave Magazine / Page publisher

We all use twitter, write emails, skype and blog all day long. There isn’t a single day we are not going to visit websites like google, youtube, the website of our favorite newspaper or social network and browse through the web. Despite that, or actually because of it, everybody tends to use the web in a different manner. Even if you are not using your computer at all, it is heavily communicating with the internet by sending and receiving so called IP-packets from all different kind of software that is installed on your PC. Beside the send information, the payload, these packets also contain information like the timestamp and the IP addresses/ports of the receiver and sender.

To visualize this fact and the traces we leave on our journey through the internet and to reveal some of the hidden patterns of our everyday online life, weave magazine asked us to create a data visualization of the online activity of several interaction designers, artists and developers.To explain the process behind the visualization and to give every reader the chance to create his own log-file and graph, we also wrote a 8-page tutorial describing how to collect, analyze and visualize this kind of data.
The source files of the logTool and the visualization sketch that are part of the tutorial are both available for download below.

By analyzing your personal internet traffic logfile using the custom written software you get a distinctive and unique visualization of your online activities.

Collecting the Data
To collect the needed data we wrote a little packet sniffing tool using processing and the open source carnivorePE library. This application listens to all internet traffic, like web, email, instant messaging on a specific local area network and records the header information of the incoming and outgoing packets of your personal computer into a log file. After 24 hours of logging you end up with a large log file containing up to 1.5 mio lines containing the following information : Timestamp:SenderIP:Port >ReceiverIP:Port

Analyzing & Visualizing
By analyzing the different IP addresses and ports we were able to determine what kind of application or service send or received the packets. Using this information and the available timestamp of each packet we sorted them into different timeslots from where we drew 5 different graphs visualizing the overall incoming and outgoing internet traffic but also the http-requests in detail, send and received emails using POP3, SMTP and IMAP and the usage of different google webservices and application.

Download LogTool
Download Visualization Sourcecode

Weave Article

How to read

Our intention while working on this data visualization was to create an artistic interpretation of the network data that stays somehow readable, or at least comparable at the same time. So to understand the graph you need to know the following. The whole day (24hours) was split into 288 timeslots, 5 minutes each, represented by a radial bar graph. The gray bars show the overall outgoing internet traffic while the purple bars show all the packets that were received by the user. The dotted line on the other hand is a more detailed view of all HTTP requests. So by comparing the bars and the line you can tell if the traffic was caused by browsing websites or not. The probably most eye-catching part are the blue bubbles that show the immense usage of different Google services like Google maps, gmail, youtube, blogger, or just Google search, but also applications like Google earth or SketchUp, that communicate with Google servers to check for updates or load data, were tracked. But the really interesting information is the one that is not directly shown. There is still some more to discover like the little gaps that represent the moment in time when the users switched off their laptops to get to work and some hours later again when they went back home. Or some continuous peaks that are evidence for some hourly auto updates or backups.

kiefer_100224

server_100209

wacker_100406

rushmeyer_100405

fontaine_100409

sauter_100331

laub_100303