ICC 2017

Using space-time cube for visualization of active transportation patterns derived from public webcams

Anna Petrasova, J. Aaron Hipp, Helena Mitasova
North Carolina State University

Public webcams

Rich source of spatio-temporal information
  • weather, traffic, changes in environment, phenology, ...
  • active transportation behavior in urban areas


The Archive of Many Outdoor Scenes
  • collection of long-term timelapse imagery from publicly accessible outdoor webcams around the world
  • 1,128,087,180 images taken from 29945 webcams
  • a project of the Media and Machines Lab Washington University in St. Louis
  • online browsing of images and download available
  • metadata and tags to improve discoverability of webcams

From image to information

How to get from image to information useful for analysis?

Artificial intelligence

Artficial artficial intelligence

  • Amazon Mechanical Turk
  • crowdsourcing marketplace platform
  • fake chess-playing machine (late 18th century)

mTurk HITs (Human Intelligence Tasks)

HITs processing workflow


Using coordinate system of the webcam image:

  • distances in the image represent varying distances in reality
  • we can't integrate other geospatial datasets (streets, POIs) or information from other webcams

Solution is to compute projective transformation by matching 4+ stable features in the webcam image to the same features in the orthophoto.

Georeferencing: example

Caveats: some webcams change orientation, many objects such as benches, traffic marking are unsuitable as GCPs, stable objects such as statues can move too


Small errors in the mTurk outlines result in large spatial errors further from the webcam

STC visualization

Space-time density of pedestrians represented as a 3D volume, computed using multivariate Kernel Density Estimation (KDE) with different spatial and temporal bandwidths

Pedestrian density visualization

webcam 9706 (July), Ehingen, Germany

Pedestrian density visualization

webcam 10823 (July), Überlingen, Germany

Effects of plaza reconstruction

webcam 3760 in 2012 (Jul - Sep), Victoria Square, Adelaide, Australia

Effects of plaza reconstruction

webcam 3760 in 2014 (Jul - Sep), Victoria Square, Adelaide, Australia

Change in pedestrian density (2014 minus 2012)

Positive values ~ increase in density in 2014
Negative values ~ decrease in density in 2014

High density of pedestrians and vehicles (webcam 5599)

if (P > percentileP99 AND V > percentileV99, V + P)

Challenges: webcam geometry and view

  • areas hidden behind trees or other objects
  • assumes pedestrians and vehicles on a horizontal plane, otherwise we get large spatial errors

Challenges: mTurk reliability

Traffic lights, statues mistakenly marked as pedestrians, machine learning approaches would avoid this type of error


  • Python libraries
  • Jupyter Notebook for data exploration
  • Georeferencing: scikit-image, GRASS GIS
  • KDE: SciPy, Statsmodels
  • Rendering: ParaView, GRASS GIS


Conclusion & Future work

  • new method for harvesting and visualization of spatio-temporal information about active transportation
  • new way for cities to detect and analyze changes in active transportation behavior in an unintrusive way
  • georeferenced data give us the ability to incorporate other geospatial data and methods (e.g., solar radiation modeling)
  • possible thanks to the synergy between crowdsourcing technologies (AMOS, mTurk, open source software)
  • machine learning techniques trained by mTurk data will enable us to analyze much larger data volume in real-time, possibly leading to the discovery of more patterns



  • Hipp, J. A., Adlakha, D., Gernes, R., Kargol, A., Pless, R., Drive, O. B., Louis, S. (2013). Do You See What I See: Crowdsource Annotation of Captured Scenes, 24–25. http://doi.org/10.1145/2526667.2526671
  • Hipp, J. A., Manteiga, A., Burgess, A., Stylianou, A., Pless, R. (2016). Webcams, Crowdsourcing, and Enhanced Crosswalks: Developing a Novel Method to Analyze Active Transportation. Front. Public Health, 4(97). http://doi.org/10.3389/fpubh.2016.00097
  • Jacobs, N., Roman, N., Pless, R. (2007). Consistent temporal variations in many outdoor scenes. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. http://doi.org/10.1109/CVPR.2007.383258

Reliability results for annotation of pedestrians in 720 webcam scenes.

Hipp, J. A., Adlakha, D., Gernes, R., Kargol, A., Pless, R., Drive, O. B., Louis, S. (2013). Do You See What I See: Crowdsource Annotation of Captured Scenes, 24–25. http://doi.org/10.1145/2526667.2526671