Vinay Bettadapura

College of Computing, Georgia Tech

vinay [at] gatech.edu

Grant Schindler

College of Computing, Georgia Tech

schindler [at] gatech.edu

Thomas Ploetz

Culture Lab, Newcastle University

thomas.ploetz [at] newcastle.ac.uk

Irfan Essa

College of Computing, Georgia Tech

irfan [at] cc.gatech.edu

We present data-driven techniques to augment Bag of Words (BoW) models, which allow for more robust modeling and recognition of complex long-term activities, especially when the structure and topology of the activities are not known a priori. Our approach specifically addresses the limitations of standard BoW approaches, which fail to represent the underlying temporal and causal information that is inherent in activity streams. In addition, we also propose the use of randomly sampled regular expressions to discover and encode patterns in activities. We demonstrate the effectiveness of our approach in experimental evaluations where we successfully recognize activities and detect anomalies in four complex datasets.

Download PDF

Download Poster

You can download the latest C++ version here: ABOW.tar.gz. See the included README file for all the details.

@inproceedings{Bettadapura:2013:ABOW,
       Author = {Vinay Bettadapura and Grant Schindler and Thomas Ploetz and Irfan Essa},
       Booktitle = {Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
       Month = {June},
       Organization = {IEEE Computer Society},
       Title = {Augmenting Bag-of-Words: Data-Driven Discovery of Temporal and Structural Information for Activity Recognition},
       Year = {2013}
}

Funding and sponsorship was provided by the U.S. Army Research Office (ARO) and Defense Advanced Research Projects Agency (DARPA) under Contract No. W911NF-11-C-0088 and W31P4Q-10-C-0262. Parts of this work have also been funded by the RCUK Research Hub on Social Inclusion through the Digital Economy (SiDE), and the German Research Foundation (DFG, Grant No. PL554/2-1). Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of our sponsors.

We thank Kitware, Disney Research and DARPA for providing the Ocean City, Soccer and WAAS datasets, respectively.

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without explicit permission of the copyright holder.