Vinay Bettadapura

vinay [at] gatech.edu

Edison Thomaz

ethomaz [at] gatech.edu

Aman Parnami

aparnami3 [at] gatech.edu

Gregory Abowd

abowd [at] gatech.edu

Irfan Essa

irfan [at] cc.gatech.edu

School of Interactive Computing, Georgia Institute of Technology, Atlanta, USA

The pervasiveness of mobile cameras has resulted in a dramatic increase in food photos, which are pictures reflecting what people eat. In this paper, we study how taking pictures of what we eat in restaurants can be used for the purpose of automating food journaling. We propose to leverage the context of where the picture was taken, with additional information about the restaurant, available online, coupled with state-of-the-art computer vision techniques to recognize the food being consumed. To this end, we demonstrate image-based recognition of foods eaten in restaurants by training a classifier with images from restaurant’s online menu databases. We evaluate the performance of our system in unconstrained, real-world settings with food images taken in 10 restaurants across 5 different types of food (American, Indian, Italian, Mexican and Thai).

Download PDF

Download Poster

@inproceedings{Bettadapura:2015:EgocentricFood,
       Author = {Vinay Bettadapura and Edison Thomaz and Aman Parnami and Gregory Abowd and Irfan Essa},
       Booktitle = {Proceedings of IEEE Winter Conference on Applications of Computer Vision (WACV)},
       Month = {January},
       Organization = {IEEE},
       Title = {Leveraging Context to Support Automated Food Recognition in Restaurants},
       Year = {2015}
}

Funding and sponsorship was provided by the U.S. Army Research Office (ARO) and Defense Advanced Research Projects Agency (DARPA) under Contract No. W911NF-11-C-0088 and the Intel Science and Technology Center for Pervasive Computing (ISTCPC). Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of our sponsors.

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without explicit permission of the copyright holder.