Studying relationships between human gaze, description, and computer vision

Kiwon Yun; Yifan Peng; Dimitris Samaras; Gregory J. Zelinsky; Tamara L. Berg

doi:10.1109/CVPR.2013.101

Studying relationships between human gaze, description, and computer vision

Kiwon Yun
, Yifan Peng
, Dimitris Samaras
, Gregory J. Zelinsky
, Tamara L. Berg

Stony Brook University

Research output: Contribution to journal › Conference article › peer-review

69 Scopus citations

Abstract

We posit that user behavior during natural viewing of images contains an abundance of information about the content of images as well as information related to user intent and user defined content importance. In this paper, we conduct experiments to better understand the relationship between images, the eye movements people make while viewing images, and how people construct natural language to describe images. We explore these relationships in the context of two commonly used computer vision datasets. We then further relate human cues with outputs of current visual recognition systems and demonstrate prototype applications for gaze-enabled detection and annotation.

Original language	English
Article number	6618945
Pages (from-to)	739-746
Number of pages	8
Journal	Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
DOIs	https://doi.org/10.1109/CVPR.2013.101
State	Published - 2013
Event	26th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2013 - Portland, OR, United States Duration: Jun 23 2013 → Jun 28 2013

Access to Document

10.1109/CVPR.2013.101

Cite this

@article{c115a63007a3489e930b42f999d96818,

title = "Studying relationships between human gaze, description, and computer vision",

abstract = "We posit that user behavior during natural viewing of images contains an abundance of information about the content of images as well as information related to user intent and user defined content importance. In this paper, we conduct experiments to better understand the relationship between images, the eye movements people make while viewing images, and how people construct natural language to describe images. We explore these relationships in the context of two commonly used computer vision datasets. We then further relate human cues with outputs of current visual recognition systems and demonstrate prototype applications for gaze-enabled detection and annotation.",

author = "Kiwon Yun and Yifan Peng and Dimitris Samaras and Zelinsky, \{Gregory J.\} and Berg, \{Tamara L.\}",

year = "2013",

doi = "10.1109/CVPR.2013.101",

language = "English",

pages = "739--746",

journal = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition",

issn = "1063-6919",

note = "26th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2013 ; Conference date: 23-06-2013 Through 28-06-2013",

}

TY - JOUR

T1 - Studying relationships between human gaze, description, and computer vision

AU - Yun, Kiwon

AU - Peng, Yifan

AU - Samaras, Dimitris

AU - Zelinsky, Gregory J.

AU - Berg, Tamara L.

PY - 2013

Y1 - 2013

N2 - We posit that user behavior during natural viewing of images contains an abundance of information about the content of images as well as information related to user intent and user defined content importance. In this paper, we conduct experiments to better understand the relationship between images, the eye movements people make while viewing images, and how people construct natural language to describe images. We explore these relationships in the context of two commonly used computer vision datasets. We then further relate human cues with outputs of current visual recognition systems and demonstrate prototype applications for gaze-enabled detection and annotation.

AB - We posit that user behavior during natural viewing of images contains an abundance of information about the content of images as well as information related to user intent and user defined content importance. In this paper, we conduct experiments to better understand the relationship between images, the eye movements people make while viewing images, and how people construct natural language to describe images. We explore these relationships in the context of two commonly used computer vision datasets. We then further relate human cues with outputs of current visual recognition systems and demonstrate prototype applications for gaze-enabled detection and annotation.

UR - https://www.scopus.com/pages/publications/84887396648

U2 - 10.1109/CVPR.2013.101

DO - 10.1109/CVPR.2013.101

M3 - Conference article

AN - SCOPUS:84887396648

SN - 1063-6919

SP - 739

EP - 746

JO - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

JF - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

M1 - 6618945

T2 - 26th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2013

Y2 - 23 June 2013 through 28 June 2013

ER -

Studying relationships between human gaze, description, and computer vision

Abstract

Access to Document

Other files and links

Fingerprint

Cite this