Real-Time Understanding of Complex Discriminative Scene Descriptions

Ramesh Manuvinakurike; Casey Kennington; David DeVault; David Schlangen

Real-Time Understanding of Complex Discriminative Scene Descriptions

Ramesh Manuvinakurike, Casey Kennington, David DeVault, David Schlangen

Computer Science Department

Research output: Chapter in Book/Report/Conference proceeding › Chapter

5 Scopus citations

Abstract

Real-world scenes typically have complex structure, and utterances about them consequently do as well. We devise and evaluate a model that processes descriptions of complex configurations of geometric shapes and can identify the described scenes among a set of candidates, including similar distractors. The model works with raw images of scenes, and by design can work word-by-word incrementally. Hence, it can be used in highly-responsive interactive and situated settings. Using a corpus of descriptions from game-play between human subjects (who found this to be a challenging task), we show that reconstruction of description structure in our system contributes to task success and supports the performance of the word-based model of grounded semantics that we use.

Original language	American English
Title of host publication	SIGDIAL 2016: 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference
Pages	232-241
Number of pages	10
ISBN (Electronic)	9781945626234
State	Published - 2016
Event	17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL 2016 - , United States Duration: 13 Sep 2016 → 15 Sep 2016

Publication series

Name	SIGDIAL 2016 - 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference

Conference

Conference	17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL 2016
Country/Territory	United States
Period	13/09/16 → 15/09/16

EGS Disciplines

Computer Sciences

Cite this

Manuvinakurike, R., Kennington, C., DeVault, D., & Schlangen, D. (2016). Real-Time Understanding of Complex Discriminative Scene Descriptions. In SIGDIAL 2016: 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference (pp. 232-241). (SIGDIAL 2016 - 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference).

Manuvinakurike, Ramesh ; Kennington, Casey ; DeVault, David et al. / Real-Time Understanding of Complex Discriminative Scene Descriptions. SIGDIAL 2016: 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference. 2016. pp. 232-241 (SIGDIAL 2016 - 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference).

@inbook{ed35a71665c041d0b7d46421938f889f,

title = "Real-Time Understanding of Complex Discriminative Scene Descriptions",

abstract = " Real-world scenes typically have complex structure, and utterances about them consequently do as well. We devise and evaluate a model that processes descriptions of complex configurations of geometric shapes and can identify the described scenes among a set of candidates, including similar distractors. The model works with raw images of scenes, and by design can work word-by-word incrementally. Hence, it can be used in highly-responsive interactive and situated settings. Using a corpus of descriptions from game-play between human subjects (who found this to be a challenging task), we show that reconstruction of description structure in our system contributes to task success and supports the performance of the word-based model of grounded semantics that we use.",

author = "Ramesh Manuvinakurike and Casey Kennington and David DeVault and David Schlangen",

note = "Publisher Copyright: {\textcopyright} 2016 Association for Computational Linguistics.; 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL 2016 ; Conference date: 13-09-2016 Through 15-09-2016",

year = "2016",

language = "American English",

series = "SIGDIAL 2016 - 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference",

pages = "232--241",

booktitle = "SIGDIAL 2016: 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference",

}

Manuvinakurike, R, Kennington, C, DeVault, D & Schlangen, D 2016, Real-Time Understanding of Complex Discriminative Scene Descriptions. in SIGDIAL 2016: 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference. SIGDIAL 2016 - 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference, pp. 232-241, 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL 2016, United States, 13/09/16.

Real-Time Understanding of Complex Discriminative Scene Descriptions. / Manuvinakurike, Ramesh; Kennington, Casey; DeVault, David et al.
SIGDIAL 2016: 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference. 2016. p. 232-241 (SIGDIAL 2016 - 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference).

Research output: Chapter in Book/Report/Conference proceeding › Chapter

TY - CHAP

T1 - Real-Time Understanding of Complex Discriminative Scene Descriptions

AU - Manuvinakurike, Ramesh

AU - Kennington, Casey

AU - DeVault, David

AU - Schlangen, David

PY - 2016

Y1 - 2016

N2 - Real-world scenes typically have complex structure, and utterances about them consequently do as well. We devise and evaluate a model that processes descriptions of complex configurations of geometric shapes and can identify the described scenes among a set of candidates, including similar distractors. The model works with raw images of scenes, and by design can work word-by-word incrementally. Hence, it can be used in highly-responsive interactive and situated settings. Using a corpus of descriptions from game-play between human subjects (who found this to be a challenging task), we show that reconstruction of description structure in our system contributes to task success and supports the performance of the word-based model of grounded semantics that we use.

AB - Real-world scenes typically have complex structure, and utterances about them consequently do as well. We devise and evaluate a model that processes descriptions of complex configurations of geometric shapes and can identify the described scenes among a set of candidates, including similar distractors. The model works with raw images of scenes, and by design can work word-by-word incrementally. Hence, it can be used in highly-responsive interactive and situated settings. Using a corpus of descriptions from game-play between human subjects (who found this to be a challenging task), we show that reconstruction of description structure in our system contributes to task success and supports the performance of the word-based model of grounded semantics that we use.

UR - https://www.sigdial.org/files/workshops/conference17/proceedings/pdf/SIGDIAL30.pdf

UR - http://www.scopus.com/inward/record.url?scp=85020697962&partnerID=8YFLogxK

M3 - Chapter

T3 - SIGDIAL 2016 - 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference

SP - 232

EP - 241

BT - SIGDIAL 2016: 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference

T2 - 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL 2016

Y2 - 13 September 2016 through 15 September 2016

ER -

Manuvinakurike R, Kennington C, DeVault D, Schlangen D. Real-Time Understanding of Complex Discriminative Scene Descriptions. In SIGDIAL 2016: 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference. 2016. p. 232-241. (SIGDIAL 2016 - 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Proceedings of the Conference).

Real-Time Understanding of Complex Discriminative Scene Descriptions

Abstract

Publication series

Conference

EGS Disciplines

Other files and links

Fingerprint

Cite this