Temporal Alignment Using the Incremental Unit Framework

Casey Kennington, Ting Han, David Schlangen

Research output: Chapter in Book/Report/Conference proceedingChapter

4 Scopus citations

Abstract

We propose a method for temporal alignments--a precondition of meaningful fusions--of multimodal systems, using the incremental unit dialogue system framework, which gives the system flexibility in how it handles alignment: either by delaying a modality for a specified amount of time, or by revoking (i.e., backtracking) processed information so multiple information sources can be processed jointly. We evaluate our approach in an offline experiment with multimodal data and find that using the incremental framework is flexible and shows promise as a solution to the problem of temporal alignment in multimodal systems.

Original languageAmerican English
Title of host publicationICMI 2017: Proceedings of the 19th ACM International Conference on Multimodal Interaction
DOIs
StatePublished - 1 Jan 2017

Keywords

  • alignment
  • fusion
  • incremental
  • multimodal

EGS Disciplines

  • Computer Sciences

Fingerprint

Dive into the research topics of 'Temporal Alignment Using the Incremental Unit Framework'. Together they form a unique fingerprint.

Cite this