Abstract
We propose a method for temporal alignments--a precondition of meaningful fusions--of multimodal systems, using the incremental unit dialogue system framework, which gives the system flexibility in how it handles alignment: either by delaying a modality for a specified amount of time, or by revoking (i.e., backtracking) processed information so multiple information sources can be processed jointly. We evaluate our approach in an offline experiment with multimodal data and find that using the incremental framework is flexible and shows promise as a solution to the problem of temporal alignment in multimodal systems.
Original language | American English |
---|---|
Title of host publication | ICMI 2017: Proceedings of the 19th ACM International Conference on Multimodal Interaction |
DOIs | |
State | Published - 1 Jan 2017 |
Keywords
- alignment
- fusion
- incremental
- multimodal
EGS Disciplines
- Computer Sciences