A multimodal in-car dialogue system that tracks the driver's attention

Spyros Kousidis; Casey Kennington; Timo Baumann; Hendrik Buschmeier; Stefan Kopp; David Schlangen

doi:10.1145/2663204.2663244

A multimodal in-car dialogue system that tracks the driver's attention

Spyros Kousidis, Casey Kennington, Timo Baumann, Hendrik Buschmeier, Stefan Kopp, David Schlangen

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

20 Scopus citations

Abstract

When a passenger speaks to a driver, he or she is co-located with the driver, is generally aware of the situation, and can stop speaking to allow the driver to focus on the driving task. In-car dialogue systems ignore these important aspects, making them more distracting than even cell-phone conversations. We developed and tested a "situationally-aware" dialogue system that can interrupt its speech when a situation which requires more attention from the driver is detected, and can resume when driving conditions return to normal. Furthermore, our system allows driver-controlled resumption of interrupted speech via verbal or visual cues (head nods). Over two experiments, we found that the situationally-aware spoken dialogue system improves driving performance and attention to the speech content, while driver-controlled speech resumption does not hinder performance in either of these two tasks.

Original language	English
Title of host publication	ICMI 2014 - Proceedings of the 2014 International Conference on Multimodal Interaction
Pages	26-33
Number of pages	8
ISBN (Electronic)	9781450328852
DOIs	https://doi.org/10.1145/2663204.2663244
State	Published - 12 Nov 2014
Externally published	Yes
Event	16th ACM International Conference on Multimodal Interaction, ICMI 2014 - Istanbul, Turkey Duration: 12 Nov 2014 → 16 Nov 2014

Publication series

Name	ICMI 2014 - Proceedings of the 2014 International Conference on Multimodal Interaction

Conference

Conference	16th ACM International Conference on Multimodal Interaction, ICMI 2014
Country/Territory	Turkey
City	Istanbul
Period	12/11/14 → 16/11/14

Keywords

In-car dialogue
Incremental dialogue
Multimodal
Speech output generation
Spoken dialogue systems

Access to Document

10.1145/2663204.2663244

Cite this

Kousidis, S., Kennington, C., Baumann, T., Buschmeier, H., Kopp, S., & Schlangen, D. (2014). A multimodal in-car dialogue system that tracks the driver's attention. In ICMI 2014 - Proceedings of the 2014 International Conference on Multimodal Interaction (pp. 26-33). (ICMI 2014 - Proceedings of the 2014 International Conference on Multimodal Interaction). https://doi.org/10.1145/2663204.2663244

@inproceedings{b676d812e5ce4b45a0fd4eb93dcaa1e1,

title = "A multimodal in-car dialogue system that tracks the driver's attention",

abstract = "When a passenger speaks to a driver, he or she is co-located with the driver, is generally aware of the situation, and can stop speaking to allow the driver to focus on the driving task. In-car dialogue systems ignore these important aspects, making them more distracting than even cell-phone conversations. We developed and tested a {"}situationally-aware{"} dialogue system that can interrupt its speech when a situation which requires more attention from the driver is detected, and can resume when driving conditions return to normal. Furthermore, our system allows driver-controlled resumption of interrupted speech via verbal or visual cues (head nods). Over two experiments, we found that the situationally-aware spoken dialogue system improves driving performance and attention to the speech content, while driver-controlled speech resumption does not hinder performance in either of these two tasks.",

keywords = "In-car dialogue, Incremental dialogue, Multimodal, Speech output generation, Spoken dialogue systems",

author = "Spyros Kousidis and Casey Kennington and Timo Baumann and Hendrik Buschmeier and Stefan Kopp and David Schlangen",

year = "2014",

month = nov,

day = "12",

doi = "10.1145/2663204.2663244",

language = "English",

series = "ICMI 2014 - Proceedings of the 2014 International Conference on Multimodal Interaction",

pages = "26--33",

booktitle = "ICMI 2014 - Proceedings of the 2014 International Conference on Multimodal Interaction",

}

Kousidis, S, Kennington, C, Baumann, T, Buschmeier, H, Kopp, S & Schlangen, D 2014, A multimodal in-car dialogue system that tracks the driver's attention. in ICMI 2014 - Proceedings of the 2014 International Conference on Multimodal Interaction. ICMI 2014 - Proceedings of the 2014 International Conference on Multimodal Interaction, pp. 26-33, 16th ACM International Conference on Multimodal Interaction, ICMI 2014, Istanbul, Turkey, 12/11/14. https://doi.org/10.1145/2663204.2663244

A multimodal in-car dialogue system that tracks the driver's attention. / Kousidis, Spyros; Kennington, Casey; Baumann, Timo et al.
ICMI 2014 - Proceedings of the 2014 International Conference on Multimodal Interaction. 2014. p. 26-33 (ICMI 2014 - Proceedings of the 2014 International Conference on Multimodal Interaction).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A multimodal in-car dialogue system that tracks the driver's attention

AU - Kousidis, Spyros

AU - Kennington, Casey

AU - Baumann, Timo

AU - Buschmeier, Hendrik

AU - Kopp, Stefan

AU - Schlangen, David

PY - 2014/11/12

Y1 - 2014/11/12

N2 - When a passenger speaks to a driver, he or she is co-located with the driver, is generally aware of the situation, and can stop speaking to allow the driver to focus on the driving task. In-car dialogue systems ignore these important aspects, making them more distracting than even cell-phone conversations. We developed and tested a "situationally-aware" dialogue system that can interrupt its speech when a situation which requires more attention from the driver is detected, and can resume when driving conditions return to normal. Furthermore, our system allows driver-controlled resumption of interrupted speech via verbal or visual cues (head nods). Over two experiments, we found that the situationally-aware spoken dialogue system improves driving performance and attention to the speech content, while driver-controlled speech resumption does not hinder performance in either of these two tasks.

AB - When a passenger speaks to a driver, he or she is co-located with the driver, is generally aware of the situation, and can stop speaking to allow the driver to focus on the driving task. In-car dialogue systems ignore these important aspects, making them more distracting than even cell-phone conversations. We developed and tested a "situationally-aware" dialogue system that can interrupt its speech when a situation which requires more attention from the driver is detected, and can resume when driving conditions return to normal. Furthermore, our system allows driver-controlled resumption of interrupted speech via verbal or visual cues (head nods). Over two experiments, we found that the situationally-aware spoken dialogue system improves driving performance and attention to the speech content, while driver-controlled speech resumption does not hinder performance in either of these two tasks.

KW - In-car dialogue

KW - Incremental dialogue

KW - Multimodal

KW - Speech output generation

KW - Spoken dialogue systems

UR - https://www.scopus.com/pages/publications/84947217152

U2 - 10.1145/2663204.2663244

DO - 10.1145/2663204.2663244

M3 - Conference contribution

T3 - ICMI 2014 - Proceedings of the 2014 International Conference on Multimodal Interaction

SP - 26

EP - 33

BT - ICMI 2014 - Proceedings of the 2014 International Conference on Multimodal Interaction

T2 - 16th ACM International Conference on Multimodal Interaction, ICMI 2014

Y2 - 12 November 2014 through 16 November 2014

ER -

A multimodal in-car dialogue system that tracks the driver's attention

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this