Semantics with Feeling: Emotions for Abstract Embedding, Affect for Concrete Grounding

An important yet underexplored aspect of meaning in both distributional and grounded models of semantics is emotion. In this paper, we explore how emotion can be predicted from descriptions of robot behaviors represented with embeddings. We then compare this approach with a grounded model that maps corresponding robot behaviors represented as internal states to the same emotion labels and discover comparable results. We then take the predictions from the second model and use them as a proxy for concrete affect (as opposed to abstract emotion) and use this derived affect to ground a semantic classifier in a retrieval task and see improvements on the retrieval task when affect is used as a grounded modality. This demonstrates that semantics can benefit from using a proxy of affect derived from human perceptions, given those perceptions are mapped to clear proxies, such the behaviors of an embodied robot.