SpeechCap: Leveraging Playful Impact Captions to Facilitate Interpersonal Communication in Social Virtual Reality

Social Virtual Reality (VR) offers immersive, interactive, and engaging mechanisms for collaborative activities within virtual environments. However, interpersonal communication in social VR remains constrained by existing mediums and channels. To address this limitation, we introduce an impact-caption-inspired approach to facilitate real-time conversations in social VR. Impact captions are a type of typographic visual effects commonly employed in videos to convey verbal messages and non-verbal cues simultaneously for enhancing viewer engagement. Starting with an exploration of the design space of impact captions, we subsequently developed a proof-of-concept system, SpeechCap, that enables users to communicate through speech-driven impact captions in VR. Using the system, we conducted a user study (N=14) to assess the effectiveness of the visual and interaction design of our approach, revealing the strengths in enhancing interactivity and integrating of verbal and non-verbal information. We conclude by discussing key findings related to visual rhetoric, interactivity of communication mediums, and ambiguity, and offer design implications aimed at improving interpersonal communication in social VR.