Enhancing Immersion in Virtual Spaces: PhytualBlend - a real-time interaction system for seamless physical feedback

As an extension of physical space, VR and MR technologies offer convenient solutions for spatial and product design by providing immersive user experiences. Nevertheless, these technologies are limited in their ability to provide physical feedback, which can lead to a perceived disconnection between visual, auditory, and bodily sensations. This paper introduces PhytualBlend, an innovative real-time interaction system designed to enhance the physical feedback experience within virtual environments. The system employs perceptual and executive hardware driven by development boards as its hardware components. User’s operation on physical entities can be synchronized seamlessly to server-based virtual world through a local information hub, including position, orientation, and haptic interaction. Diverse interaction modes within the virtual world trigger corresponding feedback in the physical space, encompassing tactile vibrations, thermal sensations, visual illumination, and object deformations. Our prototypical demonstrates remarkable performance, achieving a frame rate exceeding 60 frames per second with imperceptible network latency. The modular design of PhytualBlend enables efficient and scalable expansion of various physical feedback, interactive modes, and new virtual environments. PhytualBlend bridges the gap between the virtual and physical worlds, enabling remote collaboration participants to experience and interact with virtual contents in a more tangible and realistic manner.