Two-Way Interventive Design and Manufacturing: Realtime design and materiality visualization through mixed reality systems based on hand tracking

This research proposes a mixed reality (MR)human-computer collaboration system based on a "two-way intervention" approach, which integrates MR and hand tracking technology to enhance the design and manufacturing workflow driven by human creativity. By combining MR devices such as HoloLens2 (MR headset) with advanced design algorithms, the system not only effectively copes with uncertainties in the initial design process, but also simulates the design and manufacturing process of architectural materials by learning the physical properties of real materials. This improves the flexibility and operability of the entire process. The system adopts two-way real-time interaction mode, which breaks through the limitation of traditional design and manufacturing which is dominated by top-down linear flow, and realizes the close combination of design and manufacturing process. With the help of hand tracking technology, users can intuitively control design elements in the virtual environment, adjust and optimize the design in real time, forming a highly collaborative closed-loop design and manufacturing process. At the same time, the system can dynamically respond to the user's design intentions and changes in material conditions, providing intelligent and customized design optimization suggestions. The experimental results show that the proposed method has obvious advantages in dynamic and adaptive aspects, and can effectively deal with the challenges in complex design and manufacturing scenarios, while improving material utilization efficiency and reducing production waste.