Four-dimensional spatial-temporal packet encapsulation in the Cyber-Physical Internet: A reinforcement learning approach

This paper investigates packet encapsulation in the Cyber-Physical Internet (CPI), where boxes must be encapsulated into containers under joint three-dimensional (3D) geometric constraints and near real-time temporal requirements, governed by physical handling and transshipment tolerances. Even though the Heterogeneous Container Loading Problem (HCLP) is prevalent in logistics, it often overlooks the real-time encapsulation needs of CPI. Thus, this paper proposes a novel approach for four-dimensional (4D) spatial–temporal encapsulation in CPI. We focus on two sequential objectives: box allocation (assigning standardized boxes to appropriate containers) and box packing (efficiently loading boxes into allocated containers). To address these objectives, the 4D spatial–temporal encapsulation process in CPI is abstracted as a near real-time, large-scale HCLP (TL-HCLP), where stringent near real‑time operational constraints and massive scale requirements render traditional exact algorithms and heuristic methods insufficient. Therefore, we develop a deep reinforcement learning (DRL) algorithm. While most existing DRL algorithms are designed for single-container loading scenarios, addressing only the box packing objective, our method introduces an original reward mechanism that simultaneously handles both box allocation and packing tasks. During the implementation, we encountered computational bottlenecks in meeting the near real-time requirements. To overcome this, we further optimized the solution by incorporating GPU acceleration, significantly reducing the computation time. To validate the efficiency of the proposed algorithm, multiple computational experiments were conducted. The results demonstrate that the DRL algorithm outperforms traditional heuristics, confirming its effectiveness and applicability in real-world CPI scenarios.