Heterogeneous Edge Caching Based on Actor-Critic Learning With Attention Mechanism Aiding

In recent years, the explosive growth of network traffic has placed significant strain on backbone networks. To alleviate the content access delay and reduce additional network resource consumption resulting from large-scale requests, edge caching has emerged as a promising technology. Despite capturing substantial attention from both academia and industry, most existing studies overlook the heterogeneity of the environment and the spatial-temporal characteristics of content popularity. As a result, the potential for edge caching remains largely unexploited. To address these challenges, we propose a neighborhood-aware caching (NAC) framework in this paper. The framework leverages the perimeter information from neighboring base stations (BSs) to model the edge caching problem in heterogeneous scenarios as a Markov Decision Process (MDP). To fully exploit the environmental information, we introduce an improved actor-critic method that integrates an attention mechanism into the neural network. The actor-network in our framework is responsible for making caching decisions based on local information, while the critic network evaluates and enhances the actor's performance. The multi-head attention layer in the critic network enables integration of environmental features into the model, reducing the limitations associated with local investigation. To facilitate comparison from an engineering perspective, we also propose a heuristic algorithm, Neighbor-Influence-Least-Frequently-Use (NILFU). Our extensive experiments demonstrate that the proposed NAC framework outperforms other baseline methods in terms of average delay, hit rate, and traffic offload ratio in heterogeneous scenarios. This highlights the effectiveness of the neighborhood-aware caching approach in enhancing the performance of edge caching systems in such scenarios.