Advancing the forward-forward algorithm towards high-performance deep local learning

The Forward-Forward (FF) algorithm was recently proposed as a local learning method to address the limitations of backpropagation (BP), offering a memory-efficient and highly parallelizable alternative. However, it suffers from suboptimal performance and poor generalization in many complex tasks largely due to the lack of effective learning strategies. In this work, we reformulate FF using distance metric learning and propose a distance-forward algorithm (DF) to improve FF performance in supervised learning tasks while preserving its advantageous local computational properties. To achieve this, we reinterpret FF through the lens of centroid-based metric learning and develop a goodness-based N-pair margin loss to facilitate the learning of discriminative features. Furthermore, we integrate layer-collaboration local update strategies to reduce information loss caused by greedy local parameter updates. Finally, we extend our methods to spiking neural networks (SNNs) by developing an effective goodness function for temporal spike sequences, facilitating efficient event-driven implementation on neuromorphic hardware. Our comprehensive evaluation on eight datasets demonstrates that the proposed method surpasses existing FF models and other local learning approaches. Particularly, our DF methods require less than 40% of the memory cost compared to BP training, while exhibiting stronger robustness to multiple types of hardware-related noise. Overall, the proposed method provides an efficient and robust local learning solution, which potentially inspires future FF algorithm designs and prompts applications on many-core hardware architectures.