A real–time dynamic cluster–driven financial forecasting with an attention–based deep learning and DEA

This study proposes a novel real–time dynamic cluster–driven financial forecasting framework that utilizes Modified Dynamic Fuzzy C–Means (MdFCM) clustering, a decomposition–based Directional Distance Function Data Envelopment Analysis (DDF–DEA), and TabPFN, an attention–based prior–data–fitted network, to enhance efficiency analysis in volatile financial markets. Traditional DEA machine learning (DEA–ML) hybrids, though widely applied, are constrained by static benchmarking, an inability to accommodate negative or undesirable outputs, and limited interpretability under dynamic conditions. To address these limitations, our framework proposes a streaming–optimized architecture that optimally updates efficiency frontiers and forecasting models as new data are released. MdFCM redefines clusters of heterogeneous firms in an adaptive manner so that peer groups vary along with market regimes. Within peer clusters, DDF–DEA compares efficiency through simultaneous expansion of desirable (return) outputs and contraction of undesirable (risk) outputs, while TabPFN provides cluster–level forecasts of SHAP–powered interpretability and real–time adaptability. The system continuously monitors and updates efficiency forecasts in real time, with built–in fallback mechanisms ensuring robustness against cluster instability or data cold starts. Experimental verification on 6,177 data points (2020–2024) confirms superior accuracy and robustness through an R² of 0.9527, RMSE of 0.0485, and MAPE of 2.87%, outperforming baseline models including Random Forest, SVR, KNN, TabNet, and MLP. Using the Wilcoxon signed–rank statistical test, the robustness of our model becomes evident. Also, beyond predictive superiority, our framework provides transparent insights into the determinants of efficiency within diverse clusters of firms that inform decision support to portfolio managers, financial regulators, and corporate executives.