A Modular Knowledge-Extraction Framework for Deep Learning Forecasts of Multi-Tier Commodity Prices
Vertically linked commodity markets—global futures, regional spot, and farm-gate prices—transmit information through directed cross-market channels whose strength varies with latent volatility regimes. Standard deep learning forecasters absorb both the directed cross-market dependence and the regime dependence of intrinsic-mode-aligned latent components into shared model weights, with no explicit architectural mechanism that exposes either as an inspectable structure. This paper proposes HVB-RA, a modular framework that combines two such mechanisms with a per-tier Variational Mode Decomposition and bidirectional LSTM backbone: (i) a directed cross-market attention layer in which the upstream-to-downstream topology is supplied from domain knowledge and the time-varying upstream-source attention intensities at the farm-gate tier (the regional-spot tier, with a single upstream key, reduces algebraically to a fixed residual upstream fusion) are extracted from data, and (ii) a regime-informed modal-weighting layer that mixes two trainable softmax weight profiles over IMF-aligned latent components through a filtered Markov-switching state probability fitted in a separate stage. An auxiliary post hoc projection enforces an exact linear constraint defined by long-run sample-mean ratios across tiers; the paper does not claim that these descriptive ratios are cointegrating relations or equilibrium coefficients. The framework is evaluated on three tiers of daily natural-rubber prices spanning 2038 trading days, against three external benchmarks (random walk, ARIMA(2,0,2), and an exogenous-only LSTM) and a contemporary neural hierarchical-interpolation forecaster (NHITS). Root mean squared error is reported per tier-horizon cell; a decision-aware income-smoothing metric quantifies the operational value of h=5 farm-gate forecasts under a 5-day selling rule; and a within-method comparison evaluates the marginal contribution of the auxiliary constraint projection. On the present single-regime test window, HVB-RA attains a lower point error than the contemporary NHITS baseline at every tier-horizon cell, while no method—including HVB-RA—improves on the random-walk floor at most cells; the regime-conditional components of the architecture are not identifiable because every calibration and test origin is classified as a high-volatility regime by the trained Markov-switching model. The paper contributes to machine learning and knowledge extraction by demonstrating how time-varying upstream-source attention intensities at the farm-gate tier and regime-dependent latent-component-weight profiles—two forms of latent structure typically absorbed into model weights—can be exposed as explicit, inspectable, and individually testable components of a multi-tier forecasting architecture, and by providing a reproducibility package documenting the conditions under which each component is expected to be identifiable.