Marginal Calibration Masks Generator Differences in Held-Out Evaluation of Multivariate Time Series, with an Application to ERA5 Weather Sequences

arXiv:2609.33761v2 Announce Type: replace Abstract: Generative models for multivariate time series are routinely evaluated with pooled distributional metrics computed after marginal calibration. We show that, under a held-out protocol, the calibrated Kolmogorov-Smirnov (KS) distance differs from a real-versus-real floor by at most the distance of the calibrated sample to the training block, so that it carries little information about the generator. We audit a three-discriminator GAN with evolutionary weight adaptation (StatD2GAN) against re-implemented RCGAN and TimeGAN baselines on ERA5 weather sequences under a pre-specified protocol: the final two calendar years are held out behind a 168 hour embargo, calibration is fitted on the training block only, and the decision rules were frozen in advance. Evidence comes from 25 matched (location, seed) pairs across five locations, tested with Holm-corrected Wilcoxon signed-rank tests. First, isotonic calibration drives KS to a median of 1.019 times a per-location noise-and-shift floor, and neither baseline is distinguishable from StatD2GAN on it. Second, the sorted-representation discriminator is the only component whose removal significantly degrades cross-variable dependence (Kendall tau MAE +0.116, Holm p = 0.002); a rank-transformed variant also degrades it, consistent with quantile supervision rather than copula matching. Third, calibration raises physical constraint violations; projection removes them, but the largest marginal change (0.0110, Lhasa) exceeds the pre-specified tolerance in three locations. Fourth, the re-implemented TimeGAN is significantly better than StatD2GAN on Kendall tau and ACF MAE, RCGAN is worse on Kendall tau MAE and constraint violations, and the pre-specified collapse criterion was not met. We recommend floor-referenced marginal evaluation, matched-pair testing, and dependence and sequence-level metrics.