全部动态

本页 40 条
10月12日 周一
arXiv cs.LG64

On the Rate of Convergence of GD in Non-linear Neural Networks: An Adversarial Robustness Perspective

arXiv:2603.02095v2 Announce Type: replace Abstract: We study the convergence dynamics of Gradient Descent (GD) in a minimal binary classification setting, consisting of a two-neuron ReLU network and two training instances. We prove that even under these strong simplifying assumptions, while GD successfully converges to an optimal robustness margin, effectively maximizing the distance between the decision boundary and the training points, this convergence occurs at a prohibitively slow rate, scaling strictly as $\Theta(1/\ln(t))$. To the best of our knowledge, this establishes the first explicit lower bound on the convergence rate of the robustness margin in a non-linear model. Through empirical simulations, we further demonstrate that this inherent failure mode is pervasive, exhibiting the exact same tight convergence rate across multiple natural network initializations. Our theoretical guarantees are derived via a rigorous analysis of the GD trajectories across the distinct activation patterns of the model. Specifically, we develop tight control over the system's dynamics to bound the trajectory of the decision boundary, overcoming the primary technical challenge introduced by the non-linear nature of the architecture.

arXiv cs.LG68

Bridge Matching Sampler: Scalable Sampling via Generalized Fixed-Point Diffusion Matching

arXiv:2603.00530v2 Announce Type: replace Abstract: Sampling from unnormalized densities using diffusion models has emerged as a powerful paradigm. However, while recent approaches that use least-squares `matching' objectives have improved scalability, they often necessitate significant trade-offs, such as restricting prior distributions or relying on unstable optimization schemes. By generalizing these methods as special forms of fixed-point iterations rooted in Nelson's relation, we develop a new method that addresses these limitations, called Bridge Matching Sampler (BMS). Our approach enables learning a stochastic transport map between arbitrary prior and target distributions with a single, scalable, and stable objective. Furthermore, we introduce a damped variant of this iteration that incorporates a regularization term to mitigate mode collapse and further stabilize training. Empirically, we demonstrate that our method enables sampling at unprecedented scales while preserving mode diversity, achieving state-of-the-art results on complex synthetic densities and high-dimensional molecular benchmarks.

arXiv cs.LG68

Beyond State-Wise Mirror Descent: Offline Policy Optimization with Parametric Policies

arXiv:2602.23811v5 Announce Type: replace Abstract: We investigate the theoretical aspects of offline reinforcement learning (RL) under general function approximation. While prior works (e.g., Xie et al., 2021) have established the theoretical foundations of learning a good policy from offline data via pessimism, existing algorithms that are computationally tractable (often in an oracle-efficient sense), such as PSPI, only apply to finite and small action spaces. Moreover, these algorithms rely on state-wise mirror descent and require actors to be implicitly induced from the critic functions, failing to accommodate standalone policy parameterization which is ubiquitous in practice. In this work, we address these limitations and extend the theoretical guarantees to parameterized policy classes over large or continuous action spaces. When extending mirror descent to parameterized policies, we identify contextual coupling as the core difficulty, and show how connecting mirror descent to natural policy gradient leads to novel analyses, guarantees, and algorithmic insights, including a surprising unification between offline RL and imitation learning.

arXiv cs.LG63

Federated Learning of Nonlinear Temporal Dynamics with Graph Attention-based Cross-Client Interpretability

arXiv:2602.13485v3 Announce Type: replace Abstract: Networks of modern industrial systems are increasingly monitored by distributed sensors, where each system comprises multiple subsystems that generate high-dimensional time-series. These subsystems are often interdependent, making it important to understand how temporal patterns observed at one subsystem relate to those observed across others. This problem is particularly challenging in decentralized settings where raw measurements cannot be shared, and client observations are heterogeneous. Moreover, in practice, each subsystem (client) operates a fixed, proprietary model that cannot be modified or retrained, limiting the applicability of existing approaches. Separately, nonlinear dynamics make cross-client temporal interdependencies difficult to characterize, as they are embedded implicitly within nonlinear state-transition functions. This paper presents a vertical federated learning framework for learning one-step-ahead cross-client temporal interdependencies under these constraints. Each client maps high-dimensional local observations to low-dimensional latent states using a nonlinear state-space model, while a Graph Attention Network (GAT)-based central server learns a state-transition model over the communicated states. Given a candidate graph skeleton, the server learns state-dependent attention coefficients that capture cross-client interactions. We analytically relate the GAT attention coefficients to the Jacobian of the learned state-transition model, yielding a first-order characterization of cross-client interdependencies. We further establish bounded closeness of the learned dynamics and Jacobians to a corresponding centralized oracle. Synthetic experiments validate the theoretical results on controlled systems with known graph skeletons. Experiments on real-world datasets with unknown graph skeletons demonstrate superior federated root cause analysis over baselines.

arXiv cs.LG73

ERIS：通过联邦分片聚合改善联邦学习中的隐私、效用与可扩展性权衡

arXiv:2602.08617v3 公告类型：替换 摘要：将联邦学习（FL）扩展至十亿参数规模的模型，迫使人们在隐私、可扩展性和模型效用之间进行艰难权衡。现有解决方案往往孤立地应对这些挑战，或牺牲准确性，或依赖成本高昂的密码学工具，或引入影响收敛的通信与优化低效问题。我们提出 ERIS，这是一个以联邦分片聚合（FSA）为核心的 FL 框架。FSA 是一种新机制，将每个客户端的更新划分为互不重叠的分片，并将其聚合工作分配给多个客户端侧聚合器。FSA 消除了中心聚合瓶颈，限制了任何单一观察者可见的信息，并在重新组装后保持与中心化 FL 相同的更新。ERIS 还可以便捷地集成分布式移位压缩（DSC），以减少传输载荷和暴露的坐标。我们证明，在标准假设下，ERIS 保持收敛性，且互信息泄漏的上界由每次更新中可观测部分的比例决定；该上界随客户端侧聚合器数量的增加而降低，在启用 DSC 时还会随压缩程度的提高而降低。在图像和文本任务（包括大语言模型）上的实验表明，ERIS 实现了与 FedAvg 相当的效用，同时显著缓解了通信瓶颈，并提高了对成员推断攻击和重建攻击的鲁棒性，且无需依赖繁重的密码学运算或降低效用的扰动。

arXiv cs.LG76

COBALT：利用离线轨迹进行上下文老虎机学习以实现多轮代码生成

arXiv:2602.03806v2 公告类型：替换 摘要：训练用于多轮代码生成的大语言模型（LLMs）通常依赖在线强化学习（RL），模型通过反复生成的在线采样轨迹进行学习。尽管有效，在线 RL 的成本可能很高，且训练可能不稳定。本文提出利用离线轨迹的上下文老虎机学习（COBALT），这是一种简单的方法，通过复用离线多轮轨迹，高效训练用于多轮代码生成的 LLMs。基于多轮代码生成是一个单步可恢复的马尔可夫决策过程这一观察，COBALT 首先使用参考 LLM 收集代码生成轨迹，并将其拆分为部分轨迹。随后，在在线老虎机学习过程中，通过单步代码生成训练 LLM 补全每条部分轨迹。在 LiveCodeBench 上，COBALT 显著提升了 R1-Distill 8B 和 Qwen3 8B 的表现，使其 Pass@1 分别最多提高 9.0 和 6.2 个绝对百分点，并达到了采用 GRPO 和 VeRPO 的两种多轮在线 RL 基线的性能。我们还对上下文内奖励投机进行了基于扰动的分析，结果表明 Cobalt 能够增强模型对误导性执行反馈的鲁棒性。总体而言，我们的结果表明，Cobalt 是一种很有前景的多轮代码生成解决方案。我们的代码和数据已公开，地址为 https://github.com/OSU-NLP-Group/cobalt。

arXiv cs.LG76

超越预测集大小：通过信念集几何量化共形不确定性

arXiv:2602.01667v2 公告类型：替换 摘要：预测集大小只能粗略概括共形不确定性。我们通过信念集的几何结构提出了一个更丰富的视角：信念集是与共形预测（CP）区域相容的概率分布族。我们为分割式 CP 建立了这一表示，将近期针对完全式 CP 发现的对应关系加以推广。基于这一表示，我们为若干广泛使用的信念集不确定性度量推导了解析刻画和精确算法，使原本可能需要指数时间的计算在对共形 p 值排序后可在线性时间内完成。我们进一步通过在各误差水平上对基于 Tsallis 熵的集合大小惩罚进行积分，将其中若干度量与经典共形汇总指标统一起来。我们还证明了一项不可能性定理：对于常见的分割式共形回归构造，其诱导的信念集的不确定性汇总指标无法区分不同输入。选择性预测和分布偏移检测的分类实验表明，这些汇总指标通常能够恢复固定误差水平下预测集大小所丢失的有用排序信息，从而实现更细粒度的不确定性评估。这些结果确立了信念集几何作为量化共形不确定性及理解其局限性的一个具有理论依据的框架。

arXiv cs.LG67

Preference-Conditioned Multi-Objective Reinforcement Learning for Tactical Decision Making in Heavy-Duty Vehicles

arXiv:2601.18783v3 Announce Type: replace Abstract: A central difficulty in applying reinforcement learning to real-world applications such as autonomous driving is that conventional scalar reward formulations obtained by aggregating multiple often competing objectives may obscure the structure of their trade-offs. These challenges are particularly relevant to decision-making for heavy-duty vehicles, which requires balancing safety, efficiency, and operational costs under demanding commercial requirements. We present a preference-conditioned multi-objective reinforcement learning framework that explicitly represents these objectives and utilizes Generalized Policy Improvement and Proximal Policy Optimization to learn policies capturing different trade-offs between them. The proposed method approximates the Pareto front across three conflicting objectives: safety, quantified in terms of collisions and successful completion; energy efficiency, quantified using energy cost; and time efficiency, quantified using driver cost. We evaluate the framework in a scalable highway traffic simulation environment tailored for tactical decision making in heavy-duty trucks. It yields smooth and interpretable Pareto fronts, and evaluations across traffic densities and against an analytical baseline show that the learned policies capture meaningful trade-offs. The preference-conditioned policy enables transitions between different driving behaviors without retraining, demonstrating the potential of multi-objective reinforcement learning for adaptive decision making in autonomous heavy-duty vehicles.

arXiv cs.LG

Bayesian dynamic scheduling of multipurpose batch processes under incomplete look-ahead information

arXiv:2512.01093v2 Announce Type: replace Abstract: Multipurpose batch processes become increasingly popular in manufacturing industries since they adapt to low-volume, high-value products and shifting demands. These processes often operate in a dynamic environment, which faces disturbances such as processing delays and demand changes. To minimise long-term cost and system nervousness (i.e., disruptive changes to schedules), schedulers must design rescheduling strategies to address such disturbances effectively. Existing methods often assume complete look-ahead information over the scheduling horizon. This assumption contrasts with realistic situations where schedulers can only access incomplete look-ahead information. Sticking with existing methods may lead to suboptimal long-term costs and high-level system nervousness. In this work we propose a Bayesian dynamic scheduling method. Our method relies on learning a Bayesian Network from the probability distribution of disturbances. Specifically, the Bayesian Network represents how likely each operation will be impacted by disturbances. During the online execution, when new disturbances become observed, this method updates the posterior distribution and therefore guides the rescheduling strategy. We compare our method with the existing periodic rescheduling strategy (which generates new schedules from scratch at fixed intervals) on four benchmark problems. Computational results show that our method achieves statistically better long-term costs and system nervousness. In the theoretical aspect, we prove that if disturbances are mutually independent, the impact-quantifying variables inherently satisfy the independence assumptions required by Bayesian Networks. As an implication, practitioners can extend the method to other scheduling problems (such as job shop scheduling and continuous processes), given that they define the problem-specific dependencies between operations.

arXiv cs.LG54

A Theory of the Mechanics of Information: Generalization Through Measurement of Uncertainty (Learning is Measuring)

arXiv:2510.22809v2 Announce Type: replace Abstract: Traditional machine learning relies on explicit models and domain assumptions, often limiting flexibility and interpretability. We introduce a "model-free" framework of instance-based learning using surprisal (information theoretic uncertainty) to directly analyze and perform inferences from raw data, eliminating distribution modeling, reducing bias, and enabling efficient updates including direct edits and deletion of training data. By quantifying relevance through uncertainty, the approach enables generalizable inference across tasks including generative inference, causal discovery, anomaly detection, and time series forecasting. Our approach emphasizes traceability, interpretability, and data-driven decision making, offering a unified, human-understandable framework for machine learning, can be adapted to virtually all machine learning tasks, achieving competency across all and state-of-the-art performance on anomaly detection and some supervised tasks. The mathematical foundations create a "physics" of information, which enable these techniques to apply effectively to a wide variety of complex data types, including missing data. Our results suggest that this may be a viable foundation upon which to build scalable machine learning and artificial intelligence that can maintain human understandability of the underlying mechanics.

arXiv cs.LG62

Mixture of Many Zero-Compute Experts: A High-Rate Quantization Theory Perspective

arXiv:2510.03151v2 Announce Type: replace Abstract: This paper uses classical high-rate quantization theory to provide new insights into mixture-of-experts (MoE) models for regression tasks. Our MoE is defined by a segmentation of the input space to regions, each with a single-parameter expert that acts as a constant predictor with zero-compute at inference. Motivated by high-rate quantization theory assumptions, we assume that the number of experts is sufficiently large to make their input-space regions very small. This lets us study the approximation error of our MoE model class: (i) for one-dimensional inputs, we formulate the test error and its minimizing segmentation and experts; (ii) for multidimensional inputs, we formulate an upper bound for the test error and study its minimization. Moreover, we consider the learning of the expert parameters from a training dataset, given an input-space segmentation, and formulate their statistical learning properties. This leads us to theoretically and empirically show how the tradeoff between approximation and estimation errors in MoE learning depends on the number of experts.

arXiv cs.LG63

Multidimensional Bayesian Active Machine Learning of Working Memory Task Performance

arXiv:2510.00375v3 Announce Type: replace Abstract: While adaptive experimental design has outgrown one-dimensional, staircase-based adaptations, most cognitive experiments still control a single factor and summarize performance with a scalar. In contrast, we validate here a Bayesian, two-axis, active-classification approach in an immersive virtual testing environment, using a 5-by-5 spatial working-memory reconstruction task as an application domain. Two variables are controlled: spatial load L (number of occupied tiles) and feature-binding load K (number of distinct colors) of items. Stimulus acquisition is guided by posterior uncertainty of a nonparametric Gaussian Process (GP) probabilistic classifier, which outputs a surface over (L, K) rather than a single threshold or maximum span value. In a young adult population, we compare GP-driven Adaptive Mode (AM) with a traditional adaptive staircase Classic Mode (CM), which varies L only at K = 3. Parity between the methods is achieved for this cohort, with an intraclass correlation coefficient of 0.755 at K = 3. As a secondary exploratory result, AM reveals individual differences in interactions between spatial load and feature binding. AM estimates converge more quickly than other sampling strategies, demonstrating that only about 30 samples are required for accurate fitting of the full model.

arXiv cs.LG65

Discovering Phase Space Structure in Learned Hamiltonian Systems

arXiv:2509.23003v3 Announce Type: replace Abstract: Mechanics-inspired neural dynamical models improve interpretability and physical consistency, but they usually assume a known configuration space, fixed physical parameters, and deterministic prediction from a prescribed initial condition. We introduce Structured Phase Space GAN (SPS-GAN), a conditional generative model that learns structured dynamics from both Cartesian trajectories and video. SPS-GAN combines a port-Hamiltonian backbone with adversarial conditional generation, allowing conservative, dissipative, and forced systems to be represented in a single architecture. To infer latent mechanical structure from raw observations, we use a cyclic-coordinate loss that recovers both the system degrees of freedom and canonical coordinates without access to the true phase space. Experiments on simulated and real-world systems show that SPS-GAN improves trajectory accuracy over supervised dynamics baselines and video consistency over generative baselines, while consistently recovering the correct latent mechanical coordinates across observation modalities and dynamical regimes.

arXiv cs.LG60

User Trajectory Prediction Unifying Global and Local Temporal Information

arXiv:2508.02161v2 Announce Type: replace Abstract: Trajectory prediction is essential for formulating proactive strategies that anticipate user mobility and support advance preparation. Therefore, how to reduce the forecasting error in user trajectory prediction within an acceptable inference time arises as an interesting issue. However, trajectory data contains both global and local temporal information, complicating the extraction of the complete temporal pattern. Moreover, user behavior occurs over different time scales, increasing the difficulty of capturing behavioral patterns. To address these challenges, a trajectory prediction model based on multilayer perceptron (MLP), multi-scale convolutional neural network (MSCNN), and cross-attention (CA) is proposed. Specifically, MLP is used to extract the global temporal information of each feature. In parallel, MSCNN is employed to extract the local temporal information by modeling interactions among features within a local temporal range. Convolutional kernels with different sizes are used in MSCNN to capture temporal information at multiple resolutions, enhancing the model's adaptability to different behavioral patterns. Finally, CA is applied to fuse the global and local temporal information. Experimental results show that our model reduces mean squared error (MSE) by 5.04% and mean absolute error (MAE) by 4.35% compared with ModernTCN in 12-step prediction, while maintaining similar inference time.

arXiv cs.LG77

用于 Shapley 值和概率值的回归调整蒙特卡洛估计量

arXiv:2506.11849v3 公告类型：替换 摘要：源于博弈论的概率值，如 Shapley 值、Banzhaf 值和半值，已成为可解释人工智能中的核心工具。它们被用于特征归因、数据归因、数据估值等任务。由于精确计算可能需要指数级时间，研究主要聚焦于利用蒙特卡洛采样和线性回归建模这两种技术的高效近似方法。在本研究中，我们提出了一种结合这两种技术的新方法。我们的方法比以往的算法更加灵活，允许将线性回归替换为任何能够高效计算其概率值的函数族。这使我们能够利用 XGBoost 等基于树的模型的准确性；通过交叉拟合，该方法还可产生无偏估计。在八个数据集上的实验表明，我们的方法在概率值估计方面达到了当前最先进的性能。对于 Shapley 值，我们方法的误差可低至 Permutation SHAP（最流行的蒙特卡洛方法）的 $1/11$、Kernel SHAP（最流行的线性回归方法）的 $1/6.5$，以及 Leverage SHAP（此前最先进的 Shapley 值估计量）的 $1/3.4$。对于更一般的概率值，我们能够将误差降至以往研究中最佳估计量的 $1/390$。

arXiv cs.LG66

Optimizable Model Averaging for Ensembling Heterogeneous Model Outputs

arXiv:2505.21857v2 Announce Type: replace Abstract: Foundation model ensembles can improve accuracy and robustness, but full fine-tuning and parameter-space merging are often expensive or impossible when candidate models have different architectures. We propose Optimizable Model Averaging (OMA), a black-box approach that learns nonnegative model-combination weights, one scalar coefficient per model, to average the candidate models' predictive probabilities, leaving all model parameters frozen, by minimizing regularized predictive entropy. A central challenge is that minimizing predictive entropy can favour confidently incorrect models: increased confidence need not imply improved accuracy or calibration. We therefore examine regularization as a mechanism for controlling adaptation and empirically evaluate its effect on calibration, observing lower expected calibration error in selected configurations, alongside better performance-calibration trade-offs than model output averaging. As foundation models continue to advance, these approaches will enable the incorporation of future, potentially much better models to improve performance on challenging text and image classification tasks.

arXiv cs.LG68

Heterogeneity-Aware Client Sampling for Optimal and Efficient Federated Learning

arXiv:2505.11304v4 Announce Type: replace Abstract: Federated learning (FL) commonly involves clients with diverse communication and computational capabilities. Such heterogeneity can significantly distort the optimization dynamics and lead to objective inconsistency, where the global model converges to an incorrect stationary point potentially far from the pursued optimum. Despite its critical impact, the joint effect of communication and computation heterogeneity has remained largely unexplored, due to the intrinsic complexity of their interaction. In this paper, we reveal the fundamentally distinct mechanisms through which heterogeneous communication and computation drive inconsistency in FL. To the best of our knowledge, this is the first unified theoretical analysis of general heterogeneous FL, offering a principled understanding of how these two forms of heterogeneity jointly distort the optimization trajectory under arbitrary choices of local solvers. Motivated by these insights, we propose Federated Heterogeneity-Aware Client Sampling, FedACS, a universal method to eliminate all types of objective inconsistency. We theoretically prove that FedACS converges to the correct optimum at a rate of $O(1/\sqrt{R})$, even in dynamic heterogeneous environments. Extensive experiments across multiple datasets show that FedACS outperforms state-of-the-art and category-specific baselines by 4.3%-36%, while reducing communication costs by 22%-89% and computation loads by 14%-105%, respectively.

arXiv cs.LG67

EmbodiTTA: Resource-Efficient Test-Time Adaptation for Embodied Visual Systems

arXiv:2505.00986v4 Announce Type: replace Abstract: Continual Test-time adaptation (CTTA) continuously adapts the deployed model on every incoming batch of data. While achieving optimal accuracy, existing CTTA approaches present poor real-world applicability on resource-constrained edge devices, due to the substantial memory overhead and energy consumption. In this work, we first introduce a novel paradigm -- on-demand TTA -- which triggers adaptation only when a significant domain shift is detected. Then, we present OD-TTA, an on-demand TTA framework for accurate and efficient adaptation on edge devices. OD-TTA comprises three innovative techniques: 1) a lightweight domain shift detection mechanism to activate TTA only when it is needed, drastically reducing the overall computation overhead, 2) a source domain selection module that chooses an appropriate source model for adaptation, ensuring high and robust accuracy, 3) a decoupled Batch Normalization (BN) update scheme to enable memory-efficient adaptation with small batch sizes. Extensive experiments show that OD-TTA achieves comparable and even better performance while reducing the energy and computation overhead remarkably, making TTA a practical reality.

arXiv cs.LG74

解耦总方差与信噪比可改进扩散模型

arXiv:2502.08598v3 公告类型：替换 摘要：扩散模型作为一种强大的生成框架，已在众多科学领域得到广泛应用，但其固有的迭代采样过程仍是一个重大的计算瓶颈。虽然通常可以通过减少反向扩散步数来缓解这一问题，但样本质量会变得对所选的噪声调度高度敏感，即对每一步引入噪声和减弱信号的方式高度敏感。尽管已有研究改进了最初的方差保持和方差爆炸调度，但这些方法只是被动地调整总方差，无法对其进行直接且独立的控制。在本研究中，我们提出了一种新颖的总方差/信噪比解耦（TV/SNR）框架，使 TV 和 SNR 能够得到独立控制。我们的方法揭示，对于 TV 呈指数爆炸式增长的调度，在保留其底层 SNR 调度的同时采用恒定 TV 调度，可带来一致的改进。此外，我们证明，推广最优传输流匹配的 SNR 调度能够显著提高生成效率。关键在于，这些增益在不同的反向扩散求解器中均保持一致，并将搜索空间缩减为两个独立且可解释的参数，从而为分子结构生成等复杂科学应用实现快速且稳健的采样。

arXiv cs.LG68

Bandits with Anytime Knapsacks

arXiv:2501.18560v2 Announce Type: replace Abstract: We consider bandits with anytime knapsacks (BwAK), a novel version of the BwK problem where there is an \textit{anytime} cost constraint instead of a total cost budget. This problem setting introduces additional complexities as it mandates adherence to the constraint throughout the decision-making process. We propose SUAK (Strategic Under-utilization for Anytime Knapsacks), an algorithm that utilizes upper confidence bounds to identify the optimal mixture of arms while maintaining a balance between exploration and exploitation. SUAK is an adaptive algorithm that strategically utilizes the available budget in each round in the decision-making process and skips a round when it is possible to violate the anytime cost constraint. In particular, SUAK slightly under-utilizes the available cost budget to reduce the need for skipping rounds. We show that SUAK attains the same problem-dependent regret upper bound of $ O(K \log T)$ established in prior work under the simpler BwK framework, where $K$ denotes the number of arms, and $T$ denotes the time horizon. We also establish an instance-dependent lower bound showing that the $O(K\log T)$ dependence is unavoidable. Finally, we show via extensive simulations that SUAK achieves considerably higher rewards than naive adaptations of BwK algorithms that do not optimize learning under anytime cost constraints.

arXiv cs.LG78

Conditional Vendi Score：面向生成式 AI 模型和 LLM 的提示感知多样性评估

arXiv:2411.02817v3 公告类型：替换 摘要：文本提示引导的生成模型在保真度和提示对齐方面得到了广泛评估，但其生成多样化输出的能力仍缺乏充分研究。现有的多样性指标（如 Vendi 和 RKE）基于核矩阵的冯·诺依曼熵和 Rényi 熵，是为无条件模型开发的，无法区分由提示引起的变异与由模型引起的变异。为弥补这一不足，我们提出了 \textit{Conditional-Vendi} 和 \textit{Conditional-RKE}，这两种多样性度量由半正定矩阵的条件熵推导而来。这些评分能够分离出提示引导生成过程中由模型引起的多样性，其中 Conditional-RKE 具有 $O(1/\sqrt{n})$ 的收敛速率。对于 Conditional-Vendi，我们引入了一种截断谱近似方法，可提供具有可扩展性和一致性的估计。在文本生成图像、图像描述生成和 LLM 任务上的实验表明，这些条件评分能够恢复真实的多样性排序，还能引导扩散模型生成更多样化的样本。代码库见 https://github.com/mjalali/conditional-vendi。

arXiv cs.LG65

Deep End-to-End Survival Analysis with Temporal Consistency

arXiv:2410.06786v2 Announce Type: replace Abstract: In this study, we present a novel Survival Analysis algorithm designed to efficiently handle large-scale longitudinal data. Our approach draws inspiration from Reinforcement Learning principles, particularly the Deep Q-Network paradigm, extending Temporal Learning concepts to Survival Regression. A central argument in our method relies on temporal consistency, a recursive consistency condition between survival predictions made at consecutive states. Our framework introduces a delayed target network that decouples the generation of temporally consistent training targets from model updates, providing a more stable training signal while enabling mini-batch optimization and end-to-end training. Additionally, it supports arbitrarily complex architectures, enabling the modeling of intricate temporal dependencies. Through numerous experiments we provide empirical evidence demonstrating our framework's ability to exploit temporal consistency across datasets of varying sizes. Moreover, our algorithm outperforms benchmarks on datasets with long sequences, demonstrating its ability to capture long-term patterns. Finally, ablation studies show that the degree of target delay has a substantial effect on predictive performance and training variability. Across two distinct survival model families, delayed target updates consistently improve over the same-timescale baseline.

arXiv cs.LG66

Instance-Level Difficulty: A Missing Perspective in Machine Unlearning

arXiv:2410.03043v3 Announce Type: replace Abstract: Current research on deep machine unlearning primarily focuses on improving or evaluating the overall effectiveness of unlearning methods while overlooking the varying difficulty of unlearning individual training samples. As a result, the broader feasibility of machine unlearning remains under-explored. This paper studies the cruxes that make machine unlearning difficult through a thorough instance-level unlearning performance analysis over various unlearning algorithms and datasets. In particular, we summarize four factors that make unlearning a data point difficult, and we empirically show that these factors are independent of a specific unlearning algorithm but only relevant to the target model and its training data. Given these findings, we argue that machine unlearning research should pay attention to the instance-level difficulty of unlearning.

arXiv cs.LG77

通过杠杆分数采样实现具有可证明准确性的 Shapley 值估计

arXiv:2410.01917v3 公告类型：替换 摘要：Shapley 值最初源于博弈论，如今已成为可解释机器学习的核心工具，用于将模型预测归因于特定输入特征。然而，精确计算 Shapley 值的成本高昂：对于具有 $n$ 个特征的一般模型，需要进行 $O(2^n)$ 次模型评估。为解决这一问题，近似算法得到了广泛应用。其中最流行的算法之一是 Kernel SHAP，它不依赖于具体模型，且在实践中效果显著。在我们最初提交论文时，Kernel SHAP 尚无已知的强非渐近复杂度保证。为此，我们提出了 Leverage SHAP，这是对 Kernel SHAP 的一种轻量级修改，仅需 $O(n\log n)$ 次模型评估即可提供具有可证明准确性的 Shapley 值估计。我们的方法利用了 Shapley 值估计与不可知主动学习之间的联系，采用了杠杆分数采样这一强大的回归工具。除理论保证外，我们还发现，与广泛使用的 SHAP 库 [Lundberg & Lee, 2017] 中经过高度优化的 Kernel SHAP 实现相比，Leverage SHAP 可将误差降低约 $50\%$。

arXiv cs.LG74

TernaryLLM：三值化大语言模型

arXiv:2406.07177v2 公告类型：替换 摘要：大语言模型（LLMs）在自然语言处理（NLP）任务中取得了卓越的表现，但高昂的计算成本和内存需求限制了其应用。三值化是一种极端的量化方式，通过减少内存使用并支持高能效的浮点加法运算，为这一问题提供了解决方案。然而，将三值化应用于大语言模型时，权重和激活值中的离群值会带来挑战。在本研究中，我们观察到权重中的离群值呈非对称分布，且权重均值不为零，因此引入了双重可学习三值化（DLT），使缩放因子和平移量均可学习。我们还提出了离群值友好的特征知识蒸馏（OFF），以恢复极低比特量化中丢失的信息。所提出的 OFF 能够融入语义信息，且对离群值不敏感。OFF 的核心是利用余弦相似度，最大化三值化模型与浮点模型特征之间的互信息。大量实验表明，在不同大语言模型系列的标准文本生成和零样本基准测试中，我们的 TernaryLLM 均优于以往的低比特量化方法。具体而言，对于最强大的开源模型之一 LLaMA-3，我们的方法（W1.58A16）相较于此前最先进的方法（W2A16），在 C4 上的困惑度降低了 5.8，在零样本任务上的平均准确率提高了 8.2%。

arXiv cs.LG62

Linearized Wasserstein dimensionality reduction with approximation guarantees

arXiv:2302.07373v2 Announce Type: replace Abstract: We introduce LOT Wassmap, a computationally feasible algorithm to uncover low-dimensional structures in the Wasserstein space. The algorithm is motivated by the observation that many datasets are naturally interpreted as probability measures rather than points in $\mathbb{R}^n$, and that finding low-dimensional descriptions of such datasets requires manifold learning algorithms in the Wasserstein space. Most available algorithms are based on computing the pairwise Wasserstein distance matrix, which can be computationally challenging for large datasets in high dimensions. Our algorithm leverages approximation schemes such as Sinkhorn distances and linearized optimal transport to speed-up computations, and in particular, avoids computing a pairwise distance matrix. We provide guarantees on the embedding quality under such approximations, including when explicit descriptions of the probability measures are not available and one must deal with finite samples instead. Experiments demonstrate that LOT Wassmap attains correct embeddings and that the quality improves with increased sample size. We also show how LOT Wassmap significantly reduces the computational cost when compared to algorithms that depend on pairwise distance computations.

arXiv cs.LG72

Hide and Seek：隐藏概念以探寻黑盒模型的解释

arXiv:2610.13935v1 公告类型：交叉投稿 摘要：解释机器学习模型的预测是一项核心挑战，对于黑盒视觉模型而言尤其如此。基于概念的归因方法利用人类可理解的概念来解释模型。现有的基于概念的方法只能解释分类和回归模型。这些方法需要访问模型权重或大规模概念标注数据集，且大多数方法无法提供图像级归因。我们提出 Hide and Seek（HnS），这是一种新的概念级归因方法，无需访问模型权重或带有标签的概念数据集。HnS 能够解释多种视觉模型。给定一个概念和一个图像集，HnS 使用视觉语言模型检索与该概念相符的图像，在每张图像中定位该概念，遮蔽定位到的区域，并根据黑盒模型对原始图像和遮蔽后图像的输出变化来衡量归因。通过直接测量模型行为并支持局部归因，该方法能够实现忠实的归因。我们在 CelebA 和 COCO-Stuff 上验证了 HnS。与现有方法相比，HnS 的归因结果与真实归因的一致性更高。我们展示了 HnS 在多种任务中的适用性，包括图像描述生成和卫星图像分割，而现有概念归因方法均无法用于这些任务。我们在补充材料中提供了匿名代码。

arXiv cs.LG64

Constructive Bounds for the Topological Expressivity of Neural Networks

arXiv:2610.13931v1 Announce Type: cross Abstract: The expressivity of a neural network $\Phi$ describes the class of functions that a network can express. The term topological expressivity refers to a family of measures particularly suited to understand binary classification and is often formulated in terms of the Betti numbers of a decision boundary $\beta (\Phi^{-1}(0))$. Despite its popularity, this measure is not as useful for regression and other problems that do not involve a decision boundary. Instead, we propose to study topological expressivity in terms of the Betti numbers of the output of a neural network $\beta(\Phi(X))$. We formally prove that a ReLU network with architecture $(d,\sum\limits_{k=1}^{d } \beta_k(3k+2),\sum\limits_{k=1}^{d}\beta_k(k+1))$ can express spaces homotopy equivalent to wedge sums of spheres with arbitrary Betti numbers. This gives upper bounds for any arbitrary connected ($\beta_0=1$) sequence of Betti numbers, but by constructing spaces homotopy equivalent to products of $N$ spheres of dimension $d_1,...,d_N$ we achieve much better bounds $(\sum\limits_{n=1} d_n, 3\sum\limits_{n=1}\limits^N d_n + 2N, \sum\limits_{n=1}\limits^N d_n + N) $ for Betti numbers given by $\prod_n (1+x^{d_n})$. Finally we conjecture a much stronger tight bound which gives the architectural bound $(d,d+1,d+1)$ for a space homeomorphic to the sphere.

arXiv cs.LG63

Goal-Oriented Bundle Management and Flow Control for Deep-Space Communications via Reinforcement Learning

arXiv:2610.13923v1 Announce Type: cross Abstract: Deep-space communications operate under intermittent connectivity, long propagation delays, and relay storage constraints. In such networks, the Bundle Protocol enables store-carry-forward communication, requiring relay nodes to manage limited storage and scarce contact opportunities. Relay-side scheduling and dropping decisions are inherently goal-oriented: transmission resources should be allocated to bundles most valuable to downstream mission objectives. Conventional schedulers, such as FIFO, do not explicitly optimize task utility and may provide unfair service across heterogeneous planetary sources. This paper presents a closed-loop modeling and learning framework for relay-side bundle and flow management in a contact-driven deep-space network, where lunar and Martian updates are forwarded through a shared-buffer relay to Earth. We quantify destination-side information freshness using excess age $\Delta_i^e(t)=[\Delta_i(t)-d_i(t)]^+$, where $\Delta_i(t)$ denotes the Age of Information of source $i$ and $d_i(t)$ is a source-dependent delay baseline. Based on this metric, we define a goal-oriented loss that supports source-dependent nonlinear penalties, allowing prioritization according to mission relevance. We formulate relay control as a constrained semi-Markov decision process (CSMDP) and solve it using masked Lagrangian Proximal Policy Optimization (PPO). Feasibility-aware action masking restricts transmissions to bundles that can complete within the remaining contact window. DTNSim simulations indicate improved age-based utility and inter-flow fairness compared with baseline schedulers. The proposed framework jointly optimizes relay-side bundle scheduling, dropping, and flow control under contact-driven dynamics with explicit fairness considerations across heterogeneous sources.

arXiv cs.LG73

DyssomniaNN: Multi-Scale Spiking Neural Modeling of Human Sleep Disorders and Perceived Restedness

arXiv:2610.13918v1 Announce Type: cross Abstract: Clinical sleep medicine repeatedly documents profound divergences between objective sleep duration and subjective daytime impairment, notably exemplified by sleep state misperception in paradoxical insomnia and boundary instability in narcolepsy type 1. Here, we present DyssomniaNN: a multi-scale biophysical computational framework coupling Borb\'ely's two-process regulation of sleep homeostasis and circadian drive with Saper's hypothalamic reciprocal flip-flop circuit and a recurrent corticothalamic leaky integrate-and-fire spiking neural network. We formalize five clinical cohorts across an identical 72-hour photoperiod: healthy control, circadian rhythm sleep-wake disorder (delayed-phase), chronic insomnia, paradoxical insomnia, and narcolepsy type 1. The synthetic local field potential converges closely with empirical human electroencephalography spectra from the PhysioNet Sleep-EDF database across all canonical frequency bands (spectral mean absolute error of 0.33--0.35). Furthermore, we introduce the Well-Rested Index to quantify restorative efficacy. Simulations reveal that sleep duration fails to predict restorative recovery: paradoxical insomnia achieves normal total sleep time (21.15 h) yet yields low restoration (37.6%) from unsuppressed cortical beta-band intrusion, whereas hypocretin deficiency in narcolepsy produces severe nocturnal fragmentation (781 min of wake after sleep onset) and 35 sleep-onset rapid eye movement periods. DyssomniaNN provides an in silico platform for mechanistic sleep research and virtual neuropharmacology.

arXiv cs.LG75

Hitting-Time Bounds for Asynchronous Q-Learning

arXiv:2610.13901v1 Announce Type: cross Abstract: We study the last iterate of standard asynchronous Q-learning from a single trajectory of a finite Markov decision process. The trajectory follows a fixed behavior policy whose state--action chain is irreducible and may be periodic. For discount factor $\gamma$, write $H=(1-\gamma)^{-1}$, and let $\mu_{\min}$ be the minimum stationary state--action probability and $\mathfrak h_\star$ the largest expected hitting time of the reverse state--action chain. We prove that zero-initialized Q-learning with a constant step size achieves sup-norm error at most $\varepsilon$ with high probability using $\widetilde{O}(H^4/(\mu_{\min}\varepsilon^2)+\mathfrak h_\star)$ transitions, for $0<\varepsilon\le1$. The statistical term retains the quartic effective-horizon dependence of standard Q-learning, while initialization cost is additive in $\mathfrak h_\star$. For aperiodic behavior chains, a comparison with mixing time yields a transient term $\widetilde{O}(t_{\mathrm{mix}}/\mu_{\min})$. An empirical state--action count rule selects a constant step size for a subsequent training run and attains the same rate without prior knowledge of model parameters, target accuracy, or confidence level. The output is the last training iterate, with both stages included in the sample count. The proof uses two deterministic Bellman envelopes to separate statistical error from fixed-policy initialization error and adapts reverse-time hitting-cost compensation from TD learning to state--action propagation. An exploration lower bound holds for arbitrary predictable step sizes. Together with the existing statistical lower bound and its extension to pilot-selected constant steps, it yields matching mean-square rates up to logarithmic factors on the stated large-sample parameter classes.

arXiv cs.LG74

完全离散条件扩散模型的有限样本保证

arXiv:2610.13851v1 公告类型：cross 摘要：我们为从数据中学习并通过有限次去噪步骤实现的条件扩散模型建立有限样本保证。我们的方法使评分函数训练目标与采样网格相匹配，从而能够对统计估计误差和数值离散化误差进行联合分析。在条件密度模型光滑的假设下，我们推导出生成分布误差的高概率界，并量化所需的评分函数评估次数。对于任意光滑性指标 $\beta>0$，适当的网格都能使相对于原始目标分布的平均条件 Kullback--Leibler（KL）散度达到 $\widetilde O_p(n^{-2\beta/(d_x+d_y+2\beta)})$，无论是否采用提前停止。完整采样器和提前停止采样器的平均目标总变差（TV）误差均达到 $\widetilde O_p(n^{-\beta/(d_x+d_y+2\beta)})$。其中，$n$ 是样本量，$d_x,d_y$ 分别是数据维度和条件维度，误差则关于条件变量取平均。总变差界与现有条件分布估计理论中已确立的速率一致，仅相差对数因子。

arXiv cs.LG79

铭记之梦：利用世界模型持续引导通用机器人策略

arXiv:2610.13816v1 公告类型：cross 摘要：通用机器人策略通过模仿学习进行训练，其能力受示范数据的质量和覆盖范围限制。强化学习可以通过微调策略权重，或训练一个小型模块来引导冻结生成式策略的输入噪声，从而改进并扩展这些策略。当任务按顺序到来时，这两种方法都会遭遇灾难性遗忘。针对某个任务进行微调会损害先前任务的性能，而仅进行引导则只是将遗忘转移到了共享的引导模块上。我们提出了 Dream to Remember（D2R），它持续将一个冻结的通用策略引导至新任务，同时保留其早期技能。当引导模块学习新任务时，一个冻结的预训练世界模型通过想象中的轨迹演练先前任务，在没有回放缓冲区或新数据的情况下，使该模块保持其早期行为。当世界模型足够忠实时，D2R 完全通过想象来学习新任务。否则，它在环境中学习任务，同时仅利用世界模型演练先前任务。在 CALVIN 上，D2R 完全通过想象按顺序改进了八个任务，使平均成功率从 63.7% 提升至 74.8%；而在相同任务序列上使用 LoRA 微调则使其降至 11.7%。借助冻结的现成通用策略和世界模型 $\pi_0$ 与 A2World，D2R 在环境中学习未见过的 LIBERO-90 任务，同时保持 $\pi_0$ 在其经过微调的 LIBERO 套件上的性能，尽管该世界模型的准确度不足以进行策略优化。最后，在真实的 Franka 机器人上，D2R 完全通过想象按顺序改进了三个任务，同时保留了早先的改进。

arXiv cs.LG64

Diversity-Aware Ensemble of Quantum-Classical Models via Combinatorial Fusion

arXiv:2610.13808v1 Announce Type: cross Abstract: Variational quantum circuits (VQCs) are highly sensitive to circuit architecture, yet most quantum machine learning pipelines select a single best design and discard the remaining candidates. This paper addresses whether the structural diversity across quantum circuit search spaces can be used to produce stronger predictions through model fusion. We apply Combinatorial Fusion Analysis (CFA), a model fusion framework that selects and combines multiple scoring systems using rank-score function (RSF) and cognitive diversity (CD), to a pool of 29 architecturally diverse classical models and quantum circuits. Using a diversity-aware model selection method, we identify subsets of maximally diverse models and combine them with both score and rank combination. Experiments on four binary classification benchmarks covering different domains demonstrate that CFA-based hybrid classical-quantum fusion consistently outperform the best individual model on every dataset considered with improvements up to 6.03% in accuracy. When comparing with relevant literatures, the CFA framework further achieves superior performance across all four datasets.

arXiv cs.LG82

受约束的概率数值方法：用于带线性不等式约束的线性偏微分方程的 Gaussian Process 求解器

arXiv:2610.13804v1 公告类型：交叉 摘要：线性偏微分方程（PDE）的 Gaussian Process 求解器返回的是后验分布而非点估计，但它们仅基于等式信息进行条件化：PDE 残差、边界数据和测量值。许多我们关注的解还满足不等式约束，包括非负性、单调性和无套利性，而仅通过等式条件化无法强制满足这些约束。我们证明，对 Gaussian Process 先验同时施加线性等式观测和有限维不等式事件的条件化，在解空间上的 Borel 测度意义下，等价于依次进行条件化。因此，物理信息后验可以作为虚拟点处不等式更新的有效先验。组合后的分布虽为非高斯分布，但可以通过截断高斯矩进行计算，且无论 PDE 条件化还是不等式更新都无需修改。我们在 Poisson 基准测试、期权价格曲面和空气质量场上评估了该构造。在边界数据不完整的情况下，单调性约束使二维 Poisson 重建的误差降低了 $63\%$。在一条删减至仅保留 20 个报价的真实 S&P 500 期权链上，无套利约束使留出外推误差降低了 $22\%$，并将日历套利率从 $10\%$ 降至 $1\%$。在四个真实空气质量快照上，在监测点数量最少时，非负性使基于高斯矩的负值尾部代理指标降低了 $74\%$。不同数据情形下，平均准确率和约束诊断指标的变化各不相同。在等式后验下，带噪声的虚拟约束事件的概率衡量了更新前与指定边界的一致程度。该构造将可接受性从后处理步骤转变为概率 PDE 求解器中的条件化信息。

arXiv cs.LG74

不知道预言机光滑性的自适应方差缩减

arXiv:2610.13799v1 公告类型：cross 摘要：在平均光滑性条件下，方差缩减能够提高随机非凸优化的收敛速度。现有的自适应方差缩减方法专门针对平均光滑性设计，无法适应标准光滑性。本文研究如何在不知道光滑性常数或方差上界的情况下，使同一个算法适应这两种设置以及无噪声设置。给定梯度预算 $T$，我们提出一种两阶段方法。第一阶段比较两个独立梯度，并在它们的差异超过阈值时切换到归一化动量。该阶段不仅为标准光滑性和无噪声设置提供具有相应保证的候选点，还确保第二个方差缩减阶段中的期望函数值差距有界。由此得到的收敛速率在平均光滑性下为 $O(T^{-1/3})$，在标准光滑性下为 $O(T^{-1/4})$，在无噪声情况下为 $O(T^{-1/2})$。当 $T$ 未知时，我们提出一种双路径方法，在随机设置下以带对数因子的相应速率达到收敛。具体而言，我们使用方差缩减路径处理平均光滑性，并使用 AdaGrad-Norm 处理标准光滑性和无噪声设置。这两种方法都不会从初始点重新开始，也不要求每轮增加预言机调用次数。

arXiv cs.LG74

带约束的非光滑非凸随机优化的最优算法

arXiv:2610.13787v1 公告类型：cross 摘要：我们针对目标函数既不光滑也不凸、且带有凸约束集的随机优化问题，提出了新的算法。我们的算法能够返回一个 $(\delta,\varepsilon)$-Goldstein stationary point，该概念最近由 Liu et al. (2024) 针对带约束问题提出；同时，算法具有更优的复杂度，并且在多种情形下首次实现了该任务的最优收敛率。具体而言，我们设计了基于投影和无投影的算法：使用随机梯度时，可达到 $\widetilde O(\delta^{-1}\varepsilon^{-3})$ 的复杂度，我们证明该复杂度是最优的；使用零阶查询时，可达到 $\widetilde O(d\delta^{-1}\varepsilon^{-3})$ 的复杂度，该复杂度关于维度 $d$ 的依赖也是最优的，并且优于已有工作。值得注意的是，我们以下界的形式给出了 Frank-Wolfe gap，并且即使对于光滑问题也成立，这一结果本身也具有独立意义。这些结果表明，在非凸随机优化中，约束的存在和非光滑性都不会导致收敛率下降。我们的分析基于将问题归约为在线线性优化，并扩展了无约束优化中的相关技术。

arXiv cs.LG68

Geometry-Aware Multi-Objective Optimization: Norm-Constrained Oracles with Entropic Weight Updates

arXiv:2610.13773v1 Announce Type: cross Abstract: Multiple-gradient descent is typically formulated in Euclidean geometry, whereas matrix-aware optimizers use directions induced by other norms. We analyze a single-loop multi-objective method combining a norm-constrained linear minimization oracle with entropic updates of task weights. For any finite-dimensional norm pair, we establish an $O(T^{-1/2})$ ergodic stationarity rate in the deterministic setting, deriving the required trajectory gradient bound from smoothness and lower-boundedness in the unregularized case. With stochastic gradients and a bounded-gradient assumption, one persistent blended-gradient momentum buffer yields an $O(T^{-1/4})$ rate whose leading term has no explicit dependence on the number of tasks. We also establish static regret for the inner weighting objective on arbitrary bounded gradient sequences and, with entropy regularization and a weight floor, iteration-wise inner-gap control after burn-in while retaining ergodic stationarity, conditional on a trajectory gradient bound. The optimal blended gradient can be nonunique in spectral and sign geometries, motivating a selection-independent inner gap. Our analysis extends the spectral construction studied by concurrent MOON to general norms. Experiments compare the method with existing multi-task optimizers and examine its oracle, weighting, and momentum choices.

arXiv cs.LG81

从像素到隐私敏感特征：一个用于重构界限的信息论框架

arXiv:2610.13768v1 公告类型：cross 摘要：使用 DP-SGD 训练的模型会泄露哪些信息，使攻击者能够从中恢复数据？其 $(\varepsilon,\delta)$ 保证无法直接回答这一问题。我们推导出一个可在训练前计算的失真界限，即使攻击者能够观察训练期间的参数轨迹、知晓其他所有训练记录以及生成这些训练记录的总体分布，该界限仍然适用。分析者可以选择衡量误差所使用的表示形式，从而使该界限能够针对特定应用中对隐私敏感的重构方面。我们的信息论框架将两个贡献因素分离开来：训练所泄露的关于某条记录的信息，通过条件互信息度量；以及数据分布中原本就存在的不确定性，通过源熵度量。率失真理论将二者结合起来，为每一种重构攻击的期望误差给出一个下界。对于 DP-SGD，信息泄露界限仅取决于抽样率和噪声乘数，而源熵则可根据一个样本以有限样本置信度给出下界。正交分块构造使所得界限在高维图像上仍具有信息量。我们在像素空间和 LPIPS 特征空间中实例化了该框架，据我们所知，这是首个直接用 LPIPS 度量重构失真的下界。在一系列图像数据集上，这些界限表明，在启发式推荐的隐私预算范围内及更大范围内，都存在不可避免且非平凡的期望重构误差。实验将这些界限与强大的重构攻击进行比较，并确定了剩余松弛的主要来源。

arXiv cs.LG83

Shortcut Learning in Geospatial Foundation Model Embeddings for Cross-Disaster Damage Assessment

arXiv:2610.13766v1 Announce Type: cross Abstract: Geospatial foundation models can support disaster damage assessment, but high accuracy on familiar disasters may hide shortcut learning: prediction from event or scene characteristics rather than evidence that transfers to a new disaster. We study embeddings from TerraMind, Prithvi-EO-2.0, DOFA, and DINOv3 on the multi-disaster datasets xView2 and BRIGHT. Holding out an entire disaster, instead of randomly splitting image tiles, reduces mean balanced accuracy from 0.807 to 0.638 on xView2 and from 0.818 to 0.568 on BRIGHT. Four sets of tests examine what information supports prediction: event and acquisition context, before/after image inputs, building count, and correct image pairing. Pre-event images remain predictive, and a classifier given only the annotated number of buildings reaches balanced accuracy of 0.629 and 0.653. Combining before/after features performs best on average on xView2, but does not improve over pre-only features on BRIGHT. Shuffling image pairs within the same disaster and damage class causes modest losses. Motivated by these findings, we test whether feature transformations learned only from training disasters improve generalization to unseen disasters. These transformations improve performance, with supervised contrastive learning performing best on xView2 and PCA on BRIGHT. Our results highlight the importance of evaluation on unseen domains for assessing model robustness and show that simple feature-level transformations can improve generalization. Code and experiments can be found at https://github.com/alishibli97/gfm-shortcut-learning.