arXiv:2610.13692v1 公告类型：新论文 摘要：标准策略梯度（PG）算法依赖蒙特卡洛更新，因此存在估计方差较高的问题。尽管离策略方差缩减能够缓解这一问题，但目前要确立最优样本复杂度，仍需要重要性权重一致有界等限制性假设，或不切实际的二阶运算。在本研究中，我们提出 Loopless Variance-Reduced PG（LVR-PG），这是一个统一的理论框架，对一类将递归更新与防御性重要性采样相结合的一阶估计器进行了抽象，其中包括 PAGE-PG 和 STORM-PG 等已有估计器。我们证明，对于一般参数化策略，在弱梯度支配条件下，这类算法能够以 $\tilde{\mathcal{O}}(\epsilon^{-2})$ 的样本复杂度实现最后一次迭代的全局收敛。我们仅使用一阶更新，在极少的假设下且不使用梯度预条件器，便确立了这一保证。通过将当前最先进的下界扩展至连续参数化策略，我们确立了 $\Omega(\epsilon^{-2})$ 的样本复杂度下限，证实我们的框架在忽略对数因子和时间跨度项后具有极小极大最优性。
arXiv:2610.13690v1 公告类型：新论文 摘要：扩散模型通常学习单个目标分布的生成动力学，或学习由训练期间固定的条件变量索引的一族分布的生成动力学。这限制了它们仅凭少量示例适应此前未见分布的能力。我们提出了一种面向扩散模型的上下文解算子学习框架，其中每个目标分布都被视为从元分布中抽取的一个任务，并在推理时由一个经验上下文集指定。我们的方法学习一个上下文得分算子，将上下文样本、含噪查询点和扩散时间映射为任务相关的得分，从而无需重新训练或更新参数，即可在未见任务上进行反向扩散采样。一个关键发现是，任务相关得分具有高斯核条件均值表示。其经验 Nadaraya--Watson 近似可以由我们构建的 Transformer 类中的模型精确表示，为所提出的查询相关注意力架构提供了理论依据。该算子通过跨任务分布的摊销去噪得分匹配进行训练。我们建立了泛化误差界，量化了上下文大小、训练任务数量、任务内样本数量、模型复杂度以及内在维度和环境维度的影响。在数值实验中，我们在一系列科学与视觉数据集上评估了该方法，包括 Navier-Stokes、ERA5 气候数据、QuickDraw 草图和 Food-101。结果表明，所提出的方法在相对简单的任务族上表现良好，但其性能会随着任务复杂度的增加而下降，从而揭示了扩散模型上下文学习的潜力与局限。
arXiv:2610.13689v1 公告类型：新提交 摘要：学习局部扰动如何影响网络，对于识别哪些组件驱动或吸收系统响应至关重要。我们将这一问题表述为以种子节点为条件的响应排序：给定一个种子节点上的扰动，按响应强度对所有其他节点进行排序。当模型仅获得相互作用拓扑，而支配扰动传播的机制必须从数据中学习时，这项任务具有挑战性。我们提出 VQIL（Visible Quantum Interference Learning），一种用于以种子节点为条件进行排序的可训练 Szegedy 量子行走。边权重、连接相位以及各节点的相位旋转通过反向传播进行端到端优化，同时行走算符在整个训练过程中始终保持酉性。节点得分由量子行走振幅经过几何折扣的相干叠加得到，使量子干涉能够在最终的范数平方读出之前发生。由于每个学习参数都关联到一条具名边或一个具名节点，训练后的模型能够就各个图组件在传播机制中的作用提供可直接检验的假设。在六个输电网络的交流潮流扰动任务中，即使未向模型提供电气参数，VQIL 仍将 Spearman 秩相关系数从未训练行走的 0.147 提升至 0.831，并优于采用相同方式训练的随机游走模型。在以 Perturb-seq 为监督的五个蛋白质相互作用网络的基因扰动任务中，VQIL 将秩相关系数从 0.171 提升至 0.620，超越了表现最强的已训练经典基线模型和神经网络基线模型。这些结果表明，可训练的量子行走干涉为学习图上以种子节点为条件的响应排序提供了一种可扩展且可解释的归纳偏置。
arXiv:2610.13676v1 公告类型：新论文 摘要：时间序列基础模型（TSFMs）越来越多地支持协变量，其中日历信息是实践中最常见的类别协变量，因为每个时间点都有相应的日历信息。然而，这些特征的编码方式大多被视为实现细节，而非研究对象，尽管编码是预训练模型在推理时仍可调整的少数设计选择之一。本研究表明，信息传入模型的方式会显著影响 TSFMs 的预测准确性。因此，我们在来自多个领域的九个数据集上，对四种最先进的 TSFMs 进行零样本滚动评估，系统比较了七种日历编码，包括三种常规的、与目标无关的表示方式（序数编码、独热编码和周期编码）以及四种目标编码。这九个数据集受到周末和节假日等相关社会活动的影响，因此日历信息对预测尤为重要。我们发现，编码选择会影响所有模型的预测准确性；在同一模型内，不同编码在非节假日的 MASE 差异最高可达 9.9%，在节假日则可达 83%。编码的有效性及其被利用的程度在不同模型间存在显著差异。性能提升主要集中在节假日前后，此时日历信息提供了目标变量历史数据中无法获取的信号；同时，某些模型与编码的组合也能改善非节假日的预测。基于 SHAP 的归因分析确定了协变量相对于自回归部分的贡献，并解释了某些模型与编码的组合为何会导致性能下降。我们的研究结果表明，当前针对利用协变量的 TSFMs 的基准测试，可能会因采用固定编码策略而扭曲模型比较结果。
arXiv:2610.13660v1 Announce Type: new Abstract: The Abstraction and Reasoning Corpus (ARC) is increasingly used to evaluate whether neural networks can infer task-specific transformations from few demonstrations. Recent ARC models differ in architecture, but also in broader training and evaluation protocols, including pretraining data, data augmentation, test-time setup, and optimization budgets, making conclusions about architecture benefits debatable. In this work, we study how the performance of the vision-inspired VARC and recursion-based TRM models is affected by pretraining and test-time training (TTT) budgets under a common inductive protocol on ARC-AGI-1. We cross five pretraining budgets with four TTT budgets for each test task. Each budget specifies a task-variant count and total sample presentations, defining the controlled training budgets. We observe that increasing TTT sample presentations improves the standard ARC pass@2 metric in all settings for both models. Increasing the TTT task-variant count consistently helps only TRM, whereas VARC often performs better with fewer variants. VARC also remains competitive without pretraining, while TRM benefits much more from it. These results show that pretraining and TTT budgets affect ARC evaluation and motivate reporting at comparable budgets.
arXiv:2610.13654v1 公告类型：新论文 摘要：表格基础模型（TFMs）因其强大的预测性能而日益受到欢迎。许多此类模型还提供后验预测分布（PPDs），但这些分布的可靠性仍缺乏充分研究。我们从点预测、分布质量、边际与子群校准以及区间效率等方面评估可靠性。随后，我们推出 TabReliBench，这是一个用于上述比较和数据集层面分析的大规模分类与回归基准。我们发现，TFMs 在 TabArena 和本基准上的排名有所不同，在不同可靠性指标上的排名也会发生变化。回归 TFMs 通常能够达到接近名义水平的边际覆盖率，同时保持较窄的区间，但这种良好的边际表现并不意味着条件覆盖率也能得到保证。数据集层面的分析表明，分类难度较高往往伴随着较差的校准，而回归预测性能并不是衡量校准情况的可靠指标。集成方法能够提高预测性能，但并不总能改善 PPDs 的校准；TFMs 从保形后处理中获得的收益也很有限，这种处理无法可靠地消除子群间的差距。因此，要可靠地选择 TFM，就需要超越综合排名，开展多维度评估。代码可在 https://github.com/Tianqi-Zhao/TabReliBench 获取。
arXiv:2610.13644v1 公告类型：新论文 摘要：从表格数据集中选择特征（列）的子集，是现实应用中普遍存在的挑战。行业专家希望最大限度地降低模型训练成本，气候科学家需要一组可解释的因素子集来理解极端天气事件，临床医生则因资金和时间限制而需要在诊断检查项目之间作出选择。然而，当不同领域的实践者查阅科学文献以确定采用哪种特征选择方法时，所面对的基准评测体系缺乏代表性。尽管相关研究十分广泛，现有特征选择基准所能提供的指导仍然有限，因为它们依赖过时的模型、临时制定的评估流程，或无法反映现实问题的人工生成数据。为弥合科学基准评测与现实应用之间的差距，我们推出了现代特征选择基准 SelectArena。我们采用标准化评估流程和最先进的表格（基础）模型，在 71 个经人工精心整理、涵盖分类与回归问题的真实世界表格数据集上，对 15 种广泛使用的特征选择方法进行了比较。结果揭示了预测性能、运行时间、稳定性和有效性之间的明确权衡。在同时考虑预测性能和运行时间时，基于树的方法 CART 和 RFImportance 以及 F-Test 均达到帕累托最优。F-Test、基于信息论的方法和基于距离的方法在稳定性与有效性方面表现领先，其中 F-Test 在我们采用的各项指标上展现出均衡表现。基于搜索的方法速度慢、稳定性差，且性能较低。SelectArena 将这些发现转化为实践指导，并作为一个可扩展的平台，为未来的特征选择研究提供支持。
arXiv:2610.13635v1 Announce Type: new Abstract: Hypergraph Neural Networks (HNNs), designed to model higher-order relations beyond pairwise interactions, typically rely either on fixed structural weights or on normalised attention. Fixed weights can propagate misleading information in heterophilic settings, while normalisation makes attention coefficients competitive, in the sense that, in a message-passing framework, increasing the weight of one sender must reduce the weight of others in the same aggregation set. Our main contribution is the introduction of PANDA (Prior-guided AttentioNal Dual-path Architecture), a plug-in for two-stage message passing HNNs applicable to both undirected and directed settings. The main path interpolates between learned receiver-normalised attention and structural incidence priors, while a non-competitive (not normalised) auxiliary path lets each sender contribute its own transformed information. We also show that, with fixed structural coefficients, PANDA recovers a certain complex-valued convolution operator adopted in previous works. Across four HNNs spanning spatial and spectral methods in both directed and undirected settings, plugging PANDA in yields an average improvement of 5.18 percentage points across eight real-world benchmark datasets. The code is available at https://github.com/jainris/panda.
arXiv:2610.13631v1 Announce Type: new Abstract: Fault diagnosis in asset-intensive industries requires multiclass models trained on noisy, sensitive data collected across geographically distributed client sites. Federated learning enables collaborative training without centralizing data, but most approaches do not provide formal privacy guarantees for individual records stored at client sites and can degrade under label noise. To address these limitations, we develop a differentially private, distributionally robust framework for federated multiclass classification that provides sample-level privacy for records stored at each client. We build on a federated Wasserstein distributionally robust multiclass objective with the Crammer-Singer loss, solved using an Alternating Direction Method of Multipliers (ADMM)-based algorithm. To protect the released client updates, our method injects calibrated Gaussian noise into the proximal linearized client updates of both the local model and the local distributionally robust variable before aggregation at the central server, and we quantify the cumulative privacy loss using the moments accountant. We provide per-round sample-level $(\epsilon,\delta)$-differential privacy guarantees and an overall composed privacy guarantee across all training rounds, together with convergence results and an explicit privacy-utility tradeoff. Experiments on simulated industrial fault data and UCI benchmark datasets show that our method outperforms differentially private federated empirical risk minimization and stochastic gradient descent baselines in most tested settings, with the largest gains under strong privacy-induced noise.
arXiv:2610.13599v1 公告类型：新论文 摘要：在超声心动图等多视图医学影像检查中，诊断信息分布于不同视图中，每个视图捕捉心脏结构和功能的不同方面。在临床实践中，医生首先通过定量测量解读这些视图，再依据测量结果作出具体诊断。然而，现有的超声心动图表征学习方法并未将测量显式建模为视图与诊断之间的中间表征。因此，各项测量未能以可供下游诊断任务选择性使用的形式得到显式表示，从而妨碍了准确诊断。受这种分层临床工作流程的启发，我们提出了 EchoTab，这是一种以测量为基础的表征学习框架，使用测量特定的词元来表示多视图超声心动图检查，并将其作为超声心动图视图与诊断之间的中间表征。每个测量词元选择性地关注与其估计相关的视图，并通过表格重建进行训练，以预测对应的测量值。由此得到的词元构成一种因子化的超声心动图表征，编码了具有临床意义的结构和功能特征，可供下游诊断选择性使用。与常用的对比学习方法不同，EchoTab 不需要负样本或大批量训练。在两个数据集上的实验表明，EchoTab 能够学习有效的、以测量为基础的表征，从而提升多视图超声心动图的诊断分类性能；其学习到的注意力也与心脏科医生所采用的既定测量与视图对应关系一致。我们的代码可在 https://github.com/michelleespranita/EchoTab 获取。
arXiv:2610.13574v1 Announce Type: new Abstract: Feature selection is a central problem in high-dimensional financial regression, where large numbers of engineered variables and strong correlations among predictors reduce model stability and interpretability. $L_1$ regularization (Lasso) is widely used for sparse regression, but its shrinkage can be restrictive when informative signals are spread across correlated variables. Exact sparse regression avoids this shrinkage but requires solving a combinatorial optimization problem that is computationally difficult and is typically fit to minimize in-sample rather than out-of-sample error. This paper presents a feature selection framework based on Monte Carlo Tree Search with Generalized Nested Rollout Policy Adaptation (GNRPA). Feature selection is formulated as a sequential decision process, and candidate subsets are scored by their out-of-sample validation performance rather than training error. The method also supports a lexicographic objective accounting for subset size and multicollinearity. We study this approach on a contemporaneous explanatory regression task: explaining the weekly return of the NASDAQ-100 index from 1,072 engineered features spanning market, macroeconomic, and factor data. This isolates the value of subset selection under multicollinearity from the separate problem of predictive forecasting. On this task, the proposed method identifies sparse feature subsets with higher out-of-sample explanatory power than an optimized Lasso baseline, reaching an out-of-sample $R^2$ of 0.6961 versus 0.6313 for Lasso. These results suggest that Monte Carlo Tree Search is a practical alternative to convex regularization for sparse explanatory feature selection in financial regression, with potential value for factor attribution and risk decomposition where transparency matters as much as out-of-sample fit.
arXiv:2610.13568v1 Announce Type: new Abstract: We present Rationally Linearized Multilayer Perceptrons(RaLiMPs), a new approach to efficiently linearizing MLPs using low-degree rational approximations of standard activation functions. Inspired by polynomial-based MLP linearization, we show that rational approximations can provide accurate representations at substantially lower degree. RaLiMPs exploit these approximations to construct kernel representations with narrow bottlenecks, enabling efficient composition across MLP layers. We provide theoretical guarantees for the resulting approximation and evaluate RaLiMPs across applications ranging from implicit 3D models to large language models. Our experiments demonstrate favorable accuracy--efficiency tradeoffs, with RaLiMPs achieving competitive or improved accuracy while substantially reducing inference cost.
arXiv:2610.13557v1 Announce Type: new Abstract: Anticipation of vessel trajectories is an essential task when navigating at sea. This applies to both human navigators and autonomous surface vessels. Trajectory forecasting enables sub-routines such as collision avoidance to assess the situation and react accordingly in time. This paper investigates Transformer and Performer models for trajectory prediction of surface vessels in confined waters. Using widely available position data from the Automatic Identification System (AIS), candidate models are trained and evaluated. Historical information on how other vessels have moved in the local area surrounding each agent is conveyed using positional heat maps. The heat maps are generated using Kernel Density Estimation (KDE) and encoded with a Convolutional Neural Network (CNN) before entering the Transformer models. The models are trained with quantile loss to efficiently obtain a model-aware prediction interval, as this is crucial for timely collision avoidance. The proposed model architectures are shown to greatly outperform previous work while the inclusion of local heat maps enables the prediction of vessel movement that is not otherwise indicated from the previous positions.
arXiv:2610.13524v1 公告类型：新论文 摘要：大语言模型（LLMs）预训练过程中损失间歇性出现的“尖峰”，在理论和实证层面仍是一种尚未解明的现象。本文表明，输入服从重尾幂律分布的简单二次模型可以复现这些尖峰。我们提出了一套理论框架和仿真算法，利用此类二次模型对深度神经网络（DNNs）的逐层训练动态进行建模。在一项针对 Adam 超参数配置的案例研究中，该配置将 $\beta_2$ 设为接近 $1$ 的值，已知会在 Transformer 语言模型中引发尖峰；我们的仿真成功地从给定检查点出发，预测出了预示尖峰行为的隆起。此外，我们根据观测指标推导出一种合适的 $\beta_2$ 调度方案，并通过实证验证了其性能改进效果。
arXiv:2610.13508v1 Announce Type: new Abstract: Federated learning (FL) seeks to optimize a shared global model from decentralized client data. While effective for average performance, this process can produce unfair performance across clients, particularly under statistically heterogeneous data distributions. Existing fairness-aware and multi-objective FL methods address this through client re-weighting, gradient correction, or common-descent formulations, but often change the global optimization direction or treat client objectives symmetrically. We propose FedPCD, a priority-constrained federated optimization framework that separates the global optimization objective from client protection. FedPCD preserves the conventional federated aggregate as the primary objective, while allowing only the minimum correction necessary to meet client performance constraints. Across heterogeneous FL benchmarks, FedPCD consistently outperforms state-of-the-art baselines, achieving the highest global accuracy across all configurations and leading worst-10% client accuracy in nearly all. FedPCD further preserves competitive trade-offs between global accuracy and client performance variability. Beyond this trade-off, FedPCD demonstrates faster empirical convergence, reaching matching or superior performance in substantially fewer communication rounds.
arXiv:2610.13504v1 公告类型：新论文 摘要：生成式推荐系统越来越多地使用强化学习（RL）来优化可能在决策后数百乃至数千次交互后才出现的结果。标准 RL 根据时间邻近性而非明确的相关性传播奖励，削弱了分配给时间久远但重要的动作的信号，成为样本效率低下的一个来源。我们提出 Explanatory World Models（EWM），这是一种归因框架，利用解释识别可能促成已实现结果的早期事件。我们使用前沿语言模型实现 EWM，并通过一种守恒的层次化算子将其解释用作学习信号；该算子递归地重新分配奖励，同时保持未折扣的回合回报不变。为了探究什么样的解释才有用，我们将受 Halpern 的实际因果关系、Lombrozo 的解释简洁性以及 Deutsch 的难以变动的解释启发的三种互补变体转化为可操作的方法。我们在两个受控的长时程 RL 基准、一个公开的序列推荐基准以及一个工业规模的生成式推荐系统上评估 EWM。在 Crafter 和 WordCraft 上，EWM 显著优于 PPO 和 LaRe，且在奖励延迟最严重时取得最大提升。在 KuaiRand 上，EWM 在离线评估中优于 SFT 和 LaRe，在在线评估中优于 A2C。最后，在对工业级生成式推荐系统的离线评估中，EWM 显著提高了召回率，并提升了行为代理指标。这些结果表明，解释能够为从延迟结果中学习提供结构化且可复用的信用归因。
arXiv:2610.13435v1 公告类型：新投稿 摘要：可穿戴设备和临床设备正以日益庞大的规模记录光电容积描记（PPG）信号，越来越多的 PPG 基础模型（FM）也相继被开发出来，用于计算可供各种下游任务复用的表征。然而，这些模型难以进行比较：不同评估采用的数据集、任务定义、预处理方法、下游预测头、超参数调优方法和基线各不相同。我们提出 PPG-Bench，这是一个统一的开源基准，在九个开放访问数据集和 26 项任务上评估五个公开的 PPG 基础模型，任务涵盖心血管状态估计、活动、睡眠、筛查和身份识别。我们在适应能力逐步增强的四种下游设置下评估模型（线性探测、随机森林、冻结模型的 MLP 探测和端到端微调），并采用针对各模型的超参数优化及受试者互不重叠的交叉验证。我们还设置了三类对照：随机/常数基线、确定性的脉率变异性（PRV）特征集，以及通用时间序列基础模型（MOMENT）。在全部 68 种任务与设置组合中，35.3% 的组合未显示基础模型相较于随机/常数基线具有实质性优势，而在 83.8% 的组合中，没有任何 PPG 专用基础模型能同时优于 PRV 和 MOMENT。我们还发现，增加下游预测头的容量通常可以提升性能，但如果模型的表现处于随机猜测或常数预测水平，则可能导致性能显著下降。因此，目前的 PPG 基础模型尚不能宣称能够提供具有泛化性和可迁移性的 PPG 信号表征，该领域未来的评估应纳入与简单、非学习型对照方法的比较检验。该基准的代码及评估更多模型的说明见：https://github.com/mannas-ai/ppg-bench。
arXiv:2610.13411v1 Announce Type: new Abstract: Traditional system identification typically involves two steps: initially, training data is processed to estimate a model of the system of interest; then, the model is used to predict outputs for new input queries. In contrast, recent in-context learning approaches learn a meta model that maps directly from training data and query inputs to outputs in a zero-shot fashion, bypassing explicit model estimation. This approach leverages a meta model trained offline, on large sets of synthetically generated data, to emulate the behavior of a system identification algorithm. In this work, we advance this meta-learning framework for system identification in three directions. First, we develop a unified probabilistic meta-identification architecture that combines a Transformer-based encoder-decoder with tailored mechanisms for handling long training sequences, enabling calibrated predictions over long horizons. Second, we show how a pre-trained meta model can act as an uncertainty-aware synthetic data generator for classical system identification, providing task-specific data that regularize parametric models when real data are scarce. Third, we systematically evaluate the framework on standard nonlinear system identification benchmarks and on a challenging out-of-distribution scenario, demonstrating competitive accuracy with state-of-the-art methods and strong generalization, including cases where input-only adaptation is sufficient to handle significant distribution shifts.
arXiv:2610.13407v1 Announce Type: new Abstract: Exhaustive optimistic scoring can make action selection costly in linear bandits with a large fixed set of $K$ actions, requiring $\mathcal{O}(K)$ score evaluations per round. We introduce Persistent Candidate LinUCB (\textsc{PC-LinUCB}), which maintains $k$ active actions and samples $q$ challengers uniformly from the remaining $K-k$ actions at each round. By scoring only these $k+q$ candidates, \textsc{PC-LinUCB} reduces the number of score evaluations per round to $\mathcal{O}(k+q)$. We establish problem-independent and problem-dependent high-probability regret bounds consisting of a standard linear-bandit learning term and an additive discovery cost of order $\mathcal{O}\left((K-k)\log(K-k)/q\right)$ at fixed confidence. Notably, this additional cost does not grow with the horizon. For fixed gaps and sufficiently long horizons, we establish an $\Omega\left((K-k)/q\right)$ regret lower bound under uniform challenger sampling, showing that partial candidate access itself imposes an unavoidable regret cost. Experiments show that \textsc{PC-LinUCB} substantially accelerates LinUCB while maintaining comparable regret, and that the persistent candidate mechanism can extend beyond LinUCB.
arXiv:2610.13404v1 Announce Type: new Abstract: Linear models can predict well in complex domains even when underlying mechanisms are nonlinear. We argue that this can occur because the complexity of the world exceeds the complexity that can be learned reliably from finite, partially observed data. Flexible models can exploit curvature, thresholds, and interactions that fit one hospital, period, population, policy setting, or measurement process but fail elsewhere. When this instability lies mainly in the nonlinear component, omitting it can give linear models lower prediction error in a new environment. We connect this claim to work on simple linear rules, bias and variance, regularization, best linear prediction, measurement error, distribution shift, invariance, Rashomon sets, and underspecification. Proposition 1 gives a simple squared-error case in which a nonlinear rule learned in the training environment raises target error because it transfers too weakly; finite-sample estimation error can strengthen this result. Partial observation alone is not enough: the link between observed variables and unobserved factors must vary across environments. Stable nonlinear structure favors flexible models, and linear models remain vulnerable to linear spurious relationships. Model complexity should therefore be chosen by cross-environment stability as well as training-data accuracy.
arXiv:2610.13390v1 公告类型：新论文 摘要：地理空间机器学习通常依赖针对特定任务的特征工程，对大量社会人口、商业和环境变量进行处理。地理空间基础模型提供可复用的地点表示，但主流方法通常编码人口流动和搜索活动等动态信号，或来自卫星影像的视觉信号。很少有模型将地点的静态结构组成（谁在那里生活和工作、哪些企业在那里经营，以及它们面临哪些风险）表示为精细空间分辨率下的可迁移嵌入。我们提出 PlacePulse，这是一种结构性地理空间基础模型，将 572 个属性通过 H3 邻域聚合扩展为 2,288 个输入，再压缩为 256 维单位范数嵌入，覆盖美国 1,055,537 个 H3 分辨率为 7 的网格单元。我们还提出了一套可复用的双轴基准，在多个目标和空间支撑单元上，将表示的内在质量与下游预测效用区分开来。在内在质量方面，这些嵌入重构标准化输入的平均 $R^2 = 0.853$，利用了 256 个维度中的 251 个近似去相关维度，并呈现出显著的空间组织性：第一主成分的 Moran's $I = 0.93$，且自相关性在超出输入半径的范围内依然存在。在外在效用方面，在来自 12 个外部来源、涵盖四种空间支撑单元的 74 个留出目标上，PlacePulse 在与训练输入无关的目标上的表现优于维度相同的特征选择基线，并可与 Google 的 Population Dynamics Foundation Model 相媲美；它在健康、经济、选举、经济流动性和社会脆弱性结果方面表现领先，其相对优势因领域而异。这些发现表明，结构组成是地理空间基础模型中一种独特且互补的模态，而不只是在行为或影像数据不可用时的备选方案。
arXiv:2610.13382v1 公告类型：新提交 摘要：核岭回归（KRR）是一种具有坚实理论基础的非参数学习算法，也常被用于研究深度学习的性质。然而，精确的 KRR 形式所需的时间和内存开销使其无法适用于大型数据集，这推动了近似求解器的发展。这些算法依赖于：（i）迭代方法，以避免直接求解的计算负担；（ii）草图化或分批处理，以缩减问题规模；以及（iii）预条件，以加速收敛。虽然预条件对于快速收敛至关重要，但它也会带来内存开销。减少模型的自由参数数量可以控制这部分开销，却可能牺牲模型的表达能力，进而降低准确性。为应对这一挑战，我们提出了 BLOKS，这是一种基于共轭梯度法的求解器，用于经过 Nyström 草图化的 KRR，其核心是一种将分块 Jacobi 近似与草图化相结合的新型预条件器。BLOKS 预条件器的内存开销既不依赖于数据集大小，也不依赖于参数数量，因此能够扩展至大型模型。我们提供了一种内存高效、支持 GPU 加速的实现，并表明 BLOKS 在保持良好的时间和内存复杂度的同时，能够取得出色的准确性。事实上，在用于核求解器基准测试的标准大规模数据集上，BLOKS 的表现与最先进的方法相当或更优。
arXiv:2610.13369v1 公告类型：新论文 摘要：无分类器引导（CFG）是改善扩散模型条件生成效果的标准技术，但其激活调度通常依靠启发式方法选定。在实践中，大多数现有的基于 CFG 的方法在整个推理过程中依赖预先定义的激活策略，而近期的动态变体往往依赖人工设计的调度方案，缺乏关于条件信息何时最有用的严谨理论解释。为理解条件输入与得分函数之间的信息相关性如何随生成过程演变，我们从信息论视角探讨应在何时激活引导。在本研究中，我们将扩散轨迹建模为随时间变化的信息瓶颈，并表明条件信息的有效性在整个采样过程中并不均匀，而是在由 Fisher 信息平衡条件刻画的临界阶段之后显著增强。受这一发现启发，我们提出了一种用于推理的免训练动态 CFG 策略。我们的方法并非预先指定固定调度，而是根据条件得分与无条件得分的统计量构建随机过程，并在估计的引导增益超过预设阈值时激活引导。由此得到的自适应激活规则使条件注入与底层生成动力学相匹配，同时也将基于区间的引导涵盖为一种极限特例。在多种架构和任务上开展的大量实验表明，我们的方法有效、稳健且具有广泛适用性。除实证改进外，我们的框架还为动态引导方法提供了解释，并为条件扩散模型的推理时控制奠定了严谨的理论基础。
arXiv:2610.13350v1 公告类型：新提交 摘要：广泛复用的电子健康记录预处理流程可能隐含一些假设，涉及信息何时可用、标签代表哪一次临床诊疗过程，以及数据缺失意味着什么。这些假设很少得到系统性审查，并可能随着预处理流程的复用或调整而传播到后续研究中。我们以 MIMIC-IV 为案例，识别并量化了三种互补的预处理对齐形式，即时间对齐、语义对齐和人群对齐的违反情况，它们分别表现为时间泄漏、特征与任务不匹配以及有偏插补。我们展示了时间戳语义、诊疗过程定义、终点构建、模态选择和缺失值处理中的歧义如何改变模型可用的信息，并影响对其预测结果的解释。通过对 MIMIC-IV 数据的分析和实证案例研究，我们证明，这类选择虽然会改变模型所依赖的信号，却可能在总体预测性能中基本不显现。我们据此提出了切实可行的建议，以便对 MIMIC-IV 及相关电子健康记录数据集开展时间上有效、与临床相契合且可复现的预处理。我们的实验代码（包括所提出的预处理方法）可在以下地址获取：https://github.com/m2l-lab/mimic-iv-preprocess。
arXiv:2610.13341v1 Announce Type: new Abstract: Mamba-3 introduces input-dependent rotations into recurrent state updates, but their contribution beyond state retention remains unclear. We interpret these ro- tations as underdamped second-order dynamics, providing a framework for in- vestigating the role of oscillation in sequence modeling. Controlled comparisons show that underdamped dynamics outperform non-oscillatory damping regimes on state-tracking tasks, while language-model pretraining demonstrates benefits for long-context prediction and length generalization. Interventions in pretrained models further establish the importance of learned phase, and state analysis sug- gests that rotation regulates how retained information combines with incoming up- dates. Taken together, these findings identify underdamped dynamics as a useful mechanism for recurrent computation and suggest that oscillation supports long- context processing by regulating how information accumulates.
arXiv:2610.13335v1 Announce Type: new Abstract: Tool-using agents increasingly require a common substrate for building, training, evaluating, distilling, and maintaining agentic systems, yet existing tooling typically addresses these stages in isolation. We present AgentTune, a source-available Python library that provides a unified framework for developing and optimizing tool-using agents. At its core, AgentTune introduces a two-tier EventLog representation that preserves agent trajectories while supporting consistent projections for training, evaluation, and auditing. Building on this substrate, the library provides a shared agent runtime, tool-calling reinforcement learning, agentic distillation, retrieval-augmented generation, decision workflows, evaluation utilities, and a designed path toward self-healing agents. These components are exposed through composable interfaces so that agent strategies, training methods, evaluation procedures, and recovery mechanisms can be combined without requiring separate representations or pipelines. We demonstrate the framework across agentic tasks including legal contract search and domain-specific applications, illustrating its use for trajectory-based optimization, tool-use evaluation, and cross-task agent development. AgentTune is released as a source-available library at https://github.com/Lexsi-Labs/AgentTune
arXiv:2610.13334v1 Announce Type: new Abstract: Motivated by the success of large language models (LLMs) with broad generalizability and robustness to noisy or unreliable pre-training data, we explore in-context prediction of PDE solution fields using approximate pre-training data. In addition, inspired by the posterior predictive mean-based inference mechanism of the in-context learning in prior-data fitted networks (PFNs), we propose PDE-PFN, a prior-data fitted neural solver that approximates the posterior predictive mean of PDE solutions via in-context learning. PDE-PFN builds on a PFN architecture with self- and cross-attention mechanisms of Transformer and is pre-trained on noisy approximate solutions generated by physics-informed neural networks, serving as diverse but not necessarily exact priors. Through experiments on a range of two-dimensional PDEs, we demonstrate that PDE-PFN achieves empirical generalization across heterogeneous equations, robustness under noisy priors, and zero-gradient-update in-context inference capability. PDE-PFN achieves competitive or improved predictive accuracy relative to the evaluated baselines while supporting flexible context and query representations.
arXiv:2610.13328v1 公告类型：新提交 摘要：为混合专家（MoE）模型提供推理服务时，需要为专家精度付出两次代价：常驻内存占用取决于路由器能够访问的所有专家，而每个批大小为一的解码步骤只需为其选中的专家付出代价。这是作用于同一组逐专家位宽的两个不同码率泛函，但已发表的逐专家 MoE 分配方法仅约束前者。我们将这种差异称为存储—读取差距；在已发布的 BitsMoE 检查点中，每权重 2.54 比特（bpw）的存储量变成了 7.50 bpw 的解码流量。我们提出 DuoRate，一次存储，按工作负载读取。离线阶段，它以统一精度档位存储可按前缀读取的权重流，并预计算失真张量。在线阶段，它无需标签即可分析工作负载自身的预填充过程，并在解码泛函约束下求解拉格朗日率失真分配；可以证明，路由频率会从每个逐专家决策中消去，而求解开销仅为每个专家一次点积和一次二分搜索。因此，部署时只需保留单一制品，预填充阶段按存储码率读取，解码阶段则按工作负载自身的码率读取。在 Qwen3-30B-A3B 上，以 2.3 bpw 读取的 DuoRate 在平均准确率上比所有 2 比特级基线高出 6.3 至 47.0 个百分点；在 2.9 bpw 下，它与读取量多出 1.4–1.6 bpw 的 4 比特格式表现相当。其优势在长解码场景下最为显著：在 GSM8K 上，2.9 bpw 的 DuoRate 达到或超过所有 4 比特格式，仅比 BF16 低 0.25 个百分点；即使在 2.6 bpw 下，它仍达到或超过 4.5 比特的 NF4 和 FP4。与采用其自身格式的对照方案相比，仅分配策略一项就可节省 0.07–0.11 bpw 的读取码率。代码将在论文发表后发布。
arXiv:2610.13317v1 公告类型：新论文 摘要：偏微分方程（PDE）的代理建模通常依赖密集观测，或需要能够获取控制方程或物理约束。在实际应用中，观测可能稀疏且不完整，底层物理机制也可能仅部分已知或完全未知。我们提出 SBNO，这是一种无需方程、基于配对数据的扩散桥匹配框架，可在部分观测条件下学习用于 PDE 正问题与反问题的条件算子。SBNO 采用扩散 Schrödinger 桥匹配，从掩码感知的端点耦合中学习一对随时间变化的漂移项；该耦合在训练期间保留经验输入与解的配对关系，并在统一框架内支持 PDE 任务的正向和反向求解。在推理阶段，SBNO 通过对由所学漂移项对构建的概率流常微分方程（ODE）进行积分，生成确定性的点重建结果，无需在推理时调整观测引导或 PDE 引导的权重。在观测比例为 5% 的五个 PDE 基准测试中，SBNO 的精度优于或媲美基线方法，同时测试样本误差的离散程度更低。此外，与引导扩散基线方法相比，SBNO 的速度至少提升至 9.7 倍，GPU 显存占用至少降至原来的 1/4.3。
arXiv:2610.13307v1 Announce Type: new Abstract: The escalating environmental footprint of the Information and Communication Technol- ogy (ICT) sectorwith global Data Center electric- ity usage potentially reaching 3% by 2030 and the imperative for real-time responsiveness in smart buildings demand a foundational shift towards Edge Computing. This article proposes a holis- tic architectural and algorithmic approach for cre- ating autonomous, resilient, and privacy-preserving learning systems within smart buildings, advancing the core concept of the building itself as the data center. Our contributions, demonstrating originality compared to conventional centralized systems, are structured around three main pillars: 1) Validation of resilient data acquisition chains using the af- fordable Arduino ecosystem (Uno R4 WiFi/BME280) and secure persistence solutions including centralized archiving (Node-RED/MySQL) and secure remote transfer (Python/rsync/SSH). 2). The proposal of a lightweight, KOptim orchestration framework, inter- operable with the FIWARE standard, for the dynamic management of Service Level Agreements (SLA). 3) The adaptation and experimental valida- tion of the Generalized Hebbian Algorithm (GHA) for Principal Component Analysis (PCA) on low- power microcontrollers,We introduce and compare an iterative learning strategy using data buffers, featuring sorting and explicit preservation of extreme samples to maintain diversity. Experimental results on the high-dimensional Tour Perret sensor data confirm that this novel method, despite introducing a necessary temporal complexity of O(W log W ) due to sorting, achieves convergence and final result quality that is visually equivalent to processing the entire dataset in full memory, reliably identifying two distinct operational states.
arXiv:2610.13306v1 公告类型：新论文 摘要：在资源受限平台上进行本地在线训练，需要较低的计算开销和训练状态存储开销。我们研究了消费级 GPU 上半精度近端策略优化（PPO）的数值稳定性和计算效率。直接将 PPO 训练转换为 FP16 并不可靠，因为数值误差会通过策略更新和后续数据采集形成反馈，而 PPO 会反复比较当前策略与固定的行为策略。我们提出了 PPO-FP16，其中包含三个结构感知模块。全局标量保护用于稳定小批次或参数更新中共享的统计量。策略变化残差保护用于提取并保留当前策略与行为策略之间的微小差异。高效持久更新保护在减少状态存储的同时，稳定优化器状态和参数回写。这些改动保持了 PPO 的目标函数、超参数和训练预算不变。在六项连续控制任务和五个随机种子的实验中，PPO-FP16 在四种方法中取得了最低的回报曲线下面积（return-AUC）平均排名（1.90），并在 30 组环境与随机种子组合的比较中获得了 17 次第一。九种性能配置中有八种实现了 PPO 更新加速，加速比介于 1.16 至 2.03 倍之间。相较于 Full FP32 Adam，优化器状态内存占用减少了 71.7% 至 74.9%，GPU 已分配内存峰值最多减少了 44.5%。结果表明，保护少量在结构上具有重要作用的量，即可在无需重新调参的情况下实现稳定、高效的 FP16 PPO。
arXiv:2610.13305v1 公告类型：新投稿 摘要：我们提出 MORAD，这是一种面向骨架条件化 RNA 扩散模型的在线强化学习框架，可针对多种目标骨架训练单一共享策略。结构条件化 RNA 逆向设计旨在寻找与目标三维骨架相容的核苷酸序列，但现有强化学习方法要么为每个目标骨架单独优化策略，要么依赖离线偏好优化，因此，对可复用的骨架条件化模型进行在线后训练仍基本未得到探索。MORAD 结合了骨架条件化的分组采样、组相对奖励归一化和前向过程扩散目标。现有强化学习奖励主要关注三级结构质量，而 MORAD 则联合优化六项度量，涵盖三级结构几何、碱基配对、系综行为和序列组成。为在量纲尺度和优化方向存在差异的情况下聚合这些异质目标，MORAD 将各项度量映射为有界的合意度评分，并通过加权几何平均将其组合。与仅针对三级结构的奖励相比，这一多目标方案改善了 11 项具有明确优劣方向的评估指标中的 10 项。应用于预训练的 RIDE 时，MORAD 将 GDT-TS 提高了 16.0%（从 0.3300 提升至 0.3827），将配对 MCC 提高了 19.4%（从 0.6110 提升至 0.7295），同时将归一化系综缺陷降低了 27.5%（从 0.3906 降至 0.2830）。在测试集上，MORAD 在三级结构准确性、二级结构配对和热力学质量方面均优于 RiboDiffusion、gRNAde 和 RDesign，在结构条件化 RNA 逆向设计领域达到了新的最先进水平。
arXiv:2610.13304v1 Announce Type: new Abstract: DAG structures can be learned from observations by incorporating acyclicity via smooth constraints, such as DAGMA's log-determinant function. Typically, observations and the underlying DAG are related through linear structural equation models, with Gaussianity assumption on the disturbances at each node. Extending DAG learning methods that account for scale and tail behavior of disturbances that vary across nodes remains a challenge. We propose a node-varying Student-$t$ DAGMA that jointly estimates the graph structure, node-specific disturbance scales, and the degrees-of-freedom parameter. A Gaussian scale-mixture representation yields an iteratively reweighted formulation that down-weights large standardized residuals. Building on existing results, we also establish identifiability of the graph and disturbance parameters. Synthetic experiments demonstrate improved structural recovery under heterogeneous heavy-tailed disturbances, and experiments on real finance data show that the proposed method yields lower test negative log-likelihood than the evaluated baselines.
arXiv:2610.13292v1 Announce Type: new Abstract: Lagrangian Neural Networks (LNNs) learn mechanical systems from trajectory data by parameterizing a scalar Lagrangian and deriving the dynamics through the Euler-Lagrange equations, biasing the model toward motion that respects the variational structure of mechanics. The interactions studied so far have been pairwise-additive, leaving open whether LNNs can recover an irreducible many-body interaction from motion alone. We address this using the Axilrod-Teller triple-dipole interaction, the leading three-body dispersion term, whose dependence on the collective geometry of three particles cannot be represented by any sum of pairwise terms. We prescribe the kinetic energy exactly and learn only the potential, as a function of interparticle distances and angles. Combined with a signal-aware sampling strategy that concentrates training data where the three-body signal is strongest while preserving broad coverage of configuration space, this recovers the interaction from accelerations alone. The learned potential reproduces the radial scaling, angular dependence, and characteristic sign reversal of the Axilrod-Teller interaction, and generates stable trajectories. Embedding it within a pairwise Lennard-Jones potential lets us examine its observability: the three-body term is recovered once its dynamical signature exceeds the residual modeling error, which it does cleanly at amplified coupling but not at the coupling of a real noble gas, bracketing the transition between the two. Finally, symbolic regression on the trained potential, with no assumed functional form, returns the Axilrod-Teller expression with angular numerator, radial law, and coupling constant intact. An LNN with domain-specific structure can thus recover an irreducible three-body interaction from trajectories as a closed-form law, pointing toward learning interaction laws whose structure is not known a priori.
arXiv:2610.13278v1 Announce Type: new Abstract: Vision-language models (VLMs) exhibit complementary capabilities, yet selecting the most suitable expert for a given image-question query remains challenging due to heterogeneous multimodal inputs and varying expert strengths. Existing routing approaches rely on text-based cues or static model profiles, failing to capture fine-grained visual evidence and expert-specific failure patterns. To address this, we propose MHR-Router, a multimodal heterogeneous retrieval-enhanced adaptive routing framework for VLM inference. It accurately dispatches each multimodal query to the most suitable VLM expert via local retrieval evidence. MHR-Router consists of two stages: (1) Multimodal Routing Representation, which fuses frozen multimodal features into a comprehensive query representation; and (2) Heterogeneous Retrieval-Enhanced Adaptive Routing, which retrieves expert-specific success and failure memories as local evidence and employs a shared scorer to adaptively select the optimal expert. Experiments show that MHR-Router effectively achieves heterogeneous modality routing and consistently improves accuracy over traditional baselines.
arXiv:2610.13276v1 公告类型：新论文 摘要：基于集合的语言模型路由旨在控制推理调用次数的同时，提高至少有一个模型正确回答查询这一事件的覆盖率。现有的风险受控路由策略通过校准单个保形阈值，并将其应用于一组嵌套的模型集合，来约束全部失败风险（即所选集合中所有模型均回答失败的概率）。由于每个模型都被单独评分，这些策略忽略了模型失败之间的依赖结构。共享训练数据或架构的模型往往会在相同查询上失败，因此，两个单独表现很强的模型在组合使用时可能存在冗余，而一个较弱但具有互补性的模型却能覆盖其他模型无法正确回答的查询。为考虑这种被现有风险受控策略忽略的依赖关系，我们提出了 CopRoute。它根据每个模型相对于已选模型能够额外正确回答多少查询来对模型排序，针对每个查询决定沿该顺序选择到什么位置，然后选取仍能满足目标风险的最小截断点。我们证明，在设计数据上，CopRoute 的排序对于找到首个正确回答模型所需的平均调用次数，达到了任何高效固定排序算法所能提供的最佳保证；同时，一个独立的校准步骤可为部署时所选模型集合的全部失败风险提供认证。在 RouterBench 和 RouterEval 上，目标风险为 0.05 时，CopRoute 在十个数据集中的九个上实现了最少的经认证调用次数；目标风险为 0.10 时，则在十个数据集中的八个上实现了这一结果。在相同的经认证全部失败风险下，与最先进的风险受控策略相比，其调用次数最多减少 79%，中位数降幅为 24%。
arXiv:2610.13261v1 Announce Type: new Abstract: In this paper, we propose Data-Adaptive Mahalanobis Cross-Head Attention (DAMCHA), a simple and effective attention mechanism that generalizes Multi-Head Attention (MHA) through the lens of metric learning. Standard MHA implicitly learns a static, head-wise Euclidean similarity corresponding to the diagonal blocks of a unified cross-head metric matrix, discarding all off-diagonal interactions across heads. DAMCHA addresses this limitation via two key innovations. First, it learns the full cross-head metric matrix, inducing a Mahalanobis-like similarity that explicitly captures inter-head interactions beyond the restricted block-diagonal structure of MHA. Second, it parameterizes the metric matrix as an input-dependent function, enabling the attention geometry to dynamically adapt to the intrinsic manifold structure of the data rather than remaining fixed after training. We further introduce principled regularization strategies, most notably stack-wise parameter sharing, to ensure computational efficiency and stable optimization. DAMCHA serves as a drop-in replacement for MHA and its variants in various Transformer architectures, delivering superior expressive power and learning performance with comparable or reduced computational overhead. Extensive experiments demonstrate that DAMCHA-based Transformers consistently outperform competing baselines across a range of benchmark tasks. Code is available at https://github.com/erwinmsmith/DAMCHA.
arXiv:2610.13258v1 Announce Type: new Abstract: Neural surrogates provide computationally efficient auto-regressive modeling of wave propagation, enabling e.g. fast inverse solutions. Despite recent progress, the current surrogates are not yet sufficient for the needs of ultrasonic sensing, characterized by high frequency and heterogeneous propagation media. We show how methods building on Fourier Neural Operators (FNO) fail to capture essential signal features, such as reflections, whereas U-Net -based methods learn the wave fields well but lack the ability to transfer information across different spatial scales. We then propose a new surrogate that captures the best of the two worlds, by integrating FNOs and local-integral resampling into a U-Net architecture. We show it can be trained on limited data, and that the predictions are robust to input noise and work across spatial scales.
arXiv:2610.13257v1 公告类型：新论文 摘要：跨分词器同策略蒸馏（OPD）需要先对齐教师与学生的序列，才能比较二者的评分。解码文本对齐在文本块层面建立了这种对应关系，但仍有一个独立问题尚未解决：信用对齐。块级信号指明一个文本片段应当改变多少，却未说明该信号应如何分配到学生的各个词元上，而没有教师对应项的词元则仍然不受约束。我们提出 CAST（Chunk-aware credit Assignment with Selective Token Anchoring），这是一种“改变或保持”框架，可同时解决上述两个问题。对于已对齐的文本块，CAST 利用预测不确定性和局部教师信号的平滑性，将每个块的优势值重新分配到学生词元上，在保持该块总优势值不变的同时，重点关注由局部一致的教师反馈所支持的学习机会。由此得到的权重恰好是一个 KL 正则化信用分配问题的精确解。对于未对齐的位置，CAST 通过 KL 惩罚项选择性地将学生锚定于其初始策略，从而在缺乏教师监督的位置限制策略漂移。由于信用分配与文本块构建相互分离，CAST 也适用于共享分词方式下按标点划分或固定长度的文本块。在六项数学、科学和代码基准测试中，对于从 Qwen3-8B 到 Mistral-7B 和 Llama-3.1-8B 的跨分词器迁移，CAST 的平均得分相较于所评估的最强基线分别提高了 $2.45$ 和 $1.32$ 分；对于到 Qwen3-1.7B 的共享分词器迁移，则提高了 $2.10$ 分。
arXiv:2610.13253v1 公告类型：新论文 摘要：采用上下文学习的表格基础模型（TFMs）无需额外训练即可解决新任务，因此尤其适用于资源有限或存在隐私约束的场景。然而，其架构包含多个 transformer 块，导致较高的计算和内存开销。与此同时，梯度提升决策树（GBDTs）等传统替代方案仍具有很强的竞争力，并且能在 CPU 上高效运行，这表明表格任务可能并不天然需要深层架构。这一对比引出了一个根本问题：能否通过移除架构冗余来简化 TFMs，同时不牺牲性能？为回答这一问题，我们提出了一种具有闭式解、无需标签的仿射转换器，可替换最多占全部块数 96% 的连续块区间，同时保持其余部分不变。通过在 171 个分类数据集上评估八种前沿 TFMs，我们发现表格 transformer 存在严重的过度参数化：在所有架构中，替换模型一半的块所造成的 AUROC 中位数损失最多仅为 $0.002$，而深度减半的 TabFM 仍优于其他所有完整深度的 TFMs 和经过调优的 GBDTs。此外，不同模型家族的过度参数化方式各不相同：TabPFN-3 依赖其编码器，无论仅保留哪一个块，都能维持性能。其他单轴模型将计算分散在各个块中，实际上只需要第一个块；双轴模型则将计算集中在一个关键块中。最终，削减深度最多可减少 15 亿个参数，并以 $0.03$ 的 AUROC 中位数损失为代价，将 GPU 推理速度提升至原来的 $6\times$；而仅含一个块的 TabPFN-3 在性能损失可忽略不计的情况下，对缓存查询的处理速度达到原来的 $2.4\times$，内存用量仅为原来的六分之一。