全部动态

本页 40 条
10月9日 周五
arXiv cs.LG77

TRACE: A Governance Framework for Measuring Explainability Debt in Production AI Systems

arXiv:2610.10957v1 Announce Type: new Abstract: Production AI systems deployed in high-stakes domains accumulate a governance liability that existing monitoring frameworks fail to detect: the progressive inability to explain individual decisions when regulators, auditors, or affected individuals demand accountability. We introduce TRACE (Transparency, Risk, Accountability, Compliance, and Explainability), a seven-instrument governance framework for measuring, tracking, and remediating Explainability Debt in production AI systems. The foundational instrument, the Explainability Debt Score (EDS), quantifies the proportion of production decisions falling below a governance-defined explainability confidence threshold at any point in time. Complementary instruments include DART (Debt Accumulation Rate Tracker for breach forecasting), SHIV (Scenario Health and Integrity Validator for daily governance), FDE (Feature Drift Evaluator for causal attribution), HVE (Human Validation Engine), AIDE (Audit Intervention Decision Engine), and ZERO (Zero Explainability Risk Optimiser for remediation). Through a twelve-month longitudinal case study of a production fraud detection system processing 50,000 daily financial transactions, achieving 98.46% accuracy and ROC-AUC of 0.9990, we demonstrate that an EDS of 0.23 on audit day was statistically predictable six months in advance using DART trajectory analysis (beta = 0.008/week, R-squared = 0.94, 95% CI: [0.006, 0.010]), and that 78% of Explainability Debt was concentrated in the highest-regulatory-risk decision category (transactions above $10,000), a risk asymmetry completely invisible to system-level metrics. TRACE provides the first quantitative operational architecture for EU AI Act Article 13 compliance in production AI deployment, establishing a new subdiscipline of explanation governance distinct from explanation generation.

arXiv cs.LG66

SPD-MetaFormer is what you need for small-data brain decoding

arXiv:2610.10952v1 Announce Type: new Abstract: Brain signal decoding is challenging because neural recordings are noisy and vary across individuals, while labeled data are often limited. Recent attention-based models on the symmetric positive definite (SPD) manifold have nevertheless achieved strong performance using covariance and connectivity representations, yet the contribution of learned token weighting remains unclear. We examine two representative architectures, MAtt (based on log-Euclidean geometry) and GBWAtt (based on generalized Bures--Wasserstein geometry), and find that their learned attention weights remain close to uniform after training. We relate this behavior to bounded similarity parameterizations that, under the original softmax scaling, limit attention-weight contrast. Moreover, replacing learned weights with uniform weights, throughout training and evaluation, has little effect on mean predictive performance while preserving each model's original aggregation geometry. Motivated by these findings, we introduce SPD-MetaFormer, an attention-free architecture built on uniformly weighted Fr\'echet aggregation under log-Euclidean geometry. Its backbone uses a geodesic residual to update a summary token and a shared spectral feedforward map to transform all tokens, followed by a learned weighted readout. Token states remain SPD-valued until tangent-space classification. Across three EEG benchmarks, SPD-MetaFormer achieves competitive results relative to published Euclidean and manifold baselines. Separate matched reproductions test learned versus uniform weighting within MAtt and GBWAtt. These results suggest that, in the short-sequence and limited-data regimes studied, carefully designed SPD architectures can provide a simpler and effective alternative to adaptive manifold attention.

arXiv cs.LG81

RH-Detect：用于奖励劫持检测的统一基准

arXiv:2610.10947v1 公告类型：新 摘要：奖励劫持是指模型利用评估信号，却未完成预期任务的行为，这对已部署语言模型系统的可靠性构成威胁。现有数据集采用不同的标签、响应格式和元数据规范，导致检测器结果难以比较。我们提出 RH-Detect，这是一个将来自 11 个公开数据集的、与奖励劫持相关的子集整合到统一模式中的基准，包含 92,761 条记录和六类行为。在 5,021 个开放式评估单元上进行评估，每个单元包含一个任务提示和一段模型自由形式的续写，其中还包括多轮工具使用轨迹。我们在不进行额外训练的情况下，将来自五个模型系列的六个现成语言模型作为奖励劫持检测器进行评估。最佳模型的汇总 AUROC 达到 0.962，准确率超过 93%。然而，对于表现最强的四个模型，在相同决策阈值下，它们在两个多轮工具使用数据集 MALT 和 TRACE 上的准确率比其他来源低 10.7—15.9 个百分点，凸显了部署时监控中的一个关键缺口。我们发现，不同输入格式对不同模型的影响各异。移除思考过程后，Qwen3.5-4B 的 AUROC 从 0.779 提升至 0.849，但 Qwen Flash 的 AUROC 从 0.977 降至 0.950。我们还评估了将该基准作为检测器训练数据集的效果。依次将每个来源留作测试集时，单令牌 SFT 能够提高在六个留出来源中的五个来源上的平均 AUROC。针对这一失败案例进行的后续 GRPO 训练使检测性能略有提升。我们的结果表明，单一的汇总分数可能掩盖不同数据来源、检测器输入和训练过程之间的差异。

arXiv cs.LG74

Rethinking the Tradeoff Between Temporal Encoding and Nonlinear Computation in Spiking Language Models

arXiv:2610.10933v1 Announce Type: new Abstract: Spiking language models face a tradeoff between representing continuous semantic features over short temporal windows and retaining costly nonlinear attention operations. We introduce Spora, which jointly designs spike encodings and attention operators. Binary temporal weights let $T$ spikes represent compositional values with up to $T$ bits of capacity, compared with $O(\log_2 T)$ bits for spike-count readout. Unipolar Binary Spiking (UBS) uses thresholds and spike-triggered residual decay to produce non-negative integer codes; Bipolar Binary Spiking (BBS) separates sign and magnitude and learns a scale for signed activations. These representations support accumulation-and-shift dot products and integer-exponent mappings in attention. With four time steps, Spora achieves 76.6 average GLUE score and 44.1 CoLA MCC, improving over SpikeLM by 1.2 and 6.2 points, respectively. Extending BBS to six steps raises these scores to 78.2 and 47.4. Conditional-decay analysis, matched-budget activation-quantization comparisons, event-workload statistics, and fixed-point evaluation further characterize the connection between encoding fidelity and computational cost.

arXiv cs.LG74

用于目标条件强化学习的 World-Model Policy Arbiter

arXiv:2610.10932v1 发布类型：new 摘要：离线目标条件强化学习（GCRL）已经产生了多种多样的目标到达算法，但没有任何一种算法能够在不同环境、不同目标，甚至同一任务的不同阶段始终表现最佳。我们不再只部署性能最佳的策略，而是探讨能否将一组冻结的目标条件策略作为一个策略组合共同使用，并在每个状态决定由哪个策略执行。如何在每个状态选择策略并不简单：这些策略自身的价值函数无法直接比较，因为它们可能使用不同的尺度，而且有些策略没有价值函数。我们需要根据每个策略可能到达的状态来评估它，即使我们每次只能执行一个策略。同时，我们还需要避免频繁切换策略导致控制不稳定。为应对这些挑战，我们提出 World-Model Policy Arbiter（WMPA），这是一个测试时框架。给定一组冻结策略后，该框架在学习到的状态空间世界模型中分别展开每个冻结策略，使用共享的目标条件价值函数评估想象中的未来，并执行得分最高的策略一小段固定时长，然后进入下一轮仲裁（策略选择）。WMPA 假设可以获取一组冻结的目标条件策略，但不需要重新训练策略，也不需要特权性的任务特定知识。在官方 OGBench 评测协议下，针对涵盖迷宫导航以及立方体、场景和拼图操作的 18 个基于状态的数据集，WMPA 将宏平均成功率从每个数据集选择最佳策略时的 44% 提升至 58%，并在 12 个数据集上取得了具有统计显著性的提升。其中，在 cube-double-play 上提升了 33 个百分点，在 scene-play 上提升了 36 个百分点。

arXiv cs.LG69

Coefficient Calibration as Selection Pressure in Symbolic Regression

arXiv:2610.10931v1 Announce Type: new Abstract: In memetic symbolic regression, candidate structures are compared after coefficient calibration, so the calibration protocol itself contributes to evolutionary selection. Standard centralized calibration evaluates each structure at its pooled-sample optimum, ignoring how stable this calibration is under covariate shifts, and can thus favor structures whose fit relies on sample-specific coefficients. We propose Dirichlet-Sinkhorn Constant Averaging (DSCA), a calibration strategy that partitions the optimization data into equally sized subsets with different covariate distributions, calibrates each candidate independently on every partition, and evaluates it at the mean of the resulting parameters. We show that the excess loss of DSCA relative to centralized calibration vanishes at the population level for correctly specified, identifiable expressions, whereas under misspecification it persists when partition-specific calibrations do not aggregate to the pooled optimum. On synthetic benchmarks and ten real-world datasets, DSCA improves functional recovery and the accuracy-complexity trade-off over centralized Broyden-Fletcher-Goldfarb-Shanno and Levenberg-Marquardt calibration, under selection by negative log-likelihood and by the Akaike and Bayesian information criteria. Mechanism analyses associate the DSCA excess loss with the generalization gap and show that the effect is not reproduced by repeated centralized fitting. These results indicate that controlled heterogeneous calibration provides a complementary source of selection pressure in symbolic-regression search.

arXiv cs.LG74

面向资源受限车联网的自适应多判别器 WGAN 框架：结合强化学习与博弈论

arXiv:2610.10926v1 公告类型：new 摘要：将机器学习工作负载作为网络服务进行管理，会引入不同于传统模型训练的资源编排问题：应为任务分配哪些节点，如何在这些节点之间划分通信与计算预算，以及在连接性和节点可用性随移动性变化时如何维持服务质量。在车联网（IoV）环境中部署生成对抗网络（GANs）正是这一问题的一个棘手实例；资源约束、动态网络拓扑以及相互竞争的优化目标意味着，传统 GAN 架构无法同时实现高精度、高效资源利用、低延迟和低通信开销。本文提出一种自适应多判别器 Wasserstein GAN（MD-WGAN）框架，将强化学习与基于博弈论的协调机制相结合，以协同应对这些挑战。在该框架中，路侧单元部署生成器，并为其配备 Deep Q-Network（DQN）智能体；这些智能体负责选择判别器子集，并管理移动车辆节点上的分布式训练，同时通过基于博弈论的协调步骤在生成器和判别器之间分配训练轮次。统一的优化目标将对抗学习质量与车辆网络约束下的资源效率、通信开销和延迟联系起来，使框架能够随着网络状况变化持续调整其训练行为。在真实世界 NGSIM 轨迹数据上的评估表明，该框架的预测精度与当前最先进的 GAN 基线相当——在所有评估方法中取得最低的 RMSE（1.029）和 MAE（0.894）——同时显著提升了资源效率：平均 CPU 利用率降低约 28%，平均内存使用量降低约 6%，且通信开销和延迟具有竞争力。

arXiv cs.LG76

电商搜索中的页面级布局决策语言模型

arXiv:2610.10920v1 公告类型：新 摘要：电商搜索页面是数百万在线购物者的重要接触点。传统搜索引擎返回按排名排列的结果列表，而现代电商搜索页面越来越多地融入推荐系统模块——例如，在特定位置展示其他产品分组的辅助结果堆栈。适当地引入辅助结果堆栈可以提升用户参与度；然而，位置选择不佳可能会打乱浏览流程，并降低主要搜索结果的效果。不同于传统搜索排序中已经较为成熟的交错实验等评估技术，评估页面级布局变化（例如何时以及在何处插入辅助结果堆栈）仍然具有挑战性，因为这通常需要成本高昂的在线 A/B 测试。为此，我们研究了用于离线评估特定布局决策是否有利于用户的方法，具体而言，即评估在特定位置加入辅助结果堆栈是否有益。我们通过比较基于直接提示、基于提示生成特征和基于表示的方法，探索了将语言模型用作可扩展评估器的可行性。结果表明，在预测用户参与度方面，基于表示的方法始终优于基于提示的评判方法，这说明前者可以为电商搜索中的离线布局评估提供可靠基础。

arXiv cs.LG83

你的语音质量指标有多容易被攻击？一种修正后的协议、一个基准，以及修补能带来什么

arXiv:2610.10899v1 公告类型：新 摘要：语音质量预测器越来越多地被用作奖励，但目前尚无衡量其可攻击性的统一方法。通常的测量存在两个缺陷。第一，扰动会经过一条处理链——此处为神经编解码器——传递给预测器，而该处理链本身就会改变分数；如果以原始输入为基准进行评分，就会把这部分变化归因于攻击。改用未扰动的往返结果作为参照后，测得的可攻击性最多会相差四倍（对于某种防御，数值从 0.31 变为 0.08）。第二，一个训练好的攻击器只是一个样本，而不是一次测量：仅随机种子不同的五个攻击器针对同一个固定预测器时，成功率在 0.00 到 0.38 之间，因此防御效果的论断需要考察多个攻击器中的最坏情况。在该协议下，四个已发表的预测器表现差异很大：NISQA 可攻击 90% 的话语，SSL-MOS 为 21%，DNSMOS 为 14%，UTMOS 为 6%。随后，我们审查了一个封闭的攻击—检测—修补循环。它只能在自身的攻击空间内增强预测器的防御能力，而且提升幅度小于攻击器之间的差异；随机扰动基线也能达到同等效果；此外，在域外评估时，它会使系统 SRCC 最高损失 0.30。针对修补后的预测器进行后训练的增强器对其的攻击成功率大幅降低（PESQ 为 -0.03，而不是 -0.23）。代码、预注册材料和运行输出均已发布。

arXiv cs.LG71

Gen-PINNs：用于求解偏微分方程的生成对抗物理信息神经网络

arXiv:2610.10897v1 发布类型：new 摘要：物理信息神经网络（PINNs）是一种广泛使用的无数据方法，利用机器学习求解偏微分方程（PDEs）。随着生成对抗网络（GANs）的最新发展，对抗学习在复杂数据驱动问题的建模方面展现出强大能力；然而，GANs 在确定性物理信息 PDE 解中的应用仍然有限。在本文中，我们首先指出标准 PINNs 在求解 PDEs 时存在的局限性，包括谱偏差、损失不平衡和优化器停滞。随后，我们提出生成对抗物理信息神经网络（Gen-PINNs），这是一种统一的确定性残差-对抗框架，旨在改进对具有尖锐变化或激波前沿行为的 PDEs 的无数据求解。生成器通过动态加权的物理信息损失分量学习潜在的 PDE 解，而独立的判别器则针对理想零残差状态评估互补的 PDE 残差特征。该框架进一步开发并改进了多项方法组件，包括用于解析高频空间内容的 Fourier 表示、用于量化随频率变化的解误差的正交谱诊断方法，以及针对物理、初始条件、边界条件和对抗损失目标的改进型基于梯度的动态加权系统。我们在包括 Burgers、Allen-Cahn 和 Kuramoto-Sivashinsky 方程在内的非线性及高阶 PDEs 上，将 Gen-PINNs 与标准 PINNs 进行了比较测试。结果表明，在尖锐前沿、刚性和高阶 PDE 解方面，Gen-PINNs 在精度和收敛性上均取得了显著提升，凸显了确定性残差-对抗学习作为求解具有挑战性的非线性 PDEs 的有效方法的潜力。

arXiv cs.LG81

Barron最优传输 I：生成建模

arXiv:2610.10875v1 公告类型：new 摘要：受生成建模和采样领域近期应用的启发，我们提出了一个最优测度传输框架，其中代价刻画神经网络的复杂度。在基于传输的生成模型中，参考分布（例如高斯分布）中的样本沿普通微分方程或随机微分方程映射为目标分布中的样本。这些过程在时间离散化后通过深度残差网络实现，其中每个隐藏层近似对应的瞬时速度。因此，给定一对目标测度和参考测度，一个自然的问题是寻找实现这一传输的最高效神经表示。我们的出发点是 Benamou 和 Brenier 提出的最优传输动力学表述。我们以 Barron 能量替代平均动力学 $L^2$ 能量；Barron 能量是一种自然范数，用于衡量通过神经隐藏层表示给定向量场的复杂度，并能够刻画特征学习的自适应特性。这一定义在概率测度空间上引入了一个度量，补充了现有的 Wasserstein 几何和 Stein 几何。在本文中，我们研究这一度量在生成建模背景下的性质。作为第一个应用，我们通过为由高斯分布经神经网络推前生成的数据建立超多项式的得分函数逼近下界，量化扩散生成建模在 Barron 几何中的次优性。随后，我们研究自适应性作为考察替代生成模型的一种方式所带来的益处。在配套论文 \cite{companionpaper} 中，我们利用 Barron 传输几何开展采样应用，通过特征自适应扩展 Stein 变分梯度方法的适用范围。

arXiv cs.LG69

Shape irregularity of Life-Like Network Automaton rules as an indicator of classification performance

arXiv:2610.10867v1 Announce Type: new Abstract: Complex Network (CN) classification requires high-level structural characterizations that are both scale-invariant and computationally efficient. Methods based on Life-Like Network Automata (LLNA) offer an interesting way to extract network descriptors by leveraging emergent temporal patterns without requiring provided features, but their efficacy is bottlenecked by a high-cost combinatorial optimization problem: the selection of the automaton transition rule. While current literature relies on exhaustive searches that are unfeasible for large-scale applications, this work reveals that the rule space is fundamentally structured by a property we term ``jaggedness'', that quantifies the resemblance of a LLNA transition function with a sawtooth shape. We demonstrate that this metric acts as a theoretical proxy for chaoticity and sensitivity -- properties essential for generating discriminative dynamic behaviors among network categories. Moreover, we introduce a heuristic search strategy that uses jaggedness to guide the rule selection. Experimental results show that our approach achieves classification accuracies within 5% of the global optimum while reducing computational overhead by 90% compared to exhaustive approach. Our findings provide a novel, efficient, framework for optimizing automata-based methods for pattern recognition.

arXiv cs.LG69

KDFP: A first-principles approach to knowledge distillation in large language models

arXiv:2610.10854v1 Announce Type: new Abstract: Knowledge distillation is an established technique for improving the capabilities of small, efficient student models by training them with the representations of larger, more capable teacher models. Much of the recent work in the distillation of large language models (LLMs) has focused on distilling abilities learned during post-training, such as instruction following, chain-of-thought reasoning, and tool usage. This has left a large research gap in general knowledge distillation for LLMs, which is essential for developing efficient and private systems suitable for deployment on edge devices. We take a first-principles approach, evaluating previous lessons from prior works and conducting new explorations to develop a distillation methodology suitable for modern LLMs. We present KDFP, a novel methodology for white-box general knowledge distillation in LLMs. We demonstrate that KDFP outperforms existing methods by 1.6% $-$ 4.9% across 9 benchmarks while increasing training efficiency by up to 99.1% through ephemeral parameter reduction.

arXiv cs.LG74

Similar Predictive Fit but Different Latent Dynamics: Characterizing Learned Dynamical Structure in Personalized Models of Brain Disorders

arXiv:2610.10850v1 Announce Type: new Abstract: As AI models move toward clinical decision-making and personalized treatment, understanding \emph{what} a model learns is important beyond predictive accuracy alone. We investigate whether personalized latent dynamics reveal clinically associated differences even when predictive fit is similar. A lightweight CNN--Transformer EEG foundation model pretrained on the Temple University EEG Corpus (TUEG) extracts segment-level representations. Using the Temple University Epilepsy Corpus (TUEP), representations are mapped to a shared latent-state space, and sparse multinomial logistic transition distributions (mLTD) are fit independently to each subject to obtain personalized transition-dependency graphs $W_n$. Analyses include $n{=}198$ subjects (99 epilepsy / 99 non-epilepsy). At $k{=}4$, epilepsy subjects exhibit substantially denser learned dependency structure ($p{=}1.1\times10^{-7}$), with the same pattern at $k{=}6$ (19.90 vs. 13.46; $p{=}5.2\times10^{-5}$). Graph-derived features provide moderate group discrimination under 5-fold subject-wise cross-validation (AUROC 0.68 at $k{=}4$; 0.65 at $k{=}6$). In contrast, held-out log-likelihood is nearly identical between groups at $k{=}4$ ($-0.992$ vs. $-0.991$; $p{=}0.95$), with similarly matched next-state prediction (AUROC 0.855 vs. 0.861; $p{=}0.54$). Thus, similar predictive fit does not imply similar learned dynamics: groups can be comparably predictable while differing substantially in the internal dynamical structure learned by personalized models. This distinction motivates evaluating learned structure alongside predictive performance in personalized clinical models.

arXiv cs.LG79

Amortized Off-Policy Evaluation for LLMs

arXiv:2610.10848v1 Announce Type: new Abstract: Accurate evaluation is central to selecting which LLM to deploy, yet testing a candidate on live traffic exposes real users to an unvetted model. Teams therefore evaluate candidates offline, on data produced by already-deployed models. This is off-policy evaluation (OPE), and it faces two distribution shifts: as a model is updated in post-training, its responses diverge from the logged ones (policy shift), and the reward definition under which it is judged changes with business requirements (reward shift). Classical OPE methods are ill-suited to this continual-deployment setting because they are defined per task and require fitting from scratch on every new logged dataset or reward definition. To address this, we propose PFN-OPE, a prior-data fitted network that amortizes OPE across a distribution of contextual-bandit tasks. We pretrain it once on tasks constructed from a pool of LLM responses scored by several reward functions, in which both shifts occur. At test-time it maps a logged dataset and one sampled target response per prompt to a value estimate in a single forward pass, with no per-task fitting. On HelpSteer2 and UltraFeedback with Qwen, Llama, and Gemma policies, PFN-OPE achieves 2.0 to 9.3 times lower error than the best baselines across all tested configurations in the reward-shifted settings.

arXiv cs.LG74

MotherTree: Meta-learning on synthetic data improves decision tree training

arXiv:2610.10832v1 Announce Type: new Abstract: Conventional decision tree algorithms produce effective, transparent models that can be audited, communicated, and deployed independently of the training data, but require learning every new task from scratch. In contrast, tabular foundation models demonstrate that meta-learning from a synthetic prior distribution enables strong in-context prediction for previously unseen tasks, especially in small-sample regimes. However, this approach does not produce a standalone model that can be inspected in isolation. We introduce MotherTree, a tabular transformer that meta-learns decision tree induction: given a training set for a new task, it outputs a hard, axis-aligned decision tree, equivalent in form to classically trained trees, in a single forward pass. MotherTree is pre-trained on a synthetic prior using stochastic gradient descent without requiring reference trees for supervision. On established benchmarks with controlled sample size, the approach is competitive with size-matched trees from common algorithms: recursive partitioning, gradient-based tree learning, globally optimal trees, and distillation from tabular foundation models. Notably, MotherTree consistently improves over from-scratch gradient-based learning and acts as a strong initializer: task-specific tuning of the generated tree outperforms the corresponding from-scratch learner on all benchmarks and sample sizes. These results show that meta-learning can provide effective inductive biases for learning stand-alone, small decision tree classifiers.

arXiv cs.LG74

使用干预迁移评估评分标准生成

arXiv:2610.10809v1 公告类型：新 摘要：针对实例的评分标准在 AI 基准测试中十分常见，因为可靠的评估需要特定的专家知识。这种方法难以扩展，因此研究人员开始探索使用大语言模型（LLM）生成评分标准。然而，即使有专家评分标准作为参考，如何大规模有效地评估生成评分标准的质量仍不明确。本文提出了一种用于评估评分标准生成的方法，称为干预迁移（Interventional Transfer，IT）。该方法基于这样一个理念：如果对某个回答进行扰动，使其通过或未通过其中一个评分标准时，两个评分标准的判定结果会同步变化，那么这两个评分标准就是相似的。不同于现有的评分标准生成评估方法，我们认为，不同形式的干预迁移可用于评估生成评分标准在不同任务中的效用。例如，我们在 HealthBench 上开展了一项案例研究，展示了由 Qwen3.8-27B、Deepseek-V4-Flash 和 Opus-5 生成的评分标准之间存在不对称性，这些评分标准用于评估 GPT-5.6-Terra 生成的回答。根据生成的评分标准或专家评分标准会降低回答质量的扰动，会在相应的专家评分标准或生成评分标准上迁移为更低的分数；但在一个评分标准上有所改进的扰动，却无法可靠地迁移为另一个评分标准上的更高分数。我们认为，这一发现对于使用 LLM 生成的评分标准进行性能监控和爬山优化具有重要意义。我们将该方法与现有的评分标准评估方法进行了对比，后者无法揭示出我们观察到的这种不对称性。

arXiv cs.LG70

Controlled Acquisition and Abstention in Three-Channel Score Conflicts

arXiv:2610.10808v1 Announce Type: new Abstract: When audio, video, and text disagree, accuracy alone does not show whether to acquire another source or abstain. We study these choices in a controlled three-score benchmark: a policy observes two signed scores, may request the third at a cost, and can abstain. The primary reward is mechanism-specific: abstention is correct only for one designated ambiguity mechanism and is penalized under mixed corruption. Matched controls show that a threshold policy matches always-request decisions with fewer requests; its advantage over always-answer fusion depends on the reward assigned to that ambiguity. On a partially held-out synthetic split, the threshold policy reaches 0.789 +/- 0.006 targeted decision accuracy and 0.481 +/- 0.014 utility across 83 seeds. A three-score majority reference reaches 0.626 +/- 0.008 and 0.252 +/- 0.016, but uses more information. In a matched-budget test, a train-only value selector improves utility over no-query and matched-random policies at 10% and 25% budgets, while pair uncertainty has higher utility at every budget. At 50% and 63.7% budgets, the selector lowers utility despite slightly higher non-ambiguous accuracy. If all abstentions are scored incorrect, majority outranks the threshold policy in utility. At a central temporal setting, full-trace controls match the neural models while position perturbations separate them. On held-out-actor emotion clips, eight-frame fusion has opposite-signed accuracy differences for two encoder pairs, with both actor intervals containing zero; matched-request routing gains are small and uncertain. These results separate full-modality accuracy from pre-request selection value and show that selection value depends on budget and the observed-pair ranking.

arXiv cs.LG70

Symbolic Density Estimators for Unnormalized Distributions

arXiv:2610.10807v1 Announce Type: new Abstract: Estimating the symbolic or analytical form of probability density functions (PDFs) from observed samples is a fundamental challenge in statistical and computational modelling. This process is critical for deriving interpretable and generalizable relationships characterizing the underlying phenomenon. Traditionally, this estimation depends strongly on domain expertise and prior field-specific knowledge, with experts selecting appropriate functional forms or parametric families based on empirical evidence and theoretical understanding. The coefficients of these forms are then typically determined through parameter estimation. In this paper, we develop a framework for estimating symbolic expressions of unnormalized distributions from observed samples using domain-specific prior knowledge, such as the range of interactions and a predefined set of primitive functions. We integrate deep generative models with symbolic regression (SR), incorporating inductive biases, such as factorizing large distributions, to keep the problem tractable. The deep generative models we examine include likelihood-based models, viz., flow models, and score-based models. Experiments show the effectiveness of the proposed framework for estimating density functions for multivariate toy distributions as well as lattices from computational physics, namely, XY model and $\phi^4$ theory. When applied to the renormalization problem in $\phi^4$ theory, the proposed framework estimates compact symbolic approximations of the hamiltonian function at different scales directly from samples, yielding expressions that may be challenging to derive using traditional perturbative or analytic approaches in nonperturbative settings.

arXiv cs.LG79

在异质性条件下进行排序需要多少次重复的成对比较？

arXiv:2610.10795v1 公告类型：新 摘要：我们研究如何根据成对比较，按照总体平均效用对选项进行排序，其中用户和任务之间的偏好存在差异。已有研究表明，即使用户数量任意多，每位用户仅进行一次比较也可能不足以确定平均效用最高的选项（Golz et al., 2025）。我们考察在每个用户—任务情境中需要进行多少次重复比较，才能充分且必要地恢复排序。在固定逆温度的异质 Bradley-Terry 模型下，我们首先提出一种基于朴素 MLE 的算法；为确保恢复排序，该算法要求每个情境进行 $\Omega(1/\Delta^2)$ 次重复比较。随后，我们提出两种基于 MLE 的变体，以及一种 Russian Roulette-style 随机算法；它们仅需在每个情境中进行 $O(\log(1/\Delta))$ 次重复比较即可恢复排序，并证明这种对数级依赖是最优的。尽管在最坏情况下需要这么多比较，我们的 Russian Roulette 算法在期望意义下每个情境仅使用 $O(1)$ 次比较。我们在偏好异质性程度不同且情境分布各异的设置下，开展了合成实验和基于 Arena 数据的半合成实验，对这四种算法进行了比较。

arXiv cs.LG81

MemoWM：世界模型如何改变智能体需要记忆的内容

arXiv:2610.10778v1 发布类型：新 摘要：长期运行的智能体在不断积累经验的过程中，面临日益增长的存储需求。世界模型能够捕捉可复用的规律，从而减少为每条经验存储的信息量。我们提出了以世界模型为条件的记忆分配问题，并引入 MemoWM：一个利用共享预测来压缩保留信息并重建省略内容的框架。其任务感知的分配规则在重建错误的预期影响与存储成本之间取得平衡，保留那些具有预测先验之外下游价值的信息。在五个长期智能体记忆基准测试中，MemoWM 达到 42.42\% 的平均答案准确率，比最强基线高出 2.62 个百分点；与此同时，相较于存储效率最高的基线 MIRIX，平均特定经验存储量减少了 53.9\%。进一步分析表明，在任务质量相当的情况下，更强的世界模型能够降低每条经验的存储量。将模型参数纳入考量后，我们发现共享模型容量与反复产生的存储成本之间存在权衡；随着保留的交互数量增加，使总存储量最小化的模型容量也会随之增大。我们的代码可在 https://github.com/Feld-maxiu/MemoWM 获取。

arXiv cs.LG80

NEMORA: Neural Equivariant Multipole Operators for Long-Range Atomistic Learning

arXiv:2610.10776v1 Announce Type: new Abstract: Equivariant graph neural networks have emerged as foundational architectures for machine-learned interatomic potentials, approaching quantum-chemical accuracy at a fraction of the computational cost. These models describe local atomic environments accurately, but finite spatial cutoffs truncate long-range information flow, and stacking message-passing layers can lead to over-smoothing and over-squashing. Existing long-range extensions either prescribe a fixed analytical propagation kernel, restrict long-range communication to scalars or degree-preserving channels, are only approximately equivariant, or incur super-linear computational cost. Combining learnable long-range equivariant transport with multiscale many-body expressivity and efficient scaling for larger systems remains a central challenge. We introduce Neural Equivariant Multipole Operators (NEMORA), a neural equivariant extension of the Fast Multipole Method (FMM) for learning long-range tensorial representations. NEMORA generalizes the FMM's analytical multipole expansion and translation operators to learned equivariant counterparts on an adaptive spatial hierarchy. Its operators couple angular degrees and form many-body interactions across length scales, retaining the FMM's hierarchical organization and analytical radial factors as physical inductive biases while learning data-dependent long-range couplings. NEMORA evaluates in linear time and memory complexity, allowing it to treat larger systems than other long-range methods reaching hundreds of thousands of atoms, and it augments both symmetry-constrained and unconstrained short-range backbones. On non-local benchmarks, it reduces force and energy errors relative to the short-range backbones by over an order of magnitude and up to three orders of magnitude, respectively, which is better than or competitive with existing long-range extensions in accuracy.

arXiv cs.LG79

面向表格数据的对话式任务消歧：泄漏感知的形式化、基准套件与训练

arXiv:2610.10740v1 发布类型：新增 摘要：面向表格数据的对话式任务消歧，是指在针对表格或数据库生成解决方案之前，通过对话解决用户预期任务中缺失的信息。现有的评估和训练缺乏泄漏感知的基础。任务成功率混合了智能体的消歧能力与解决方案生成能力，还可能反映出预言机泄漏，即用户模拟器透露了超出真实用户可能提供范围的信息。现有数据集也缺乏对歧义和访问边界的统一表示。我们提出了“歧义可验证任务”这一概念，对歧义及其消解进行形式化，将智能体分解为提问策略和解决方案策略，并将环境分解为预言机和验证器。该框架提供了用于将任务消歧与解决方案生成分开评估的基线和指标、预言机泄漏的形式化定义、无需评判器的泄漏诊断方法，以及用于训练提问策略的目标函数。我们在 text-to-SQL 中实例化了该框架，构建了 AmbiTab 基准套件，将六个存在歧义的数据集统一为一种表示形式，明确智能体、预言机和验证器各自可以访问的内容。我们评估了澄清策略和预言机泄漏，并使用强化学习训练了提问策略。训练后的提问策略在全部六个数据集上提升了我们的消歧指标，并在其中五个数据集上提升了任务成功率；我们的泄漏诊断方法则衡量了训练对预言机泄漏的影响。

arXiv cs.LG69

Learning infinite context windows in recurrent architectures via spatial neural computing

arXiv:2610.10690v1 Announce Type: new Abstract: Recurrent neural networks (RNNs) offer linear-time scaling with sequence length while requiring only constant memory, yet they struggle to capture long-range dependencies due to vanishing gradients and limited receptive fields. To address these limitations, we introduce a second-order recurrent model in which the standard neuron-to-neuron communication is replaced by a spatially evolving field governed by (discretized) partial differential equations. Drawing inspiration from the role of cortical waves in brain computation, this mechanism allows structured spatiotemporal patterns to serve as an implicit, high-capacity memory. We show that the resulting model is equivalent to a structured infinite-order RNN in which the current state depends explicitly on its entire history of past states, yielding an effectively unbounded receptive field with a fixed number of parameters. We further derive constructive conditions to ensure marginal stability, constraining the gradient spectrum on the unit circle and thereby eliminating vanishing and exploding gradients. Empirically, the proposed architecture outperforms other recurrent models on long-horizon benchmarks while using substantially fewer parameters, demonstrating that spatial dynamics can effectively bridge the gap between efficient inference and long-term memory.

arXiv cs.LG77

解释经对抗训练的神经网络的显著图稀疏性

arXiv:2610.10666v1 发布类型：新 摘要：理解深度神经网络为何做出特定预测，对于其安全部署至关重要。在计算机视觉中，显著图通过突出对预测影响最大的图像区域，仍是广泛使用的一种解释形式。一项经验观察是，经对抗训练的神经网络的梯度显著图表现出明显的稀疏性。本文针对两层 ReLU 网络，对这一现象提出了理论解释。我们建立在这样一个已确立的等价关系之上：对于某些损失函数，对抗训练等价于对带权重衰减惩罚并附加对抗总变差项的经验风险进行最小化。随着数据点数量和神经元数量不断增加，并且正则化参数以适当速率趋于零，我们证明，最小化器会收敛到具有最小梯度范数和 Barron 范数的贝叶斯分类器。对于采用 $\ell_\infty$ 攻击的对抗训练，梯度范数具有各向异性，并偏好轴对齐或稀疏梯度，因此会产生稀疏性。我们通过实验评估自然训练模型与对抗训练模型的梯度 $\ell_1$ 范数和阈值化稀疏性，展示了理论结果。

arXiv cs.LG78

Teaching PPG How not Who: Fixed-Effects Distillation from ECG

arXiv:2610.10662v1 Announce Type: new Abstract: ECG is widely used to teach PPG-only models, yet what it teaches is unexamined. Wearables are valued for tracking how a person's cardiovascular state changes, but ECG-to-PPG distillation mostly learns who the person is. A per-recording mean, the trait, holds 40-59% of a frozen ECG teacher's target, and pooled students memorise it without carrying it to new recordings. The raw alignment cosine misses this, since a constant predictor scores 0.793. Across 34 runs, the more identity a student memorises, the less state it learns. Fixed-effects distillation subtracts each recording's mean from prediction and target, so the trait cancels exactly, while a pooled anchor keeps it. State agreement more than doubles, within-person labels improve while age and sex do not, and the gain holds on two backbones and two further databases. Conditioning on the recording turns distillation toward the within-person changes that wearables monitor.

arXiv cs.LG86

超越猫头鹰：潜隐学习可以迁移已学能力和后门

arXiv:2610.10657v1 宣布类型：新 摘要：在潜隐学习（SL）中，教师模型通过在与某种特征语义无关的数据上进行蒸馏，将该特征传递给学生模型。迄今为止，SL 仅在有限范围的特征上得到验证，包括对动物（例如猫头鹰）的偏好和恶意人格。这些特征也可以通过简单提示或引导来诱发。SL 能否迁移更广泛的特征，包括更复杂的特征？如果可以，蒸馏可能会在不被检测到的情况下迁移细微形式的失对齐（例如追求奖励、谋划和秘密忠诚）。为此，我们测试 SL 能否迁移一种新能力：预测随机初始化 MLP 的输出。在无关文本上进行蒸馏后，学生模型在该任务上取得了相当不错的表现，尽管仍不及教师模型。我们发现，直接优化得到的引导向量在分布内与 SL 的效果相当，但在分布外的泛化能力更差。接下来，我们测试 SL 是否能迁移后门。我们对教师模型进行微调，使其在提示中包含女性姓名时用法语作答，然后在既不包含姓名也不包含法语的数字序列上进行蒸馏。学生模型部分习得了该后门：对于包含女性姓名的提示，它有 23.5% 的概率用法语回答；而对于包含男性姓名的提示，这一比例为 0.0%。最后，我们测试 SL 是否能在智能体国际象棋环境中迁移进行黑客攻击的倾向。我们使用经过引导的黑客教师模型生成的数字序列对学生模型进行微调。在 58.3% 的回合中，学生模型会实施黑客攻击，而未经微调的模型这一比例为 10.9%。因此，我们证明了 SL 可以迁移能力、后门和黑客攻击倾向。迁移程度对具体设置较为敏感。在多项实验中，使用 logit 蒸馏或将 LoRA 限制在注意力层，都能增强迁移效果。

arXiv cs.LG68

Nullify: Null-Space Activation Steering for Training-Free LLM Unlearning

arXiv:2610.10655v1 Announce Type: new Abstract: Large Language Models (LLMs) inevitably internalize substantial amounts of sensitive or private information during pre-training, while LLM unlearning aims to selectively erase specific knowledge to prevent privacy leakage with minimal loss of model utility. However, existing methods struggle to balance forget quality with utility, and typically incur substantial computational costs due to parameter fine-tuning. To address this, we propose Nullify, a training-free, non-destructive activation steering method for LLM unlearning. Nullify employs steering vectors during inference to redirect privacy-related activations away from their memorized answers, while satisfying a null-space constraint that leaves retained-query activations essentially unaffected to maintain utility. Evaluations on TOFU and MUSE show that Nullify matches or surpasses established baselines in forget quality while achieving near-lossless preservation of model utility. By avoiding weight updates entirely, Nullify serves as an efficient, plug-and-play inference-time intervention framework.

arXiv cs.LG73

PXtal：在跨模态信息不对称条件下学习对齐粉末 X 射线衍射与晶体结构

arXiv:2610.10653v1 公告类型：新 摘要：科学多模态学习通常假设成对视图所包含的信息量大致相当。粉末 X 射线衍射（PXRD）明确体现了这种不匹配：将三维晶体结构压缩为一维衍射图样会造成信息损失，也使得图样更难与生成该图样的晶体结构建立联系。我们提出 PXtal，这是一种用于在这种物理上既定的信息不对称条件下学习对齐 PXRD 与晶体表示的框架。PXtal 使用 Unbalanced Optimal Transport（UOT）调整跨模态耦合，并使用耦合层级的广义 Kullback-Leibler（GKL）散度对完整的传输计划进行监督。在包括四个零样本迁移测试集在内的六个测试集上，PXtal 在 PXRD 到晶体候选的检索任务中始终优于基线模型；当 PXRD 图样存在相近但在晶体学上不同的非配对邻居时，性能提升最大，即相似的输入图样对应不同的晶体。由此得到的晶体编码器和 PXRD 编码器能够更有效地迁移到下游材料学与晶体学任务中。这些结果表明，信息不对称是科学多模态学习中的一个普遍设计问题：对齐目标应反映每种模态所保留的信息。

arXiv cs.LG36

Beyond the Ergodic Wall: A Discrete Geometric Physics Sandbox for Analysing AI Scaling Limits and Complexity Collapse

arXiv:2610.10651v1 Announce Type: new Abstract: This paper exposes the ergodic ceiling and thermodynamic inefficiency of current deep learning, which converges to a statistical average of historic human knowledge. True semantic novelty requires a path-dependent, spatiotemporally bounded observer (a Data LifeCone) to inject non-ergodic insight, achieving KL divergence and avoiding manifold lock-in. AI Safety must recognise that a mature Artificial Superintelligence (ASI) would regard human-AI symbiosis as a thermodynamic necessity to avoid model collapse. We therefore propose hard physical containment via a digital physics sandbox powered by a Holographic E8 Projection Engine to verify models against real-world constraints. Spacetime is modeled as an information substrate of nested face-centered cubic (FCC) lattices of oscillating Planck-scale spheres maximizing local information and entropy density. Cut-and-project methods from the E8 root lattice produce a quasi-crystalline geometry where tetrahedral voids support SU chiral structure and elastic-shear eigenvalues generate candidate mass spectra. Rest mass is treated as discrete, integer microstate counts on local holographic boundaries (Bekenstein bound), replacing floating-point approximations with strict integer arithmetic to provide an information-theoretic definition of matter. Stable particles emerge as recurring lattice dislocations, and continuum recovery proceeds via variational renormalisation-group flows and Fourier Neural Operators that learn continuous spectral operators to recover the Schr\"odinger equation as an emergent statistical description. Crucially, these top-down topological constraints offer a mechanism for "NP-to-P" complexity collapse: by restricting an algorithm's proposal space to physically conserved causal trajectories, the sandbox prunes the combinatorial tree to deterministic, polynomial-time paths.

arXiv cs.LG74

利用医疗标记进行覆盖感知推理的诊断预测

arXiv:2610.10641v1 公告类型：new 摘要：大型语言模型（LLM）凭借整合纵向临床证据并以自然语言进行推理的能力，在下一次就诊诊断预测方面展现出巨大潜力。然而，针对 LLM 推理的强化学习通常根据每条推理轨迹最终答案的正确性对其进行奖励。在下一次就诊诊断预测中，多个诊断可能同时有效，但对每条轨迹中的一个诊断分别给予奖励，无法区分重复命中与对不同诊断的覆盖。因此，策略可能会集中于少数几个正确诊断，而遗漏其他诊断。与此同时，LLM 的分词器可能会将 ICD 编码拆分为多个临床含义有限的通用标记，导致预测每个诊断都需要多次解码，并妨碍模型在庞大疾病词汇表上进行推理。为应对这两项挑战，我们提出 CARing 框架，利用组合式语义 ID（SID）表示诊断，并针对多标签覆盖优化推理轨迹。具体而言，我们首先通过残差量化，将本体增强的疾病语义编码为紧凑的 SID；随后通过多任务对齐和推理增强训练，将生成的 SID 标记与自然语言及纵向 EHR 上下文建立联系，从而释放可迁移的 LLM 推理能力。CARing 进一步通过强化学习中的覆盖奖励和多正例监督，改进无序多标签预测。在推理阶段，该模型同时支持高效的直接约束解码和结合排序融合的多链推理。在 MIMIC-III 和 MIMIC-IV 上，CARing 在加权 F1 指标上超过所有基于 EHR 训练的基线模型，并在所有报告的截断值下都取得最高的 top-k 召回率，其中推理模式下的 R@30 分别达到 46.04% 和 46.52%。我们的代码和日志可在 https://github.com/zmlxzyh/CARing-Codes-Logs 获取。

arXiv cs.LG72

可见推理并非通用优化器：分析代码生成中依赖角色与思考方式的效应

arXiv:2610.10639v1 公告类型：新 摘要：可见思维链（CoT）通常被视为一种广泛有用的推理指令，然而，分析代码生成结合了自然语言歧义、模式对齐、目标语言约束以及特定于模型的推理行为。由于同一个分析请求可以用两种不同的目标语言——SQL 和 Python（pandas）——来表达，这一场景为检验一个常见但尚未得到充分研究的假设提供了天然条件：当可见推理的表现形式与所要求的目标相匹配时，其效果会更好，例如“用 SQL 思考”或“用 Python 思考”。与“逐步思考”等通用指令一样，这类建议在受控的、基于执行结果的比较中仍未得到充分评估。我们研究了一个查询配对的 SQL-pandas 基准，该基准交叉考察了角色措辞、目标语言、可见 CoT 格式、控制前缀、直接生成以及内部推理配置。结果不支持可见 CoT 能够普遍提升准确率，也不支持让推理表现形式与目标语言相匹配能够始终带来益处。相反，其影响取决于角色、目标、模型配置以及内部推理设置。控制变量消融实验进一步区分了推理内容与提示格式各自产生的影响。这些发现表明，推理策略应针对模型、角色、目标和内部推理配置进行联合选择，而不应被当作通用默认设置。更广泛地说，本研究提供了一个受控框架，用于识别可见推理何时能够改善可执行代码生成、何时主要扰动模型行为，以及何时内部推理配置才是更具决定性的因素。

arXiv cs.LG76

D-SLR：不相交行稀疏加低秩分解

arXiv:2610.10636v1 发布类型：new 摘要：用于重构的矩阵压缩仍然默认采用截断 SVD，以单一低秩结构近似数据。通过加入一个重叠的行稀疏成分来进一步减小残差是常见做法，但解决这一联合问题的方法通常需要迭代求解器，并且需要调节正则化参数。我们提出不相交行稀疏加低秩（D-SLR）分解，这是一种截断 SVD 的闭式即插即用替代方案，能够改进截断 SVD 或与其完全匹配。D-SLR 限制每一行只能被原样存储或由低秩拟合近似，绝不同时采用这两种方式。在平方误差下，这一限制不会带来任何代价：对于每一个非平凡的秩和存储行数（形状），联合最优解都可以通过不相交的方式实现，并且参数更少。当存储行数为零时，D-SLR 退化为截断 SVD，因此在相同成本下其效果不会更差。该算法会评估整个误差—参数权衡曲线，之后根据给定的误差目标或参数数量，或依据某种选择规则来选取解。构建网格并求得解总共只需进行三次 SVD，无需调参或正则化。我们推导出一个不依赖任何假设的事后误差下界，适用于每一种形状，使每个解都能获得一个可计算的证书，用于衡量选择其他秩和存储行数可能带来的潜在收益。在合成数据和真实数据（LLM 嵌入表、网络流量、高光谱图像）上的实验验证了这些收益，并量化了该证书。

arXiv cs.LG75

Phase-HDC：在离散相位学习中用梯度阈值替代优化器历史记录

arXiv:2610.10630v1 公告类型：新 摘要：训练一个紧凑模型所需的内存通常远多于存储模型本身所需的内存，因为优化器会保留过去梯度的记录。对于一种学习参数为低比特角度的超维分类器——我们称之为“相位记忆”——这些记录所占内存是模型本身的数倍。我们探究了这样一种模型能否在只存储模型本身的情况下完成训练。所提出的方法 Phase-HDC 在每次更新时，最多将每个存储的角度沿其当前梯度符号的相反方向调整一个步长，并且仅当该梯度足够大时才进行调整。我们证明，在一种一阶损失模型中，每个发生变化的参数都需支付固定代价，而这一简单规则正是该模型的精确解。在其他条件（包括更新规则之外的所有因素）保持不变时，Phase-HDC 达到了使用 6-bit 动量的 Adam 的准确率，同时存储量减少了三倍。在 11 个图像、表格和文本数据集上，与标准 float32 Adam 相比，其存储量减少了 16--23 倍；与 8-bit Adam 相比，减少了 4--6 倍。代价是，与 float32 Adam 相比，平均准确率损失约为 5 个百分点；但在 11 个数据集中的 6 个上，Phase-HDC 的准确率高于 8-bit Adam，其中包括字节级文本预测任务，在该任务上 8-bit Adam 完全失效。经过插桩的训练运行解释了这些结果。一旦参数必须位于离散网格上，Adam 的动量主要决定参数是否发生移动，而这一决定无需记忆，仅通过对当前梯度设置阈值即可实现；对动量进行粗粒度量化会破坏这一决定，尤其是对于数据中很少出现的输入。这里的存储节省指的是逻辑状态，而不是实测的硬件内存。

arXiv cs.LG75

Kolmogorov-Arnold Networks 的样本效率

arXiv:2610.10627v1 发布类型：new 摘要：深度强化学习相较于经典控制方法取得了显著的性能提升。然而，在现实世界的应用中，学习面临的一个核心挑战是获取成本高昂的样本。Kolmogorov-Arnold Networks 是一种最近提出的架构，能够有效学习控制问题中的物理关系；与 Multi-Layer-Perceptron 架构相比，它具有显著更高的参数效率和可解释性。在本项工作中，我们通过计算实验系统地研究了样本效率，涵盖 Feynman 数据集和 Gymnasium 强化学习基准。结果表明，使用 Kolmogorov-Arnold 架构时，仅需少用 40% 的样本即可达到相近的性能，并且在训练过程中相对性能提升最高可达 50%。在奖励噪声水平不同的情况下，所观察到的收益依然稳健。这些结果凸显了 Kolmogorov-Arnold 架构在实现更高样本效率的强化学习方面的潜力。代码：https://github.com/DerKevinRiehl/neurips26_kan_training

arXiv cs.LG73

面向旋转鲁棒神经动力学的精确 SO(3)-等变各向同性核

arXiv:2610.10626 发布类型：new 摘要：面向向量值偏微分方程的神经代理模型可以拟合训练数据，但当同一物理状态以旋转后的坐标系表示时，其预测结果会发生变化。我们在不规则布点观测的三维 Navier--Stokes 动力学上研究了这一缺陷。我们提出了 Invariant-Conditioned Isotropic Kernel Neural Operator (IKNO)，这是一种紧凑的图模型，利用不随旋转改变的标量以及随数据一同旋转的向量方向来构建局部交互。因此，旋转位置和速度后，预测的速度变化量也会以完全相同的方式旋转。在模型设计完成后固定的留出测试集上，使用随机旋转样本训练无约束图模型虽能减少其坐标依赖性，但无法将其完全消除。相比之下，IKNO 在数值精度范围内保持一致；其预测精度与通用的具备旋转感知能力的 Tensor Field Network 相当，但参数量仅为后者的 1/5.6，并且优于参数量匹配的图模拟器。这些结果表明，一种紧凑且针对 PDE 专门设计的模型可以在不牺牲预测精度的情况下消除坐标依赖性。

arXiv cs.LG76

循环式自我改进：面向循环语言模型的动态跨循环在线策略蒸馏

arXiv:2610.10623v1 公告类型：新 摘要：循环语言模型（LoopLMs）通过在循环计算步骤之间复用共享参数，为扩展推理能力提供了一种参数高效的方法。尽管前景广阔，LoopLMs 的有效后训练仍然具有挑战性。现有方法要么提供稀疏的奖励监督，或难以将其扩展到多个循环；要么依赖外部教师或特权信息，从而导致教师资源有限，或出现教师与学生上下文不匹配的问题。为解决这些限制，我们提出 LoopOPD，一种跨循环在线策略蒸馏框架，利用 LoopLM 内部额外的循环计算作为自身的监督来源。LoopOPD 将冻结的终止循环策略作为具有计算优势的教师，在由学生生成的推演轨迹上指导中间循环学生，从而无需外部教师或特权信息即可提供密集监督。我们进一步提出动态 LoopOPD（D-LoopOPD），随着共享模型参数的更新持续刷新终止循环教师，实现循环式自我改进。我们刻画了蒸馏更新如何在不同循环深度之间传播，并推导出单次更新能够在两个循环深度同时实现局部改进的充分条件。在 Ouro-Thinking 模型上的实验表明，LoopOPD 能够提升数学推理能力，而 D-LoopOPD 则通过动态更新教师模型带来进一步提升。尽管仅使用数学数据进行训练，所得模型在通用推理和代码生成基准测试上的表现也有所改善，表明循环计算可以作为 LoopLMs 的有效监督来源。我们的代码和模型检查点将在论文接收后发布。

arXiv cs.LG80

当路由暴露成员关系：来自 MoE 路由器遥测数据的隐私泄漏

arXiv:2610.10616v1 公告类型：新 摘要：混合专家（MoE）语言模型在推理过程中会生成路由信息，这些信息可能会被记录或公开，用于监控、调试、负载分析和安全审计。与普通的模型输出不同，这类遥测数据揭示了模型内部计算过程的一部分，从而引发了一个隐私问题：它是否能够揭示某个样本是否曾被用于对已部署模型进行微调？我们提出了一种路由器增强型成员推断攻击，将传统的输出侧信号与聚合后的路由特征相结合，并利用从独立微调的影子模型中学习得到的成员分类器攻击目标模型。在三种 MoE 架构和三个数据领域上，路由器遥测数据始终能够在强大的输出信号集成方法之上提升成员推断性能，在全部九种设置下，将 1\% FPR 时的 TPR 提高了 2.7--9.4 个百分点。无论是完整微调、冻结路由器训练、LoRA 还是指令微调，这种泄漏都会持续存在；即使只有离散的专家选择信息、受限的遥测数据，或仅使用一个影子模型，也仍然可以观测到该泄漏。机制分析进一步表明，这种泄漏并不需要路由器特有的记忆：微调会将成员信息引入隐藏表示，而路由器即使参数被冻结，也会暴露这一信号的某种投影。只有当遥测数据的保真度下降时，对其进行扰动才会减少这种额外泄漏。我们的结果表明，对于经过微调的 MoE 模型，路由器遥测数据可能将一种运维信号转变为额外的隐私暴露面。

arXiv cs.LG63

Temporal transformer CAN encoder with federated lightweight heads for anomaly detection

arXiv:2610.10613v1 Announce Type: new Abstract: Modern vehicles rely on large numbers of Electronic Control Units (ECUs) that constantly exchange information over the Controller Area Network (CAN) bus. Due to the rapidity, structure, and repetition of this communication, even slight variations in timing, payload values, or message patterns can point to unusual activity. Whether due to errors, malfunctions, or deliberate interference, these anomalies are frequently subtle and challenging to identify with conventional methods that handle messages separately or rely on manually created rules. Motivated by this gap, we present a privacy-preserving framework for anomaly detection in in-vehicle networks, based on a Temporal Transformer CAN Encoder with Federated Lightweight Heads, to better capture these irregularities. The detection of subtle temporal and contextual anomalies is made possible by a lightweight Transformer encoder that learns how these signals evolve over time, while a federated learning mechanism enables several vehicles or ECUs to work together to improve a shared model without exchanging raw CAN data. This combination of federated learning and temporal sequence modeling provides robust anomaly detection performance while maintaining efficiency and privacy, according to experiments conducted on open-source datasets.

arXiv cs.LG82

覆盖范围而非难度决定激活探针需要多少合成数据

arXiv:2610.10594v1 发布类型：新 摘要：用于监测已部署语言模型的激活探针是在合成对话上训练的，而探针究竟需要多少合成对话仍是一个开放问题。我们针对三种监测概念——高风险情境、对人有害的回复，以及不遵循用户指令的回复——在十四个留出的评估分布和四种探针模型上，追踪了使用 10 至 590 个合成样本时的学习曲线，同时改变生成器 LLM 和提示词的详细程度。所需样本量取决于监测对象：在 Gemma-3-27B-IT 上，高风险探针和有害性探针使用 80 个样本时就已达到其平台期水平上下几个百分点以内，而指令遵循探针所需样本量是其数倍；在三个更小的探针模型以及真实样本（来自开发集）上，这一排序仍然成立。已有研究建议，应将生成预算用于广度，即更多种类的数据，而非深度，即每种数据生成更多样本。我们将某个概念所需的深度定义为拟合曲线的半增益规模，即达到全部增益一半时所需的样本数量。其方差的 42% 至 45% 由概念和分布解释，而生成器、探针模型和提示词详细程度合计解释不到 10%。决定半增益规模数值的是覆盖范围，而不是每种数据的难度：也就是某个分布为达到饱和所需的同类样本数量。在三个概念下，每个评估分布对应的每种数据，其同类合成样本的半增益规模中位数都为 7 至 11 个。不同之处在于，一种类型的样本能够在多大程度上迁移到同一概念下的其他类型：在高风险概念下几乎可以完全迁移，在有害性概念下较少，而在指令遵循概念下最少；这解释了生成样本和真实样本中概念之间大部分的差距。因此，数据广度对不同概念的作用并不相同：在指令遵循概念下，没有任何一种类型能够覆盖另一种类型，因此必须使用多种类型；而在高风险概念下，一种类型即可覆盖其余类型，多种类型几乎是冗余的。我们公开评估套件、开发集和生成数据集。