全部动态

本页 40 条
10月12日 周一
arXiv cs.LG73

VEPA:面向 LLM 智能体终身护栏的经验证演进式形式化策略

arXiv:2610.12813v1 公告类型:交叉发布 摘要:LLM 智能体越来越多地操作私有文档和可变系统状态等敏感资源,这可能带来严重的安全与安保风险。因此,运行时护栏应在执行可能不可逆的智能体操作之前,作出可检查、可复现且确定性的允许或拒绝决策。然而,安全与安保要求在部署前很少完备,因此护栏需要随着新风险的出现而持续适应。现有自适应护栏会从部署经验中学习,但通常通过 LLM 推理来应用安全策略,这可能导致决策不一致和策略解释模糊。另一方面,形式化护栏能够以确定性方式执行策略,但通常依赖部署前制定的策略或特定任务策略,缺乏适应新观测风险的机制。我们提出 VEPA,一个用于形式化智能体护栏经验证演进的框架。VEPA 将护栏表示为一种形式化策略,由针对带类型的智能体事件的可执行符号规则构成。在部署过程中,我们使用部署反馈引导 LLM 提出策略补丁,以修复新观测到的失效,同时保留此前正确的行为。随后,确定性验证器仅接受结构有效且无冲突的策略。最后,一个证据感知的授权层仅在已验证规则获得充分支持证据时才将其激活,从而防止不确定的更新直接影响护栏决策。在 ASSEBench-Safety、ASSEBench-Security 和 ATBench 上,VEPA 在扩大确定性形式化策略执行范围的同时,始终优于基线方法。这些提升在更具挑战性的部署条件下同样成立,包括混合流、受损反馈和更长的运行周期。

arXiv cs.LG64

Trainable Quantum-Native Language Models: Dissipative Memory, Capacity Limits, and Hardware Validation

arXiv:2610.12806v1 Announce Type: cross Abstract: We study byte-level autoregressive language models whose hidden state is a small quantum register, whose transitions are learned token-conditioned quantum channels, and whose next-byte probabilities come directly from Born-rule measurements without a learned classical vocabulary projection or softmax. In the fixed-width recurrent setting, unitary dynamics cannot contract or reset stored distinctions; experimentally, unitary recurrence loses to its retrained one-token control in every tested variant. Introducing learned dissipation, then token-conditioned dissipative gating, yields useful memory beyond one token. The gated model improves test negative log-likelihood by 0.20 +/- 0.03 nats/byte over retrained parameter-matched one-token controls at context length 12, and by 0.29-0.33 at longer contexts, but never beats retrained two-token controls. A key-value recall benchmark shows a width-dependent threshold: one stored pair meets the pre-registered accuracy bar at 4-5 memory qubits, two pairs at 6-7, and three pairs remain unsolved. This is consistent with, but does not establish, a capacity explanation for the short effective memory horizon. As a secondary result, one-qubit models reach lower stationary memory entropy than a heuristically optimized classical predictive frontier at matched error in 9/9 seeds. On IBM Heron hardware, computational-basis memory-register distributions show signal against a uniform null in 12/12 circuits across prefixes of up to 12 tokens, next-bit probabilities pass pre-registered accuracy bars at four-token prefixes, and bytes are generated autoregressively by sampling physically measured bit probabilities. The tested models are classically simulable and substantially worse than classical n-gram baselines. We claim no computational quantum advantage.

arXiv cs.LG68

Polyconvex neural-network surrogates with learned paired directional features for anisotropic hyperelastic homogenization

arXiv:2610.12801v1 Announce Type: cross Abstract: Approximating the effective anisotropic hyperelastic response of a representative volume element (RVE) requires constitutive expressiveness, mechanical consistency, and computational efficiency. This work develops a polyconvex physics-augmented neural network (PANN) construction based on paired directional features whose orientations, stretch powers, and determinant exponents adapt to the RVE response. The features are composed with convex, componentwise nondecreasing cores, realized as an input-convex neural network (ICNN) or an input-convex Kolmogorov-Arnold network (ICKAN). The complete energies satisfy objectivity, an exactly stress-free reference, two-dimensional polyconvexity, and prescribed growth conditions by construction, and nonnegative energy is certified separately for the selected models. On a multicavity and a single-cavity periodic cell, these energies are compared with an Unconstrained energy, trained identically without the curvature and growth constraints, and with three hyperreduced projection-based reduced-order models (HPROMs), which serve as physics-based references. On the first cell, learning the features lowers the stress error sevenfold, and six features suffice on both. The constrained energies remain below one percent in stress error and in the displacement error of a tensile-coupon FE$^2$ application with a full-order reference, whereas the Unconstrained energy and the HPROMs are more accurate inside the data. Beyond the data, however, the Unconstrained energy loses rank-one convexity, and under confined compression it stops below half of the 500 MPa that the constrained energies reach. These results quantify what the guarantees cost in accuracy and what they secure beyond the data.

arXiv cs.LG

Credit Attribution in Timestamped Soccer Networks Using Multivariate Hawkes Processes

arXiv:2610.12775v1 Announce Type: cross Abstract: The two main traditional player statistics used in soccer are goals scored and assists, with an assist credited when a player's pass immediately leads to a teammate scoring. Attributing credit solely to the player who makes the final pass before a goal fails to account for the contributions of multiple players in the buildup to the goal. Consequently, traditional statistics provide only a limited view of player involvement. In this paper, we use multivariate Hawkes processes (MHPs) to model timestamped sequences of passes and shots, which form continuous-time networks. MHPs provide a branching mechanism where an event (a pass or shot) can be generated as a child of a previous event, which allows us to quantify each pass's contribution to a subsequent pass or shot. This approach distributes credit across multiple passes within an attacking sequence, making it applicable across a wide range of situations. Building on this framework, we propose expected goal involvements (xGI), a metric that measures a player's involvement in creating goal scoring opportunities based on their passes. This metric identifies players who play key roles in creating goal scoring opportunities even if they do not make the final pass, offering a more comprehensive measure of attacking involvement and player performance.

arXiv cs.LG71

Trajectory Prediction for Autonomous Driving in Road Construction Zones Using Human-in-the-Loop Driving Data

arXiv:2610.12747v1 Announce Type: cross Abstract: Ego-trajectory prediction in urban roadway work zones remains an underexplored topic in autonomous-driving research, with limited dedicated data and systematic evaluation. Roadway work zones involve temporary changes in the lane layouts and connectivity of the roads, complicating interactions with surrounding vehicles and workers. These changes make future vehicle trajectories difficult to predict. To address these challenges, we collect WZDrive-Data, a multimodal dataset comprising 1,183 work-zone driving segments from 33 drivers, using a human-in-the-loop driving simulator with prerecorded worker animations and VR-controlled workers. Driver control and gaze signals are recorded from human participants, while vehicle trajectories and scene information are obtained from the simulator. Our proposed multimodal ego-trajectory predictor, WZDrive, achieves a mean minADE (K=6) of 1.09 m and minFDE (K=6) of 2.12 m. Compared with the best-performing state-of-the-art baseline for each metric in our benchmark, WZDrive reduces minADE (K=6) by 7.18% and minFDE (K=6) by 13.18%. This research addresses a gap in autonomous-driving research on urban work zones by combining human-in-the-loop data collection with multimodal learning from human driving signals and in the context of work zones.

arXiv cs.LG62

Minimax Rates for Estimating Powered Sliced Wasserstein Functionals

arXiv:2610.12740v1 Announce Type: cross Abstract: We study the two-sample estimation of the powered Sliced Wasserstein functional under projected $r$-th moment assumptions. Let $N=n\wedge m$, where $n$ and $m$ are two sample sizes. We establish the minimax rates in the noncritical regimes. When $p 2p$ it is of order $N^{-1/2}$. In both cases, the natural plug-in estimator is minimax rate-optimal. At the endpoint $r=p$, the minimax risk is bounded away from zero, so uniformly consistent estimation is impossible. At the critical order $r=2p$, we obtain the exact worst-case rate of the plug-in estimator and a lower bound for the minimax risk, leaving a logarithmic gap between the lower and upper bounds. The exact minimax rate in this critical case remains open.

arXiv cs.LG75

StereoEngine:无需双目相机即可扩展双目视觉双臂机器人数据规模

arXiv:2610.12734v1 公告类型:交叉收录 摘要:双目视觉正越来越多地用于机器人操作,越来越多的研究将其融入视觉-语言-动作(VLA)模型。然而,大规模真实世界双目视觉数据集的匮乏,仍是此类策略规模化发展的根本瓶颈。近期的生成式单目转双目方法主要面向 VR 内容和自动驾驶等场景。此前尚无研究为机器人操作生成逼真的合成双目数据,该领域的双目监督信号主要来自仿真。我们提出 StereoEngine,引入两种互补的方法来合成机器人操作所需的双目数据。第一种方法利用估计深度,将单目(左视角)视频变换为右视角,并使用视频扩散模型修复因遮挡解除而显露的区域,从而无需使用真实双目数据进行训练,即可将单目示范转换为双目视图对。第二种方法先在仿真中渲染双目视图对,再联合生成两个照片级逼真视图,无需任何真实视频作为输入,即可生成左右一致的数据。StereoEngine 在生成双目数据的质量上优于现有基线方法,且将其合成数据用于策略的中期训练或特定任务微调,能够显著提升下游双臂操作性能。项目网站:https://stereoengine.github.io

arXiv cs.LG78

Zebra3D:面向可扩展三维重建的高效混合基础模型

arXiv:2610.12727v1 公告类型:交叉发布 摘要:前馈式三维基础模型能够从无位姿图像中重建场景,但全局注意力的二次复杂度开销限制了其向大规模视图集的扩展。我们提出 Zebra3D,这是一种逐层稀疏—线性混合注意力框架,无需从头训练即可提升预训练三维基础模型的效率。在 VGGT、HunyuanWorld-Mirror 和 $\pi^3$ 中,我们观察到一种大体一致的随网络深度变化的模式:早期阶段对线性注意力替换的容忍度相对较高,而敏感的中后期阶段往往呈现出更集中的注意力。Zebra3D 利用这一结构,为容忍度较高的阶段分配以线性注意力为核心的注意力机制,为敏感阶段分配分块稀疏注意力,并使用地标摘要保留未选中的上下文。闭式仿射拟合支持免训练转换,而可选的转换后训练则将任务监督与输出蒸馏、关系特征蒸馏相结合。在全部三种骨干网络上,Zebra3D 均在高注意力稀疏度下实现了出色的重建精度和相机位姿精度,其免训练变体的表现仍接近对应的稠密模型。端到端加速比随输入视图数量增加而提升,在 1,024 个视图时最高达到 $7.1\times$。经过转换后训练,Zebra3D-VGGT 在平均注意力稀疏度为 94\% 时,RealEstate10K AUC@30 达到 80.5\%,与使用相同数据和训练方案微调的稠密 VGGT 相差不超过 1.1 个百分点。

arXiv cs.LG71

A Distributional Framework for Behavioral Characterization and Risk Management of Large Language Models

arXiv:2610.12718v1 Announce Type: cross Abstract: Large language models are stochastic by nature. Repeated executions under the same experimental conditions may produce different responses. We propose a framework that can be applied across the model lifecycle, from task design to residual behavior and continuous monitoring. The method consists of building an empirical response distribution estimated through Monte Carlo simulation. Semantic interventions are introduced to study how these distributions change. We illustrate the framework with two different tasks, borrower classification and music album recommendation, and quantify the resulting distributional changes. The framework is linked to model risk management and governance by measuring robustness and control effectiveness, and characterizing residual behavior. Since it relies only on observed responses, it can also be applied to black-box models accessed through commercial APIs.

arXiv cs.LG70

PVBRF-ID: Private, Verifiable and Byzantine-Robust Federated Intrusion Detection for Oilfield Enterprise Networks

arXiv:2610.12704v1 Announce Type: cross Abstract: Federated learning allows dispersed oilfield sites to train a shared intrusion detector without pooling network traffic, yet its aggregation step forces a choice between privacy and robustness. Byzantine-robust rules must read each update in the clear, whereas secure aggregation hides updates from the checks that detect poisoning. We present PVBRF-ID, a two-server protocol in which sites secret-share their updates and the servers jointly test each update's norm and its cosine similarity to a reference computed on a small clean root set, revealing one accept bit per site. Commitments, Merkle roots and a homomorphic audit make the inclusion of accepted updates publicly verifiable. On a 15-class Edge-IIoTset task with ten sites holding differently distributed data and ten random seeds, PVBRF-ID matched FedAvg on clean data (macro-F1 0.708 vs. 0.706) and withstood sign-flip attacks (0.696 vs. 0.091). Against an adaptive attacker controlling 30% of sites, PVBRF-ID with secure clipping reached 0.641, whereas FedAvg fell to 0.046; plaintext multi-Krum remained higher by 0.052. An update-inference attacker recovered each site's class mix from plaintext updates (Spearman correlation 0.81) but not from one server's view. Backdoors passed the filter, and noise-based mitigation gave only heuristic protection. The implementation uses Python 3.13, NumPy 2.3, 30 rounds and 10 seeds.

arXiv cs.LG69

Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation

arXiv:2610.12703v1 Announce Type: cross Abstract: High-risk applications of deep learning require precise characterization of model decision-making. We conduct a systematic explainability evaluation across Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and Vision Mamba (ViM). Unlike predominantly qualitative studies, we quantify the alignment between model attention and human visual perception using eye-tracking ground truth. We formalize this comparison via a standardized normalization pipeline mapping post-hoc attribution maps and gaze heatmaps onto spatial probability distributions. Alignment is evaluated across fine-grained classification, medical imaging, and emotion recognition using architecture-specific XAI methods. We measure fidelity using Jensen-Shannon similarity, Chi-square similarity, and Pearson correlation, three complementary estimators capturing support overlap, mass discrepancy, and structural co-variation. Our results reveal distinct attention dynamics: CNNs tend to exhibit a more global focus, with spatially more diffuse saliency maps that correlate well with human perception in general contexts, while ViTs demonstrate fine-grained, patch-level attention, often focused on discriminative microfeatures. The Mamba architecture shows biased behavior: it achieves strong explainability in medical tasks but exhibits divergent intensity distributions in natural images; our analytical characterization of the metrics makes this dissociation explicit, since near-unit support overlap is attained simultaneously with a vanishing linear correlation. We further analyze the effect of backbone freezing and ensemble strategies on model interpretability. The results highlight how architectural and training choices influence the alignment between model saliency and human visual attention.

arXiv cs.LG75

Product Monte Carlo:密度乘积下的期望

arXiv:2610.12702v1 公告类型:交叉发布 摘要:汇总独立计算的子样本推断结果,是大规模贝叶斯计算、联邦学习和意见汇聚中一个众所周知的难题。核心问题是近似一个概率密度与各个密度之积成正比的分布。我们提出 Product Monte Carlo(PMC),这是一种仅使用来自各个密度的蒙特卡洛样本,即可计算密度乘积下任意期望的算法。我们通过一族可计算的条件期望来表示难以直接计算的密度乘积下的期望,从而将其转化为回归问题。除点估计外,我们的全贝叶斯回归框架还提供一致的不确定性量化和具有理论依据的诊断方法。我们为 PMC 估计量建立了一致性的理论保证,并刻画了其收敛速率。在改变各组成分布的数量、维度和异质性的模拟实验与真实数据实验中,相比现有方法,PMC 以更低的计算成本实现了更高的估计精度、校准更好的不确定性覆盖率以及更强的可扩展性。此外,PMC 无需计算密度值,因此适用于基于模拟的推断;我们通过一项蛋白质图像应用展示了这一点。

arXiv cs.LG62

Toward Maximizing Algebraic Connectivity via Density-Dependent Backbone Construction and Reinforcement Learning Completion

arXiv:2610.12697v1 Announce Type: cross Abstract: This paper studies an open network design problem of constructing graph topologies with maximum algebraic connectivity under fixed numbers of vertices and edges. A density-dependent approach is proposed that decomposes the design task into two stages: a high-quality, low-cost structural backbone construction and a reinforcement learning completion stage. The method selects the backbone based on density, using complete multipartite graphs in dense regimes and Cayley graphs in low-to-intermediate regimes, and then refines the remaining edges through a learning-based procedure. Numerical results show that the proposed method achieves high algebraic connectivity with favorable runtime scalability, outperforming existing methods. These results support a structural-family-first strategy for scalable network design.

arXiv cs.LG77

MagicBeanStalk:通过递归数据合成摆脱 RLVR 零优势陷阱

arXiv:2610.12659v1 公告类型:交叉发布 摘要:以数据为中心的 RLVR 扩展一直侧重于收集或生成可验证的训练问题。然而,仅凭语料库规模并不能判断数据集是否能为目标策略提供有效的学习信号。为指导有效 RLVR 数据的合成,我们通过实验,将数据集的有效性刻画为其所含问题、目标策略和训练 rollout 预算的函数。基于这一刻画,我们发现,一个汇集了广泛使用的 RLVR 数据集的大型开源数学语料库几乎完全陷入了零优势陷阱:对于固定的目标策略和 rollout 预算,其中约 88% 的问题要么在每次 rollout 中都能解决,要么在每次 rollout 中都无法解决。筛选可以提高剩余有效问题的集中度,但它只能进行选择,同时丢弃难题。因此,有效的合成问题应当要求使用解决这些难题所需的推理操作及其组合。不过,这些问题的难度必须经过校准,使部分 rollout 成功、部分失败,并且至少有一个正确解答能够在训练预算内完成。直接提示教师模型对难题进行变式改写并不能达到这一目的。在这些实证原则的指导下,我们提出了 MagicBeanStalk,一种结合自顶向下分解与自底向上递归数据合成的算法。它在保留种子问题推理蓝图的同时,将种子问题扩展为一系列在实质上全新、且针对策略和预算进行了校准的训练问题。在 8 个数学基准测试中,在 16K token 预算下,使用 MagicBeanStalk 问题持续进行 RLVR 训练,使 gpt-oss-120b 的成绩从 61.2 提升至 69.8(+8.6),使 DeepSeek v4Flash 从 72.4 提升至 77.9(+5.4)。使用完整的 7.8 万道问题源语料库进行训练仅提升了 1.1 分,而使用其中经筛选得到的最佳真实数据子集则提升了 5.8 分。在金融领域,我们仅基于 340 个解答就合成了超过 1.4 万道可验证问题,并在 SecQue 和 FinanceBench 上分别比基础模型提升了 6.9 分和 2.4 分。

arXiv cs.LG76

SpireBench:使用 Slay the Spire 评估智能体模型的长程策略对局能力

arXiv:2610.12596v1 公告类型:交叉收录 摘要:智能体大语言模型如今已能处理复杂任务,但对其长程策略推理能力的研究仍显不足:大多数智能体基准测试流程较短,且以独立回合为单位,而模型在长程任务中的表现又难以拆解为已有知识、推理能力与运气各自的贡献。我们提出 SpireBench,让模型游玩完整的商业牌组构筑游戏 Slay the Spire(StS);在该游戏中,早期决策决定了长期成败。我们在游戏最高难度下测试了 20 个大语言模型(LLM),发现前沿模型超越了启发式基线,并达到了普通人类玩家的平均水平,但仍远逊于专家玩家。随着单局进程推进,这一差距进一步扩大:最强模型通过第一幕的频率与专家相当,随后却逐渐落后。强弱模型之间的区别在于每场战斗中的表现;所有模型都会误判整局游戏中的风险,倾向于保留生命值,而不是消耗生命值来增强实力。对所有游戏术语进行全面且内部一致的重命名后,我们发现,模型的表现部分依赖于对游戏的先验知识;此外,模型也未能从自己在多局游戏中记录的记忆中获得提升,这些记忆往往有误且过于谨慎。我们的测试框架首次向模型提供了完整的原版游戏,并与现有框架进行了比较。总体而言,SpireBench 提供了一个尚未达到性能饱和的长程基准,并配备了用于定位智能体失败环节及原因的工具。

arXiv cs.LG74

Fly-by-Code:基于主动视觉与物理反馈的空中操作具身编程智能体

arXiv:2610.12549v1 公告类型:交叉收录 摘要:具身编程智能体通过生成可执行程序,并根据执行反馈对其进行改进,将自然语言指令转化为机器人策略。这种改进过程假定执行反馈能够揭示任务是否成功,并在失败后提供足够的证据来指导程序修订。空中操作在两个方面违背了这一假定。一方面,载荷约束限制了机载相机的覆盖范围,使对任务至关重要的结果处于机器人视野之外(部分可观测性);另一方面,强动态耦合与有限的控制裕度使物理交互容易失败(交互鲁棒性有限),且失败原因难以明确。反复审视同一份反馈,既无法补回从未采集到的观测,也无法区分现有证据无法辨别的失败原因。为获取这些缺失的证据,我们提出 Active Feedback(AF),一种主动式“观察与探测”框架。当执行反馈仍留有未解问题时,智能体会合成一个独立的诊断程序,在仿真中运行,从信息丰富的视角检查此前未被观察到的结果(观察),或通过有针对性的物理交互检验关于失败原因的假设(探测)。在六项仿真空中操作任务中,每种条件均进行 60 次试验;在相同的编程智能体配置和任务执行次数限制下,与终止时反馈相比,AF 将误报完成的次数从 25 次降至 5 次,并将最终任务成功率从 31.7% 提升至 75%。借助共用的机器人 API,在仿真中通过 AF 改进的任务程序无需修改任务程序源代码,即可在实体空中操作机器人上运行,并在真实世界中完成多阶段操作任务。这些结果表明,可执行诊断能够弥补具身编程智能体执行反馈中的信息缺口,从而同时改善空中操作中的完成状态评估与程序改进。

arXiv cs.LG76

AutoIndexer:通过在编辑链上训练因果模型实现灵活顺序解码

arXiv:2610.12545v1 公告类型:交叉收录 摘要:自回归解码不太适用于非因果顺序的数据,例如交叉引用或对后续内容的依赖,因为与人类的迭代推理不同,其输出无法在事后修订。扩散式 LLM 在一定程度上缓解了这一局限,但仍无法在预先生成的上下文中灵活插入或删除任意数量的词元。我们设计了 AutoIndexer,这是一种采用新颖训练与解码机制的生成模型,通过模拟编辑链,使模型能够在事后通过插入、替换和删除词元,灵活修改先前的输出。该模型学习预测用于开启和结束编辑的标记词元,同时将光标移动到现有词元序列中的相应位置,类似于文本编辑器。训练序列通过对真实标签施加逆向链式编辑操作进行扰动而获得,模型则通过撤销这些逆向操作来学习编辑。这种扰动训练使 AutoIndexer 对错误上下文具有鲁棒性,并能适应代码纠错等编辑任务。我们证明,该方法具备大多数自回归模型和扩散模型所缺乏的生成后上下文编辑能力,同时不会损害其在标准 LLM 基准测试上的表现。代码已发布于 https://github.com/AutodeskAILab/autoindexer。

arXiv cs.LG69

Equal-Budget Monte Carlo Tree Search versus Randomized-Restart List Scheduling for DAG Workflows: A Controlled Equal-Budget Computational Study

arXiv:2610.12542v1 Announce Type: cross Abstract: Monte Carlo Tree Search (MCTS) is often proposed for scheduling directed-acyclic-graph (DAG) workflows, yet most studies never run the comparison that could refute it - giving a much simpler method the same search effort. We run that comparison under a controlled protocol. All arms receive an identical budget - the same number of candidate schedules built and evaluated: an MCTS scheduling method, preregistered as the main arm; a randomized-restart list scheduler with no search tree as the control; and a textbook-UCT MCTS arm as a registered follow-up on the same domain. The analysis plan was locked before data collection; an independent checker confirmed all 768 confirmation-stage schedules valid with correct finish times. At the main budget of 2048 evaluated schedules, MCTS is 2.88% worse than the restarts (95% CI [-3.62%,-2.15%]; the 72 of 96 held-out instances with positive baseline-relative headroom). The deficit is not budget starvation: on the separate exploration pool restarts self-improve by 1.85% where MCTS gains 0.24%, and 700 of 720 of MCTS's first results are merely the warm start it was handed. The studied selection rule locks deterministically onto the first-expanded child, keeping its trials correlated while restarts draw gains from the tails; the causal link to makespan remains an inference. A structural map locates where the tree stops paying: restarts win on sparse irregular workflows, no difference is detectable on regular layered ones, and dense ones leave no room to improve. Swapping the rule for textbook UCT removes most of the deficit (-0.26%, 95% CI [-0.71%,+0.23%]) without reversing its sign. Results are confined to a controlled synthetic domain (N in {8,16,32} tasks, 2-8 processors, budgets up to 2048); we make no claim about larger budgets.

arXiv cs.LG73

LLM 驱动的音色生成与插值中的结构性与表征性能动性

arXiv:2610.12532v1 公告类型:交叉收录 摘要:生成式创意界面通常会显示用户在可能性空间中的位置。这类显示暗示,所展示的数值是从生成产物中测量得出的;然而,在越来越多的系统中,这些数值实际上是由描述自身输出的语言模型预测的,而两者在屏幕上无法区分。大多数创意领域缺乏公认的测量方法来核验这种自我报告,但音频领域具备这样的测量方法。我们对一个实际运行的 LLM 驱动音色界面进行了测量研究,结果表明,用户能够促成的结果与界面所报告的结果可以彼此分离。前者可以被精确刻画:可达集是一个 zonotope,其维度随着每放置一个声音而增加一维,因此可以证明,模型可在其中操作的空间确实由用户掌控。后者则不准确。显示位置与实测位置之间的偏差约为该区域宽度的三倍,而且误差有两个独立来源:一是模型预测的坐标,其准确性在所评估的两个模型之间相差一倍;二是插值步骤产生的误差,它源于合成器参数的组合,不受模型质量影响。在端到端测试中,36 条细化指令中有 26 条未产生可感知的变化;在使用较弱模型时,该功能未执行任何操作,却报告已经收敛。因此,实效能动性(用户能够促成什么结果)与认知能动性(用户能否知晓这些结果)可以彼此分离,也可以独立设计:通过结构强制实施的约束在模型失效时依然有效,而依赖模型自我报告的保证则不然。最终仍然存在的失效模式,是将毫无反应呈现为成功。最后,我们提出了一套用于构建和审计此类显示的简短规程。

arXiv cs.LG73

MolP2SBench:LLMs 真的能从分子性质推理出结构吗?

arXiv:2610.12522v1 公告类型:交叉收录 摘要:大型语言模型(LLMs)正越来越多地用于分子发现与结构解析,这些领域的决策依赖于物理观测。现有评估主要关注端到端的分子重建,而正确的预测并不一定意味着模型依赖了所提供的性质信息。我们提出 MolP2SBench,这是一个用于评估 LLMs 从分子性质到结构(P2S)推理能力的基准。每项任务提供一个部分被掩蔽的分子骨架及给定性质,要求模型从合理的候选项中选出缺失的结构。我们通过打乱或替换性质,检验模型对正确性质—结构配对的依赖程度,以及对反事实性质变化的响应能力。按信息通道分别评估的排行榜采用基于性质的恢复指标(Property-Grounded Recovery,PGR),对原始条件与反事实条件下的恢复表现赋予相同权重。通过评估七个模型,我们发现 LLMs 能够利用物理性质进行结构预测,但在不同性质类型上的表现差异显著。使用某种性质时准确率更高,并不一定意味着对该性质的响应更强:模型从 $^1$H NMR 中获得的收益可能大于从 $^{13}$C NMR 中获得的收益,但在 $^1$H 数据被替换时,反而更不容易改变预测。LLMs 的表现也未必反映人类化学家利用这些性质进行推断时的相对难度。尽管 UV-Vis 光谱提供的直接结构信息少于 NMR 光谱,Astra 使用 UV-Vis 数据时的准确率却高于使用 NMR 谱峰时。MolP2SBench 支持研究 LLMs 如何提取和利用物理性质信息,揭示可能不同于传统化学直觉的模式,并为构建更可靠地以性质为依据、用于分子与量子化学推断的模型奠定基础。代码和数据集:https://github.com/DUAL-Xiao/MolP2SBench。

arXiv cs.LG68

Looking Under the Streetlight: Evaluation in Generative Molecular Dynamics

arXiv:2610.12518v1 Announce Type: cross Abstract: This position paper argues that the generative molecular dynamics (GenMD) community is not properly evaluating its progress: current evaluations let a model match what is reported while being arbitrarily wrong on what was claimed, which risks over-claiming advantage over classical baselines. This reflects a structural misalignment between common evaluation practices and the scientific tasks the field sets for itself. The reason is that the ground truth---exhaustive molecular dynamics on the systems of interest---is computationally costly, and this cost is what motivates the field. With ground truth inaccessible, evaluation drifts toward convenient proxies, and the resulting norms entrench easily. We identify misaligned evaluation patterns that persist as a community-level equilibrium, and propose concrete observables, reporting standards, and benchmark designs, both computational and experimental, that better reflect what GenMD aspires to deliver. Our aim is to give the field shared language and tools for evaluation that match its scientific ambitions before today's conventions calcify into norms.

arXiv cs.LG76

CORAL-KV:用于自回归视频 KV 缓存的紧凑跨层预测与残差编码

arXiv:2610.12514v1 公告类型:交叉收录 摘要:自回归视频扩散模型维护着不断增长的注意力键值(KV)缓存,使内存成为长视频生成的主要瓶颈。近期,基于聚类的量化已成为压缩视频 KV 缓存的有效方法:先将相似的键向量和值向量分组,再量化残差误差。然而,当后续帧关注压缩后的历史信息时,微小的重建误差会随着视频长度增加而不断累积。除了聚类所利用的层内相似性,我们还发现视频模型的缓存层之间存在很强的依赖关系。这促使我们提出 Coral-KV——离线校准、残差编码与局部适配——这一 KV 缓存压缩框架。它利用各层的缓存状态预测后续层的键和值,再量化剩余的预测残差。我们构建了紧凑的块对角加低秩(block-DPLR)预测器:块对角分量对每个注意力头内部的通道关系建模,而全局低秩项则对注意力头之间的依赖关系建模。Coral-KV 提供了灵活的保真度与内存权衡:既可以跨视频复用预测器,也可以针对当前视频重新拟合轻量级块对角预测器,以获得更高的保真度。在 HY-WorldPlay 和 MAGI-1 上的广泛评估表明,与 QVG-4 相比,我们的可复用配置分别将 PSNR 提高了 0.32 dB 和 4.95 dB,同时将缓存内存占用分别减少了 48% 和 37%。与 QVG-Pro 相比,我们的视频适配配置将 MAGI-1 的 PSNR 提高了 3.77 dB,同时将缓存内存占用减少了 33.7%。Coral-KV 还能在长时间生成过程中保持更高的重建保真度,减少时间上的不连续性,同时保持相当的生成延迟。

arXiv cs.LG

EarLog: Opportunistic Ear Physiology Sensing toward Personalized Hearing Risk Awareness

arXiv:2610.12501v1 Announce Type: cross Abstract: Many headphone users risk irreversible hearing damage during high-load listening, yet current protection still relies on sound-pressure limits. Clinical tests detect damage after it becomes measurable, leaving the risk-accumulating process unobserved. The acoustic reflex (AR), an involuntary response to high acoustic load, marks when the ear enters a protective state. AR traces are weak, and existing measurements require dedicated probes and controlled conditions that are difficult to maintain during everyday audio. We present EarLog, which turns playback, the dominant interference, into an opportunistic probe. EarLog introduces content-conditioned residual sensing to suppress variations caused by nonstationary content, individual ear acoustics, and device response. It identifies weak AR through loudness-dependent changes and localized spectral irregularities in the residual. A resonance-constrained sparse finite impulse response (FIR) model captures the short-window response. Dual-microphone observations, mild-distortion recovery, and cross-session reference updates keep detection usable under low signal-to-noise ratio (SNR), fit changes, and environmental drift. EarLog runs in real time on an embedded controller using a low-cost dual-microphone add-on. We evaluate it with 81 participants, 16 headphone models, and more than 400 hours of recordings. EarLog achieves 99.0% sensitivity and 98.7% specificity on a clinically referenced benchmark. EarLog records listener-specific AR events during everyday listening, providing data for future longitudinal studies of hearing risk.

arXiv cs.LG45

Acoustic Biomarkers for Alzheimer's Disease Detection from Spontaneous Speech

arXiv:2610.12500v1 Announce Type: cross Abstract: Alzheimer's disease (AD) remains a major public health challenge, and early identification is crucial for timely intervention and care planning. Recent work has shown that speech-based digital biomarkers can capture subtle changes in vocal production, prosody, and timing that often precede or accompany cognitive decline. In this study, we investigate whether acoustic features extracted from spontaneous speech can reliably distinguish individuals with AD from healthy controls. Using a structured speech dataset derived from the Cookie Theft picture-description task, we extract clinically informative low-level descriptors spanning prosodic, spectral, and temporal domains, and evaluate a range of machine learning classifiers. Our results indicate that acoustic biomarkers carry strong discriminative value, with the best-performing models achieving clinically meaningful separation between diagnostic groups. These findings support the use of non-invasive, low-cost audio-based screening tools and motivate further work on multimodal and longitudinal assessment frameworks for dementia risk detection.

arXiv cs.LG61

Faithfulness Without Anchors: A Position Paper on the Structural Misalignments of Explainable AI Faithfulness Metrics for Feature Importance

arXiv:2610.12493v1 Announce Type: cross Abstract: Explainable Artificial Intelligence (XAI) tools have often been presented as methods to increase trust in machine learning models by making aspects of their algorithms more human-understandable. However, evidence accumulated over the last decade points to an overall failure of this objective, instead promoting overreliance on incorrect predictions, which can subsequently lead to algorithmic aversion. Rather, XAI appears to function primarily as a debugging tool, useful for obtaining anecdotal insights into, for example, features learned by the model. One of the main issues underlying this limitation is that XAI methods often oversimplify the inner workings of the model, resulting in misalignment between the explanation and the model itself - a behavior referred to as unfaithfulness or infidelity. Faithfulness can be interpreted as the degree of approximation between an explanation and the model. Computing faithfulness in a rigorous manner would thus make it possible to quantify one of the various elements of trustworthiness of an explanation. However, existing metrics present significant limitations, making this quantification either infeasible or computationally expensive. In this position paper, we outline five major shortcomings of current faithfulness metrics and propose four possible avenues for research to bolster their reliability.

arXiv cs.LG75

TTS-Q:感知引导的文本转语音扩散 Transformer 训练后量化

arXiv:2610.12474v1 公告类型:交叉收录 摘要:零样本文本转语音(TTS)可以仅凭几秒钟的音频克隆声音。然而,这种高质量表现依赖于拥有数十亿参数的扩散 Transformer,其模型规模和多步采样使最需要它们的端侧、辅助技术和实时应用难以采用。训练后量化(PTQ)无需重新训练即可压缩此类模型,但现有方法主要面向语言和视觉,而非语音。在 4 比特精度下,即使是性能强劲的量化器,也只能让语音可懂度接近全精度水平,却会持续损害说话人的音色,因此错误率仍然较低,声音却偏离了目标说话人。我们探究量化为何损害语音、损害发生在哪里,以及能否在不重新训练的情况下修复这些损害。为此,我们首次针对 10 亿和 35 亿参数规模的 TTS 扩散 Transformer 开展了权重与激活均为 4 比特(W4A4)的 PTQ 研究。音色损失来自权重量化和激活量化两方面,但只有激活量化部分具有可供精度调度利用的时间步结构。因此,我们提出 TTS-Q,从与任务匹配的校准候选项中选择难以量化的项,并采用预算平衡的混合精度调度,为后期时间步的前馈激活分配更高精度。TTS-Q 在两种模型规模上均实现了最佳的音色恢复效果,且不牺牲语音可懂度。此外,部署后的经校准 W4A4 参考方案降低了 GPU 峰值内存占用,为资源受限条件下的 TTS 部署提供了实用参考。

arXiv cs.LG68

Neuromorphic Satellite Intelligence: A Formal Framework for Event-Driven Architectures, Spike-Based Algorithms, and Quantitative Energy-Accuracy Trade-offs

arXiv:2610.12471v1 Announce Type: cross Abstract: Satellite onboard computing faces an irreducible tension: mission intelligence requirements grow monotonically while power, mass, and radiation budgets remain fixed. This paper advances the state of the art on three fronts. First, we develop a formal mathematical model of spiking neural computation for satellite workloads, deriving closed-form expressions for energy consumption as a function of network sparsity, firing rate, and synaptic fan-out. Second, we propose three original algorithms -- a Spiking Kalman-Complementary Filter (SKCF) for attitude estimation, a Temporal Contrast Anomaly Detector (TCAD) for spacecraft health monitoring, and a Spike-Based Adaptive Modulation and Coding (SAMC) scheme for cognitive satellite links -- each with pseudocode and analytical complexity bounds. Third, we introduce a quantitative Energy-Accuracy-Latency (EAL) trade-off surface and populate it with numerical estimates derived from published silicon measurements of Loihi and SpiNNaker, enabling direct comparison with FPGA and GPU baselines. A taxonomy of four radiation-aware neuromorphic architecture classes is refined with fault-rate models. Continual on-orbit learning is formalized through a constrained Spike-Timing Dependent Plasticity (STDP) rule that provably bounds weight drift. Case studies for a 12U CubeSat Earth observation mission, a GEO communications payload, and an autonomous deep-space probe provide concrete, quantified design points. Our analysis shows that neuromorphic implementations can achieve 8 to 47 times energy reduction over GPU baselines while maintaining inference accuracy within 2.1\% for representative satellite workloads.

arXiv cs.LG79

关于 AI 时间跨度的估计与有效性——对 METR 图的统计考察

arXiv:2610.12466v1 公告类型:交叉发布 摘要:METR 的 50\% 时间跨度衡量的是人类完成某类软件任务所需的时间,而 AI 解决这类任务的概率为 50\%,从而使 AI 能力能够用易于理解的单位来表达。基于 228 项任务和 26 个 AI,我们使用样条和项目反应理论重新计算时间跨度,以放宽任务对 AI 的难度与人类完成时间的对数呈线性关系这一假设。我们拟合的样条可解释为一个将人类完成时间\emph{转换}为 AI 任务难度的函数;该函数在 2–30 分钟区间内几乎平坦,而在其他区间则接近线性。因此,尽管同样增长了 $10 \times$,时间跨度从 3 分钟跃升至 30 分钟,远比从 30 分钟跃升至 5 小时容易。总体而言,我们提供了在一套经交叉验证的适当评分规则下表现更好的时间跨度点估计,以及用于评估时间跨度构念效度的诊断图。我们建议结合诊断图来解读时间跨度,尤其是在提出新的基于时间跨度的基准,或扩展现有基准以纳入耗时更长的任务时。

arXiv cs.LG76

想象未来,内化要义:通过内化时空想象实现高效 VLA 推理

arXiv:2610.02626v2 公告类型:交叉发布 摘要:视觉-语言-动作(VLA)模型越来越多地引入中间推理来改进机器人操作,但现有方法主要针对已观测状态进行推理,并未显式预判未来场景的演变。然而,若将此类推理扩展为在每个推理步骤中显式推演未来,则会引入大量计算开销。我们提出了 IG-VLA,这是一种使模型能够想象未来并内化要义的 VLA 推理框架。我们的 Latent Spatiotemporal Reasoning 直接在视觉表征空间中学习想象与任务相关的未来场景演变,无需高成本的像素级视频生成即可指导动作预测。为进一步降低推理开销,我们引入了 Scene Gist Memory,将推理得到的场景-行为关联内化为紧凑的 Scene Gist Token,在保留未来推理优势的同时,绕过推理阶段的显式未来想象。在 LIBERO、LIBERO-Plus 和 VLABench 上开展的大量实验验证了 IG-VLA 的有效性和效率。在 LIBERO-Plus Language 测试套件上,推理策略与要义策略的成功率均比最强基线高出近 6%。要义策略相较于基线还实现了最高 6.38 倍的加速,在单张 NVIDIA A6000 GPU 上,将每个动作块的推理延迟从 1081 毫秒降至 169.5 毫秒。这些结果表明,未来时空推理能够被有效内化,从而实现高效的 VLA 部署。

arXiv cs.LG

A Sampling Complexity-aware Framework for Discrete-time Fractional-Order Dynamical System Identification

arXiv:2501.17499v1 Announce Type: cross Abstract: A variety of complex biological, natural and man-made systems exhibit non-Markovian dynamics that can be modeled through fractional order differential equations, yet, we lack sample comlexity aware system identification strategies. Towards this end, we propose an affine discrete-time fractional order dynamical system (FoDS) identification algorithm and provide a detailed sample complexity analysis. The algorithm effectively addresses the challenges of FoDS identification in the presence of noisy data. The proposed algorithm consists of two key steps. Firstly, it avoids solving higher-order polynomial equations, which would otherwise result in multiple potential solutions for the fractional orders. Secondly, the identification problem is reformulated as a least squares estimation, allowing us to infer the system parameters. We derive the expectation and probabilistic bounds for the FoDS parameter estimation error, assuming prior knowledge of the functions \( f \) and \( g \) in the FoDS model. The error decays at a rate of \( N = O\left( \frac{d}{\epsilon} \right) \), where \( N \) is the number of samples, \( d \) is the dimension of the state variable, and \( \epsilon \) represents the desired estimation accuracy. Simulation results demonstrate that our theoretical bounds are tight, validating the accuracy and robustness of this algorithm.

arXiv cs.LG82

AI 模型可被证明能够隐藏任意能力

arXiv:2610.13928v1 公告类型:新提交 摘要:能力评估旨在部署前揭示模型的危险行为,但其可靠性依赖于一个假设:隐藏能力可以被诱发。我们通过构造后门攻击,证明这一假设在对抗条件下并不成立。这些攻击将加密的神经网络嵌入宿主模型的权重中,只有在收到秘密触发信号时才会解密并执行它们。借助数字锁箱,在密码学假设下,即使能够完全访问模型权重,这些隐藏电路仍可被证明难以诱发和解释,从而将先前隐藏固定字符串的研究扩展到隐藏任意计算。我们在 PyTorch 中实现了概念验证构造,并在一个小型化学反应预测任务上,通过微调、权重加噪、基于梯度的触发信号搜索以及潜在空间对抗训练来评估能力诱发效果;该任务被用作化学、生物、放射性和核(CBRN)相关能力的替代指标。结果表明,这些构造能够抵抗上述方法;结合我们的密码学保证,这揭示了基于能力诱发的审计方法的局限。我们公开了模型,以支持防御措施的改进。

arXiv cs.LG

KL Before and After Mixing: A Round-Complexity Separation in Parallel Sampling

arXiv:2610.13924v1 Announce Type: new Abstract: A parallel sampler can choose at random which coordinates to generate together. For a target $p$ and the output law $q_U$ of a schedule $U$ drawn afresh for each output, we compare the schedule-averaged divergence $\mathbb{E}_U D_{\mathrm{KL}}(p\|q_U)$ with the output-law divergence $D_{\mathrm{KL}}(p\|\mathbb{E}_U q_U)$, in nats. Rounds commit independent draws from exact conditional marginals frozen at the round's start, each output's schedule is drawn before sampling from a finite mixture of schedules, and every scheduled round is charged. For the uniform distribution on binary strings of even length $n$ with $n/2$ ones, the least expected number of rounds at a fixed tolerance $\varepsilon>0$ is $\Theta_\varepsilon((\log n)^2)$ under the first criterion but $(e^{-\varepsilon}+o(1))\log_2\ln n$ under the second, and the double-logarithmic order is attainable with the same round count on every run. For this target, a count-state recursion gives the exact least output divergence at every round cap over all causal randomized policies. The separation extends to weighted fixed-count targets with bounded weight ratios and quota fractions bounded away from zero and one, without permutation symmetry, and to the full-support Beta$(1,1)$--Bernoulli target, where the sharp expected cost under the output-law criterion is $(e^{-\varepsilon}/2+o(1))\log_2\log_2n$, attained with rare $n$-round runs. Prior work implies the order of the schedule-averaged cost on the balanced target and its $O_\varepsilon((\log n)^2)$ upper bounds on both extensions.

arXiv cs.LG84

基于自然语言的蛋白质结构设计

arXiv:2610.13917v1 公告类型:新投稿 摘要:从头蛋白质设计的一项核心挑战是生成具有所需结构和功能特性的蛋白质。现有设计模型通常要求为每种特性提供专门的输入表示,从而限制了其支持的设计目标。我们开发了 Prompteina,这是首个支持以自然语言指定全新且更广泛设计目标的全原子结构生成模型,类似于文本到图像生成。为训练 Prompteina,我们构建了 Prompteina-Data,这是一个文本与蛋白质结构配对数据集,包含 180 万个结构和超过 5000 万条文本描述,涵盖整体及残基特异性的结构和功能特性。我们通过文本条件预训练,以及将残基与相应文本对齐的微调,对预训练结构生成模型 Proteina-Complexa 进行了适配。Prompteina 在 18 项结构特性中的 15 项上优于基线模型,并且生成的蛋白质更符合其预期功能的描述。在结合蛋白设计中,它在 20 个靶标中的至少 13 个上领先于基线模型,界面置信度较最强基线模型提高了 14\%。引入迭代式计算模拟反馈后,在最具挑战性的靶标上,Prompteina 相较于非迭代生成的性能增益比次优基线模型的相应增益高出 57\%。最后,我们通过实验验证了 Prompteina 针对 GM2A 的设计,此前尚无针对该靶标的从头设计结合蛋白报道。我们鉴定出九种结合蛋白,其中结合能力最强者的表观 $K_{\mathrm{D}}$ 为 107 nM。这些结果表明,自然语言可作为蛋白质结构生成的有效且强大的条件控制接口。

arXiv cs.LG75

重新审视表格数据深度回归:目标函数与全局早停的局限

arXiv:2610.13909v1 公告类型:新发布 摘要:尽管面向表格数据的深度学习模型近来受到了研究界的广泛关注,但回归任务中目标函数的选择仍是一个研究不足的自由度。多个研究方向提出了标准均方误差(MSE)的替代方案,但数据集、骨干网络和评估协议的差异,使得这些方案的实际收益难以比较。在本研究中,我们通过在大量成熟的表格数据集和多种神经网络骨干上,对回归目标函数进行系统性的重新评估,弥补了这一空白。比较结果表明,相较于经过充分调优的 MSE 基线,许多已提出的目标函数带来的提升有限或不稳定,而分位数损失在各个受测骨干网络上,相较于 MSE 均取得了稳定的总体提升。随后,我们发现,即使采用这一表现优异的目标函数,仍存在一个局限:目标噪声水平不同的区域,可能在不同的训练轮次达到最佳预测性能。因此,即使目标函数能够表示异方差目标分布,单一的全局早停检查点仍可能迫使模型在不同不确定性区域之间作出折中。针对这一局限,我们提出了一种感知异方差性的分位数损失扩展方法,使用依赖于输入的标签平滑作为潜在的解决方案,并在多种骨干网络上取得了优于分位数损失的表现。总体而言,我们的结果为表格数据深度学习中的回归目标函数提供了新的实证图景,并揭示了不同不确定性区域之间最优停止时机的差异如何限制全局早停的有效性。代码可在 https://github.com/yandex-research/tabular-regression 获取。

arXiv cs.LG74

预测并非充分的记忆状态

arXiv:2610.13881v1 公告类型:新论文 摘要:能够复现当前所有预测的记忆,仍可能已经遗忘了如何从下一次观测中学习。我们针对精确累积学习和注意力机制刻画了这一区别。对于特征和标签均为不受限制的实数的光滑凸残差损失,当且仅当损失为多项式时,才存在固定维度的连续递归状态。对于普通的、未加权的观测,$D$ 次多项式恰好需要 $r=\binom{d+D}{D}-1$ 个坐标;即使在当前预测器固定为同一个预测器的历史中,继续学习仍可能恰好需要 $r-d$ 个坐标。对于非多项式损失,这类预测相同的历史所需的状态维度没有上界。在有界标量逻辑斯蒂示例中,即便固定任意有限阶导数喷射,这一障碍仍然存在。对于岭回归,隐藏的证据是设计矩阵:精确状态需要 $dp+d(d+1)/2$ 个坐标,而预测器只需 $dp$ 个。在一个条件良好的零预测族上,最优 $B$ 比特压缩的平均单次更新误差量级为 $d^{-1}2^{-2B/s_d}$,其中 $s_d=d(d+1)/2$,即使允许任意随机编码也是如此。核注意力提供了第二种精确闭包分类。这些结果揭示了未来更新会暴露哪些信息,而非提出新的回归算法,也不声称能提升已训练模型的性能。

arXiv cs.LG74

通过零阶残差跟踪实现部分信息随机优化

arXiv:2610.13871v1 公告类型:新论文 摘要:许多优化问题仅能直接获取部分梯度,而其余分量必须通过函数求值来推断。这种结构出现在施为预测、施为强化学习以及下层为黑箱的双层优化中。我们提出 Partially Informed Stochastic Optimization(PISO),将可获取的梯度分量与残差的动态估计相结合。通过以该估计为中心对每次零阶观测进行中心化,PISO 使随机探测引起的方差取决于跟踪误差,而非残差的大小。由此得到一种具有显式收缩机制的随机化草图迭代,并为动量提供了有理论依据的解释。对于光滑非凸目标,我们建立了依赖于残差变化量及其初始大小的收敛保证,而不要求不可获取的梯度分量具有一致界。在采用独立采样的函数求值以及高斯或球面探测方向时,PISO 在梯度满足 Lipschitz 条件下达到的函数求值复杂度主导项为 \(O(d^{2}\varepsilon^{-6})\),在额外假设 Hessian 矩阵满足 Lipschitz 条件下则为 \(O(d^{2}\varepsilon^{-5})\)。实现这些速率,每次更新仅需一个探测方向、两个函数样本和一个已知分量样本。分析还量化了批处理如何通过增加观测次数来减少部署次数。在五种实验设置中,PISO 的变体在其中三种设置下,以相当的预言机访问条件优于所测试的基线方法;在双层路由问题中更快达到相当的解;在施为强化学习中,则在不使用响应模型导数的方法中表现最佳。

arXiv cs.LG74

跨骨架与表位的推理时引导,实现高样本效率的从头纳米抗体设计

arXiv:2610.13844v1 公告类型:新论文 摘要:生成式分子设计通常依赖大量生成和事后筛选。Feynman-Kac(FK)引导等推理时引导方法通过使生成轨迹偏向高奖励候选对象来提高效率,但其优化仅限于单一固定配置。在从头纳米抗体设计中,预先确定骨架类型或靶标表位等条件选择会形成瓶颈。由于不同条件下的产出率各异,对前景不佳的配置进行引导会浪费计算资源。我们将生成条件选择重新表述为推理时的资源分配问题,并提出两种动态预算分配方法:一种是联合的全局序贯蒙特卡洛(SMC)方案,可同时跨配置进行引导并分配计算资源;另一种是基于批量逐次淘汰(BaSE)改造的解耦方法。实证结果表明,在不同治疗靶标和生成配置下,这些动态分配方法将每 GPU 小时的可设计候选对象产出提高至近 5 倍,同时不牺牲多样性。

arXiv cs.LG69

Returning to Development States: When Full Sandbox Snapshots Help Coding Agents

arXiv:2610.13841v1 Announce Type: new Abstract: Coding agents often make useful progress before they fail, but retry-from-scratch discards the machine state created along the way. This makes failure recovery an interesting search problem: if intermediate environment states preserve meaningful progress, restoring and branching from them could outperform repeatedly restarting from the initial state. We implement Go-Explore-style search over full Daytona snapshots and find that higher and more efficient solve rates depend mainly on progress that sits outside the repository, such as databases, services, and other runtime state. Our results suggest that full-snapshot search is most effective when long-horizon coding work spans multiple services whose state cannot be reconstructed from a single diff.

arXiv cs.LG75

专才学得更快:预训练中的知识组织方式影响知识更新效率

arXiv:2610.13835v1 公告类型:新提交 摘要:理想的(语言)模型必须持续更新其知识。现有工作主要关注如何在训练后高效地进行这些更新。我们研究了预训练期间时变知识的组织方式如何影响后续更新的效率。按时间顺序排列的知识训练出的模型,专精于最近接触到的知识;我们将其称为专才。相比之下,以随机打乱的顺序在相同知识上训练出的模型,在所涵盖的所有时期均表现良好;我们将其称为通才。令人惊讶的是,专才获取新知识所需的更新步数远少于通才。为解释这一优势,我们从第一性原理出发,推导出一条将更新成本与模型优化动态联系起来的定律,该定律能够准确预测实证结果。在这些发现的指导下,我们设计了一种预训练数据组织策略,在保留稳定的历史知识的同时,实现对当前知识的高效更新。我们还表明,按时间组织知识带来的益处也适用于现实世界中的预训练模型和现有知识更新工具,并可推广至其他知识排序方式。项目页面:https://horizon-lab.org/specialist/

arXiv cs.LG77

论连续扩散语言模型的设计空间

arXiv:2610.13834v1 公告类型:新论文 摘要:连续扩散语言模型(CDLMs)有望将连续流建模的优势引入语言领域,包括加速采样和可控生成。该领域发展迅速,CDLMs 的能力已开始媲美离散扩散模型。然而,与图像或视频等连续模态不同,我们对训练 CDLMs 时哪些关键因素起作用的理解仍然有限。在本研究中,我们聚焦其中两个因素,即自条件化和嵌入几何,并揭示其背后的部分机制,进而提出成本更低、效果更好的设计。我们首先表明,自条件化通过让模型接触训练中原本罕见的、类似推理时的状态,纠正了训练与推理之间的不匹配。通过一种名为 $\mathrm{DoubleCat}$ 的双层插值方案,无需额外的前向传播即可获得这些收益,使 TinyGSM 上的准确率提升至原来的三倍以上。自条件化还有助于识别被大范数高斯噪声主导的词元,这一问题被称为词元可识别性。为直接解决该问题,我们在 $\mathrm{DoubleCat}$ 中引入离散揭示机制,得到 $\mathrm{SemiCat}$。该模型在 TinyGSM 上达到与离散扩散相当的性能,使用标准训练超参数和经过调优的采样器时,准确率达到 $49.94\%$。在 $1.7$B 参数规模下,$\mathrm{SemiCat}$ 在六项基于似然的评估中的四项上优于离散扩散,并在无条件生成方面超越独热流模型。在嵌入空间中,我们发现交叉熵会推动嵌入形成一种几何结构,其解码错误率与独热表示的解码错误率非常接近。我们还简要探索了一系列其他选项,例如混合扰动过程和噪声调度,包括异步调度。我们相信,这些探索能够帮助研究社区更好地理解并拓展 CDLMs 的设计空间。