全部动态

本页 40 条
10月9日 周五
arXiv cs.LG76

Generative Adversarial Loops

arXiv:2610.11458v1 Announce Type: new Abstract: AI research progress can be viewed as the interaction between two processes: benchmark creation and method discovery. Historically, both were driven by human intelligence. However, recent advances in AI have accelerated automated method discovery, while automated benchmark creation has received comparatively less attention. To enable self-advancing systems, we propose Generative Adversarial Loop (GAL), a generator-discriminator framework alternating between two agentic searches: (1) a discriminator that generates adversarial data to expose weaknesses in current systems, and (2) a generator that discovers algorithms to overcome them. We apply this framework to approximation algorithms for efficient inference. Unlike existing auto research systems, which primarily focus on algorithm discovery, GAL introduces a discriminator agent that automates goalpost setting by continually searching for weaknesses in the current algorithm. We demonstrate adversarial data generation across four tasks: KV compression, sparse video generation, sparse attention, and context extension, where the discriminator identifies weaknesses in state of the art techniques. We further show that GAL enables autonomous improvement, with newly discovered algorithms improving not only on adversarially generated data, but also on established benchmarks. Specifically, GAL improves CompactorPress on KV compression with Qwen3-4B at 4x, raising performance on the discriminator dataset from 0.35 to 0.97, while also outperforming RULER-HARD (+0.77 pts). For context extension, GAL boosts Dual Chunk Attention from 0.20 to 0.90 on the discriminator dataset, while yielding gains on standard benchmarks(ScienceFiction (+6 pts) and PG19 32K (-0.33 PPL)). GAL thus provides a path toward autonomous goalpost setting and algorithmic improvement, where AI systems continually discover their own weaknesses and develop methods to overcome them.

arXiv cs.LG77

Zatom-2: Multitask Pretraining on Atomistic Data for Generative Modeling across Domains

arXiv:2610.11454v1 Announce Type: new Abstract: Unified atomistic modeling has the potential to accelerate discovery in chemistry, materials science, and biology by bridging data-rich chemical domains and data-scarce biological contexts. However, existing generative approaches to atomistic modeling remain highly specialized to scientific disciplines (chemistry vs. biology) or do not leverage both high-volume organic (molecule) and inorganic (material) data for general-purpose pretraining. To this end, we introduce Zatom-2, an atomistic generative model pretrained on approximately five million structures from the OMol25 and OMat24 electronic structure datasets. Zatom-2 features a multiscale Transformer architecture coupled with conditional flow matching that supports force conditioning and foundational pretraining tasks such as generation, structure prediction, and prediction of molecular and material energies and forces. Empirically, Zatom-2 achieves better molecular distribution fidelity than Zatom-1 and achieves strong performance on existing molecule and material generation benchmarks. Zatom-2 demonstrates the ability to control sample generation across low- and high-force regimes, and enhances protein generation in a low-data setting through joint generative-predictive pretraining and transfer learning, increasing protein backbone designability in a length extrapolation setting from 67.8% without pretraining to 74.8% after finetuning on 2,000 protein domains.

arXiv cs.LG76

MotiveMob: Motivation as Semantic Action for Closed-Loop Human Mobility Generation

arXiv:2610.11442v1 Announce Type: new Abstract: Human mobility generation, an important task in urban research, synthesizes trajectory data for urban planning and transportation management. Human mobility can be characterized as a "why-where-when" decision process: people form an intention to move and then determine where and when the corresponding activity will take place. Trajectory generation under user-level and temporal distribution shifts may benefit from explicitly modeling this decision structure. However, many existing human mobility generation methods either represent behavioral intent at a coarse granularity, such as a daily plan or a trajectory-level description, or directly predict future locations without explicitly reasoning about a possible motivation for each movement step. We introduce MotiveMob, a motivation-driven autoregressive framework for human mobility generation that first forms a hypothesis about why the next movement may occur and then jointly generates where and when it may occur. At each step, a motivation predictor conditions on the current mobility state, a long-term behavioral report, and the mobility history to infer a plausible motivation or determine whether the trajectory should terminate. Given the hypothesized motivation, a state predictor grounds it in a candidate next location and arrival time. The candidate then undergoes speed-feasibility and repetition checks before being fed back for the next decision. We evaluate MotiveMob under distribution shifts involving unseen users and unseen temporal periods, including seasonal changes and the substantial behavioral disruption caused by the COVID-19 pandemic. Experiments show that MotiveMob consistently achieves better distributional fidelity than competitive pretraining-based and prompting-based methods under user-level and temporal distribution shifts, demonstrating robust generalization to out-of-distribution mobility patterns.

arXiv cs.LG72

Policy Alignment: New Signals for Membership Auditing in On-Policy Distillation

arXiv:2610.11423v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student model by aligning its policy with a teacher model on trajectories generated by the student model itself. Through this process, the student policy moves toward the teacher on the prompts used for distillation. However, these prompts are often private and costly, creating a need for prompt-level membership auditing. Existing methods mainly rely on likelihood-based confidence signals or student policy drift between checkpoints, but they do not capture the teacher-induced direction of the student update. In this paper, we propose Policy Alignment Membership Auditing (PAMA), a new auditing framework tailored for OPD. Our key observation is that a member prompt directly contributes to the teacher-guided policy update, while a non-member prompt only experiences indirect effects through cross-prompt generalization. Based on this directional trace, PAMA measures whether the student update moves toward reducing the teacher loss on a candidate prompt. Specifically, we introduce Teacher Alignment Gain (TAG) to estimate the teacher-aligned update direction from model outputs, and further combine it with student drift and uncertainty alignment signals for reliable membership auditing. We evaluate PAMA on six datasets and three teacher-student model families. On MATH, the primary evaluation benchmark, PAMA achieves AUC values of 0.791--0.941, improving AUC by 14.6--20.6% over state-of-the-art baselines.

arXiv cs.LG72

MC-TRCM：面向不完整移动与可穿戴心理健康特征视图的观测感知递归融合

arXiv:2610.11408v1 公告类型：新论文 摘要：公开的移动与可穿戴心理健康数据集通常提供汇总特征表，而非同步的原始传感器流。在这些发布的数据中，每个锚点对应一次调查或标签的时间，并可能结合手机或可穿戴设备的汇总信息、既往症状评分、人口统计信息、临床变量以及来源可用性指标。我们提出 Modality-Conditioned Temporal Recursive Context Model（MC-TRCM），将每个特征来源保留为独立的词元，并将缺失情况纳入输入上下文。对于已观测到的来源，使用其数值和缺失情况摘要进行编码；对于缺失的来源，使用可学习的缺失词元；数据集和任务嵌入为融合提供条件；递归预测头则按照通过验证集选定的步数，逐步细化各项输出。我们采用参与者级别的数据划分，并仅基于验证集进行模型选择，在 DepreST-CAT 和 Prediction of Severity Change-Depression（PSYCHE-D）的六个预定义终点上评估了 MC-TRCM。MC-TRCM 在 DepreST-CAT 的患者健康问卷-9（PHQ-9）和广泛性焦虑障碍量表-7（GAD-7）严重程度预测中取得了最低的平均绝对误差，相较于最佳表格数据基准模型，分别降低了 0.181 和 0.217 个量表分值。分类终点的表现因任务而异：MC-TRCM 在 GAD-7 类别预测中取得了与最佳结果相同的舍入后平衡准确率；在 PSYCHE-D 多分类预测中，其平衡准确率在数值上最高，领先 0.002；在 PHQ-9 类别预测和 PSYCHE-D 二分类预测中，其表现仍接近最强基准模型。消融实验支持了逐特征线性调制、缺失词元、缺失情况投影和递归细化的作用，而校准分析和特征来源对照实验则刻画了各终点的表现特征。我们的代码已公开于 https://github.com/Botwwt/MC-TRCM。

arXiv cs.LG68

Learning from Hetero Density for Cryo-EM Protein Reconstruction

arXiv:2610.11403v1 Announce Type: new Abstract: Reconstructing protein structures from cryo-electron microscopy (cryo-EM) maps is essential for understanding macromolecular assemblies. Although learning-based methods have improved protein reconstruction, information from hetero components remains underused. Our analysis finds both false predictions and reference protein sites near hetero components; filtering nearby candidates can improve or impair chain construction. We introduce CryoCue, a framework that uses hetero information to guide protein reconstruction. An anchor-supervised detector learns hetero representations across five component classes. Multiscale hetero features guide backbone localization, while predicted hetero candidates condition structure refinement through their class, confidence, and frame-relative geometry. Experiments show that CryoCue improves backbone localization near hetero components and achieves more accurate protein structure reconstruction.

arXiv cs.LG78

CoPoE：通过可分解的疾病坐标专家乘积实现用于模态缺失情况下阿尔茨海默病诊断的多模态融合

arXiv:2610.11394v1 公告类型：新论文 摘要：多模态阿尔茨海默病（AD）诊断受益于异质性临床、影像、基因组和生物标志物证据的整合，但临床队列中经常存在不规则的模态缺失。现有融合方法往往合成缺失的输入，面临引入人工替代数据的风险，或将可用信号汇集到不可解释的潜在空间中。我们提出 CoPoE（疾病坐标专家乘积），这是一个疾病坐标框架，将多模态证据映射到结构化潜在空间，并将该空间划分为四个不同的生物学和临床轴：遗传风险、分子病理、神经退行性变和临床分期（R/P/N/S）。每个已观测模态都对覆盖完整 RPNS 向量的对角高斯专家进行参数化，而带掩码的专家乘积架构仅融合可用模态。因此，缺失模态不会向融合路径添加任何因子，使网络能够针对任意非空模态子集保留稳健、可分解的后验分布，而无须在 RPNS 路径中进行合成插补。通过在 ADNI 数据集上开展广泛的模态缺失实验，在共享的非 PET ADNI 嵌入基准下，与标准化的模态缺失融合基线相比，CoPoE 取得了最佳的全模态性能，并在全部 15 种已观测模态子集评估中获得最高的平均 AUROC，同时显著改善了原始概率的 ECE、Brier 分数和 NLL。此外，PET 监督探测显示，在全模态条件下，病理（P）分块内存在证据富集；即使不提供直接的体液生物样本输入，仍能保留与 tau 相关的信号。我们的代码已公开，地址为 https://github.com/labhai/CoPoE。

arXiv cs.LG73

From a Prompt to Repertoires: Evolving Functional REpertoires Enable LLM Continual Learning

arXiv:2610.11373v1 Announce Type: new Abstract: Continual learning remains challenging for large language models, which must enable models to acquire new skills and knowledge without degrading existing capabilities. Existing approaches typically address this challenge by carefully designing how model parameters are updated. In contrast, prompt optimization avoids costly parameter updates while achieving competitive or even superior performance to reinforcement learning methods such as GRPO on individual knowledge-intensive and reasoning tasks. This raises a natural question: \textit{Can prompt optimization, as an efficient adaptation approach, be directly applied to continual learning?} Our analysis shows that, under sequential task adaptation, it suffers from catastrophic forgetting, while optimized prompts accumulate rules that overfit to local task distributions. To address these limitations, we propose \emph{Evolving Functional REpertoires} (EFRE), which replaces a single prompt with a repertoire of functions that evolves as new tasks arrive: compatible updates refine existing functions, while conflicting updates trigger the emergence of new ones. On a three-task continual-learning stream, EFRE achieves a final average performance 7.50 percentage points higher than GRPO. Moreover, after adaptation to the Bio task, its performance on FinQA decreases by only 1.56 percentage points, compared with 25.10 percentage points for the base prompt optimization method. We further instantiate EFRE in a minimal agent system and observe consistent improvements across different backbone models. Overall, these results demonstrate EFRE's strong performance in continual learning for large language models and highlight its substantial potential for continual learning in advanced agent systems.

arXiv cs.LG69

SpatialOPSD: Self-Distilling Spatial Intelligence from Verified Coding Agent Traces

arXiv:2610.11366v1 Announce Type: new Abstract: Spatial coding agents significantly improve spatial reasoning in Multimodal Large Language Models (MLLMs) by using external tools to generate verified execution traces. However, this paradigm inherently suffers from prohibitive inference-time overhead and external dependencies. In this paper, we explore whether an MLLM can internalize this agentic capability to operate entirely tool-free. We begin with a simple observation: prompting an MLLM with summarized execution traces of a spatial coding agent naturally unlocks the model's internal spatial Chain-of-Thought (CoT). Motivated by this, we introduce SpatialOPSD, an on-policy self-distillation framework that internalizes spatial reasoning into a standalone MLLM by formulating verified agent traces as privileged information. To mitigate privileged-information leakage during distillation, we introduce Repetition-Aware Distillation, which combines repetition masking with unlikelihood regularization. Experiments across multiple benchmarks demonstrate that self-distilling SpatialOPSD achieves higher average accuracy than SFT and GRPO on both spatial and OOD datasets, exhibiting superior performance and generalization.

arXiv cs.LG72

Bernoulli Flow Models: Self-Consistent Generative Modeling for Binary Data

arXiv:2610.11362v1 Announce Type: new Abstract: Binary diffusion models typically require a large number of function evaluations (NFEs) to generate high-quality samples, making practical inference computationally expensive. Reducing NFEs while preserving sample quality without distillation or additional training remains a significant challenge. Existing binary diffusion models define a discrete one-step forward path and then derive the reverse posterior. In low-NFE settings requiring cross-step sampling, they approximate the true multi-step likelihood with a single-step likelihood transition, which severely degrades sample quality. To address this fundamental limitation and decouple the generative dynamics from fixed discrete time steps, we propose Bernoulli Flow Models (BFM). Rather than relying on sequential one-step Markov diffusion chains, BFM defines a unified continuous global Bernoulli probability flow path between data distributions and pure noise, from which we derive analytical closed-form posterior transitions over arbitrary time intervals. Consequently, reducing the inference NFE is no longer an approximation based on skipping discrete steps; it only requires re-evaluating the analytical posterior over a new time grid. This eliminates the structural training-inference mismatch inherent to discrete chains and yields self-consistent low-NFE sampling. Experiments show that BFM is highly robust to aggressive NFE reduction. On LSUN Churches 256x256, a BFM trained with 256 steps achieves an FID of 9.22 using only 16 sampling steps, whereas the state-of-the-art discrete baseline degrades to 204.10. BFM also remains competitive with continuous and discrete generative baselines under standard full-step inference. These results establish BFM as a theoretically rigorous, self-consistent, and practically effective framework for fast binary data generation.

arXiv cs.LG73

样本高效的生成式保形预测

arXiv:2610.11349v1 公告类型：新增 摘要：生成式保形预测从条件生成器的样本中构建不确定性集合，但只有当这些样本能够良好地表示响应分布时，这种方法才是高效的。这可能需要大量样本，而每个样本的生成成本都可能很高，例如在大型扩散模型和科学模拟器中，因此必须高效利用采样预算。现有方法在每个输入处抽取相同数量的样本，导致在响应分布简单的地方浪费样本，而在分布复杂的地方采样不足，从而使集合膨胀，并导致这些输入的覆盖率不足。我们提出 CASA（保形自适应样本分配），通过刻画额外样本的边际价值，在满足边际覆盖率和平均采样预算约束的条件下，在各个输入之间分配样本，以最小化期望集合大小。理论分析表明，在相同预算下，自适应分配得到的集合小于固定数量分配得到的集合：遗漏一个模态会迫使半径跨越模态之间的间隔，即使是全知半径也无法弥补这一点。在合成任务和真实任务中，CASA 在相同预算下生成的集合显著更小，通常还能改善条件覆盖率，并且可以与现有的半径自适应方法互补。

arXiv cs.LG75

Deflating the Hessian: Rank-4 W4A4 Quantization for Multimodal Diffusion Transformers

arXiv:2610.11315v1 Announce Type: new Abstract: In diffusion transformers, low-rank branches can mitigate 4-bit weight--activation (W4A4) post-training quantization (PTQ) loss by decomposing each weight into a low-bit residual and a high-precision low-rank component. Existing low-rank PTQ approaches, however, either optimize low-rank compensation and residual quantization separately, often requiring higher ranks, or rely on second-order weight updates without explicitly modeling activation quantization error, which becomes particularly pronounced under 4-bit quantization. To address these limitations, we present \method{}, a unified framework modeling low-rank-assisted W4A4 PTQ as a coupled calibration problem and deriving optimization-based solvers from the joint objective. Eliminating the output-side low-rank factor yields a \emph{deflated Hessian} that discounts residual errors already captured by the low-rank component, while an activation-noise surrogate is incorporated to suppress activation quantization error. Across five diffusion backbones, rank-4 \method{} consistently outperforms rank-4 SVDQuant in PSNR and LPIPS. It further surpasses rank-32 SVDQuant on SANA-1.6B, FLUX.1-schnell, and FLUX.1-dev with an $8\times$ smaller rank and up to $6.25\times$ faster quantization. Furthermore, on the Qwen3-8B LLM, rank-4 \method{} improves MMLU accuracy from 61.50\% to 68.17\% over rank-32 SVDQuant. Overall, \method{} achieves better W4A4 performance with substantially lower rank and quantization cost.

arXiv cs.LG64

NP-Hardness of Minimizing Neurons in Two-Hidden-Layer ReLU Neural Networks

arXiv:2610.11313v1 Announce Type: new Abstract: A fundamental question in neural network architecture optimization is whether the minimum hidden-neuron count required to approximate a target function within a prescribed tolerance can be computed efficiently. This paper resolves this question for two-hidden-layer ReLU networks under an $L^p(\mathbb{R}^d,\mathbb{R}^m)$ approximation constraint. For every fixed $d \ge 1$, $m \ge 1$, and $1 \le p < \infty$, we prove that computing the optimum exactly is NP-hard. The result holds even when the target is represented by a rational ReLU network whose realization is nonzero, componentwise nonnegative, compactly supported, globally Lipschitz, and continuous piecewise affine. The polynomial-time reduction from 3-SAT produces an architecture gap in which unsatisfiable formulas yield an optimum of zero, whereas satisfiable formulas yield an optimum of at least $d+2$. The proof constructs compactly supported polyhedral frustum functions realized by two-hidden-layer ReLU networks and establishes the $L^p$-density of finite linear combinations of box-frustum functions. The results offer theoretical justification for employing heuristic approximation methods in the design of ReLU neural networks, illustrating that attaining a minimal configuration within polynomial time is computationally unachievable.

arXiv cs.LG74

From Geometry to Generalization: Why Row Normalization Can Beat Adam and Muon

arXiv:2610.11309v1 Announce Type: new Abstract: Different optimizers can fit the same training data while selecting classifiers with substantially different geometries, but whether this difference provably affects population performance remains unclear. We show that row-wise normalization can achieve strictly higher population accuracy than full-batch Adam, a proxy for random-reshuffling Adam, and exact-SVD Muon in high-dimensional multiclass classification. Under an isotropic Gaussian-cloud data model, this advantage arises because row normalization's class-wise Euclidean geometry asymptotically preserves the population decision-boundary directions, whereas Adam's coordinate-wise geometry and Muon's spectral geometry introduce nonvanishing distortions. Beyond isotropy, the advantage persists for full-batch training on class means with independently oriented class-mean and test-noise covariances. It holds for power-law spectra with class-mean exponent below one, even under heavily anisotropic test noise. When both covariances are diagonal and sufficiently close, the advantage over Adam can reverse, while applying the same random rotation to both restores it by changing only their alignment with Adam's coordinate axes. Synthetic and last-layer language-model experiments support the predicted advantage.

arXiv cs.LG71

Memorization and Malign Generalization in Conditional Diffusion Models with Random Features

arXiv:2610.11288v1 Announce Type: new Abstract: Conditional diffusion models generate diverse, novel, and high-quality samples under prescribed conditions. However, theoretical understanding of their memorization and generalization remains limited, while recent works have characterized these behaviors primarily in unconditional settings. In this work, we analyze a random-feature conditional score model in the high-dimensional proportional limit, deriving asymptotic expressions for training and test losses. By decomposing the test loss, we show that in the overparameterized regime, increasing model width improves prediction of the condition-dependent mean while reducing within-condition prediction variance, a phenomenon we term "malign generalization." Furthermore, analyzing the training loss reveals that more informative conditions lead to memorization of training samples at smaller widths. These theoretical findings are supported by experiments with U-Net architectures on realistic data.

arXiv cs.LG70

Residual spectral instabilities in representation learning

arXiv:2610.11257v1 Announce Type: new Abstract: Learned representations can lose latent degrees of freedom successively, suggesting a cascade of transitions whose underlying stability principle remains unclear. Here we formulate dimension-wise posterior collapse in variational autoencoder (VAE) as a fluctuation theory around partially collapsed states. Interpreting the negative evidence lower bound as an effective free energy, its quadratic expansion defines a Gaussian theory whose Hessian acts as a mass matrix for latent fluctuations. We show that the collapsed directions form an invariant fluctuation sector and derive its exact mass spectrum in terms of a conditional residual operator. A local reactivation direction lowers the free energy when the decoder variance falls below the residual spectral upper edge, with equality marking marginality. The criterion recovers principal component thresholds in the linear Gaussian VAE limit. Viewed in reverse along continuously connected branches, the reactivation boundary provides a local criterion for successive collapse. Numerical continuation experiments show successive loss of latent dimensions near these spectral marginalities. These results support a spectral cascade interpretation governed by residual information left unexplained by the surviving representation.

arXiv cs.LG71

Neuro-Memory Fuzzy Inference System for Mimicking Human-like Car Following Behavior

arXiv:2610.11252v1 Announce Type: new Abstract: This study presents the Neuro-Memory Fuzzy Inference System (NeMeFIS), a hierarchical machine learning architecture that asymmetrically models acceleration and deceleration in car following behavior by integrating five human memory types procedural, working, episodic, semantic, and declarative. By linking external variables to memory functions via metaheuristics and validating them through factor and p-value analyses, NeMeFIS uncovers latent cognitive influences across Arterial, Collector, and Rural Highway corridors for different types of vehicles. Results from 54 different trained models emphasize cognitive thresholds shaped by driver perception limits and cognitive load. The trained NeMeFIS models outperform traditional statistical and conventional machine learning models in replicating realistic driving behavior, including comparisons with Linear Regression, ANFIS, and LSTM architectures. Fuzzy rule analysis reveals that declarative memory demands the highest rule, especially during deceleration, indicating complex braking decisions. Procedural memory drives acceleration, while semantic and declarative memory guide deceleration. Risk perception also emerges as a key factor, particularly on urban roads. Validated on both heterogeneous and homogeneous datasets, NeMeFIS offers a robust framework for modeling driver cognition. The findings support psychotherapeutic applications and the development of adaptive, human-like decision systems in Connected and Autonomous Vehicles (CAVs) to enhance traffic safety.

arXiv cs.LG79

Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals

arXiv:2610.11247v1 Announce Type: new Abstract: On-policy distillation (OPD) enables effective capability transfer between language models, yet the mechanisms underlying its failures are not fully understood. Across code generation and mathematical reasoning, OPD with larger-scale teachers exhibits early loss plateaus, with an average final loss reduction of 25.1% after 200 updates, compared with 96.2% for self-RL teachers, obtained by further reinforcement learning (RL) training of the initial student. To understand this difference, we analyze OPD as an idealized continuous-time dynamical system in the small-learning-rate limit. Our training-log diagnostics associate these plateaus with an early decline in a gradient-based learning-signal proxy while substantial loss remains; these measurements do not establish why the underlying gradient weakens. We further prove a local recovery guarantee for teachers sufficiently close to the initial student in a shared parameterization under regularity conditions, offering a conditional explanation for the success of self-RL teachers in our experiments. Across runs with and without loss plateaus, we observe small relative parameter changes (0.025-0.098%) and high similarity between the student's representations before and after OPD (linear CKA $>0.98$ across layers). These observations suggest that limited representation adaptation may contribute to learning-signal collapse, a hypothesis that remains to be tested. Code is available at https://github.com/leizhao7/opd-learning-signals.

arXiv cs.LG80

Read What Matters: Query-Adaptive Quantization for KV Caches

arXiv:2610.11245v1 Announce Type: new Abstract: KV-cache entries are stored before their future queries are known, but each decoding query needs precision in different places. We study this mismatch using separate budgets for retained bits and bits fetched per query. ReadKV stores each key and value in a progressive code whose prefixes support different reconstruction precisions. For each query, it allocates key-channel prefixes using the query, computes attention from the reconstructed keys, and then allocates value-token prefixes using that attention. Stored entries remain unchanged. Each stage optimizes a calibrated distortion objective under a fixed budget; we prove exact allocation under diminishing refinement gains and relate these objectives to attention-output error. We also exhibit a finite-dimensional attention family where query-dependent access strictly outperforms every query-independent reader at the same read budget, even with unrestricted competing encoders and decoders. Across six base models, reading four bits on average from an eight-bit cache increases C4 perplexity by at most 0.66%, using about one quarter of the logical reads and half the retained capacity of a 16-bit cache. It is consistently more accurate than storing and fully reading four bits at the same payload-read budget. Retaining more bits than each query fetches is aimed at long-context decoding, where the cache bytes moved per step, rather than the weights, dominate cost. Long-context question answering and retrieval on two instruction-tuned models provide additional quality evidence. On the tested 8K-token, batch-one, single-layer workload on an NVIDIA A10G, a restricted eight-bit ReadKV reader with a two-bit mean payload-read budget has 39% lower latency than the tested TurboQuant codec.

arXiv cs.LG78

Low-Cost Sensor Calibration for Indoor Air Quality Monitoring: A Dataset, Evaluation Scenarios, and a Lightweight Model

arXiv:2610.11236v1 Announce Type: new Abstract: Low-cost sensors enable scalable indoor air quality monitoring but require calibration because of nonlinear distortions, noise, and temporal drift. The conventional strict pairwise calibration setting requires a co-located reference sensor at each deployment location and does not account for spatial and temporal heterogeneity. To address these limitations, we introduce a six-month dataset comprising multivariate indoor air-quality measurements from low-cost and reference sensors with contextual metadata collected at five locations. Using this dataset, we define four evaluation scenarios. The reference-efficient and location-transfer scenarios evaluate spatial generalization, whereas the long-term drift and event-conditioned scenarios assess robustness to gradual and abrupt distribution shifts. Based on these scenarios, we derive design requirements and propose a lightweight temporal model that combines input-window compression with residual temporal and feature fusion. Experiments show strong calibration performance across all four scenarios with low edge-inference cost.

arXiv cs.LG74

SteerCast：用于仅解码器时间序列预测的基于检索的潜在空间引导

arXiv:2610.11229v1 公告类型：new 摘要：时间序列预测旨在根据历史观测值和辅助特征预测未来值。我们提出了 SteerCast，这是一种基于检索的潜在空间引导方法，可在推理时改进仅解码器预测器，而无需更新其参数。SteerCast 通过存储每个历史窗口的表示以及在预测器潜在空间中计算得到的引导向量，从训练集构建数据库。该引导向量定义为：由真实后续序列诱导的表示与由模型自身预测诱导的表示之差。在测试时，SteerCast 为查询历史检索最近邻样本，聚合其引导向量，并在自回归生成的每一步将所得信号注入预测器的隐藏状态，从而引导预测结果趋向于与相似训练案例一致的轨迹。在涵盖多种多变量基准数据集和多个预测跨度的实验中，SteerCast 始终比微调后的骨干模型和基于检索的基线方法取得更高的预测精度，同时无需在原始微调之外进行额外训练，并且仅使用训练集作为检索语料库。

arXiv cs.LG76

Do Flatter Minima Drive Better Generalization? An Algorithmic Separation in Grokking

arXiv:2610.11206v1 Announce Type: new Abstract: Flat loss landscapes have long been linked to better generalization in neural networks. However, its role as a causal mechanism for generalization is less established. Grokking provides an unique testbed to understand this distinction: models are prone to fit observed data using non-generalizing structure and remain in that regime for prolonged periods, transitioning to generalization only under particular training conditions. In this work, we study whether flat loss landscapes can act as a driving mechanism in this transition. While recent work has argued for flatness as a necessary geometric condition for this transition, we find that biasing training toward flatter solutions using sharpness-aware minimization (SAM) is insufficient to reliably induce this transition, despite producing flatter solutions. However, when SAM is paired with mechanisms that drive generalization such as weight decay, an interesting property emerges: SAM can accelerate the transition to generalizing solutions by up to 4x at the epoch-level. We theoretically untangle this relationship between SAM and weight decay using a minimal interpolating two-layer ReLU model with both memorizing and generalizing solutions. We show that even in this simple setup, flatness alone cannot distinguish a memorizing solution from a generalizing one, while weight decay favors generalizing solutions. However, under a local stability analysis, there exists a window where a memorizing interpolant is locally stable under gradient descent but unstable under SAM in the low-norm regime, which can explain SAM's ability to accelerate this transition. Overall, our results provide a more interpretable account of the role of flatness in driving generalization, especially in settings where models are vulnerable to minimizing loss through learning non-generalizing structure.

arXiv cs.LG76

细胞命运指派中的预测多重性：无标签 Rashomon 集与逐细胞认证的局限

arXiv:2610.11185v1 公告类型：新论文 摘要：单细胞轨迹推断将转录组测量结果映射到发育连续谱上，但对数据拟合程度同样好的配置可能会指派出相互冲突的细胞命运。FateMultiplicity 是一个无标签框架，无需谱系标签即可构建统计上可接受的模型集合，即 Rashomon 集；其方法是在以随机种子变异校准的非劣效性检验下，利用交叉拟合的留出基因评估模型偏差。预测多重性很大，且更多地取决于模型空间的多样性，而非其规模：第二种算法的十二种配置揭示了 20.0% 的细胞存在多重性，而第一种算法的二十四种配置仅揭示了 3.8%。随后，我们检验逐细胞认证的命运间隔 FM 是否能比拟合模型本身提供更可靠的指派，结果表明不能。在具有真实值的模拟数据上，针对同一批细胞，FM 判别错误指派的 AUC 为 0.682，而基线配置自身决策间隔的 AUC 为 0.965（p = 0.003），基于随机种子离散程度的基线方法则为 0.854。信息量取决于可接受集合的广度，而非其基数：当基数为四时，更换随机种子重新拟合得到的集合对应 AUC 为 0.933，扰动超参数得到的集合则为 0.701。将下确界放宽为 q 分位数可恢复判别能力，但会趋近于单个模型自身的置信度；上确界达到 0.973，是因为 theta* 属于集合这一事实为其提供了下界，而下确界则缺乏锚定。轨迹推断中的多重性值得测量和报告，但在无标签 Rashomon 集上进行逐细胞认证，并不能带来更可靠的命运判定。两种构造仍然有效：间隔侵蚀比可在模拟中区分真实与虚假的分支点（AUC 为 0.890，尚未在真实数据上检验）；以克隆观测到的命运为参照，未通过认证的细胞与其所属克隆结局不一致的比例，比通过认证的细胞高出 16.4 个百分点（p < 0.001）。

arXiv cs.LG70

SACQ: Structured Decoding with Memory-Conditioned Refinement for Long-Horizon Forecasting

arXiv:2610.11170v1 Announce Type: new Abstract: Long-term time series forecasting (LTSF) models predominantly employ patch-based encoders terminated by a flatten readout head that maps the entire encoded historical memory to all future steps through a single shared projection. This implicit coupling of future positions obscures position-specific historical-to-future alignment and amplifies sensitivity to corrupted inputs and extreme supervision noise. We present SACQ, a plug-in structured prediction head that replaces flatten readout while keeping the encoder unchanged. SACQ adopts a two-stage decoding pipeline: it first establishes a coarse patch-grid forecast scaffold, then refines each future position through cross-attention over historical memory and merges the attention-derived correction with the coarse scaffold via a learned per-patch gate. To stabilize optimization under long horizons and noisy labels, we further propose a batch-adaptive scaled log-cosh loss that automatically calibrates robustness to the current residual scale, suppressing outlier gradients while preserving MSE-like sensitivity for typical errors. SACQ attains top-tier test MSE/MAE across PatchTST, DLinear, and patch-Mamba backbones with only modest incremental overhead in parameters and latency. Under inference-time input corruption and training-set label-noise stress tests, SACQ substantially outperforms flatten readouts, with ablation studies validating each architectural component.

arXiv cs.LG75

CARE: A Lightweight Plug-in Gated Correction and Uncertainty-aware Module for Long-term Time Series Forecasting

arXiv:2610.11165v1 Announce Type: new Abstract: Multivariate long-horizon forecasting is critical to electricity load scheduling and traffic flow management, and to financial risk control. Existing deterministic backbones output a single trajectory, masking heterogeneous prediction difficulty across horizons and channels and providing no localized reliability signal. We present CARE (Corrective branch with Aligned context and Relative-error Estimation), a lightweight plug-in that enhances any deterministic forecaster without architectural redesign. Operating in parallel with the base model, CARE resamples historical context to match the forecast horizon, learns residual correction patterns from this aligned history, and applies scale-aware bounded updates modulated by per-coordinate sigmoid risk gates. A multi-objective loss jointly optimizes forecast accuracy, residual tracking, risk alignment, and base-model anchoring. Across eight benchmarks with three representative backbones, CARE improves accuracy with marginal parameter and latency overhead. Its risk gates reliably identify high-error regions: on Weather, the highest-gate tertile exhibits nearly four times the error of the lowest-gate tertile, offering planners an interpretable per-step trust signal. Code is available at https://github.com/CG-BNYC/CARE.

arXiv cs.LG80

用于信用风险建模的 Ranking Prior Alignment：外部先验何时发挥作用？

arXiv:2610.11146v1 公告类型：新论文 摘要：冷启动信用评分——在标注数据稀缺、特征较弱或模型容量极小的条件下部署模型——是金融机器学习中反复出现的问题。当新的信贷产品上线时，带标注的违约数据稀缺，特征流水线尚不成熟，而模型必须以极小的容量部署，以避免过拟合。标准应对措施依赖的仍是同一批有限数据；真正需要的是一种以领域知识为基础的外部正则化来源。我们提出 Ranking Prior Alignment，这是一个模型无关框架，通过温度缩放的 KL 散度损失，将外部排序先验（来自领域专家、教师模型或 LLM）蒸馏到任意评分模型中。该框架通过统一公式 L = L_task + gamma(t) * KL(P_agent || P_model)，将神经网络架构（MIL 注意力机制）与基于树的架构（XGBoost 自定义目标函数）统一起来，其中 gamma(t) 遵循指数衰减调度。该方法在推理时无需外部模型，其基于树的实现可容忍高达 eta = 0.5 的标注噪声。在一个包含超过 150 万商户的工业数据集上，MIL 对齐在使用 3,000 个包时，7/7 个评估项均取得正向提升（1 项 ID 指标 + 3 项 OOT 指标 + 3 项退化指标；在 OOT-1 上，Delta AUC 峰值为 +0.020），而 XGBoost 消融实验在使用 300 个包时，9/9 项指标均取得正向提升。在公开 Amex 数据集上进行的跨数据集验证显示，5/5 个交叉验证折均取得正向提升（平均 Delta AUC = +0.041）。四类模型（MIL、XGBoost、LightGBM、Logistic Regression）和四种教师架构的实验表明，该框架既不依赖特定模型，也不依赖特定先验来源。我们进一步观察到，对齐收益呈现逆向缩放规律：随着数据丰富度 N、模型容量 C 和特征质量 Q 的下降，收益反而增加，这有助于从业者判断何时值得投入先验标注。

arXiv cs.LG75

ActiveMedAgent：面向多模态医学诊断的成本感知轨迹学习

arXiv:2610.11140v1 公告类型：新投稿 摘要：临床诊断本质上是一个序贯过程：只有在预期新增证据能够消除诊断不确定性时，临床医生才会从低成本检查逐步转向高成本检查。我们提出 ActiveMedAgent，这是一个将这种成本感知的序贯逻辑引入多模态医学人工智能的框架。在使用参数冻结、通过 API 访问的视觉语言模型的情况下，ActiveMedAgent 跟踪候选诊断的概率分布，并以每一步的诊断效用减去成本，对每次信息获取进行评分。随后，基于这些已评分的轨迹离线训练一个轻量级 MLP 控制器，使其学习何时请求更多证据，以及何时作出最终诊断。在三个常用基准上，基于轨迹的策略学习始终优于无引导的信息获取方法和全模态基线。值得注意的是，我们发现了一种信息过载效应。在 175 个病例中，该智能体利用更少的信息通道得出了正确诊断，而全模态基线却未能正确诊断，这表明，学习省略哪些信息可能与学习获取哪些信息同样重要。

arXiv cs.LG81

QUILT：通过共享查询执行重新思考稀疏注意力预填充

arXiv:2610.11134v1 公告类型：新论文 摘要：稀疏注意力降低了长上下文注意力的开销，但现有内核通常独立处理各个查询，反复加载并反量化多个查询共享的 KV 条目。我们观察到，相邻查询所选的 KV 条目存在大量重叠，为跨查询复用创造了机会。我们提出 QUILT，一种工作负载感知的稀疏注意力执行机制，通过联合处理相邻查询并复用共享 KV 条目，减少冗余的内存流量和计算。QUILT 引入 Shift-and-Compare Set Decomposition（SCSD），将不规则的集合操作转化为适用于现代加速器的规则数据并行原语，并将 SCSD 与注意力计算以流水线方式执行，以隐藏其开销。级联共享在多个粒度上分层实现复用。一种分块感知的执行策略在共享粒度与硬件分块利用率之间取得平衡，并选择性地移除重要性较低的查询专属尾部，以消除利用不足的分块。我们使用 GLM-5.3 和 DeepSeek-3.2，在张量并行和序列并行两种设置下，基于 LongBench 评估 QUILT。与最先进的稀疏注意力内核相比，QUILT 将平均内核延迟最多降低 55.1%，将处理的 KV 数据量最多减少 55.9%，同时将首个词元生成时间（TTFT）延迟最多降低 36.8%，且准确率下降可忽略不计。

arXiv cs.LG60

Machine Learning Optimization for Enhanced OS Fingerprinting

arXiv:2610.11133v1 Announce Type: new Abstract: Operating System (OS) Fingerprinting is a technique that can be used to identify a network's operating systems by evaluating network traffic in the form of TCP/IP packets. This research will explore the effectiveness of passively identifying operating systems on the CIC-IDS2017 dataset, a collection of over 47 gigabytes of pcap files with their corresponding operating systems. This research also proposes a new command line interface, OsirisML, which uses nPrint to preprocess the data into tabular data and XGBoost to apply ML to the data to generate, retrain, and test ML models. When packets are split randomly between training and testing, OsirisML models reach an accuracy of 97.66% on a down-sampled subset of the Friday capture and 84.69% on the entire capture. On the entire Monday capture, which contains no attacks, OsirisML reaches an accuracy of 73.83% and an F-1 score of 79.38%.

arXiv cs.LG77

Cova-PINN：用于复杂几何结构中流固共轭传热的跨域守恒物理信息神经网络

arXiv:2610.11108v1 公告类型：新投稿 摘要：多域物理信息神经网络（PINNs）能够灵活地构建针对不同介质的表征，以求解流固共轭传热（CHT）问题。然而，标准多域 PINNs 在各域分别采样的支撑点集上施加控制方程和界面条件约束，这可能产生看似合理的温度场，却导致端到端能量传递和出口温度不准确。我们提出 Cova-PINN，这是一种多域 PINN 框架，可使守恒约束的支撑与复杂几何结构中的热相互作用路径相匹配。Cova-PINN 联合优化局部尺度上的跨域复合控制体平衡与换热器全局尺度上的配对壁面闭合。我们采用统一方案，在四种三重周期极小曲面（TPMS）换热器和一种几何结构不同的 DualMS 设计上，将 Cova-PINN 与针对 CHT、面向优化以及适用于复杂几何结构的 PINN 基线方法进行比较评估。相较于最接近的基线 MUSA-PINN-CHT，Cova-PINN 在四种 TPMS 拓扑结构上的平均出口温度误差和设备级闭合误差分别降低了 $37.7\%$ 和 $60.2\%$，同时还提高了全场预测和换热量的准确性，并在 DualMS 上取得了一致的改进。

arXiv cs.LG68

DaCe-DT: Data-Centric Offline Multi-Task Reinforcement Learning via Adaptive Prompts and Trajectory Correction for Heterogeneous Tasks

arXiv:2610.11085v1 Announce Type: new Abstract: Offline multi-task reinforcement learning (Offline MTRL) heavily depends on the quality and distribution of pre-collected data. However, existing methods mainly focus on algorithmic optimization, with less emphasis on data-level improvements to enhance learning ability and generalization performance. This paper, from a data perspective, reveals three key bottlenecks that limit Offline MTRL performance:(i) ineffective utilization of prompts length under diverse task complexities, and (ii) semantic irrelevance of randomly sampled prompt segments, (iii) misleading supervision induced by fragmented and discontinuous trajectories. To address these challenges, we propose DaCe-DT, a robust offline MTRL framework designed to be insensitive to heterogeneous task complexities and data quality, featuring length-gated prompt masking (LGPM), retrieval-augmented prompt construction (RAPC), and value-adaptive return calibration (VARC). Together, these mechanisms enable DaCe-DT to deliver data-centric prompt adaptation and trajectory refinement, resulting in robust multi-task generalization and stable policy learning amid heterogeneous offline data and tasks. Experimental results on Meta-World show that DaCe-DT consistently outperforms state-of-the-art methods, achieving an average improvement of 11.73% on optimal datasets and an improvement of 13.34% on suboptimal datasets, demonstrating its effectiveness in learning stably from imperfect data and improving overall multi-task performance.

arXiv cs.LG73

Optimally Pacing Budget Spending and Learning

arXiv:2610.11074v1 Announce Type: new Abstract: We establish near-optimal regret bounds for budget-constrained online learning against arbitrary classes of budget-pacing experts in the adversarial setting. In particular, given any class of $F$ experts and a candidate budget pacing schedule, we provide a full-information algorithm which obtains regret $O(D \sqrt{\log F}+ \sqrt{T\log F})$ against all experts whose cumulative spending stays within distance $D$ of this schedule, matching lower bounds established by Braverman et al. (2025). We additionally show that our technique extends to various problems in online resource allocation, where the learner gets to see the rewards and costs of the current options available to them, and establish $O(D\sqrt{\log F})$ regret bounds when fractional allocation is allowed. This is the first algorithm we are aware of which can achieve $o(\sqrt{T})$ guarantees for such tasks.

arXiv cs.LG74

F$^3$NO：具有跨尺度条件调控的频率分解有限时间流映射神经算子

arXiv:2610.10998v1 公告类型：新 摘要：神经算子能够实现快速的PDE预测，但重复预测会累积误差，且细尺度结构仍然难以解析。我们提出一种频率分解有限时间流映射神经算子（F$^3$NO），利用更新后的低频特征引导高频信息的非线性细化。在每一层中，这种跨尺度条件调控将全局频谱处理与局部细节细化连接起来。该模型直接预测指定未来时刻的状态，并根据预测时间间隔调整两条分支的贡献。对于更长的轨迹，它将短时间片段内的并行预测与片段之间的递归传播相结合。在五个PDE基准测试上的实验表明，与自回归基线和直接预测基线相比，该方法提升了预测精度。消融实验显示，频率分解细化能够以更少的参数提高精度，而分段带来的收益取决于空间分辨率和动力学状态。

arXiv cs.LG63

Invariant-Measure Reasoners: Stable Representations for Latent Reasoning

arXiv:2610.10996v1 Announce Type: new Abstract: Latent reasoning models repeatedly update a latent state using the same recurrent block. As the recurrent depth increases, the sequence of latent states may converge to a compact subset of the state space without necessarily converging to a fixed point. Existing models typically predict by applying a prediction head to a single latent state. However, the latent state can continue to change even after many updates, potentially making predictions unstable across recurrent depths. To address this instability, we introduce invariant-measure reasoners (ImR), a framework that uses an invariant measure as a stable representation. This measure describes the long-run distribution of latent states on the compact subset and is invariant under updates by the recurrent block. ImR predicts from the expectation of the prediction head's output under this measure. We use ImR in two ways: fine-tuning only the prediction head of existing models and training models from scratch. Both approaches reduce prediction instability and improve accuracy in many settings on maze and Sudoku tasks. In some settings, models trained with ImR exhibit non-fixed-point behavior more frequently than existing models yet achieve high accuracy even with such behavior, unlike existing models. These results suggest that ImR can leverage otherwise destabilizing dynamics for latent reasoning.

arXiv cs.LG72

多带宽 Distribution Matching Distillation：论 Distribution Matching Distillation 与 Drifting Models 的等价性

arXiv:2610.10989v1 公告类型：new 摘要：研究人员一直在持续探索有效的一步生成模型，而 Drifting Models（Deng et al., 2026）近期展现出了一步生成的巨大潜力。已有研究揭示了 Diffusion & Flow Style Generative Models（DFSGMs）（Ho et al., 2020；Song et al., 2020a；b；Lipman et al., 2022；Liu et al., 2022）与 Drifting Models（Li & Zhu, 2026；Lai et al., 2026；Turan et al., 2026）之间的联系。然而，据我们所知，尽管二者的优化目标公式几乎完全相同，尚无人在 Drifting Model 与广泛使用的蒸馏方法 Distribution Matching Distillation（DMD/DMD2）（Yin et al., 2024b；a）之间建立精确的对应关系。在本文中，我们证明，通过将预训练 DFSGMs 中的速度场/噪声场转换为 Drifting Models 中的吸引力场，并从生成分布中估计排斥力场，训练 Drifting Model 在本质上等价于 Distribution Matching Distillation。基于这一等价概念，我们从 Drifting Model 的视角出发，提出了一种基于 DMD 的改进方法——Multi-Bandwidth Distribution Matching Distillation（MBDMD）。

arXiv cs.LG76

使用 Chained LMOs 优化大型语言模型

arXiv:2610.10975v1 公告类型：new 摘要：Muon 促成了一系列不断壮大的优化器，这些优化器组合了多种矩阵归一化方法，但这类方法仍然彼此割裂，缺乏统一视角。我们引入链式线性最小化预言机（chained linear minimization oracles，chained LMOs），将这些方法视为多个 LMO 的组合。尽管许多链式方法在经验上表现出色，但其中不少超出了标准 LMO 框架的范围，并且可能在光滑凸目标上发散。为解释组合为何仍能带来帮助，我们转向线性联想记忆，并表明在各向异性嵌入下，链式方法能够优于 Muon。在实验方面，我们提出了 TensorChain，这是一种处于该框架内的新型优化器，可将不同层中相互兼容的权重矩阵堆叠起来，并沿各个轴对所得三维张量进行归一化。在 Qwen3 0.6B 和 1.7B 的预训练中，TensorChain 在平均 token 效率方面超过了所有链式基线方法；在验证损失相同的情况下，相比 Muon 平均节省 9.6% 的 token。

arXiv cs.LG74

预算约束下用于离策略评估的多源反事实标注

arXiv:2610.10974v1 公告类型：new 摘要：离策略评估（OPE）利用已记录数据估计目标策略的价值，但行为策略覆盖范围有限，可能迫使人们进行高方差重加权或奖励模型外推。反事实标注能够为未观测动作提供额外证据，但包括领域专家和大型语言模型（LLMs）在内的实际标注来源可能成本高昂、存在偏差或含有噪声。我们研究了上下文 bandit OPE 中此类标注的预算约束获取问题。在给定各来源的成本和误差特征后，我们针对上下文—动作对与标注来源制定整数分配问题，以最小化估计量方差中取决于标注方案的部分。我们通过首次标注阈值和局部标注价值区间刻画标注何时具有价值。针对耦合的多源问题，我们提出一种结合动态规划子程序的主化—最小化算法，能够单调改善目标函数。合成临床 bandit 和由 LLM 标注的教育 bandit 实验表明，与不进行标注相比，我们的分配方法分别将固定特征下的均方误差（MSE）降低了 20.58% 和 10.77%。

arXiv cs.LG79

Transferability of Learned States in Neural PDE Solvers

arXiv:2610.10972v1 Announce Type: new Abstract: Assessing useful reuse in neural PDE solvers is challenging: final accuracy can reflect source learning and target-time computation. Our reuse contract separates solution accuracy, learning contribution, and numerical utility through paired state comparisons, matched target information and budgets, and cost accounting. A literature audit extracts 18 version-specific protocol records from 12 papers, documenting retained states, target-time resources, and reported controls. For a fixed linear system and residual tolerance, we construct two initial guesses with identical solution-error, energy-error, and residual norms, reaching the same solution with different conjugate-gradient (CG) iteration counts. Across 240 source-training trajectories, two linear PDE families, Fourier neural operators and convolutional networks, a fixed predictor's benefit reverses across correction algorithms. Among pairs with both relative prediction errors less than or equal to 5 percent on 64 in-distribution tasks (63 by 63 interior grids), reductions in all three norms accompany more CG iterations, at mean taskwise rates of 23.5 percent and 23.9 percent in two libraries. Work-based selection saves 2.50-3.33 CG iterations on held-out in-distribution tasks; matched adaptation demonstrates finite-budget pretraining value. Independent batches confirm a 0.73 percent complete online saving for one physics-trained Fourier neural operator against zero-initialized Poisson-preconditioned CG. Reuse requires matched state comparisons and downstream computational evidence.

arXiv cs.LG77

ASPIRE：通过集合预测与物理精化发现鞍点

arXiv:2610.10969v1 公告类型：新 摘要：利用事件驱动模型预测热激活扩散与缺陷演化，需要识别原子重排机制及其激活能垒。发现相关鞍点是其中的主要计算瓶颈：一个亚稳态可能对应多种重排，而代价高昂的局部搜索可能失败，或反复收敛到同一个鞍点。为应对这一挑战，我们提出 ASPIRE（Atomistic Saddle-Point Inference with Refinement for Events），这是一个从单个初始原子环境中预测一组鞍点候选，并通过在原始原子间势上执行 Dimer 搜索对其进行精化的框架。该框架的等变集合预测器 Ev-Quiformer 集成了以下组件：(i) 由几何条件控制的标量-向量事件槽，用于生成多个鞍点候选；(ii) 一个解码器，通过结合原子、槽以及相对于锚点的向量与不变量系数，将每个槽映射为完整的原子位移场。我们还贡献了两个数据集：(i) BCCFE4VACAV-4000，包含4,000个体心立方铁四空位构型和65,450个参考事件，这些事件按初始状态分组，用于集合监督和精化后评估；(ii) BCCFE-1TO4VAC，包含5,372个构型，每个构型含有1至4个空位。在理论方面，我们建立了提案等变性的条件。在实验方面，ASPIRE在该基准上的参考事件覆盖率达到77.20%，而传统 Dimer 基线为75.73%，同时所需的 Dimer 力评估次数约减少一半。在对50个构型进行的计时评估中，在所述硬件设置下，ASPIRE将每个构型的实际运行时间从478.8秒缩短至176.3秒。

arXiv cs.LG79

Spectrally Targeted Muon

arXiv:2610.10965v1 Announce Type: new Abstract: The Muon optimizer orthogonalizes each update matrix, setting all of its singular values to one, and has proven highly effective for training large language models. It remains unclear, however, whether this success comes from amplifying small singular directions that gradient descent neglects or from suppressing large, degenerate directions that disrupt training. We introduce Spectrally Targeted Muon, which orthogonalizes only the singular values above or below a threshold $\tau$, so that varying $\tau$ interpolates between normalized SGD and Muon. It isolates the relevant singular subspaces with projections computed by Newton-Schulz iteration on a shifted Gram matrix, so no SVD is needed. We evaluate these variants on the CIFAR-10 and NanoGPT speedruns, tracking the effective rank of gradient, update, and weight matrices and a new metric, the alignment of updates with the tangent space of the weight matrix's isospectral manifold. We find three things. First, the small singular values of the momentum are not noise. Orthogonalizing everything except the few largest singular values of each matrix nearly matches Muon while touching only a small fraction of the momentum, whereas orthogonalizing only the top falls well short even though it holds almost all of it. On language models every momentum singular value is far below one, so targeted orthogonalization can only amplify, and Muon wins by making directions that are too small to train on at their raw scale trainable. Second, shrinking the largest singular values is what keeps the parameter spectrum flat; this is cheap, and it is not what drives the loss. Third, AdamW differs from Muon mainly in how slowly it builds structure, which explains its slower start and why warmup helps AdamW but only hurts Muon.