全部动态

本页 40 条
10月12日 周一
arXiv cs.LG70

Neurograf v1: An End-to-End Romanian-First Language Model Family

arXiv:2610.13134v1 Announce Type: cross Abstract: We present Neurograf v1, an experimental end-to-end Romanian-first language-model pipeline developed to investigate the feasibility of training language models from scratch for a lower-resource European language. The pipeline comprises RoSfera, a 635 GB Romanian corpus containing 175.7 million documents; an 80k byte-level BPE tokenizer; five decoder-only models ranging from 160M to 6B parameters, trained on 131 billion tokens; three instruction-tuned assistants; and two contrastively trained embedding models. Training and evaluation were conducted using computing infrastructure at ELI-NP/IFIN-HH. The model family exhibits consistent scaling of validation loss and increasing Romanian factual and cultural knowledge with model size. On the 1,000-question WWTBM benchmark, Neurograf-6B achieves 69.89% accuracy on Romanian-context questions, while Neurograf-3.2B reaches 67.38%. Compute-optimal analysis yields a parameter-scaling exponent of approximately 0.423. Neurograf v1 serves as an experimental demonstrator rather than a production system. The study identifies practical challenges involving diacritic preservation, corpus quality, data composition, and tokenization efficiency, while demonstrating the feasibility of developing Romanian-first base models, instruction-following assistants, and semantic embedding models within a unified training pipeline.

arXiv cs.LG75

基于梯度的连续动态时间规整快速近似

arXiv:2610.13129v1 发布类型：cross 摘要：Dynamic Time Warping（DTW）是衡量时间序列差异的常用方法。它通过动态规划寻找时间戳之间的最优规整路径，但计算速度较慢且不可微。因此，人们提出了多种变体。其中，Soft-DTW 提出了 DTW 的平滑形式，便于计算梯度；Continuous-DTW（CDTW）则通过可参数化的规整函数引入了连续形式。然而，对于这两种方法，计算成本仍然较高，在现代机器学习流程的大规模数据集上难以实际应用。本文从多个方面重新审视 CDTW。在证明其在温和条件下具有可微性之后，我们提出了 FastCDTW，这是一种高效的近似求解器，包含两种不同的计算策略：针对分段线性规整函数的精确闭式方案，以及针对平滑规整函数（例如隐式神经网络）的无偏蒙特卡洛估计器。两种策略都能计算关于规整函数和输入时间序列的梯度，所得算法也适用于 GPU。我们将其集成到更复杂的学习问题中，展示了它的优势，包括时间序列平均、分类和深度领域自适应。

arXiv cs.LG75

为什么重尾权重谱能够预测模型质量

arXiv:2610.13123v1 公告类型：交叉收录 摘要：我们仍未能扎实理解深度神经网络（DNNs）为何在预测任务中表现出色。例如，尽管已有许多依赖数据的指标试图预测 DNNs 的模型质量，但对于大型数据集，这些指标的计算成本可能很高，而且当训练数据集未公开时，这些指标可能根本无法计算。一种实用且有效的模型质量预测方法是分析 DNN 权重矩阵，因为先前的研究已表明，训练后权重矩阵中的重尾谱分布能够有力地预测模型质量。然而，此类指标缺乏严格的统计学依据。在本研究中，我们旨在从统计学角度理解这些指标为何表现良好。为此，我们采用概率框架推导 DNN 的边际似然，并仅使用最后一层的权重进行评估。在大规模卷积 DNNs 和 LLMs 上，我们发现该边际似然能够预测模型质量。为理解重尾谱在预测性能中的作用，我们针对服从（重尾的）高温 Marchenko-Pastur（HTMP）分布的谱，推导了对数边际似然的极限。我们证明，这一极限量是尾部厚重程度参数的凸函数，并推导出一个最优尾部厚重程度，其在实证中能够预测大规模 DNNs 的过拟合。

arXiv cs.LG74

Does Location Matter? A Black-Box Causal Audit Reveals Model-Dependent Audio Grounding in LALM Hallucination

arXiv:2610.13112v1 Announce Type: cross Abstract: Large audio-language models (LALMs) frequently exhibit object hallucination, affirming absent sounds. The dominant mitigation, Audio-Aware Decoding (AAD), contrasts logits under real audio against an all-zero "blank" waveform, flagged by prior review as out-of-distribution. We introduce a black-box causal test that isolates the claimed evidence itself: localize a claim's evidence region via CLAP similarity, remove it by spectral gating, and compare against a random-window control across a 2x2x2 grid on three LALMs. On Qwen2-Audio-7B, localized ablation's raw AUROC trails blank's (0.613 vs. 0.721); this is a duration confound, not a localization failure, since blank removes the entire clip versus the grid's 2-4 s. Holding duration fixed, localization beats random removal at every cell on both models. Raw detection value remains model-dependent, especially on Audio Flamingo 3. The same evidence corrects decoding: contrastive decoding combining blank and language-prior contrasts, gated and ensembled, raises Qwen2-Audio accuracy from 57.1% to 71.9%, beating AAD alone. A unified causal-grounding profile does not transfer between models, indicating different grounding regimes. The source code and a detailed report of our findings, including identified limitations, are available at https://github.com/khalooei/causa.

arXiv cs.LG80

针对分布式模型提取的查询监测局限性

arXiv:2610.13083v1 公告类型：cross 摘要：如今，模型提供商面临着通过数千个账户发起的蒸馏活动，而每个账户看起来都像普通用户。当网络和支付元数据无法将这些账户关联起来时，剩下的唯一证据就是它们提出的问题。我们使用一百万条真实聊天轮次进行开发，并用之后17周内、未受任何设计选择影响的数据进行一次固定评估，研究提供商仅凭这些查询能够、以及不能够了解到什么。答案与提取行为本身的特殊性关系不大，更多取决于良性协调：只有当某项活动比同一周内最具鲜明特征的普通用户群体更加独特时，它才是可见的。我们对此进行了形式化，表明活动与用户之间一个微小但为正的差异带来的是检测所需时间，而不是可检测性，并确定了最难检测的活动。实证结果显示，经过校准的监测器能够捕捉到按账户监测器遗漏的朴素分布式提取活动（覆盖0.86至1.00的周数）；其中一种变体——在扣除每个账户的主题性之后，选择行动步调一致的账户——即使活动中填充了自然文本，仍能捕捉到其中的活动（覆盖0.50至0.96的周数），而另外两个谱检测器一个也捕捉不到。将每项活动蒸馏成一个小型学生模型后发现，在我们的测试环境中，能够躲过所有检测器的活动，其能力最多只有可检测活动的42%。这些局限同样具体：缓慢升级的活动会污染所有检测器的校准，良性的评估流程与模型蒸馏看起来完全一样；当串通者所占比例为2.5%或更低时，我们没有找到任何可部署的检测器；而转发真实用户请求的活动，可能对任何仅依赖查询的监测器都不可见。

arXiv cs.LG75

CoBe：对话式反事实文本编辑基准

arXiv:2610.13057v1 公告类型：交叉发布 摘要：反事实文本编辑是指根据某个假设性变化修改给定文本，同时保留在因果上不受该变化影响的事实的能力。当被问及如果医生没有建议手术会发生什么时，模型的输出应保留患者的病史，仅修改该决定所带来的后果。这需要因果理解能力：模型必须在没有明确指示的情况下，判断哪些事件会受到假设性变化的影响，以及哪些内容应保持不变。这项能力是人类推理的核心，尤其是在医疗或法律等安全关键领域。相比之下，关联式文本生成任务仅生成与改变后的条件存在统计关联的文本。在上述医疗示例中，关联式回答可能会减轻患者的症状，因为病情较轻与不进行手术存在相关性。这类回答构造了一个看似合理的情境，却并非给定情境的反事实版本。以往的反事实文本数据集并未清晰区分这两种推理模式。我们提出 Counterfactual Benchmark（CoBe），用于测试模型在收到自然对话式提示时，对给定文本情境进行反事实编辑的零样本能力。CoBe 包含 2000 个问题，涵盖医疗、科学与工程、商业及日常情境。我们测试了一系列前沿模型和较小规模模型，其中前沿系统的平均得分为 $58.05\%$。研究结果表明，模型经常默认采用关联式生成，而非保留原始情境的因果结构。此外，性能并未随模型规模或输出词元长度的增加而稳定提升，这表明此类失败可能源于模型的训练目标，而非模型容量不足。

arXiv cs.LG77

空间重于统计量，重训练重于排序：仅标签后门下的训练标签审计

arXiv:2610.13051v1 公告类型：交叉发布 摘要：有些后门攻击从不改动图像。FLIP 只修改标签：在 CIFAR-10 上，翻转 3% 的标签，就能让 ResNet-32 将 99.7% 带有触发器的源类别图像归入攻击者指定的类别。常见防御方法会在使用这些相同标签训练的模型内部寻找投毒痕迹。因此，当这些方法效果不佳时，问题究竟出在所用的检验方法，还是执行检验的空间？我们通过受控替换来回答这个问题：保持检验方法、标签和指标不变，仅改变特征空间，并用各防御方法保留的数据重新训练受害模型，以此评估其防御效果。起决定作用的是空间。在狗$\to$猫的攻击中，同一种最近邻检验在投毒模型特征上的 AUROC 为 0.546，在从未接触过标签的冻结 DINOv2 特征上为 0.967，在使用干净标签训练的同一网络上则为 0.995；最后一种情况是理想参照对照，而非防御方法。我们复现了 FLIP 作者推荐的防御方法 SPECTRE，得到 90.94 / 0.97，与其公布的 91.47 / 1.7 相比接近，随后评估其代价：在卡车$\to$鹿的攻击中，它会删除每个类别 20% 的数据，而基于冻结特征的原型审计仅删除 6%，就能获得更高的干净样本准确率和更低的攻击成功率。在三个更难的类别对上，该审计仍留下约四分之一的攻击效果。在实施任何攻击之前，根据编码器几何结构和干净标签得出的一个计数，能够区分审计可消除攻击的类别对与只能部分消除攻击的类别对，不过这一观察仅涉及一个计数为零的类别对。三项结果提醒我们，不应轻率地下结论。更好的检测分数并不意味着更安全的模型：在四个类别对中的两个上，Confident Learning 的 AUROC 更高，但在狗$\to$猫的攻击中，其残留攻击成功率却是原型审计的 2.6 倍。原型方法族在我们的静态评估中表现最佳，却是自适应攻击者最先绕过的方法。而且，超过某个临界点后，检测出更多投毒样本也不再能降低攻击成功率。应在从未学习过这些标签的空间中检查标签，并根据重新训练后的模型，而非检测分数，来评判防御方法。

arXiv cs.LG56

Lightweight Time-Frequency Vibration Sensing for Consumer Device Condition Monitoring

arXiv:2610.13040v1 Announce Type: cross Abstract: Consumer rotating devices, including washing machines, air conditioners, electric fans, robotic vacuum cleaners, and home fitness equipment, increasingly rely on vibration sensing for intelligent condition monitoring and predictive maintenance. Ensuring reliable operation in these resource-constrained appliances requires efficient, lightweight artificial intelligence (AI) solutions capable of processing non-stationary signals under real-world operational interference. This paper proposes a lightweight AI-assisted vibration sensing framework that employs continuous wavelet transform (CWT) and Hilbert-Huang transform (HHT) to convert one-dimensional (1-D) vibration time-series signals into two-dimensional (2-D) time-frequency representations. These representations are subsequently analyzed using lightweight deep learning architectures, specifically MobileNetV3-Small, ConvNeXt-Tiny, and the You Only Look Once version 11 classification model (YOLOv11-cls). Public bearing benchmark datasets, including Case Western Reserve University (CWRU), Paderborn University (PU), and Intelligent Maintenance Systems (IMS), are utilized as representative rotating components to investigate model generalization. The proposed framework is designed with eventual embedded deployment in consumer electronics in mind. Experimental results demonstrate that CWT combined with YOLOv11-cls achieves high diagnostic accuracy, while MobileNetV3-Small provides an attractive low-latency alternative for computationally constrained edge platforms. Conventional random splitting is also compared with dataset-specific structured isolation. The results highlight the importance of avoiding data leakage and quantify tradeoffs in classification accuracy, parameter count, floating-point operations (FLOPs), model footprint, preprocessing cost, and central processing unit (CPU) inference latency.

arXiv cs.LG69

Scaling Empowered Agents

arXiv:2610.13002v1 Announce Type: cross Abstract: Empowerment measures the capacity of an agent to actively control its environment. In this work, we explain the connection between skill learning and empowerment. While skill learning is not empowerment, we introduce a simple scalable recipe for empowerment maximization: empowerment corresponds to the number of executable skills from a state, so maximizing empowerment corresponds to reaching states that are good starting points for many skills. Our recipe applies to arbitrary underlying skill learning methods. Whereas most prior work on empowerment looks at tabular settings, this recipe enables us to study empowerment at a scale where we can realize theoretically-predicted but empirically-elusive properties of empowerment (e.g., gathering resources, environment modification, survival). Experiments show that empowerment maximization unlocks interesting emergent properties, such as building a house in Crafter. Website and code at \href{https://empowerment-geometry.github.io/scaling-empowerment/}{this https url}.

arXiv cs.LG76

SFT 应该学习什么？用于可泛化后训练的条件信息增益

arXiv:2610.12995v1 公告类型：交叉收录 摘要：监督微调（SFT）仍是大语言模型后训练中广泛采用的方法，因为固定的示范可以直接引入期望的推理策略和行为模式。然而，强制模型模仿外部提供的轨迹，也可能干扰预训练已支持的行为，从而限制模型在训练分布之外的泛化能力。近期的动态 SFT 方法根据预测概率、先验支持程度或分布兼容性等模型侧信号，对词元级监督进行重新加权，以缓解这一问题。但与模型的兼容性并不等同于学习价值：当前模型支持程度较低的词元，既可能是不期望的不匹配，也可能恰恰是示范旨在教授的任务特定决策。我们提出 CIG-SFT，这是一种信息论框架，用于估计每个示范词元所携带的、以任务为条件的信息。在教师强制机制下，我们保持标准解答前缀不变，比较冻结的参考模型在能够和不能访问任务输入两种情况下的词元似然。由此得到的条件信息增益，量化了任务本身在多大程度上降低了对示范决策的不确定性。随后，我们将这一信息转化为当前策略的 KL 正则化投影，并推导出等价的标量加权 SFT 目标，其 logit 梯度与投影目标的梯度完全一致。在数学推理和代码生成基准测试中，CIG-SFT 始终优于标准 SFT 和近期的自适应微调基线，尤其在对具有挑战性的评估分布的泛化方面取得了显著提升，同时既不需要额外训练数据，也不需要在线轨迹采样。

arXiv cs.LG67

Forecasting Corpus-Sampling Disagreement in Masked Text Reconstruction

arXiv:2610.12987v1 Announce Type: cross Abstract: If a second text corpus were collected from the same population, how much would a predictor's outputs change? We study this question for masked-text reconstruction using a smoothed predictor built from nearby token-pair counts and evaluated over the full 50,257-token GPT-2 vocabulary. Treating each source document as one observation, we derive a first-order formula for the expected disagreement between predictors fitted to two independent corpora. The formula accounts for variable document length, random context counts, and correlations across token offsets, and it can be evaluated from a separate reference corpus without access to either target corpus. On TinyStories, WikiText-103, and CNN/DailyMail, growing each corpus from 512 to 2,048 chunks of 64 tokens lowers disagreement by only 5--40\%, far short of the 75\% implied by $1/n$ scaling. The forecast tracks this slow decline, which we trace to smoothing of rarely seen contexts. Forecast accuracy depends less on how these correlations are modeled than on the size of the reference corpus: small references systematically underpredict disagreement. A source-split jackknife that uses only reference data reduces this calibration error by 57--59\%. After this correction, a simpler smoothed-multinomial approximation is most accurate when the reference corpus is smaller than the target corpus, and a nonlinear source bootstrap is most accurate when it is larger. Finally, small masked diffusion language models trained on the same corpora show that the count-based forecasts partly identify which inputs are most sensitive to the corpus, but miss the neural models' much faster decline in disagreement.

arXiv cs.LG72

用于事件驱动 Latent Jump Models 的 Scan-Parallel Variational Inference

arXiv:2610.12986v1 公告类型：交叉发布 摘要：连续时间潜在状态空间模型广泛用于不规则观测的时间序列，通常通过随机初始状态及其后平滑的常微分方程（ODE）动力学，或通过随机微分方程（SDE）累积的布朗扰动来表示不确定性。虽然这些构造对于平滑系统或扩散系统而言很自然，但它们并不能直接编码有限个未观测冲击，而这些冲击的发生时间、幅度及其对记忆的影响本身也具有不确定性。为此，我们提出 Latent Jump Models（LJMs）：潜在状态在泊松过程驱动的事件时刻之间遵循闭式确定性动力学，并在每次事件发生时经历一个学习得到的瞬时跳变，其强度和标记分布均依赖于状态。这定义了一个时间齐次马尔可夫过程，其中的跳变是内在的状态转移，而非数值积分步骤。在训练方面，我们推导出一个易于计算的事件增强路径空间证据下界（ELBO），并提出 Scan-Parallel Variational Inference（SPVI）：以观测为条件的变分后验先采样完整的潜在事件配置，再使用三角仿射扫描计算状态递推。这使 LJMs 能够以对数级时间维度计算深度实现快速且可扩展的训练。在合成随机系统和真实的不规则时间序列上，LJMs 的预测精度与连续时间潜变量基线模型相比保持竞争力，其潜在状态更新次数通常少于数值求解器的求值次数，同时训练速度比序贯基线模型快一到两个数量级。

arXiv cs.LG70

NucleoNet: A Nuclear-Aware Determinant Ansatz for Multi-Geometry Neural Variational Monte Carlo

arXiv:2610.12985v1 Announce Type: cross Abstract: Neural-network variational Monte Carlo provides an accurate route to solving the electronic Schr\"odinger equation, but many neural wavefunctions are optimized for a single nuclear geometry, requiring separate calculations to map out a potential energy surface. We introduce NucleoNet, a nuclear-aware determinant ansatz in which electrons and nuclei form a unified particle set propagated through a shared electron--nucleus backbone that jointly evolves one-particle and two-particle representations. Nuclei carry learned representations that evolve alongside the electronic representations and condition orbital construction, while antisymmetry is imposed only by electronic determinants. On equilibrium small-molecule benchmarks, NucleoNet achieves accuracy comparable to established neural-wavefunction baselines with substantially fewer parameters. For each molecular system, a single model optimized jointly across geometries describes the N$_2$ dissociation curve, Li displacement in LiCN, and the ethane and formamide minimum-energy paths for bond dissociation. Sparse-geometry training generalizes to unseen geometries for N$_2$ and LiCN without further optimization, and N$_2$ ablations show that nuclear streams substantially improve this interpolation. These results show that a compact wavefunction with jointly evolved nuclear representations can describe the geometries of a molecular system with a single set of parameters.

arXiv cs.LG60

Minions: A Dynamic and Distributed Virtual GPU Cloud Built on Idle Apple Silicon Devices

arXiv:2610.12970v1 Announce Type: cross Abstract: The increasing demand for GPU-accelerated machine learning inference has exposed limitations in centralized GPU infrastructures, including high cost, limited elasticity, and long wait times under resource constraints. At the same time, modern Apple Silicon-based macOS devices are equipped with powerful GPUs that remain largely idle during normal usage. This paper presents Minions, a dynamic and distributed virtual GPU cloud that aggregates idle Apple Silicon GPUs across macOS devices to execute large-scale machine learning inference workloads. Minions enables teams to leverage existing hardware without requiring additional GPU procurement. The system provides flexible GPU inference task execution, fault tolerance, and efficient resource utilization. Our experience shows that idle macOS devices can be transformed into a practical distributed GPU inference platform in environments where centralized GPU resources are constrained.

arXiv cs.LG66

JoyInside-VLM-Agent: Product-Conditioned Multimodal Policy Learning for Embodied Agents

arXiv:2610.12957v1 Announce Type: cross Abstract: Embodied agents must translate multimodal observations and user requests into executable actions while adapting to product constraints, changing goals, and execution feedback. We present JoyInside-VLM-Agent, a ReAct-style multimodal agent developed through product-conditioned trajectory construction and staged policy learning for robotic platforms and smart hardware. A closed-loop sandbox generates multimodal interaction trajectories covering goal revisions and failure recovery, which are checked against execution outcomes and semantic criteria. Supervised fine-tuning and multi-teacher on-policy distillation precede reinforcement learning with DPER (Dual-Source Privileged On-Policy Self-Distillation with Token-Level Evidence Routing). DPER routes token-level evidence between a frozen privileged teacher and a privileged student sharing the rollout policy's parameters, modulating reward-derived token advantages while preserving their signs. Training rewards combine generative process evaluation, multi-judge rankings, and task-queue consistency. JoyInside-VLM-Agent achieves a BFCL v4 overall score of 77.44, exceeding the evaluated baselines; results on the other public benchmarks are mixed. On JoyInside-Devices and JoyInside-Robotics, the evaluated system, which includes domain-specific skill refinement, leads the baselines on all reported metrics. Two real-world quadruped case studies illustrate visually grounded search, replanning after execution failure, and explicit failure reporting.

arXiv cs.LG68

From Imbalance to Evidence: Deep Nonparametric Testing for Complex Data under Severe Sample Imbalance

arXiv:2610.12946v1 Announce Type: cross Abstract: Comparing distributions under severe sample imbalance requires effectively using abundant majority observations while retaining sensitivity to differences involving the smaller sample. This challenge is particularly relevant for complex, non-Euclidean data, such as graphs, whose structural characteristics call for flexible testing procedures. Classifier two-sample tests (C2STs) provide such flexibility, but existing procedures are primarily designed for balanced or comparably sized samples. In this paper, we study C2STs in a highly imbalanced regime, and show that severe sample imbalance may substantially degrade testing performance. Different from conventional imbalanced learning approaches that treat sample imbalance merely as a nuisance for classifier training, we turn it into a source of repeated testing evidence. Building on this insight, we propose an Enhanced Imbalance-Aware Deep Classifier Two-Sample Test (EIAD-C2ST), a classifier-agnostic framework designed for sample imbalance that accommodates diverse data modalities by tailoring classifiers to data structure. For computationally expensive deep classifiers, we further develop a scalable representation-amortized implementation strategy. Theoretically, we establish rigorous guarantees on asymptotic size and characterize conditions for power consistency, allowing the imbalance ratio to diverge. We demonstrate the empirical gains using simulations and an application to blockchain forensics. The results reveal that the majority-sample surplus, when properly leveraged, can be organized into useful statistical evidence, rather than an obstacle, for distributional comparisons with complex data.

arXiv cs.LG69

Sparse Dictionary Learning for Measure-Valued Data via Linearized Optimal Transport: Efficient Algorithms and Sample Complexity

arXiv:2610.12924v1 Announce Type: cross Abstract: We study dictionary learning for probability measure-valued data by embedding measures into Hilbert spaces using features derived from optimal transport geometry. We learn data-adapted dictionaries through sparse autoencoder parameterizations, producing fast, scalable representations of probability measures as sparse finite-dimensional codes. The resulting framework combines the rich, geometric structure of optimal transport with the interpretability and efficiency of sparse representation learning. We demonstrate its utility across diverse data modalities, including hyperspectral images, geometric point clouds, and document embeddings in large language model latent spaces. We also prove generalization bounds for sparse dictionary learning objectives on broad classes of $L^2$-valued measure embeddings, expressed in terms of the metric complexity of the atom class. For optimal transport and entropic transport map dictionaries, we derive new covering number bounds that yield explicit rates. Together, our computational and theoretical results support a new, scalable model for representation learning in the space of probability measures via embedding into Hilbert spaces.

arXiv cs.LG65

Beyond Second-Order Information: Paid Logistic Regression with Noisy Features

arXiv:2610.12919v1 Announce Type: cross Abstract: We consider online binary prediction with paid stochastic features, where the learner can spend resources to obtain better observations of the underlying features. This setting is motivated by applications such as medical testing, where additional cost can provide more precise measurements of patient's features for predicting a binary outcome. At each round, a latent feature is drawn i.i.d., the response is generated through a logistic model, and the learner chooses how much to pay before observing a noisy version of the feature. The goal of learner is to jointly control prediction loss and acquisition cost. In contrast to the former result on the online linear regression with paid features, covariance information alone is no longer sufficient: two noise distributions with the same covariance may induce different optimal logistic losses, and lower variance can even lead to higher loss. Under a structural ordering in which larger payments yield better observations by removing independent noise, we obtain $\widetilde O(\sqrt{T})$ regret when the noise structure is known and $\widetilde O(T^{2/3})$ regret when it is unknown, with matching lower bounds up to logarithmic factors.

arXiv cs.LG68

BLEND: Balancing Personalization vs. Generalization in Federated Vision-Language Models

arXiv:2610.12907v1 Announce Type: cross Abstract: Federated vision-language models (FedVLMs) represent one of the emerging frontiers of machine learning, aiming to integrate federated learning within the fine-tuning pipelines of vision-language models (VLMs). In this work, we introduce BLEND, a new framework designed to jointly enhance the personalization and generalization capabilities of FedVLMs. BLEND proposes a selective VLM parameter fine-tuning and aggregation strategy that consists of (i) global vision and text adapters shared across clients, and (ii) a personalized vision projection adapter tailored to each client. BLEND further introduces a novel loss function that blends the outputs of the global and personalized adapters to achieve personalization, while simultaneously promoting generalization through a principled anchor design implemented via the projection heads of the zero-shot VLM. Extensive experiments demonstrate that BLEND consistently outperforms state-of-the-art baselines, highlighting its ability to effectively balance personalization and generalization across diverse system settings. (Code is available at: https://github.com/payamsiabd/BLEND.git)

arXiv cs.LG77

重新思考扩散与解码的作用：用于并行语言生成的 Predictive-State Flows

arXiv:2610.12898v1 公告类型：交叉发布 摘要：在连续扩散语言模型（DLMs）中，扩散先验生成的潜变量通过解码器或词元嵌入矩阵，以近乎确定性的方式映射为词元。与视觉细节可以连续变化的图像生成不同，扩散先验必须在连续空间中捕捉细粒度的离散词元选择，这可能使潜变量的联合分布难以建模。我们认为，生成任务的职责应当重新划分：扩散先验应生成高层次的预测信息，而解码器则负责解决剩余的词元级选择。基于这一观点，我们提出 Predictive-State Flows（PSF），将序列划分为多个块，并为每个块关联一个仅根据前文上下文计算的预测状态。整流流先验生成预测状态序列，随后由块因果解码器并行生成所有块，仅在各块内部进行自回归生成。这一设计将扩散处理的状态从 L 个词元级状态减少为 K << L 个预测状态，并将词元级的顺序计算深度降至 L/K。在 OpenWebText 无条件生成任务中，一个拥有 1.97 亿参数、K=8 的模型，每生成一篇含 1024 个词元的文档仅需 0.31 TFLOPs，在熵为 5.16 时实现了 22.8 的生成困惑度；相比之下，采用 32 个采样步的 ELF-B 的生成困惑度为 24.0，计算量为 7.2 TFLOPs。在对应的实测运行配置下，两者的 MAUVE@512 分别为 0.93 和 0.73。受控比较进一步表明，额外编码了自身块信息的内容状态，其生成困惑度比预测状态高 8–16%；而将采样得到的预测状态轨迹替换为共享均值，会使 MAUVE@512 降低 0.11。这些结果表明，连续 DLMs 的潜变量设计不仅应考虑文本表示的紧凑程度，还应考虑先验与解码器之间的生成职责划分。

arXiv cs.LG67

Generalized Determinantal Point Processes for Attraction and Repulsion with Distinct Interaction Kernels

arXiv:2610.12894v1 Announce Type: cross Abstract: Determinantal point processes (DPPs) model subset selection probability with repulsion, but many applications also exhibit attraction arising from pairwise relations or shared latent structures. Nonsymmetric DPPs (NDPPs) capture both effects, but their kernels are difficult to interpret and specify. We propose the determinant-ratio point process (DRPP), which models repulsion and attraction via a ratio of the determinants of two symmetric positive-definite kernels. This formulation preserves the feature-based interpretation of standard DPPs while allowing the two effects to be specified separately. Furthermore, we introduce an auxiliary-variable representation that enables efficient MCMC sampling and yields tractable expectation formulas for key statistics including the normalizing constant, which together facilitate maximum-likelihood learning. We theoretically characterize the interaction structures well suited to DRPPs: for clustered attraction with $C$ latent groups, DRPPs exactly realize the target pairwise interactions with an attraction rank $C$, whereas NDPPs can require a rank that grows with the number of items given the same repulsion component. Conversely, nontransitive pair-specific attraction may require compensation through the DRPP repulsion kernel. Experiments support these predictions, showing that DRPPs fit transitive attraction with substantially fewer parameters than NDPPs.

arXiv cs.LG73

自归一化 Kullback-Leibler 估计的通用界

arXiv:2610.12851v1 公告类型：交叉发布 摘要：本研究考察了一种基于熵的 Kullback-Leibler（KL）散度估计量，该估计量由归一化重要性权重构造而成。我们证明了一个通用的有限样本界，表明对于任意样本量，该估计量的期望至多比目标分布相对于提议分布的 KL 散度高出 $\log 2$。随后，我们刻画了重要性加权重采样所诱导的分布与提议分布之间的 KL 散度，并表明，基于熵的估计量的期望以及目标分布相对于提议分布的 KL 散度均为其上界。重要的是，这些界既不要求重要性权重有界，也不要求高阶矩有限。我们将这些结果应用于通过 Soft Best-of-$N$ 采样实现的生成模型测试时对齐，以及相对熵编码，得到有限样本下的策略散度保证和无需矩条件的编码成本界。Soft Best-of-$N$ 采样是 Verdun 等人（2025）提出的一种基于重要性采样的扩展技术。

arXiv cs.LG71

双向引导：面向生成式贝叶斯的摘要统计量同步学习

arXiv:2610.12850v1 公告类型：交叉收录 摘要：随着观测样本量增大，基于模拟的贝叶斯推断（SBI）可能难以恢复日益集中的后验分布。我们将充分摘要统计量与后验分布的联合学习视为一项通用原则，以实现各种 SBI 方法中的可扩展推断。SBI 中的一种常见做法是通过预先确定的摘要统计量进行降维，但这可能丢失部分后验信息。相比之下，我们的方法对后验学习器依赖于所学摘要统计量的方式施加结构约束（例如仿射性），并将摘要统计量与后验分布同步学习。我们发现了一种重要机制，称为双向引导：在对摘要统计量施加结构约束的条件下，所学摘要统计量的后验充分性简化了后验学习，而恰当的后验目标函数则引导所学摘要统计量趋向后验充分性。这一机制解释了一个反直觉的发现：对摘要统计量施加结构约束能够改善后验学习，而非限制其性能。我们针对后验估计误差以及所学摘要统计量对充分性的偏离程度均建立了理论保证，从而明确了随着观测样本量增大，为恢复日益集中的后验分布，模拟预算需要满足的条件。

arXiv cs.LG75

APEX：用于极致压缩的异步前缀去噪

arXiv:2610.12843v1 公告类型：交叉发布 摘要：更少的潜在词元有望降低扩散模型的计算成本。然而，激进的潜在空间压缩会造成重建与生成之间的两难：增加通道数可以改善重建，却会使生成更加困难。我们提出 APEX，其全称为 Asynchronous Prefix denoising for EXtreme compression，将分层潜在空间与针对通道组的异步去噪相结合。我们的自编码器将通道前缀按从结构到细节的顺序排列，而一个共享去噪器则在错开的局部时间步生成这些通道组，无需增加词元。在 ImageNet-512 上，在 64 倍和 128 倍压缩条件下，APEX 在所有评估的通道数设置中均优于现有方法。在训练和引导采样计算量相同的情况下，它在所有评估的计算预算下也均优于一个 256 词元模型，且计算量越低，优势越明显。代码见 https://daviddinkevich.github.io/APEX。

arXiv cs.LG72

COFFEQA：用于大语言模型问答的反事实引导少样本蒸馏

arXiv:2610.12837v1 公告类型：交叉收录 摘要：知识蒸馏旨在训练较小的学生模型，使其模仿较大的教师模型，以适用于资源受限的环境。然而，蒸馏过程通常需要大量数据，且计算成本高昂，尤其是对于生成式任务。少样本蒸馏提供了一种替代方案，通过有限的训练样本迁移教师模型的知识。然而，我们观察到，少样本蒸馏的性能在很大程度上取决于这些训练样本的选择。事实上，我们证明，使用教师模型决策边界附近不确定性更高的样本，会显著降低蒸馏性能。受此观察启发，我们提出了用于开卷问答的反事实引导少样本蒸馏方法（COFFEQA）。这是一种新策略，能够巧妙利用反事实，为具有挑战性的事实样本提供额外监督。在生成式问答场景中，我们将反事实定义为经过最小幅度修改的上下文，该上下文会促使教师模型将其回答改为另一个答案。我们从几何角度阐释了事实—反事实样本对如何约束教师模型与学生模型决策边界之间的偏差。我们还引入了一项新指标——上下文遵循难度（CFD），根据回答与预期替代答案的一致性，以及对修改后上下文的依赖程度，在蒸馏前进一步验证反事实。COFFEQA 使用经 CFD 验证的事实—反事实样本对进行少样本蒸馏。采用 T5 和 Qwen2.5 系列模型，在五个域内和三个域外问答基准上开展的实验表明，在相同的少样本预算下，COFFEQA 优于仅使用事实样本的蒸馏基线。例如，在 TriviaQA 上，使用 T5 模型和 32 个少样本对时，COFFEQA 相较于仅使用事实样本的基线，平均 EM 分数提高了 8.77 分，平均 F1 分数提高了 9.98 分。

arXiv cs.LG61

Scalable Learning of Determinantal Point Processes with Kernel-Structured Interactions

arXiv:2610.12829v1 Announce Type: cross Abstract: Determinantal point processes (DPPs) on a finite set are probability models that exhibit repulsion where inter-item similarities are encoded by a positive definite kernel matrix. Scaling DPP learning to large datasets typically requires imposing structural constraints on the kernel matrix, such as a low-rank-plus-diagonal form. However, structural assumptions introduce a severe trade-off between expressive power and computational scalability. Our study addresses this challenge by (a) modeling the full-rank interactions via a nonlinear kernel function and latent features, (b) designing a surrogate function for the maximum likelihood estimator, and (c) adopting stochastic optimization using random features of the nonlinear kernel. Numerical experiments on real-world datasets demonstrate that our method significantly improves this trade-off, achieving better generalization performance with remarkable parameter efficiency.

arXiv cs.LG73

VEPA：面向 LLM 智能体终身护栏的经验证演进式形式化策略

arXiv:2610.12813v1 公告类型：交叉发布 摘要：LLM 智能体越来越多地操作私有文档和可变系统状态等敏感资源，这可能带来严重的安全与安保风险。因此，运行时护栏应在执行可能不可逆的智能体操作之前，作出可检查、可复现且确定性的允许或拒绝决策。然而，安全与安保要求在部署前很少完备，因此护栏需要随着新风险的出现而持续适应。现有自适应护栏会从部署经验中学习，但通常通过 LLM 推理来应用安全策略，这可能导致决策不一致和策略解释模糊。另一方面，形式化护栏能够以确定性方式执行策略，但通常依赖部署前制定的策略或特定任务策略，缺乏适应新观测风险的机制。我们提出 VEPA，一个用于形式化智能体护栏经验证演进的框架。VEPA 将护栏表示为一种形式化策略，由针对带类型的智能体事件的可执行符号规则构成。在部署过程中，我们使用部署反馈引导 LLM 提出策略补丁，以修复新观测到的失效，同时保留此前正确的行为。随后，确定性验证器仅接受结构有效且无冲突的策略。最后，一个证据感知的授权层仅在已验证规则获得充分支持证据时才将其激活，从而防止不确定的更新直接影响护栏决策。在 ASSEBench-Safety、ASSEBench-Security 和 ATBench 上，VEPA 在扩大确定性形式化策略执行范围的同时，始终优于基线方法。这些提升在更具挑战性的部署条件下同样成立，包括混合流、受损反馈和更长的运行周期。

arXiv cs.LG64

Trainable Quantum-Native Language Models: Dissipative Memory, Capacity Limits, and Hardware Validation

arXiv:2610.12806v1 Announce Type: cross Abstract: We study byte-level autoregressive language models whose hidden state is a small quantum register, whose transitions are learned token-conditioned quantum channels, and whose next-byte probabilities come directly from Born-rule measurements without a learned classical vocabulary projection or softmax. In the fixed-width recurrent setting, unitary dynamics cannot contract or reset stored distinctions; experimentally, unitary recurrence loses to its retrained one-token control in every tested variant. Introducing learned dissipation, then token-conditioned dissipative gating, yields useful memory beyond one token. The gated model improves test negative log-likelihood by 0.20 +/- 0.03 nats/byte over retrained parameter-matched one-token controls at context length 12, and by 0.29-0.33 at longer contexts, but never beats retrained two-token controls. A key-value recall benchmark shows a width-dependent threshold: one stored pair meets the pre-registered accuracy bar at 4-5 memory qubits, two pairs at 6-7, and three pairs remain unsolved. This is consistent with, but does not establish, a capacity explanation for the short effective memory horizon. As a secondary result, one-qubit models reach lower stationary memory entropy than a heuristically optimized classical predictive frontier at matched error in 9/9 seeds. On IBM Heron hardware, computational-basis memory-register distributions show signal against a uniform null in 12/12 circuits across prefixes of up to 12 tokens, next-bit probabilities pass pre-registered accuracy bars at four-token prefixes, and bytes are generated autoregressively by sampling physically measured bit probabilities. The tested models are classically simulable and substantially worse than classical n-gram baselines. We claim no computational quantum advantage.

arXiv cs.LG68

Polyconvex neural-network surrogates with learned paired directional features for anisotropic hyperelastic homogenization

arXiv:2610.12801v1 Announce Type: cross Abstract: Approximating the effective anisotropic hyperelastic response of a representative volume element (RVE) requires constitutive expressiveness, mechanical consistency, and computational efficiency. This work develops a polyconvex physics-augmented neural network (PANN) construction based on paired directional features whose orientations, stretch powers, and determinant exponents adapt to the RVE response. The features are composed with convex, componentwise nondecreasing cores, realized as an input-convex neural network (ICNN) or an input-convex Kolmogorov-Arnold network (ICKAN). The complete energies satisfy objectivity, an exactly stress-free reference, two-dimensional polyconvexity, and prescribed growth conditions by construction, and nonnegative energy is certified separately for the selected models. On a multicavity and a single-cavity periodic cell, these energies are compared with an Unconstrained energy, trained identically without the curvature and growth constraints, and with three hyperreduced projection-based reduced-order models (HPROMs), which serve as physics-based references. On the first cell, learning the features lowers the stress error sevenfold, and six features suffice on both. The constrained energies remain below one percent in stress error and in the displacement error of a tensile-coupon FE$^2$ application with a full-order reference, whereas the Unconstrained energy and the HPROMs are more accurate inside the data. Beyond the data, however, the Unconstrained energy loses rank-one convexity, and under confined compression it stops below half of the 500 MPa that the constrained energies reach. These results quantify what the guarantees cost in accuracy and what they secure beyond the data.

arXiv cs.LG71

Trajectory Prediction for Autonomous Driving in Road Construction Zones Using Human-in-the-Loop Driving Data

arXiv:2610.12747v1 Announce Type: cross Abstract: Ego-trajectory prediction in urban roadway work zones remains an underexplored topic in autonomous-driving research, with limited dedicated data and systematic evaluation. Roadway work zones involve temporary changes in the lane layouts and connectivity of the roads, complicating interactions with surrounding vehicles and workers. These changes make future vehicle trajectories difficult to predict. To address these challenges, we collect WZDrive-Data, a multimodal dataset comprising 1,183 work-zone driving segments from 33 drivers, using a human-in-the-loop driving simulator with prerecorded worker animations and VR-controlled workers. Driver control and gaze signals are recorded from human participants, while vehicle trajectories and scene information are obtained from the simulator. Our proposed multimodal ego-trajectory predictor, WZDrive, achieves a mean minADE (K=6) of 1.09 m and minFDE (K=6) of 2.12 m. Compared with the best-performing state-of-the-art baseline for each metric in our benchmark, WZDrive reduces minADE (K=6) by 7.18% and minFDE (K=6) by 13.18%. This research addresses a gap in autonomous-driving research on urban work zones by combining human-in-the-loop data collection with multimodal learning from human driving signals and in the context of work zones.

arXiv cs.LG62

Minimax Rates for Estimating Powered Sliced Wasserstein Functionals

arXiv:2610.12740v1 Announce Type: cross Abstract: We study the two-sample estimation of the powered Sliced Wasserstein functional under projected $r$-th moment assumptions. Let $N=n\wedge m$, where $n$ and $m$ are two sample sizes. We establish the minimax rates in the noncritical regimes. When $p 2p$ it is of order $N^{-1/2}$. In both cases, the natural plug-in estimator is minimax rate-optimal. At the endpoint $r=p$, the minimax risk is bounded away from zero, so uniformly consistent estimation is impossible. At the critical order $r=2p$, we obtain the exact worst-case rate of the plug-in estimator and a lower bound for the minimax risk, leaving a logarithmic gap between the lower and upper bounds. The exact minimax rate in this critical case remains open.

arXiv cs.LG75

StereoEngine：无需双目相机即可扩展双目视觉双臂机器人数据规模

arXiv:2610.12734v1 公告类型：交叉收录 摘要：双目视觉正越来越多地用于机器人操作，越来越多的研究将其融入视觉-语言-动作（VLA）模型。然而，大规模真实世界双目视觉数据集的匮乏，仍是此类策略规模化发展的根本瓶颈。近期的生成式单目转双目方法主要面向 VR 内容和自动驾驶等场景。此前尚无研究为机器人操作生成逼真的合成双目数据，该领域的双目监督信号主要来自仿真。我们提出 StereoEngine，引入两种互补的方法来合成机器人操作所需的双目数据。第一种方法利用估计深度，将单目（左视角）视频变换为右视角，并使用视频扩散模型修复因遮挡解除而显露的区域，从而无需使用真实双目数据进行训练，即可将单目示范转换为双目视图对。第二种方法先在仿真中渲染双目视图对，再联合生成两个照片级逼真视图，无需任何真实视频作为输入，即可生成左右一致的数据。StereoEngine 在生成双目数据的质量上优于现有基线方法，且将其合成数据用于策略的中期训练或特定任务微调，能够显著提升下游双臂操作性能。项目网站：https://stereoengine.github.io

arXiv cs.LG78

Zebra3D：面向可扩展三维重建的高效混合基础模型

arXiv:2610.12727v1 公告类型：交叉发布 摘要：前馈式三维基础模型能够从无位姿图像中重建场景，但全局注意力的二次复杂度开销限制了其向大规模视图集的扩展。我们提出 Zebra3D，这是一种逐层稀疏—线性混合注意力框架，无需从头训练即可提升预训练三维基础模型的效率。在 VGGT、HunyuanWorld-Mirror 和 $\pi^3$ 中，我们观察到一种大体一致的随网络深度变化的模式：早期阶段对线性注意力替换的容忍度相对较高，而敏感的中后期阶段往往呈现出更集中的注意力。Zebra3D 利用这一结构，为容忍度较高的阶段分配以线性注意力为核心的注意力机制，为敏感阶段分配分块稀疏注意力，并使用地标摘要保留未选中的上下文。闭式仿射拟合支持免训练转换，而可选的转换后训练则将任务监督与输出蒸馏、关系特征蒸馏相结合。在全部三种骨干网络上，Zebra3D 均在高注意力稀疏度下实现了出色的重建精度和相机位姿精度，其免训练变体的表现仍接近对应的稠密模型。端到端加速比随输入视图数量增加而提升，在 1,024 个视图时最高达到 $7.1\times$。经过转换后训练，Zebra3D-VGGT 在平均注意力稀疏度为 94\% 时，RealEstate10K AUC@30 达到 80.5\%，与使用相同数据和训练方案微调的稠密 VGGT 相差不超过 1.1 个百分点。

arXiv cs.LG71

A Distributional Framework for Behavioral Characterization and Risk Management of Large Language Models

arXiv:2610.12718v1 Announce Type: cross Abstract: Large language models are stochastic by nature. Repeated executions under the same experimental conditions may produce different responses. We propose a framework that can be applied across the model lifecycle, from task design to residual behavior and continuous monitoring. The method consists of building an empirical response distribution estimated through Monte Carlo simulation. Semantic interventions are introduced to study how these distributions change. We illustrate the framework with two different tasks, borrower classification and music album recommendation, and quantify the resulting distributional changes. The framework is linked to model risk management and governance by measuring robustness and control effectiveness, and characterizing residual behavior. Since it relies only on observed responses, it can also be applied to black-box models accessed through commercial APIs.

arXiv cs.LG70

PVBRF-ID: Private, Verifiable and Byzantine-Robust Federated Intrusion Detection for Oilfield Enterprise Networks

arXiv:2610.12704v1 Announce Type: cross Abstract: Federated learning allows dispersed oilfield sites to train a shared intrusion detector without pooling network traffic, yet its aggregation step forces a choice between privacy and robustness. Byzantine-robust rules must read each update in the clear, whereas secure aggregation hides updates from the checks that detect poisoning. We present PVBRF-ID, a two-server protocol in which sites secret-share their updates and the servers jointly test each update's norm and its cosine similarity to a reference computed on a small clean root set, revealing one accept bit per site. Commitments, Merkle roots and a homomorphic audit make the inclusion of accepted updates publicly verifiable. On a 15-class Edge-IIoTset task with ten sites holding differently distributed data and ten random seeds, PVBRF-ID matched FedAvg on clean data (macro-F1 0.708 vs. 0.706) and withstood sign-flip attacks (0.696 vs. 0.091). Against an adaptive attacker controlling 30% of sites, PVBRF-ID with secure clipping reached 0.641, whereas FedAvg fell to 0.046; plaintext multi-Krum remained higher by 0.052. An update-inference attacker recovered each site's class mix from plaintext updates (Spearman correlation 0.81) but not from one server's view. Backdoors passed the filter, and noise-based mitigation gave only heuristic protection. The implementation uses Python 3.13, NumPy 2.3, 30 rounds and 10 seeds.

arXiv cs.LG69

Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation

arXiv:2610.12703v1 Announce Type: cross Abstract: High-risk applications of deep learning require precise characterization of model decision-making. We conduct a systematic explainability evaluation across Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and Vision Mamba (ViM). Unlike predominantly qualitative studies, we quantify the alignment between model attention and human visual perception using eye-tracking ground truth. We formalize this comparison via a standardized normalization pipeline mapping post-hoc attribution maps and gaze heatmaps onto spatial probability distributions. Alignment is evaluated across fine-grained classification, medical imaging, and emotion recognition using architecture-specific XAI methods. We measure fidelity using Jensen-Shannon similarity, Chi-square similarity, and Pearson correlation, three complementary estimators capturing support overlap, mass discrepancy, and structural co-variation. Our results reveal distinct attention dynamics: CNNs tend to exhibit a more global focus, with spatially more diffuse saliency maps that correlate well with human perception in general contexts, while ViTs demonstrate fine-grained, patch-level attention, often focused on discriminative microfeatures. The Mamba architecture shows biased behavior: it achieves strong explainability in medical tasks but exhibits divergent intensity distributions in natural images; our analytical characterization of the metrics makes this dissociation explicit, since near-unit support overlap is attained simultaneously with a vanishing linear correlation. We further analyze the effect of backbone freezing and ensemble strategies on model interpretability. The results highlight how architectural and training choices influence the alignment between model saliency and human visual attention.

arXiv cs.LG75

Product Monte Carlo：密度乘积下的期望

arXiv:2610.12702v1 公告类型：交叉发布 摘要：汇总独立计算的子样本推断结果，是大规模贝叶斯计算、联邦学习和意见汇聚中一个众所周知的难题。核心问题是近似一个概率密度与各个密度之积成正比的分布。我们提出 Product Monte Carlo（PMC），这是一种仅使用来自各个密度的蒙特卡洛样本，即可计算密度乘积下任意期望的算法。我们通过一族可计算的条件期望来表示难以直接计算的密度乘积下的期望，从而将其转化为回归问题。除点估计外，我们的全贝叶斯回归框架还提供一致的不确定性量化和具有理论依据的诊断方法。我们为 PMC 估计量建立了一致性的理论保证，并刻画了其收敛速率。在改变各组成分布的数量、维度和异质性的模拟实验与真实数据实验中，相比现有方法，PMC 以更低的计算成本实现了更高的估计精度、校准更好的不确定性覆盖率以及更强的可扩展性。此外，PMC 无需计算密度值，因此适用于基于模拟的推断；我们通过一项蛋白质图像应用展示了这一点。

arXiv cs.LG62

Toward Maximizing Algebraic Connectivity via Density-Dependent Backbone Construction and Reinforcement Learning Completion

arXiv:2610.12697v1 Announce Type: cross Abstract: This paper studies an open network design problem of constructing graph topologies with maximum algebraic connectivity under fixed numbers of vertices and edges. A density-dependent approach is proposed that decomposes the design task into two stages: a high-quality, low-cost structural backbone construction and a reinforcement learning completion stage. The method selects the backbone based on density, using complete multipartite graphs in dense regimes and Cayley graphs in low-to-intermediate regimes, and then refines the remaining edges through a learning-based procedure. Numerical results show that the proposed method achieves high algebraic connectivity with favorable runtime scalability, outperforming existing methods. These results support a structural-family-first strategy for scalable network design.

arXiv cs.LG77

MagicBeanStalk：通过递归数据合成摆脱 RLVR 零优势陷阱

arXiv:2610.12659v1 公告类型：交叉发布 摘要：以数据为中心的 RLVR 扩展一直侧重于收集或生成可验证的训练问题。然而，仅凭语料库规模并不能判断数据集是否能为目标策略提供有效的学习信号。为指导有效 RLVR 数据的合成，我们通过实验，将数据集的有效性刻画为其所含问题、目标策略和训练 rollout 预算的函数。基于这一刻画，我们发现，一个汇集了广泛使用的 RLVR 数据集的大型开源数学语料库几乎完全陷入了零优势陷阱：对于固定的目标策略和 rollout 预算，其中约 88% 的问题要么在每次 rollout 中都能解决，要么在每次 rollout 中都无法解决。筛选可以提高剩余有效问题的集中度，但它只能进行选择，同时丢弃难题。因此，有效的合成问题应当要求使用解决这些难题所需的推理操作及其组合。不过，这些问题的难度必须经过校准，使部分 rollout 成功、部分失败，并且至少有一个正确解答能够在训练预算内完成。直接提示教师模型对难题进行变式改写并不能达到这一目的。在这些实证原则的指导下，我们提出了 MagicBeanStalk，一种结合自顶向下分解与自底向上递归数据合成的算法。它在保留种子问题推理蓝图的同时，将种子问题扩展为一系列在实质上全新、且针对策略和预算进行了校准的训练问题。在 8 个数学基准测试中，在 16K token 预算下，使用 MagicBeanStalk 问题持续进行 RLVR 训练，使 gpt-oss-120b 的成绩从 61.2 提升至 69.8（+8.6），使 DeepSeek v4Flash 从 72.4 提升至 77.9（+5.4）。使用完整的 7.8 万道问题源语料库进行训练仅提升了 1.1 分，而使用其中经筛选得到的最佳真实数据子集则提升了 5.8 分。在金融领域，我们仅基于 340 个解答就合成了超过 1.4 万道可验证问题，并在 SecQue 和 FinanceBench 上分别比基础模型提升了 6.9 分和 2.4 分。

arXiv cs.LG76

SpireBench：使用 Slay the Spire 评估智能体模型的长程策略对局能力

arXiv:2610.12596v1 公告类型：交叉收录 摘要：智能体大语言模型如今已能处理复杂任务，但对其长程策略推理能力的研究仍显不足：大多数智能体基准测试流程较短，且以独立回合为单位，而模型在长程任务中的表现又难以拆解为已有知识、推理能力与运气各自的贡献。我们提出 SpireBench，让模型游玩完整的商业牌组构筑游戏 Slay the Spire（StS）；在该游戏中，早期决策决定了长期成败。我们在游戏最高难度下测试了 20 个大语言模型（LLM），发现前沿模型超越了启发式基线，并达到了普通人类玩家的平均水平，但仍远逊于专家玩家。随着单局进程推进，这一差距进一步扩大：最强模型通过第一幕的频率与专家相当，随后却逐渐落后。强弱模型之间的区别在于每场战斗中的表现；所有模型都会误判整局游戏中的风险，倾向于保留生命值，而不是消耗生命值来增强实力。对所有游戏术语进行全面且内部一致的重命名后，我们发现，模型的表现部分依赖于对游戏的先验知识；此外，模型也未能从自己在多局游戏中记录的记忆中获得提升，这些记忆往往有误且过于谨慎。我们的测试框架首次向模型提供了完整的原版游戏，并与现有框架进行了比较。总体而言，SpireBench 提供了一个尚未达到性能饱和的长程基准，并配备了用于定位智能体失败环节及原因的工具。