全部动态

本页 40 条
10月12日 周一
arXiv cs.LG68

Shared Canonical Component Removal Beyond Expert Views

arXiv:2610.13819v1 Announce Type: new Abstract: In clinical practice, experts often begin by converting raw inputs into a set of expert features before making a decision. While black-box models trained on raw input can outperform models trained on expert views, it remains unclear what predictive signal the raw model exploits beyond what the expert view captures. We introduce an audit framework to quantify this gap using Shared Canonical Component Removal (SCCR). Unlike standard residualization or concept erasure, SCCR selectively removes only those directions strongly shared between learned raw and expert embeddings. Evaluations on synthetic data, coronary artery disease (CAD) detection, antidepressant treatment dropout prediction, and skin lesion classification using SkinCon annotations show that SCCR identifies the ways in which expert views are insufficient and recovers residual predictive signal while minimizing redundancy with the expert view.

arXiv cs.LG71

TabMF：利用表格基础模型学习数据集表示

arXiv:2610.13818v1 公告类型：新提交 摘要：监督学习数据集的表示可支持数据集检索和分布偏移监测等更高层次的机器学习任务。然而，现有方法通常依赖于手工设计的元特征，或专门为表示学习训练的数据集编码器。我们提出 TabMF，这是一个将预训练表格基础模型（TFM）用作监督学习数据集编码器的框架。TabMF 为预测型 TFM 配备了架构感知的数据集级读出机制，并利用合成监督学习数据集的随机视图进行对比式后训练，以端到端方式适配模型。我们基于 TabICLv2 和 TabPFN-2.5 实现 TabMF，并在未见过的真实数据集上，通过六项任务评估所得表示。简单的冻结 TFM 读出机制即可产生具有竞争力的表示，这表明预测式预训练已经形成了丰富的数据集级结构。这两个 TabMF 编码器在所评估的学习型和经典数据集表示方法中取得了最佳综合性能，尤其在数据集检索、紧凑基准构建和分布偏移严重程度评估方面表现突出。

arXiv cs.LG73

通过精确高斯条件化统一扩散与基于流的后验采样

arXiv:2610.13812v1 公告类型：新论文 摘要：流匹配和扩散模型日益被用作线性逆问题的先验，然而，大多数方法都是针对某一类轨迹先验专门设计的。虽然已有即插即用方法被提出，但这些方法隐式地消除了引导轨迹的相关性，导致先验分布过于宽泛，在数据稀疏时可能表现不佳。本研究提出 $\beta$-Flow：一种通用的即插即用方法，由相关性参数 $\beta$ 和置信度参数 $\kappa$ 参数化；前者将流先验、扩散先验和去相关先验统一起来，后者控制对路径分布的约束强度。每一步仅需对去噪网络进行一次前向计算，随后执行高斯条件化步骤。该步骤是不依赖网络梯度的精确 Kalman 更新。我们表明，许多现有的引导采样器可被解释为我们方法的实例或近似，每种方法都隐式地采用了不同的轨迹先验，并使用不同的算子替代 Kalman 增益。从这一视角出发，我们根据各方法的隐式偏置和假设进行了定性比较。我们在 4 个数据集上的 7 类逆问题中，均在 PSNR 和 SSIM 指标上取得了达到最先进水平的性能。此外，我们还展示了这种先验控制的实用性：当观测所含信息较少时，高置信度、高相关性的设置（严格先验）效果最佳；而当观测所含信息较多时，低相关性更为适合，这体现了先验与似然在贝叶斯推断中的互补作用。

arXiv cs.LG69

QTestBench: A Test Framework for Evaluating LLM-Generated Quantum Code

arXiv:2610.13811v1 Announce Type: new Abstract: Large language models (LLMs) have shown strong results in classical code generation and are increasingly used to write the unit tests that safeguard that code. Whether they can do the same for quantum software is less clear, because quantum programs are probabilistic, depend on numerical tolerances, and are judged by domain-specific checkers, so a well-formed test can still disagree with the benchmark's verdict. Existing quantum benchmarks measure whether a model produces a correct solution, but not whether its tests can distinguish correct implementations from faulty ones. We introduce QTestBench, a framework that evaluates LLM-generated quantum test suites against independent benchmark checkers, never treating the tests themselves as ground truth. For each task, QTestBench prompts models to produce syntactic, semantic, and behavioral tests, injects candidate solutions at runtime, and compares each test verdict with the checker's label under three validation cohorts, each answering a distinct diagnostic question. We benchmark five LLMs on eight PennyLane QHack challenges and ten Qiskit HumanEval tasks. Generated suites almost always execute, but discrimination lags behind: the best generator reaches a test quality of 0.71 on QHack and 0.34 on Qiskit HumanEval, so executability alone overstates test quality. Oracle-verified synthetic mutants stabilize bug-detection estimates when natural faulty solutions are scarce, as on three of ten Qiskit tasks that yield none. QTestBench offers a reproducible, cross-framework benchmark for the trustworthiness of LLM-generated quantum tests.

arXiv cs.LG78

FORTE：面向工业推荐系统的特征评估与弃用生产框架

arXiv:2610.13806v1 公告类型：新论文 摘要：生产环境中的推荐系统存在特征生命周期不对称性：添加新特征必须在某项指标上带来收益，而移除特征则不能导致任何指标退化。在实践中，特征弃用很难获得优先处理，因为以下情况均可能导致特征无法被移除：它与其他特征存在交互，其重要性每天都在变化，无法在生产模型发生变化或数据漂移出现之前完成评估，需要与其他上线项目争夺 A/B 测试资源，在在线 A/B 测试中导致指标退化，或被多个模型使用而需要协调弃用。我们提出 FORTE，这是一个与模型无关的生产框架，用于发现、验证和移除特征。其高效的扰动实现支持每日特征评估。为考虑特征间的交互，该框架通过分组消融来评估由所有排名较低的特征构成的候选特征集；其离线结果对在线 A/B 测试具有足够的预测能力，因此可以有把握地将弃用方案推进到在线测试阶段。血缘追踪能够识别所有使用相关特征的下游模型，从而充分实现从训练集中移除特征所带来的成本节约。我们在两个大规模推荐系统上验证了 FORTE，发现它能够识别出不会导致在线指标退化的可弃用特征集，而其对效率的影响取决于模型特性。在一个搜索相关性模型上，FORTE 生成了三个嵌套的候选特征集，分别包含 32、48 和 59 个特征。在线护栏指标否决了最大的实验方案。最终选定包含 48 个特征的集合进行弃用，移除了该模型 43.3% 的特征。在一个大型排序模型上，FORTE 识别出了 8 个待移除的候选特征。重新训练和 A/B 测试确认未出现指标退化。该方案上线后，可训练参数减少了 29.9%，推理服务吞吐量提高了 19.8%–25.5%，每年训练成本降低了数十万美元。

arXiv cs.LG70

Randomized Momentum Transport with Cheap Predictors

arXiv:2610.13789v1 Announce Type: new Abstract: Momentum methods such as STORM and second-order momentum track the gradient by adding a transport correction to momentum, at the cost of an extra oracle call per iteration. We introduce randomized momentum transport, which evaluates the correction with probability $q$ and otherwise uses a cheap prediction, with importance weighting that preserves the conditional mean and adds $(q^{-1}-1)$ times the prediction loss to the variance. Under transport-bias, second-moment, and residual-control assumptions, linear minimization oracle (LMO) updates, including normalized, sign, and exact spectral (Muon) directions, retain an $O(T^{-1/3})$ stationarity bound for each fixed $q>0$ with a single-gradient initialization. For unnormalized STORM, some $q<1$ lowers the leading coefficient of our sufficient gradient budget if and only if the gradient-difference noise modulus exceeds the smoothness constant, by at most about $10\%$. An online diagonal predictor admits a regret-to-convergence guarantee. At approximately equal oracle cost, querying a quarter of the corrections lowers the final training loss by $18$ to $43\%$ on MNIST1D and raises the test accuracy of Hessian-vector transport on CIFAR-10 by $1.5$ points.

arXiv cs.LG73

你的离散图扩散模型正在悄悄记忆训练数据

arXiv:2610.13786v1 公告类型：新稿 摘要：离散图扩散模型（GDMs）在马尔可夫噪声模型下，通过向节点和边独立添加噪声来扰动图结构。在没有节点特征的无条件设定下，仅在单个图上训练的等变去噪器就能够生成新图，而边重叠率和 WL 核相似度等样本级指标并未发现记忆现象的证据。在本研究中，我们挑战了这一观点，并首次证明图扩散模型确实存在记忆现象。通过分析在双块随机块模型（SBMs）上训练的 GDMs 的反向扩散轨迹，我们发现了显著的训练数据偏向：与留出的测试图相比，模型能够近乎完美地恢复被扰动的训练图。我们将其称为条件记忆，因为它仅在反向轨迹以被扰动的训练图为条件时才会显现，而在推理过程中并不明显。这一现象伴随着不断扩大的泛化差距，即训练损失下降的同时，留出图上的损失却在上升，而且无法通过早停来缓解。记忆现象的出现始终早于生成图中社区结构的涌现，因此，在此之前的每个检查点生成的图都不具备结构。这种行为在我们测试的所有节点规模下均存在。

arXiv cs.LG79

ReplaySSM：缓存 SSM 输入，而非状态

arXiv:2610.13778v1 公告类型：新论文 摘要：状态空间模型（SSM）维护一个固定大小的循环状态，但在每个解码步骤中读写该状态可能成为混合模型的性能瓶颈。这种即时状态管理会产生算术强度较低的大量状态 I/O，使推测解码缺乏低成本的回滚方式，并将状态量化误差传递到后续循环更新中。我们提出 ReplaySSM，通过展开递推过程，将状态表示为一个检查点和一个存储近期 SSM 输入的缓冲区，从而将输出计算与检查点更新分离。通过周期性更新检查点，ReplaySSM 避免了大多数步骤中的状态写入，并将占主导地位的状态内存流量减少约一半。保留近期输入还使推测回滚简化为缓冲区指针操作，并支持仅验证输出，无需显式生成中间状态。同样，这种分离也允许在大多数步骤中使用低精度检查点进行输出计算，同时使用更高精度的检查点进行循环更新，因此低精度副本中的舍入误差永远不会进入后续更新。我们在 vLLM 中为 Mamba-2 和 Gated DeltaNet 实现了 ReplaySSM。在推理服务批大小为 256 时，相较于经过调优的 vLLM 基线，它在四个混合模型上将标准解码速度提升至 $1.20$--$1.47$ 倍，并在两个 120B 级别模型上实现了约为 vLLM 推测解码 $1.8$ 倍的吞吐量。与共享低精度检查点相比，为状态分别采用不同精度还能降低输出 KL 散度，并减少下游任务的准确率损失。

arXiv cs.LG81

DualQuant：面向 LLM 的双尺度免校准权重与激活量化

arXiv:2610.13767v1 公告类型：新稿 摘要：在原生支持 4 位运算的加速器上进行低比特推理，需要同时量化权重和激活，才能充分发挥这些格式带来的吞吐量优势。尽管训练后权重量化已得到广泛研究，激活量化仍然更具挑战性，且通常需要校准数据。在本研究中，我们提出 DUALQUANT，一种双尺度、免校准的训练后量化框架，可直接最小化权重重构均方误差（MSE），并由此直接得到用于激活量化前的逐通道激活重缩放因子。我们的关键观察是，通过最小化权重重构误差获得的缩放因子能够有效地对激活进行重缩放，从而无需校准数据即可实现具有竞争力的激活量化。我们将 DUALQUANT 表述为一个不可微的权重重构问题，并基于变量分裂和块坐标下降开发了一种高效求解器，其中每个子问题均具有闭式更新解。DUALQUANT 提供了一种统一算法，支持包括完整 MX 系列和 NVFP4 在内的多种格式。据我们所知，DUALQUANT 是首个正式支持 MX 和 NV 块格式的双重缩放方法。在 WikiText-2、C4 和零样本下游任务上对 Llama 和 Qwen 模型的评估表明，DUALQUANT 在大多数模型和格式上的精度达到或超过了基于校准的 SmoothQuant，并显著优于免校准基线方法，尤其在 MX 和 NV 块格式上取得了大幅提升。我们在 RTX 4090 上，使用针对 Llama-3.1-8B 的实际打包 W4A4 推理路径验证了这些收益。该实现的精度与伪量化非常接近，同时相较于 BF16，内存占用最多降至约 1/2.61，吞吐量最高提升至 2.46 倍。

arXiv cs.LG66

When Edge Independence Fails: Joint Graph Diffusion with Latent Sociability Priors

arXiv:2610.13765v1 Announce Type: new Abstract: Discrete graph diffusion models corrupt graph structure through independent edge updates with a Markovian noise model. Combined with the exchangeability requirement on the output distribution, the terminal prior is necessarily an Erd\H{o}s--R\'enyi (E-R) random graph. This has a fundamental consequence for unconditional graph generation without node features: in the large graph limit, any GNN denoiser applied to an E-R graph approximately outputs an E-R graph, making effective denoising difficult. In practice, generated graphs fail to recover meaningful node heterogeneity and higher-order structure. Previous models introduced specialized node features to compensate, framing this as an expressivity problem, while the role of the prior remained unexamined. We instead address the root cause by introducing the Latent Sociability Prior (LSP) and a join} diffusion process that co-evolves latent node sociability variables and graph structure, removing edge independence while preserving exchangeability. Unlike existing latent graph diffusion methods, the sociability mechanism requires no complex encoder: its sole role is to guide edge updates and preserve node heterogeneity throughout the trajectory. We validate our approach on multiple graph generation datasets and show that the proposed framework more accurately captures graph distributions than baseline methods without node features.

arXiv cs.LG69

Matched-Context Group Policy Optimization for LLM Agent Training

arXiv:2610.13755v1 Announce Type: new Abstract: Training long-horizon language model agents requires assigning credit to intermediate decisions from sparse terminal feedback. Within a finite group of sibling rollouts, useful action comparisons depend on their context, evidence for both alternatives, and how that evidence enters learning. We propose Matched-Context Group Policy Optimization (MCGPO), a procedure for constructing weighted action preferences from already collected trajectories. MCGPO defines comparison contexts with pre-action state summaries and observed rubric progress, annotated by task rules. Two-action support selects among ordered exact and coarse matching rules. Fixed folds separate candidate decisions from the terminal outcomes used to orient their preferences, and support-dependent evidence weights scale an auxiliary pairwise objective alongside shared GraphGPO task learning. MCGPO reuses rollout data and each response's original history, preserving critic-free training. Across six environment-backbone settings with three training seeds each, MCGPO improves mean success over matched GraphGPO: by 0.47-1.46 percentage points on ALFWorld and WebShop with Qwen2.5-1.5B and 7B, and by 3.96 and 5.16 points on WebShop with Qwen3-4B and 8B. The ALFWorld gains build on baseline success above 92%, where remaining headroom is limited. Integrated success improves in three of four Qwen2.5 settings. Three-seed factorial controls support opposite-fold evidence and evidence weighting; joint state-rubric matching achieves the highest mean in all four representation-control settings.

arXiv cs.LG61

Studying the Effects of Task Diversity Using Small Mathematical Worlds

arXiv:2610.13750v1 Announce Type: new Abstract: Basic mathematical tasks like modular addition have been a valuable tool for illuminating aspects of neural network behavior that are challenging to see in noisy, unverifiable, large-scale, real-world settings. These studies have typically focused on a single task. However, modern language models are exposed to a vast number of tasks despite being trained with a next-token prediction objective. In this work, we introduce two mathematical task ecosystems, IntWorld and PermWorld, built from distinct but related tasks on integers and permutations. Both are designed to simulate task diversity within a cohesive mathematical 'world' where tasks are distinct but related. Because the datasets are procedurally generated, we can precisely vary the number of tasks, allowing us to begin to disentangle the role of task variety from other factors in training. After describing the datasets, we provide some initial results obtained from these datasets which support the idea that increasing task diversity increases the richness of a model's internal representations.

arXiv cs.LG70

SafeDiSTL: Decomposing Joint Constraints for Safe Multi-Agent Reinforcement Learning

arXiv:2610.13737v1 Announce Type: new Abstract: Cooperative multi-agent tasks often impose constraints on a shared objective or collective safety. A team level violation cost function can measure this constraint outcome, but does not distinguish which agents' actions most strongly influence that violation. We present SafeDiSTL, a constrained multi-agent reinforcement learning framework that allows for credit assignment even with joint constraint measurements. We use Signal Temporal Logic (STL) robustness to measure episode-level violation through running-maximum cost increments. Smooth full-episode robustness and differentiable dynamics provide action sensitivities, which allocate each cost increment across agents at the same decision time. These sensitivities account for delayed effects when weighting agents' shares of each cost increment. We study the framework in cooperative payload transport, connected navigation, and networked traffic control tasks. SafeDiSTL achieves 1.6-5.1x the safe-success rate in transport, improves navigation safe-success by 11.1%, and delivers 50-68% more throughput in traffic when compared to constrained baselines. Results demonstrate a strong improvement of the learned policy in dealing with shared constraint while retaining the performance of unconstrained policies.

arXiv cs.LG76

Orthros：面向高效 Transformer 的阶段感知异构注意力

arXiv:2610.13735v1 公告类型：新稿 摘要：线性注意力和滑动窗口注意力等高效注意力机制旨在将 Transformer 的计算复杂度从二次降至线性。然而，这些机制的上下文内召回能力有所下降，往往需要交错引入全注意力层，导致长上下文处理中的二次复杂度瓶颈仍未解决。为此，我们提出 Orthros 注意力，将效率优化范式从结构分配转向阶段感知解耦。作为一个统一模块，它在序列处理的不同阶段动态切换计算模式，实现线性复杂度的预填充和全注意力解码。在预填充负载重、解码负载轻的常见长上下文场景中，这一设计使 Orthros 注意力的渐近复杂度低于标准 Transformer。除理论形式化描述和复杂度分析外，广泛的实证评估也表明 Orthros 在多个维度上具有优势。首先，我们验证了其基础语言建模能力，展示了其在不同模型规模下的可扩展性，以及在全面架构比较中的强大竞争力。其次，通过高数据效率的 Transformer 到 Orthros 转换，Orthros 展现出极佳的转换可行性，转换后的性能甚至超越原始 Transformer。最后，Orthros 相较于 Transformer 实现了 $21.9\times$ 的预填充加速，并带来了端到端推理性能提升，同时保留了长上下文能力：其在 128K 大海捞针测试中取得近乎完美的结果，在真实场景的上下文内召回任务中也达到了 Transformer 水平。总体而言，这些在高性能、灵活性和效率方面的优势，使 Orthros 有望在工业应用中接替 Transformer。

arXiv cs.LG78

SeoulMMOD：大规模多模态起讫点流量基准

arXiv:2610.13730v1 公告类型：新投稿 摘要：起讫点（OD）流量预测通过预测城市范围内人员流动的规模和模式，为城市出行决策提供支持。现有大多数公开 OD 基准仅提供全市出行的局部视图，涵盖单一交通服务或至多两种出行方式。在出行方式覆盖不完整的情况下，很难区分观测到的需求变化究竟源于交通方式之间的转移，还是出行总量的变化。我们提出 SeoulMMOD，这是一个面向首尔全市多模态出行的大规模公开基准。SeoulMMOD 提供了覆盖三年的逐小时 OD 流量数据，用于估计 25 个区和 426 个下辖行政区域内 6 种城市出行方式的总出行量，并配有出行时间、出行距离、日历、降雨量和兴趣点（POI）信号。利用 SeoulMMOD，我们在统一协议下评测了 14 个预测基线，并分析了空间尺度变化、联合训练与按出行方式分别训练，以及跨年泛化的影响。结果表明，若干在区级尺度上有效的时空模型无法扩展到下辖行政区域级别的 OD 预测，而简单的多出行方式联合训练也不能稳定地提高准确性。该数据集与基准共同构建了一个可复现的测试平台，用于城市尺度下多出行方式、跨多年的 OD 预测。

arXiv cs.LG72

基于图神经网络的摊销式反事实预测

arXiv:2610.13724v1 公告类型：新论文 摘要：反事实推断探究的是，在不同情境下，已观测到的结果可能会有何不同，例如，患者如果接受另一种治疗是否会康复。当决策依赖于未被观测到的备选方案的结果时，就会出现此类问题。回答反事实查询通常需要了解底层因果结构，而这在实践中往往无法获得。因此，一种常见方法是先根据观测数据估计因果图，再进行反事实推断。然而，估计因果图中的误差可能会传递至下游反事实预测。为应对这一挑战，我们提出 Counterfactual Graph Neural Process（CFGNP），这是一种端到端方法，无需预先估计因果图，即可直接根据观测数据回答反事实查询。CFGNP 利用图神经网络（GNN），将跨世界消息传递限制在相互对应的事实变量与反事实变量之间，从而利用双图结构。在合成数据和基于图像的基准测试中，CFGNP 的预测性能均优于现有方法，其中，在 German Loan 基准测试中，它将均方误差（MSE）从 $4.51 \times 10^{-2}$ 降至 $0.19 \times 10^{-2}$。此外，在合成因果图上，仅使用 10 节点图训练的模型，无需重新训练，即可在节点数多达 30 的图上保持出色的预测性能。

arXiv cs.LG75

面向联邦社会公平 k-Means 聚类的机器遗忘

arXiv:2610.13717v1 公告类型：新论文 摘要：联邦聚类可支持跨医院的患者分层、分布式金融数据集中的客户细分等应用。负责任的人工智能要求公平地表征各人口群体，并支持数据删除，这在一定程度上源于 GDPR 的删除权等权利要求。这些目标相互影响：删除记录可能改变群体之间的平衡，并改变剩余人群的聚类结果。我们研究联邦社会公平 $k$-means 的精确机器遗忘，其目标是最小化各群体到最近中心的平均平方距离中的最大值。我们的方法为每个群体分别保留本地摘要，因此，当全局公平性权重发生变化时，未受影响机构的摘要仍然有效。对于两个群体的情形，我们证明了在理想采样条件下，初始化具有独立于客户端群体比例的近似保证，并表明使用相同随机种子进行重放能够精确复现重新训练的结果。当整个客户端退出时，复用带来的收益最大：在三个数据集上，对于仅包含初始化的模型，重放的速度为重新训练的 $35.8\times$；加入一轮和两轮公平性细化后，速度仍分别为重新训练的 $2.4\times$ 和 $1.7\times$，而这些细化轮次能够降低表现最差群体的代价。对于单条记录的删除，收益较为有限（约为 $1.06\times$），且验证缓存分配结果的开销可能超过其节省的开销。这些结果明确了精确遗忘能够通过复用计算获得收益的适用范围。代码和可复现性材料见 https://github.com/arareddy/mufsc。

arXiv cs.LG71

Towards Understanding the Training of GFlowNets: A Theoretical Perspective

arXiv:2610.13711v1 Announce Type: new Abstract: Generative Flow Networks (GFlowNets) learn policies that sample discrete objects in proportion to a positive reward. They are trained by minimizing balance objectives, whose exact satisfaction guarantees correct sampling. However, a comprehensive theory linking nonzero balance loss and approximate stationarity to sampling accuracy, especially under evolving sampling distributions and backward policies, is still lacking. To bridge this gap, we develop a unified theory for Flow Matching, Detailed Balance, and Trajectory Balance in this regime. We first bound terminal symmetric KL by the square root of the expected balance loss. Then, we show that approximate stationarity controls this distribution error up to a tangent-space residual measuring parameterization incompleteness. Finally, we establish finite-time convergence under fixed or controlled sampling and fixed, balance-trained, or separately updated backward policies. Under standard smoothness assumptions, with fixed or controlled sampling and a fixed or balance-trained backward policy, the expected loss is $O(\bar\epsilon^2+T^{-1})$ and expected terminal symmetric KL is $O(\bar\epsilon+T^{-1/2})$ under uniform coverage. For a separately updated backward policy, we characterize the sufficient update decay to approach the residual neighborhood and preserve these rates. These results connect balance objectives to sampling accuracy and clarify how parameterization, coverage, and moving components govern GFlowNet training.

arXiv cs.LG75

一步完成校准：联合学习准确且校准良好的神经网络

arXiv:2610.13693v1 公告类型：新论文 摘要：神经网络（NNs）虽然具有高度的灵活性和通用性，但往往校准不佳。事后校准方法使用单独的校准数据集和额外的校准参数，调整已训练模型的输出。这一过程需要单独优化校准器并进行后续模型选择，从而增加了开销。本文表明，通过双层 Stackelberg 形式化建模，标准的（事后）校准器可以与骨干网络一起进行端到端训练。该建模方式利用交叉熵联合优化准确性与校准效果，无需单独的校准集或专门的校准损失，仅引入一个额外的超参数：校准器学习率。尽管我们的框架支持从温度缩放、矩阵缩放到神经网络层等多种校准器，但实验结果表明，直接将最终分类层用作校准器，即可在不增加任何参数的情况下，得到准确且校准良好的模型。我们的方法仅增加极少的复杂度，便能超越当前最先进的校准器，同时还能避免在多任务学习等场景中，独立校准组件带来的成倍开销。

arXiv cs.LG73

方差缩减策略梯度近乎极小极大最优

arXiv:2610.13692v1 公告类型：新论文 摘要：标准策略梯度（PG）算法依赖蒙特卡洛更新，因此存在估计方差较高的问题。尽管离策略方差缩减能够缓解这一问题，但目前要确立最优样本复杂度，仍需要重要性权重一致有界等限制性假设，或不切实际的二阶运算。在本研究中，我们提出 Loopless Variance-Reduced PG（LVR-PG），这是一个统一的理论框架，对一类将递归更新与防御性重要性采样相结合的一阶估计器进行了抽象，其中包括 PAGE-PG 和 STORM-PG 等已有估计器。我们证明，对于一般参数化策略，在弱梯度支配条件下，这类算法能够以 $\tilde{\mathcal{O}}(\epsilon^{-2})$ 的样本复杂度实现最后一次迭代的全局收敛。我们仅使用一阶更新，在极少的假设下且不使用梯度预条件器，便确立了这一保证。通过将当前最先进的下界扩展至连续参数化策略，我们确立了 $\Omega(\epsilon^{-2})$ 的样本复杂度下限，证实我们的框架在忽略对数因子和时间跨度项后具有极小极大最优性。

arXiv cs.LG73

基于 Transformer 的扩散模型上下文得分算子学习

arXiv:2610.13690v1 公告类型：新论文 摘要：扩散模型通常学习单个目标分布的生成动力学，或学习由训练期间固定的条件变量索引的一族分布的生成动力学。这限制了它们仅凭少量示例适应此前未见分布的能力。我们提出了一种面向扩散模型的上下文解算子学习框架，其中每个目标分布都被视为从元分布中抽取的一个任务，并在推理时由一个经验上下文集指定。我们的方法学习一个上下文得分算子，将上下文样本、含噪查询点和扩散时间映射为任务相关的得分，从而无需重新训练或更新参数，即可在未见任务上进行反向扩散采样。一个关键发现是，任务相关得分具有高斯核条件均值表示。其经验 Nadaraya--Watson 近似可以由我们构建的 Transformer 类中的模型精确表示，为所提出的查询相关注意力架构提供了理论依据。该算子通过跨任务分布的摊销去噪得分匹配进行训练。我们建立了泛化误差界，量化了上下文大小、训练任务数量、任务内样本数量、模型复杂度以及内在维度和环境维度的影响。在数值实验中，我们在一系列科学与视觉数据集上评估了该方法，包括 Navier-Stokes、ERA5 气候数据、QuickDraw 草图和 Food-101。结果表明，所提出的方法在相对简单的任务族上表现良好，但其性能会随着任务复杂度的增加而下降，从而揭示了扩散模型上下文学习的潜力与局限。

arXiv cs.LG72

VQIL：用于图上扰动响应排序的可见量子干涉学习

arXiv:2610.13689v1 公告类型：新提交 摘要：学习局部扰动如何影响网络，对于识别哪些组件驱动或吸收系统响应至关重要。我们将这一问题表述为以种子节点为条件的响应排序：给定一个种子节点上的扰动，按响应强度对所有其他节点进行排序。当模型仅获得相互作用拓扑，而支配扰动传播的机制必须从数据中学习时，这项任务具有挑战性。我们提出 VQIL（Visible Quantum Interference Learning），一种用于以种子节点为条件进行排序的可训练 Szegedy 量子行走。边权重、连接相位以及各节点的相位旋转通过反向传播进行端到端优化，同时行走算符在整个训练过程中始终保持酉性。节点得分由量子行走振幅经过几何折扣的相干叠加得到，使量子干涉能够在最终的范数平方读出之前发生。由于每个学习参数都关联到一条具名边或一个具名节点，训练后的模型能够就各个图组件在传播机制中的作用提供可直接检验的假设。在六个输电网络的交流潮流扰动任务中，即使未向模型提供电气参数，VQIL 仍将 Spearman 秩相关系数从未训练行走的 0.147 提升至 0.831，并优于采用相同方式训练的随机游走模型。在以 Perturb-seq 为监督的五个蛋白质相互作用网络的基因扰动任务中，VQIL 将秩相关系数从 0.171 提升至 0.620，超越了表现最强的已训练经典基线模型和神经网络基线模型。这些结果表明，可训练的量子行走干涉为学习图上以种子节点为条件的响应排序提供了一种可扩展且可解释的归纳偏置。

arXiv cs.LG75

论利用协变量的时间序列基础模型中日历特征的编码

arXiv:2610.13676v1 公告类型：新论文 摘要：时间序列基础模型（TSFMs）越来越多地支持协变量，其中日历信息是实践中最常见的类别协变量，因为每个时间点都有相应的日历信息。然而，这些特征的编码方式大多被视为实现细节，而非研究对象，尽管编码是预训练模型在推理时仍可调整的少数设计选择之一。本研究表明，信息传入模型的方式会显著影响 TSFMs 的预测准确性。因此，我们在来自多个领域的九个数据集上，对四种最先进的 TSFMs 进行零样本滚动评估，系统比较了七种日历编码，包括三种常规的、与目标无关的表示方式（序数编码、独热编码和周期编码）以及四种目标编码。这九个数据集受到周末和节假日等相关社会活动的影响，因此日历信息对预测尤为重要。我们发现，编码选择会影响所有模型的预测准确性；在同一模型内，不同编码在非节假日的 MASE 差异最高可达 9.9%，在节假日则可达 83%。编码的有效性及其被利用的程度在不同模型间存在显著差异。性能提升主要集中在节假日前后，此时日历信息提供了目标变量历史数据中无法获取的信号；同时，某些模型与编码的组合也能改善非节假日的预测。基于 SHAP 的归因分析确定了协变量相对于自回归部分的贡献，并解释了某些模型与编码的组合为何会导致性能下降。我们的研究结果表明，当前针对利用协变量的 TSFMs 的基准测试，可能会因采用固定编码策略而扭曲模型比较结果。

arXiv cs.LG68

Pretraining and Test-Time Training Budgets Matter for ARC: A Controlled Study of VARC and TRM

arXiv:2610.13660v1 Announce Type: new Abstract: The Abstraction and Reasoning Corpus (ARC) is increasingly used to evaluate whether neural networks can infer task-specific transformations from few demonstrations. Recent ARC models differ in architecture, but also in broader training and evaluation protocols, including pretraining data, data augmentation, test-time setup, and optimization budgets, making conclusions about architecture benefits debatable. In this work, we study how the performance of the vision-inspired VARC and recursion-based TRM models is affected by pretraining and test-time training (TTT) budgets under a common inductive protocol on ARC-AGI-1. We cross five pretraining budgets with four TTT budgets for each test task. Each budget specifies a task-variant count and total sample presentations, defining the controlled training budgets. We observe that increasing TTT sample presentations improves the standard ARC pass@2 metric in all settings for both models. Increasing the TTT task-variant count consistently helps only TRM, whereas VARC often performs better with fewer variants. VARC also remains competitive without pretraining, while TRM benefits much more from it. These results show that pretraining and TTT budgets affect ARC evaluation and motivate reporting at comparable budgets.

arXiv cs.LG77

TabReliBench：超越预测准确性的表格基础模型评估

arXiv:2610.13654v1 公告类型：新论文 摘要：表格基础模型（TFMs）因其强大的预测性能而日益受到欢迎。许多此类模型还提供后验预测分布（PPDs），但这些分布的可靠性仍缺乏充分研究。我们从点预测、分布质量、边际与子群校准以及区间效率等方面评估可靠性。随后，我们推出 TabReliBench，这是一个用于上述比较和数据集层面分析的大规模分类与回归基准。我们发现，TFMs 在 TabArena 和本基准上的排名有所不同，在不同可靠性指标上的排名也会发生变化。回归 TFMs 通常能够达到接近名义水平的边际覆盖率，同时保持较窄的区间，但这种良好的边际表现并不意味着条件覆盖率也能得到保证。数据集层面的分析表明，分类难度较高往往伴随着较差的校准，而回归预测性能并不是衡量校准情况的可靠指标。集成方法能够提高预测性能，但并不总能改善 PPDs 的校准；TFMs 从保形后处理中获得的收益也很有限，这种处理无法可靠地消除子群间的差距。因此，要可靠地选择 TFM，就需要超越综合排名，开展多维度评估。代码可在 https://github.com/Tianqi-Zhao/TabReliBench 获取。

arXiv cs.LG78

SelectArena：面向真实世界表格数据的现代特征选择基准

arXiv:2610.13644v1 公告类型：新论文 摘要：从表格数据集中选择特征（列）的子集，是现实应用中普遍存在的挑战。行业专家希望最大限度地降低模型训练成本，气候科学家需要一组可解释的因素子集来理解极端天气事件，临床医生则因资金和时间限制而需要在诊断检查项目之间作出选择。然而，当不同领域的实践者查阅科学文献以确定采用哪种特征选择方法时，所面对的基准评测体系缺乏代表性。尽管相关研究十分广泛，现有特征选择基准所能提供的指导仍然有限，因为它们依赖过时的模型、临时制定的评估流程，或无法反映现实问题的人工生成数据。为弥合科学基准评测与现实应用之间的差距，我们推出了现代特征选择基准 SelectArena。我们采用标准化评估流程和最先进的表格（基础）模型，在 71 个经人工精心整理、涵盖分类与回归问题的真实世界表格数据集上，对 15 种广泛使用的特征选择方法进行了比较。结果揭示了预测性能、运行时间、稳定性和有效性之间的明确权衡。在同时考虑预测性能和运行时间时，基于树的方法 CART 和 RFImportance 以及 F-Test 均达到帕累托最优。F-Test、基于信息论的方法和基于距离的方法在稳定性与有效性方面表现领先，其中 F-Test 在我们采用的各项指标上展现出均衡表现。基于搜索的方法速度慢、稳定性差，且性能较低。SelectArena 将这些发现转化为实践指导，并作为一个可扩展的平台，为未来的特征选择研究提供支持。

arXiv cs.LG68

PANDA: Prior-guided Attentional Dual-path Architecture

arXiv:2610.13635v1 Announce Type: new Abstract: Hypergraph Neural Networks (HNNs), designed to model higher-order relations beyond pairwise interactions, typically rely either on fixed structural weights or on normalised attention. Fixed weights can propagate misleading information in heterophilic settings, while normalisation makes attention coefficients competitive, in the sense that, in a message-passing framework, increasing the weight of one sender must reduce the weight of others in the same aggregation set. Our main contribution is the introduction of PANDA (Prior-guided AttentioNal Dual-path Architecture), a plug-in for two-stage message passing HNNs applicable to both undirected and directed settings. The main path interpolates between learned receiver-normalised attention and structural incidence priors, while a non-competitive (not normalised) auxiliary path lets each sender contribute its own transformed information. We also show that, with fixed structural coefficients, PANDA recovers a certain complex-valued convolution operator adopted in previous works. Across four HNNs spanning spatial and spectral methods in both directed and undirected settings, plugging PANDA in yields an average improvement of 5.18 percentage points across eight real-world benchmark datasets. The code is available at https://github.com/jainris/panda.

arXiv cs.LG68

Private and Distributionally Robust Federated Multiclass Classification

arXiv:2610.13631v1 Announce Type: new Abstract: Fault diagnosis in asset-intensive industries requires multiclass models trained on noisy, sensitive data collected across geographically distributed client sites. Federated learning enables collaborative training without centralizing data, but most approaches do not provide formal privacy guarantees for individual records stored at client sites and can degrade under label noise. To address these limitations, we develop a differentially private, distributionally robust framework for federated multiclass classification that provides sample-level privacy for records stored at each client. We build on a federated Wasserstein distributionally robust multiclass objective with the Crammer-Singer loss, solved using an Alternating Direction Method of Multipliers (ADMM)-based algorithm. To protect the released client updates, our method injects calibrated Gaussian noise into the proximal linearized client updates of both the local model and the local distributionally robust variable before aggregation at the central server, and we quantify the cumulative privacy loss using the moments accountant. We provide per-round sample-level $(\epsilon,\delta)$-differential privacy guarantees and an overall composed privacy guarantee across all training rounds, together with convergence results and an explicit privacy-utility tradeoff. Experiments on simulated industrial fault data and UCI benchmark datasets show that our method outperforms differentially private federated empirical risk minimization and stochastic gradient descent baselines in most tested settings, with the largest gains under strong privacy-induced noise.

arXiv cs.LG

A small cognitive core explains human behavior at scale

arXiv:2610.13603v1 Announce Type: new Abstract: Psychology has proposed many principles of cognition. Each, however, was developed in isolation, leaving their combined explanatory power unknown. We assembled a small set of established principles into one model, which we call COntextual Retrieval and Encoding (CORE). CORE consists of a hierarchy of episodic memories into which prediction errors are written and whose encoding, retrieval, and forgetting depend on context. Fitted jointly to 7,036,442 choices from 137 experiments, the model predicts human choices better than black-box models and reproduces 75% of reported behavioral effects when simulated. Moreover, its latent variables show classic signatures of Bayesian learning and event segmentation. Taken together, these findings suggest that a small cognitive core accounts for much of human behavior. They also open a path toward making psychology more than the sum of its parts, by building computational models on the entire experimental record.

arXiv cs.LG76

EchoTab：通过表格重建学习以测量为基础的表征，用于超声心动图诊断

arXiv:2610.13599v1 公告类型：新论文 摘要：在超声心动图等多视图医学影像检查中，诊断信息分布于不同视图中，每个视图捕捉心脏结构和功能的不同方面。在临床实践中，医生首先通过定量测量解读这些视图，再依据测量结果作出具体诊断。然而，现有的超声心动图表征学习方法并未将测量显式建模为视图与诊断之间的中间表征。因此，各项测量未能以可供下游诊断任务选择性使用的形式得到显式表示，从而妨碍了准确诊断。受这种分层临床工作流程的启发，我们提出了 EchoTab，这是一种以测量为基础的表征学习框架，使用测量特定的词元来表示多视图超声心动图检查，并将其作为超声心动图视图与诊断之间的中间表征。每个测量词元选择性地关注与其估计相关的视图，并通过表格重建进行训练，以预测对应的测量值。由此得到的词元构成一种因子化的超声心动图表征，编码了具有临床意义的结构和功能特征，可供下游诊断选择性使用。与常用的对比学习方法不同，EchoTab 不需要负样本或大批量训练。在两个数据集上的实验表明，EchoTab 能够学习有效的、以测量为基础的表征，从而提升多视图超声心动图的诊断分类性能；其学习到的注意力也与心脏科医生所采用的既定测量与视图对应关系一致。我们的代码可在 https://github.com/michelleespranita/EchoTab 获取。

arXiv cs.LG64

$L_0$ Feature Selection via Monte Carlo Tree Search in High-Dimensional Financial Regression

arXiv:2610.13574v1 Announce Type: new Abstract: Feature selection is a central problem in high-dimensional financial regression, where large numbers of engineered variables and strong correlations among predictors reduce model stability and interpretability. $L_1$ regularization (Lasso) is widely used for sparse regression, but its shrinkage can be restrictive when informative signals are spread across correlated variables. Exact sparse regression avoids this shrinkage but requires solving a combinatorial optimization problem that is computationally difficult and is typically fit to minimize in-sample rather than out-of-sample error. This paper presents a feature selection framework based on Monte Carlo Tree Search with Generalized Nested Rollout Policy Adaptation (GNRPA). Feature selection is formulated as a sequential decision process, and candidate subsets are scored by their out-of-sample validation performance rather than training error. The method also supports a lexicographic objective accounting for subset size and multicollinearity. We study this approach on a contemporaneous explanatory regression task: explaining the weekly return of the NASDAQ-100 index from 1,072 engineered features spanning market, macroeconomic, and factor data. This isolates the value of subset selection under multicollinearity from the separate problem of predictive forecasting. On this task, the proposed method identifies sparse feature subsets with higher out-of-sample explanatory power than an optimized Lasso baseline, reaching an out-of-sample $R^2$ of 0.6961 versus 0.6313 for Lasso. These results suggest that Monte Carlo Tree Search is a practical alternative to convex regularization for sparse explanatory feature selection in financial regression, with potential value for factor attribution and risk decomposition where transparency matters as much as out-of-sample fit.

arXiv cs.LG

Simultaneous Competitive and Regret Guarantees for TCP Acknowledgment

arXiv:2610.13571v1 Announce Type: new Abstract: We study simultaneous competitive and regret guarantees for TCP acknowledgment. Each acknowledgment serves all pending packets, and the objective is to minimize the total acknowledgment and waiting costs. It is well known that no deterministic online algorithm achieves a competitive ratio better than $2$. Given $k$ randomized online experts and $0<\varepsilon<1$, we construct a randomized online algorithm whose realized total cost is always at most $(2+\varepsilon)$ times the offline optimal cost $\mathrm{OPT}$ and whose expected regret against the expert with the smallest realized cost is $O(\sqrt{\mathrm{OPT}\log(k+1)}+\log(1/\varepsilon))$. The algorithm requires no advance knowledge of the number of packets or $\mathrm{OPT}$. Our algorithm controls switching costs between experts and uses the gap between the permitted competitive bound and its current cost to determine when to learn. We also prove lower bounds showing that, for $k\ge 2$, this regret bound is asymptotically optimal in $\mathrm{OPT}$, $k$, and $\varepsilon$, and that $\varepsilon>0$ is necessary for sublinear regret.

arXiv cs.LG71

Rationally Linearized MLPs

arXiv:2610.13568v1 Announce Type: new Abstract: We present Rationally Linearized Multilayer Perceptrons(RaLiMPs), a new approach to efficiently linearizing MLPs using low-degree rational approximations of standard activation functions. Inspired by polynomial-based MLP linearization, we show that rational approximations can provide accurate representations at substantially lower degree. RaLiMPs exploit these approximations to construct kernel representations with narrow bottlenecks, enabling efficient composition across MLP layers. We provide theoretical guarantees for the resulting approximation and evaluate RaLiMPs across applications ranging from implicit 3D models to large language models. Our experiments demonstrate favorable accuracy--efficiency tradeoffs, with RaLiMPs achieving competitive or improved accuracy while substantially reducing inference cost.

arXiv cs.LG68

Probabilistic Ship Trajectory Forecasting using Transformer Models on AIS Data

arXiv:2610.13557v1 Announce Type: new Abstract: Anticipation of vessel trajectories is an essential task when navigating at sea. This applies to both human navigators and autonomous surface vessels. Trajectory forecasting enables sub-routines such as collision avoidance to assess the situation and react accordingly in time. This paper investigates Transformer and Performer models for trajectory prediction of surface vessels in confined waters. Using widely available position data from the Automatic Identification System (AIS), candidate models are trained and evaluated. Historical information on how other vessels have moved in the local area surrounding each agent is conveyed using positional heat maps. The heat maps are generated using Kernel Density Estimation (KDE) and encoded with a Convolutional Neural Network (CNN) before entering the Transformer models. The models are trained with quantile loss to efficiently obtain a model-aware prediction interval, as this is crucial for timely collision avoidance. The proposed model architectures are shown to greatly outperform previous work while the inclusion of local heat maps enables the prediction of vessel movement that is not otherwise indicated from the previous positions.

arXiv cs.LG74

通过逐层随机特征模型模拟深度神经网络训练中的尖峰

arXiv:2610.13524v1 公告类型：新论文 摘要：大语言模型（LLMs）预训练过程中损失间歇性出现的“尖峰”，在理论和实证层面仍是一种尚未解明的现象。本文表明，输入服从重尾幂律分布的简单二次模型可以复现这些尖峰。我们提出了一套理论框架和仿真算法，利用此类二次模型对深度神经网络（DNNs）的逐层训练动态进行建模。在一项针对 Adam 超参数配置的案例研究中，该配置将 $\beta_2$ 设为接近 $1$ 的值，已知会在 Transformer 语言模型中引发尖峰；我们的仿真成功地从给定检查点出发，预测出了预示尖峰行为的隆起。此外，我们根据观测指标推导出一种合适的 $\beta_2$ 调度方案，并通过实证验证了其性能改进效果。

arXiv cs.LG62

FedPCD: Priority-Constrained Federated Aggregation

arXiv:2610.13508v1 Announce Type: new Abstract: Federated learning (FL) seeks to optimize a shared global model from decentralized client data. While effective for average performance, this process can produce unfair performance across clients, particularly under statistically heterogeneous data distributions. Existing fairness-aware and multi-objective FL methods address this through client re-weighting, gradient correction, or common-descent formulations, but often change the global optimization direction or treat client objectives symmetrically. We propose FedPCD, a priority-constrained federated optimization framework that separates the global optimization objective from client protection. FedPCD preserves the conventional federated aggregate as the primary objective, while allowing only the minimum correction necessary to meet client performance constraints. Across heterogeneous FL benchmarks, FedPCD consistently outperforms state-of-the-art baselines, achieving the highest global accuracy across all configurations and leading worst-10% client accuracy in nearly all. FedPCD further preserves competitive trade-offs between global accuracy and client performance variability. Beyond this trade-off, FedPCD demonstrates faster empirical convergence, reaching matching or superior performance in substantially fewer communication rounds.

arXiv cs.LG75

超越时间邻近性：用于强化学习信用分配的 Explanatory World Models

arXiv:2610.13504v1 公告类型：新论文 摘要：生成式推荐系统越来越多地使用强化学习（RL）来优化可能在决策后数百乃至数千次交互后才出现的结果。标准 RL 根据时间邻近性而非明确的相关性传播奖励，削弱了分配给时间久远但重要的动作的信号，成为样本效率低下的一个来源。我们提出 Explanatory World Models（EWM），这是一种归因框架，利用解释识别可能促成已实现结果的早期事件。我们使用前沿语言模型实现 EWM，并通过一种守恒的层次化算子将其解释用作学习信号；该算子递归地重新分配奖励，同时保持未折扣的回合回报不变。为了探究什么样的解释才有用，我们将受 Halpern 的实际因果关系、Lombrozo 的解释简洁性以及 Deutsch 的难以变动的解释启发的三种互补变体转化为可操作的方法。我们在两个受控的长时程 RL 基准、一个公开的序列推荐基准以及一个工业规模的生成式推荐系统上评估 EWM。在 Crafter 和 WordCraft 上，EWM 显著优于 PPO 和 LaRe，且在奖励延迟最严重时取得最大提升。在 KuaiRand 上，EWM 在离线评估中优于 SFT 和 LaRe，在在线评估中优于 A2C。最后，在对工业级生成式推荐系统的离线评估中，EWM 显著提高了召回率，并提升了行为代理指标。这些结果表明，解释能够为从延迟结果中学习提供结构化且可复用的信用归因。

arXiv cs.LG80

PPG-Bench：面向 PPG 基础模型的统一基准

arXiv:2610.13435v1 公告类型：新投稿 摘要：可穿戴设备和临床设备正以日益庞大的规模记录光电容积描记（PPG）信号，越来越多的 PPG 基础模型（FM）也相继被开发出来，用于计算可供各种下游任务复用的表征。然而，这些模型难以进行比较：不同评估采用的数据集、任务定义、预处理方法、下游预测头、超参数调优方法和基线各不相同。我们提出 PPG-Bench，这是一个统一的开源基准，在九个开放访问数据集和 26 项任务上评估五个公开的 PPG 基础模型，任务涵盖心血管状态估计、活动、睡眠、筛查和身份识别。我们在适应能力逐步增强的四种下游设置下评估模型（线性探测、随机森林、冻结模型的 MLP 探测和端到端微调），并采用针对各模型的超参数优化及受试者互不重叠的交叉验证。我们还设置了三类对照：随机/常数基线、确定性的脉率变异性（PRV）特征集，以及通用时间序列基础模型（MOMENT）。在全部 68 种任务与设置组合中，35.3% 的组合未显示基础模型相较于随机/常数基线具有实质性优势，而在 83.8% 的组合中，没有任何 PPG 专用基础模型能同时优于 PRV 和 MOMENT。我们还发现，增加下游预测头的容量通常可以提升性能，但如果模型的表现处于随机猜测或常数预测水平，则可能导致性能显著下降。因此，目前的 PPG 基础模型尚不能宣称能够提供具有泛化性和可迁移性的 PPG 信号表征，该领域未来的评估应纳入与简单、非学习型对照方法的比较检验。该基准的代码及评估更多模型的说明见：https://github.com/mannas-ai/ppg-bench。

arXiv cs.LG67

In-Context Learning for System Identification: Probabilistic Formulation, Long-Sequence Handling, and Benchmark Validation

arXiv:2610.13411v1 Announce Type: new Abstract: Traditional system identification typically involves two steps: initially, training data is processed to estimate a model of the system of interest; then, the model is used to predict outputs for new input queries. In contrast, recent in-context learning approaches learn a meta model that maps directly from training data and query inputs to outputs in a zero-shot fashion, bypassing explicit model estimation. This approach leverages a meta model trained offline, on large sets of synthetically generated data, to emulate the behavior of a system identification algorithm. In this work, we advance this meta-learning framework for system identification in three directions. First, we develop a unified probabilistic meta-identification architecture that combines a Transformer-based encoder-decoder with tailored mechanisms for handling long training sequences, enabling calibrated predictions over long horizons. Second, we show how a pre-trained meta model can act as an uncertainty-aware synthetic data generator for classical system identification, providing task-specific data that regularize parametric models when real data are scarce. Third, we systematically evaluate the framework on standard nonlinear system identification benchmarks and on a challenging out-of-distribution scenario, demonstrating competitive accuracy with state-of-the-art methods and strong generalization, including cases where input-only adaptation is sufficient to handle significant distribution shifts.

arXiv cs.LG71

Accelerating Linear Bandits with Persistent Candidate Sets

arXiv:2610.13407v1 Announce Type: new Abstract: Exhaustive optimistic scoring can make action selection costly in linear bandits with a large fixed set of $K$ actions, requiring $\mathcal{O}(K)$ score evaluations per round. We introduce Persistent Candidate LinUCB (\textsc{PC-LinUCB}), which maintains $k$ active actions and samples $q$ challengers uniformly from the remaining $K-k$ actions at each round. By scoring only these $k+q$ candidates, \textsc{PC-LinUCB} reduces the number of score evaluations per round to $\mathcal{O}(k+q)$. We establish problem-independent and problem-dependent high-probability regret bounds consisting of a standard linear-bandit learning term and an additive discovery cost of order $\mathcal{O}\left((K-k)\log(K-k)/q\right)$ at fixed confidence. Notably, this additional cost does not grow with the horizon. For fixed gaps and sufficiently long horizons, we establish an $\Omega\left((K-k)/q\right)$ regret lower bound under uniform challenger sampling, showing that partial candidate access itself imposes an unavoidable regret cost. Experiments show that \textsc{PC-LinUCB} substantially accelerates LinUCB while maintaining comparable regret, and that the persistent candidate mechanism can extend beyond LinUCB.