全部动态

本页 40 条
10月12日 周一
arXiv cs.LG81

DualQuant：面向 LLM 的双尺度免校准权重与激活量化

arXiv:2610.13767v1 公告类型：新稿 摘要：在原生支持 4 位运算的加速器上进行低比特推理，需要同时量化权重和激活，才能充分发挥这些格式带来的吞吐量优势。尽管训练后权重量化已得到广泛研究，激活量化仍然更具挑战性，且通常需要校准数据。在本研究中，我们提出 DUALQUANT，一种双尺度、免校准的训练后量化框架，可直接最小化权重重构均方误差（MSE），并由此直接得到用于激活量化前的逐通道激活重缩放因子。我们的关键观察是，通过最小化权重重构误差获得的缩放因子能够有效地对激活进行重缩放，从而无需校准数据即可实现具有竞争力的激活量化。我们将 DUALQUANT 表述为一个不可微的权重重构问题，并基于变量分裂和块坐标下降开发了一种高效求解器，其中每个子问题均具有闭式更新解。DUALQUANT 提供了一种统一算法，支持包括完整 MX 系列和 NVFP4 在内的多种格式。据我们所知，DUALQUANT 是首个正式支持 MX 和 NV 块格式的双重缩放方法。在 WikiText-2、C4 和零样本下游任务上对 Llama 和 Qwen 模型的评估表明，DUALQUANT 在大多数模型和格式上的精度达到或超过了基于校准的 SmoothQuant，并显著优于免校准基线方法，尤其在 MX 和 NV 块格式上取得了大幅提升。我们在 RTX 4090 上，使用针对 Llama-3.1-8B 的实际打包 W4A4 推理路径验证了这些收益。该实现的精度与伪量化非常接近，同时相较于 BF16，内存占用最多降至约 1/2.61，吞吐量最高提升至 2.46 倍。

arXiv cs.LG66

When Edge Independence Fails: Joint Graph Diffusion with Latent Sociability Priors

arXiv:2610.13765v1 Announce Type: new Abstract: Discrete graph diffusion models corrupt graph structure through independent edge updates with a Markovian noise model. Combined with the exchangeability requirement on the output distribution, the terminal prior is necessarily an Erd\H{o}s--R\'enyi (E-R) random graph. This has a fundamental consequence for unconditional graph generation without node features: in the large graph limit, any GNN denoiser applied to an E-R graph approximately outputs an E-R graph, making effective denoising difficult. In practice, generated graphs fail to recover meaningful node heterogeneity and higher-order structure. Previous models introduced specialized node features to compensate, framing this as an expressivity problem, while the role of the prior remained unexamined. We instead address the root cause by introducing the Latent Sociability Prior (LSP) and a join} diffusion process that co-evolves latent node sociability variables and graph structure, removing edge independence while preserving exchangeability. Unlike existing latent graph diffusion methods, the sociability mechanism requires no complex encoder: its sole role is to guide edge updates and preserve node heterogeneity throughout the trajectory. We validate our approach on multiple graph generation datasets and show that the proposed framework more accurately captures graph distributions than baseline methods without node features.

arXiv cs.LG69

Matched-Context Group Policy Optimization for LLM Agent Training

arXiv:2610.13755v1 Announce Type: new Abstract: Training long-horizon language model agents requires assigning credit to intermediate decisions from sparse terminal feedback. Within a finite group of sibling rollouts, useful action comparisons depend on their context, evidence for both alternatives, and how that evidence enters learning. We propose Matched-Context Group Policy Optimization (MCGPO), a procedure for constructing weighted action preferences from already collected trajectories. MCGPO defines comparison contexts with pre-action state summaries and observed rubric progress, annotated by task rules. Two-action support selects among ordered exact and coarse matching rules. Fixed folds separate candidate decisions from the terminal outcomes used to orient their preferences, and support-dependent evidence weights scale an auxiliary pairwise objective alongside shared GraphGPO task learning. MCGPO reuses rollout data and each response's original history, preserving critic-free training. Across six environment-backbone settings with three training seeds each, MCGPO improves mean success over matched GraphGPO: by 0.47-1.46 percentage points on ALFWorld and WebShop with Qwen2.5-1.5B and 7B, and by 3.96 and 5.16 points on WebShop with Qwen3-4B and 8B. The ALFWorld gains build on baseline success above 92%, where remaining headroom is limited. Integrated success improves in three of four Qwen2.5 settings. Three-seed factorial controls support opposite-fold evidence and evidence weighting; joint state-rubric matching achieves the highest mean in all four representation-control settings.

arXiv cs.LG61

Studying the Effects of Task Diversity Using Small Mathematical Worlds

arXiv:2610.13750v1 Announce Type: new Abstract: Basic mathematical tasks like modular addition have been a valuable tool for illuminating aspects of neural network behavior that are challenging to see in noisy, unverifiable, large-scale, real-world settings. These studies have typically focused on a single task. However, modern language models are exposed to a vast number of tasks despite being trained with a next-token prediction objective. In this work, we introduce two mathematical task ecosystems, IntWorld and PermWorld, built from distinct but related tasks on integers and permutations. Both are designed to simulate task diversity within a cohesive mathematical 'world' where tasks are distinct but related. Because the datasets are procedurally generated, we can precisely vary the number of tasks, allowing us to begin to disentangle the role of task variety from other factors in training. After describing the datasets, we provide some initial results obtained from these datasets which support the idea that increasing task diversity increases the richness of a model's internal representations.

arXiv cs.LG70

SafeDiSTL: Decomposing Joint Constraints for Safe Multi-Agent Reinforcement Learning

arXiv:2610.13737v1 Announce Type: new Abstract: Cooperative multi-agent tasks often impose constraints on a shared objective or collective safety. A team level violation cost function can measure this constraint outcome, but does not distinguish which agents' actions most strongly influence that violation. We present SafeDiSTL, a constrained multi-agent reinforcement learning framework that allows for credit assignment even with joint constraint measurements. We use Signal Temporal Logic (STL) robustness to measure episode-level violation through running-maximum cost increments. Smooth full-episode robustness and differentiable dynamics provide action sensitivities, which allocate each cost increment across agents at the same decision time. These sensitivities account for delayed effects when weighting agents' shares of each cost increment. We study the framework in cooperative payload transport, connected navigation, and networked traffic control tasks. SafeDiSTL achieves 1.6-5.1x the safe-success rate in transport, improves navigation safe-success by 11.1%, and delivers 50-68% more throughput in traffic when compared to constrained baselines. Results demonstrate a strong improvement of the learned policy in dealing with shared constraint while retaining the performance of unconstrained policies.

arXiv cs.LG76

Orthros：面向高效 Transformer 的阶段感知异构注意力

arXiv:2610.13735v1 公告类型：新稿 摘要：线性注意力和滑动窗口注意力等高效注意力机制旨在将 Transformer 的计算复杂度从二次降至线性。然而，这些机制的上下文内召回能力有所下降，往往需要交错引入全注意力层，导致长上下文处理中的二次复杂度瓶颈仍未解决。为此，我们提出 Orthros 注意力，将效率优化范式从结构分配转向阶段感知解耦。作为一个统一模块，它在序列处理的不同阶段动态切换计算模式，实现线性复杂度的预填充和全注意力解码。在预填充负载重、解码负载轻的常见长上下文场景中，这一设计使 Orthros 注意力的渐近复杂度低于标准 Transformer。除理论形式化描述和复杂度分析外，广泛的实证评估也表明 Orthros 在多个维度上具有优势。首先，我们验证了其基础语言建模能力，展示了其在不同模型规模下的可扩展性，以及在全面架构比较中的强大竞争力。其次，通过高数据效率的 Transformer 到 Orthros 转换，Orthros 展现出极佳的转换可行性，转换后的性能甚至超越原始 Transformer。最后，Orthros 相较于 Transformer 实现了 $21.9\times$ 的预填充加速，并带来了端到端推理性能提升，同时保留了长上下文能力：其在 128K 大海捞针测试中取得近乎完美的结果，在真实场景的上下文内召回任务中也达到了 Transformer 水平。总体而言，这些在高性能、灵活性和效率方面的优势，使 Orthros 有望在工业应用中接替 Transformer。

arXiv cs.LG78

SeoulMMOD：大规模多模态起讫点流量基准

arXiv:2610.13730v1 公告类型：新投稿 摘要：起讫点（OD）流量预测通过预测城市范围内人员流动的规模和模式，为城市出行决策提供支持。现有大多数公开 OD 基准仅提供全市出行的局部视图，涵盖单一交通服务或至多两种出行方式。在出行方式覆盖不完整的情况下，很难区分观测到的需求变化究竟源于交通方式之间的转移，还是出行总量的变化。我们提出 SeoulMMOD，这是一个面向首尔全市多模态出行的大规模公开基准。SeoulMMOD 提供了覆盖三年的逐小时 OD 流量数据，用于估计 25 个区和 426 个下辖行政区域内 6 种城市出行方式的总出行量，并配有出行时间、出行距离、日历、降雨量和兴趣点（POI）信号。利用 SeoulMMOD，我们在统一协议下评测了 14 个预测基线，并分析了空间尺度变化、联合训练与按出行方式分别训练，以及跨年泛化的影响。结果表明，若干在区级尺度上有效的时空模型无法扩展到下辖行政区域级别的 OD 预测，而简单的多出行方式联合训练也不能稳定地提高准确性。该数据集与基准共同构建了一个可复现的测试平台，用于城市尺度下多出行方式、跨多年的 OD 预测。

arXiv cs.LG72

基于图神经网络的摊销式反事实预测

arXiv:2610.13724v1 公告类型：新论文 摘要：反事实推断探究的是，在不同情境下，已观测到的结果可能会有何不同，例如，患者如果接受另一种治疗是否会康复。当决策依赖于未被观测到的备选方案的结果时，就会出现此类问题。回答反事实查询通常需要了解底层因果结构，而这在实践中往往无法获得。因此，一种常见方法是先根据观测数据估计因果图，再进行反事实推断。然而，估计因果图中的误差可能会传递至下游反事实预测。为应对这一挑战，我们提出 Counterfactual Graph Neural Process（CFGNP），这是一种端到端方法，无需预先估计因果图，即可直接根据观测数据回答反事实查询。CFGNP 利用图神经网络（GNN），将跨世界消息传递限制在相互对应的事实变量与反事实变量之间，从而利用双图结构。在合成数据和基于图像的基准测试中，CFGNP 的预测性能均优于现有方法，其中，在 German Loan 基准测试中，它将均方误差（MSE）从 $4.51 \times 10^{-2}$ 降至 $0.19 \times 10^{-2}$。此外，在合成因果图上，仅使用 10 节点图训练的模型，无需重新训练，即可在节点数多达 30 的图上保持出色的预测性能。

arXiv cs.LG75

面向联邦社会公平 k-Means 聚类的机器遗忘

arXiv:2610.13717v1 公告类型：新论文 摘要：联邦聚类可支持跨医院的患者分层、分布式金融数据集中的客户细分等应用。负责任的人工智能要求公平地表征各人口群体，并支持数据删除，这在一定程度上源于 GDPR 的删除权等权利要求。这些目标相互影响：删除记录可能改变群体之间的平衡，并改变剩余人群的聚类结果。我们研究联邦社会公平 $k$-means 的精确机器遗忘，其目标是最小化各群体到最近中心的平均平方距离中的最大值。我们的方法为每个群体分别保留本地摘要，因此，当全局公平性权重发生变化时，未受影响机构的摘要仍然有效。对于两个群体的情形，我们证明了在理想采样条件下，初始化具有独立于客户端群体比例的近似保证，并表明使用相同随机种子进行重放能够精确复现重新训练的结果。当整个客户端退出时，复用带来的收益最大：在三个数据集上，对于仅包含初始化的模型，重放的速度为重新训练的 $35.8\times$；加入一轮和两轮公平性细化后，速度仍分别为重新训练的 $2.4\times$ 和 $1.7\times$，而这些细化轮次能够降低表现最差群体的代价。对于单条记录的删除，收益较为有限（约为 $1.06\times$），且验证缓存分配结果的开销可能超过其节省的开销。这些结果明确了精确遗忘能够通过复用计算获得收益的适用范围。代码和可复现性材料见 https://github.com/arareddy/mufsc。

arXiv cs.LG71

Towards Understanding the Training of GFlowNets: A Theoretical Perspective

arXiv:2610.13711v1 Announce Type: new Abstract: Generative Flow Networks (GFlowNets) learn policies that sample discrete objects in proportion to a positive reward. They are trained by minimizing balance objectives, whose exact satisfaction guarantees correct sampling. However, a comprehensive theory linking nonzero balance loss and approximate stationarity to sampling accuracy, especially under evolving sampling distributions and backward policies, is still lacking. To bridge this gap, we develop a unified theory for Flow Matching, Detailed Balance, and Trajectory Balance in this regime. We first bound terminal symmetric KL by the square root of the expected balance loss. Then, we show that approximate stationarity controls this distribution error up to a tangent-space residual measuring parameterization incompleteness. Finally, we establish finite-time convergence under fixed or controlled sampling and fixed, balance-trained, or separately updated backward policies. Under standard smoothness assumptions, with fixed or controlled sampling and a fixed or balance-trained backward policy, the expected loss is $O(\bar\epsilon^2+T^{-1})$ and expected terminal symmetric KL is $O(\bar\epsilon+T^{-1/2})$ under uniform coverage. For a separately updated backward policy, we characterize the sufficient update decay to approach the residual neighborhood and preserve these rates. These results connect balance objectives to sampling accuracy and clarify how parameterization, coverage, and moving components govern GFlowNet training.

arXiv cs.LG75

一步完成校准：联合学习准确且校准良好的神经网络

arXiv:2610.13693v1 公告类型：新论文 摘要：神经网络（NNs）虽然具有高度的灵活性和通用性，但往往校准不佳。事后校准方法使用单独的校准数据集和额外的校准参数，调整已训练模型的输出。这一过程需要单独优化校准器并进行后续模型选择，从而增加了开销。本文表明，通过双层 Stackelberg 形式化建模，标准的（事后）校准器可以与骨干网络一起进行端到端训练。该建模方式利用交叉熵联合优化准确性与校准效果，无需单独的校准集或专门的校准损失，仅引入一个额外的超参数：校准器学习率。尽管我们的框架支持从温度缩放、矩阵缩放到神经网络层等多种校准器，但实验结果表明，直接将最终分类层用作校准器，即可在不增加任何参数的情况下，得到准确且校准良好的模型。我们的方法仅增加极少的复杂度，便能超越当前最先进的校准器，同时还能避免在多任务学习等场景中，独立校准组件带来的成倍开销。

arXiv cs.LG73

方差缩减策略梯度近乎极小极大最优

arXiv:2610.13692v1 公告类型：新论文 摘要：标准策略梯度（PG）算法依赖蒙特卡洛更新，因此存在估计方差较高的问题。尽管离策略方差缩减能够缓解这一问题，但目前要确立最优样本复杂度，仍需要重要性权重一致有界等限制性假设，或不切实际的二阶运算。在本研究中，我们提出 Loopless Variance-Reduced PG（LVR-PG），这是一个统一的理论框架，对一类将递归更新与防御性重要性采样相结合的一阶估计器进行了抽象，其中包括 PAGE-PG 和 STORM-PG 等已有估计器。我们证明，对于一般参数化策略，在弱梯度支配条件下，这类算法能够以 $\tilde{\mathcal{O}}(\epsilon^{-2})$ 的样本复杂度实现最后一次迭代的全局收敛。我们仅使用一阶更新，在极少的假设下且不使用梯度预条件器，便确立了这一保证。通过将当前最先进的下界扩展至连续参数化策略，我们确立了 $\Omega(\epsilon^{-2})$ 的样本复杂度下限，证实我们的框架在忽略对数因子和时间跨度项后具有极小极大最优性。

arXiv cs.LG73

基于 Transformer 的扩散模型上下文得分算子学习

arXiv:2610.13690v1 公告类型：新论文 摘要：扩散模型通常学习单个目标分布的生成动力学，或学习由训练期间固定的条件变量索引的一族分布的生成动力学。这限制了它们仅凭少量示例适应此前未见分布的能力。我们提出了一种面向扩散模型的上下文解算子学习框架，其中每个目标分布都被视为从元分布中抽取的一个任务，并在推理时由一个经验上下文集指定。我们的方法学习一个上下文得分算子，将上下文样本、含噪查询点和扩散时间映射为任务相关的得分，从而无需重新训练或更新参数，即可在未见任务上进行反向扩散采样。一个关键发现是，任务相关得分具有高斯核条件均值表示。其经验 Nadaraya--Watson 近似可以由我们构建的 Transformer 类中的模型精确表示，为所提出的查询相关注意力架构提供了理论依据。该算子通过跨任务分布的摊销去噪得分匹配进行训练。我们建立了泛化误差界，量化了上下文大小、训练任务数量、任务内样本数量、模型复杂度以及内在维度和环境维度的影响。在数值实验中，我们在一系列科学与视觉数据集上评估了该方法，包括 Navier-Stokes、ERA5 气候数据、QuickDraw 草图和 Food-101。结果表明，所提出的方法在相对简单的任务族上表现良好，但其性能会随着任务复杂度的增加而下降，从而揭示了扩散模型上下文学习的潜力与局限。

arXiv cs.LG72

VQIL：用于图上扰动响应排序的可见量子干涉学习

arXiv:2610.13689v1 公告类型：新提交 摘要：学习局部扰动如何影响网络，对于识别哪些组件驱动或吸收系统响应至关重要。我们将这一问题表述为以种子节点为条件的响应排序：给定一个种子节点上的扰动，按响应强度对所有其他节点进行排序。当模型仅获得相互作用拓扑，而支配扰动传播的机制必须从数据中学习时，这项任务具有挑战性。我们提出 VQIL（Visible Quantum Interference Learning），一种用于以种子节点为条件进行排序的可训练 Szegedy 量子行走。边权重、连接相位以及各节点的相位旋转通过反向传播进行端到端优化，同时行走算符在整个训练过程中始终保持酉性。节点得分由量子行走振幅经过几何折扣的相干叠加得到，使量子干涉能够在最终的范数平方读出之前发生。由于每个学习参数都关联到一条具名边或一个具名节点，训练后的模型能够就各个图组件在传播机制中的作用提供可直接检验的假设。在六个输电网络的交流潮流扰动任务中，即使未向模型提供电气参数，VQIL 仍将 Spearman 秩相关系数从未训练行走的 0.147 提升至 0.831，并优于采用相同方式训练的随机游走模型。在以 Perturb-seq 为监督的五个蛋白质相互作用网络的基因扰动任务中，VQIL 将秩相关系数从 0.171 提升至 0.620，超越了表现最强的已训练经典基线模型和神经网络基线模型。这些结果表明，可训练的量子行走干涉为学习图上以种子节点为条件的响应排序提供了一种可扩展且可解释的归纳偏置。

arXiv cs.LG75

论利用协变量的时间序列基础模型中日历特征的编码

arXiv:2610.13676v1 公告类型：新论文 摘要：时间序列基础模型（TSFMs）越来越多地支持协变量，其中日历信息是实践中最常见的类别协变量，因为每个时间点都有相应的日历信息。然而，这些特征的编码方式大多被视为实现细节，而非研究对象，尽管编码是预训练模型在推理时仍可调整的少数设计选择之一。本研究表明，信息传入模型的方式会显著影响 TSFMs 的预测准确性。因此，我们在来自多个领域的九个数据集上，对四种最先进的 TSFMs 进行零样本滚动评估，系统比较了七种日历编码，包括三种常规的、与目标无关的表示方式（序数编码、独热编码和周期编码）以及四种目标编码。这九个数据集受到周末和节假日等相关社会活动的影响，因此日历信息对预测尤为重要。我们发现，编码选择会影响所有模型的预测准确性；在同一模型内，不同编码在非节假日的 MASE 差异最高可达 9.9%，在节假日则可达 83%。编码的有效性及其被利用的程度在不同模型间存在显著差异。性能提升主要集中在节假日前后，此时日历信息提供了目标变量历史数据中无法获取的信号；同时，某些模型与编码的组合也能改善非节假日的预测。基于 SHAP 的归因分析确定了协变量相对于自回归部分的贡献，并解释了某些模型与编码的组合为何会导致性能下降。我们的研究结果表明，当前针对利用协变量的 TSFMs 的基准测试，可能会因采用固定编码策略而扭曲模型比较结果。

arXiv cs.LG68

Pretraining and Test-Time Training Budgets Matter for ARC: A Controlled Study of VARC and TRM

arXiv:2610.13660v1 Announce Type: new Abstract: The Abstraction and Reasoning Corpus (ARC) is increasingly used to evaluate whether neural networks can infer task-specific transformations from few demonstrations. Recent ARC models differ in architecture, but also in broader training and evaluation protocols, including pretraining data, data augmentation, test-time setup, and optimization budgets, making conclusions about architecture benefits debatable. In this work, we study how the performance of the vision-inspired VARC and recursion-based TRM models is affected by pretraining and test-time training (TTT) budgets under a common inductive protocol on ARC-AGI-1. We cross five pretraining budgets with four TTT budgets for each test task. Each budget specifies a task-variant count and total sample presentations, defining the controlled training budgets. We observe that increasing TTT sample presentations improves the standard ARC pass@2 metric in all settings for both models. Increasing the TTT task-variant count consistently helps only TRM, whereas VARC often performs better with fewer variants. VARC also remains competitive without pretraining, while TRM benefits much more from it. These results show that pretraining and TTT budgets affect ARC evaluation and motivate reporting at comparable budgets.

arXiv cs.LG77

TabReliBench：超越预测准确性的表格基础模型评估

arXiv:2610.13654v1 公告类型：新论文 摘要：表格基础模型（TFMs）因其强大的预测性能而日益受到欢迎。许多此类模型还提供后验预测分布（PPDs），但这些分布的可靠性仍缺乏充分研究。我们从点预测、分布质量、边际与子群校准以及区间效率等方面评估可靠性。随后，我们推出 TabReliBench，这是一个用于上述比较和数据集层面分析的大规模分类与回归基准。我们发现，TFMs 在 TabArena 和本基准上的排名有所不同，在不同可靠性指标上的排名也会发生变化。回归 TFMs 通常能够达到接近名义水平的边际覆盖率，同时保持较窄的区间，但这种良好的边际表现并不意味着条件覆盖率也能得到保证。数据集层面的分析表明，分类难度较高往往伴随着较差的校准，而回归预测性能并不是衡量校准情况的可靠指标。集成方法能够提高预测性能，但并不总能改善 PPDs 的校准；TFMs 从保形后处理中获得的收益也很有限，这种处理无法可靠地消除子群间的差距。因此，要可靠地选择 TFM，就需要超越综合排名，开展多维度评估。代码可在 https://github.com/Tianqi-Zhao/TabReliBench 获取。

arXiv cs.LG78

SelectArena：面向真实世界表格数据的现代特征选择基准

arXiv:2610.13644v1 公告类型：新论文 摘要：从表格数据集中选择特征（列）的子集，是现实应用中普遍存在的挑战。行业专家希望最大限度地降低模型训练成本，气候科学家需要一组可解释的因素子集来理解极端天气事件，临床医生则因资金和时间限制而需要在诊断检查项目之间作出选择。然而，当不同领域的实践者查阅科学文献以确定采用哪种特征选择方法时，所面对的基准评测体系缺乏代表性。尽管相关研究十分广泛，现有特征选择基准所能提供的指导仍然有限，因为它们依赖过时的模型、临时制定的评估流程，或无法反映现实问题的人工生成数据。为弥合科学基准评测与现实应用之间的差距，我们推出了现代特征选择基准 SelectArena。我们采用标准化评估流程和最先进的表格（基础）模型，在 71 个经人工精心整理、涵盖分类与回归问题的真实世界表格数据集上，对 15 种广泛使用的特征选择方法进行了比较。结果揭示了预测性能、运行时间、稳定性和有效性之间的明确权衡。在同时考虑预测性能和运行时间时，基于树的方法 CART 和 RFImportance 以及 F-Test 均达到帕累托最优。F-Test、基于信息论的方法和基于距离的方法在稳定性与有效性方面表现领先，其中 F-Test 在我们采用的各项指标上展现出均衡表现。基于搜索的方法速度慢、稳定性差，且性能较低。SelectArena 将这些发现转化为实践指导，并作为一个可扩展的平台，为未来的特征选择研究提供支持。

arXiv cs.LG68

PANDA: Prior-guided Attentional Dual-path Architecture

arXiv:2610.13635v1 Announce Type: new Abstract: Hypergraph Neural Networks (HNNs), designed to model higher-order relations beyond pairwise interactions, typically rely either on fixed structural weights or on normalised attention. Fixed weights can propagate misleading information in heterophilic settings, while normalisation makes attention coefficients competitive, in the sense that, in a message-passing framework, increasing the weight of one sender must reduce the weight of others in the same aggregation set. Our main contribution is the introduction of PANDA (Prior-guided AttentioNal Dual-path Architecture), a plug-in for two-stage message passing HNNs applicable to both undirected and directed settings. The main path interpolates between learned receiver-normalised attention and structural incidence priors, while a non-competitive (not normalised) auxiliary path lets each sender contribute its own transformed information. We also show that, with fixed structural coefficients, PANDA recovers a certain complex-valued convolution operator adopted in previous works. Across four HNNs spanning spatial and spectral methods in both directed and undirected settings, plugging PANDA in yields an average improvement of 5.18 percentage points across eight real-world benchmark datasets. The code is available at https://github.com/jainris/panda.

arXiv cs.LG68

Private and Distributionally Robust Federated Multiclass Classification

arXiv:2610.13631v1 Announce Type: new Abstract: Fault diagnosis in asset-intensive industries requires multiclass models trained on noisy, sensitive data collected across geographically distributed client sites. Federated learning enables collaborative training without centralizing data, but most approaches do not provide formal privacy guarantees for individual records stored at client sites and can degrade under label noise. To address these limitations, we develop a differentially private, distributionally robust framework for federated multiclass classification that provides sample-level privacy for records stored at each client. We build on a federated Wasserstein distributionally robust multiclass objective with the Crammer-Singer loss, solved using an Alternating Direction Method of Multipliers (ADMM)-based algorithm. To protect the released client updates, our method injects calibrated Gaussian noise into the proximal linearized client updates of both the local model and the local distributionally robust variable before aggregation at the central server, and we quantify the cumulative privacy loss using the moments accountant. We provide per-round sample-level $(\epsilon,\delta)$-differential privacy guarantees and an overall composed privacy guarantee across all training rounds, together with convergence results and an explicit privacy-utility tradeoff. Experiments on simulated industrial fault data and UCI benchmark datasets show that our method outperforms differentially private federated empirical risk minimization and stochastic gradient descent baselines in most tested settings, with the largest gains under strong privacy-induced noise.

arXiv cs.LG

A small cognitive core explains human behavior at scale

arXiv:2610.13603v1 Announce Type: new Abstract: Psychology has proposed many principles of cognition. Each, however, was developed in isolation, leaving their combined explanatory power unknown. We assembled a small set of established principles into one model, which we call COntextual Retrieval and Encoding (CORE). CORE consists of a hierarchy of episodic memories into which prediction errors are written and whose encoding, retrieval, and forgetting depend on context. Fitted jointly to 7,036,442 choices from 137 experiments, the model predicts human choices better than black-box models and reproduces 75% of reported behavioral effects when simulated. Moreover, its latent variables show classic signatures of Bayesian learning and event segmentation. Taken together, these findings suggest that a small cognitive core accounts for much of human behavior. They also open a path toward making psychology more than the sum of its parts, by building computational models on the entire experimental record.

arXiv cs.LG76

EchoTab：通过表格重建学习以测量为基础的表征，用于超声心动图诊断

arXiv:2610.13599v1 公告类型：新论文 摘要：在超声心动图等多视图医学影像检查中，诊断信息分布于不同视图中，每个视图捕捉心脏结构和功能的不同方面。在临床实践中，医生首先通过定量测量解读这些视图，再依据测量结果作出具体诊断。然而，现有的超声心动图表征学习方法并未将测量显式建模为视图与诊断之间的中间表征。因此，各项测量未能以可供下游诊断任务选择性使用的形式得到显式表示，从而妨碍了准确诊断。受这种分层临床工作流程的启发，我们提出了 EchoTab，这是一种以测量为基础的表征学习框架，使用测量特定的词元来表示多视图超声心动图检查，并将其作为超声心动图视图与诊断之间的中间表征。每个测量词元选择性地关注与其估计相关的视图，并通过表格重建进行训练，以预测对应的测量值。由此得到的词元构成一种因子化的超声心动图表征，编码了具有临床意义的结构和功能特征，可供下游诊断选择性使用。与常用的对比学习方法不同，EchoTab 不需要负样本或大批量训练。在两个数据集上的实验表明，EchoTab 能够学习有效的、以测量为基础的表征，从而提升多视图超声心动图的诊断分类性能；其学习到的注意力也与心脏科医生所采用的既定测量与视图对应关系一致。我们的代码可在 https://github.com/michelleespranita/EchoTab 获取。

arXiv cs.LG64

$L_0$ Feature Selection via Monte Carlo Tree Search in High-Dimensional Financial Regression

arXiv:2610.13574v1 Announce Type: new Abstract: Feature selection is a central problem in high-dimensional financial regression, where large numbers of engineered variables and strong correlations among predictors reduce model stability and interpretability. $L_1$ regularization (Lasso) is widely used for sparse regression, but its shrinkage can be restrictive when informative signals are spread across correlated variables. Exact sparse regression avoids this shrinkage but requires solving a combinatorial optimization problem that is computationally difficult and is typically fit to minimize in-sample rather than out-of-sample error. This paper presents a feature selection framework based on Monte Carlo Tree Search with Generalized Nested Rollout Policy Adaptation (GNRPA). Feature selection is formulated as a sequential decision process, and candidate subsets are scored by their out-of-sample validation performance rather than training error. The method also supports a lexicographic objective accounting for subset size and multicollinearity. We study this approach on a contemporaneous explanatory regression task: explaining the weekly return of the NASDAQ-100 index from 1,072 engineered features spanning market, macroeconomic, and factor data. This isolates the value of subset selection under multicollinearity from the separate problem of predictive forecasting. On this task, the proposed method identifies sparse feature subsets with higher out-of-sample explanatory power than an optimized Lasso baseline, reaching an out-of-sample $R^2$ of 0.6961 versus 0.6313 for Lasso. These results suggest that Monte Carlo Tree Search is a practical alternative to convex regularization for sparse explanatory feature selection in financial regression, with potential value for factor attribution and risk decomposition where transparency matters as much as out-of-sample fit.

arXiv cs.LG

Simultaneous Competitive and Regret Guarantees for TCP Acknowledgment

arXiv:2610.13571v1 Announce Type: new Abstract: We study simultaneous competitive and regret guarantees for TCP acknowledgment. Each acknowledgment serves all pending packets, and the objective is to minimize the total acknowledgment and waiting costs. It is well known that no deterministic online algorithm achieves a competitive ratio better than $2$. Given $k$ randomized online experts and $0<\varepsilon<1$, we construct a randomized online algorithm whose realized total cost is always at most $(2+\varepsilon)$ times the offline optimal cost $\mathrm{OPT}$ and whose expected regret against the expert with the smallest realized cost is $O(\sqrt{\mathrm{OPT}\log(k+1)}+\log(1/\varepsilon))$. The algorithm requires no advance knowledge of the number of packets or $\mathrm{OPT}$. Our algorithm controls switching costs between experts and uses the gap between the permitted competitive bound and its current cost to determine when to learn. We also prove lower bounds showing that, for $k\ge 2$, this regret bound is asymptotically optimal in $\mathrm{OPT}$, $k$, and $\varepsilon$, and that $\varepsilon>0$ is necessary for sublinear regret.

arXiv cs.LG71

Rationally Linearized MLPs

arXiv:2610.13568v1 Announce Type: new Abstract: We present Rationally Linearized Multilayer Perceptrons(RaLiMPs), a new approach to efficiently linearizing MLPs using low-degree rational approximations of standard activation functions. Inspired by polynomial-based MLP linearization, we show that rational approximations can provide accurate representations at substantially lower degree. RaLiMPs exploit these approximations to construct kernel representations with narrow bottlenecks, enabling efficient composition across MLP layers. We provide theoretical guarantees for the resulting approximation and evaluate RaLiMPs across applications ranging from implicit 3D models to large language models. Our experiments demonstrate favorable accuracy--efficiency tradeoffs, with RaLiMPs achieving competitive or improved accuracy while substantially reducing inference cost.

arXiv cs.LG68

Probabilistic Ship Trajectory Forecasting using Transformer Models on AIS Data

arXiv:2610.13557v1 Announce Type: new Abstract: Anticipation of vessel trajectories is an essential task when navigating at sea. This applies to both human navigators and autonomous surface vessels. Trajectory forecasting enables sub-routines such as collision avoidance to assess the situation and react accordingly in time. This paper investigates Transformer and Performer models for trajectory prediction of surface vessels in confined waters. Using widely available position data from the Automatic Identification System (AIS), candidate models are trained and evaluated. Historical information on how other vessels have moved in the local area surrounding each agent is conveyed using positional heat maps. The heat maps are generated using Kernel Density Estimation (KDE) and encoded with a Convolutional Neural Network (CNN) before entering the Transformer models. The models are trained with quantile loss to efficiently obtain a model-aware prediction interval, as this is crucial for timely collision avoidance. The proposed model architectures are shown to greatly outperform previous work while the inclusion of local heat maps enables the prediction of vessel movement that is not otherwise indicated from the previous positions.

arXiv cs.LG74

通过逐层随机特征模型模拟深度神经网络训练中的尖峰

arXiv:2610.13524v1 公告类型：新论文 摘要：大语言模型（LLMs）预训练过程中损失间歇性出现的“尖峰”，在理论和实证层面仍是一种尚未解明的现象。本文表明，输入服从重尾幂律分布的简单二次模型可以复现这些尖峰。我们提出了一套理论框架和仿真算法，利用此类二次模型对深度神经网络（DNNs）的逐层训练动态进行建模。在一项针对 Adam 超参数配置的案例研究中，该配置将 $\beta_2$ 设为接近 $1$ 的值，已知会在 Transformer 语言模型中引发尖峰；我们的仿真成功地从给定检查点出发，预测出了预示尖峰行为的隆起。此外，我们根据观测指标推导出一种合适的 $\beta_2$ 调度方案，并通过实证验证了其性能改进效果。

arXiv cs.LG62

FedPCD: Priority-Constrained Federated Aggregation

arXiv:2610.13508v1 Announce Type: new Abstract: Federated learning (FL) seeks to optimize a shared global model from decentralized client data. While effective for average performance, this process can produce unfair performance across clients, particularly under statistically heterogeneous data distributions. Existing fairness-aware and multi-objective FL methods address this through client re-weighting, gradient correction, or common-descent formulations, but often change the global optimization direction or treat client objectives symmetrically. We propose FedPCD, a priority-constrained federated optimization framework that separates the global optimization objective from client protection. FedPCD preserves the conventional federated aggregate as the primary objective, while allowing only the minimum correction necessary to meet client performance constraints. Across heterogeneous FL benchmarks, FedPCD consistently outperforms state-of-the-art baselines, achieving the highest global accuracy across all configurations and leading worst-10% client accuracy in nearly all. FedPCD further preserves competitive trade-offs between global accuracy and client performance variability. Beyond this trade-off, FedPCD demonstrates faster empirical convergence, reaching matching or superior performance in substantially fewer communication rounds.

arXiv cs.LG75

超越时间邻近性：用于强化学习信用分配的 Explanatory World Models

arXiv:2610.13504v1 公告类型：新论文 摘要：生成式推荐系统越来越多地使用强化学习（RL）来优化可能在决策后数百乃至数千次交互后才出现的结果。标准 RL 根据时间邻近性而非明确的相关性传播奖励，削弱了分配给时间久远但重要的动作的信号，成为样本效率低下的一个来源。我们提出 Explanatory World Models（EWM），这是一种归因框架，利用解释识别可能促成已实现结果的早期事件。我们使用前沿语言模型实现 EWM，并通过一种守恒的层次化算子将其解释用作学习信号；该算子递归地重新分配奖励，同时保持未折扣的回合回报不变。为了探究什么样的解释才有用，我们将受 Halpern 的实际因果关系、Lombrozo 的解释简洁性以及 Deutsch 的难以变动的解释启发的三种互补变体转化为可操作的方法。我们在两个受控的长时程 RL 基准、一个公开的序列推荐基准以及一个工业规模的生成式推荐系统上评估 EWM。在 Crafter 和 WordCraft 上，EWM 显著优于 PPO 和 LaRe，且在奖励延迟最严重时取得最大提升。在 KuaiRand 上，EWM 在离线评估中优于 SFT 和 LaRe，在在线评估中优于 A2C。最后，在对工业级生成式推荐系统的离线评估中，EWM 显著提高了召回率，并提升了行为代理指标。这些结果表明，解释能够为从延迟结果中学习提供结构化且可复用的信用归因。

arXiv cs.LG80

PPG-Bench：面向 PPG 基础模型的统一基准

arXiv:2610.13435v1 公告类型：新投稿 摘要：可穿戴设备和临床设备正以日益庞大的规模记录光电容积描记（PPG）信号，越来越多的 PPG 基础模型（FM）也相继被开发出来，用于计算可供各种下游任务复用的表征。然而，这些模型难以进行比较：不同评估采用的数据集、任务定义、预处理方法、下游预测头、超参数调优方法和基线各不相同。我们提出 PPG-Bench，这是一个统一的开源基准，在九个开放访问数据集和 26 项任务上评估五个公开的 PPG 基础模型，任务涵盖心血管状态估计、活动、睡眠、筛查和身份识别。我们在适应能力逐步增强的四种下游设置下评估模型（线性探测、随机森林、冻结模型的 MLP 探测和端到端微调），并采用针对各模型的超参数优化及受试者互不重叠的交叉验证。我们还设置了三类对照：随机/常数基线、确定性的脉率变异性（PRV）特征集，以及通用时间序列基础模型（MOMENT）。在全部 68 种任务与设置组合中，35.3% 的组合未显示基础模型相较于随机/常数基线具有实质性优势，而在 83.8% 的组合中，没有任何 PPG 专用基础模型能同时优于 PRV 和 MOMENT。我们还发现，增加下游预测头的容量通常可以提升性能，但如果模型的表现处于随机猜测或常数预测水平，则可能导致性能显著下降。因此，目前的 PPG 基础模型尚不能宣称能够提供具有泛化性和可迁移性的 PPG 信号表征，该领域未来的评估应纳入与简单、非学习型对照方法的比较检验。该基准的代码及评估更多模型的说明见：https://github.com/mannas-ai/ppg-bench。

arXiv cs.LG67

In-Context Learning for System Identification: Probabilistic Formulation, Long-Sequence Handling, and Benchmark Validation

arXiv:2610.13411v1 Announce Type: new Abstract: Traditional system identification typically involves two steps: initially, training data is processed to estimate a model of the system of interest; then, the model is used to predict outputs for new input queries. In contrast, recent in-context learning approaches learn a meta model that maps directly from training data and query inputs to outputs in a zero-shot fashion, bypassing explicit model estimation. This approach leverages a meta model trained offline, on large sets of synthetically generated data, to emulate the behavior of a system identification algorithm. In this work, we advance this meta-learning framework for system identification in three directions. First, we develop a unified probabilistic meta-identification architecture that combines a Transformer-based encoder-decoder with tailored mechanisms for handling long training sequences, enabling calibrated predictions over long horizons. Second, we show how a pre-trained meta model can act as an uncertainty-aware synthetic data generator for classical system identification, providing task-specific data that regularize parametric models when real data are scarce. Third, we systematically evaluate the framework on standard nonlinear system identification benchmarks and on a challenging out-of-distribution scenario, demonstrating competitive accuracy with state-of-the-art methods and strong generalization, including cases where input-only adaptation is sufficient to handle significant distribution shifts.

arXiv cs.LG71

Accelerating Linear Bandits with Persistent Candidate Sets

arXiv:2610.13407v1 Announce Type: new Abstract: Exhaustive optimistic scoring can make action selection costly in linear bandits with a large fixed set of $K$ actions, requiring $\mathcal{O}(K)$ score evaluations per round. We introduce Persistent Candidate LinUCB (\textsc{PC-LinUCB}), which maintains $k$ active actions and samples $q$ challengers uniformly from the remaining $K-k$ actions at each round. By scoring only these $k+q$ candidates, \textsc{PC-LinUCB} reduces the number of score evaluations per round to $\mathcal{O}(k+q)$. We establish problem-independent and problem-dependent high-probability regret bounds consisting of a standard linear-bandit learning term and an additive discovery cost of order $\mathcal{O}\left((K-k)\log(K-k)/q\right)$ at fixed confidence. Notably, this additional cost does not grow with the horizon. For fixed gaps and sufficiently long horizons, we establish an $\Omega\left((K-k)/q\right)$ regret lower bound under uniform challenger sampling, showing that partial candidate access itself imposes an unavoidable regret cost. Experiments show that \textsc{PC-LinUCB} substantially accelerates LinUCB while maintaining comparable regret, and that the persistent candidate mechanism can extend beyond LinUCB.

arXiv cs.LG67

Why Linear Models Can Still Work in a Complex and Changing World

arXiv:2610.13404v1 Announce Type: new Abstract: Linear models can predict well in complex domains even when underlying mechanisms are nonlinear. We argue that this can occur because the complexity of the world exceeds the complexity that can be learned reliably from finite, partially observed data. Flexible models can exploit curvature, thresholds, and interactions that fit one hospital, period, population, policy setting, or measurement process but fail elsewhere. When this instability lies mainly in the nonlinear component, omitting it can give linear models lower prediction error in a new environment. We connect this claim to work on simple linear rules, bias and variance, regularization, best linear prediction, measurement error, distribution shift, invariance, Rashomon sets, and underspecification. Proposition 1 gives a simple squared-error case in which a nonlinear rule learned in the training environment raises target error because it transfers too weakly; finite-sample estimation error can strengthen this result. Partial observation alone is not enough: the link between observed variables and unobserved factors must vary across environments. Stable nonlinear structure favors flexible models, and linear models remain vulnerable to linear spurious relationships. Model complexity should therefore be chosen by cross-environment stability as well as training-data accuracy.

arXiv cs.LG75

PlacePulse：从结构性地理数据中学习可迁移的地点表示

arXiv:2610.13390v1 公告类型：新论文 摘要：地理空间机器学习通常依赖针对特定任务的特征工程，对大量社会人口、商业和环境变量进行处理。地理空间基础模型提供可复用的地点表示，但主流方法通常编码人口流动和搜索活动等动态信号，或来自卫星影像的视觉信号。很少有模型将地点的静态结构组成（谁在那里生活和工作、哪些企业在那里经营，以及它们面临哪些风险）表示为精细空间分辨率下的可迁移嵌入。我们提出 PlacePulse，这是一种结构性地理空间基础模型，将 572 个属性通过 H3 邻域聚合扩展为 2,288 个输入，再压缩为 256 维单位范数嵌入，覆盖美国 1,055,537 个 H3 分辨率为 7 的网格单元。我们还提出了一套可复用的双轴基准，在多个目标和空间支撑单元上，将表示的内在质量与下游预测效用区分开来。在内在质量方面，这些嵌入重构标准化输入的平均 $R^2 = 0.853$，利用了 256 个维度中的 251 个近似去相关维度，并呈现出显著的空间组织性：第一主成分的 Moran's $I = 0.93$，且自相关性在超出输入半径的范围内依然存在。在外在效用方面，在来自 12 个外部来源、涵盖四种空间支撑单元的 74 个留出目标上，PlacePulse 在与训练输入无关的目标上的表现优于维度相同的特征选择基线，并可与 Google 的 Population Dynamics Foundation Model 相媲美；它在健康、经济、选举、经济流动性和社会脆弱性结果方面表现领先，其相对优势因领域而异。这些发现表明，结构组成是地理空间基础模型中一种独特且互补的模态，而不只是在行为或影像数据不可用时的备选方案。

arXiv cs.LG73

BLOKS：利用草图化分块预条件扩展核求解器的规模

arXiv:2610.13382v1 公告类型：新提交 摘要：核岭回归（KRR）是一种具有坚实理论基础的非参数学习算法，也常被用于研究深度学习的性质。然而，精确的 KRR 形式所需的时间和内存开销使其无法适用于大型数据集，这推动了近似求解器的发展。这些算法依赖于：（i）迭代方法，以避免直接求解的计算负担；（ii）草图化或分批处理，以缩减问题规模；以及（iii）预条件，以加速收敛。虽然预条件对于快速收敛至关重要，但它也会带来内存开销。减少模型的自由参数数量可以控制这部分开销，却可能牺牲模型的表达能力，进而降低准确性。为应对这一挑战，我们提出了 BLOKS，这是一种基于共轭梯度法的求解器，用于经过 Nyström 草图化的 KRR，其核心是一种将分块 Jacobi 近似与草图化相结合的新型预条件器。BLOKS 预条件器的内存开销既不依赖于数据集大小，也不依赖于参数数量，因此能够扩展至大型模型。我们提供了一种内存高效、支持 GPU 加速的实现，并表明 BLOKS 在保持良好的时间和内存复杂度的同时，能够取得出色的准确性。事实上，在用于核求解器基准测试的标准大规模数据集上，BLOKS 的表现与最先进的方法相当或更优。

arXiv cs.LG75

通过信息瓶颈揭开无分类器引导的黑箱

arXiv:2610.13369v1 公告类型：新论文 摘要：无分类器引导（CFG）是改善扩散模型条件生成效果的标准技术，但其激活调度通常依靠启发式方法选定。在实践中，大多数现有的基于 CFG 的方法在整个推理过程中依赖预先定义的激活策略，而近期的动态变体往往依赖人工设计的调度方案，缺乏关于条件信息何时最有用的严谨理论解释。为理解条件输入与得分函数之间的信息相关性如何随生成过程演变，我们从信息论视角探讨应在何时激活引导。在本研究中，我们将扩散轨迹建模为随时间变化的信息瓶颈，并表明条件信息的有效性在整个采样过程中并不均匀，而是在由 Fisher 信息平衡条件刻画的临界阶段之后显著增强。受这一发现启发，我们提出了一种用于推理的免训练动态 CFG 策略。我们的方法并非预先指定固定调度，而是根据条件得分与无条件得分的统计量构建随机过程，并在估计的引导增益超过预设阈值时激活引导。由此得到的自适应激活规则使条件注入与底层生成动力学相匹配，同时也将基于区间的引导涵盖为一种极限特例。在多种架构和任务上开展的大量实验表明，我们的方法有效、稳健且具有广泛适用性。除实证改进外，我们的框架还为动态引导方法提供了解释，并为条件扩散模型的推理时控制奠定了严谨的理论基础。

arXiv cs.LG80

临床机器学习中隐藏的数据挑战

arXiv:2610.13350v1 公告类型：新提交 摘要：广泛复用的电子健康记录预处理流程可能隐含一些假设，涉及信息何时可用、标签代表哪一次临床诊疗过程，以及数据缺失意味着什么。这些假设很少得到系统性审查，并可能随着预处理流程的复用或调整而传播到后续研究中。我们以 MIMIC-IV 为案例，识别并量化了三种互补的预处理对齐形式，即时间对齐、语义对齐和人群对齐的违反情况，它们分别表现为时间泄漏、特征与任务不匹配以及有偏插补。我们展示了时间戳语义、诊疗过程定义、终点构建、模态选择和缺失值处理中的歧义如何改变模型可用的信息，并影响对其预测结果的解释。通过对 MIMIC-IV 数据的分析和实证案例研究，我们证明，这类选择虽然会改变模型所依赖的信号，却可能在总体预测性能中基本不显现。我们据此提出了切实可行的建议，以便对 MIMIC-IV 及相关电子健康记录数据集开展时间上有效、与临床相契合且可复现的预处理。我们的实验代码（包括所提出的预处理方法）可在以下地址获取：https://github.com/m2l-lab/mimic-iv-preprocess。

arXiv cs.LG69

Understanding Mamba-3 Through Oscillatory Dynamics

arXiv:2610.13341v1 Announce Type: new Abstract: Mamba-3 introduces input-dependent rotations into recurrent state updates, but their contribution beyond state retention remains unclear. We interpret these ro- tations as underdamped second-order dynamics, providing a framework for in- vestigating the role of oscillation in sequence modeling. Controlled comparisons show that underdamped dynamics outperform non-oscillatory damping regimes on state-tracking tasks, while language-model pretraining demonstrates benefits for long-context prediction and length generalization. Interventions in pretrained models further establish the importance of learned phase, and state analysis sug- gests that rotation regulates how retained information combines with incoming up- dates. Taken together, these findings identify underdamped dynamics as a useful mechanism for recurrent computation and suggest that oscillation supports long- context processing by regulating how information accumulates.

arXiv cs.LG71

AgentTune: A Unified Training Lifecycle for Tool-Using LLM Agents

arXiv:2610.13335v1 Announce Type: new Abstract: Tool-using agents increasingly require a common substrate for building, training, evaluating, distilling, and maintaining agentic systems, yet existing tooling typically addresses these stages in isolation. We present AgentTune, a source-available Python library that provides a unified framework for developing and optimizing tool-using agents. At its core, AgentTune introduces a two-tier EventLog representation that preserves agent trajectories while supporting consistent projections for training, evaluation, and auditing. Building on this substrate, the library provides a shared agent runtime, tool-calling reinforcement learning, agentic distillation, retrieval-augmented generation, decision workflows, evaluation utilities, and a designed path toward self-healing agents. These components are exposed through composable interfaces so that agent strategies, training methods, evaluation procedures, and recovery mechanisms can be combined without requiring separate representations or pipelines. We demonstrate the framework across agentic tasks including legal contract search and domain-specific applications, illustrating its use for trajectory-based optimization, tool-use evaluation, and cross-task agent development. AgentTune is released as a source-available library at https://github.com/Lexsi-Labs/AgentTune

arXiv cs.LG67

PDE-PFN: Prior-Data Fitted Neural PDE Solver

arXiv:2610.13334v1 Announce Type: new Abstract: Motivated by the success of large language models (LLMs) with broad generalizability and robustness to noisy or unreliable pre-training data, we explore in-context prediction of PDE solution fields using approximate pre-training data. In addition, inspired by the posterior predictive mean-based inference mechanism of the in-context learning in prior-data fitted networks (PFNs), we propose PDE-PFN, a prior-data fitted neural solver that approximates the posterior predictive mean of PDE solutions via in-context learning. PDE-PFN builds on a PFN architecture with self- and cross-attention mechanisms of Transformer and is pre-trained on noisy approximate solutions generated by physics-informed neural networks, serving as diverse but not necessarily exact priors. Through experiments on a range of two-dimensional PDEs, we demonstrate that PDE-PFN achieves empirical generalization across heterogeneous equations, robustness under noisy priors, and zero-gradient-update in-context inference capability. PDE-PFN achieves competitive or improved predictive accuracy relative to the evaluated baselines while supporting flexible context and query representations.