Bayesian Anytime Pareto Set Identification for Multi-Objective Multi-Armed Bandits

arXiv:2606.18785v2 Announce Type: replace Abstract: Identifying Pareto optimal solutions is critical to support multi-objective decision-making. In this work, we present Top-Two Pareto Front Thompson Sampling (TTPFTS), the first anytime algorithm for Pareto Set Identification in Multi-Objective Multi-Armed Bandits (MOMABs), taking a Bayesian approach. We benchmark TTPFTS against state-of-the-art fixed-confidence and fixed-budget Pareto Set Identification algorithms on synthetic environments. Next, we demonstrate its practical utility in a challenging real-world multi-objective multi-agent molecular discovery setting by efficiently exploring an ultra-large synthesis-on-demand molecular library, outperforming industry-standard and state-of-the-art baselines. Furthermore, we introduce a novel uncertainty quantification metric that estimates our algorithm's confidence in the predicted Pareto set. We demonstrate that this metric effectively proxies true performance, yielding a robust methodology for monitoring learning progress in complex settings. Finally, we complement these empirical findings with the first finite-time theoretical analysis of an anytime MOMAB Pareto Set Identification algorithm.