Self-Organized Conformal Prediction: Reducing Regional Coverage Gaps with Unsupervised Group Discovery

arXiv:2606.29403v3 Announce Type: replace-cross Abstract: Conformal prediction guarantees marginal coverage, but a pooled calibration quantile can hide systematic undercoverage across heterogeneous regions of the feature space. We introduce Self-Organized Conformal Prediction (SOCP), a calibration scheme that discovers input-space groups with an unsupervised Self-Organizing Map (SOM) trained without calibration data. At prediction time, the query's best-matching unit (BMU) retrieves a calibration buffer from one cell, a fixed grid neighborhood, or a prototype enlargement (Regimes 1 to 3). When fixed neighborhoods are too sparse, Regime 3 adds cells by prototype distance, using a global budget selected from training cell occupancies and the planned calibration size before any calibration score is observed. The predictor and nonconformity score remain unchanged. Cell-only retrieval has exact cell-conditional validity, and each fixed union of cells has exact retrieved-set validity. Interpreting a neighborhood threshold at its central cell incurs an explicit Kolmogorov-Smirnov (KS) bias term. Across ten regression and classification benchmarks, SOCP lowers the mean weighted coverage gap (WCovGap) relative to pooled split conformal prediction (SCP) on all ten datasets. The mean relative change is -14.8%, with a mean increase of 3.0% in output size. Under fixed-neighborhood retrieval, SO calibration lowers mean WCovGap across ten seeds in 43 of 50 score and dataset comparisons, while SO-SCP lowers it for every dataset at all three tested external partition granularities. These results provide local group calibration without supervised partitions or predictor retraining, while the diagnostics make the cost and limits of locality explicit. The code is available at https://github.com/LouisTier/SOCP.