LightSplit: Practical Privacy-Preserving Split Learning via Orthogonal Projections (Full Version)

arXiv:2605.13265v2 Announce Type: replace Abstract: Split learning (SL) enables collaborative training by partitioning a neural network across clients and a central server. However, the cut-layer interface introduces a key challenge: high-dimensional activations incur substantial communication overhead while exposing representations vulnerable to reconstruction attacks. Existing approaches typically address efficiency or privacy in isolation. We propose LightSplit, which limits information exposure and reduces communication overhead via a lightweight fixed orthogonal random projection at the cut layer. Based on Shannon's information theory, this projection acts as an information bottleneck that restricts instance-specific information and suppresses exploitable per-sample signals. With low-dimensional projections sent instead of raw activations, the server operates on lifted representations without requiring changes to existing SL architectures. Adding no trainable client components, LightSplit remains lightweight and suitable for edge devices while preserving end-to-end differentiability via exact gradient propagation. As the projection is non-invertible, part of the original representation is irreversibly discarded at the client, so LightSplit reduces the information available for reconstruction. We evaluate LightSplit on seven benchmarks spanning image classification, tabular data, and language-model fine-tuning, under IID and non-IID client data. At up to 32x compression it retains at least 94% of baseline accuracy on CIFAR-10, FashionMNIST, GTSRB, and UCI HAR, 86% on CIFAR-100, and 91% on average under non-IID client data; at 8x it retains 92% of ImageNet-1k top-1 and raises GPT-2 perplexity by at most 1.15x. Against state-of-the-art reconstruction attacks LightSplit resists inversion; the strongest, co-trained attack loses up to 87.3% of its reconstruction SSIM, and subject re-identification on UCI HAR drops by more than half.