SPARC: Scalable 3D Panoptic Segmentation with Reinforcement-driven Clustering

Haoling Huang, Ting Han, Jie Ouyang, Yiping Chen

Abstract. Large-scale 3D panoptic segmentation is critical for digital twins and geospatial analysis, demanding models that process massive point clouds while distinguishing instances across highly diverse spatial scales. However, prevailing graph-based approaches rely on one-shot optimization, suffering from short-sighted decisions where irreversible local errors propagate globally, leading to severe under-segmentation at boundaries between objects of disparate scales. To overcome this short-sightedness, we present SPARC, a scalable framework that reframes graph clustering as a sequential, self-correcting decision process driven by hierarchical reinforcement learning. Specifically, SPARC employs a dual-level agent where a meta-controller adaptively determines instance completeness while a low-level policy iteratively refines edge affinities, enabling the model to revise early mistakes based on long-horizon rewards rather than greedy local cues. Complementing this, we introduce Semantic Voxel Partitioning (SVP) to generate semantically coherent superpoints, ensuring robust primitives that mitigate noise before clustering begins. Extensive experiments demonstrate that SPARC achieves state-of-the-art performance on the DALES dataset with a Panoptic Quality of 62.4%, surpassing existing methods by 9.8% and effectively resolving multi-scale segmentation ambiguities.