Automatic non-parametric image parsing via hierarchical semantic voting based on sparse–dense reconstruction and spatial–contextual cues

Image parsing is vital for many high-level image understanding tasks. Although both parametric and non-parametric approaches have achieved remarkable success, many technical challenges still prevail for images containing things/objects with broad-coverage and high-variability, because it still lacks versatile and effective strategies to seamlessly integrate local–global features selection, contextual cues exploitation, spatial layout encoding, data-driven coherency exploration, and flexible accommodation of newly annotated labels. To ameliorate, this paper develops a novel automatic non-parametric image parsing method with advantages of both parametric and non-parametric methodologies by resorting to new modeling and inferring strategies. The originality of our new approach is to employ sparse–dense reconstruction as a latent learning model to conduct candidate-label probability analysis over multi-level local regions, and synchronously leverage context-specific local–global label confidence propagation and global semantic spatial–contextual cues to guide holistic scene parsing. Towards this goal, we devise several novel technical components to comprise a lightweight parsing framework, including local region representation integrating complementary features, anisotropic consistency propagation based on bi-harmonic distance metric, bottom-up label voting, semantic string generation of image-level spatial–contextual cues based on Hilbert space-filling curve, and co-occurrence priors analysis based on relaxed string matching algorithm, which collectively enable us to effectively combat the aforementioned obstinate problems. Moreover, we conduct comprehensive experiments on public benchmarks, and make extensive and quantitative evaluations with state-of-the-art methods, which demonstrate the advantages of our method in accuracy, versatility, flexibility, and efficiency.