Large-scale continual road inspection: Visual infrastructure assessment in the wild

This work develops a method to inspect the quality of pavement conditions based on images captured from moving vehicles. This task is challenging because the appearance of road surfaces varies tremendously, depending on the construction materials (e.g., concrete, asphalt), the weather conditions (e.g., rain, snow), the illumination conditions (e.g., sunny, shadow), and the interference of other structures (e.g., manholes, road marks). This problem is amplified by the lack of a sufficiently large and diverse dataset for training a pavement classifier. Our first contribution in this paper is the development of a method to create a large-scale dataset of pavement images. Specifically, using map and GPS information, we match the ratings by government inspectors found in public databases to Google Street View images, creating a dataset containing more than 700K images from 70K street segments. We use the dataset to develop a deep-learning method for road assessment, which is based on Convolutional Neural Networks, Fisher Vector encoding, and UnderBagging random forests. This method achieves an accuracy of 58.2% and significantly outperforms various other texture classification methods.