Effectiveness of Image Augmentation Techniques on Detection of Building Characteristics from Street View Images Using Deep Learning

Abstract

Two key building characteristics, namely the number of stories and typology, is vital across various domains such as construction management and architectural design. These aspects are particularly critical for disaster risk assessment and infrastructure planning. Although deep learning models are adept at extracting this information from Street view images (SVIs), their success is contingent upon the availability of large and diverse data sets with high accuracy. Image augmentation presents an alternative method to artificially broaden data set diversity. However, the impact of image augmentation techniques on identifying building stories and typologies from SVIs has not been adequately explored. This study proposes a methodology employing eight distinct image augmentation techniques—brightness, contrast, perspective, rotation, scale, shearing, and translation augmentations—as well as a combined approach using all these methods. The study evaluates the efficacy of models trained with these techniques by comparing the accuracy of different classes and architectures for each task, both with and without the application of augmentation. The findings revealed that while most augmentation methods enhance model accuracy, their effectiveness is task-dependent. Furthermore, it was observed that the most effective augmentation techniques differ among building classes and architectures within each task. This suggests that augmentation strategies need to be custom-designed to align with the unique features of each class and architectures for precise estimation of the number of stories and building typologies. While the focus of this research is on specific tasks, the evaluated augmentation techniques could also extend to related areas, such as ascertaining the age of buildings or identifying window types. In this study, the efficacy of augmentation techniques is explored within the framework of identifying the number of stories and building typologies. The models were assessed for average accuracy and class-specific accuracy across various architectures, comparing outcomes with and without the implementation of the proposed augmentation methods. A key finding is that the most effective augmentation method varies between architectures and individual classes. Contrary to common practice in deep learning, where applying multiple augmentation techniques is standard for accuracy enhancement, this study observed that such a strategy did not uniformly improve performance. Specifically, while combining augmentation methods generally resulted in higher average accuracy, this was not the case for some classes within MobileNetV3 when detecting the number of stories. Similarly, for ResNet-152, employing all augmentation techniques together led to the lowest accuracy in certain classes for building typology classification. These results indicate that augmentation strategies may require customization to cater to the distinct characteristics of each class and architecture for accurate estimation of number of stories and building typologies.