A Hybrid Game Engine–Generative AI Framework for Overcoming Data Scarcity in Open-Pit Crack Detection

Rohan Le Roux, Siavash Khaksar, Mohammadali Sepehri, Iain Murray

Open-pit mining operations rely heavily on visual inspection to identify indicators of slope instability such as surface cracks. Early identification of these geotechnical hazards enables timely safety interventions to protect both workers and assets in the event of slope failures or landslides. While computer vision (CV) approaches offer a promising avenue for autonomous crack detection, their effectiveness remains constrained by the scarcity of labelled geotechnical datasets. Deep learning (DL)-based models, in particular, require large amounts of representative training data to generalize to unseen conditions; however, collecting such data from operational mine sites is limited by safety, cost, and data confidentiality constraints. To address this challenge, this study proposes a novel hybrid game engine–generative artificial intelligence (AI) framework for large-scale dataset generation without requiring real-world training data. Leveraging a parameterized virtual environment developed in Unreal Engine 5 (UE5), the framework generates realistic images of open-pit surface cracks and enhances their fidelity and diversity using StyleGAN2-ADA. The synthesized datasets were used to train the YOLOv11 real-time object detection model and evaluated on a held-out real-world dataset of open-pit slope imagery to assess the effectiveness of the proposed framework in improving model generalizability under extreme data scarcity. Experimental results demonstrated that models trained using the proposed framework consistently outperformed the UE5 baseline, with average precision (AP) at intersection over union (IoU) thresholds of 0.5 and [0.5:0.95] increasing from 0.792 to 0.922 (+16.4%) and 0.536 to 0.722 (+34.7%), respectively, across the best-performing configurations. These findings demonstrate the effectiveness of hybrid generative AI frameworks in mitigating data scarcity in CV applications and supporting the development of scalable automated slope monitoring systems for improved worker safety and operational efficiency in open-pit mining.