AI-Driven Detection, Segmentation, and Classification of Medicinal Plant Leaves Using Computer Vision and Deep Learning

Mahalakshmi B, Laxmi B Rananavare

Ayurvedic medicines play a significant role in maintaining the physical and mental health of people. Currently, these medicines are widely adopted to cure various diseases. Therefore, precise detection and classification of medicinal crops are critical for ensuring effective treatments. However, the scarcity of experts in this field poses a challenge, making the process of detecting and classifying herbal plants both complex and time-consuming. To address this issue, a fully automatic system for medicinal plant classification is much required. Currently, Computer Vision (CV) and Machine Learning (ML)/Deep Learning (DL) based systems have gained attention to provide automated detection and classification solutions for these types of problems i.e. detection, segmentation and classification. Therefore, this study adopts this concept and proposes a comprehensive framework for medicinal plant leaf classification, integrating state-of-the-art DL and feature extraction techniques. Initially, data augmentation methods such as rotation, flipping, scaling, and color jittering are applied to enhance dataset diversity and improve model generalization. In next stage, image segmentation task is applied by utilizing advanced deep learning Yolov8 mode which helps to accurately extract leaf regions from complex backgrounds. The segmented images are then analysed to extract features using pre-trained VGG16 deep learning models, which are further enriched with handcrafted texture, color, and shape features to ensure robust representation. These diverse feature sets are fused and classified using an ensemble of ML classifiers, including Random Forest (RF), Support Vector Machine (SVM), and Gradient Boosting (GB). The proposed system achieves classification accuracies of 87.65%, 83.20%, and 89.32% for RF, SVM, and GB, respectively. Experimental results demonstrate the system's effectiveness in achieving good accuracy and reliability in medicinal plant leaf classification. This approach provides a significant step toward automating plant identification, promoting the efficient use of Ayurvedic medicinal resources, and reducing dependency on domain experts.