Multi-Modal Deepfake Detection System Using Hybrid Deep Learning on Visual and Audio Features
Abstract: Deepfake technology, driven by generative models such as GANs and diffusion architectures, has enabled the creation of highly realistic manipulated media capable of deceiving both visual and auditory perception. Such forgeries pose significant risks to identity verification, digital forensics, and public trust. Existing detection methods are often restricted to single modalities or handcrafted features, limiting their robustness against high-quality cross-modal deepfakes. This work presents a multi-modal deepfake detection framework that integrates visual and audio analysis. The system employs a ResNet-based backbone for spatial feature extraction, Bi-LSTM encoders for temporal modeling in videos, and a 2D CNN with Random Forest classification for speech-based anomaly detection. An attention-based late-fusion strategy combines outputs to generate reliable predictions with calibrated confidence scores. Experimental results demonstrate high performance, with training accuracy reaching 98% and validation accuracy of 92% on benchmark datasets. The system is deployed as a Flask-based web application, enabling real-time detection and interpretable outputs.