Enhancing Data Security in Satellite Communication Systems: Integrating Quantum Cryptography with CatBoost Machine Learning

Mohd Nadeem, Syed Anas Ansar, Sakshi Halwai, Arpita Singh, Rajeev Kumar

In modern communication networks, particularly satellite-based systems, data security faces significant challenges from vulnerabilities such as signal interception, jamming, and latency during long distance transmissions. Traditional cryptographic methods are increasingly vulnerable to quantum computing threats, underscoring the need for advanced solutions to protect data integrity, confidentiality, and availability. This research investigates the fusion of quantum cryptography and Machine Learning (ML) to improve security in satellite communication. The Quantum Key Distribution (QKD), which is grounded in quantum mechanics, enables unbreakable encryption by detecting eavesdropping via quantum state disturbances. The CatBoost ML algorithm is applied to a dataset of 10,000 records featuring categorical attributes for prioritizing security elements such as anomaly detection, encryption types, and access controls. The model yields an accuracy of 89.23% and Area under Curve the Receiver Operating Characteristic (AUC-ROC) score of 94.56%, effectively predicting threat levels. Feature importance reveals anomaly detection (28.5%) and quantum encryption (22.3%) as primary contributors. While hurdles such as high implementation costs and transmission range limitations persist, this quantum ML synergy provides a proactive, adaptive framework for resilient, future-ready communication networks.