FHEON: A Configurable Framework for Developing Privacy-Preserving Encrypted Neural Networks

Nges Brian Njungle, Eric Jahns, Michel A. Kinsy

TL;DR: FHEON is presented, an open-source configurable framework for developing privacy-preserving neural network models for inference using the CKKS scheme of HE, and outperform all state-of-the-art HE inference works in both latency and memory utilization.

The widespread adoption of Machine Learning as a Service raises critical privacy and security concerns, particularly about data confidentiality and trust in both cloud providers and the machine learning models provided. Homomorphic Encryption (HE) has emerged as a promising solution to these problems, allowing computations on encrypted data without decryption. Despite its potential, existing works that integrate HE into neural networks are often limited to specific architectures or classes. This leaves a wide gap in providing a framework for easy development of HE-friendly privacy-preserving neural network models similar to what we have in the broader field of machine learning. In this paper, we present FHEON, an open-source configurable framework for developing privacy-preserving neural network models for inference using the CKKS scheme of HE. FHEON introduces optimized and configurable implementations of privacy-preserving neural network layers, including convolution layers, average pooling layers, ReLU activation functions, and fully connected layers. These layers are configured using standard parameters such as input channels, output channels, kernel size, stride, and padding to support arbitrary convolution neural network (CNN) architectures. Furthermore, FHEON provides utility functions that ease usage and adoption. We assess the performance of FHEON using several CNN architectures, including LeNet-5, VGG-11, VGG-16, ResNet-20, and ResNet-34. FHEON maintains encrypted-domain accuracies within +-1% of their plaintext counterparts for ResNet-20 and LeNet-5 models. Notably, on a consumer-grade CPU, the models built on FHEON achieved 98.5% accuracy with a latency of 13 seconds on MNIST using LeNet-5, and 92.2% accuracy with a latency of 403 seconds on CIFAR-10 using ResNet-20. Though configurable, FHEON outperform all state-of the-art HE inference works in both latency and memory utilization. Additionally, FHEON operates within a practical memory budget requiring not more than 42.3 GB for VGG-16.