Google's HEIR Compiler Aims for Private Machine Learning
AI-generated image · US National Wire
A new compiler allows programs to operate on encrypted data, enabling private inference for ML models.
Google has developed HEIR, a homomorphic encryption (HE) compiler designed to convert input programs into versions that operate directly on encrypted data, as first reported by Jeremy Kun via Hacker News. According to Kun, the technology ensures that the computer running the program never accesses a single bit of cleartext information regarding the inputs, outputs, or intermediate values.
Reporting from Kun's article highlights HEIR's capability to compile pre-trained machine learning models to provide private inference. In a demonstration using a credit card fraud detector—a three-layer feed-forward network—the HE inference took approximately 2 seconds on a single-threaded CPU, compared to 0.5 milliseconds for cleartext execution. While this represents a 4,000x slowdown, Kun notes that execution times are improving.
Other examples compiled with HEIR show varying latencies on a single-threaded CPU: a network anomaly ensemble takes 30 seconds, a Criteo recommender model takes 5 minutes, and a 10-layer convolutional network for hotword detection takes 20 minutes. However, Kun reports that integrating HEIR with GPUs has significantly reduced these times; specifically, the Criteo workload ran in approximately 500ms on a single GPU, such as an H100, reducing the slowdown to roughly 50x compared to cleartext.