Homomorphic Encryption Schemes for ML Inference Comparison

Homomorphic encryption lets you run a machine learning model against data it never sees in the clear. A hospital keeps patient records encrypted, a fintech company keeps its fraud model proprietary, and inference happens anyway, with neither side handing over what it wants kept private. But "use FHE" isn't one decision. It's a stack of them, and the scheme you pick first decides what math you're allowed to do, how much of it survives before the ciphertext falls apart, and what your compute bill looks like at the end of the month.

Homomorphic computation runs four to five orders of magnitude slower than plaintext, and it burns five to six orders of magnitude more energy doing it. That gap isn't spread evenly across the board. Some schemes close part of it for specific tasks and blow it wide open for others, and sorting out which is which is most of the real work in evaluating FHE for a product you plan to ship, as opposed to a paper you plan to publish.

How the four major scheme families handle noise differently, and why that determines what they can compute

Every LWE-based FHE scheme buries your plaintext under noise on purpose. That noise is what makes the encryption secure, but it also grows with every homomorphic operation, especially multiplication, until it's big enough that decryption fails outright and you get garbage back instead of an answer. So underneath the marketing, the whole design space of FHE is really a design space of noise management. Multiplicative depth, meaning how many multiplications in a row a ciphertext can survive, is the ceiling that noise puts on your computation.

There are two ways to deal with it. Parameter management fixes your modulus and ring dimension before you start, which means budgeting a depth up front with no way to rescue a computation that runs past it. Bootstrapping refreshes the noise budget mid-computation instead, at the cost of being one of the most expensive operations in the entire FHE toolkit. Depth versus bootstrapping cost is the tension sitting under every scheme in this piece, and it shows up again and again, wearing a slightly different outfit each time.

The four families also split on how they represent numbers. BGV and BFV do exact integer arithmetic with modulus-based noise control. CKKS does approximate real arithmetic using a scaling factor instead of exact modular reduction. TFHE, also called CGGI after its authors, works bit by bit and bootstraps after nearly every gate. Before you write a line of code, these differences have already decided whether your ML workload fits the scheme at all.

BGV and BFV: exact integer arithmetic, where they fit, and why ML inference usually isn't it

BGV and BFV are cousins, both built on ring-learning-with-errors, both working over integers with modular arithmetic. The textbook difference is how they handle relinearization during multiplication; in practice, with modern library optimization at moderate problem sizes, that difference mostly washes out. BGV tends to pull ahead at very large and very small scales, and BFV is easier to pick up if you're building your first pipeline. Choosing between them is more an ergonomics question than a capability one.

Where these two actually earn their keep is private information retrieval, private set intersection, encrypted database queries: workloads built on large vectors of small integers where exactness is the entire point. Neural network inference is a different animal. Weights and activations in a trained model are real-valued, and BGV and BFV only speak integers. You can quantize weights into fixed-point representations to force them through the door, but that pushes complexity onto the model and costs accuracy on the way in. At that point you're bending the workload to fit the scheme, which is backwards.

BGV and BFV still show up constantly in hybrid pipelines, doing feature encoding or integer-valued preprocessing that sits next to the inference layer. That's a real, useful role. It's just a different one from running the inference step itself.

CKKS: why approximate arithmetic made floating-point ML inference over encrypted data possible

CKKS, introduced by Cheon, Kim, Kim, and Song in 2017, made a bet that turned out right for machine learning. If your application already tolerates numerical imprecision, and every neural network does, then a scheme that bakes controlled approximation error into the ciphertext itself stops being a weakness and becomes the whole point.

Mechanically, CKKS encodes floating-point values as large integers scaled by a factor that governs precision. That factor gets used up with each multiplication, which is why depth still matters even though the arithmetic underneath is approximate. The other piece that makes CKKS the ML workhorse is SIMD batching: it packs many values into one ciphertext and runs the same operation on all of them at once, which lines up almost eerily well with how a neural network layer wants to be computed anyway. Thousands of weights, one operation. CKKS is, as of now, the only mainstream FHE scheme offering both floating-point arithmetic and this kind of parallel batching together, and that combination is what lets standard ML operations map onto encrypted computation without redesigning the model.

The catch is activation functions. FHE natively gives you addition and multiplication, full stop. ReLU has a conditional branch built into it, and branches don't exist in this world. The workaround is polynomial approximation: fit a polynomial to the shape of ReLU and evaluate that instead. Higher-degree polynomials get you closer to the real function, but they also add multiplicative depth, which means more bootstrapping, which means more latency. Lee et al. pushed this to 91.31% accuracy on CIFAR-10 using a polynomial ReLU approximation, with inference taking 2,271 seconds. That's the depth-latency tradeoff made concrete instead of abstract. Compare that to LeNet-5, where a simpler square activation costs only 0.2% accuracy loss against plaintext. Sometimes approximation is nearly free, sometimes it's brutal, and which one you get depends on the architecture in a way that resists a clean rule of thumb.

There's a memory story here too, and it doesn't get enough airtime outside engineering teams who've actually shipped this stuff. CKKS multiplicative keys range from 2.6 MB at shallow depth up to 2.6 GB at depth 150; rotation keys run from 10.5 MB up to 10 GB across that same range. Those aren't numbers you wave away when you're thinking about deployment footprint, client-server bandwidth, or where the keys physically have to live.

Under all of it sits the bootstrapping bottleneck. CKKS bootstrapping is expensive, deep networks chew through their multiplicative budget fast, and that forces repeated bootstrapping calls. This is exactly where CKKS's edge over TFHE starts to thin.

TFHE: what gate-by-gate bootstrapping gains and what it costs on ML workloads

TFHE, also known as CGGI, takes the opposite approach. It works on individual bits or small integers and bootstraps after nearly every gate, which sounds expensive per operation, and it is. But TFHE's bootstrapping does something CKKS's version can't: programmable bootstrapping, or PBS, evaluates any non-linear univariate function in a single bootstrapping step. No polynomial approximation, no depth budget to haggle over. You define the function, bootstrap it in, and you're done.

The memory profile favors TFHE too. In OpenFHE, a TFHE public key sits at a steady 113 MB, the secret key is 4 KB, and the bootstrapping key lands around 25 MB for 128-bit security. Set that against CKKS's multi-gigabyte keys at depth and it's obvious why TFHE gets picked for anything resource-constrained.

So where does TFHE actually beat CKKS on real operations? Comparison, and pretty much only comparison. A 2025 benchmark from Krüger, Moriya, and Schoop (IACR ePrint 2025/1460) tested both schemes on 2^16 32-bit integers, and comparison was the one clear TFHE win, because comparing two numbers in CKKS needs multiple bootstrapping rounds thanks to the multiplicative depth involved. Everywhere else in that same benchmark, addition, multiplication, polynomial evaluation, square root, CKKS came out ahead. TFHE's bit-level representation doesn't map cleanly onto floating-point weights and activations, and forcing a modern neural net into bit-wise encoding is cumbersome enough to cap how deep a network you can realistically run.

TFHE's real home turf is logic-heavy or binarized computation, edge devices where a compact key matters more than raw floating-point throughput, and comparison-dominated pipelines. As a general-purpose engine for deep neural network inference, it's the weaker of the two, at least among people who've actually benchmarked both side by side.

How the 2025 CKKS vs. TFHE benchmark clarifies when each scheme is the better choice

The Krüger, Moriya, and Schoop paper matters because it's the first controlled, direct comparison of CKKS and TFHE across the operations that actually show up in ML inference: addition, multiplication, division, square root, polynomial evaluation, comparison, all run against a shared dataset of 2^16 32-bit integers.

What falls out is a fairly clean pattern. CKKS wins most operations, especially the ones that parallelize across many values at once, which is basically what an ML inference pipeline is made of end to end. TFHE's one decisive win is comparison, for the bootstrapping-depth reason covered above. CKKS's lead isn't fixed, though. It narrows as depth requirements climb and bootstrapping fires more often, which is exactly the situation deep networks create for it.

Reach for CKKS when the workload parallelizes well and comparisons are rare. Reach for TFHE, or a hybrid, when comparison-heavy branching or non-linear functions that resist polynomial approximation sit at the center of the pipeline. What the benchmark doesn't tell you is how these operations behave once you chain them into a real inference graph, because networks trigger bootstrapping in patterns that depend on architecture, not just on the operation type taken on its own. The benchmark hands you the building blocks. The specific network still needs its own analysis, and that part doesn't come off the shelf.

GPU acceleration and hardware-aware optimization as the practical path to reducing CKKS inference latency

CKKS is structurally parallel by design, which makes it an unusually good fit for GPU acceleration, better than most cryptographic primitives manage. The Chameleon GPU implementation reports homomorphic addition speedups of 110.7x over Microsoft SEAL, 192.7x over OpenFHE, and 19.7x over HElib; multiplication speedups land at 115.9x, 76.6x, and 73.1x against those same three libraries. These aren't small gains around the edges. They're the difference between a workload being impractical and a workload shipping to real users.

Purpose-built silicon pushes further still. The LP-HENN accelerator, built specifically for FHE neural network inference, reports a 31.82x performance improvement and an extraordinary energy efficiency improvement for HE-CNN inference against a CPU baseline. Numbers like that should make anyone skeptical of a scheme comparison that only looks at CPU benchmarks, because the "practical" scheme for a given latency target can flip entirely once you factor in the hardware you're actually deploying on.

The toolchain matters here too, maybe more than it gets credit for, since that's where a lot of the real engineering hours go. CKKS is backed by a genuinely mature set of libraries: Microsoft SEAL, TenSEAL, HE Transformer, HEAAN, with a correspondingly deeper GPU-acceleration story behind them. TFHE libraries like Concrete, TFHE-rs, and cuFHE are solid, but they're built with edge and embedded targets in mind, not large-scale GPU throughput. Even inside CKKS, the bottleneck isn't always where you'd guess: research on secure transformer inference (the THOR work) found that key switching, not the matrix multiplication itself, dominates latency in transformer workloads. That argues for designing plaintext packing and matrix multiplication together, rather than just throwing more GPU parallelism at the problem and hoping the bottleneck moves.

Hybrid CKKS+TFHE schemes and client-offload architectures as engineering responses to the remaining bottlenecks

Once it sinks in that CKKS handles linear layers well and non-linear functions poorly, and TFHE is the mirror image of that, one question shows up on its own: why not use both? That's the premise behind hybrid CKKS-TFHE pipelines. CKKS takes the matrix multiplications and convolutions, the linear layers where its parallelism and floating-point support carry the load. TFHE's programmable bootstrapping steps in for activation functions, where CKKS would otherwise need an expensive polynomial approximation. Switching a ciphertext between the two representations costs something real, though, and it only pays off when the bootstrapping you avoid in CKKS outweighs the switching overhead you take on to get there. That calculation depends entirely on the workload in front of you. It doesn't generalize into a universal win.

A more radical answer showed up in 2025 with Safhire: skip approximating non-linear functions on the server altogether, and hand them off to the client, who already holds the secret key anyway. Linear layers stay encrypted and run server-side; model confidentiality survives through shuffling, so the client can't reconstruct the proprietary model from what it sees. This dodges the bootstrapping cost for non-linear layers entirely and sidesteps the accuracy hit from polynomial approximation. The tradeoff is real, though: this only works when the client is actually present and taking part during inference, which rules it out for pure server-side batch processing, and it adds round-trip communication overhead between client and server. Even so, benchmarks on standard CNNs show Safhire beating Orion, a leading pure-FHE inference system, on end-to-end latency by a wide margin.

There's also work happening entirely inside CKKS's own toolbox, no hybrid needed. Low-complexity multiplexed parallel convolutions achieved a 4.67x reduction in inference latency and a 134x improvement in amortized runtime for ResNet-20 at 128-bit security, no scheme switching or client offload involved at all. So "pick the right scheme" and "optimize inside the scheme" are both live questions at once, and answering one doesn't close off the other.

Where does this leave things? Pointed at active research rather than a settled answer, and I'd guess the question practitioners ask in a couple years shifts from "CKKS or TFHE" to "which combination, and where does the handoff boundary sit."

Matching workload characteristics to scheme choice: a decision framework

The decision isn't really about picking a scheme in the abstract. It's about matching a scheme to your workload's arithmetic profile: how deep it runs, how linear it is, where it deploys, how much approximation error you can absorb before the result stops being useful.

For real-valued deep networks, CNNs, transformers, MLPs, with activations that tolerate polynomial approximation reasonably well, CKKS is the sensible default. Engineering time belongs on GPU acceleration and smart packing strategy, not on relitigating the scheme choice itself. For networks built around heavy comparison logic, branching activations, or binarized weights, TFHE or a hybrid CKKS-TFHE setup makes more sense; you trade floating-point convenience for programmable bootstrapping's expressiveness. Integer-valued preprocessing, feature encoding, database-style lookups sitting next to the inference pipeline belong on BGV or BFV. Forcing them into the inference layer just because it's already sitting there is a mistake I've watched teams make more than once, usually under deadline pressure that makes the easy path look like the right one.

For very deep networks where CKKS bootstrapping starts eating the latency budget, that's the signal to look at hybrid scheme-switching or a client-offload architecture like Safhire, rather than squeezing for more depth inside CKKS alone. For edge or resource-constrained deployment, TFHE's compact footprint (113 MB for the public key, roughly 25 MB for the bootstrapping key) makes it workable in places where CKKS's multi-gigabyte keys at depth simply won't fit on the device.

One thing that doesn't fit neatly into any framework: if the application genuinely cannot tolerate approximation error, an exact financial audit trail is the obvious case, CKKS is the wrong tool no matter how fast it runs. Speed doesn't override a correctness requirement. The library ecosystem is its own quiet factor in all this too. CKKS's GPU stack is more mature today, TFHE's libraries lean toward edge optimization, and that asymmetry shapes what's buildable on a given timeline nearly as much as the underlying math does.

The four-to-five-order-of-magnitude overhead against plaintext, the number this piece opened with, sticks around through every one of these choices. GPU acceleration, hybrid architectures, client offload: each chips away at the overhead in a different place, for a different workload, and none of them erase it outright. Anyone deploying FHE for ML inference is really choosing where on that overhead curve they're willing to sit. That's the decision underneath all the scheme names, and it's worth sitting with rather than rushing past.