Quantum Computing Fundamentals for Data Engineers

Quantum computing occupies an awkward position in data engineering discourse. It is simultaneously over-hyped — vendors selling “quantum-ready data platforms” to people whose actual bottleneck is a badly partitioned Parquet table — and under-appreciated, because the one concrete way it will affect data infrastructure this decade has nothing to do with running quantum algorithms at all.

This post tries to be useful about both. It covers the fundamentals honestly (what a qubit is, what the famous algorithms actually do), then spends most of its length on the questions a working data engineer would actually ask: does this speed up my pipeline? Does this help my models? What do I need to change, and when?

The short answers, up front:

• Will quantum computers speed up your ETL? No, and there is a structural reason why, not just an engineering-maturity reason. Getting classical data into a quantum machine costs at least as much as processing it classically, which eats the speedup for exactly the workloads data engineers care about.
• Will they speed up your ML? Almost certainly not the way the 2019-era papers implied. Several of the headline “exponential speedups” have been matched by classical algorithms under the same assumptions.
• Is there anything you should actually do? Yes — migrate your cryptography. That work has fixed deadlines, real compliance consequences, and is entirely classical engineering. It is the section to read if you read only one.

Where the hardware actually is

One result reframed the whole field, and it is worth starting there.

Google’s Willow (December 2024) is a 105-qubit superconducting chip, but the qubit count is not the point. Willow was the first public demonstration of below-threshold error correction: adding more physical qubits to a logical qubit made the logical error rate go down. Before that, error correction was theoretically sound but empirically unproven at scale — every additional qubit added more noise than it suppressed. Willow showed the curve bends the right way. Google followed it with a verifiable-advantage result in 2025 and, in July 2026, reinforcement-learning-based calibration that improved logical stability by around 3.5x and pushed surface-code memory errors below one per thousand cycles.

IBM took a different tack: less single-milestone, more published engineering roadmap with dated deliverables. Nighthawk (120 qubits) targets demonstrable quantum advantage on a practical problem by the end of 2026, and the Loon chip is validating the fault-tolerant architecture intended to underpin Starling in 2029. IBM’s roadmap is the most explicit in the industry, which also makes it the easiest to hold to account.

Elsewhere: IonQ reported 99.9923% two-qubit gate fidelity (“four nines”) in October 2025. Quantinuum’s H-series crossed several logical-qubit thresholds, in part through its collaboration with Microsoft. Microsoft announced Majorana 1 in February 2025, a topological qubit approach that would dramatically cut the physical-to-logical overhead if it scales — the “if” is doing real work in that sentence, and the result remains contested. Neutral-atom platforms (Atom Computing, QuEra) demonstrated systems with over a thousand qubits.

The reality check that goes with all of it. Error correction overhead is still the dominant constraint. A distance-7 surface code spends 49 physical qubits to produce one better logical qubit. Running Shor’s algorithm against RSA-2048 needs millions of physical qubits. None of the above moved the consensus central estimate for “Q-Day” — when a cryptographically relevant quantum computer exists — which sits around 2033-2035.

The honest summary is that this is not a “quantum computers now work” story. It is a “the engineering is now serious and the path is validated” story. That distinction matters enormously for how you plan.

Quantum Computing vs. Classical Computing

In classical computing, the bit is the fundamental unit of information, existing in one of two states: $0$ or $1$. Operations on bits are governed by classical logic gates, which perform deterministic computations.

Quantum computing leverages the principles of quantum mechanics. Its fundamental unit, the quantum bit or qubit, can exist in a superposition of states, effectively representing both $0$ and $1$ simultaneously:

where $\alpha$ and $\beta$ are complex probability amplitudes such that $| \alpha |^2 + | \beta |^2 = 1$.

Key Differences:

• Data Representation: An $n$-qubit register holds a superposition over all $2^n$ basis states, described by $2^n$ complex amplitudes.
• Computational Power: A unitary applied to that register acts on all $2^n$ amplitudes at once. This is the origin of the “quantum parallelism” claim — with an important caveat below.
• Entanglement: Qubits can be correlated in ways that have no classical analogue, so the joint state cannot be written as a product of individual qubit states.
• Algorithmic Approach: Quantum algorithms exploit interference between amplitudes to make wrong answers cancel and right answers reinforce, which is how a speedup is actually obtained.

Core Quantum Concepts

• A qubit is the quantum counterpart of a classical bit. It is represented by a vector in a two-dimensional Hilbert space. Physically, qubits can be realized using various quantum systems, including electron spins, photon polarization, and atomic energy levels.

  A qubit’s state is generally expressed as:

  \[\begin{aligned} |\psi\rangle = \cos\left(\frac{\theta}{2}\right)|0\rangle + e^{i\phi} \sin\left(\frac{\theta}{2}\right)|1\rangle, \end{aligned}\]

  where $ \theta $ and $ \phi $ define the qubit’s position on the Bloch sphere.

• Superposition allows qubits to exist in a combination of $ |0⟩ $ and $ |1⟩ $ states simultaneously, enabling quantum computers to explore multiple solutions at once.

  Example:

  • A qubit in superposition:
  \[\begin{aligned} |\psi\rangle = \frac{1}{\sqrt{2}}(|0\rangle + |1\rangle). \end{aligned}\]
  • Upon measurement, the state collapses to either $ |0⟩ $ or $ |1⟩ $ with equal probability.

• Entanglement is a quantum phenomenon in which the state of one qubit is directly related to the state of another, even over large distances.

  Example:

  • A Bell state, representing two entangled qubits:
  \[\begin{aligned} |\Phi^+\rangle = \frac{1}{\sqrt{2}}(|00\rangle + |11\rangle). \end{aligned}\]
  • Measuring one qubit instantly determines the state of the other.

Quantum Gates and Circuits

Quantum gates manipulate qubits by applying unitary operations, altering their states and enabling complex quantum computations. Quantum circuits are composed of these gates arranged in a specific sequence, performing transformations on quantum information. Here are some common quantum gates:

• Pauli-X Gate (NOT Gate): Flips the state of a qubit. If the qubit is in state $|0 \rangle$, applying $ X $ gate changes it to $ | 1 \rangle $, and vice versa.

  \[\begin{aligned} X = \begin{bmatrix} 0 & 1 \\ 1 & 0 \\ \end{bmatrix} \end{aligned}\]

• Hadamard Gate (H Gate): Places a qubit in an equal superposition of $|0\rangle$ and $|1\rangle $. This is essential for creating quantum states that represent multiple possibilities simultaneously.

  \[\begin{aligned} H = \frac{1}{\sqrt{2}}\begin{bmatrix} 1 & 1 \\ 1 & -1 \\ \end{bmatrix} \end{aligned}\]

• Controlled-NOT (CNOT) Gate: A two-qubit gate that flips the target qubit if the control qubit is $ |1\rangle $. This gate is crucial for creating entanglement between qubits.

  \[\begin{aligned} \text{CNOT} = \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 \\ 0 & 0 & 1 & 0 \\ \end{bmatrix} \end{aligned}\]

• Phase Gate (S Gate): Adds a phase shift of $\pi/2$ to the $|1\rangle$ component of a qubit’s state. This is a foundational gate for many quantum algorithms involving phase manipulation.

  \[\begin{aligned} S = \begin{bmatrix} 1 & 0 \\ 0 & i \\ \end{bmatrix} \end{aligned}\]

• T Gate (π/4 Phase Shift): Adds a phase of $\pi/4$ to the $|1\rangle$ state. It’s useful for precise phase control and often appears in error correction.

  \[\begin{aligned} T = \begin{bmatrix} 1 & 0 \\ 0 & e^{i\pi/4} \\ \end{bmatrix} \end{aligned}\]

The diagram below represents a simple quantum circuit with the following sequence:

1. Hadamard Gate on the first qubit $ q_0 $: This gate places the qubit in a superposition of $ |0\rangle $ and $ |1\rangle $, enabling the representation of multiple states simultaneously.
2. CNOT Gate between $ q_0 $ (control) and $ q_1 $ (target): This entangles the two qubits. If $ q_0 $ is in the state $ |1\rangle $, the CNOT gate flips the state of $ q_1 $; otherwise, it leaves $ q_1 $ unchanged.
3. Measurement on $ q_0 $: Measures the state of $ q_0 $, collapsing the superposition to a definite state, either $ |0\rangle $ or $ |1\rangle $.

In this circuit, the combination of the Hadamard and CNOT gates creates an entangled state between $ q_0 $ and $ q_1 $. This setup is foundational for building more complex quantum algorithms, as entangled states are a key resource in quantum computing.

Quantum Algorithms in Data

Quantum Fourier Transform

The Quantum Fourier Transform (QFT) is the quantum analogue of the discrete Fourier transform, essential in quantum algorithms involving periodicity and phase estimation. The QFT maps an input state into the frequency domain, making it valuable for analyzing periodicity in quantum states and for applications in algorithms like Shor’s algorithm for factorization.

For an $ n $-qubit input state $ |x\rangle $, the QFT is defined as:

where $ |x\rangle $ is an integer in the range $ 0 \leq x < 2^n $.

The QFT is constructed by applying a series of Hadamard gates and controlled phase rotations. For each qubit $ q_i $ (where $ i = 0, 1, \ldots, n-1 $), we perform a Hadamard transform followed by controlled rotations with angles depending on the separation of the qubits. This process creates a superposition that represents the Fourier transform of the input state.

For the $i$-th qubit in an $ n $-qubit system, the transformation steps are as follows:

1. Apply the Hadamard gate:

   \[H |q_i\rangle = \frac{1}{\sqrt{2}} \left(|0\rangle + |1\rangle\right).\]

2. Follow with controlled rotations. For each pair $(i, j)$, where $ j > i $, apply a controlled-$R_k$ gate with rotation angle $ 2\pi / 2^{j-i+1} $:

   \[R_k = \begin{bmatrix} 1 & 0 \\ 0 & e^{2\pi i / 2^k} \end{bmatrix}.\]

   For instance, on the first qubit, this becomes:

   \[H \cdot R_{k} = \frac{1}{\sqrt{2}} \left(|0\rangle + e^{2\pi i/2^k} |1\rangle \right),\]

   where each controlled rotation $ R_k $ introduces a phase shift, creating interference patterns that encode the frequency information.

3. After applying all necessary gates, a swap gate sequence reorders the qubits to finalize the QFT computation in the correct order.

Example QFT Circuit for 3 Qubits

In a 3-qubit QFT circuit, the process involves:

1. Hadamard Gate on Qubit 0: Begin by placing qubit 0 in a superposition:

   \[H|q_0\rangle = \frac{1}{\sqrt{2}} \left(|0\rangle + |1\rangle\right).\]

2. Controlled Rotations on Qubit 0: Apply controlled-$R_2$ and controlled-$R_3$ gates between qubit 0 and the subsequent qubits, introducing phases:

   • Apply $R_2$ between qubit 0 and qubit 1.
   • Apply $R_3$ between qubit 0 and qubit 2.

3. Hadamard Gate on Qubit 1: Apply a Hadamard gate to qubit 1 to further entangle it with the other qubits:

   \[H|q_1\rangle = \frac{1}{\sqrt{2}} \left(|0\rangle + |1\rangle\right).\]

4. Controlled Rotation on Qubit 1: Apply a controlled-$R_2$ gate between qubit 1 and qubit 2.

5. Final Hadamard Gate on Qubit 2: Apply a Hadamard gate to qubit 2.

6. Swap Operation: To finalize, perform a swap between qubits 0 and 2 to ensure the qubits are in the correct order for measurement.

The resulting QFT circuit efficiently transforms the input state into the frequency domain, ready for analysis or further algorithmic steps.

Applications of the QFT include:

• Signal Processing: The QFT accelerates Fourier transforms on large datasets, useful in applications requiring quantum frequency analysis, such as in signal processing within quantum data states.
• Period Finding and Shor’s Algorithm: The QFT is critical in period-finding algorithms and enables efficient solutions to periodicity and factorization problems, a foundation for Shor’s algorithm in quantum cryptanalysis.

Grover’s Algorithm

Grover’s Algorithm provides a powerful quadratic speedup for unstructured search problems, reducing search complexity from $ O(N) $ to $ O(\sqrt{N}) $. This efficiency gain makes it particularly valuable for applications involving large datasets where specific items or optimal solutions need to be identified quickly. Grover’s approach relies on iteratively amplifying the probability amplitude of the target state, isolating it from other possible states through a process known as amplitude amplification.

The key to amplitude amplification in Grover’s Algorithm is the diffusion operator $ D $, which can be mathematically expressed as:

where $ H^{\otimes n} $ represents the Hadamard gate applied to each of the $ n $ qubits. This operator amplifies the target state’s probability by reflecting all states around their mean amplitude, boosting the likelihood of measuring the correct result.

Steps of Grover’s Algorithm

1. Initialization: Begin by initializing a superposition across all possible states:

   \[|\psi\rangle = \frac{1}{\sqrt{N}} \sum_{x=0}^{N-1} |x\rangle\]

   where each $ |x\rangle $ represents a potential solution.

2. Oracle Function $ O $: Use an oracle function to mark the target state by flipping its amplitude. Mathematically, this is represented as:

   \[O|x\rangle = \begin{cases} -|x\rangle, & \text{if $x$ is the target} \\ |x\rangle, & \text{otherwise} \end{cases}\]

3. Amplitude Amplification (Diffusion): Following the oracle, apply the diffusion operator $ D $ to amplify the target state’s amplitude while suppressing others:

   \[D = 2|\psi\rangle\langle\psi| - I,\]

   where $ |\psi\rangle $ is the initial superposition. The operator $ D $ increases the probability of measuring the target state by reflecting all amplitudes about their average.

4. Iteration: Repeat the oracle and diffusion steps approximately $ \frac{\pi}{4}\sqrt{N} $ times to maximize the target state’s probability amplitude.

5. Measurement: Measure the quantum state to collapse it onto the target state with high probability.

Applications of Grover’s Algorithm

• Unstructured search: Grover applies where the only way to recognise a solution is to test it — SAT-style constraint satisfaction, brute-force preimage search, collision finding.
• Optimisation: Amplitude amplification is a subroutine inside several quantum optimisation heuristics, giving a quadratic reduction in the number of oracle calls.
• Cryptanalysis: Grover halves the effective key length of a symmetric cipher, which is exactly why the standard advice is AES-256 rather than AES-128 in a post-quantum world.

Quantum Principal Component Analysis

Quantum PCA offers a quantum-enhanced approach to Principal Component Analysis (PCA), widely used for dimensionality reduction in machine learning and data engineering. By leveraging quantum computation, Quantum PCA can significantly accelerate the extraction of principal components, making it particularly advantageous for high-dimensional datasets.

The classical covariance matrix $ \Sigma $, which captures the variance structure of the data, can be approximated as a quantum density matrix $ \rho $ as follows:

where each data vector $ | x_i \rangle $ is represented as a quantum state. The density matrix $ \rho $, defined as:

serves as the quantum equivalent of the covariance matrix and allows for the representation of the dataset’s principal components.

The principal components are found by solving the eigenvalue problem for $ \rho $, where eigenvalues $ \lambda_i $ and eigenvectors $ | u_i \rangle $ satisfy:

Using Quantum Phase Estimation (QPE), we can efficiently approximate the eigenvalues $ \lambda_i $ and their corresponding eigenvectors $ | u_i \rangle $.

Steps of Quantum PCA

1. Encoding the Covariance Matrix as a Quantum State

   First, we represent the classical covariance matrix $ \Sigma $ of a dataset $ X \in \mathbb{R}^{n \times d} $ (where $ n $ is the number of observations and $ d $ is the number of features) as a quantum density matrix $ \rho $. Given by:

   \[\begin{aligned} \Sigma = \frac{1}{n} X^T X, \end{aligned}\]

   the matrix $ \Sigma $ is encoded as $ \rho $, a $ d \times d $ Hermitian matrix with similar properties to the covariance matrix. Quantum density matrix encoding leverages quantum states corresponding to each row of $ X $, capturing the structure of the data in the quantum realm.

2. Quantum Phase Estimation to Find Eigenvalues and Eigenvectors

   To extract the principal components, we use Quantum Phase Estimation (QPE), which estimates the eigenvalues and eigenvectors of the density matrix $ \rho $. Through quantum parallelism, QPE simultaneously approximates multiple eigenvalues, yielding an eigenvalue decomposition of $ \rho $:

   \[\begin{aligned} \rho = \sum_{i=1}^{d} \lambda_i | u_i \rangle \langle u_i |, \end{aligned}\]

   where:

   • $ \lambda_i $ represent the variance explained by each principal component,
   • $ | u_i \rangle $ are the eigenvectors, or principal components.

   The QPE algorithm applies a unitary transformation associated with $ \rho $ to a quantum register, allowing the quantum system to find eigenvalues and eigenvectors with high efficiency.

3. Measurement to Obtain Principal Components

   After applying QPE, a measurement is performed, collapsing the quantum state onto the eigenvector corresponding to the largest eigenvalue, representing the primary principal component. Repeated measurements reveal the dominant principal components in order of explained variance, extracting key features without explicitly computing all eigenvalues and eigenvectors.

Applications of Quantum PCA

• The headline claim: classical diagonalisation of a $d \times d$ matrix is $O(d^3)$; qPCA using quantum phase estimation runs in time polylogarithmic in $d$ — often quoted as $O(\log d)$ — for the dominant components.

• Efficiency in high-dimensional data: the state $\rho$ occupies $\log_2 d$ qubits rather than $d^2$ floats, so the representation is exponentially compact.

The data engineering reality check

Everything above is real physics and real computer science. This section is about why almost none of it touches a data pipeline, and it is the part I would most want a data engineer to internalise, because it lets you evaluate a vendor claim in about thirty seconds.

The input problem

A quantum computer’s advantage lives in a $2^n$-dimensional state space that it can manipulate with $\mathrm{poly}(n)$ gates. To use that on your data, your data has to get in. Encoding an arbitrary classical vector of length $N$ into the amplitudes of $\log_2 N$ qubits requires, in general, a circuit of $O(N)$ gates. There is no shortcut for arbitrary data — the information-theoretic content is $N$ numbers and you have to write $N$ numbers.

So for any workload whose cost is dominated by touching every record, the state preparation is already as expensive as just doing the computation classically. This is not an engineering immaturity that better hardware fixes. It is structural.

QRAM — quantum random access memory, which would return $\sum_i \alpha_i \lvert i \rangle \lvert x_i \rangle$ in $O(\log N)$ time — is the assumed escape hatch in most quantum machine learning papers. It has not been built at any useful scale, the leading bucket-brigade proposal requires $O(N)$ physical components that all have to be error-corrected, and the cost of that error correction is widely argued to reintroduce the polynomial factor the architecture was supposed to remove. Any paper claiming an exponential speedup on classical data is, in practice, assuming a device that does not exist.

The output problem

Symmetrically: a quantum computation ends in a state, and measurement gives you one sample. If your answer is a large object — a transformed table, a full set of principal components, a sorted result set — you need $O(N)$ measurements (each requiring a full re-run of the circuit) to extract it. Quantum algorithms only stay fast when the answer is small: a single number, a distribution you want to sample from, a yes/no.

Putting the two together

The combination gives a useful filter. Quantum computing can plausibly help when:

Data engineering workloads have the opposite profile in almost every case: enormous input, enormous output, and a computation that is mostly I/O, shuffles and joins rather than a hard combinatorial core. Your Spark job is slow because of a skewed partition key and a shuffle spilling to disk. Neither of those is a computational-complexity problem, and nothing about superposition addresses them.

The dequantisation lesson

The deepest correction of the last several years came from a then-undergraduate, Ewin Tang. The quantum recommendation systems algorithm of Kerenidis and Prakash was one of the flagship exponential quantum machine learning speedups. Tang showed in 2018 that if you grant a classical algorithm the same sampling access to the input that the quantum algorithm’s QRAM assumption grants it, a classical algorithm matches the polylogarithmic scaling. The quantum speedup was an artefact of comparing a quantum algorithm with a strong input model against a classical algorithm with a weak one.

The technique — now a small industry called dequantisation — has since been applied to low-rank matrix inversion, principal component analysis, supervised clustering, semidefinite programming and more. The pattern is consistent: exponential quantum speedups on classical low-rank data tend not to survive a fair comparison.

The practical takeaway is a question to ask of any claim: what input model is assumed, and is the classical baseline given the same one? If the answer is “QRAM” and the baseline is a naive dense algorithm, the speedup is probably not real.

Where the linear-algebra story actually landed

The HHL algorithm for linear systems is the other one that gets quoted at data engineers. It solves $A\vec{x} = \vec{b}$ in time polylogarithmic in the system size — subject to $A$ being sparse and well-conditioned, $\vec{b}$ being preparable, and the output being a state $\lvert x \rangle$ rather than the vector $\vec{x}$. Read the conditions carefully and the applicability narrows dramatically. It is a legitimate result; it is not a replacement for your least-squares solver.

Where quantum genuinely might help

Being sceptical is not the same as being dismissive. There are areas where the case is strong and where the input/output filter above is satisfied:

• Quantum simulation. Simulating molecules and materials was Feynman’s original motivation and remains the most defensible application. The input is a Hamiltonian — a compact description, not a dataset — and the output is an energy or a spectrum. The classical cost is exponential in system size for genuinely correlated systems. Catalysis, battery chemistry, nitrogen fixation, and high-temperature superconductivity are the standard targets. If quantum computing changes an industry first, it will be pharmaceuticals and materials, not analytics.
• Cryptanalysis. Shor’s algorithm gives an exponential speedup on factoring and discrete logarithms, and unlike the machine learning cases, the input is a single integer. This one is not in doubt; only the timeline is. It is the subject of the next section.
• Optimisation, cautiously. QAOA and quantum annealing attract enormous interest for portfolio optimisation, routing and scheduling. As of 2026 there is still no convincing demonstration of advantage over good classical heuristics on a real instance — classical solvers keep improving in response, and several claimed advantages have been matched classically within months. Worth watching; not worth building on.
• Sampling and generative models. Quantum circuits produce probability distributions that are hard to sample classically. Whether any of those distributions is one that anyone actually wants is the open question.

For data engineering specifically, the realistic near-term shape is not “quantum replaces Spark” but “a quantum subroutine becomes one more remote accelerator call inside a classical pipeline” — the same architectural position as a GPU inference endpoint, invoked for a small, hard, well-defined subproblem. That is a solved integration pattern; you already know how to build it.

The part that is actually on your roadmap: post-quantum cryptography

Here is the section with deadlines in it.

Shor’s algorithm breaks RSA, Diffie-Hellman and elliptic-curve cryptography. That is essentially all public-key cryptography in production today: TLS, SSH, code signing, VPNs, database connections, cloud KMS envelopes, JWTs, and the mutual authentication between your services. A cryptographically relevant quantum computer needs millions of physical qubits, and consensus estimates for when one exists cluster around 2033-2035.

It would be a mistake to conclude that this is a 2033 problem.

Harvest now, decrypt later

An adversary can capture encrypted traffic today and store it until a quantum computer can decrypt it. Nothing about that requires the quantum computer to exist yet. It only requires the data to still be sensitive when it does.

For a data engineer, this reframes the question usefully. Look at what flows through your pipelines and ask how long each thing stays sensitive:

• Medical records, genomic data: decades.
• Financial records, tax data: 7-10 years by statute, longer in practice.
• Government and defence material: 25 years or more.
• Long-lived credentials, signing keys, root CA material: for as long as they remain trusted.
• Today’s clickstream: essentially worthless in 2035.

Anything in the first four categories crossing a network encrypted with classical public-key cryptography is exposed to harvest-now-decrypt-later today. That is what makes this a current-year concern rather than a next-decade one.

The standards exist and are final

This is no longer speculative. NIST published the first three post-quantum standards in August 2024:

NIST additionally selected HQC in March 2025 as a backup key-encapsulation mechanism built on error-correcting codes rather than lattices, precisely so that a future break of the lattice assumption does not take out the whole ecosystem at once. A draft standard is expected around 2026 with finalisation targeted for 2027.

FN-DSA (FALCON) is also in progress as a signature scheme with smaller signatures than ML-DSA, at the cost of trickier floating-point implementation requirements.

The dates that matter

NIST IR 8547 sets the migration timeline for US federal systems, and it is the schedule the wider industry is converging on:

• 2030 — RSA, ECDSA, EdDSA, Diffie-Hellman and ECDH at 112-bit security are deprecated.
• 2035 — those algorithms are disallowed. Federal systems are expected to have completed migration.

The UK NCSC has published a comparable timeline, with organisations expected to have completed discovery by 2028 and high-priority migration by 2031. If you work with government, healthcare, or financial data in any jurisdiction, these dates will reach your compliance requirements well before they reach the news.

It is already in production around you

The migration has quietly started, and you are probably using it without having noticed:

• Chrome and Firefox have shipped hybrid X25519MLKEM768 key agreement for TLS 1.3. A large and growing share of TLS connections from modern browsers are already post-quantum protected against harvest-now-decrypt-later.
• Cloudflare and other major CDNs support hybrid post-quantum key agreement across their networks.
• OpenSSH added support for post-quantum key exchange, and made it the default in recent releases; OpenSSH 10 warns when a connection negotiates a non-post-quantum key exchange.
• AWS KMS, ACM and Secrets Manager support hybrid post-quantum TLS endpoints. Azure and GCP have published migration roadmaps.
• Signal and Apple iMessage (PQ3) both deployed post-quantum ratchets.

The pattern everywhere is hybrid: run classical and post-quantum key exchange together and combine the secrets, so the connection is secure if either holds. That hedges against implementation bugs in the new, less-battle-tested schemes. Expect hybrid to remain the norm for years.

What this means for a data platform

Concretely, for the systems a data engineer owns:

In transit. Every TLS connection in your architecture — clients to API, service to service, service to database, connector to Kafka, worker to object store — currently negotiates a classical key exchange unless you have deliberately configured otherwise. Most of this is fixed by upgrading TLS libraries and enabling hybrid groups, not by rewriting application code. The work is inventory and upgrade coordination, which is unglamorous but tractable.

At rest. Symmetric encryption is fine. AES-256 against Grover retains roughly 128 bits of effective security, and even that assumes a Grover attack that is far less practical than the raw complexity suggests because it does not parallelise well. You do not need to re-encrypt your data lake. What you do need to look at is the key wrapping: envelope encryption schemes protect a symmetric data key with an asymmetric key, and that outer layer is the quantum-vulnerable part.

Signatures and long-lived trust. Code signing, artefact signing, certificate chains and long-lived JWT signing keys all need a migration path. Signatures have a different risk profile from encryption — a signature verified today and never again is not at risk retrospectively — but anything that has to remain verifiable for a decade does need planning. ML-DSA signatures are substantially larger than ECDSA ones (kilobytes rather than tens of bytes), which has real consequences for protocols with tight size budgets, embedded systems, and anything that stores a signature per row.

Size and performance. ML-KEM-768 public keys are around 1.2 kB against 32 bytes for X25519. The CPU cost is generally comparable or better than classical ECC, so the pain is bandwidth and packet sizes rather than compute — but it is enough to break protocols that assumed a handshake fits in one round trip, and enough to matter if you sign individual messages at high volume.

What a data engineer should actually do in 2026

A practical, non-panicked sequence:

1. Build a cryptographic inventory. You cannot migrate what you cannot find. Enumerate where public-key cryptography appears across your platform: TLS termination points, service meshes, database connection settings, message brokers, object store clients, KMS integrations, code signing, SSH keys and host keys, JWT and OIDC signing, and any hardcoded certificates. This is the step everyone underestimates and the one every published guidance document puts first.
2. Classify by data lifetime. Cross-reference the inventory against how long the data it protects stays sensitive. Anything protecting data with a lifetime past 2035 is the priority queue; anything protecting ephemeral data can wait for the vendor upgrade cycle.
3. Design for crypto-agility. The real lesson is not “adopt ML-KEM”, it is that algorithms will change again — HQC is arriving, FN-DSA is arriving, and something will be broken or deprecated in the next twenty years. Push algorithm selection behind an interface, keep algorithm identifiers in your data formats and key metadata, avoid hardcoding key and signature sizes, and make key rotation a routine operation rather than an incident. A platform that can swap primitives without a rewrite has solved the general problem, not just this instance of it.
4. Enable hybrid where you get it for free. Upgrading OpenSSL, Go, the JDK and your OpenSSH to current versions gets you a large fraction of the transport-layer migration with configuration rather than code. Turn on hybrid key exchange on the connections that matter and measure the handshake overhead in your own environment.
5. Push your vendors. Your managed database, streaming platform, warehouse and SaaS connectors control cryptography you do not. Asking for their post-quantum roadmap now costs you an email and gets the requirement into their backlog while there is still time.
6. Do not buy a quantum key distribution box. QKD requires dedicated fibre or line-of-sight optics, does not solve authentication (it needs a classical authenticated channel to bootstrap, which is the problem you were trying to solve), and does not scale to internet topology. The NSA, the UK NCSC and other national agencies have all published guidance recommending post-quantum cryptography over QKD for securing communications. Post-quantum cryptography is software, deploys over existing networks, and is the actual answer.

This is the concrete quantum-computing work item for a data platform this decade, and note that all of it is classical engineering. You do not need a quantum computer, a physicist, or a strategy deck. You need an inventory, a version bump, and a design that assumes the algorithms will change again.

Current quantum computing platforms

If you want to experiment, access is genuinely easy and mostly free at small scale.

• IBM Quantum Platform — the largest fleet of publicly accessible superconducting processors, programmed with Qiskit. Note that the old “IBM Quantum Experience” name is retired and the platform migrated to a new IBM Cloud-based service in 2025; documentation now lives at quantum.cloud.ibm.com/docs. Qiskit 2.x is a significant break from the 1.x code in most older tutorials — see the note below.
• Google Quantum AI — the Willow and Sycamore lineage, programmed with Cirq. More research-oriented; hardware access is by collaboration rather than open sign-up.
• Microsoft Azure Quantum — a broker rather than a manufacturer, giving one API over IonQ, Quantinuum, Rigetti and Pasqal hardware, plus Q# and the QDK. The Majorana topological programme is separate and considerably more speculative.
• Amazon Braket — the AWS equivalent, with the Braket SDK over IonQ, IQM, Rigetti and QuEra, and the useful property that it sits inside an AWS account you already have, with the IAM and S3 integration you already understand.
• Quantinuum — trapped-ion H-series systems with the highest published gate fidelities and all-to-all connectivity; fewer qubits, much better ones.
• IonQ — trapped ion, reporting 99.9923% two-qubit gate fidelity in 2025, available through all three major clouds.

Rigetti’s standalone QCS remains, but most access now routes through Braket or Azure Quantum rather than direct.

Getting started

The learning curve is real but shorter than it looks — linear algebra plus a simulator gets you a long way, and you do not need hardware access to learn anything in this post.

1. Linear algebra first. Complex vector spaces, unitary matrices, tensor products, eigendecomposition. Quantum computing is linear algebra with a physical interpretation; if the algebra is comfortable, the rest follows.
2. Use a simulator, not hardware. Up to about 30 qubits a laptop simulator is faster, noiseless, and infinitely more debuggable than a real device. Real hardware is currently useful for studying noise, not for getting answers.
3. Implement the classics. Deutsch-Jozsa, Bernstein-Vazirani, Grover, quantum phase estimation. Implementing them is what makes interference click, in a way that reading about it does not.
4. Read a resource-estimate paper. Nothing calibrates expectations faster than seeing how many physical qubits and how many hours a “simple” algorithm actually needs.
5. Then spend your remaining effort on post-quantum cryptography, which is the part of this that will appear in your actual job.

A working “hello world” on current Qiskit — the API changed enough in 2.x that most tutorials you will find are broken:

# pip install qiskit qiskit-aer
from qiskit import QuantumCircuit, transpile
from qiskit_aer import AerSimulator

# Bell state: entangle two qubits.
qc = QuantumCircuit(2)
qc.h(0)          # superposition on q0
qc.cx(0, 1)      # entangle q0 with q1
qc.measure_all()

sim = AerSimulator()
result = sim.run(transpile(qc, sim), shots=1024).result()
print(result.get_counts())
# ~50% '00', ~50% '11', and essentially no '01' or '10' — that is entanglement.

Three migration notes if you are updating older code: execute() was removed in Qiskit 1.0, so build a circuit, transpile() it against the backend, then call backend.run(); the simulators moved out into the separate qiskit-aer package; and qiskit.opflow was removed in favour of qiskit.quantum_info.SparsePauliOp. Hardware submission now goes through the V2 primitives, Sampler and Estimator, rather than direct circuit execution.

Conclusion

Three things are true at once, and holding all three is the useful position.

Quantum computing is real and progressing faster than the sceptics expected. Below-threshold error correction was the milestone that mattered, and it arrived. The remaining problems are engineering problems of scale, not open questions of principle. Fault-tolerant machines are a matter of when.

Almost none of the “quantum for big data” story survives contact with the input/output bottleneck. Getting classical data into a quantum computer costs what the computation would have cost. Getting large answers out costs the same again. Dequantisation removed the exponential speedup from a string of flagship machine learning results by the simple expedient of giving the classical baseline the same assumptions. Grover is asymptotically worse than the index you already have. Your pipeline is slow for reasons that have nothing to do with computational complexity, and the honest advice for making it faster remains: fix the partitioning, fix the file sizes, fix the shuffle, and reach for a bigger cluster before you reach for a different kind of physics.

One part of this is a real, dated, current work item — and it is entirely classical. Post-quantum cryptography has finalised standards, published deprecation dates of 2030 and 2035, an active harvest-now-decrypt-later threat to any long-lived sensitive data crossing your network, and a migration that is already underway in the browsers and SSH clients you use daily. If you take one action from this post, make it an inventory of where public-key cryptography lives in your platform, and a design that lets you swap the algorithms without a rewrite.

The genuinely interesting quantum applications — simulating molecules and materials — are ones where the input is a Hamiltonian rather than a dataset. That is a fascinating field. It is just not, mostly, our field. What is ours is making sure that the data we are responsible for today is still protected when the machines arrive.

Further reading

Foundations

• Nielsen, M. A., & Chuang, I. L. (2010). Quantum Computation and Quantum Information. Cambridge University Press. — still the standard reference.
• Qiskit documentation and learning platform — the current home, replacing the old qiskit.org/documentation.

Hardware milestones

• Google Quantum AI (2024). Quantum error correction below the surface code threshold. Nature. — the Willow result.
• Arute, F., et al. (2019). Quantum supremacy using a programmable superconducting processor. Nature, 574, 505-510. — for the historical record, and for how contested such claims are.

The sceptical literature — the most useful reading in this list

• Aaronson, S. Shtetl-Optimized — consistently the best correction to quantum hype, from someone who works in the field.
• Tang, E. (2019). A quantum-inspired classical algorithm for recommendation systems. — the dequantisation paper.
• Aaronson, S. (2015). Read the fine print. Nature Physics, 11, 291-293. — the definitive short account of why quantum machine learning speedups need caveats.

Post-quantum cryptography

• NIST Post-Quantum Cryptography project — FIPS 203, 204 and 205.
• NIST IR 8547: Transition to Post-Quantum Cryptography Standards — the 2030 and 2035 dates.
• UK NCSC: Timelines for migration to post-quantum cryptography — the UK schedule, and good practical guidance on discovery.
• Open Quantum Safe — implementations you can actually test against.