Randomized benchmarking over Clifford sequences

What it fills

• Measure what the machine can actually do

  Run a protocol whose answer is already known, on the hardware, and read the machine's own performance off how far the result falls short. Nothing here computes anything a user wanted — the point is that the answer is known in advance, because that is what makes the shortfall a measurement.

When it applies

The protocol's shape is stated exactly: *"Generate a sequence of m + 1 quantum operations with the first m operations chosen uniformly at random from some group G ⊆ U(d) and the final operation chosen so that the net sequence (if realized without errors) is the identity operation"*, with the Clifford group chosen *"because each element of the Clifford group can be realized efficiently on a quantum processor"*. The number that comes out is a per-gate average, read off the decay: $r = 1 - p - (1-p)/d$. **What "robust" means here is the paper's own contribution and it is a correction rather than a refinement:** earlier randomized benchmarking could be defeated outright — *"it is easy to show (via a counter example with gate-dependent errors that consist of the exact inverse of the gate applied) that the decay rate estimated via RB methods can be totally unrelated to the actual error-rate"* — and this analysis *"is valid for a realistic noise model admitting time-dependent and gate-dependent errors and also accounts for state preparation and measurement errors"*, requiring only that the variation across the gate set is not too strong.

Requires

Every step this method names moves its route along, so there is nothing it needs alongside them.

Example

given  a programmable device realizing the Clifford group G on n qubits (d = 2^n),
       with each element efficiently compilable and its correcting inverse
       efficiently pre-computable
           # (p.1, "Step 1" paragraph)
       K := |G|, the size of the Clifford group on n qubits
           # (p.2, text around Eq. 8)
       an initial state rho_psi (folding in preparation error) and a POVM
       element E_psi (folding in measurement error) -- ideally
       rho_psi = E_psi = |psi><psi|                                          (Eq. 2, "Step 2")
       an accuracy eps and a confidence delta for the fidelity estimate at
       each sequence length m, and a range of m to sweep, with m >> 1
                                      (paragraph after Eq. 18)

requires  the noise Lambda_{i,j} may depend on the time-step j and on which
    gate C_i is applied, but the environment's correlation time must be
    negligible against one operation's duration -- the only structural
    assumption on the noise model this protocol targets
        # (p.1, paragraph after Eq. 1)

# --- Step 1: build one random length-(m+1) sequence -------------------------
draw i_1, ..., i_m independently and uniformly at random from G
    # exact uniform sampling needs an exhaustive list of G, which is only
    #   feasible for a few qubits; for larger n, sample instead from a
    #   generating set G' (CNOTs on every pair, plus single-qubit H and S)
    #   whose compiled length b scales polynomially in n, or from the
    #   2-qubit Clifford group per pair, whose b is a smaller-degree
    #   polynomial in n -- both are cited constructions, not proved here
    #   (p.4, "conclusion" paragraphs, Refs [20],[24])
C_{i_{m+1}} := the unique Clifford making the length-(m+1) sequence equal
    the identity if every operation were realized without error             (Step 1, Eq. 1)
S_{i_m} = compose over j = 1..m+1 of ( Lambda_{i_j,j} then C_{i_j} )        (Eq. 1)

# --- Step 2: execute the sequence once and measure survival ----------------
run the sequence on rho_psi, measure E_psi
record  s := Tr[ E_psi  S_{i_m}(rho_psi) ]                                   (Step 2, Eq. 2 context)

# --- Step 3: average over k random draws at this m --------------------------
repeat Steps 1-2 for k independently-drawn sequences, where
    k := ln(2/delta) / (2 * eps^2)
        # (Hoeffding bound, unlabelled eq. after Eq. 18)
    # proved, not fitted: this many trials suffice to put the averaged
    #   sequence fidelity within eps of its true value at confidence delta,
    #   and k does not depend on m or on the qubit count n -- but the bound
    #   assumes the i_1..i_m are drawn UNIFORMLY from the Clifford group
F_seq(m, psi) := mean of the k recorded s values                             (Step 3, Eq. 2-3)

# --- Step 4: sweep m, fit the decay ------------------------------------------
repeat Steps 1-3 at every m in the chosen range
fit { (m, F_seq(m,psi)) } to ONE of the two models below (pick one, not both):

    F_seq^(0)(m,psi) = A0 p^m + B0                                           (Eq. 6)
        # exact only if Lambda_{i,j} = Lambda for every gate and every
        #   time-step -- errors precisely gate- and time-independent. The
        #   twirl is then over the CUMULATIVE products
        #   D_{i_j} := C_{i_j} after ... after C_{i_1}, not over the
        #   single-step C_{i_j}: substituting the net identity turns the
        #   sequence into an m-fold gate-independent twirl
        #   Lambda_twirl = (1/K) sum_{i_j} D_{i_j}^dagger Lambda D_{i_j},
        #   which is exactly depolarizing; A0, B0 absorb state-prep and
        #   measurement error and an edge effect from the error on the
        #   final gate
        #   (Eq. 9 and following paragraph; paragraph after Eq. 6)

    F_seq^(1)(m,psi) = A1 p^m + B1 + C1 (m-1)(q - p^2) p^(m-2)               (Eq. 4)
        # perturbs each Lambda_{i,j} to first order about the gate-set
        #   mean Lambda_bar; valid once gamma := (1/K) sum_i
        #   ||Lambda_i - Lambda_bar||_{1->1}^H satisfies gamma << 2/m for
        #   TIME-independent noise -- noise that also varies across
        #   time-steps needs the paper's stronger per-step bound on each
        #   gamma_j instead of this one scalar; this is an assumed regime,
        #   not something proved to hold                                    (paragraph after Eq. 18)
        # q - p^2 is this model's own measure of how gate-dependent the
        #   errors are; A1, B1, C1 absorb state-prep/measurement error and
        #   the same final-gate edge effect as A0, B0
        #   (Eq. 15-17; paragraph after Eq. 4)

read p off the fit that was used

return  r := 1 - p - (1 - p)/d                                               (Eq. 5)
# r averages over the Haar measure on each gate AND over the whole gate
#   set G -- it is a group-average per-gate error, never a worst-case or
#   single-gate figure
#   (Eq. 7 and surrounding text)
# this fit is provably reliable for a general time- and gate-dependent
#   noise model, PROVIDED the gate-dependence stays small enough for the
#   model used above; without that qualifier RB is not automatically
#   trustworthy -- a counterexample where the error on each gate is exactly
#   that gate's own inverse can make an unqualified RB decay rate totally
#   unrelated to the true error-rate                                        (p.1, intro paragraph)
# gate compilation is NOT free: one length-m sequence costs mb elementary
#   gates, b = poly(n) per Clifford element, once C is realized via a
#   generating set rather than looked up directly
#   (p.4, "conclusion" paragraphs)

Cost, as the source states it

The paper prices two things and proves only one. Proved: by Hoeffding's inequality, estimating the averaged sequence fidelity at a single sequence length $m$ needs at most $k=\ln(2/\delta)/2\epsilon^2$ randomly drawn Clifford sequences, where $\epsilon$ is the accuracy and $\delta$ is what this paper itself calls the confidence; $k$ is independent of $m$ and of the qubit number $n$, and the bound assumes uniform sampling from the Clifford group. Exhaustive averaging is ruled out instead: the group's size scales as $2^{O(n^2)}$, so the number of length-$m$ sequences scales as $2^{mO(n^2)}$. Cited rather than proved here: compiling each Clifford element into $b$ generating-set gates, with $b$ polynomial in $n$, makes one length-$m$ sequence cost $mb$ elementary gates. Nothing here bounds how many distinct sequence lengths the fit itself requires.

Implementations

• Single-qubit numerics, four noise models

  • Robust randomized benchmarking of quantum processes

  This paper reports no hardware run. Its only demonstration of the protocol is a simulation: the abstract says the authors "illustrate the protocol through numerical examples", and the conclusion says they "numerically illustrate the relevance of both models". The experimental randomized benchmarking the introduction points at — "initial experimental implementations of RB in atomic ions for different types of traps [13, 14], NMR [15], superconducting qubits [6, 16], and atoms in optical lattices [17]" (p. 1, left column) — is cited, not performed in this Letter. What is reported is a simulated single-qubit benchmarking experiment against four constructed noise models, and its purpose is to separate the two fitting models the paper derives: to show where the zeroth-order curve $F_{seq}^{(0)}(m,\psi)=A_0p^m+B_0$ (Eq. 6) suffices and where the first-order curve $F_{seq}^{(1)}(m,\psi)=A_1p^m+B_1+C_1(m-1)(q-p^2)p^{m-2}$ (Eq. 4) is needed.

  The example benchmarks "a single qubit under time-independent unitary errors with no state-preparation or measurement errors". The error on each Clifford is manufactured out of the gate itself, which is what makes it gate-dependent by construction: "For each $C_j$, the unitary error was constructed by finding the Hamiltonian that generates the Clifford operation via $C_j(\rho) = \exp(-iH_j)\rho\exp(iH_j)$. For each $H_j$, the unitary $\exp(-iH_j)$ was diagonalized and to simulate the error one of the eigenvalues was multiplied by $e^{i\delta}$ and the other by $e^{-i\delta}$", which the paper reads as "over/under rotations around $H_j$". (The symbol $\delta$ does double duty in this paper: the rotation angle here, and the confidence level in the Hoeffding bound elsewhere.) Two further models add a channel on top of that unitary error — "unitary error with depolarizing noise and unitary error with amplitude damping". Fig. 1 plots $F_{seq}(m,\psi)-1/2$ rather than $F_{seq}$ itself because "we have subtracted the DC offset in the model so that pure exponentials appear as straight lines on the semilog plot", and for the two unitary cases that offset is exactly $B_1 = B_0 = 1/2$ "since the noise is unital and there are no state preparation or measurement errors". **Two things a reader needs are never reported for these numerics.** The paper's own Step 3 is "Average over random realizations of the sequence", yet how many random Clifford sequences were averaged at each $m$ is not given anywhere — the Hoeffding bound $k=\ln(2/\delta)/2\epsilon^2$ is stated in this same paper, but no $k$, no accuracy $\epsilon$ and no confidence $\delta$ is quoted for the simulation, so the bound describes what would suffice rather than what was run. And no software, package, language or machine is named anywhere in the Letter — that is a reporting convention rather than something the four-step protocol demands, but it is why nothing here can be re-run as written.

  Case A fixes the over/under-rotation angle: "$\delta = 0.1$ (case A)". Case B draws it, "$\delta$ chosen uniformly at random in the range $[0.075, 1.125]$ (case B)" — quoted exactly as the paper prints the interval, confirmed in a 500 dpi render of p. 3 rather than from the extracted text alone. As printed, the upper endpoint is more than ten times case A's $\delta = 0.1$; this entry reproduces the paper's interval and makes no correction to it. For the other two models, "the depolarizing and damping parameters were chosen randomly in $0.9875 \pm 0.01$ with the unitary error chosen in the same way as case A". Sequence lengths appear only in Fig. 1 and only for the two unitary cases: the numerics are plotted at $m = 3, 5, 7, \ldots, 51$ — twenty-five sequence lengths per case, read off the figure by measuring the plotted points against the axis tick marks, since no sentence in the paper states a sequence-length range or list. There is no dataset here in any other sense; the inputs are the noise-model parameters above.

  Table I reports the fitted decay parameter $p$, the average error-rate $r$ — Eq. (5) ties the two, the paper writing that "$p$ determines the average error-rate $r$ according to the relation" $r = 1 - p - (1-p)/d$ — and the paper's own gate-dependence measure $q - p^2$, for the four simulated noise models. Unitary A: $p = 0.980$, $r = 1.05\times10^{-2}$, $q-p^2 = -2.73\times10^{-4}$. Unitary B: $p = 0.943$, $r = 2.85\times10^{-2}$, $q-p^2 = -6.83\times10^{-3}$. Unitary and depolarizing: $p = 0.982$, $r = 8.75\times10^{-3}$, $q-p^2 = -2.77\times10^{-8}$. Unitary and $T_1$ (the prose calls it amplitude damping): $p = 0.988$, $r = 5.85\times10^{-3}$, $q-p^2 = -2.80\times10^{-8}$. **No uncertainty is attached to any of these** — Table I prints point values only, and no error bars appear in Fig. 1. One caveat for anyone recomputing: at $d = 2$ Eq. (5) reproduces Unitary B's $r$ from its $p$ exactly, and the last two columns once $p$ is rounded to three decimals, but not Unitary A, whose printed $r = 1.05\times10^{-2}$ implies $p = 0.979$ against the $0.980$ the table prints; both are reproduced above exactly as the paper prints them. These are outputs of the simulation's own constructed error models, not measurements: no hardware and no simulator is named. The comparison the numbers are for splits the four cases in two. For the unitary pair, "the first order result fits the data extremely well (green line) while the zero'th order (red dashed) only approximates the sequence fidelity when the variation in $\delta$ is small (case A)", and in case B "the non-exponential behaviour of the average sequence fidelity is clearly visible". For the other two it runs the other way: "in both these cases the simulations are well approximated by the zero'th order solution", which the paper takes to show "that the zero'th-order randomized benchmarking model gives a robust estimate of the error-rate for a variety of error models provided that the variation in the noise is small enough". The text describes its gate-dependence measure as "much larger for case B than case A", while Table I prints both of those values as negative. Fig. 1 covers the unitary cases alone — its caption reads in full "(color online) Average sequence fidelity as a function of sequence length for a error model with unitary noise. See text for details.", the paper's own "a error model" included — so the depolarizing and damping models have no plotted decay curve, only a Table I column.

What it needs

Nobody has taken this apart yet. That is a gap in this graph, not a claim that the method has no parts.

Other ways to fill the same slot

Different approaches

• Quantum volume from random square circuits

  Run random circuits that are as deep as they are wide, and ask how often the machine returns one of the outputs that should be more likely than the median. Widen and deepen together until it can no longer beat that bar; the last size it managed is the number.

In the Atlas

• Robust randomized benchmarking of quantum processes

  Estimate an average error rate for a set of operations (gates) on a quantum information processor, under a noise model general enough to allow errors that depend on both the time and the gate at which they occur.

Sources

• Robust randomized benchmarking of quantum processes↗Easwar Magesan, J. M. Gambetta, Joseph Emerson · 2010theory yes · simulation yes · hardware no