Measuring Performance

OK, so you have your program running on a quantum computer. How do you know it is any good?

Here are some of the key metrics to compare - take a look at the results from one of our hackathons to see how these look - you can play with the parameters in the dashboard tab.

Number of repeats, R

Let's assume that to find the ground state of a molecule (e.g. Helium) you've repeated the entire computation R times.

Precision δ and success probability, s

For each repeat, we consider a run successful if the lowest energy found is equal to the known ground state for that molecule to given precision δ (e.g. δ=0.1). We call the success probability s.

Total number of shots, t

We count the average number of calls to the quantum computer over the R runs. (Quantum computers are expensive so we want to use them as little as possible!)

Time-To-Solution

The Time-To-Solution(TTS) metric quantifies the reliability of the result against the number of times you used the quantum computer. For TTS, lower is better as this means you got a better result with fewer calls to the quantum computer.

If the probability of success is s, then the probability of failing to find the ground state after R runs is (1−s)^R. Therefore, the probability of finding the ground state at least once after R runs is p=1−(1−s)^R, and the expected number of calls to the quantum computer is

TTS = t x log(1-p)/log(1-s)

NB: You may find it easier to read the above formulae in the Jupyter Notebook from the 1st QCK Hackathon.