Test suite: simpler = better?

[alexfleury-sb]

Hello, I have been working on a no-pyscf version of Tangelo. During this process, I had to change code in many test folders. I compile the list of molecular systems needed for our test:

• H2_sto3g
• H4_sto3g
• H4_sto3g (with only MOs=1,2,3 active)
• H4_sto3g (with only MOs=1,2 active)
• H4_cation_sto3g
• H4_doublecation_minao
• H4_sto3g (with symmetry tuned on)
• NaH_sto3g (with only MOs=5,9 active)
• H2O_sto3g (with MOs=0,7 frozen)

I feel this is a bit all over the place, with different reference values even with the very same molecular systems. I wonder if it would be beneficial to restrict ourselves to 2-3 molecules in our testing. The code could be improved to centralize the reference values to a single file, for e.g.

class TangeloTest(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        from tangelo.molecule_library import mol_H2_sto3g, mol_H3_sto3g

        cls.mol_H2_sto3g = mol_H2_sto3g
        cls.mol_H3_sto3g = mol_H3_sto3g

        cls.energy_H2_sto3g = -1.13 # Energy for H2 STO-3G
        cls.energy_H3_sto3g = -1.11 # Energy for H3 STO-3G

        cls.energy_places = 4 # Compare energies up to 4 decimals.

And in another file, where TangeloTest is imported.

class SuperCoolNewAnsatzTest(TangeloTest):

    def energy_closed_shell_test(self):
        """Verify super-cool-new ansatz CS energy."""

        # Build circuit.
        ansatz = SuperCoolNewAnsatz(self.mol_H2_sto3g)
        ansatz.build_circuit()

        # Build qubit hamiltonian for energy evaluation.
        qubit_hamiltonian = jordan_wigner(self.mol_H2_sto3g.fermionic_hamiltonian)

        # Assert energy returned is as expected for given parameters
        sim = Simulator()
        ansatz.update_var_params([0., 1., 2., 3.])
        energy = sim.get_expectation_value(qubit_hamiltonian, ansatz.circuit)
        self.assertAlmostEqual(energy, self.energy_H2_sto3g places=self.energy_places)

    def energy_open_shell_test(self):
        """Verify super-cool-new ansatz OS energy."""

        # Build circuit.
        ansatz = SuperCoolNewAnsatz(self.mol_H3_sto3g)
        ansatz.build_circuit()

        # Build qubit hamiltonian for energy evaluation.
        qubit_hamiltonian = jordan_wigner(self.mol_H3_sto3g.fermionic_hamiltonian)

        # Assert energy returned is as expected for given parameters
        sim = Simulator()
        ansatz.update_var_params([0., 1., 2., 3.])
        energy = sim.get_expectation_value(qubit_hamiltonian, ansatz.circuit)
        self.assertAlmostEqual(energy, self.energy_H3_sto3g, places=self.energy_places)

Potential improvements:

• No more "data" folders in our tests. Data could be centralized into a TangeloTest and/or LinqTest classes.
• Framework for test writing (guide people on which molecular systems to test on).
• Same default decimal for float checks.
• Easier to decouple molecular computation from the relevant tests. With many tests depending on the molecular integrals, they may fail in the future if some changes are made to pyscf (or other backend we would be using).

[JamesB-1qbit]

I think this would certainly be an improvement! One point is that the energy used for testing does sometimes does need to be different as not all the algorithms converge to the exact answer. But that can be handled on a case by case basis.

Another option for energy is that we add known values to each molecule in molecule_library. For example, mol_H2_sto3g.fci_energy = -1.1372701746 Also could add excited state energies if known. This would have the added benefit of allowing users to have reference values at hand without running CCSDSolver/FCISolver/CASCI.

[alexfleury-sb]

That is why I thought of keeping it small as H2 would, with most of the ansatze that consider single and double excitations, give the FCI energy or something near. For other ansatze, we could only test the according to the number of gates or change the energy for each case.

For the second point you raised, we could indeed put references energies (FCI, CCSD, ...) directly into a data structure for test (a little like MolecularData does in the openfermionpyscf package. I worked on a MockSecondQuantizedMolecule class (https://github.com/AlexandreF-1qbit/Tangelo/blob/minimal_install/tangelo/helpers/testing.py) to perform test without pyscf, and other attributes could be added for reference values. With that in mind, the TangeloTest example class above would contained several MockSecondQuantizedMolecule objects with their reference values embedded in them. TangeloTest would also define the decimal points we choose for float comparison or other general options in testing.