single run feature

[danlavu]

No description provided.

[danlavu]

I don't like the logging output; it prints 'SKIPPED' and 'Skipped' for one test case; I'm not sure where it's coming from.

PASSED [ 31%]SKIPPED [ 37%]Single run marker set, skipping ipa

Skipped

The following marker values are tested, logs attached.

@pytest.mark.importance("critical")
@pytest.mark.topology(KnownTopologyGroup.AnyProvider)
def test_identity__single_run_no_marker(client: Client, provider: GenericProvider):


@pytest.mark.single_run()
@pytest.mark.importance("critical")
@pytest.mark.topology(KnownTopologyGroup.AnyProvider)
def test_identity__single_run_no_value(client: Client, provider: GenericProvider):


@pytest.mark.single_run("ad")
@pytest.mark.importance("critical")
@pytest.mark.topology(KnownTopologyGroup.AnyProvider)
def test_identity__single_run_value_specified(client: Client, provider: GenericProvider):


@pytest.mark.single_run("ad")
@pytest.mark.single_run("samba")
@pytest.mark.importance("critical")
@pytest.mark.topology(KnownTopologyGroup.AnyProvider)
def test_identity__single_run_invalid_values_specified(client: Client, provider: GenericProvider):

pytest-single-run.txt

[pbrezina]

I'm not a big fun of single_run. How about preferred_topology? Its still not perfect, so suggestion is welcomed. You should also not take the string but the topology marker object instead. It would be also good to allow lists. That is:

# By default, runs only LDAP
@pytest.mark.preferred_topology(KnownTopology.LDAP)
@pytest.mark.topology(KnownTopologyGroup.AnyProvider)
def test_exampe(...):
    pass

# By default, runs only LDAP and IPA
@pytest.mark.preferred_topology(KnownTopology.LDAP)
@pytest.mark.preferred_topology(KnownTopology.IPA)
@pytest.mark.topology(KnownTopologyGroup.AnyProvider)
def test_exampe(...):
    pass

# By default, runs only LDAP and IPA
@pytest.mark.preferred_topology([KnownTopology.LDAP, KnownTopology.IPA])
@pytest.mark.topology(KnownTopologyGroup.AnyProvider)
def test_exampe(...):
    pass

[pbrezina]

This is SSSD specific, can't be here. Please, remove it.

[danlavu]

Ack.

[pbrezina]

Move this into a standalone method.

[pbrezina]

It should probably rely on data.topology_mark rather then the global field. And maybe place it into pytest_runtest_call? It may solve the output problem that you see.

[danlavu]

I'm not a big fun of single_run. How about preferred_topology? Its still not perfect, so suggestion is welcomed. You should also not take the string but the topology marker object instead. It would be also good to allow lists. That is:

Linguistically, preferred_topology is unclear. It sounds like it makes the defined topology run first, while in reality, the new default is that it only runs against the selected topology. Is it the phrase or one of the words in particular? Maybe the negation sounds better, @mark.skip_extra_topologies? I want it to be clear that it skips/bypasses/ignores/omits additional topologies and is an exclusive/unique/single/solo/distinctive run. One of those words might make it sound better?

[pbrezina]

I think "single run" does not describe it at all. Negation would be easier to describe, I think, but harder to use (as the use case is to run 1 instead of 4).

The use case is "run these topologies if there is no topology filter". default_selection, default_topology, default_topology_selection? We probably can't find short phrase that does not have to be backed up by documentation.