STAR_Test.py needs more cases

[mikefranze]

Currently we just have a basic example in our test cases. We should add more to cover all the different scenarios, ties, true ties, multiwinner, etc. See STAR_Test.py for example on how to set up the test cases. This can also help us determine if the logs that we're outputting are sufficient.

[endolith]

#2

[masiarek]

Thank you Mike. This is long overdue on my end.

[endolith]

(I started working on this with the examples I listed, by the way)

[masiarek]

Did we agree on file format to use for upload?
How do we indicate election configuration?

• Number of seats

Or, for now assume the basic scenario / format:

n,Allie,Billy,Candace, Dennis, Edith, Frank, Georgie, Harold
12:5,4,3,3,2,1,0

[mikefranze]

See the existing unit test format in STAR_Test.py for an example. The code doesn't have any functionality for reading text files or csv yet.

To add a test add another function under the STARTest class like the test_original_example function. You can change the number of winners with the optional parameter numwinners

results = STAR(S, numwinners=2)

The format might change though if you want to switch to pytest.

[endolith]

These ones don't work as I would expect:

        # 7. One highest-scoring, two or more tied for second-highest. No
        # Condorcet winner among tied, break 2nd place (B or C). A is preferred
        # in runoff regardless:
        election = [[5, 2, 3],
                    [5, 2, 3],
                    [4, 3, 2],
                    [1, 4, 3],
                    ]
        ballots = pd.DataFrame(data=election)
        results = STAR(ballots)
        assert results['elected'] == [0]

        # 8. One highest-scoring, two or more tied for second-highest. No
        # Condorcet winner among tied, break 2nd place (B or C). Both are
        # also tied in runoff regardless, and highest-scoring wins (A):
        election = [[5, 2, 3],
                    [5, 2, 3],
                    [1, 3, 2],
                    [1, 4, 3],
                    ]
        ballots = pd.DataFrame(data=election)
        results = STAR(ballots)
        assert results['elected'] == [0]

        # 11. Three or more tied for highest-scoring. One Condorcet winner in
        # tiebreaker and two or more tied for 2nd place CW. Break tie (B or C).
        # Regardless, Condorcet winner A wins runoff:
        election = [[2, 1, 1],
                    [2, 1, 1],
                    [3, 5, 5],
                    ]
        ballots = pd.DataFrame(data=election)
        results = STAR(ballots)
        assert results['elected'] == [0]

All produce the same actual result:

results['elected'][0].tied
Out[116]: [1, 2, 0]

[akdienes]

@endolith I believe according to stated tiebreakers these are all expected behaviors, since the tie in the scoring round cannot be resolved. It is a good point that, no matter how it is resolved, the same candidate will win the runoff. I suppose we could check for that case by saying if 1. there exists a candidate surely making the runoff and 2. that candidate beats all other tied in the scoring round, then we can resolve the election in their favor.

[mikefranze]

That could be resolved with just resolving the tie randomly right? Which should be an option in the event of the true tie.

But we should also try to make it clear in the output results that the tie is between 1 and 2 in the score round, and not 0, 1, and 2.

[endolith]

That could be resolved with just resolving the tie randomly right? Which should be an option in the event of the true tie.

Yes, that's what I do in my implementation, and I had listed in the comments, but I removed those comments because it seemed like you weren't doing that here. https://github.com/endolith/elsim/blob/master/tests/test_star.py#L452

[mikefranze]

@akdienes, I believe we had settled on just reporting true ties by default, but should we add an optional parameter for resolving them randomly? This would mimic real world implementations where its up the election host to determine in advance how ties are resolved.