Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Loop Handling and Reporting Issues in Z3 Constraints and one-iteration-checker #1116

Merged
merged 12 commits into from
Nov 29, 2024

Conversation

Raine-Yang-UofT
Copy link
Contributor

@Raine-Yang-UofT Raine-Yang-UofT commented Nov 21, 2024

Proposed Changes

(Describe your changes here. Also describe the motivation for your changes: what problem do they solve, or how do they improve the application or codebase? If this pull request fixes an open issue, use a keyword to link this pull request to the issue.)

  • Resolved an issue where CFG edges from the loop body to the loop condition block were ignored during the augmentation of Z3 constraints.
  • Fixed incorrect reporting of messages for while loops in the one-iteration-checker when the z3 option is enabled.

...

Screenshots of your changes (if applicable)

Type of Change

(Write an X or a brief description next to the type or types that best describe your changes.)

Type Applies?
🚨 Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change that adds functionality)
🐛 Bug fix (non-breaking change that fixes an issue) X
♻️ Refactoring (internal change to codebase, without changing functionality)
🚦 Test update (change that only adds or modifies tests)
📚 Documentation update (change that only updates documentation)
📦 Dependency update (change that updates a dependency)
🔧 Internal (change that only affects developers or continuous integration)

Checklist

(Complete each of the following items for your pull request. Indicate that you have completed an item by changing the [ ] into a [x] in the raw text, or by clicking on the checkbox in the rendered description on GitHub.)

Before opening your pull request:

  • I have performed a self-review of my changes.
    • Check that all changed files included in this pull request are intentional changes.
    • Check that all changes are relevant to the purpose of this pull request, as described above.
  • I have added tests for my changes, if applicable.
    • This is required for all bug fixes and new features.
  • I have updated the project documentation, if applicable.
    • This is required for new features.
  • I have updated the project Changelog (this is required for all changes).
  • If this is my first contribution, I have added myself to the list of contributors.

After opening your pull request:

  • I have verified that the pre-commit.ci checks have passed.
  • I have verified that the CI tests have passed.
  • I have reviewed the test coverage changes reported by Coveralls.
  • I have requested a review from a project maintainer.

Questions and Comments

(Include any questions or comments you have regarding your changes.)

Implement z3 option for one_iteration_checker
Create unit tests
Check whether the loop has at least one iteration by z3 constraints
Implement a test case that fails the current implementation
Add cycles to paths during CFG path traverse
Update unit tests
Fix an error in one-iteration-checker that causes error message not
correctly reported for while loops
@coveralls
Copy link
Collaborator

coveralls commented Nov 21, 2024

Pull Request Test Coverage Report for Build 12076085936

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 4 of 4 (100.0%) changed or added relevant lines in 2 files are covered.
  • 1 unchanged line in 1 file lost coverage.
  • Overall coverage decreased (-0.03%) to 91.968%

Files with Coverage Reduction New Missed Lines %
python_ta/cfg/graph.py 1 93.83%
Totals Coverage Status
Change from base Build 11961351945: -0.03%
Covered Lines: 3103
Relevant Lines: 3374

💛 - Coveralls

if (
current_edge.target == self.end
or current_edge.target in visited_nodes
or all(edge in visited_edges for edge in current_edge.target.successors)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can be simplified into a set subset operation

expected_while_true_path = [
{x > 5, y > 10},
{x > 5, y > 10, x + y > 15},
{},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems quite strange for this to be here. What edge is this supposed to correspond to?

All edges should, at a minimum, have the precondition constraints. So if you're getting an empty set something seems off. (Note: {} is an empty dict literal, which is even more suspicious.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Professor, I think this is the expected behavior. This edge corresponds to the highlighted one pointing from loop body to condition. Since both x and y are reassigned in the loop body, all z3 constraints that involves x and y, including precondition, are removed.

image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay that makes sense, thanks. The value is fine, but see my other comment about {} not being the appropriate data type. It's strange that the tests didn't pick up on that, you should investigate why not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Professor, thanks for the reminder. Even though we do expect an empty set here, but {} is an empty dictionary. I will definitely look into what's going on here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Professor, I’ve changed the empty dictionary to set(). The reason the tests passed previously is that they iterate through and compare each z3 constraint using zip(actual, expected). When the collection is empty—whether it’s a set or a dictionary—this iteration doesn’t execute. This behavior shouldn’t cause any issues since we’ve already verified that the lengths of expected and actual are equal beforehand.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay great 👍

@david-yz-liu david-yz-liu merged commit 9b7b4a1 into pyta-uoft:master Nov 29, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants