-
Notifications
You must be signed in to change notification settings - Fork 268
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix: Wording of assertion failure closer to semantics #3324
base: master
Are you sure you want to change the base?
Conversation
This PR fixes #3216 It completely replaces the wording ".... might not hold" by "could not prove..."
@@ -297,7 +297,7 @@ public class AssertStatement : ProofObligationDescription { | |||
: $"error is impossible: {customErrMsg}"; | |||
|
|||
public override string FailureDescription => | |||
customErrMsg ?? "assertion might not hold"; | |||
customErrMsg ?? "Could not prove assertion"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
customErrMsg ?? "Could not prove assertion"; | |
customErrMsg ?? "could not prove assertion"; |
The convention in the rest of the messages is for them to start with a lower-case letter.
@@ -279,7 +279,7 @@ public class PreconditionSatisfied : ProofObligationDescription { | |||
: $"error is impossible: {customErrMsg}"; | |||
|
|||
public override string FailureDescription => | |||
customErrMsg ?? "function precondition might not hold"; | |||
customErrMsg ?? "Could not prove function precondition"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
customErrMsg ?? "Could not prove function precondition"; | |
customErrMsg ?? "could not prove function precondition"; |
The convention in the rest of the messages is for them to start with a lower-case letter.
docs/DafnyRef/UserGuide.md
Outdated
|
||
### 25.6.1. Verification debugging when verification fails {#sec-verification-debugging} | ||
|
||
Let's assume one assertion is failing ("assertion might not hold" or "postcondition might not hold"). What should you do next? | ||
Let's assume one assertion is failing ("Could not prove assertion" or "Could not prove postcondition"). What should you do next? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same case distinction here.
docs/HowToFAQ/ERROR_SeqComp.md
Outdated
@@ -1,5 +1,5 @@ | |||
--- | |||
title: "Error: function precondition might not hold" | |||
title: "Error: Cannot prove function precondition" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto.
docs/HowToFAQ/onepage.md
Outdated
@@ -2213,7 +2213,7 @@ To declare `formula` as _contravariant_ use `formula<-T>`. Then `formula<U>` is | |||
|
|||
Type parameter characteristics are discussed in [the reference manual](../DafnyRef/DafnyRef.html#sec-type-parameter-variance) | |||
|
|||
# "Error: function precondition might not hold" | |||
# "Error: Cannot prove function precondition" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And here.
@@ -7233,7 +7233,7 @@ public ForceCheckToken(IToken tok) | |||
Contract.Ensures(Contract.Result<Bpl.Ensures>() != null); | |||
|
|||
Bpl.Ensures ens = new Bpl.Ensures(ForceCheckToken.Unwrap(tok), free, condition, comment); | |||
ens.Description = new PODesc.AssertStatement(errorMessage ?? "This is the postcondition that could not be proven."); | |||
ens.Description = new PODesc.AssertStatement(errorMessage ?? "this is the postcondition that could not be proven."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you are going with the lower case style, then remove the ending periods as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is usually in a full sentence, though, of the form "Error: here is something in lowercase."
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, but the style in dafny elsewhere is predominantly to start with lower-case and not have a period, except when there are clearly multiple sentences. I'm not wedded to this style, and am open to a team discussion, but I'd like it to be consistent.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, good point. I'm fine with leaving out the periods.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
This PR is the companion of dafny-lang/dafny#3324 We established numerous times this year that the wording "might not hold" still incorrectly provides the feeling that the postcondition is incorrect. Rather than focusing on whether assertions hold or not, this PR fixes the status by focusing on the fact that the verifier was not able to prove the assertion.
Requires |
* Fix: Error wording closer to semantics This PR is the companion of dafny-lang/dafny#3324 We established numerous times this year that the wording "might not hold" still incorrectly provides the feeling that the postcondition is incorrect. Rather than focusing on whether assertions hold or not, this PR fixes the status by focusing on the fact that the verifier was not able to prove the assertion. * Fixed tests * proven => proved. Fixed tests * Fixed one more en->ed * Fixed a test * Fixed all the boogie files with messages inline * Fixed 3 more tests * Fixed last 2 CI errors * Support for the last two CI tests * Wording for invariants as well
- Added the trace of proof even in successes - Replaced intermediate "Could not prove" and "Did prove" by "Inside " to indicate this is just a trace - Use backticks to indicate quoted code - Better error message instead of "error is impossible: This is the precondition that might not hold" - First-class support of {:error} in hover messages.
…obligation descriptions.
# Conflicts: # Source/DafnyLanguageServer.Test/Lookup/HoverVerificationTest.cs
…t text Fixed CI tests
# Conflicts: # Source/DafnyLanguageServer/Language/DiagnosticErrorReporter.cs
… feat-better-error-messages # Conflicts: # Source/DafnyLanguageServer/Language/DiagnosticErrorReporter.cs
# Conflicts: # Source/DafnyCore/Verifier/ProofObligationDescription.cs # Source/DafnyCore/Verifier/Translator.cs # Source/DafnyLanguageServer.Test/Lookup/HoverVerificationTest.cs # Source/DafnyLanguageServer.Test/Synchronization/DiagnosticsTest.cs # Source/DafnyLanguageServer.Test/Various/ConcurrentInteractionsTest.cs # Source/DafnyLanguageServer/Handlers/DafnyHoverHandler.cs # Source/DafnyLanguageServer/Language/DiagnosticErrorReporter.cs # Test/allocated1/Allocated1.dfy.expect # Test/allocated1/dafny0/AutoContracts.dfy.expect # Test/allocated1/dafny0/AutoReq.dfy.expect # Test/allocated1/dafny0/BindingGuards.dfy.expect # Test/allocated1/dafny0/ChainingDisjointTests.dfy.expect # Test/allocated1/dafny0/CoPrefix.dfy.expect # Test/allocated1/dafny0/CoinductiveProofs.dfy.expect # Test/allocated1/dafny0/Datatypes.dfy.expect # Test/allocated1/dafny0/Definedness.dfy.expect # Test/allocated1/dafny0/DiamondImports.dfy.expect # Test/allocated1/dafny0/Fuel.dfy.expect # Test/allocated1/dafny0/FunctionSpecifications.dfy.expect # Test/allocated1/dafny0/IndexIntoUpdate.dfy.expect # Test/allocated1/dafny0/InductivePredicates.dfy.expect # Test/allocated1/dafny0/Inverses.dfy.expect # Test/allocated1/dafny0/LitTriggers.dfy.expect # Test/allocated1/dafny0/Matrix-OOB.dfy.expect # Test/allocated1/dafny0/MultiDimArray.dfy.expect # Test/allocated1/dafny0/NestedMatch.dfy.expect # Test/allocated1/dafny0/OpaqueFunctions.dfy.expect # Test/allocated1/dafny0/Parallel.dfy.expect # Test/allocated1/dafny0/PredExpr.dfy.expect # Test/allocated1/dafny0/Predicates.dfy.expect # Test/allocated1/dafny0/Protected.dfy.expect # Test/allocated1/dafny0/Refinement.dfy.expect # Test/allocated1/dafny0/SmallTests.dfy.expect # Test/allocated1/dafny0/Superposition.dfy.expect # Test/allocated1/dafny0/Tuples.dfy.expect # Test/allocated1/dafny0/TypeAntecedents.dfy.expect # Test/allocated1/dafny0/TypeParameters.dfy.expect # Test/allocated1/dafny0/TypeSynonyms.dfy.expect # Test/allocated1/dafny0/Unchanged.dfy.expect # Test/allocated1/dafny0/UnfoldingPerformance.dfy.expect # Test/allocated1/dafny0/one-message-per-failed-precondition.dfy.expect # Test/cli/zeroCores.dfy.expect # Test/dafny0/AutoContracts.dfy.expect # Test/dafny0/CoinductiveProofs.dfy.expect # Test/dafny0/CustomErrorMesage.dfy.expect # Test/dafny0/DefaultParameters.dfy.expect # Test/dafny0/Definedness.dfy.expect # Test/dafny0/Fuel.dfy.expect # Test/dafny0/FunctionSpecifications.dfy.expect # Test/dafny0/Include.dfy.expect # Test/dafny0/NoMoreAssume2Less2.dfy.expect # Test/dafny0/Parallel.dfy.expect # Test/dafny0/QuantificationNewSyntax.dfy.expect # Test/dafny0/Refinement.dfy.expect # Test/dafny0/RevealConsistency.dfy.expect # Test/dafny0/SmallTests.dfy.expect # Test/dafny0/Superposition.dfy.expect # Test/dafny0/one-message-per-failed-precondition.dfy.expect # Test/dafny2/SnapshotableTrees.dfy.expect # Test/dafny4/git-issue245.dfy.expect # Test/exports/OpaqueFunctions.dfy.expect # Test/git-issues/git-issue-1180b.dfy.expect # Test/git-issues/git-issue-2026.dfy.expect # Test/git-issues/git-issue-2299.dfy.expect # Test/git-issues/git-issue-2703.dfy.expect # Test/git-issues/git-issue-384.dfy.expect # Test/git-issues/git-issue-889a.dfy.expect # Test/hofs/Naked.dfy.expect # Test/traits/TraitOverride1.dfy.expect # docs/DafnyRef/UserGuide.md
…n to" where appropriate
# Conflicts: # Source/DafnyCore/Verifier/Translator.cs # Source/DafnyLanguageServer.Test/Lookup/HoverVerificationTest.cs # Test/allocated1/dafny0/AutoContracts.dfy.expect # Test/allocated1/dafny0/BindingGuards.dfy.expect # Test/allocated1/dafny0/ControlStructures.dfy.expect # Test/allocated1/dafny0/DTypes.dfy.expect # Test/allocated1/dafny0/DirtyLoops.dfy.expect # Test/allocated1/dafny0/Fuel.dfy.expect # Test/allocated1/dafny0/FunctionSpecifications.dfy.expect # Test/allocated1/dafny0/MultiSets.dfy.expect # Test/allocated1/dafny0/OpaqueFunctions.dfy.expect # Test/allocated1/dafny0/Parallel.dfy.expect # Test/allocated1/dafny0/Refinement.dfy.expect # Test/allocated1/dafny0/SmallTests.dfy.expect # Test/allocated1/dafny0/TypeAntecedents.dfy.expect # Test/allocated1/dafny0/TypeParameters.dfy.expect # Test/allocated1/dafny0/one-message-per-failed-precondition.dfy.expect # Test/dafny0/BindingGuards.dfy.expect # Test/dafny0/ByMethod.dfy.expect # Test/dafny0/ControlStructures.dfy.expect # Test/dafny0/CustomErrorMesage.dfy.expect # Test/dafny0/DTypes.dfy.expect # Test/dafny0/DefaultParameters.dfy.expect # Test/dafny0/DirtyLoops.dfy.expect # Test/dafny0/ForLoops.dfy.expect # Test/dafny0/Fuel.dfy.expect # Test/dafny0/FunctionSpecifications.dfy.expect # Test/dafny0/Include.dfy.expect # Test/dafny0/Includee.dfy.expect # Test/dafny0/Iterators.dfy.expect # Test/dafny0/NoMoreAssume2Less2.dfy.expect # Test/dafny0/OpaqueFunctions.dfy.expect # Test/dafny0/Parallel.dfy.expect # Test/dafny0/Predicates.dfy.expect # Test/dafny0/Refinement.dfy.expect # Test/dafny0/SmallTests.dfy.expect # Test/dafny0/Twostate-Verification.dfy.expect # Test/dafny0/TypeAntecedents.dfy.expect # Test/dafny0/TypeParameters.dfy.expect # Test/dafny0/one-message-per-failed-precondition.dfy.expect # Test/dafny0/snapshots/Snapshots8.run.dfy.expect # Test/dafny1/InductionOptions.dfy.expect # Test/dafny2/SnapshotableTrees.dfy.expect # Test/exports/RevealProvideAll.dfy.expect # Test/git-issues/git-issue-1207.dfy.expect # Test/git-issues/git-issue-1812.dfy.expect # Test/git-issues/git-issue-1989.dfy.expect # Test/git-issues/git-issue-19b.dfy.expect # Test/git-issues/git-issue-2026.dfy.expect # Test/git-issues/git-issue-2299.dfy.expect # Test/git-issues/git-issue-2597-verification.dfy.expect # docs/OnlineTutorial/Lemmas.2.expect # docs/OnlineTutorial/guide.13.expect
Fixes #3686 - Added the trace of proof even in successes - Replaced intermediate "Could not prove" and "Did prove" by "Inside " to indicate this is just a trace - Use backticks to indicate quoted code - Better error message instead of "error is impossible: This is the precondition that might not hold" - First-class support of {:error} in hover messages. - BONUS: Not described in the issue: Hover support for `{:error}` in failed postconditions. Acts as a precursor to #3324 by harmonizing the change made in boogie "might not hold" => "could not be proved" in preconditions and postconditions ``` before: function precondition might not hold after: function precondition could not be proved before: the calculation step between the previous line and this line might not hold after: the calculation step between the previous line and this line could not be proved before: This postcondition might not hold on a return path. after: this postcondition could not be proved on a return path ``` The screenshots of the issue are now fixed w.r.t. the described errors. ![image](https://user-images.githubusercontent.com/3601079/222830841-be6e4157-fc1d-44b1-bf8c-603efe6146e4.png) ![image](https://user-images.githubusercontent.com/3601079/222830776-6edb430d-7765-4f52-860b-c5a8f4221f7e.png) With support for `{:error}` in postconditions ![image](https://user-images.githubusercontent.com/3601079/222868101-f2956c8e-8d54-4fe5-ad54-6ca7b02cd701.png) <small>By submitting this pull request, I confirm that my contribution is made under the terms of the [MIT license](https://github.com/dafny-lang/dafny/blob/master/LICENSE.txt).</small>
# Conflicts: # Source/DafnyCore/Verifier/ProofObligationDescription.cs # Source/DafnyCore/Verifier/Translator.ClassMembers.cs # Source/DafnyCore/Verifier/Translator.ExpressionWellformed.cs # Source/DafnyCore/Verifier/Translator.cs # Source/DafnyLanguageServer.Test/Lookup/HoverVerificationTest.cs # Source/DafnyLanguageServer.Test/Various/ConcurrentInteractionsTest.cs # Source/DafnyLanguageServer/Handlers/DafnyHoverHandler.cs # Source/DafnyLanguageServer/Language/DiagnosticErrorReporter.cs # Test/allocated1/Allocated1.dfy.expect # Test/allocated1/dafny0/Array.dfy.expect # Test/allocated1/dafny0/AssertBy.dfy.expect # Test/allocated1/dafny0/AutoReq.dfy.expect # Test/allocated1/dafny0/Basics.dfy.expect # Test/allocated1/dafny0/BindingGuards.dfy.expect # Test/allocated1/dafny0/Calculations.dfy.expect # Test/allocated1/dafny0/ChainingDisjointTests.dfy.expect # Test/allocated1/dafny0/Char.dfy.expect # Test/allocated1/dafny0/CoPrefix.dfy.expect # Test/allocated1/dafny0/CoinductiveProofs.dfy.expect # Test/allocated1/dafny0/Comprehensions.dfy.expect # Test/allocated1/dafny0/ComputationsLoop.dfy.expect # Test/allocated1/dafny0/ComputationsLoop2.dfy.expect # Test/allocated1/dafny0/ComputationsNeg.dfy.expect # Test/allocated1/dafny0/ControlStructures.dfy.expect # Test/allocated1/dafny0/DTypes.dfy.expect # Test/allocated1/dafny0/Datatypes.dfy.expect # Test/allocated1/dafny0/Definedness.dfy # Test/allocated1/dafny0/Definedness.dfy.expect # Test/allocated1/dafny0/DiamondImports.dfy.expect # Test/allocated1/dafny0/DirtyLoops.dfy.expect # Test/allocated1/dafny0/Fuel.dfy.expect # Test/allocated1/dafny0/FunctionSpecifications.dfy.expect # Test/allocated1/dafny0/IndexIntoUpdate.dfy.expect # Test/allocated1/dafny0/InductivePredicates.dfy.expect # Test/allocated1/dafny0/LetExpr.dfy.expect # Test/allocated1/dafny0/LitTriggers.dfy.expect # Test/allocated1/dafny0/Maps.dfy.expect # Test/allocated1/dafny0/ModifyStmt.dfy.expect # Test/allocated1/dafny0/Modules1.dfy.expect # Test/allocated1/dafny0/MultiDimArray.dfy.expect # Test/allocated1/dafny0/MultiSets.dfy.expect # Test/allocated1/dafny0/NatTypes.dfy.expect # Test/allocated1/dafny0/NestedMatch.dfy.expect # Test/allocated1/dafny0/OpaqueFunctions.dfy.expect # Test/allocated1/dafny0/Parallel.dfy.expect # Test/allocated1/dafny0/PredExpr.dfy.expect # Test/allocated1/dafny0/Predicates.dfy.expect # Test/allocated1/dafny0/Protected.dfy.expect # Test/allocated1/dafny0/Reads.dfy.expect # Test/allocated1/dafny0/RealCompare.dfy.expect # Test/allocated1/dafny0/RealTypes.dfy.expect # Test/allocated1/dafny0/Refinement.dfy.expect # Test/allocated1/dafny0/SmallTests.dfy.expect # Test/allocated1/dafny0/StatementExpressions.dfy.expect # Test/allocated1/dafny0/SubsetTypes.dfy.expect # Test/allocated1/dafny0/Tuples.dfy.expect # Test/allocated1/dafny0/Twostate-Functions.dfy.expect # Test/allocated1/dafny0/Twostate-Verification.dfy.expect # Test/allocated1/dafny0/TypeAntecedents.dfy.expect # Test/allocated1/dafny0/TypeParameters.dfy.expect # Test/allocated1/dafny0/TypeSynonyms.dfy.expect # Test/allocated1/dafny0/Unchanged.dfy.expect # Test/allocated1/dafny0/UnfoldingPerformance.dfy.expect # Test/allocated1/dafny0/columns.dfy.expect # Test/allocated1/dafny0/one-message-per-failed-precondition.dfy.expect # Test/dafny0/ArrayElementInit.dfy.expect # Test/dafny0/ArrayElementInitERR.dfy.expect # Test/dafny0/AutoReq.dfy.expect # Test/dafny0/Basics.dfy.expect # Test/dafny0/Char.dfy.expect # Test/dafny0/Comprehensions.dfy.expect # Test/dafny0/ComprehensionsNewSyntax.dfy.expect # Test/dafny0/DefaultParameters.dfy.expect # Test/dafny0/DirtyLoops.dfy.expect # Test/dafny0/Fuel.dfy.expect # Test/dafny0/FunctionSpecifications.dfy.expect # Test/dafny0/GhostAllocations.dfy.expect # Test/dafny0/Iterators.dfy.expect # Test/dafny0/LabeledAsserts.dfy.expect # Test/dafny0/LabelsOldAt.dfy.expect # Test/dafny0/LetExpr.dfy.expect # Test/dafny0/MiscTypeInferenceTests.dfy.expect # Test/dafny0/ModifyStmt.dfy.expect # Test/dafny0/MultiSets.dfy.expect # Test/dafny0/NoReferencesVerification.dfy.expect # Test/dafny0/OpaqueTypeWithMembers.dfy.expect # Test/dafny0/Parallel.dfy.expect # Test/dafny0/SmallTests.dfy.expect # Test/dafny0/SubsetTypes.dfy.expect # Test/dafny0/Twostate-Verification.dfy.expect # Test/dafny0/TypeParameters.dfy.expect # Test/git-issues/git-issue-1207.dfy.expect # Test/git-issues/git-issue-1989.dfy.expect # Test/git-issues/git-issue-2299.dfy.expect # Test/git-issues/git-issue-2605.dfy.expect # Test/hofs/Classes.dfy.expect # Test/hofs/Field.dfy.expect # Test/hofs/FnRef.dfy.expect # Test/hofs/Frame.dfy.expect # Test/hofs/Simple.dfy.expect # Test/hofs/Twice.dfy.expect # Test/triggers/splitting-triggers-yields-better-precondition-related-errors.dfy.expect # Test/unicodechars/dafny0/Char.dfy.expect # docs/DafnyRef/UserGuide.md
Looking for an approval of an active Dafny contributor
This PR fixes #3216
It completely replaces the wording ".... might not hold" by "could not prove..."
There are many reasons for that: We already use the wording "could not prove..." in other places and, funny enough, we (I) previously justified the introduction of "might not hold" by
If we replace "might not hold" by "could not prove", then "might not hold" does not requires explanation anymore.
I got also several complaints from the wording that, although softer than the previous "assertion violation", still indicates a point where Dafny is unsure what to do.
Moreover, the implication of "assertion might not hold" places the doubt on the programmer, which can be frustrating: If the programmer wants to find Dafny's counter-examples, he would be upset to discover they are incomplete, and that their assertion is actually valid. Saying "Could not prove" shifts the doubt from the programmer to Dafny by finally recognizing that the programmer might hit a limitation of Dafny and perhaps just need to introduce things to help Dafny prove its result. It also emphasizes the need to "prove" rather than the "make the assertion hold".
Moreover, we did a first test of having "Could not prove" at the beginning of the error message, but this is uninformative as now most error messages would start by "could not prove", so we thought it was better to use the passive voice so that the kind of error can be spot in the first two words as usual.
Here are the messages that were replaced: (before/after)
I also made the error homogeneous by removing the first capital letter (errors messages are always prefixed by "Error: "), and remove the final periods because they are not needed in error messages that don't start with a capital letter (and can lead to confusion if the error message ends with a name), and could be added as post-processing if it was needed, e.g. in a report.
So I changed:
I also changed the UserGuide.md to reflect this wording, section verification debugging.
Will be merged after #3324
By submitting this pull request, I confirm that my contribution is made under the terms of the MIT license.