Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the stack trace of check-expect forms #228

Merged
merged 6 commits into from
Nov 9, 2024

Conversation

shhyou
Copy link
Collaborator

@shhyou shhyou commented Oct 25, 2024

In #lang SLs, check-expect and similar forms are not annotated by errortrace.
This PR adds the errortrace:annotate syntax property to request for annotation.

Before:

#lang htdp/isl+
;; error at test-engine/racket-tests.rkt:XXX:XX
(check-expect 0 zero?)
;; check failure at lang/private/teachprims.rkt:XXX:XX
(check-expect zero? 0)

After: error at line 3, column 0 and line 5, column 0.


In addition to stack trace fixes, this PR refines and fixes somes check-form error messages:

  • check-random and check-expect should NOT have identical error messages.
  • Have check-expect messages refer to "the second argument" like other check-forms.
  • Format the third argument of check-within with ~s instead of ~a for otherwise strings like "NNN.mm" would be printed like numbers (i.e. just NNN.mm).

When reporting "not given an inexact number" in check-within, the given
has to be formatted with ~s since it could have been a string:

    (check-within (sqrt 2) 3/2 "0.1")
@@ -109,9 +109,9 @@

(define (do-check-expect test expected src)
(error-check (lambda (v) (if (number? v) (exact? v) #t))
expected INEXACT-NUMBERS-FMT #t)
expected INEXACT-NUMBERS-FMT #t (list expected expected))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why expected is listed twice. This feels wrong.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the error message to:

check-expect: cannot compare inexact numbers, but the second argument is ~a. Try (check-within test ~a error-range).

Where the ", but the second argument is ~a" part is new.
So there are two ~a referring to expected now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you made sure that this change still works with the stepper?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup! This PR does not change the expansion but only attaches an extra syntax property.

@mfelleisen
Copy link
Contributor

Thanks for getting this started. I don't have time to review the changes inside the file yet but three of them raise questions already.

@mfelleisen
Copy link
Contributor

Can you do me a favor and check in Guillaume's paper whether "advice on how to fix things should go into error messages"? My recollection is that we should not do so unless obvious. I consider the suggestion in the modified inexact error message non-obvious.

@shhyou
Copy link
Collaborator Author

shhyou commented Oct 25, 2024

Can you do me a favor and check in Guillaume's paper whether "advice on how to fix things should go into error messages"? My recollection is that we should not do so unless obvious. I consider the suggestion in the modified inexact error message non-obvious.

Sort of -- Guillaume Marceau (et al.) say

  • Error messages should not propose solutions.

but also say that

The second warns against proposing corrections to syntax errors.

which is not directly targeting this error message. The suggestion in the message has been there since 2010, do you want to remove it?

@mfelleisen
Copy link
Contributor

mfelleisen commented Oct 25, 2024 via email

@shhyou
Copy link
Collaborator Author

shhyou commented Oct 26, 2024

Tests added to module-lang-test.rkt in racket/drracket#689.

@mikesperber
Copy link
Member

@rfindler and I had a conversation about this issue at ICFP - Robby, could you look at this?

@shhyou
Copy link
Collaborator Author

shhyou commented Nov 3, 2024

Is this ready to be merged?

@rfindler
Copy link
Member

rfindler commented Nov 3, 2024

I'm sorry, I've lost track of this conversation. I don't see any issues from a look at the diff, but if there is something specific I should be remembering, could you remind me @mikesperber ?

@shhyou shhyou merged commit 3eab980 into racket:master Nov 9, 2024
2 checks passed
@shhyou shhyou deleted the check-form-stack branch November 9, 2024 15:58
shhyou added a commit to racket/drracket that referenced this pull request Nov 11, 2024
 (#689)

* Add some tests for racket/htdp#228, racket/htdp#229, and racket/htdp#230 to
  `tests/drracket/module-lang-test`.

  + Tests of racket/htdp#229 should first save the buffer to disk before running
  + Some racket/htdp#229 tests are ported from htdp-test:intm-lam.rktl

* Update `tests/drracket/language-test` to match racket/htdp#229.

* Manually check for empty stderr in tests/drracket/module-lang-test

  Somehow `-e`/`--check-stderr` does not work with `tests/drracket/module-lang-test`
  (perhaps due to using `test-log` + `exit 0` in `fire-up-drracket-and-run-tests`?)

* Test utils: sleep for 0.1s after printing error msg

  In tests/drracket/module-lang-test, stderr goes through a pipe to be
  checked for the absence of error messages. Therefore, sleep for 0.1s
  before existing to let the background thread pipe the messages to terminal.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants