Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add wide & unicode string literal support #527

Merged
merged 26 commits into from
Oct 21, 2023
Merged

Add wide & unicode string literal support #527

merged 26 commits into from
Oct 21, 2023

Conversation

ehaas
Copy link
Collaborator

@ehaas ehaas commented Oct 20, 2023

Adds support for L"" / u"" / U"" literals.

String contents are written to p.retained_strings

Also fixes 2 unrelated bugs:

  1. Max diagnostics for a char or string literal is currently capped at 4; we drop any after that. So if 4 warnings happened first, followed by an error, the error would get dropped. New behavior is to replace the last warning if adding an error fails.
  2. Fix a crash when passing a non-string to an attribute that expects a string (introduced in Value: Use a start/end range instead of slice for byte values #503)

Copy link
Owner

@Vexu Vexu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on these! The invalid token is now almost exclusively used for incorrectly encoded bytes.

Would you mind taking a look at the remaining cases? I think some of them can simply be marked unreachable thanks to line splicing and the others could be changed to empty_char_literal and unterminated_{char,string}_literal with better error messages.

The split between CharKind and StringKind with nearly identical APIs seems a bit odd to me but if you think it's better then it's fine.

@ehaas
Copy link
Collaborator Author

ehaas commented Oct 20, 2023

I think the only reason I made them separate is to prevent string and char literals from concatenating but you're right, they can probably be combined. I'll take a look at that and the invalid token.

@Vexu Vexu merged commit 070fbf8 into Vexu:master Oct 21, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants