Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CharInfo: implement C23 rules for identifiers #540

Merged
merged 2 commits into from
Nov 8, 2023

Conversation

ehaas
Copy link
Collaborator

@ehaas ehaas commented Nov 8, 2023

This is the first part of the c23 unicode update; the second part will be adding identifier normalization.

@Vexu
Copy link
Owner

Vexu commented Nov 8, 2023

the second part will be adding identifier normalization.

There might already be a Zig library for this that we could use.

@ehaas
Copy link
Collaborator Author

ehaas commented Nov 8, 2023

It looks like maybe we can get away with just rejecting non-normalized identifiers?

https://open-std.org/JTC1/SC22/WG14/www/docs/n2836.pdf

Therefore, normalization of identifiers must happen before object files are emitted. It is either the
responsibility of programmers to provide normalized identifiers, and compilers to either warn or error
on non-normalized ones, or for compilers to normalize Unicode input. For non-Unicode input, the
transformation to normalized form is the simple table based lookup used to translate now.

This project provides code that handles the normalization form we'll need https://codeberg.org/dude_the_builder/ziglyph

@Vexu
Copy link
Owner

Vexu commented Nov 8, 2023

It looks like maybe we can get away with just rejecting non-normalized identifiers

That would be nice. We should definitely detect non-normalized identifiers since there is a simple check for it and the tables for NFC_Quick_Check={No,Maybe} are quite small.

@Vexu Vexu merged commit 50dbfb0 into Vexu:master Nov 8, 2023
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants