SNOW-1803811: Allow mixed-case field names for struct type columns #2640
Conversation
CHANGELOG.md
Outdated
| - Added support for mixed case field names in struct type columns. | ||
|
|
There was a problem hiding this comment.
does this only affected structured types or all StructTypes? If it is the latter, then it would be a bcr, no?
There was a problem hiding this comment.
This only affects structured types because currently semi-structured objects don't become StructType columns, but instead get converted to MapType for some reason.
| f"{field.name} {convert_sp_to_sf_type(field.datatype)}" | ||
| f"{field.raw_name} {convert_sp_to_sf_type(field.datatype)}" |
There was a problem hiding this comment.
I kinda have the same problem as Afroz's on BCR.. previous we have name upper cased but now we do not, this won't break users when user references keys in the map object?
There was a problem hiding this comment.
As far as I know nobody is using StructType columns yet due to them requiring structured types be enabled for their account.
| @@ -106,7 +106,7 @@ def _create_test_dataframe(s): | |||
| StructType( | |||
| [ | |||
| StructField("A", StringType(16777216), nullable=True), | |||
| StructField("B", DoubleType(), nullable=True), | |||
| StructField('"b"', DoubleType(), nullable=True), | |||
There was a problem hiding this comment.
do we need extra double quote here?
| @@ -407,6 +407,7 @@ class ColumnIdentifier: | |||
| """Represents a column identifier.""" | |||
|
|
|||
| def __init__(self, normalized_name: str) -> None: | |||
| self.raw_name = normalized_name | |||
There was a problem hiding this comment.
If this is internal usage only then I prefer marking them as private _raw_name
| @@ -487,6 +488,10 @@ def name(self) -> str: | |||
| """Returns the column name.""" | |||
| return self.column_identifier.name | |||
|
|
|||
| @property | |||
| def raw_name(self) -> str: | |||
There was a problem hiding this comment.
same as the the public vs private api comment
Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.
Fixes SNOW-1803811
Fill out the following pre-review checklist:
Please describe how your code solves the related issue.
This PR addresses an issue with StructType columns where names for StructFields are always uppercased. Snowflake always uppercases column names, but fields that are internal to StructType columns do not have casing enforced. Ideally we would not treat StructFields as column objects, but separating out that responsibility would require a BCR.
This change gets around the BCR requirement by additionally storing the unmodified version of a field name inside of the ColumnIdentifier class and using that when generating the sql for StructType columns instead of the column formatted name.