feat(rust,python): add unset tblproperties operation

[ion-elgreco]

Description

It adds the unset tblproperties operation.

I'm currently parsing the string value as DeltaConfigKey to check if the properties that are going to be unset are valid. The reason for that is to prevent unsetting of other configurations like delta.constraints etc.

The downside here is that our ConfigKey enum should be up to date, on the other hand we could also check each property against a "protected" list of properties that can only be changed by their respective operation. If a user tries to unset a property that is in this protected list it will raise.

While writing this, I noticed that ChangeDataFeed and AppendOnly are WriterFeatures and part of the DeltaConfigKey, not sure if that can be problematic

@roeap can you give some feedback here on the approach?

I will add the rust tests once it's a bit clear which way want to go :)

Related Issue(s)

• closes Add UNSET TBLPROPERTIES #2073

[roeap]

While writing this, I noticed that ChangeDataFeed and AppendOnly are WriterFeatures and part of the DeltaConfigKey, not sure if that can be problematic

@ion-elgreco, not entirely sure if this is what you are referring to, but generally speaking we have to distinguish two things.

1. If a table supports a feature, which means readers / writers have to adhere to the specs of that feature in the protocol. This is what is reflected via readerFeatures and writerFeatures.

2. If a feature is enabled - this is what is in some cases reflected via table properties - like CDC or deletion vectors.

The downside here is that our ConfigKey enum should be up to date

I expect that the configuration keys are eventually up-streamed to kernel, where they always have to be valid anyhow. A related question would be if we want to allw setting things that delta-rs does not yet support. E.g. setting delta.deletionVecotrsEnabled to true would make the table un-readable for us right now. (though hopefully not for too long :))

[ion-elgreco]

@roeap on your last question I think we should allow that because I can think of usecases where users quickly adjust table features and properties with delta-rs to then just write with spark-delta.

Back on the main point of my note, what order is done to check for example whether CDC is enabled below table feature protocol and above?

Let's take this example:

If we are below V7 protocol we only check the config so we can safely unset the property.

But what if you have a table at V6 with enableChangeDataFeed=true, then we upgrade to V7. Will that config setting be translated into a CDC writer feature enabled?

Also what if we have a table with table feature CDC enabled, then I assume any setting or unsetting the table property: enableChangeDataFeed will not affect it

[roeap]

Thanks, left a few small comments.

[roeap]

here is was a little bit unsure what behaviour I would expect. From the name, my first instinct was that this would set the property, not append it to any existing ones.

What do you think?

[ion-elgreco]

So, I was thinking people could either chain multiple .with_property().with_property() to add the configkeys or they use the with_properties to pass a Vec.

[ion-elgreco]

Putting this in draft until I address some properties that are apparently not allowed to be unset :)