Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: configurable extra allowable metadata field attributes #73

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

hughevans
Copy link

According to the Avro specification any attribute can be added to a field and used as extra metadata provided it does not clash with the other attributes specified.

To enable the use of specific custom metadata attributes you can define them with:

Avro::Builder.extra_metadata_attributes(
  :sensitivity_level,
  :deprecated_by,
  :reference
)

I also solved #9 when adding this.

@hughevans hughevans requested a review from a team as a code owner December 19, 2024 04:35
@hughevans hughevans force-pushed the extra-metadata-dsl branch 3 times, most recently from 97af4f8 to fefefa3 Compare December 19, 2024 04:48
@hughevans hughevans changed the title feat: configurable extra allowable metadata fields feat: configurable extra allowable metadata field attributes Dec 19, 2024

# Define extra allowable metadata attributes for fields
def self.extra_metadata_attributes(*attrs)
Avro::Builder::Field.extra_metadata_attributes(attrs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the spec supports metadata attributes on all JSON object in the serialized Avro schema. Is there a generalized version of this change that supports attributes on constructs other than just fields e.g. records?

Copy link
Author

@hughevans hughevans Dec 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jturkel Yes, I can make that change no problem. Would you rather seperate config blocks for declaring the attributes for for each object type? Or is all together ok?

Avro::Builder.extra_metadata_attributes(:sensitivity_level, :deprecated_by)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with how people are using metadata on Avro schemas but my instinct is to keep it simple for now with a single set of attributes that applies to all types/fields. We can always extend this in the future to support both global and type specific metadata attributes.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jturkel Fantastic. I’ve pushed that change. There is the possibility still to have individual setters for records vs fields in the future if you want too.

# Define extra allowable metadata attributes for fields
def self.extra_metadata_attributes(*attrs)
Avro::Builder::Field.extra_metadata_attributes(attrs)
Avro::Builder::Record.extra_metadata_attributes(attrs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the Avro spec supports metadata on all types rather than just record types. If that's correct, then should this be on Avro::Builder::Types::Type rather than Avro::Builder::Types::RecordType?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jturkel sorry I haven’t got back to this because I hit some code here that could benefit from a refactor. Any possibility we could merge this as is and then circle back later?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants