Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

paramgen: add support for custom field (un)marshalling #142

Open
hariso opened this issue Nov 20, 2024 · 1 comment
Open

paramgen: add support for custom field (un)marshalling #142

hariso opened this issue Nov 20, 2024 · 1 comment
Labels
feature New feature or request

Comments

@hariso
Copy link
Contributor

hariso commented Nov 20, 2024

Feature description

The problem: Here's an example from a connector middleware struct:

type SourceWithSchemaExtraction struct {
    SchemaType schema.Type `json:"sdk.schema.extract.type" validate:"inclusion=avro" default:"avro"`
}

where schema.Type is an alias of schema.Type in conduit-commons.

When parsing SourceWithSchemaExtraction with sdk.Util.ParseConfig, we get an error saying that there's a type mismatch, because the default value is "avro" (a string), and the target value is an int. The reason is that in Config.Validate() we validate the types.

Possible solution:

  1. Remove type validation in Config.Validate() altogether. In the Connector SDK, it's called only when parsing a config, and after that we decode the map into a struct, where the types are validated too.
  2. Use one of mapstructure's hooks (link), so that developers can implement a MarshalText/UnmarshalText method and customize parsing.
@hariso
Copy link
Contributor Author

hariso commented Nov 26, 2024

We have a workaround for this, but it's a useful feature that we plan to add in one of the next releases.

@hariso hariso removed this from Conduit Roadmap Nov 27, 2024
@hariso hariso removed this from Conduit Main Nov 27, 2024
@hariso hariso added the feature New feature or request label Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant