Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenCDC proto #7

Merged
merged 14 commits into from
Jan 3, 2024
Merged

OpenCDC proto #7

merged 14 commits into from
Jan 3, 2024

Conversation

lovromazgon
Copy link
Member

@lovromazgon lovromazgon commented Dec 21, 2023

Description

This PR adds the following:

  • OpenCDC proto files moved from conduit-connector-protocol to this repo, as they will be shared between the connector and processor SDK.
  • The repo now contains a GitHub action to validate proto files* and upload them to the schema registry on pushes to main.
  • Added FromProto and ToProto methods on Record and related types. Methods are tested and have benchmarks. Compared to the equivalent methods in conduit-connector-protocol the methods are roughly 20% faster and will allow us to relieve the pressure on GC in the future as we will be able to reuse proto structs instead of allocating new ones each time.
  • Added UnmarshalJSON method on Record which allows you to unmarshal a JSON record. Previously unmarshalling failed because opencdc.Data is an interface.

*Note that the validation action will fail in this PR, because it compares the file with the previous state in main and checks for breaking changes. The action right now doesn't find any proto files in main, so it fails.

Quick checks:

  • I have followed the Code Guidelines.
  • There is no other pull request for the same update/change.
  • I have written unit tests.
  • I have made sure that the PR is of reasonable size and can be easily reviewed.

@lovromazgon lovromazgon changed the title [WIP] OpenCDC proto OpenCDC proto Dec 22, 2023
@lovromazgon lovromazgon marked this pull request as ready for review December 22, 2023 11:09
@lovromazgon lovromazgon requested a review from a team as a code owner December 22, 2023 11:09
opencdc/fromproto.go Outdated Show resolved Hide resolved
opencdc/fromproto.go Outdated Show resolved Hide resolved
opencdc/toproto.go Outdated Show resolved Hide resolved
proto/buf.gen.yaml Outdated Show resolved Hide resolved
opencdc/toproto.go Outdated Show resolved Hide resolved
opencdc/fromproto_test.go Outdated Show resolved Hide resolved
opencdc/fromproto_test.go Outdated Show resolved Hide resolved
opencdc/fromproto_test.go Outdated Show resolved Hide resolved
Copy link
Contributor

@hariso hariso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@lovromazgon lovromazgon merged commit 680d445 into main Jan 3, 2024
4 of 5 checks passed
@lovromazgon lovromazgon deleted the lovro/opencdc-proto branch January 3, 2024 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants