Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add schema syncing implementation #273

Merged
merged 4 commits into from
Oct 22, 2024
Merged

Add schema syncing implementation #273

merged 4 commits into from
Oct 22, 2024

Conversation

andrewstucki
Copy link
Contributor

@andrewstucki andrewstucki commented Oct 22, 2024

This adds a high-level schema syncing client for CRD --> schema creation. Here's how it roughly works:

  1. A user calls Sync with the CRD
  2. The client pulls both schema data and compatability levels and merges them into a comparable in-memory object.
  3. The client also pulls a special SchemaHash field on the CRD status which is just a non-cryptographic hash of the contents of the schema text string that has previously been synced (since the Redpanda schema creation operations have some normalization of the schema string and may no longer be 1-1 comparable with what's in the CRD)
  4. All of these values are compared against what's currently in the schema CRD itself
  5. If anything changes, then we create a new schema version and hash the contents of its schema text, returning both all of the version numbers of the schema and the hash of its contents.
  6. The above will be used in patching the status of the CRD with both the hash of the schema and all of the versions of the schema that we've created in the registry.

Delete operations simply delete every version of the schema by their subject, and since a CRD deletion should leave things in a clean state, it issues both a soft and then hard delete.

This is the second broken-out piece of #263 with only the controller implementation and acceptance test additions remaining.

Copy link
Contributor

@chrisseto chrisseto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking

operator/pkg/client/factory.go Outdated Show resolved Hide resolved
operator/pkg/client/schemas/schema.go Outdated Show resolved Hide resolved
operator/pkg/client/schemas/schema.go Outdated Show resolved Hide resolved
operator/pkg/functional/compare.go Outdated Show resolved Hide resolved
operator/pkg/functional/compare.go Outdated Show resolved Hide resolved
operator/pkg/functional/compare.go Outdated Show resolved Hide resolved
operator/pkg/client/spec_sasl.go Outdated Show resolved Hide resolved
operator/pkg/client/schemas/syncer_test.go Outdated Show resolved Hide resolved
}

// Delete removes the schema in Redpanda.
func (s *Syncer) Delete(ctx context.Context, o *redpandav1alpha2.Schema) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this need to be idempotent (Should it eat not found errors)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The general pattern I've followed most places is to just return the error from the client and then swallow it if we so desire in the controller by saying, "hey this is a non-retryable 'terminal' error while we're trying to clean up, so just ignore it"

@andrewstucki andrewstucki merged commit 4976e07 into main Oct 22, 2024
5 checks passed
@andrewstucki andrewstucki deleted the schema-reg-2 branch October 22, 2024 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants