Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resume connector change stream using the timestamp of the first event #2472

Merged
merged 2 commits into from
Nov 17, 2023

Conversation

Kerkesni
Copy link
Contributor

@Kerkesni Kerkesni commented Nov 15, 2023

There is currently a lag between when a bucket has a workflow set and when its events get picked up by the oplog populator.

This can cause issues as we might miss the first oplog events of the bucket.

We start the change stream from the first event that triggered the creation of the connector using startup.mode=timestamp to avoid loosing the first events.

Issue: BB-471

@bert-e
Copy link
Contributor

bert-e commented Nov 15, 2023

Hello kerkesni,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Status report is not available.

@Kerkesni Kerkesni force-pushed the bugfix/BB-471 branch 4 times, most recently from 45c0910 to 8082835 Compare November 15, 2023 16:35
@Kerkesni Kerkesni marked this pull request as ready for review November 15, 2023 16:50
francoisferrand added a commit to scality/Zenko that referenced this pull request Nov 15, 2023
francoisferrand added a commit to scality/Zenko that referenced this pull request Nov 15, 2023
To avoid loosing initial events, we start the change stream from the
first event that triggered the creation of the connector.

Issue: BB-471
@scality scality deleted a comment from bert-e Nov 16, 2023
@bert-e
Copy link
Contributor

bert-e commented Nov 16, 2023

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

@Kerkesni
Copy link
Contributor Author

/approve

@bert-e
Copy link
Contributor

bert-e commented Nov 17, 2023

Integration data created

I have created the integration data for the additional destination branches.

  • this pull request will merge bugfix/BB-471 into
    development/8.6
  • w/8.7/bugfix/BB-471 will be merged into development/8.7

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.5

You can set option create_pull_requests if you need me to create
integration pull requests in addition to integration branches, with:

@bert-e create_pull_requests

The following options are set: approve

@bert-e
Copy link
Contributor

bert-e commented Nov 17, 2023

Build failed

The build for commit did not succeed in branch bugfix/BB-471.

The following options are set: approve

@bert-e
Copy link
Contributor

bert-e commented Nov 17, 2023

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

  • ✔️ development/8.6

  • ✔️ development/8.7

The following branches will NOT be impacted:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.5

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

  • Any commit you add on the source branch will trigger a new cycle after the
    current queue is merged.
  • Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

@bert-e
Copy link
Contributor

bert-e commented Nov 17, 2023

I have successfully merged the changeset of this pull request
into targetted development branches:

  • ✔️ development/8.6

  • ✔️ development/8.7

The following branches have NOT changed:

  • development/7.10
  • development/7.4
  • development/7.70
  • development/8.5

Please check the status of the associated issue BB-471.

Goodbye kerkesni.

@bert-e bert-e merged commit 8b97080 into development/8.6 Nov 17, 2023
7 checks passed
@bert-e bert-e deleted the bugfix/BB-471 branch November 17, 2023 17:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants