feat(dev): Use newer kafka & zookeeper images to support Apple arm64 #29084
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
armenzg
added
the
Component: Developer Environment
armenzg
changed the base branch from
armenzg/dev/confluent-kafka-apple-m1
to
master
The current zookeeper image we use fails to execute on Apple's arm64 machines (see [issue][1]) for details). Only upgrading the version of both when executing on Apple arm64 machines. Fixes #28570. [1]: confluentinc/kafka-images#80 (comment)
armenzg
force-pushed
the
armenzg/devservices/kafka-zookeeper-arm64
branch
from
c8c89c1 to
e23a576
Compare
armenzg
commented
Oct 6, 2021
| @@ -158,12 +158,16 @@ runs: | |||
|
|
|||
| # TODO: Use devservices kafka. See https://github.com/getsentry/sentry/pull/20986#issuecomment-704510570 | |||
| if [ "$NEED_KAFKA" = "true" ]; then | |||
| # This is *not* the production version. Unclear reason as to why this was chosen | |||
| # https://github.com/getsentry/ops/blob/c823e62f930ecc6c97bb08898c71e49edc7232f6/cookbooks/getsentry/attributes/default.rb#L631 | |||
There was a problem hiding this comment.
Just keeping track of the discrepancy. Production uses 3.4.11.
| docker run \ | ||
| --name sentry_zookeeper \ | ||
| -d --network host \ | ||
| -e ZOOKEEPER_CLIENT_PORT=2181 \ | ||
| confluentinc/cp-zookeeper:4.1.0 | ||
|
|
||
| # This is the production version; do not change w/o changing it there as well | ||
| # https://github.com/getsentry/ops/blob/c823e62f930ecc6c97bb08898c71e49edc7232f6/cookbooks/getsentry/attributes/default.rb#L643 |
There was a problem hiding this comment.
Just taking note of it. Production uses 4.1.2-3 instead of 5.1.2.
src/sentry/conf/server.py
Outdated
| @@ -1697,6 +1698,8 @@ def build_cdc_postgres_init_db_volume(settings): | |||
| ) | |||
|
|
|||
|
|
|||
| APPLE_ARM64 = pf().startswith("mac") and pf().endswith("arm64-arm-64bit") | |||
There was a problem hiding this comment.
For the curious:
❯ python -c "import platform; print(platform.platform())"
macOS-11.6-arm64-arm-64bit| # See details https://github.com/confluentinc/kafka-images/issues/80#issuecomment-855511438 | ||
| "image": "confluentinc/cp-zookeeper:6.2.0" | ||
| if APPLE_ARM64 | ||
| else "confluentinc/cp-zookeeper:5.1.2", |
There was a problem hiding this comment.
I'm selectively upgrading the version on Apple's arm64 since there was a bug that only affects Intel (You can read more details in #28672).
I've added a card to investigate this later on.
There was a problem hiding this comment.
I would add this explanation in the comments
|
On an Apple M1 machine this can be tested with |
billyvg
reviewed
Oct 6, 2021
| # See details https://github.com/confluentinc/kafka-images/issues/80#issuecomment-855511438 | ||
| "image": "confluentinc/cp-zookeeper:6.2.0" | ||
| if APPLE_ARM64 | ||
| else "confluentinc/cp-zookeeper:5.1.2", |
There was a problem hiding this comment.
I would add this explanation in the comments
src/sentry/conf/server.py
Outdated
| @@ -9,6 +9,7 @@ | |||
| import sys | |||
| import tempfile | |||
| from datetime import timedelta | |||
| from platform import platform as pf | |||
There was a problem hiding this comment.
is there a reason to alias to pf? I would just keep it as platform
billyvg
approved these changes
Oct 6, 2021
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The current zookeeper image we use fails to execute on Apple's arm64 machines (see issue for details).
Only change the version when executing on Apple arm64 machines.
Fixes #28570.