Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added script to install confluent_kafka module and do a sanity test with multiple python versions. #1743

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 18 additions & 6 deletions tools/RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ v0.11.4rc3 for the 3rd v0.11.4 release candidate.
With the addition of prebuilt binary wheels we make use of Semaphore CI
to build OSX, Linux and Windows binaries which are uploaded to build's
artifact directory. These artifacts are downloaded and then uploaded manually
to PyPi.
to PyPI.

**Note**: Python package versions use a lowercase `rcN` suffix to indicate
release candidates while librdkafka uses `-RCN`. The Python format
Expand All @@ -36,10 +36,10 @@ time line:
* Create test tag to trigger CI builds, test.
* Work out any release build issues.
* librdkafka is released, update librdkafka version.
* Create release candidate tag, wait for CI builds, deploy to test-PyPi.
* Create release candidate tag, wait for CI builds, deploy to test-PyPI.
* Create PR, get it reviewed.
* When it is time for final release, merge the PR.
* On master, create release tag, wait for CI builds, deploy to PyPi.
* On master, create release tag, wait for CI builds, deploy to PyPI.
* Update confluent.io docs.
* Announce release.

Expand Down Expand Up @@ -290,7 +290,7 @@ With the PR merged to master, check out and update master:
Now go back to 5.1 and start the final RELEASE ITERATION.


### 5.6. Upload wheel packages to PyPi
### 5.6. Upload wheel packages to PyPI

**CANDIDATE ITERATION:** To upload binary packages to test.pypi.org, use:

Expand All @@ -301,7 +301,7 @@ Now go back to 5.1 and start the final RELEASE ITERATION.
$ twine upload dl-v0.11.4rc1/*


### 5.7. Upload source packages to PyPi
### 5.7. Upload source packages to PyPI

When uploading source packages make sure to have checked out the correct tag
and that you do not have any uncommited modifications and that the `build/`
Expand All @@ -316,7 +316,19 @@ directory is empty.
$ python setup.py sdist upload


### 5.8. Verify installation from PyPi
### 5.8. Verify installation from PyPI

#### New way

Use `tools/test-released-wheels` script to test installation from PyPI

**CANDIDATE ITERATION:**
$ ./tools/test-released-wheels 0.11.4rc1 test

**RELEASE ITERATION:**
$ ./tools/test-released-wheels 0.11.4

#### Old way

In the same virtualenv as created above:

Expand Down
56 changes: 56 additions & 0 deletions tools/test-released-wheels.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
#!/bin/bash

set -e

# Check if an argument is present
if [ -z "$1" ]; then
echo "Please provide a version of confluent_kafka to install."
exit 1
fi

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should move $1 and $2 to params for better readability and use them instead

CONFLUENT_KAFKA_VERSION=$1
USE_TEST_PYPI=$2

# Check if pyenv is installed
if ! command -v pyenv &> /dev/null; then
echo "pyenv is not installed. Please install pyenv before running this script."
exit 1
fi

# List of Python versions to run the code with
python_versions=("3.7.17" "3.8.18" "3.9.18" "3.10.11" "3.11.7" "3.12.1" "3.13.0")

eval "$(pyenv init -)"


# Loop through each Python version and execute the code
for version in "${python_versions[@]}"; do

echo -e "\e[1;34mRunning with Python version $version in env $environment_name \e[0m"

echo Installing python $version
pyenv install -s $version > /dev/null 2>&1

echo Using $version
pyenv shell $version > /dev/null 2>&1

echo Python version: `python -V`
echo Pip version: `pip -V`

pip install --upgrade pip

echo "Uninstalling confluent_kafka if already installed"
pip uninstall -y confluent_kafka > /dev/null 2>&1 || true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add venv here to not make changes in the global space?


if [ "$2" = "test" ]; then
echo "Installing confluent_kafka from test PyPI"
pip install --index-url https://test.pypi.org/simple/ confluent_kafka==$1 > /dev/null 2>&1
else
echo "Installing confluent_kafka"
pip install confluent_kafka==$1
fi

echo "Testing confluent_kafka"
output=$(python -c 'import confluent_kafka as ck ; print("py:", ck.version(), "c:", ck.libversion())')
echo -e "\e[1;32m$output\e[0m"
echo "Successfully tested confluent_kafka version $1 with Python version $version"
echo
echo
done