Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update all connectors to latest sdk #173

Closed
27 of 57 tasks
raulb opened this issue Aug 12, 2024 · 5 comments
Closed
27 of 57 tasks

Update all connectors to latest sdk #173

raulb opened this issue Aug 12, 2024 · 5 comments

Comments

@raulb
Copy link
Member

raulb commented Aug 12, 2024

⚠️ We need to make sure those connectors are also released once upgraded.

Important

After updating the SDK run this script to fix 95% of the code automatically: ConduitIO/conduit#1433 (comment)

@raulb raulb converted this from a draft issue Aug 12, 2024
@raulb raulb self-assigned this Aug 12, 2024
@hariso
Copy link
Contributor

hariso commented Aug 12, 2024

For NATS JetStream, the work is done here: conduitio-labs/conduit-connector-nats-jetstream#137.

@raulb
Copy link
Member Author

raulb commented Aug 12, 2024

Generated by using this script

#!/bin/bash

# Check that a tag was provided as an argument
if [ -z "$1" ]; then
  echo "Usage: $0 <tag>"
  exit 1
fi

connectors=$(curl -s https://conduit.io/connectors.json | jq -c '.[].url')
connectors_outdated=()
connectors_without_sdk=()
contributor_connectors_outdated=()

for url in $(echo "$connectors"); do
  github_url=${url#*github.com/}
  github_url=${github_url%%\"*}
  raw_url="https://raw.githubusercontent.com/$github_url/main/go.mod"

  if ! curl -sL "$raw_url" >/dev/null 2>&1; then
    connectors_without_sdk+=("https://github.com/$github_url")
    continue
  fi

  go_mod=$(curl -s "$raw_url")

  commit=$(awk -F' ' '/github.com\/conduitio\/conduit-connector-sdk/{print $NF}' <<< "$go_mod")

  if [ "$commit" = "github.com/conduitio/conduit-connector-sdk" ]; then
    commit=$(awk -F' ' '/require github.com\/conduitio\/conduit-connector-sdk/{print $3}' <<< "$go_mod")
  fi

  if ! grep -q "github.com/conduitio/conduit-connector-sdk $1" <<< "$go_mod"; then
    if [ -z "$commit" ]; then
      connectors_without_sdk+=("https://github.com/$github_url")
    else
      if ! grep -q "conduitio-labs" <<< "$github_url" && ! grep -q "ConduitIO" <<< "$github_url"; then
        contributor_connectors_outdated+=("https://github.com/$github_url @ $commit")
      else
        connectors_outdated+=("https://github.com/$github_url @ $commit")
      fi
    fi
  else
    echo "- [x] https://github.com/$github_url @ $commit"
  fi
done

if [ "${#connectors_outdated[@]}" -gt 0 ]; then
  for connector in "${connectors_outdated[@]}"; do
    echo "- [ ] $connector"
  done
fi

if [ "${#contributor_connectors_outdated[@]}" -gt 0 ]; then
  echo "
### Connectors owned by contributors:
"
  for connector in "${contributor_connectors_outdated[@]}"; do
    echo "- [ ] $connector"
  done
fi

if [ "${#connectors_without_sdk[@]}" -gt 0 ]; then
  echo "
### Connectors without conduit-connector-sdk:
"
  for connector in "${connectors_without_sdk[@]}"; do
    echo "- [ ] $connector"
  done
fi
exit 0

@alarbada
Copy link
Contributor

@raulb do you mind if I help you out with some of my connectors?

Specifically with these:

  • pinecone
  • activemq artemis & classic
  • pulsar
  • rabbitmq

Also, I took a look at some of these:

  • Pinecone needs the {{ secrets.API_KEY }} set, for some reason it is not there anymore.
  • activemq artemis has many dependabot prs blocked, might be misconfigured.
  • activemq classic got a failed to create review: GraphQL: GitHub Actions is not permitted to approve pull requests. (addPullRequestReview), might also be misconfigured.

@raulb
Copy link
Member Author

raulb commented Aug 12, 2024

@alarbada please do so!!! thank you!

@raulb
Copy link
Member Author

raulb commented Oct 15, 2024

Closing this one since we'll address the upgrade to the latest SDK for each connector independently.

@raulb raulb closed this as completed Oct 15, 2024
@github-project-automation github-project-automation bot moved this from In Progress to Done in Conduit Main Oct 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

3 participants