Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: On Destination - Issue with loading credentials via pipeline yaml. #212

Open
anna-cross opened this issue Apr 24, 2024 · 0 comments
Open
Labels
bug Something isn't working

Comments

@anna-cross
Copy link

anna-cross commented Apr 24, 2024

Bug description

When running conduit with the pipeline yaml below, the connector seems to be unable to authenticate and validate the PK provided.


version: 2.0
pipelines:
  - id: clickhouse-to-google-pub
    status: running
    connectors:
      - id: sf.in
        type: source
        plugin: standalone:[email protected]
        settings:
          url: # ex https://default:[email protected]:8443/default?secure
          table: #test
          orderingColumn: #name
          batchSize: #10
      - id: sf.out
        type: destination
        plugin: standalone:gcp-pubsub@0299d3b-dirty
        settings:
          privateKey: # the entirity of private key, must start with -----BEGIN PRIVATE KEY----- and end with -----END PRIVATE KEY-----\n
          clientEmail: # ex - [email protected]
          projectId: # meroxa
          topicId: # test

However when passing following json as options in code manually in client/pubsub.go, the same private key provided seems to work.

`options := `{
		"type": "service_account",
		"project_id": "meroxa",
		"private_key_id": "b5c7aa1bc56e1759bb781ca53406016b6c587cc1",
		"private_key": "PRIVATEKEY",
		"client_email": "[email protected]",
		"client_id": "109884669801399525038",
		"auth_uri": "https://accounts.google.com/o/oauth2/auth",
		"token_uri": "https://oauth2.googleapis.com/token",
		"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
		"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/meroxatest%40meroxa.iam.gserviceaccount.com",
		"universe_domain": "googleapis.com"
	  }
	  `
	cli, err := pubsub.NewClient(ctx, cfg.ProjectID, option.WithCredentialsJSON([]byte(options)))
`

Steps to reproduce

  1. Use the pipeline yaml above in your local conduit with clickhouse and pubsub credentials
  2. Try passing your pubsub PK in the yaml

Version

v0.1.0

@anna-cross anna-cross added the bug Something isn't working label Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant