Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

echo command ignores env variables and resources #3043

Open
peczenyj opened this issue Nov 28, 2024 · 0 comments
Open

echo command ignores env variables and resources #3043

peczenyj opened this issue Nov 28, 2024 · 0 comments

Comments

@peczenyj
Copy link
Contributor

peczenyj commented Nov 28, 2024

I was expecting that, running the echo command, it can return the full configuration including the resources and values expanded from env vars.

Example

If I have this .env fime

BUFFER_MEMORY_LIMIT=2048

this config.yaml

input:
  resource: the_input

pipeline:
  processors:
    - mapping: root = content().uppercase()

buffer:
  memory:
    limit: ${BUFFER_MEMORY_LIMIT:1024}

output:
  resource: the_output

and this resources.yaml file

---
input_resources:
  - label: the_input
    stdin: {}

output_resources:
  - label: the_output
    stdout: {}

By running the latest version of redpandadata/connect I got

$ docker run --env-file ./foo/.env --net host -v ./foo/:/c ${IMAGE} echo -r "/c/resources.yaml" /c/config.yaml

http:
  enabled: true
  address: 0.0.0.0:4195
  root_path: /benthos
  debug_endpoints: false
  cert_file: ""
  key_file: ""
  cors:
    enabled: false
    allowed_origins: []
  basic_auth:
    enabled: false
    realm: restricted
    username: ""
    password_hash: ""
    algorithm: sha256
    salt: ""
input:
  resource: the_input
buffer:
  memory:
    limit: 1024           # should be 2048 no?
pipeline:
  threads: -1
  processors:
    - mapping: root = content().uppercase()
output:
  resource: the_output
input_resources: []       # no signal of the_input
processor_resources: []
output_resources: []      # no signal of the_output 
cache_resources: []
rate_limit_resources: []
logger:
  level: INFO
  format: logfmt
  add_timestamp: false
  level_name: level
  timestamp_name: time
  message_name: msg
  static_fields:
    '@service': redpanda-connect
  file:
    path: ""
    rotate: false
    rotate_max_age_days: 0
metrics:
  prometheus: {}
  mapping: ""
tracer:
  none: {}
shutdown_delay: 0s
shutdown_timeout: 20s
tests: []
redpanda:
  seed_brokers: []
  client_id: benthos
  tls:
    enabled: false
    skip_cert_verify: false
    enable_renegotiation: false
    root_cas: ""
    root_cas_file: ""
    client_certs: []
  sasl: []
  metadata_max_age: 5m
  pipeline_id: ""
  logs_topic: ""
  logs_level: info
  status_topic: ""
  idempotent_write: true
  timeout: 10s
  max_message_bytes: 1MB
  broker_write_max_bytes: 100MB

but if I call the endpoint /debug/config/yaml I can retrieve the full configuration.

based on the help message

   This simple command is useful for sanity checking a config if it isn't
   behaving as expected, as it shows you a normalised version after environment
   variables have been resolved:

I should see, at least, a different value in the buffer memory limit, no ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant