Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quoted variable values incorrectly converted to integers #453

Closed
jamest-gaia opened this issue May 2, 2023 · 6 comments · Fixed by #511
Closed

Quoted variable values incorrectly converted to integers #453

jamest-gaia opened this issue May 2, 2023 · 6 comments · Fixed by #511
Assignees
Labels
impact/reliability Something that feels unreliable or flaky kind/bug Some behavior is incorrect or out of spec resolution/fixed This issue was fixed

Comments

@jamest-gaia
Copy link

jamest-gaia commented May 2, 2023

What happened?

My Pulumi.develop.yaml file

config:
  aws:profile: 028111111111_AWSAdministratorAccess
  aws:region: ap-southeast-2
  myprojectname:aws_account_id: "028111111111" # I also tried single quotes here
  myprojectname:dns_domain: "redacted"
  myprojectname:aws_route53_zoneid: "redacted"

When I attempt to use this value aws_account_id in the code, it strips the zero. The stripping doesn't happen to the aws:profile. Perhaps because it contains some alpha characters too, or perhaps because the aws provider handles the variable differently.

Someone on Slack (Scott Lowe) suggested:

Do you have a reference to that configuration value in the config block of your Pulumi.yaml file? If so, you can specify a type property there (set it to “string”), and that may help resolve the issue.

I tried that, perhaps I'm missing something simple.
So this value aws_account_id is different for each stack hence its original location in the stack files i.e. Pulumi.develop.yaml, Pulumi.staging.yaml etc.
I tried to implement your suggestion by adding a couple of lines under config: at the top of Pulumi.yaml like so:
config:

  aws_account_id:
    type: string

Then I printed the output like so:

outputs:
  # Debug printouts
  accIDString: "${aws_account_id}"

And it's still getting converted.

Outputs:
  + accIDString               : 2.809102044e+10

I also tried a slightly different way, but this just printed the variable name

config:
  awsAccountID: 
    type: string
    default: ${aws_account_id}
...
outputs:
  # Debug printouts
  accIDString: "${awsAccountID}"
~~~~~~~~
Outputs:
  + accIDString               : "${aws_account_id}"

Expected Behavior

I expected variable values entered with quotes to stay as strings

Steps to reproduce

  1. Set a variable with an integer as the value, but cast it as a string.
  2. Try to use the value in the yaml code. It will get converted into a 'number'

This is where I tried to use it (in an IAM policy)

 # Allow to send emails with SES.
            - Action:
                - ses:SendEmail
              Effect: Allow
              Resource: "*" # "arn:aws:ses:${aws:region}:${aws_account_id}:identity/${ses_DomainIdentity.domain}" # can't use this due to bug with yaml stripping leading 0 from aws_account_id

Output of pulumi about

CLI          
Version      3.68.0 # (originally reported on 3.65.1)
Go Version   go1.20.4 # (originally reported on go1.20.3)
Go Compiler  gc

Plugins
NAME  VERSION
yaml  unknown

Host     
OS       debian
Version  11.5
Arch     x86_64

This project is written in yaml

Additional context

Have tried double and single quotes, have tried !!str '5432'

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

@jamest-gaia jamest-gaia added kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels May 2, 2023
@justinvp justinvp added impact/reliability Something that feels unreliable or flaky and removed needs-triage Needs attention from the triage team labels May 3, 2023
@jamest-gaia
Copy link
Author

jamest-gaia commented May 22, 2023

Another example of this happening.
Variable definition (I've tried both double and single quotes):
image

Variable usage:
image

Have also tried:

- name: POSTGRES_PORT
  value: !!str "${confPOSTGRES_PORT}"

Error:

 aws:ecs:TaskDefinition (myFarGateService):
    error: aws:ecs/taskDefinition:TaskDefinition resource 'myFarGateService' has a problem: ECS Task Definition container_definitions is invalid: Error decoding JSON: json: cannot unmarshal number into Go struct field KeyValuePair.Environment.Value of type string. Examine values at 'TaskDefinition.ContainerDefinitions'.

Not sure if this is related? yaml/pyyaml#98

Update: ended up hardcoding it.

image

@jamest-gaia
Copy link
Author

jamest-gaia commented May 26, 2023

Bump on this. It seems like it happens with booleans as well. Pulumi somehow converts things into a type.
It's making me unable to deploy to ECS...
Again, I've tried with double quotes, single quotes, and also trying to cast it using yaml !!str keyword
Variable definition:

config:
  biosys:ALLOW_PUBLIC_REGISTRATION: !!str 'True'

Code:

environment:
            # https://www.pulumi.com/registry/packages/awsx/api-docs/ecs/fargatetaskdefinition/#taskdefinitionkeyvaluepair
            - name: ALLOW_PUBLIC_REGISTRATION
              value: !!str "${ALLOW_PUBLIC_REGISTRATION}"

Error:

aws:ecs:TaskDefinition (biosys-440-service):
    error: aws:ecs/taskDefinition:TaskDefinition resource 'biosys-440-service' has a problem: ECS Task Definition container_definitions is invalid: Error decoding JSON: json: cannot unmarshal bool into Go struct field KeyValuePair.Environment.Value of type string. Examine values at 'TaskDefinition.ContainerDefinitions'.

@jamest-gaia
Copy link
Author

Seems like using this yaml syntax solves it

 biosys:ALLOW_PUBLIC_REGISTRATION: |-
   "True"

I found this out by using the pulumi config set command to add the variable to the stack file instead of adding it myself manually.

@tgummerer tgummerer self-assigned this Oct 9, 2023
tgummerer added a commit to pulumi/pulumi that referenced this issue Oct 11, 2023
Currently we're only passing the config as strings through the
RunRequest method.  This means that the program handling the RPC
can't know the right types, and has to infer it.  This leads to issues
such as pulumi/pulumi-yaml#453.

Passing the types through we'll be able to parse the strings correctly
on the receiving end.
tgummerer added a commit to pulumi/pulumi that referenced this issue Oct 13, 2023
Currently we're only passing the config as strings through the
RunRequest method.  This means that the program handling the RPC
can't know the right types, and has to infer it.  This leads to issues
such as pulumi/pulumi-yaml#453.

Passing the types through we'll be able to parse the strings correctly
on the receiving end.
tgummerer added a commit to pulumi/pulumi that referenced this issue Oct 13, 2023
Currently we're only passing the config as strings through the
RunRequest method.  This means that the program handling the RPC
can't know the right types, and has to infer it.  This leads to issues
such as pulumi/pulumi-yaml#453.

Passing the types through we'll be able to parse the strings correctly
on the receiving end.
tgummerer added a commit to pulumi/pulumi that referenced this issue Oct 13, 2023
Currently we're only passing the config as strings through the
RunRequest method.  This means that the program handling the RPC
can't know the right types, and has to infer it.  This leads to issues
such as pulumi/pulumi-yaml#453.

Passing the types through we'll be able to parse the strings correctly
on the receiving end.
tgummerer added a commit to pulumi/pulumi that referenced this issue Oct 13, 2023
Currently we're only passing the config as strings through the
RunRequest method.  This means that the program handling the RPC
can't know the right types, and has to infer it.  This leads to issues
such as pulumi/pulumi-yaml#453.

Passing the types through we'll be able to parse the strings correctly
on the receiving end.
tgummerer added a commit to pulumi/pulumi that referenced this issue Oct 13, 2023
Currently we're only passing the config as strings through the
RunRequest method.  This means that the program handling the RPC
can't know the right types, and has to infer it.  This leads to issues
such as pulumi/pulumi-yaml#453.

Passing the types through we'll be able to parse the strings correctly
on the receiving end.
tgummerer added a commit to pulumi/pulumi that referenced this issue Oct 13, 2023
Currently we're only passing the config as strings through the
RunRequest method.  This means that the program handling the RPC
can't know the right types, and has to infer it.  This leads to issues
such as pulumi/pulumi-yaml#453.

Passing the types through we'll be able to parse the strings correctly
on the receiving end.
tgummerer added a commit to pulumi/pulumi that referenced this issue Oct 13, 2023
Currently we're only passing the config as strings through the
RunRequest method.  This means that the program handling the RPC
can't know the right types, and has to infer it.  This leads to issues
such as pulumi/pulumi-yaml#453.

Passing the types through we'll be able to parse the strings correctly
on the receiving end.
tgummerer added a commit to pulumi/pulumi that referenced this issue Oct 13, 2023
Currently we're only passing the config as strings through the
RunRequest method.  This means that the program handling the RPC
can't know the right types, and has to infer it.  This leads to issues
such as pulumi/pulumi-yaml#453.

Passing the types through we'll be able to parse the strings correctly
on the receiving end.
@pulumi-bot pulumi-bot added the resolution/fixed This issue was fixed label Oct 27, 2023
@jamest-pin
Copy link

This now seems to be happening in reverse. I have values I want to be booleans, but they're being force converted into strings.

@jamest-pin
Copy link

pulumi seems to be converting all stack config values to strings. I want to store a boolean type there. Is there a way to store a value as a boolean, or to convert a string to a bool? I tried forcing it but didn't work.
I suppose it's possible that options: section doesn't accept variables..but I hope not.

# Pulumi.develop.yaml
config:
  myproject:myvar: false

# Pulumi.yaml

resources:
  myresource:
    options:
      protect: ${pulumi.stack}.${myvar}
    properties:
      ...

The above gives an error

error: failed to discover plugin requirements: 
-error: Pulumi.yaml:71,16-42: protect must be a boolean value;

If I try to force it:

 options:
      protect: !!bool ${pulumi.stack}.${myvar}

It doesn't work:

error: could not unmarshal '.../Pulumi.yaml': invalid YAML file: yaml: cannot decode !!str `${pulumi.stack}.${protect_dynamodb_tables}` as a !!bool

I also tried passing it through the variables: section in Pulumi.yaml

# Pulumi.yaml
...
variables:
  varProtectTablesAsBool:
    type: bool
    value: ${pulumi.stack}.${protect_dynamodb_tables} # tried both with !!bool and without

Still doesn't work, giving the first error protect must be a boolean value;

@Frassle
Copy link
Member

Frassle commented Nov 14, 2023

I think protect is set to only accept a literal bool expression, not an interpolated expression. Some recent work in #510 to loosen this restriction for "version", probably doable for "protect" as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
impact/reliability Something that feels unreliable or flaky kind/bug Some behavior is incorrect or out of spec resolution/fixed This issue was fixed
Projects
None yet
6 participants