Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use of parameters with the Redshift Data API in aws-sdk-pandas #3024

Open
ArtOfStew opened this issue Nov 19, 2024 · 1 comment
Open

Use of parameters with the Redshift Data API in aws-sdk-pandas #3024

ArtOfStew opened this issue Nov 19, 2024 · 1 comment
Labels
question Further information is requested

Comments

@ArtOfStew
Copy link

We are attempting to use a Lambda to read from a Redshift Serverless Workgroup.

We need to use parameters, but I can't figure out the syntax to make it work. This may be a bug, but we may simply be missing the correct way of doing it.

Here's a sample lambda:

import json
import awswrangler as wr

def lambda_handler(event, context):
    print(wr.__version__)
    rs_data_api = wr.data_api.redshift.RedshiftDataApi(
        workgroup_name="my-workgroup",
        database="dev",
        secret_arn="arn:aws:secretsmanager:REGION:ACCOUNT:secret:my/redshift/secret-5OCdpE"
    )

    parameters = [{
        "name": "ittype",
        "value": "E"
    }]


    df = rs_data_api.execute("SELECT itemcode FROM my_schema.itemtable where itemtype = :ittype LIMIT 5", parameters = parameters)
    # This fails with "Request 0c4f411e-9dbe-4e4f-bc03-0184fcf26e59 failed with status FAILED and error ERROR: syntax error at or near \":\" in context \"where ittype = :\", at line 1\n  Position: 60"


    df = rs_data_api.execute("SELECT itemcode FROM my_schema.itemtable where itemtype = ? LIMIT 5", parameters = parameters)
    # This fails with "Request 3f9dbece-3a2b-4a8d-a15f-909afae5ad6c failed with status FAILED and error No value specified for parameter 1."
    
    print(df)

We've also tried the %s format defined in PEP249, but it gets the same error as the colon; the error is a syntax error.

We have tried different combinations for the parameters according to various documentation we found on the web. For example this:

    parameters = [{
        "stringValue": "E"
    }]

... as well as many more. We've even tried string-ifying the parameters with json.dumps(parameters).

It seems like parameters either aren't being fed through, or there is a different way of submitting them.

Appreciate the help!

Lambda Python Runtime: 3.12 ARM64
Wrangler Layer: AWSSDKPandas-Python312-Arm64 version 14 (awswranger == 3.10.0)

@ArtOfStew ArtOfStew added the question Further information is requested label Nov 19, 2024
@jaidisido
Copy link
Contributor

I am afraid the current implementation of execute in the RedshiftDataAPI class of awswrangler does not support parameters:
https://github.com/aws/aws-sdk-pandas/blob/main/awswrangler/data_api/redshift.py#L116
This is why the parameters keyword is not listed in the docs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants