Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature - Adding basic support for timestream-influxdb #8523

Open
jflim opened this issue Jan 24, 2025 · 9 comments · May be fixed by #8587
Open

Feature - Adding basic support for timestream-influxdb #8523

jflim opened this issue Jan 24, 2025 · 9 comments · May be fixed by #8587
Assignees

Comments

@jflim
Copy link
Contributor

jflim commented Jan 24, 2025

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/timestream-influxdb.html

  • create_db_instance
  • delete_db_instance
  • get_db_instance
  • list_db_instances
  • list_tags_for_resource
  • tag_resource
  • untag_resource
@jflim jflim changed the title Adding basic support for timestream-influxdb Feature - Adding basic support for timestream-influxdb Jan 24, 2025
@jflim
Copy link
Contributor Author

jflim commented Jan 27, 2025

Currently working on this!

@jflim
Copy link
Contributor Author

jflim commented Jan 27, 2025

@bblommers I ran the script/scaffold.py script, and it crashed with an error about 'aws' key errors but it seemed to create most of the files needed to create the mocked service.

At the moment, I'm trying to mock a basic test such as

def test_create_db_instance_success():
    client = boto3.client("timestream-influxdb", region_name="us-east-1")

    response = client.create_db_instance(
        name="test-instance",
        password="password123",
        dbInstanceType="db.influx.medium",
        vpcSubnetIds=["subnet-0123456789abcdef0"],
        vpcSecurityGroupIds=["sg-0123456789abcdef0"],
        allocatedStorage=123,
    )

but the test is erroring out the below:

    def __getitem__(self, region_name: str) -> SERVICE_BACKEND:
        # Some services, like S3, used to be truly global - meaning one Backend serving all
        # Now that we support partitions (AWS, AWS-CN, AWS-GOV, etc), there will be one backend per partition
        # Because the concept of 'region' doesn't exist in a global service, we use the partition name to keep the backends separate
        # We used to use the term 'global' in lieu of a region name, and users may still use this
        # It should resolve to 'aws', to ensure consistency
        if region_name == "global":
            region_name = "aws"
        if region_name in self.keys():
            return super().__getitem__(region_name)
        # Create the backend for a specific region
        with backend_lock:
            if region_name in self.regions and region_name not in self.keys():
                super().__setitem__(
                    region_name, self.backend(region_name, account_id=self.account_id)
                )
            if region_name not in self.regions and allow_unknown_region():
                super().__setitem__(
                    region_name, self.backend(region_name, account_id=self.account_id)
                )
>       return super().__getitem__(region_name)
E       KeyError: 'aws'

moto/core/base_backend.py:306: KeyError
============================================================= short test summary info =============================================================
FAILED tests/test_timestreaminfluxdb/test_timestreaminfluxdb.py::test_create_db_instance_success - KeyError: 'aws'

Do you have any ideas what I could be doing wrong in regards to registering the backend?(or if I can provide code segments that might be able to help debug the issue)

@bpandola
Copy link
Collaborator

@jflim Can you submit a draft PR with your current changeset? Hard to debug without seeing the code.

@jflim
Copy link
Contributor Author

jflim commented Jan 28, 2025

Sure, I can try to push a draft PR. In the meantime, I was able to reproduce on a separate branch. Is this expected?

==Current Implementation Status==
[ ] create_db_instance
[ ] create_db_parameter_group
[ ] delete_db_instance
[ ] get_db_instance
[ ] get_db_parameter_group
[ ] list_db_instances
[ ] list_db_parameter_groups
[ ] list_tags_for_resource
[ ] tag_resource
[ ] untag_resource
[ ] update_db_instance
=================================
Select Operation: create_db_instance
        Initializing service    timestream-influxdb
        creating        moto/timestreaminfluxdb
        creating        moto/timestreaminfluxdb/models.py
        creating        moto/timestreaminfluxdb/exceptions.py
        creating        moto/timestreaminfluxdb/__init__.py
        creating        moto/timestreaminfluxdb/responses.py
        creating        moto/timestreaminfluxdb/urls.py
        skip creating   tests/test_timestreaminfluxdb
        creating        tests/test_timestreaminfluxdb/__init__.py
        creating        tests/test_timestreaminfluxdb/test_server.py
        creating        tests/test_timestreaminfluxdb/test_timestreaminfluxdb.py
        inserting code  moto/timestreaminfluxdb/responses.py
        inserting code  moto/timestreaminfluxdb/models.py
        inserting code  tests/test_timestreaminfluxdb/test_timestreaminfluxdb.py
Updating backend index...

Please select another operation, or Ctrl-X/Ctrl-C to cancel.

Remaining steps after development is complete:
- Run scripts/implementation_coverage.py,


Traceback (most recent call last):
  File "/Users/jlim/moto/scripts/scaffold.py", line 697, in <module>
    main()
  File "/Users/jlim/moto/.venv/lib/python3.11/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jlim/moto/.venv/lib/python3.11/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/jlim/moto/.venv/lib/python3.11/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jlim/moto/.venv/lib/python3.11/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jlim/moto/scripts/scaffold.py", line 666, in main
    operation = select_operation(service)
                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jlim/moto/scripts/scaffold.py", line 92, in select_operation
    implemented, operation_names = get_operations(service_name)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jlim/moto/scripts/scaffold.py", line 109, in get_operations
    moto_client, _name = get_moto_implementation(service_name)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jlim/moto/scripts/implementation_coverage.py", line 20, in get_moto_implementation
    else backends[0]["global"]
         ~~~~~~~~~~~^^^^^^^^^^
  File "/Users/jlim/moto/moto/core/base_backend.py", line 306, in __getitem__
    return super().__getitem__(region_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'aws'
(.venv) jlim@603e5f5a8790 moto % 

@jflim
Copy link
Contributor Author

jflim commented Jan 28, 2025

@bpandola @bblommers I was made aware that there was a comment saying that the scaffolding may crash on certain services. I had tried scaffolding on a few other nonimplemented services and did not get this error. Wondering how to proceed in debugging scaffolding or the service backend?

@bpandola
Copy link
Collaborator

@jflim This error is likely due to the fact that timestream-influxdb doesn't currently have any regions returned by botocore:

regions = boto3.session.Session().get_available_regions("timestream-influxdb")
assert regions == []

You can manually update your botocore/data/endpoints.json file with an entry for timestream-influxdb and any supported regions, which should allow the scaffold script to execute without issue.

Ref: boto/botocore#3028
Ref: https://aws.amazon.com/about-aws/whats-new/2024/03/amazon-timestream-influxdb-available/

@jflim
Copy link
Contributor Author

jflim commented Jan 29, 2025

@jflim This error is likely due to the fact that timestream-influxdb doesn't currently have any regions returned by botocore:

regions = boto3.session.Session().get_available_regions("timestream-influxdb")
assert regions == []

You can manually update your botocore/data/endpoints.json file with an entry for timestream-influxdb and any supported regions, which should allow the scaffold script to execute without issue.

Ref: boto/botocore#3028 Ref: https://aws.amazon.com/about-aws/whats-new/2024/03/amazon-timestream-influxdb-available/

Adding to the aws partition for botocore seemed to work! What changes would I need to make so that timestream-influxdb could work with all moto users?

@bpandola
Copy link
Collaborator

@jflim Things should work just fine for moto users. This issue only affects the scaffolding script, as far as I can tell. Or am I missing something?

@bblommers
Copy link
Collaborator

You probably also have to specify the regions manually when creating the Backend-class.

See TimestreamQuery for an example, that service has the same problem:

additional_regions=[
"us-east-1",
"us-east-2",
"us-west-2",
"eu-central-1",
"eu-west-1",
"ap-southeast-2",
"ap-northeast-1",
],

@jflim jflim linked a pull request Feb 11, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants