r.2023.10.31.0

Summary:

This release contains improvements via tech debt cleanup, performance enhancements for subs processing, as well as adding the cost management operator version in kafka messages to the ROS team.

What's Changed

• [COST-4345] delete ReportingCommonDBAccessor and region_map tests by @maskarb in #4747
• [COST-4350] delete AccountAliasAccessor, AWSAccountAlias, AccountLabel by @maskarb in #4749
• [COST-4346] replace ReportStatsDBAccessor with Django methods by @maskarb in #4748
• [COST-4242] Performance enhancements for resource id subs processing by @cgoodfred in #4730
• [COST-4358] Provide operator version in the messages to ROS by @chambridge in #4750
• RHIROS-1292 Make ROS OCP API spec available under Cost-Mgmt API doc by @srdeotarse in #4728
• Revert "RHIROS-1292 Make ROS OCP API spec available under Cost-Mgmt API doc" by @chambridge in #4753

New Contributors

• @srdeotarse made their first contribution in #4728

Full Changelog: r.2023.10.23.0...r.2023.10.31.0

r.2023.10.23.0

Summary:

This release contains database migrations to partition one of our upcoming UI view tables, subs sql improvements, and minor api updates.
As well as new logic to skip clusters without a matching billing report date for OCP on Cloud processing.

What's Changed

• [COST-4252] Add tag counts to meta in GET response by @samdoran in #4731
• [COST-4028] drop OCPPodSummaryByNodeP table by @maskarb in #4736
• [COST-4028] recreate OCPPodSummaryByNodeP as partitioned table by @maskarb in #4737
• [COST-4328] Fix subs vCPU trino filter and ensure columns exist on aws_line_items table by @cgoodfred in #4740
• [COST-4028] - redo table drop and create by @maskarb in #4741
• [COST-4028] make migrations no-op by @maskarb in #4742
• [COST-4328] Fix where clause SQL and check for results before attempting to upload by @cgoodfred in #4745
• [COST-3341] Add 'cost_type' param in OCP AWS Forecas Report by @bacciotti in #4739
• Add a unittest to ensure django models are partitioned in postgres. by @myersCody in #4738
• [COST-4331] Parse tags for ELS not EUS and use ID 204 if ELS is present for SUBS processing by @cgoodfred in #4744
• [COST-4289] - Skip OCP clusters without a matching billing report date for OCP on Cloud processing by @lcouzens in #4735

New Contributors

• @bacciotti made their first contribution in #4739

Full Changelog: r.2023.10.17.0...r.2023.10.23.0

r.2023.10.17.0

Summary:

• Handling null values passed into tags settings API
• Fix fetching latest currency values when API is unreachable
• Start moving to PG 14

What's Changed

• [COST-4235] Properly handle null values in the tags settings API by @samdoran in #4708
• [COST-4224] Only read currency API data from a good response by @samdoran in #4699
• [COST-4315] - upgrade local and ephemeral to Postgres 14 by @maskarb in #4705

Full Changelog: r.2023.10.10.0...r.2023.10.17.0

r.2023.10.10.0

Summary:

• Adding storage class filtering for persistent volumes
• Improved rhel processing tracking by distinct resource id
• Fixing OCP processing exception when cluster_id not available
• Improvements to internal SRE endpoints for better observability and control

What's Changed

• Add storage class filter, and lax serializer checks. by @myersCody in #4725
• More info on what celery tasks are in queue by @cgoodfred in #4454
• add the ability to purge a specific queue of tasks by @lcouzens in #4727
• [COST-4242] process and track metered RHEL by resource id by @cgoodfred in #4713
• [COST-4268] Do not attempt creating a new cluster without cluster_id by @djnakabaale in #4718

Full Changelog: r.2023.10.09.0...r.2023.10.10.0

r.2023.10.09.0

Summary:

• dependency and environment variable updates

What's Changed

• unpin Pandas and dependencies update by @maskarb in #4722
• Adding cache timeout params by @lcouzens in #4724

Full Changelog: r.2023.10.06.0...r.2023.10.09.0

r.2023.10.06.0

Summary:

• deprecate old settings endpoint
• bug fixes

What's Changed

• Turn fallback on for all labels unleash flag. by @myersCody in #4716
• [COST-4120] Fix capacity instance counts. by @myersCody in #4644
• [COST-3797] Deprecate Settings endpoint by @myersCody in #4670
• Fix flags.json by @myersCody in #4721
• Improve logging around ocp on cloud summary dates. by @myersCody in #4715
• [COST-4257] Fix unused percentage for first days of the month. by @myersCody in #4720
• Bump the worker timout for large customers by @lcouzens in #4719

Deprecation Notice

• Endpoint: /api/cost-management/v1/settings/
• Deprecated: Friday, Oct. 6th 2023
• End of Life: Wednesday, Jan. 31st 2024

Overview

In this release, we are excited to announce a significant change regarding one of our API endpoints. We have moved the functionality in our Setting's workflow which was previously built on Data Driven Forms (DDF) to a new location. This transition enables us to pave the way for exciting new functionalities and improvements in future releases.

Key Changes

1. Endpoint Relocation: The endpoint that was previously built on DDF has been relocated to a new home within our system. This transition ensures better alignment with our evolving architecture and future development plans.

2. Improved Performance: With this move, we have optimized the endpoint's performance, resulting in faster response times and reduced latency.

3. Scalability: The new implementation is designed to scale seamlessly as our user base and data volume grow, ensuring a consistent and reliable experience.

How Does This Affect You?

For most users, this transition will be seamless, with no immediate action required. However, if you have direct integrations or dependencies on the previous endpoint, we recommend reviewing your configurations and updating them as needed to point to the new location. The functionalities available on the previous endpoint can still be controlled through their new locations:

1. Setting a Default AWS Cost Type

• Endpoint: /api/cost-management/v1/account-settings/cost-type/
• Acceptable Methods: [GET, PUT]
• Example Request Body:

{
    "cost_type": "unblended_cost"
}

2. Setting a Default Currency

• Endpoint: /api/cost-management/v1/account-settings/currency/
• Acceptable Methods: [GET, PUT]
• Example Request Body:

{
    "currency": "EUR"
}

NOTE: To see your currency options use: /api/cost-management/v1/currency/

3. Enabling and Disabling Tags & Labels

• Endpoint to view tags: /api/cost-management/v1/settings/tags/
• Acceptable Methods: [GET]
• Example Response:

{
    "meta": {
        "count": 17,
        "limit": 100,
        "offset": 0
    },
    "links": {
        "first": "/api/cost-management/v1/settings/tags/?limit=100&offset=0",
        "next": null,
        "previous": null,
        "last": "/api/cost-management/v1/settings/tags/?limit=100&offset=0"
    },
    "data": [
        {
            "uuid": "a1e5d730-c987-4507-a333-439972f886d6",
            "key": "app",
            "enabled": true,
            "source_type": "OCP"
        }
     ]
  }

NOTE: This endpoint will return a unique identifier, a1e5d730-c987-4507-a333-439972f886d6 in the example above, which will be used in the following endpoints to control enablement:

• Endpoint to Enable: /api/cost-management/v1/settings/tags/enable/
• Endpoint to Disable: /api/cost-management/v1/settings/tags/disable/
  • Acceptable Methods: [PUT]
  • Example Request Body:

{
    "ids": ["a1e5d730-c987-4507-a333-439972f886d6"]
}

4. Enabling and Disabling AWS Cost Categories

• Endpoint to view tags: api/cost-management/v1/settings/aws_category_keys/
• Acceptable Methods: [GET]
• Example Response:

{
    "meta": {
        "count": 17,
        "limit": 100,
        "offset": 0
    },
    "links": {
        "first": "/api/cost-management/v1/settings/tags/?limit=100&offset=0",
        "next": null,
        "previous": null,
        "last": "/api/cost-management/v1/settings/tags/?limit=100&offset=0"
    },
    "data": [
        {
            "uuid": "d75540f4-e030-4ce5-8782-a44ae25f2a17",
            "key": "volume",
            "enabled": true
        },
        {
            "uuid": "46060dc7-a544-4fd0-be5b-880e1b37601d",
            "key": "qe",
            "enabled": true
        },
     ]
  }

NOTE: This endpoint will return a unique identifier, d75540f4-e030-4ce5-8782-a44ae25f2a17 in the example above, which will be used in the following endpoints to control enablement:

• Endpoint to Enable: /api/cost-management/v1/settings/aws_category_keys/enable/
• Endpoint to Disable: /api/cost-management/v1/settings/aws_category_keys/disable/
  • Acceptable Methods: [PUT]
  • Example Request Body:

{
    "ids": ["d75540f4-e030-4ce5-8782-a44ae25f2a17", "46060dc7-a544-4fd0-be5b-880e1b37601d""]
}

What's New?

Enhanced Response Headers

We have added new response headers to our API responses, specifically tailored to provide insights into upcoming deprecations. These headers are designed to assist you in staying ahead of any modifications to our APIs and help ensure a smooth transition.

1. Deprecation: Fri, 29 Sep 2023 00:00:00 - The date the endpoint is no longer supported.
2. Sunset: Wed, 31 Jan 2024 00:00:00 - The date the endpoint will be permanently removed.

How Does This Benefit You?

By examining these response headers in your workflows and integrations, you can:

• Stay Informed: Be notified in advance of any deprecations or breaking changes planned for our APIs.

• Proactive Action: Take proactive measures to update your configurations and adapt to upcoming changes, minimizing potential disruptions to your operations.

Recommended Action

To leverage the benefits of these enhanced response headers, we recommend the following actions:

1. Review Your Integration: Take some time to review your existing integrations and workflows that rely on our APIs.

2. Check for New Headers: Ensure that your integrations are equipped to capture and interpret the new response headers.

3. Stay Updated: Periodically monitor these headers to receive timely notifications of any deprecations or changes.

4. Adjust as Needed: When you receive notifications related to upcoming deprecations, make the necessary adjustments to your configurations or code to align with the changes.

Conclusion

We are excited about the possibilities that this transition opens up for our platform. Our commitment to providing you with the best possible experience remains unwavering, and we look forward to delivering new functionalities and improvements in the near future.

Full Changelog: r.2023.10.03.0...r.2023.10.06.0

r.2023.10.03.0

Summary

• migration to consolidate pod and volume labels into an all-labels field
• add ordering by PersistentVolumeClaims and StorageClasses on the OCP volumes endpoints
• bug fixes

What's Changed

• Add schema name to the provider map by @myersCody in #4703
• Add attempting to delete log. by @myersCody in #4704
• [COST-4101] - tech debt: remove ocp on gcp summarization by node. by @myersCody in #4626
• [COST-4231] - handle empty OCP daily file by @maskarb in #4702
• [COST-4110] - handle duplicate usage ids from subs processing by @cgoodfred in #4679
• [COST-4240] - ingress-flow: ensure provider belongs to customer sending ingress reports by @lcouzens in #4697
• [COST-3083] - enable volume labels on OCP cost endpoint by @myersCody in #4687
• Improve PostgreSQL and minio container builds by @samdoran in #4696
• [COST-4198] Order by pvc & storage_class by @myersCody in #4692
• [COST-4263] add sources-psk to priority worker by @maskarb in #4711
• Bump urllib3 from 1.26.16 to 1.26.17 by @dependabot in #4710
• make the tag population a no-op by @maskarb in #4712

Full Changelog: r.2023.09.22.0...r.2023.10.03.0

r.2023.09.22.0

Summary:

Updated base container image, local development improvements, orchestrator changes.

What's Changed

• Update base image in #4691
• [COST-4209] remove schedule flag from orchestrator by @maskarb in #4694
• Local dev minio and PostgreSQL changes by @maskarb in #4693

Full Changelog: r.2023.09.20.0...r.2023.09.22.0

r.2023.09.20.0

Summary:

Improvements to database accessors and internal APIs

What's Changed

• [COST-4185] Make ROS presigned URL timeout configurable by @cgoodfred in #4688
• Create new Provider Managers and remove CURAccountsInterface, CURAccountsDB, AccountsAccessor by @maskarb in #4567
• [COST-4208] Remove CustomerDBAccessor and ProviderAuthDBAccessor by @maskarb in #4676
• [COST-4209] Enable OCP polling ingest only when DEBUG is true by @maskarb in #4689
• [COST-4209] Fix Masu download endpoint query params by @maskarb in #4690

Full Changelog: r.2023.09.18.0...r.2023.09.20.0

r.2023.09.18.0

Summary:

• Various subs improvements
• Improvements to development workflow
• Improvements OCP data processing
• Better handling of null values for OCP.

What's Changed

• [COST-4033] Use source application.extra.metered value for subs check by @cgoodfred in #4630
• [COST-4168] null storage usage by @maskarb in #4669
• update dependencies by @maskarb in #4681
• Daily archives minor bug fixes + code cleanup by @lcouzens in #4668
• Update CI status badge by @samdoran in #4672
• [COST-4178] - (OCP) determine metadata during csv creation by @maskarb in #4673
• fix: compare integers not integer strings by @djnakabaale in #4683
• [COST-4109] Start subs processing at midnight day before source creation by @cgoodfred in #4678
• [COST-4178] use the filename-stem instead of the report-type to track clearing s3 by @maskarb in #4685
• Ensure consistent permissions for bind mounts by @samdoran in #4682
• [COST-4182] Return status code 412 when enabled tags limit is exceeded by @samdoran in #4675

Full Changelog: r.2023.09.11.0...r.2023.09.18.0