v2.0.4

corrected yum update

v2.0.3

Updated

• CUMULUS-3270

  • Upgraded to Python 3.10, updated build jobs. Removed Python 3.8 and 3.6(EOL) build jobs.
  • Updated SDK boto3 version to support new Python runtime versions.
  • Updated pyinstaller and jsonschema versions.
  • Fixed pylint errors from Too Broad Exceptions.

Updated

• removed extraneous dependencies from requests.txt
  • typed-ast
  • six (remains as a subdependency)

v2.0.2

Release 2.0.2

[v2.0.2] 2022-01-10

BREAKING CHANGES

• CUMULUS-2751
  • Changes the key for the values populated in meta.workflow_tasks from the step function node name to an internally indexed integer string
  • Updates load_nested_event to no longer do a legacy task name lookup from either the step function API or cumulus_meta for configuration from event.workflow_config

Fixed

• CUMULUS-2751

  • Resolve inter-compatibility bug/regression with cumulus-message-adapter-js, as the current releases strictly check the message return schema for the existence of a config object. This fix restores the original behavior.
  • Fix regression in 2.0.0 that caused tasks without task_config defined cause CMA to fail

Compatibility

This version of the Cumulus Message Adapter requires at minimum the following versions of the client libraries:

• https://github.com/nasa/cumulus-message-adapter-js: v1.2.0+
• https://github.com/nasa/cumulus-message-adapter-python: v2.0.0+
• https://github.com/nasa/cumulus-message-adapter-java: v1.3.0+

Updating a Cumulus Deployment to CMA v2.0.2

The Cumulus Message Adapter has been updated in release 2.0.0 to no longer utilize the AWS step function API to look up the defined name of a step function task for population in meta.workflow_tasks, but instead use an incrementing integer field.

Additionally a bugfix was released in the form of v2.0.1/v2.0.2 following the initial 2.0.0 release, so all users should update to release 2.0.2

The update is not tied to a particular version of Core, however the update should be done across all task components in order to ensure consistent execution records.

Changes

Execution Record Update

This update functionally means that Cumulus tasks/activities using the CMA will now record a record that looks like the following in meta.workflowtasks, and more importantly in the tasks column for an execution record:

Original

      "DiscoverGranules": {
        "name": "jk-tf-DiscoverGranules",
        "version": "$LATEST",
        "arn": "arn:aws:lambda:us-east-1:xxxxx:function:jk-tf-DiscoverGranules"
      },
      "QueueGranules": {
        "name": "jk-tf-QueueGranules",
        "version": "$LATEST",
        "arn": "arn:aws:lambda:us-east-1:xxxx:function:jk-tf-QueueGranules"
      }

New

      "0": {
        "name": "jk-tf-DiscoverGranules",
        "version": "$LATEST",
        "arn": "arn:aws:lambda:us-east-1:xxxxx:function:jk-tf-DiscoverGranules"
      },
      "1": {
        "name": "jk-tf-QueueGranules",
        "version": "$LATEST",
        "arn": "arn:aws:lambda:us-east-1:xxxx:function:jk-tf-QueueGranules"
      }

Actions Required

The following should be done as part of a Cumulus stack update to utilize cumulus message adapter > 2.0.2:

• Python tasks that utilize cumulus-message-adapter-python should be updated to use > 2.0.0, their lambdas rebuilt and Cumulus workflows reconfigured to use the updated version.

• Python activities that utilize cumulus-process-py should be rebuilt using > 1.0.0 with updated dependencies, and have their images deployed/Cumulus configured to use the new version.

• The cumulus-message-adapter v2.0.2 lambda layer should be made available in the deployment account, and the Cumulus deployment should be reconfigured to use it (via the cumulus_message_adapter_lambda_layer_version_arn variable in the cumulus module). This should address all Core node.js tasks that utilize the CMA, and many contributed node.js/JAVA components.

Once the above have been done, redeploy Cumulus to apply the configuration and the updates should be live.

v2.0.1

This release is superseded by release 2.0.2

v2.0.0

This release is superseded by release 2.0.2

Original release notes are retained below:

BREAKING CHANGES

CUMULUS-2751

• Changes the key for the values populated in meta.workflow_tasks from the step function node name to an internally indexed integer string
• Updates load_nested_event to no longer do a legacy task name lookup from either the step function API or cumulus_meta for configuration from event.workflow_config

Compatibility

This version of the Cumulus Message Adapter requires at minimum the following versions of the client libraries:

• https://github.com/nasa/cumulus-message-adapter-js: v1.2.0+
• https://github.com/nasa/cumulus-message-adapter-python: v2.0.0+
• https://github.com/nasa/cumulus-message-adapter-java: v1.3.0+

v1.3.1

Updated

• CUMULUS-2751
  • Update CMA logging to explicitly flush stderr on log write
  • Add diagnostic logging to CMA 'steps'

v1.3.0

BREAKING CHANGES

CUMULUS-1896

CMA precompiled binary files are now copied into a subdirectory ./cma_bin. Clients who utilize the pre-compiled binary should update their pathing expectations to make use of that directory. If you are using the CMA via cumulus-message-adapter-js, then this change is handled for you transparently as of version 1.2.0.

Added

CUMULUS-1896

Added a 'streaming' mode to the CMA. This will allow client libraries to avoid requiring three subprocess/python invocations when running the CMA commands in sequence, thus lowering the overhead of utilizing the CMA.

Updated

Updated deployment to not bundle all associated .so files for pyinstaller in the executable. This results in slightly better performance/memory requirements at the cost of having all of the associated .so files in the layer directory structure.

Compatibility

This version of the Cumulus Message Adapter requires at minimum the following versions of the client libraries:

• https://github.com/nasa/cumulus-message-adapter-js: v1.2.0+
• https://github.com/nasa/cumulus-message-adapter-python: v1.2.0+
• https://github.com/nasa/cumulus-message-adapter-java: v1.3.0+

v1.2.0

BREAKING CHANGES

• CUMULUS-1486
  Removed all python 2 support. CMA now supports python 3.6+ only.
• Class in message_adapter/message_adapter.py renamed from message_adapter to MessageAdapter

Compatibility

This version of the Cumulus Message Adapter requires at minimum the following versions of the client libraries:

• https://github.com/nasa/cumulus-message-adapter-js: v1.0.8+
• https://github.com/nasa/cumulus-message-adapter-python: v1.2.0+
• https://github.com/nasa/cumulus-message-adapter-java: v1.3.0+

v1.1.3

Fixed

• Removed unsafe use of exec and eval in message_adapter.__assignJsonPathValue

v1.1.2

Compatibility

This version of the Cumulus Message Adapter requires at minimum the following versions of the client libraries:

• https://github.com/nasa/cumulus-message-adapter-js: v1.0.8+
• https://github.com/nasa/cumulus-message-adapter-python: v1.0.5+
• https://github.com/nasa/cumulus-message-adapter-java: v1.2.7+

Added

• Updated release process to utilize pyinstaller[CUMULUS-1447] to create a pre-package python+CMA release binary compatible with AWS Linux 2, added as the executable cma to the release package.