Skip to content

Commit

Permalink
Rename cumulus-sled to cumulus-message-adpater (#8)
Browse files Browse the repository at this point in the history
* Rename everything to cumulus-message-adapter
* Remove .zip and gitignore
  • Loading branch information
Aimee Barciauskas authored Jan 25, 2018
1 parent 92393be commit 6d7e775
Show file tree
Hide file tree
Showing 11 changed files with 71 additions and 70 deletions.
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
*.pyc
*~
*.egg-info
.vscode
.vscode
cumulus-message-adapter.zip
22 changes: 11 additions & 11 deletions CONTRACT.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
# `cumulus-sled` Contract
# `cumulus-message-adapter` Contract

`cumulus-sled` is a command-line interface for preparing and outputting Cumulus Messages for Cumulus Tasks. This contract defines how libraries should call `cumulus-sled` to integrate a task into a Cumulus Workflow.
`cumulus-message-adapter` is a command-line interface for preparing and outputting Cumulus Messages for Cumulus Tasks. This contract defines how libraries should call `cumulus-message-adapter` to integrate a task into a Cumulus Workflow.

Every Cumulus Task includes a business function. `cumulus-sled` and a language-specific library for interacting with `cumulus-sled` are required to integrate a business function as a Cumulus Task into a Cumulus Workflow.
Every Cumulus Task includes a business function. `cumulus-message-adapter` and a language-specific library for interacting with `cumulus-message-adapter` are required to integrate a business function as a Cumulus Task into a Cumulus Workflow.

`cumulus-sled` functions are invoked via the command line and read from stdin. Calling `./cumulus-sled <function_name>` should be followed by a json blob sent to stdin as detailed below.
`cumulus-message-adapter` functions are invoked via the command line and read from stdin. Calling `./cumulus-message-adapter <function_name>` should be followed by a json blob sent to stdin as detailed below.

Note: The `python ./cumulus-sled.zip` is interchangeable with `__main__.py` if the `cumulus-sled.zip` is up-to-date with the code.
Note: The `python ./cumulus-message-adapter.zip` is interchangeable with `__main__.py` if the `cumulus-message-adapter.zip` is up-to-date with the code.

```bash
# Cumulus Message or Cumulus Remote Message in:
python ./cumulus-sled.zip loadRemoteEvent
python ./cumulus-message-adapter.zip loadRemoteEvent
'{
"event": <event_json>
}'

# Cumulus Message and Lambda Context in:
python ./cumulus-sled.zip loadNestedEvent
python ./cumulus-message-adapter.zip loadNestedEvent
'{
"event": <event_json>,
"context": <context_json>
Expand All @@ -25,7 +25,7 @@ python ./cumulus-sled.zip loadNestedEvent
# Call inner handler

# Send result as <handler_response_json> to produce Cumulus Message out:
python ./cumulus-sled.zip createNextEvent
python ./cumulus-message-adapter.zip createNextEvent
'{
"event": <event_json>,
"handler_response": <handler_response_json>,
Expand Down Expand Up @@ -82,7 +82,7 @@ Cumulus Messages come in 2 flavors: The full **Cumulus Message** and the **Cumul

### `loadRemoteEvent` input

* `<event_json>` to cumulus-sled `loadRemoteEvent` should be either a full Cumulus Message or a Cumulus Remote Message, as defined above.
* `<event_json>` to cumulus-message-adapter `loadRemoteEvent` should be either a full Cumulus Message or a Cumulus Remote Message, as defined above.

### `loadRemoteEvent` output

Expand All @@ -92,9 +92,9 @@ loadRemote output is a full Cumulus Message as a json blob.

### `loadNestedEvent` input

* `<event_json>` to cumulus-sled `loadNestedEvent` should be a full Cumulus Message as defined above.
* `<event_json>` to cumulus-message-adapter `loadNestedEvent` should be a full Cumulus Message as defined above.

* `<context_json>` to cumulus-sled `loadNestedEvent` should be the context from the lambda.
* `<context_json>` to cumulus-message-adapter `loadNestedEvent` should be the context from the lambda.

**`loadNestedEvent` Details:**

Expand Down
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Cumulus Sled Message Transformation
# Cumulus Message Adapter

[![CircleCI](https://circleci.com/gh/cumulus-nasa/cumulus-sled.svg?style=svg)](https://circleci.com/gh/cumulus-nasa/cumulus-sled)
[![CircleCI](https://circleci.com/gh/cumulus-nasa/cumulus-message-adapter.svg?style=svg)](https://circleci.com/gh/cumulus-nasa/cumulus-message-adapter)

## Development

Expand All @@ -25,20 +25,20 @@ And then you can check tests pass with the following nosetests command:
$ CUMULUS_ENV=testing nosetests -v -s
```

Some example node.js code for handling the cumulus-sled can be found and executed:

```bash
./examples/example-node-sled-lib.js
```

### Linting

$ pylint message

### Contributing

If changes are made to the codebase, a new zip archive should be created to replace `cumulus-sled.zip` for libraries that require it:
If changes are made to the codebase, a new zip archive should be created to replace `cumulus-message-adapter.zip` for libraries that require it:

```bash
$ zip -r cumulus-message-adapter.zip .
```

Then you can run some integration tests:

```bash
$ zip -r cumulus-sled.zip .
./examples/example-node-message-adapter-lib.js
```
4 changes: 2 additions & 2 deletions __main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@
import json
import sys

from message import message
from message_adapter import message_adapter

if __name__ == '__main__':
functionName = sys.argv[1]
transformer = message.message()
transformer = message_adapter.message_adapter()
exitCode = 1
allInput = json.loads(raw_input())
event = allInput['event']
Expand Down
Binary file removed cumulus-sled.zip
Binary file not shown.
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ function loadJsonFromFile(fileName) {
/**
* Integration Test for loadRemoteEvent
*/
var child = cp.spawn('python', ['./cumulus-sled.zip', 'loadRemoteEvent'], { env: env });
var child = cp.spawn('python', ['./cumulus-message-adapter.zip', 'loadRemoteEvent'], { env: env });

child.stderr.pipe(process.stderr);

Expand All @@ -40,7 +40,7 @@ child.stdout.on('data', (data) => {
* test :(.
* TODO(aimee) Mock the response from AWS Step Functions API.
*/
var child = cp.spawn('python', ['./cumulus-sled.zip', 'loadNestedEvent'], { env: env });
var child = cp.spawn('python', ['./cumulus-message-adapter.zip', 'loadNestedEvent'], { env: env });

child.stderr.pipe(process.stderr);

Expand All @@ -54,7 +54,7 @@ child.stdin.end();
/*
* Integration Test for createNextEvent
*/
var child = cp.spawn('python', ['./cumulus-sled.zip', 'createNextEvent']);
var child = cp.spawn('python', ['./cumulus-message-adapter.zip', 'createNextEvent']);

child.stderr.pipe(process.stderr);

Expand Down
File renamed without changes.
File renamed without changes.
18 changes: 9 additions & 9 deletions message/message.py → message_adapter/message_adapter.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
from datetime import datetime,timedelta
import uuid
from jsonpath_ng import jsonpath, parse
import aws_sled
import aws

class message:
class message_adapter:
"""
transforms the cumulus message
"""
Expand Down Expand Up @@ -55,7 +55,7 @@ def __getTaskNameFromExecutionHistory(self, executionHistory, arn):
elif step['type'] == 'TaskStateEntered': return step['stateEnteredEventDetails']['name'];
raise LookupError('No task found for ' + arn);

def __getCurrentSfnTask(self, stateMachineArn, executionName, arn):
def _message__getCurrentSfnTask(self, stateMachineArn, executionName, arn):
"""
* Given a state machine ARN, an execution name, and an optional Activity or Lambda ARN returns
* the most recent task name started for the given ARN in that execution, or if no ARN is
Expand All @@ -69,7 +69,7 @@ def __getCurrentSfnTask(self, stateMachineArn, executionName, arn):
* @param {string} arn An ARN to an Activity or Lambda to find. See "IMPORTANT!"
* @returns {string} The name of the task being run
"""
sfn = aws_sled.stepFn()
sfn = aws.stepFn()
executionArn = self.__getSfnExecutionArnByName(stateMachineArn, executionName);
executionHistory = sfn.get_execution_history(
executionArn=executionArn,
Expand All @@ -93,7 +93,7 @@ def loadRemoteEvent(self, event):
* @returns {*} the full event data
"""
if ('replace' in event):
s3 = aws_sled.s3()
s3 = aws.s3()
data = s3.Object(event['replace']['Bucket'], event['replace']['Key']).get();
if (data is not None):
return json.loads(data['Body'].read().decode('utf-8'));
Expand Down Expand Up @@ -132,7 +132,7 @@ def __loadStepFunctionConfig(self, event, context):
"""
meta = event['cumulus_meta'];
arn = context['invokedFunctionArn'] if 'invokedFunctionArn' in context else context.get('activityArn');
taskName = self.__getCurrentSfnTask(meta['state_machine'],meta['execution_name'],arn);
taskName = self._message__getCurrentSfnTask(meta['state_machine'],meta['execution_name'],arn);
return self.__getConfig(event, taskName) if taskName is not None else None;

def __loadConfig(self, event, context):
Expand Down Expand Up @@ -294,7 +294,7 @@ def __assignJsonPathValue(self, message, jspath, value):
exec ("message" + dictPath + " = value");
return message;

def __assignOutputs(self, handlerResponse, event, messageConfig):
def _message__assignOutputs(self, handlerResponse, event, messageConfig):
"""
* Applies a task's return value to an output message as defined in config.cumulus_message
*
Expand Down Expand Up @@ -329,7 +329,7 @@ def __storeRemoteResponse(self, event):

if (roughDataSize < self.MAX_NON_S3_PAYLOAD_SIZE): return event;

s3 = aws_sled.s3();
s3 = aws.s3();
s3Bucket = event['ingest_meta']['message_bucket']
s3Key = ('/').join(['events', str(uuid.uuid4())])
s3Params = {
Expand All @@ -354,7 +354,7 @@ def createNextEvent(self, handlerResponse, event, messageConfig):
* @param {*} messageConfig The cumulus_message object configured for the task
* @returns {*} the output message to be returned
"""
result = self.__assignOutputs(handlerResponse, event, messageConfig);
result = self._message__assignOutputs(handlerResponse, event, messageConfig);
result['exception'] = 'None';
if 'replace' in result: del result['replace'];
return self.__storeRemoteResponse(result);
Expand Down
8 changes: 4 additions & 4 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
# Fields marked as "Optional" may be commented out.

setup(
name='cumulus-sled', # Required
name='cumulus-message-adpater', # Required

# Versions should comply with PEP 440:
# https://www.python.org/dev/peps/pep-0440/
Expand All @@ -29,9 +29,9 @@
# This is a one-line description or tagline of what your project does. This
# corresponds to the "Summary" metadata field:
# https://packaging.python.org/specifications/core-metadata/#summary
description='A python implementation of cumulus message transformation for sled', # Required
description='A python implementation of cumulus message adapter', # Required
long_description=long_description, # Optional
url='https://github.com/cumulus-nasa/sled-message', # Optional
url='https://github.com/cumulus-nasa/cumulus-message-adpater', # Optional
author='Cumulus Authors', # Optional
author_email='[email protected]', # Optional
classifiers=[ # Optional
Expand All @@ -43,7 +43,7 @@
'Programming Language :: Python :: 2',
'Programming Language :: Python :: 2.7',
],
keywords='nasa cumulu sled', # Optional
keywords='nasa cumulus message adapter', # Optional
packages=find_packages(exclude=['.circleci', 'contrib', 'docs', 'tests']), # Required
install_requires=['jsonpath-ng', 'boto3'], # Optional
)
Loading

0 comments on commit 6d7e775

Please sign in to comment.