Skip to content

Commit

Permalink
Add toc for new_connector.md
Browse files Browse the repository at this point in the history
  • Loading branch information
t94j0 committed Apr 25, 2024
1 parent 35e3bdc commit e2e52ac
Showing 1 changed file with 4 additions and 4 deletions.
8 changes: 4 additions & 4 deletions docs/new_connector.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ Regardless of the connector actions, it will need to somehow save the following
| Project name | PROJECT-X |
| Expiration days (or date) | 100 (or 01/01/2024) |

# Download Processing
## Download Processing

File processing is the one flow that differs from other structured data ingestion. First, the file bytes need to be uploaded to Nemesis, and second, a metadata message needs to be posted to kick off processing.

## Step 1 - File Upload
### Step 1 - File Upload

For a file to be processed, the raw file bytes first need to be posted to the correct API route for storage in the data lake. This is accomplished by POSTing the file bytes to the `https://<NEMESIS_URL>/api/file` which returns a simple JSON response with an `object_id` field containing a UUID that references the uploaded file. For example, to do this in Python (as shown in [mythic-connector](https://github.com/SpecterOps/Nemesis/blob/main/cmd/connectors/mythic-connector/sync.py)), you would run something like this:

Expand All @@ -32,7 +32,7 @@ curl -H "Content-Type: application/octet-stream" -v --user 'nemesis:Qwerty12345'

The `nemesis_file_id` is used in the `file_data` message in Step 2 below. This UUID is the unique reference for the file in Nemesis.

## Step 2 - File Data Message
### Step 2 - File Data Message

After the file is uploaded to Nemesis, a [file_data](odr/references/file_data.md) ODR message needs to be posted with file metadata information. The example from the [mythic-connector](https://github.com/SpecterOps/Nemesis/blob/main/cmd/connectors/mythic-connector/sync.py) is:

Expand Down Expand Up @@ -62,7 +62,7 @@ r = requests.request("POST", f"{NEMESIS_URL}/data", auth=basic, data=data, heade
*Note that timestamps need to be in ISO 8601 UTC form, e.g., 2023-08-01T22:51:35*


# Other Structured Data
## Other Structured Data

For other types of structured data, only a single message needs to be posted to the `http://<NEMESIS_URL>/api/data` API route, e.g. Step 2 in the downloading processing example. The `metadata["data_type"]` field should be one of the types defined in the [ODR](odr/references/). The appropriate ODR document will also define the fields and structure needed for the datatype.

Expand Down

0 comments on commit e2e52ac

Please sign in to comment.