Skip to content

Commit

Permalink
MINIFICPP-2469 Create GetCouchbaseKey processor
Browse files Browse the repository at this point in the history 
Closes #1881

Signed-off-by: Marton Szasz <[email protected]>
  • Loading branch information
@lordgamez @szaszm
lordgamez authored and szaszm committed Nov 27, 2024
1 parent bd1182e commit ea16935
Show file tree
Hide file tree
Showing 14 changed files with 682 additions and 26 deletions.
40 changes: 40 additions & 0 deletions PROCESSORS.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ limitations under the License.
- [FetchSmb](#FetchSmb)
- [FocusArchiveEntry](#FocusArchiveEntry)
- [GenerateFlowFile](#GenerateFlowFile)
- [GetCouchbaseKey](#GetCouchbaseKey)
- [GetFile](#GetFile)
- [GetTCP](#GetTCP)
- [HashContent](#HashContent)
Expand Down Expand Up @@ -1096,6 +1097,45 @@ In the list below, the names of required properties appear in bold. Any other pr
| success | success operational on the flow record |


## GetCouchbaseKey

### Description

Get a document from Couchbase Server via Key/Value access. The ID of the document to fetch may be supplied by setting the <Document Id> property. NOTE: if the Document Id property is not set, the contents of the FlowFile will be read to determine the Document Id, which means that the contents of the entire FlowFile will be buffered in memory.

### Properties

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

| Name | Default Value | Allowable Values | Description |
|------------------------------------------|---------------|----------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Couchbase Cluster Controller Service** | | | A Couchbase Cluster Controller Service which manages connections to a Couchbase cluster. |
| **Bucket Name** | default | | The name of bucket to access.<br/>**Supports Expression Language: true** |
| Scope Name | | | Scope to use inside the bucket. If not specified, the _default scope is used.<br/>**Supports Expression Language: true** |
| Collection Name | | | Collection to use inside the bucket scope. If not specified, the _default collection is used.<br/>**Supports Expression Language: true** |
| **Document Type** | Json | Json<br/>Binary<br/>String | Content type of the retrieved value. |
| Document Id | | | A static, fixed Couchbase document id, or an expression to construct the Couchbase document id.<br/>**Supports Expression Language: true** |
| Put Value to Attribute | | | If set, the retrieved value will be put into an attribute of the FlowFile instead of a the content of the FlowFile. The attribute key to put to is determined by evaluating value of this property.<br/>**Supports Expression Language: true** |

### Relationships

| Name | Description |
|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| success | Values retrieved from Couchbase Server are written as outgoing FlowFiles content or put into an attribute of the incoming FlowFile and routed to this relationship. |
| failure | All FlowFiles failed to fetch from Couchbase Server and not retry-able are routed to this relationship. |
| retry | All FlowFiles failed to fetch from Couchbase Server but can be retried are routed to this relationship. |
| original | The original input FlowFile is routed to this relationship when the value is retrieved from Couchbase Server and routed to 'success'. |

### Output Attributes

| Attribute | Relationship | Description |
|----------------------|--------------|---------------------------------------|
| couchbase.bucket | success | Bucket where the document was stored. |
| couchbase.doc.id | success | Id of the document. |
| couchbase.doc.cas | success | CAS of the document. |
| couchbase.doc.expiry | success | Expiration of the document. |


## GetFile

### Description
Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ The next table outlines CMAKE flags that correspond with MiNiFi extensions. Exte
| AWS | [AWSCredentialsService](CONTROLLERS.md#awscredentialsservice)<br/>[PutS3Object](PROCESSORS.md#puts3object)<br/>[DeleteS3Object](PROCESSORS.md#deletes3object)<br/>[FetchS3Object](PROCESSORS.md#fetchs3object)<br/>[ListS3](PROCESSORS.md#lists3) | -DENABLE_AWS=ON |
| Azure | [AzureStorageCredentialsService](CONTROLLERS.md#azurestoragecredentialsservice)<br/>[PutAzureBlobStorage](PROCESSORS.md#putazureblobstorage)<br/>[DeleteAzureBlobStorage](PROCESSORS.md#deleteazureblobstorage)<br/>[FetchAzureBlobStorage](PROCESSORS.md#fetchazureblobstorage)<br/>[ListAzureBlobStorage](PROCESSORS.md#listazureblobstorage)<br/>[PutAzureDataLakeStorage](PROCESSORS.md#putazuredatalakestorage)<br/>[DeleteAzureDataLakeStorage](PROCESSORS.md#deleteazuredatalakestorage)<br/>[FetchAzureDataLakeStorage](PROCESSORS.md#fetchazuredatalakestorage)<br/>[ListAzureDataLakeStorage](PROCESSORS.md#listazuredatalakestorage) | -DENABLE_AZURE=ON |
| CivetWeb | [ListenHTTP](PROCESSORS.md#listenhttp) | -DENABLE_CIVET=ON |
| Couchbase | [CouchbaseClusterService](CONTROLLERS.md#couchbaseclusterservice)<br/>[PutCouchbaseKey](PROCESSORS.md#putcouchbasekey) | -DENABLE_COUCHBASE=ON |
| Couchbase | [CouchbaseClusterService](CONTROLLERS.md#couchbaseclusterservice)<br/>[PutCouchbaseKey](PROCESSORS.md#putcouchbasekey)<br/>[GetCouchbaseKey](PROCESSORS.md#getcouchbasekey) | -DENABLE_COUCHBASE=ON |
| Elasticsearch | [ElasticsearchCredentialsControllerService](CONTROLLERS.md#elasticsearchcredentialscontrollerservice)<br/>[PostElasticsearch](PROCESSORS.md#postelasticsearch) | -DENABLE_ELASTICSEARCH=ON |
| ExecuteProcess (Linux and macOS) | [ExecuteProcess](PROCESSORS.md#executeprocess) | -DENABLE_EXECUTE_PROCESS=ON |
| Google Cloud Platform | [DeleteGCSObject](PROCESSORS.md#deletegcsobject)<br>[FetchGCSObject](PROCESSORS.md#fetchgcsobject)<br>[GCPCredentialsControllerService](CONTROLLERS.md#gcpcredentialscontrollerservice)<br>[ListGCSBucket](PROCESSORS.md#listgcsbucket)<br>[PutGCSObject](PROCESSORS.md#putgcsobject) | -DENABLE_GCP=ON |
Expand Down
4 changes: 2 additions & 2 deletions docker/test/integration/cluster/containers/CouchbaseServerContainer.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ def run_post_startup_commands(self):
["couchbase-cli", "cluster-init", "-c", "localhost", "--cluster-username", "Administrator", "--cluster-password", "password123", "--services", "data,index,query",
"--cluster-ramsize", "2048", "--cluster-index-ramsize", "256"],
["couchbase-cli", "bucket-create", "-c", "localhost", "--username", "Administrator", "--password", "password123", "--bucket", "test_bucket", "--bucket-type", "couchbase",
"--bucket-ramsize", "1024"]
"--bucket-ramsize", "1024", "--max-ttl", "36000"]
]
for command in commands:
(code, _) = self.client.containers.get(self.name).exec_run(command)
Expand All @@ -47,7 +47,7 @@ def deploy(self):
return

self.docker_container = self.client.containers.run(
"couchbase:community-7.6.2",
"couchbase:enterprise-7.2.5",
detach=True,
name=self.name,
network=self.network.name,
Expand Down
Loading

0 comments on commit ea16935

Please sign in to comment.