Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MINIFICPP-2469 Create GetCouchbaseKey processor #1881

Closed
wants to merge 6 commits into from
Closed

MINIFICPP-2469 Create GetCouchbaseKey processor #1881

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8cdbfa6
MINIFICPP-2469 Create GetCouchbaseKey processor
lordgamez Oct 8, 2024
7bd1cb4
Add extenstion docs and refactor docker tests
lordgamez Oct 21, 2024
cd1e638
Add test logs
lordgamez Oct 22, 2024
b8ba62a
Fix after rebase
lordgamez Nov 19, 2024
6b8d35c
Review update
lordgamez Nov 22, 2024
de69b7f
Fix formatting and clang issues
lordgamez Nov 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions PROCESSORS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ limitations under the License.
- [FetchSmb](#FetchSmb)
- [FocusArchiveEntry](#FocusArchiveEntry)
- [GenerateFlowFile](#GenerateFlowFile)
- [GetCouchbaseKey](#GetCouchbaseKey)
- [GetFile](#GetFile)
- [GetTCP](#GetTCP)
- [HashContent](#HashContent)
Expand Down Expand Up @@ -1096,6 +1097,45 @@ In the list below, the names of required properties appear in bold. Any other pr
| success | success operational on the flow record |


## GetCouchbaseKey

### Description

Get a document from Couchbase Server via Key/Value access. The ID of the document to fetch may be supplied by setting the <Document Id> property. NOTE: if the Document Id property is not set, the contents of the FlowFile will be read to determine the Document Id, which means that the contents of the entire FlowFile will be buffered in memory.

### Properties

In the list below, the names of required properties appear in bold. Any other properties (not in bold) are considered optional. The table also indicates any default values, and whether a property supports the NiFi Expression Language.

| Name | Default Value | Allowable Values | Description |
|------------------------------------------|---------------|----------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Couchbase Cluster Controller Service** | | | A Couchbase Cluster Controller Service which manages connections to a Couchbase cluster. |
| **Bucket Name** | default | | The name of bucket to access.<br/>**Supports Expression Language: true** |
| Scope Name | | | Scope to use inside the bucket. If not specified, the _default scope is used.<br/>**Supports Expression Language: true** |
| Collection Name | | | Collection to use inside the bucket scope. If not specified, the _default collection is used.<br/>**Supports Expression Language: true** |
| **Document Type** | Json | Json<br/>Binary<br/>String | Content type of the retrieved value. |
| Document Id | | | A static, fixed Couchbase document id, or an expression to construct the Couchbase document id.<br/>**Supports Expression Language: true** |
| Put Value to Attribute | | | If set, the retrieved value will be put into an attribute of the FlowFile instead of a the content of the FlowFile. The attribute key to put to is determined by evaluating value of this property.<br/>**Supports Expression Language: true** |

### Relationships

| Name | Description |
|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| success | Values retrieved from Couchbase Server are written as outgoing FlowFiles content or put into an attribute of the incoming FlowFile and routed to this relationship. |
| failure | All FlowFiles failed to fetch from Couchbase Server and not retry-able are routed to this relationship. |
| retry | All FlowFiles failed to fetch from Couchbase Server but can be retried are routed to this relationship. |
| original | The original input FlowFile is routed to this relationship when the value is retrieved from Couchbase Server and routed to 'success'. |

### Output Attributes

| Attribute | Relationship | Description |
|----------------------|--------------|---------------------------------------|
| couchbase.bucket | success | Bucket where the document was stored. |
| couchbase.doc.id | success | Id of the document. |
| couchbase.doc.cas | success | CAS of the document. |
| couchbase.doc.expiry | success | Expiration of the document. |


## GetFile

### Description
Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ The next table outlines CMAKE flags that correspond with MiNiFi extensions. Exte
| AWS | [AWSCredentialsService](CONTROLLERS.md#awscredentialsservice)<br/>[PutS3Object](PROCESSORS.md#puts3object)<br/>[DeleteS3Object](PROCESSORS.md#deletes3object)<br/>[FetchS3Object](PROCESSORS.md#fetchs3object)<br/>[ListS3](PROCESSORS.md#lists3) | -DENABLE_AWS=ON |
| Azure | [AzureStorageCredentialsService](CONTROLLERS.md#azurestoragecredentialsservice)<br/>[PutAzureBlobStorage](PROCESSORS.md#putazureblobstorage)<br/>[DeleteAzureBlobStorage](PROCESSORS.md#deleteazureblobstorage)<br/>[FetchAzureBlobStorage](PROCESSORS.md#fetchazureblobstorage)<br/>[ListAzureBlobStorage](PROCESSORS.md#listazureblobstorage)<br/>[PutAzureDataLakeStorage](PROCESSORS.md#putazuredatalakestorage)<br/>[DeleteAzureDataLakeStorage](PROCESSORS.md#deleteazuredatalakestorage)<br/>[FetchAzureDataLakeStorage](PROCESSORS.md#fetchazuredatalakestorage)<br/>[ListAzureDataLakeStorage](PROCESSORS.md#listazuredatalakestorage) | -DENABLE_AZURE=ON |
| CivetWeb | [ListenHTTP](PROCESSORS.md#listenhttp) | -DENABLE_CIVET=ON |
| Couchbase | [CouchbaseClusterService](CONTROLLERS.md#couchbaseclusterservice)<br/>[PutCouchbaseKey](PROCESSORS.md#putcouchbasekey) | -DENABLE_COUCHBASE=ON |
| Couchbase | [CouchbaseClusterService](CONTROLLERS.md#couchbaseclusterservice)<br/>[PutCouchbaseKey](PROCESSORS.md#putcouchbasekey)<br/>[GetCouchbaseKey](PROCESSORS.md#getcouchbasekey) | -DENABLE_COUCHBASE=ON |
| Elasticsearch | [ElasticsearchCredentialsControllerService](CONTROLLERS.md#elasticsearchcredentialscontrollerservice)<br/>[PostElasticsearch](PROCESSORS.md#postelasticsearch) | -DENABLE_ELASTICSEARCH=ON |
| ExecuteProcess (Linux and macOS) | [ExecuteProcess](PROCESSORS.md#executeprocess) | -DENABLE_EXECUTE_PROCESS=ON |
| Google Cloud Platform | [DeleteGCSObject](PROCESSORS.md#deletegcsobject)<br>[FetchGCSObject](PROCESSORS.md#fetchgcsobject)<br>[GCPCredentialsControllerService](CONTROLLERS.md#gcpcredentialscontrollerservice)<br>[ListGCSBucket](PROCESSORS.md#listgcsbucket)<br>[PutGCSObject](PROCESSORS.md#putgcsobject) | -DENABLE_GCP=ON |
Expand Down
4 changes: 2 additions & 2 deletions docker/test/integration/cluster/containers/CouchbaseServerContainer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ def run_post_startup_commands(self):
["couchbase-cli", "cluster-init", "-c", "localhost", "--cluster-username", "Administrator", "--cluster-password", "password123", "--services", "data,index,query",
"--cluster-ramsize", "2048", "--cluster-index-ramsize", "256"],
["couchbase-cli", "bucket-create", "-c", "localhost", "--username", "Administrator", "--password", "password123", "--bucket", "test_bucket", "--bucket-type", "couchbase",
"--bucket-ramsize", "1024"]
"--bucket-ramsize", "1024", "--max-ttl", "36000"]
]
for command in commands:
(code, _) = self.client.containers.get(self.name).exec_run(command)
Expand All @@ -47,7 +47,7 @@ def deploy(self):
return

self.docker_container = self.client.containers.run(
"couchbase:community-7.6.2",
"couchbase:enterprise-7.2.5",
szaszm marked this conversation as resolved.
Show resolved Hide resolved
detach=True,
name=self.name,
network=self.network.name,
Expand Down
Loading