Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NR-347971 | OpenLineage event consumer writer #1887

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open
5 changes: 5 additions & 0 deletions entity-types/agent-openlineage/definition.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
domain: AGENT
type: OPENLINEAGE
configuration:
entityExpirationTime: MANUAL
alertable: false
27 changes: 27 additions & 0 deletions entity-types/dpo-dataset_versions/definition.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
domain: DPO
type: DATASET_VERSIONS
synthesis:
rules:
- compositeIdentifier:
separator: ":"
attributes:
- dataset.namespace
devendra-nr marked this conversation as resolved.
Show resolved Hide resolved
- dataset.name
- updatedAt
name: dataset.name
encodeIdentifierInGUID: true
conditions:
- attribute: dataset.name
tags:
dataset.namespace:
entityTagName: olin.dataset.namespace
multiValue: false
dataset.name:
entityTagName: olin.dataset.name
multiValue: false
dataset.updatedAt:
entityTagName: olin.dataset.updatedAt
multiValue: false
configuration:
entityExpirationTime: MANUAL
alertable: true
23 changes: 23 additions & 0 deletions entity-types/dpo-datasets/definition.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
domain: DPO
type: DATASETS
synthesis:
rules:
- compositeIdentifier:
separator: ":"
attributes:
- dataset.namespace
- dataset.name
name: dataset.name
encodeIdentifierInGUID: true
conditions:
- attribute: dataset.name
tags:
dataset.namespace:
entityTagName: olin.dataset.namespace
multiValue: false
dataset.name:
entityTagName: olin.dataset.name
multiValue: false
configuration:
entityExpirationTime: MANUAL
alertable: true
30 changes: 30 additions & 0 deletions entity-types/dpo-job_versions/definition.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
domain: DPO
type: JOB_VERSIONS
synthesis:
rules:
- compositeIdentifier:
separator: ":"
attributes:
- job.namespace
- job.name
- updatedAt
name: job.name
encodeIdentifierInGUID: true
conditions:
- attribute: job.name
tags:
job.namespace:
entityTagName: olin.job.namespace
multiValue: false
job.name:
entityTagName: olin.job.name
multiValue: false
job.updatedAt:
entityTagName: olin.job.updatedAt
multiValue: false
pipeline.name:
entityTagName: olin.pipeline.name
multiValue: false
configuration:
entityExpirationTime: MANUAL
alertable: true
27 changes: 27 additions & 0 deletions entity-types/dpo-jobs/definition.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
domain: DPO
type: JOBS
synthesis:
rules:
- compositeIdentifier:
separator: ":"
attributes:
- job.namespace
- job.name
- eventTime
name: job.name
encodeIdentifierInGUID: true
conditions:
- attribute: job.name
tags:
job.namespace:
entityTagName: olin.job.namespace
multiValue: false
job.name:
entityTagName: olin.job.name
multiValue: false
pipeline.name:
entityTagName: olin.pipeline.name
multiValue: false
configuration:
entityExpirationTime: MANUAL
alertable: true
23 changes: 23 additions & 0 deletions entity-types/dpo-pipelines/definition.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
domain: DPO
type: PIPELINES
synthesis:
rules:
- compositeIdentifier:
separator: ":"
attributes:
- pipeline.namespace
- pipeline.name
name: pipeline.name
encodeIdentifierInGUID: true
conditions:
- attribute: pipeline.name
tags:
pipeline.namespace:
entityTagName: olin.pipeline.namespace
multiValue: false
pipeline.name:
entityTagName: olin.pipeline.name
multiValue: false
configuration:
entityExpirationTime: MANUAL
alertable: true