Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enable diagram detect based on file instance id #1887

Merged
merged 14 commits into from
Sep 3, 2024

Conversation

olacognite
Copy link
Contributor

@olacognite olacognite commented Aug 26, 2024

Description

Make diagram detect available by file_instance_id.

I have done it similarly to id and external id, but could consider to only make it available with file references, since file references with page ranges is the recommended way to run (at least when documents have variable numbers of pages).

PR in context-api is here

Checklist:

  • Tests added/updated.
  • Documentation updated. Documentation is generated from docstrings - these must be updated according to your change.
    If a new method has been added it should be referenced in cognite.rst in order to generate docs based on its docstring.
  • Changelog updated in CHANGELOG.md.
  • Version bumped. If triggering a new release is desired, bump the version number in _version.py and pyproject.toml per semantic versioning.

Copy link

codecov bot commented Aug 26, 2024

Codecov Report

Attention: Patch coverage is 88.88889% with 2 lines in your changes missing coverage. Please review.

Project coverage is 90.69%. Comparing base (95178fd) to head (9c61005).

Files with missing lines Patch % Lines
cognite/client/data_classes/contextualization.py 81.81% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1887      +/-   ##
==========================================
+ Coverage   90.62%   90.69%   +0.06%     
==========================================
  Files         126      126              
  Lines       19609    19617       +8     
==========================================
+ Hits        17771    17791      +20     
+ Misses       1838     1826      -12     
Files with missing lines Coverage Δ
cognite/client/_api/diagrams.py 85.26% <100.00%> (+0.48%) ⬆️
cognite/client/_version.py 100.00% <100.00%> (ø)
cognite/client/data_classes/contextualization.py 92.44% <81.81%> (+0.24%) ⬆️

... and 5 files with indirect coverage changes

@olacognite olacognite marked this pull request as ready for review August 30, 2024 09:29
@olacognite olacognite requested review from a team as code owners August 30, 2024 09:29
@olacognite olacognite requested review from dimitryhoff and removed request for a team August 30, 2024 09:29
Copy link
Contributor

@doctrino doctrino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some nit picking :)

cognite/client/data_classes/contextualization.py Outdated Show resolved Hide resolved
Comment on lines 376 to 381
if self.file_id is None and self.file_external_id is not None:
item: dict[str, str | int | dict[str, int]] = {"fileExternalId": self.file_external_id}
item: dict[str, str | int | dict[str, int] | dict[str, str]] = {"fileExternalId": self.file_external_id}
if self.file_id is not None and self.file_external_id is None:
item = {"fileId": self.file_id}
if self.file_instance_id is not None:
item = {"fileInstanceId": self.file_instance_id.dump(include_instance_type=False)}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A situation that should never happen, but this is more robust:

        item: dict[str, str | int | dict[str, int] | dict[str, str]] = {}
        if self.file_id is None and self.file_external_id is not None and self.file_instance_id is None:
            item["fileExternalId"] = self.file_external_id
        elif self.file_id is not None and self.file_external_id is None and self.file_instance_id is None:
            item["fileId"] = self.file_id
        elif self.file_instance_id is not None and self.file_id is None and self.file_external_id is None:
            item["fileInstanceId"] = self.file_instance_id.dump(include_instance_type=False)
        else:
            raise ValueError("Exactly one of file_id, file_external_id and file_instance_id must be set")


PNID_FILE_ID = 3261066797848581
DIAGRAM_SPACE = "diagram_space"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make a fixture to guarantee that this space will exist?

Something like

@pytest.fixture(scope="session")
def diagram_space(cognite_client: CogniteClient) -> Space:
    return cognite_client.data_modeling.spaces.apply(SpaceApply(space=DIAGRAM_SPACE))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, could do that, but should also make sure that the file exists in that case. If the space does not exist, then the file cant exist either, so maybe add a file to the repo and upload it if it is not there or what do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I missed that. I would also make another fixture for the file then. If the file is not too big or sensitive then commit it to the repo.

@@ -103,3 +106,20 @@ def test_run_diagram_detect_in_pattern_mode(self, cognite_client):

assert len(detected_by_resource_type["file_reference"]) >= 10 # 14 seen when making the test
assert len(detected_by_resource_type["instrument"]) >= 60 # 72 seen when making the test

def test_run_diagram_detect_with_file_instance_id(self, cognite_client):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def test_run_diagram_detect_with_file_instance_id(self, cognite_client):
@pytest.mark.usefixtures("diagram_space")
def test_run_diagram_detect_with_file_instance_id(self, cognite_client):

@doctrino doctrino self-requested a review September 2, 2024 12:04
@olacognite olacognite merged commit bdde64c into master Sep 3, 2024
14 checks passed
@olacognite olacognite deleted the DOG-3974_dm_reference_in_diagram_detect branch September 3, 2024 05:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants