[PY-407][PY-534][PY-404] Item + ItemQuery objects #715

Nathanjp91 · 2023-11-13T09:06:43Z

Problem

Item + ItemQuery doesn't exist, need to be added in so can manage and download these items. Also add in associated methods and features to objects that have Items as well as methods that items should have

Solution

Added in these objects

Item Meta now exists along with ItemQuery
delete() method on both item and itemquery that requests the item/s be deleted on the server
objects that interact with items have an items property that maps to a ItemQuery
Small changes to QueryString for easier typing

Changelog

Items now queryable with paginated support

linear · 2023-11-13T09:06:46Z

PY-407 Create ItemQuery object and relevant functions/filters

https://www.notion.so/v7labs/Item-Management-Functionality-Scoping-df5f846a135e46e1bf31c473afda32ee?pvs=4#16946909a4aa4929a43e4592ffd209a4

Create an ItemQuery object and functions and filters to work with items as needed.

item = team.items.get(id)
# - or -
item = team.item.where(query).first() # or .one() etc., depending on API

Note: Should plan to only support Advanced Filters

Nathanjp91 · 2023-11-13T09:35:23Z

darwin/future/core/types/common.py

+                    output += f"{k}={x}&"
+            else:
+                output += f"{k}={v}&"
+        return output[:-1]  # remove trailing &



This had to be changed and added to facilitate that a key can map to multiple values, eg dataset_id=0000&dataset_id=0001

Nathanjp91 · 2023-11-13T09:36:08Z

darwin/future/data_objects/item.py

@@ -75,7 +75,7 @@ def validate_fps(cls, values: dict) -> dict:
        elif isinstance(value, (int, float)):
            type = values.get("type")
            if type == "image":
-                assert value == 0, "fps must be 0 for images"
+                assert value == 0 or value == 1.0, "fps must be '0' or '1.0' for images"


turns out on our api, 1.0 also exists for items that are images. Found in dogfooding.

darwin/future/core/items/delete_items.py

Nathanjp91 · 2023-11-13T09:43:10Z

darwin/future/core/types/common.py


 JSONType = Union[Dict[str, Any], List[Dict[str, Any]]]  # type: ignore


+class Implements_str(Protocol):


so this protocol should mean it'll work with anything that can be converted to a string, so things like ints, floats etc will work.

Nathanjp91 · 2023-11-13T09:44:36Z

darwin/future/core/types/common.py


-    def __init__(self, value: Dict[str, str]) -> None:
+    def __init__(self, value: Mapping[str, list[Stringable] | Stringable]) -> None:


Mapping here is a more free-form Dict. As per the mypy docs, it's better to use the constructs like Mapping/Sequence as input parameters, and more concrete ones like dict/list as return types.

Nathanjp91 · 2023-11-13T09:46:08Z

darwin/future/core/types/common.py

+        ...
+
+
+Stringable = Union[str, Implements_str]


I'm not entirely sure why this union has to exist, but I couldn't get the type system to properly work without it. One would expect implements_str to encompass str natively, but it doesn't.

linear · 2023-11-13T10:14:15Z

PY-534 Change stage.item_ids to stage.items

Currently, WFStage has a member, item_ids which returns a list of the UUIDs of the stage items.

Change this to the actual stage objects, when possible.

Acceptance Criteria

WFStage has items member populated with item data structures, instead of a list of IDs

PY-404 Create delete method on item

https://www.notion.so/v7labs/Item-Management-Functionality-Scoping-df5f846a135e46e1bf31c473afda32ee?pvs=4#16946909a4aa4929a43e4592ffd209a4

items = team.items.where(query)

[item.delete() for item in items]

Both items and query should have their own delete. (This applies to all batch actions)

owencjones

Some comments for consideration, but overall LGTM

owencjones · 2023-11-13T09:32:13Z

darwin/future/core/items/delete_items.py

    payload = {
        "filters": {
-            "dataset_ids": dataset_ids
+            "dataset_ids": [str(item) for item in dataset_ids]


Wait, so this accepts "all" as a possible input? That surprises me.

all our endpoints do, although I have just removed that because I do think it's dangerous. Also it makes typing a bit of a nightmare anyway.

darwin/future/data_objects/item.py

owencjones · 2023-11-13T10:45:21Z

darwin/future/tests/core/types/test_querystring.py

@@ -13,12 +11,23 @@ def test_querystring_happy_path() -> None:
    assert str(query_string_2) == "?foo=bar&baz=qux"

    query_string_3 = QueryString({})
-    assert str(query_string_3) == "?"
+    assert str(query_string_3) == ""


Yeah, protocols do work much better for this eh.

Nathan Perkins and others added 17 commits November 8, 2023 16:05

ItemCore changes for Query

242e264

ItemQuery

5b276a4

Changes for Item Query

fc1130c

filters addition

77b74c9

filters addition

a5cd5f5

teams + dataset: ItemQuery

db90889

linting fixes

3a10558

changes to core api delete

afcc21b

delete typing

241219f

fixes to delete and item instantiation

9ff039c

linting

918ce11

stage items

bfe9706

items query test basics

e4bcdc1

linting

b78bb16

tests for item + query

1f9227c

linting

927b19e

cleanup

75f2e4a

Nathanjp91 added 3 commits November 13, 2023 09:20

merge master and tests changes

ab47096

workflow items

a2e1b9b

item documentation

fab19e6

Nathanjp91 commented Nov 13, 2023

View reviewed changes

darwin/future/core/items/delete_items.py Outdated Show resolved Hide resolved

Nathanjp91 commented Nov 13, 2023

View reviewed changes

Nathanjp91 changed the title ~~[PY-407] Item + ItemQuery objects~~ [PY-407][PY-534][PY-404] Item + ItemQuery objects Nov 13, 2023

owencjones approved these changes Nov 13, 2023

View reviewed changes

Nathanjp91 added 2 commits November 13, 2023 10:55

comments on delete

0efaedd

non-working validator removed

945cdde

Nathanjp91 merged commit 0efb835 into master Nov 13, 2023
13 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[PY-407][PY-534][PY-404] Item + ItemQuery objects #715

[PY-407][PY-534][PY-404] Item + ItemQuery objects #715

Nathanjp91 commented Nov 13, 2023

linear bot commented Nov 13, 2023

Nathanjp91 Nov 13, 2023

Nathanjp91 Nov 13, 2023

Nathanjp91 Nov 13, 2023

Nathanjp91 Nov 13, 2023

Nathanjp91 Nov 13, 2023

linear bot commented Nov 13, 2023

owencjones left a comment

owencjones Nov 13, 2023

Nathanjp91 Nov 13, 2023

owencjones Nov 13, 2023


		JSONType = Union[Dict[str, Any], List[Dict[str, Any]]] # type: ignore


		class Implements_str(Protocol):


		def __init__(self, value: Dict[str, str]) -> None:
		def __init__(self, value: Mapping[str, list[Stringable] \| Stringable]) -> None:

[PY-407][PY-534][PY-404] Item + ItemQuery objects #715

[PY-407][PY-534][PY-404] Item + ItemQuery objects #715

Conversation

Nathanjp91 commented Nov 13, 2023

Problem

Solution

Changelog

linear bot commented Nov 13, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

linear bot commented Nov 13, 2023

owencjones left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment