Error: Object with ID -198880863 already exists in the frame

[diogomafra]

We recently updated our system from Savant 0.2.11 to 0.4 and started seeing the errors below. These errors happen on a step that creates a custom ROI.

 ERROR python::exception                                    > [python.exception.type=<class 'AttributeError'>, python.exception.value='ObjectMeta' object has no attribute '_attributes', python.exception.traceback=Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/savant/deepstream/pyfunc.py", line 174, in process_buffer
    self.process_frame(buffer, frame_meta)
  File "/opt/savant/samples/license_plate_recognition/custom_roi.py", line 141, in process_frame
    roi_attribute = custom_roi_object.get_attr_meta(
  File "/usr/local/lib/python3.10/dist-packages/savant/meta/object.py", line 158, in get_attr_meta
    attrs = self.get_attr_meta_list(element_name, attr_name)
  File "/usr/local/lib/python3.10/dist-packages/savant/meta/object.py", line 147, in get_attr_meta_list
    return self._attributes.get((element_name, attr_name), None)
, python.version=3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]] Exception occurred

 ERROR python::exception                                    > [python.exception.type=<class 'ValueError'>, python.exception.value=Object with ID -198880863 already exists in the frame., python.version=3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0], python.exception.traceback=Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/savant/deepstream/pipeline.py", line 815, in update_frame_meta
    self._update_meta_for_single_frame(
  File "/usr/local/lib/python3.10/dist-packages/savant/deepstream/pipeline.py", line 886, in _update_meta_for_single_frame
    obj_meta = nvds_obj_meta_output_converter(
  File "/usr/local/lib/python3.10/dist-packages/savant/deepstream/metadata.py", line 84, in nvds_obj_meta_output_converter
    return video_frame.add_object(
] Exception occurred
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/savant/deepstream/pipeline.py", line 815, in update_frame_meta
    self._update_meta_for_single_frame(
  File "/usr/local/lib/python3.10/dist-packages/savant/deepstream/pipeline.py", line 886, in _update_meta_for_single_frame
    obj_meta = nvds_obj_meta_output_converter(
  File "/usr/local/lib/python3.10/dist-packages/savant/deepstream/metadata.py", line 84, in nvds_obj_meta_output_converter
    return video_frame.add_object(
ValueError: Object with ID -198880863 already exists in the frame.

Additional details

Here's a branch where I'm reproducing the issue. It only happens when many objects are in the frame, so I had to force that scenario by creating multiple YOLO steps.

The pipeline in the example does the following:

• runs YOLO
• creates a custom ROI based on all yolo objects (here's the code)
• runs a fake OCR step that adds an attribute to the custom ROI object (code)
• moves the attribute from the custom object to the original one (code)

While looking at the Savant's source code I noticed this PR, it changed a call to video_frame.create_object with video_frame.add_object, which is the function that triggers the error.

[bwsw]

@diogomafra we will investigate the problem soon.

[bwsw]

@diogomafra just to potentially offer you a workaround, could you tell me why you need to create a custom ROI per object detected by YOLO? I mean, if you want to tweak boxes, you could use the preprocess object meta feature of the downstream OCR nvinfer block.

[diogomafra]

TL;DR: We create the custom ROI because a model step cannot have multiple inputs.

Our pipeline has a YOLO model with multiple labels, some of these labels must be passed to the OCR step. The ideal pipeline would look like the following:

• yolo step
  • output
    • labels: tractor, tractor_number, license_plate
• OCR step:
  • input: [yolo.tractor_number, yolo.license_plate]

I know we could have one OCR step per label, but we're not doing that for 3 reasons:

• load the model only once
• use batching
• the same model cannot be used in multiple steps, it only works if we copy it to a different folder (that's probably a bug)

Do you have any other suggestions?

By the way, this is important but not urgent right now, enjoy the holidays 😄

[bwsw]

Ah, I see the use case. Definitely, the use case must be supported. Please keep tracking the issue.

[bwsw]

@diogomafra could you please check the fix?

https://github.com/insight-platform/Savant/pull/897/files

Clone the PR branch; run make publish-local and test the operation. If it works well, we will merge it and port it to 0.4.x as well.

[diogomafra]

@bwsw your PR fixed one of the errors (Object already exists in the frame), but I'm still getting the other ('ObjectMeta' object has no attribute '_attributes').

I tested it with both versions, your PR (develop) and 0.4.9.

 ERROR insight::savant::pyfunc                              > Error in process_buffer() call for PyFunc(module='samples.custom_roi_error.custom_roi', class_name='MergeROI', kwargs={'roi': 'ocr.roi', 'attribute': 'fake-ocr.text'}, dev_mode=False).
Aliases for entries in sys.path:
    <distpkg>: /usr/local/lib/python3.10/dist-packages
    <pwd>    : /opt/savant
    <pwd>    : /opt/savant
Traceback (most recent call last):
    <pwd> /gst_plugins/python/pyfunc.py:249            do_transform_ip     self.pyfunc.instance.process_buffer(buffer)
    <distpkg> /savant/deepstream/pyfunc.py:174         process_buffer      self.process_frame(buffer, frame_meta)
    <pwd> /samples/custom_roi_error/custom_roi.py:141  process_frame       roi_attribute = custom_roi_object.get_attr_meta(
    <distpkg> /savant/meta/object.py:158               get_attr_meta       attrs = self.get_attr_meta_list(element_name, attr_name)
    <distpkg> /savant/meta/object.py:147               get_attr_meta_list  return self._attributes.get((element_name, attr_name), None)
AttributeError: 'ObjectMeta' object has no attribute '_attributes'

[diogomafra]

EDIT: the problem is more complex than that. As I understand the object should have either _attributes or object_meta_impl, but both are None, causing the issue.

I can see only one reason for the ObjectMeta not having the attribute _attributes... The method _from_be_object_meta creates a new instance of ObjectMeta calling __new__, so __init__ is not called and the attribute is not defined.

_from_be_object_meta is called inside the parent property.

[bwsw]

We will explore it a bit later. I have no required knowledge. We have samples working with attributes, do you follow them?

[diogomafra]

@bwsw I noticed something interesting. frame_meta.objects is returning duplicated objects (same uid twice).

The problem I'm seeing occurs because I'm removing the object from the frame, and the next time the same object appears in the loop the attribute object_meta_impl is already None.

I can work around this issue, but I don't think the frame_meta.objects should return the same object twice, right?

# this list can have the same object twice in some cases
custom_roi_objs = [o for o in frame_meta.objects if obj.element_name == "custom_roi"]

# here we iterate over the list and remove from the frame_meta all the objects
# for duplicated objects, the second time we see them the `object_meta_impl` is None an the code fails
for obj in custom_roi_objs:

    # do something

    # remove object
    frame_meta.remove_obj_meta(custom_roi_object)

[diogomafra]

@bwsw can you please release the fix you already have? The second problem I can work around in my code.

[bwsw]

@diogomafra I cannot merge it because I do not understand the root cause. We need to wait developers get back from vacations and investigate your case under a more closer look to elaborate the qualified solution. So, for now, you can use a workaround but I do not want to publish it because we need a general fix after understanding the root cause and the use pattern.

[bwsw]

@diogomafra, we have pulled the issue into the development. Please wait a bit more :)

[denisvmedyantsev]

@diogomafra The problem is the non-unique object uid, you duplicate objects and this leads to the issue you described. We changed the way the uid is generated. Could you please check the PR #912?

[diogomafra]

Thank you for working on that @denisvmedyantsev . I'll test it tomorrow and get back to you ASAP.

[diogomafra]

@denisvmedyantsev I just confirmed your PR fixes both errors I was seeing!