Problem (bug?) with RegionOfInterest X and Y

[PedroACFerreira]

Hey there,
Easy to replicate, open a video on a FileCapture node, connect to Crop node, take out the RegionOfInterest X or Y. The values do not update when you make a new crop, and default to 0. I've tried a couple of things, but can't get it to update while the workflow is ongoing. Is it not possible to get current X and Y values of the drawn ROI? Or am I missing something?
Thanks!

[PedroACFerreira]

Height and Width update just fine, but X and Y are stuck to 0

[bruno-f-cruz]

Can you add a workflow that reproduces the bug? thanks

[PedroACFerreira]

ROI X Bug.zip

Just open literally any video with it (I'm using .avi files). You can see there that regardless of Crop Region values, X does not change in the Member Selector, although the Width and Height do and represent the correct values.

[bruno-f-cruz]

I am not sure this is a bug. I would defer to @glopesdev since i am not sure if the image metadata should be updated on crop or not. Currently, the image object doesn't seem to have any sort of metadata information associated with past transformations, this would effectively break that.

Anyway, for a potential solution, you could instead define your crop roi and use that information:

Workflow (Copy+Paste me)

<?xml version="1.0" encoding="utf-8"?>
<WorkflowBuilder Version="2.8.5"
                 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                 xmlns:cv="clr-namespace:Bonsai.Vision;assembly=Bonsai.Vision"
                 xmlns:p1="clr-namespace:OpenCV.Net;assembly=OpenCV.Net"
                 xmlns:rx="clr-namespace:Bonsai.Reactive;assembly=Bonsai.Core"
                 xmlns="https://bonsai-rx.org/2018/workflow">
  <Workflow>
    <Nodes>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="cv:FileCapture">
          <cv:FileName />
          <cv:PlaybackRate>0</cv:PlaybackRate>
          <cv:StartPosition>0</cv:StartPosition>
          <cv:PositionUnits>Frames</cv:PositionUnits>
          <cv:Loop>false</cv:Loop>
          <cv:Playing>true</cv:Playing>
        </Combinator>
      </Expression>
      <Expression xsi:type="PropertySource" TypeArguments="cv:Crop,p1:Rect">
        <MemberName>RegionOfInterest</MemberName>
        <Value>
          <X>82</X>
          <Y>80</Y>
          <Width>26</Width>
          <Height>27</Height>
        </Value>
      </Expression>
      <Expression xsi:type="rx:BehaviorSubject">
        <Name>roi</Name>
      </Expression>
      <Expression xsi:type="PropertyMapping">
        <PropertyMappings>
          <Property Name="RegionOfInterest" />
        </PropertyMappings>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="cv:Crop">
          <cv:RegionOfInterest>
            <cv:X>82</cv:X>
            <cv:Y>80</cv:Y>
            <cv:Width>26</cv:Width>
            <cv:Height>27</cv:Height>
          </cv:RegionOfInterest>
        </Combinator>
      </Expression>
      <Expression xsi:type="SubscribeSubject">
        <Name>roi</Name>
      </Expression>
      <Expression xsi:type="MemberSelector">
        <Selector>Width</Selector>
      </Expression>
    </Nodes>
    <Edges>
      <Edge From="0" To="4" Label="Source1" />
      <Edge From="1" To="2" Label="Source1" />
      <Edge From="2" To="3" Label="Source1" />
      <Edge From="3" To="4" Label="Source2" />
      <Edge From="5" To="6" Label="Source1" />
    </Edges>
  </Workflow>
</WorkflowBuilder>

[PedroACFerreira]

Let me see if I can explain myself better. ROIs are, as far as I understand, OpenCV.Rect objects in the (X,Y,Width,Height) format.
Just assum I need the values coming out of the RegionOfInterest that is drawn in the Crop node, and that I need it to be updated if someone using the workflow redraws the square. The width and height I can get easily, and update if I update the crop. X and Y I can't, they are always 0, although they are a part of the same Object.

And even if we are not talking about updating values on the go, the X value is not even accurate when you start the workflow. Imagine I draw a ROI that is a OpenCV.Rect with (50,50,100,100). If you click the Crop node it will say on the right of the screen that the X value is indeed 50, and the crop is indeed drawn properly. However, using the Member Selector for X and Y is showing 0, but for Width and Height it shows the correct value of 100.

So, if its all coming out of the same object, and 2 of the 4 values display properly, why don't X and Y?
Btw, this happens with any node that draws a ROI, not just Crop

[bruno-f-cruz]

You can edit the ROI directly in the ROI constructor instead of the Crop operator. You need to pass the original image such that the ROI visualizer can have something to draw on.

So, if its all coming out of the same object, and 2 of the 4 values display properly, why don't X and Y?

The width and height are properties of the image that can always be calculated whether the image is an orphan or not. I can give you an image and you will always be able to know width and height. Offsets are not the case, I can change those values and the image will not change, since they are simply tracking metadata from a previous pipeline. Once again, I don't think this is technical problem (bug), it's more a conceptual one, does it make sense to keep track of this information or not? From very little thought, a few questions are already popping up in my mind:

1. If you apply 2 crops in a row, do you use some sort of composition from the very first image source OR do you reset on each operation?
2. If you rescale the image, how do you make sense of these values?
3. Merging 2 images?
   This list is likely to grow....

[PedroACFerreira]

So, somehow, using the RegionOfInterest Constructor to set the ROI like you suggested here, connected to FileCapture, I can get an updated X value to use elsewhere in the workflow. Why the X value gets updated if you use the RegionOfInterest node but not the Crop node, no idea, but my problem is solved for now! Thanks!