Ground truthing can only work once per image

[simbamangu]

Problem: When an image has been ground truthed, it cannot be reviewed again even if it is added to a new task.

Expected behaviour: Images should be able to be added to a new task then reviewed / ground truthed again.

[TanyaStere42]

Reason it is how it is

When working with a team that is annotating the same images and the goal is performance stats between users, you don't need to ground truth repeatedly. A single GT gets you the analytics you need regarding user performance.

That reason still seems applicable, so I think this comes down to "we need the platform to be more flexible and not lock us into a single mode of processing". Something where there's a toggle to either allow or prevent multiple reviews of an image. The important thing here would be designing in a way where the user still has a concept of "100% complete"

Proposed design

Create task forms gets a new checkbox labeled "Ground Truth All Images" (default set to on). If checked, lab leads will need to GT all images in that task, regardless of if they've been reviewed before. If unchecked, the system will check for a GT for that image and accept it as the final GT.

Feedback needed

What happens when there are two ground truths? How would you expect that to be resolved in the annotation export? Does it take the latest GT as fact? Does it merge them? Do you have to GT your GT to have a single source of truth?

[simbamangu]

Given the experimental nature of some of this work, and the fact that ground-truthing isn't necessarily 100% accurate, having the option on a per-task basis to GT again would be important.

I'd propose that any task can be GT'd again - though having the option in the 'create task' dialog also needed.

[TanyaStere42]

So, in your mind, we would just store the latest version of the GT? If you chose to redo the ground truth, it would wipe the previous state?