-
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
0da61fc
commit 30ea70e
Showing
21 changed files
with
152 additions
and
146 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
# Delete Data | ||
|
||
|
||
## Delete Users | ||
At present, there is no way to delete users. | ||
|
||
## Delete Images | ||
At present, there is no way through the UI to delete images from the system. Instead, you must make edits to the mongoDB directly. | ||
|
||
## Delete Tasks | ||
When you delete a task, the associated annotations are removed from the system. When you delete a task that has been through ground truth, the ground truth confirmations are removed from the system, which can impact annotation export analysis. When you delete a task that has the last instance of a tag on it, the tag is not removed from the system. To remove tags, you must delete them individually. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
docs/lab-lead/hot-keys.md → docs/administration/manage-labels.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
2 changes: 1 addition & 1 deletion
2
...ab-lead/evaluate-annotator-performance.md → docs/evaluate-annotator-performance.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
# Data Exports | ||
|
||
## Exporting Task Annotation and Image Data for Analysis | ||
|
||
Data can be exported for third-party application analysis and modeling by generating image and annotation data as `CSV` for one or more tasks selected in the Tasks view. Here's how to export data: | ||
|
||
1. Click **Tasks** in the header navigation menu to open the Tasks view. | ||
2. In the Tasks view, use the filters to filter the table down to only those tasks that you want to export. | ||
3. Click the **Export Filtered Results** button and then select **Export Annotations** or **Export Images** options to choose the content of the `CSV` file to export. | ||
4. A progress bar indicates the compilation progress of the downloadable `CSV` file. When the progress bar completes, click Download to download the `CSV` file. | ||
|
||
![ttexport](assets/images/tasktable-exportoptions.png) | ||
|
||
## Export Images CSV Data | ||
|
||
The following fields are available in the Export Images CSV file: | ||
|
||
* **Task Name** - Name of the task this image was included in. An image may be included in multiple tasks, and if so, the task name is repeated on multiple lines in this file. | ||
* **Task ID** - The **Scout internal ID** of the task this image belongs to. | ||
* **Image Filename** - Original filename of the image. | ||
* **WIC Confidence** - *Machine learning score* predicting confidence of one or more labels present in the image. The score ranges from 0 (nothing predicted in the image) to 1 (maximum confidence that at least one ML model category would be found in the image). *If ML has not been run on this image, the WIC Confidence is empty.* | ||
* **Ground Truth Status** - True/False value indicating whether review of this image has been completed in the Ground Truth task stage. | ||
* **Exclusion Side** - The side of the Division Line that is exclude in counts of animals in overlapping images. This value is always: **right.** | ||
* **Inclusion Top X Fraction** - The decimal fraction (0 to 1) of the top insertion point of the division line. This is the fraction of the image width from the left-side of the image at which the top of the division line intersects the bottom of the image. | ||
* **Inclusion Bottom X Fraction** - The decimal fraction (0 to 1) of the bottom insertion point of the division line. This is the fraction of the image width from the left-side of the image at which the bottom of the division line intersects the bottom of the image. | ||
|
||
## Export Annotation CSV Data | ||
|
||
The following fields are available in the Export Annotations CSV file: | ||
|
||
* **Task Name** - The name of the task that the annotation is included. | ||
* **Task ID** - The *Scout internal ID* of the task this annotation belongs to. | ||
* **Image Filename** - Original filename of the image. | ||
* **Image ID** - *Scout internal ID* of the image. | ||
* **Box X** - *X coordinate* of the annotation bounding box’s top left corner. | ||
* **Box Y** - *Y coordinate* of the annotation bounding box’s top left corner. | ||
* **Box W** - Pixel width of the bounding box. | ||
* **Box H** - Pixel height of the bounding box. | ||
* **Box Confidence** - If machine learning created an annotation, this is the confidence score of the bounding box of this annotation. The score ranges from 0 *(no confidence)* to 1 *(maximum confidence that the box is correctly predicted)*. | ||
* **Classification** - The predicted classification (i.e. *type*) of the bounding box, such as *elephant* or *buffalo*. | ||
* **Classification Confidence** - If machine learning created this annotation, this is the confidence score of the classification (e.g., *elephant*) assigned to this annotation. The score ranges from 0 *(no confidence)* to 1 *(maximum confidence that the classification is correct)*. | ||
* **Assignee** - The Scout user that drew the bounding box. | ||
* **Timestamp** - Timestamp of annotation creation. | ||
* **Is Ground Truth** - Whether this annotation has been approved in the Ground Truth process. Annotations that have this field as True are likely to be more reliably accurate as they have been reviewed independently. | ||
* **Excluded By Line** - Whether the centroid of this bounding box falls outside or inside the division line. | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
# Getting Started | ||
|
||
Your Lab Lead will provide you with the URL to Scout, as well as your username and password. Once you have these, open your browser on your local computer and connect to Scout. Use your username and password to log in. | ||
|
||
```{note} | ||
Log out of Scout after your work. Scout does not automatically log you out, so doing it manually protects your account and work. | ||
``` | ||
|
||
## Finding Your Tasks | ||
|
||
After login, you should see the **Tasks page** as a landing page. The page is filtered beforehand to your assigned tasks. Use filters to find specific tasks to annotate. | ||
![taskpage](assets/images/task-table-simple.png) | ||
|
||
The **Tasks view** has a **Filters** section, which allows you to filter tasks visible on the page. Click **Filter** to apply the filters and **Reset** to set all filter parameters back to their default unfiltered state. Available filters include: | ||
|
||
* **Task Name** - display tasks to those matching the specified selection of characters | ||
* **Date Created Start and Date Created End** - filter displayed results to the tasks created within this range. Note that this is task creation and has nothing to do with the dates of images in the tasks. | ||
* **Assignee** - filters tasks to specific users | ||
* **Stage** - filters tasks to their stage in the annotation, ground truth, and line division workflow | ||
* A table of Scout tasks filtered to your user account as well as selections in the **Filters** section. **Table columns** include: | ||
* **Task Name** - name given to the task when it was created. | ||
* **Side** - side of the airplane that images in the task were from, *Left or Right* | ||
* **Assignee** - user assigned to the task | ||
* **Date Created** - creation date of the task | ||
* **A%** - percentage of image annotation complete in the task | ||
* **G%** - percentage of image ground truth review complete in the task | ||
* **D%** - percentage of image division line annotation complete in the task | ||
* **Annotate button** - if images are still available for annotation in the task, provides access to the annotation page | ||
* **Next button** - provides access to subsequent rows of table data if more tasks match the filter criteria than can be displayed in the table | ||
* **Previous button** - provides access to previous rows of table data if more tasks match the filter criteria than can be displayed in the table |
Oops, something went wrong.