diff --git a/docs/administration/delete.md b/docs/administration/delete.md new file mode 100644 index 0000000..a97a93b --- /dev/null +++ b/docs/administration/delete.md @@ -0,0 +1,11 @@ +# Delete Data + + +## Delete Users +At present, there is no way to delete users. + +## Delete Images +At present, there is no way through the UI to delete images from the system. Instead, you must make edits to the mongoDB directly. + +## Delete Tasks +When you delete a task, the associated annotations are removed from the system. When you delete a task that has been through ground truth, the ground truth confirmations are removed from the system, which can impact annotation export analysis. When you delete a task that has the last instance of a tag on it, the tag is not removed from the system. To remove tags, you must delete them individually. \ No newline at end of file diff --git a/docs/lab-lead/first-scout-login.md b/docs/administration/index.md similarity index 80% rename from docs/lab-lead/first-scout-login.md rename to docs/administration/index.md index c3754bc..fc2ea4b 100644 --- a/docs/lab-lead/first-scout-login.md +++ b/docs/administration/index.md @@ -1,4 +1,8 @@ -# Creating the First Login on Scout +# Administration + +Once Scout is running on your server, you can set up the system to be ready for data processing. You'll need to create the first login to do any other administration. + +## Create the First Login As a Lab Lead, your first contact with **Scout** is after the System Administrator installed it and provided you with a link. Open the link using the **Google Chrome browser** and create your Lab Lead account. *(Note: This step is not needed if the System Administrator provided you with login credentials.)* @@ -13,3 +17,12 @@ Here's how to create your Lab Lead account: ```{note} While you can perform all of the Lab Lead duties in Scout from any Chrome browser that can access it, you may also want to obtain access to the Scout Server laptop to be able to add aerial survey images to Scout’s image ingestion folder (see the Add survey images to Scout section of this document). Ask your System Administrator for an Ubuntu Desktop account to log into the server. Your account also needs access to the image ingestion folder created by the System Administrator. ``` + +```{toctree} +:maxdepth: 1 + +manage-users +manage-labels +manage-task +delete +``` diff --git a/docs/lab-lead/hot-keys.md b/docs/administration/manage-labels.md similarity index 98% rename from docs/lab-lead/hot-keys.md rename to docs/administration/manage-labels.md index 6837ba5..641c367 100644 --- a/docs/lab-lead/hot-keys.md +++ b/docs/administration/manage-labels.md @@ -1,4 +1,4 @@ -# Hot Keys +# Labels and Hot Keys Lab Leads should set up hot keys and labels before assigning tasks to annotators. diff --git a/docs/lab-lead/manage-task.md b/docs/administration/manage-task.md similarity index 100% rename from docs/lab-lead/manage-task.md rename to docs/administration/manage-task.md diff --git a/docs/lab-lead/manage-users.md b/docs/administration/manage-users.md similarity index 81% rename from docs/lab-lead/manage-users.md rename to docs/administration/manage-users.md index 2ecfa87..911c0f2 100644 --- a/docs/lab-lead/manage-users.md +++ b/docs/administration/manage-users.md @@ -1,4 +1,5 @@ -# Creating User Accounts +# Manage Users +## Creating User Accounts Only a Lab Lead can create Annotator and other Lab Lead user accounts. Here's how to create a new user account: @@ -8,8 +9,8 @@ Only a Lab Lead can create Annotator and other Lab Lead user accounts. Here's ho * A unique username *(Note: case sensitive)* * A unique password *(Note: case sensitive and needs to be confirmed on the **Confirm password** field )* * The role for the new user in the **Role** field. Your options are: - * **Annotator** \- an account that only let the user draw and label bounding boxes on images in assigned tasks\. - * **Lab Lead** \- an account that can also manage users and tasks\, provide ground truth for annotated data\, draw division lines\, and export data\. + * **Annotator** - an account that only let the user draw and label bounding boxes on images in assigned tasks. + * **Lab Lead** - an account that can also manage users and tasks, provide ground truth for annotated data, draw division lines, and export data. 4. Your password to ensure that new users cannot be created without your consent. 5. Click **Submit** to create a new user account. To make sure that it's successful, the new username will be listed on the Users page. @@ -17,7 +18,7 @@ Only a Lab Lead can create Annotator and other Lab Lead user accounts. Here's ho Make sure to convey the new username and password to the new user so that they may begin using the account. ``` -# Editing User Accounts +## Editing User Accounts Here's how to edit an existing user account: diff --git a/docs/all-users/find-task.md b/docs/all-users/find-task.md deleted file mode 100644 index 991bd5a..0000000 --- a/docs/all-users/find-task.md +++ /dev/null @@ -1,22 +0,0 @@ -# Finding Your Tasks - -After login, you should see the **Tasks page** as a landing page. The page is filtered beforehand to your assigned tasks. Use filters to find specific tasks to annotate. -![taskpage](../assets/images/task-table-simple.png) - -The **Tasks view** has a **Filters** section, which allows you to filter tasks visible on the page. Click **Filter** to apply the filters and **Reset** to set all filter parameters back to their default unfiltered state. Available filters include: - -* **Task Name** \- display tasks to those matching the specified selection of characters -* **Date Created Start and Date Created End** \- filter displayed results to the tasks created within this range\. Note that this is task creation and has nothing to do with the dates of images in the tasks\. -* **Assignee** \- filters tasks to specific users -* **Stage** \- filters tasks to their stage in the annotation\, ground truth\, and line division workflow -* A table of Scout tasks filtered to your user account as well as selections in the **Filters** section. **Table columns** include: - * **Task Name** \- name given to the task when it was created\. - * **Side** \- side of the airplane that images in the task were from\, *Left or Right* - * **Assignee** \- user assigned to the task - * **Date Created** \- creation date of the task - * **A%** \- percentage of image annotation complete in the task - * **G%** \- percentage of image ground truth review complete in the task - * **D%** \- percentage of image division line annotation complete in the task - * **Annotate button** \- if images are still available for annotation in the task\, provides access to the annotation page - * **Next button** \- provides access to subsequent rows of table data if more tasks match the filter criteria than can be displayed in the table - * **Previous button** \- provides access to previous rows of table data if more tasks match the filter criteria than can be displayed in the table \ No newline at end of file diff --git a/docs/all-users/index.md b/docs/all-users/index.md deleted file mode 100644 index 2e08fce..0000000 --- a/docs/all-users/index.md +++ /dev/null @@ -1,9 +0,0 @@ -# All Users - -```{toctree} -:hidden: - -logging-in -find-task -annotate-task -``` diff --git a/docs/all-users/logging-in.md b/docs/all-users/logging-in.md deleted file mode 100644 index 2af4ed8..0000000 --- a/docs/all-users/logging-in.md +++ /dev/null @@ -1,9 +0,0 @@ -# How to Login to Scout - -Your Lab Lead will provide you with the URL to Scout, as well as your username and password. Once you have these, open the **Google Chrome** browser on your local computer and connect to Scout. Use your username and password to log in. - -```{note} -Log out of Scout after your work. Scout does not automatically log you out, so doing it manually protects your account and work. -``` - - diff --git a/docs/all-users/annotate-task.md b/docs/annotate-task.md similarity index 84% rename from docs/all-users/annotate-task.md rename to docs/annotate-task.md index ec06e23..b729cf0 100644 --- a/docs/all-users/annotate-task.md +++ b/docs/annotate-task.md @@ -1,20 +1,20 @@ -# Annotating Your Assigned Tasks +# Manually Annotate Tasks If an unfinished task is assigned to you directly, you can click the **Annotate button** next to the task on the **Tasks table** to begin annotating randomly presented images. The annotation screen allows you to *click-drag* to create a bounding box. Use the down arrow key or begin typing to select the correct label for the annotation. In addition to the *click-drag* motion for creating annotations, the following tools are available to you in the annotation interface: -* zoom in button ![zoomin](../assets/images/zoom-plus.png) -* zoom out button ![zoomout](../assets/images/zoom-minus.png) +* zoom in button ![zoomin](assets/images/zoom-plus.png) +* zoom out button ![zoomout](assets/images/zoom-minus.png) * **scroll bars** to pan across the image to look for parts to annotate -* back button ![back](../assets/images/back-button.png) +* back button ![back](assets/images/back-button.png) * **Done** to finish the ground truth stage of this image, making it as completely and accurately done. If any annotations were created, modified, or destroyed, they are now saved as ground truth. If no annotations were present when Done was clicked, the image is considered empty. ```{note} If you refresh your Chrome browser, the interface presents a new random image, and your work on the image you saw before is not stored. ``` -![annotate](../assets/images/annotation-interface.png) +![annotate](assets/images/annotation-interface.png) Once you finish annotating the images, a page noting the successful completion of annotation for the task appears. Click the **Tasks button** on the menu to return to the **Tasks view and start** your next assigned task. -![success](../assets/images/annotation-complete-success.png) +![success](assets/images/annotation-complete-success.png) ## Annotation Toggle diff --git a/docs/contribute/code-guide.md b/docs/contribute/code-guide.md index 17615aa..7841189 100644 --- a/docs/contribute/code-guide.md +++ b/docs/contribute/code-guide.md @@ -13,7 +13,7 @@ Generally speaking, we want to avoid in-line styling to allow for greater consis ## Internationalization All Wild Me tools are intended for an international audience, so we want to provide translated versions. An active goal for 2024 is to get ensure that all product copy has been internationalized according to the following standard. -At present, in Wildbook, all copy except for site name, notification messages, and species names are wrapped and localized to 4 languages: english, french, spanish, and italian. +At present, Scout is not internationalized for any language. ## EXIF data handling EXIF data is notoriously inconsistent between different cameras. Unless we want to provide an entire suite of EXIF management tools (we do not), we must make assumptions about the data coming in, prepare for those assumptions to be wrong, and fail gracefully. Any development focused on EXIF data management should catch exceptions for the following cases: diff --git a/docs/contribute/index.md b/docs/contribute/index.md index d800b4f..5c75c06 100644 --- a/docs/contribute/index.md +++ b/docs/contribute/index.md @@ -17,14 +17,10 @@ Wild Me provides open source software intended to combat extinction. Historicall If you want to help, there's any number of ways that contribute directly to the mission. -* Provide data: If you have relevant species data that you want to provide to researchers, check out the available Wildbooks and see if there's a match. We're also putting together a list of high-interest data deficient species +* Provide data: If you have relevant species data that you want to provide to researchers, contact us through the Development Server or via email. Data can be included as just training material for models or can be added to public data sets to further conservation more broadly, whatever works for you! * Develop code: unit tests, tech debt, new features, bug fixes. All of our products are available for direct contribution. See [](pr-workflow.md) for more information. * Donate: As a non-profit, we are working to provide maximum value at minimum cost. However, we must still cover the costs of things such as storage and processing for our various platforms. - -## Add to documentation -All Wild Me documentation, be it community, user, or developer docs, is hosted on GitHub. As such, you can contribute to the docunentation through the standard PR flow. - -_Guidelines coming soon_ +* Contribute to documentation: All Wild Me documentation, be it community, user, or developer docs, is hosted on GitHub. As such, you can contribute to the docunentation through the standard PR flow. ## Coming soon... * Test and issue reporting diff --git a/docs/lab-lead/division-lines.md b/docs/division-lines.md similarity index 88% rename from docs/lab-lead/division-lines.md rename to docs/division-lines.md index aa2d7f4..75462b1 100644 --- a/docs/lab-lead/division-lines.md +++ b/docs/division-lines.md @@ -1,4 +1,4 @@ -# Drawing Division Lines +# Draw Division Lines Division Lines allow you to note a single area of overlap between two images that meet the following criteria: @@ -8,7 +8,7 @@ Division Lines allow you to note a single area of overlap between two images tha Division Lines are intended to help note areas of overlap in successive images so that overcounting can be minimized. Division Lines are drawn on a task-by-task basis, and drawing Division Lines is only possible after a task has achieved 100% annotation and growth truth review. *(Note: Drawing division lines is optional.)* To draw **Division Lines,** go to the Tasks view and select a task from the table. If the task is ready for Division Lines, the **Division Lines** button becomes clickable. -![rowdivision](../assets/images/tasktable-row-division.png) +![rowdivision](assets/images/tasktable-row-division.png) ```{note} Not all images are made available for division lines. Division lines only need to be set between successive pairs of images that might have overlapping areas, and both must have annotations to be considered for division lines. @@ -16,9 +16,9 @@ Not all images are made available for division lines. Division lines only need t The Division Lines screen provides you with a side-by-side view of two successive images in a sequence. Drawing Division Lines is a stage that requires the side of the plane (Left or Right) to be accurately set on the task to ensure the images are sequenced correctly. The tools available in the Division Lines screen are: -* Back button ![back](../assets/images/back-button.png) +* Back button ![back](assets/images/back-button.png) * **Done** - completes the division line stage of this image pair, making it as completely and accurately *done*. * **Sliders** - the top and button sliders control the start, end, and slope of the line. To the best of your ability, use the slider to define the overlap area between the left and right images. -![divisionlinedefault](../assets/images/divisionline-default.png) +![divisionlinedefault](assets/images/divisionline-default.png) diff --git a/docs/lab-lead/evaluate-annotator-performance.md b/docs/evaluate-annotator-performance.md similarity index 98% rename from docs/lab-lead/evaluate-annotator-performance.md rename to docs/evaluate-annotator-performance.md index 51839cd..63a099a 100644 --- a/docs/lab-lead/evaluate-annotator-performance.md +++ b/docs/evaluate-annotator-performance.md @@ -1,4 +1,4 @@ -# How to Evaluate Annotator Performance +# Evaluate Annotator Performance Use both the Image and Annotation CSV exports allows evaluation of the whole image classifier (WIC). For this performance method, we use the following definition of ground truth: diff --git a/docs/export-data.md b/docs/export-data.md new file mode 100644 index 0000000..7a45f8f --- /dev/null +++ b/docs/export-data.md @@ -0,0 +1,46 @@ +# Data Exports + +## Exporting Task Annotation and Image Data for Analysis + +Data can be exported for third-party application analysis and modeling by generating image and annotation data as `CSV` for one or more tasks selected in the Tasks view. Here's how to export data: + +1. Click **Tasks** in the header navigation menu to open the Tasks view. +2. In the Tasks view, use the filters to filter the table down to only those tasks that you want to export. +3. Click the **Export Filtered Results** button and then select **Export Annotations** or **Export Images** options to choose the content of the `CSV` file to export. +4. A progress bar indicates the compilation progress of the downloadable `CSV` file. When the progress bar completes, click Download to download the `CSV` file. + +![ttexport](assets/images/tasktable-exportoptions.png) + +## Export Images CSV Data + +The following fields are available in the Export Images CSV file: + +* **Task Name** - Name of the task this image was included in. An image may be included in multiple tasks, and if so, the task name is repeated on multiple lines in this file. +* **Task ID** - The **Scout internal ID** of the task this image belongs to. +* **Image Filename** - Original filename of the image. +* **WIC Confidence** - *Machine learning score* predicting confidence of one or more labels present in the image. The score ranges from 0 (nothing predicted in the image) to 1 (maximum confidence that at least one ML model category would be found in the image). *If ML has not been run on this image, the WIC Confidence is empty.* +* **Ground Truth Status** - True/False value indicating whether review of this image has been completed in the Ground Truth task stage. +* **Exclusion Side** - The side of the Division Line that is exclude in counts of animals in overlapping images. This value is always: **right.** +* **Inclusion Top X Fraction** - The decimal fraction (0 to 1) of the top insertion point of the division line. This is the fraction of the image width from the left-side of the image at which the top of the division line intersects the bottom of the image. +* **Inclusion Bottom X Fraction** - The decimal fraction (0 to 1) of the bottom insertion point of the division line. This is the fraction of the image width from the left-side of the image at which the bottom of the division line intersects the bottom of the image. + +## Export Annotation CSV Data + +The following fields are available in the Export Annotations CSV file: + +* **Task Name** - The name of the task that the annotation is included. +* **Task ID** - The *Scout internal ID* of the task this annotation belongs to. +* **Image Filename** - Original filename of the image. +* **Image ID** - *Scout internal ID* of the image. +* **Box X** - *X coordinate* of the annotation bounding box’s top left corner. +* **Box Y** - *Y coordinate* of the annotation bounding box’s top left corner. +* **Box W** - Pixel width of the bounding box. +* **Box H** - Pixel height of the bounding box. +* **Box Confidence** - If machine learning created an annotation, this is the confidence score of the bounding box of this annotation. The score ranges from 0 *(no confidence)* to 1 *(maximum confidence that the box is correctly predicted)*. +* **Classification** - The predicted classification (i.e. *type*) of the bounding box, such as *elephant* or *buffalo*. +* **Classification Confidence** - If machine learning created this annotation, this is the confidence score of the classification (e.g., *elephant*) assigned to this annotation. The score ranges from 0 *(no confidence)* to 1 *(maximum confidence that the classification is correct)*. +* **Assignee** - The Scout user that drew the bounding box. +* **Timestamp** - Timestamp of annotation creation. +* **Is Ground Truth** - Whether this annotation has been approved in the Ground Truth process. Annotations that have this field as True are likely to be more reliably accurate as they have been reviewed independently. +* **Excluded By Line** - Whether the centroid of this bounding box falls outside or inside the division line. + diff --git a/docs/get-started.md b/docs/get-started.md new file mode 100644 index 0000000..10a7632 --- /dev/null +++ b/docs/get-started.md @@ -0,0 +1,30 @@ +# Getting Started + +Your Lab Lead will provide you with the URL to Scout, as well as your username and password. Once you have these, open your browser on your local computer and connect to Scout. Use your username and password to log in. + +```{note} +Log out of Scout after your work. Scout does not automatically log you out, so doing it manually protects your account and work. +``` + +## Finding Your Tasks + +After login, you should see the **Tasks page** as a landing page. The page is filtered beforehand to your assigned tasks. Use filters to find specific tasks to annotate. +![taskpage](assets/images/task-table-simple.png) + +The **Tasks view** has a **Filters** section, which allows you to filter tasks visible on the page. Click **Filter** to apply the filters and **Reset** to set all filter parameters back to their default unfiltered state. Available filters include: + +* **Task Name** - display tasks to those matching the specified selection of characters +* **Date Created Start and Date Created End** - filter displayed results to the tasks created within this range. Note that this is task creation and has nothing to do with the dates of images in the tasks. +* **Assignee** - filters tasks to specific users +* **Stage** - filters tasks to their stage in the annotation, ground truth, and line division workflow +* A table of Scout tasks filtered to your user account as well as selections in the **Filters** section. **Table columns** include: + * **Task Name** - name given to the task when it was created. + * **Side** - side of the airplane that images in the task were from, *Left or Right* + * **Assignee** - user assigned to the task + * **Date Created** - creation date of the task + * **A%** - percentage of image annotation complete in the task + * **G%** - percentage of image ground truth review complete in the task + * **D%** - percentage of image division line annotation complete in the task + * **Annotate button** - if images are still available for annotation in the task, provides access to the annotation page + * **Next button** - provides access to subsequent rows of table data if more tasks match the filter criteria than can be displayed in the table + * **Previous button** - provides access to previous rows of table data if more tasks match the filter criteria than can be displayed in the table diff --git a/docs/lab-lead/ground-truth.md b/docs/ground-truth.md similarity index 73% rename from docs/lab-lead/ground-truth.md rename to docs/ground-truth.md index 2a4f0a4..d3dd807 100644 --- a/docs/lab-lead/ground-truth.md +++ b/docs/ground-truth.md @@ -1,4 +1,4 @@ -# Ground Truth Review of Tasks +# Ground Truth Review The Ground Truth stage of a task involves a review of its images and annotations for accuracy by a lab lead. The purposes of the ground truth stage are to: @@ -14,17 +14,17 @@ When exporting annotations, it is important to understand which Annotations are If a task is fully annotated but has not been 100% through the Ground Truth stage yet, you can click the **Ground Truth** button next to the task on the Tasks table to begin reviewing randomly presented images and their annotations. 1. The **Annotation screen** allows you to *click-drag* to create a new bounding box and then press the down arrow or begin typing to select the correct label for the annotation. -2. You can click on any previously created annotation to edit it. A **trash can** ![trash](../assets/images/trash.png) +2. You can click on any previously created annotation to edit it. A **trash can** ![trash](assets/images/trash.png) 3. In addition to the *click-drag motion* for creating annotations, the following tools are available to you in the annotation interface: -* ![zoomplus](../assets/images/zoom-plus.png) zooms in on the image. -* ![zoomminus](../assets/images/zoom-minus.png) zooms out of the image +* ![zoomplus](assets/images/zoom-plus.png) zooms in on the image. +* ![zoomminus](assets/images/zoom-minus.png) zooms out of the image * **scroll bars** for the image allow you to pan across the image looking for parts to annotate -* ![back](../assets/images/back-button.png) returns to the last image annotated in this annotation session +* ![back](assets/images/back-button.png) returns to the last image annotated in this annotation session * **Done** to finish the ground truth stage of this image, making it as completely and accurately *done*. If any annotations were created, modified, or destroyed, they are now saved as ground truth. If no annotations were present when **Done** was clicked, the image is considered empty. -* ![panel](../assets/images/panel-button.png) opens a side-by-side display that allows you to compare annotations for this image to other tasks that contain this image. The **Reference Task** pulldown allows you to switch between other tasks that include annotations on this image. This allows you to perform your ground truth review in the context of multiple annotators (*including ML*) who may find and miss correct annotations in different ways. However, what you annotate in the left window on the image is considered the final and correct Ground Truth for the image. +* ![panel](assets/images/panel-button.png) opens a side-by-side display that allows you to compare annotations for this image to other tasks that contain this image. The **Reference Task** pulldown allows you to switch between other tasks that include annotations on this image. This allows you to perform your ground truth review in the context of multiple annotators (*including ML*) who may find and miss correct annotations in different ways. However, what you annotate in the left window on the image is considered the final and correct Ground Truth for the image. -![gtcompare](../assets/images/groundtruth-compare.png) +![gtcompare](assets/images/groundtruth-compare.png) Once Ground Truth is complete *(i.e. reaches 100%),* a window will pop out confirming that the changes have been saved. ## Annotation Toggle diff --git a/docs/lab-lead/image-gallery.md b/docs/image-gallery.md similarity index 100% rename from docs/lab-lead/image-gallery.md rename to docs/image-gallery.md diff --git a/docs/index.md b/docs/index.md index cac1e86..489c8b0 100644 --- a/docs/index.md +++ b/docs/index.md @@ -13,8 +13,8 @@ With Scout, you can: ## Where Should I Start? * Are you planning on setting up Scout on your computer and managing the system, including system updates? You should start with the [System Setup and Maintenance](setup-and-maintenance/index.md) section. -* Are you going to manage image upload, user accounts, and data management? Do you have the final say in annotation state? You should start with [Lab Lead](lab-lead/index.md) section. -* Are you going to be reviewing images and editing annotations? You should start with the [Annotators](all-users/index.md) section. +* Are you going to manage image upload, user accounts, and data management? Do you have the final say in annotation state? You should start with the [Administration](administration/index.md) section. +* Are you going to be reviewing images and editing annotations? You should start with the [Get Started](get-started.md) section. ## We're here to help! Have questions on setup and maintenance? Join the [Wild Me Development Discord](https://discord.gg/zw4tr3RE4R) and we'll help you get to the root of the problem. @@ -22,16 +22,24 @@ Have questions on how to annotate, what data will work out of the box, or anythi ```{toctree} :hidden: +get-started +annotate-task +image-gallery +ground-truth +division-lines +export-data +evaluate-annotator-performance setup-and-maintenance/index -lab-lead/index -all-users/index +administration/index +scoutbot/index support-and-faq ``` -## How Can You Help? +## How Can I Help? Want to put your talents to use fighting the 6th mass extinction? We're excited to have you! Get started here: ```{toctree} +:maxdepth: 2 contribute/index ``` diff --git a/docs/lab-lead/export-data.md b/docs/lab-lead/export-data.md deleted file mode 100644 index 41b5872..0000000 --- a/docs/lab-lead/export-data.md +++ /dev/null @@ -1,44 +0,0 @@ -# Exporting Task Annotation and Image Data for Analysis - -Data can be exported for third-party application analysis and modeling by generating image and annotation data as `CSV` for one or more tasks selected in the Tasks view. Here's how to export data: - -1. Click **Tasks** in the header navigation menu to open the Tasks view. -2. In the Tasks view, use the filters to filter the table down to only those tasks that you want to export. -3. Click the **Export Filtered Results** button and then select **Export Annotations** or **Export Images** options to choose the content of the `CSV` file to export. -4. A progress bar indicates the compilation progress of the downloadable `CSV` file. When the progress bar completes, click Download to download the `CSV` file. - -![ttexport](../assets/images/tasktable-exportoptions.png) - -## Export Images CSV Data - -The following fields are available in the Export Images CSV file: - -* **Task Name** \- Name of the task this image was included in\. An image may be included in multiple tasks\, and if so\, the task name is repeated on multiple lines in this file\. -* **Task ID** \- The **Scout internal ID** of the task this image belongs to. -* **Image Filename** \- Original filename of the image\. -* **WIC Confidence** - *Machine learning score* predicting confidence of one or more labels present in the image. The score ranges from 0 (nothing predicted in the image) to 1 (maximum confidence that at least one ML model category would be found in the image). *If ML has not been run on this image, the WIC Confidence is empty.* -* **Ground Truth Status** \- True/False value indicating whether review of this image has been completed in the Ground Truth task stage\. -* **Exclusion Side** \- The side of the Division Line that is exclude in counts of animals in overlapping images\. This value is always: **right.** -* **Inclusion Top X Fraction** \- The decimal fraction \(0 to 1\) of the top insertion point of the division line\. This is the fraction of the image width from the left\-side of the image at which the top of the division line intersects the bottom of the image\. -* **Inclusion Bottom X Fraction** \- The decimal fraction \(0 to 1\) of the bottom insertion point of the division line\. This is the fraction of the image width from the left\-side of the image at which the bottom of the division line intersects the bottom of the image\. - -## Export Annotation CSV Data - -The following fields are available in the Export Annotations CSV file: - -* **Task Name** \- The name of the task that the annotation is included\. -* **Task ID** \- The *Scout internal ID* of the task this annotation belongs to. -* **Image Filename** \- Original filename of the image\. -* **Image ID** - *Scout internal ID* of the image. -* **Box X** - *X coordinate* of the annotation bounding box’s top left corner. -* **Box Y** - *Y coordinate* of the annotation bounding box’s top left corner. -* **Box W** \- Pixel width of the bounding box\. -* **Box H** \- Pixel height of the bounding box\. -* **Box Confidence** \- If machine learning created an annotation\, this is the confidence score of the bounding box of this annotation\. The score ranges from 0 *(no confidence)* to 1 *(maximum confidence that the box is correctly predicted)*. -* **Classification** \- The predicted classification \(i\.e\. *type*) of the bounding box, such as *elephant* or *buffalo*. -* **Classification Confidence** \- If machine learning created this annotation\, this is the confidence score of the classification \(e\.g\.\, *elephant*) assigned to this annotation. The score ranges from 0 *(no confidence)* to 1 *(maximum confidence that the classification is correct)*. -* **Assignee** \- The Scout user that drew the bounding box\. -* **Timestamp** \- Timestamp of annotation creation\. -* **Is Ground Truth** \- Whether this annotation has been approved in the Ground Truth process\. Annotations that have this field as True are likely to be more reliably accurate as they have been reviewed independently\. -* **Excluded By Line** \- Whether the centroid of this bounding box falls outside or inside the division line\. - diff --git a/docs/lab-lead/index.md b/docs/lab-lead/index.md deleted file mode 100644 index 21c4e68..0000000 --- a/docs/lab-lead/index.md +++ /dev/null @@ -1,26 +0,0 @@ -# Lab Leads - -```{toctree} -:hidden: - -first-scout-login -manage-users -manage-task -hot-keys -image-gallery -ground-truth -division-lines -export-data -evaluate-annotator-performance -``` - -In Scout, a lab lead is responsible for: - -- Creating the first Scout login -- Creating and editing annotator and other lab lead user accounts -- Adding survey images to Scout (see the Add survey images to Scout section of this document) -- Creating and assigning annotation tasks -- Annotating tasks (if assigning to self or another lab lead) -- Ground truthing tasks annotated by other users -- Drawing division lines on image sequences -- Exporting tasks for analysis diff --git a/docs/scoutbot/index.md b/docs/scoutbot/index.md new file mode 100644 index 0000000..4902be1 --- /dev/null +++ b/docs/scoutbot/index.md @@ -0,0 +1,11 @@ +# Scoutbot +_This page is under development_ + +## Understand the ML +TBD + +## Data Format +TBD + +## Run Scoutbot Independently +TBD \ No newline at end of file