Readme.md

Visualization Scripts

We provide various visualization functionlaities on TAO-Amodal, including:

• Visualize Modal Mask & Amodal Box
• Visualize Tracker Predictions
• Parameter Definitions

Usage

Visualize mask and amodal bounding box

1. Randomly visualize 5 videos from each subset in TAO-Amodal training set.

Modify --mask-annotations, --annotations, --images-dir to your local paths. Videos will be saved in --output-dir.

python ./vis_amodal_mask_videos.py \
    --mask-annotations /path/to/BURST_annotations/train/train_visibility.json \
    --annotations /path/to/amodal_annotations/train.json \
    --output-dir /path/to/output_dir \
    --images-dir /path/to/TAO-Amodal/frames \
    --split train \
    --slow-down 25 \
    --speed-up -1 \
    --random-quality-check True \
    --random-quality-check-size 5

Example output video

More examples could be found here.

---

2. Visualize a specific track in a video using --filter-tracks and --video-name.

# Visualize amodal boxes
python ./vis_amodal_mask_videos.py \
    --mask-annotations /path/to/TAO-Amodal/BURST_annotations/val/all_classes_visibility.json \
    --annotations /path/to/TAO-Amodal/amodal_annotations/validation.json \
    --output-dir /path/to/output-dir \
    --images-dir /path/to/TAO-Amodal/frames \
    --slow-down 15 \
    --speed-up 2 \
    --split val \
    --show-categories False \
    --transparent True \
    --filter-tracks 14226 \
    --interpolate True \
    --video-name val/LaSOT/cattle-7

# Visualize modal boxes
python ./vis_amodal_mask_videos.py \
    --mask-annotations /path/to/TAO-Amodal/BURST_annotations/val/all_classes_visibility.json \
    --annotations /path/to/TAO-Amodal/amodal_annotations/validation.json \
    --output-dir /path/to/output-dir \
    --images-dir /path/to/TAO-Amodal/frames \
    --slow-down 15 \
    --speed-up 2 \
    --split val \
    --show-categories False \
    --transparent True \
    --filter-tracks 14226 \
    --interpolate False \
    --video-name val/LaSOT/cattle-7 \
    --modal True

• Example output:

Modal	Amodal

---

Visualize Tracker Prediction

1. Visualize Predictions (JSON)

The prediction JSON should be structured following this format. Example prediction JSON file generated by our Amodal Expander could be found here. We provide two example commands below.

• Display the specified track with a specific color using --filter-tracks and --color.

python ./vis_prediction.py \
    --annotations /path/to/TAO-Amodal/amodal_annotations/validation_lvis_v1.json \
    --predictions /path/to/prediction.json \
    --output-dir /path/to/output-dir \
    --images-dir /path/to/TAO-Amodal/frames \
    --split val \
    --slow-down 15 \
    --speed-up 1 \
    --transparent True \
    --filter-tracks 138807920001 \
    --color 255 80 133 \
    --video-name val/AVA/Ic0LMbDyc9Y_scene_7_61166-62253 

Example Output Video

• Visualize all tracks from randomly picked videos.

python ./vis_prediction.py \
    --annotations /path/to/TAO-Amodal/amodal_annotations/validation_lvis_v1.json \
    --predictions /path/to/prediction.json \
    --split val \
    --output-dir /path/to/output-dir \
    --images-dir /path/to/TAO-Amodal/frames \
    --slow-down 15 \
    --speed-up 1 \
    --random-quality-check True \
    --random-quality-check-size 5

2. Compare Tracker Predictions

Use the following command to compare predictions, --predictions and --predictions2, generated by two trackers. For example, you can compare the modal and amodal predictions generated by our Amodal Expander.

# Show specified track with a certain color
python ./vis_prediction_comparison.py \
    --annotations /path/to/TAO-Amodal/amodal_annotations/validation_lvis_v1.json \
    --predictions /path/to/modal_prediction.json \
    --predictions2 /path/to/prediction.json \
    --output-dir /path/to/output-dir \
    --images-dir /path/to/TAO-Amodal/frames \
    --split val \
    --slow-down 15 \
    --speed-up 1 \
    --transparent True \
    --filter-tracks 138807920001 \
    --color 255 80 133 \
    --video-name val/AVA/Ic0LMbDyc9Y_scene_7_61166-62253 

Modal	Amodal

More examples could be found here.

Parameter Definition

We provide different parameters for better customization (e.g., bounding box/video) of your visualization results. For example, you can do interpolation, specify colors or making the background transparent to emphasize the bounding boxes.

Check utils.py for the definition of each parameter.