Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove correspondences from translation averaging outliers downstream #755

Merged
merged 4 commits into from
Jan 23, 2024

Conversation

akshay-krishnan
Copy link
Collaborator

No description provided.

@akshay-krishnan
Copy link
Collaborator Author

Based on evaluation on larger datasets, this change produces small improvements (1% Pose AUC post BA) for some datasets (Barn, Kings college, Skansen, Gerrard hall) and small regressions of similar magnitude for others (skydio 501). this also somewhat depends on the frontend being used.

Some example numbers:

dataset BA pose AUC with change BA pose AUC without change
Barn - LOFTR 22.71 22.69
Barn - SIFT 40.94 39.39
Gerrard Hall - SIFT 6.36 5.90
Gerrard Hall - LOFTR 4.71 7.52
Kings college - SIFT 42.80 40.84
Kings college - LOFTR 66.12 59.55
Palace - SIFT 82.45 82.88
Skansen - SIFT 87.86 87.19
Skansen - superglue 74.92 74.73
skydio501 - LOFTR 5.16 5.70
skydio501 - SIFT 1.93 10.1
south building - LOFTR 44.45 44.08
south building - SIFT 49.66 48.77

more numbers are in the spreadsheet (ta-outliers-x)

@akshay-krishnan
Copy link
Collaborator Author

requesting high-level thoughts on whether we should proceed with this change. Shall update the code if thats the case.

@travisdriver
Copy link
Collaborator

What degree threshold are the previous pose AUC statistics at?

@akshay-krishnan
Copy link
Collaborator Author

Most of them are at 1 deg, I used 2.5 degrees for a couple (including skydio501) because their pose AUCs at 1 deg are 0. Ive highlighted the rows in the spreadsheet that I used.

@@ -196,3 +199,16 @@ def get_2d_tracks(
) -> List[SfmTrack2d]:
tracks_estimator = CppDsfTracksEstimator()
return tracks_estimator.run(corr_idxs_dict, keypoints_list)


def filter_corr_by_idx(correspondences, idxs):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: missing type annotations

@johnwlambert
Copy link
Collaborator

Btw @akshay-krishnan I believe CI is failing:

=========================== short test summary info ============================
FAILED tests/averaging/translation/test_averaging_1dsfm.py::TestTranslationAveraging1DSFM::test_circle_all_edges - ValueError: too many values to unpack (expected 2)
FAILED tests/averaging/translation/test_averaging_1dsfm.py::TestTranslationAveraging1DSFM::test_computation_graph - ValueError: too many values to unpack (expected 2)
FAILED tests/averaging/translation/test_averaging_1dsfm.py::TestTranslationAveraging1DSFM::test_lund_door - ValueError: too many values to unpack (expected 2)
FAILED tests/averaging/translation/test_averaging_1dsfm.py::TestTranslationAveraging1DSFM::test_panorama - ValueError: too many values to unpack (expected 2)
FAILED tests/averaging/translation/test_averaging_1dsfm.py::Test1dsfmAllOutliers::test_outlier_case_missing_value - ValueError: too many values to unpack (expected 2)
FAILED tests/frontend/detector_descriptor/test_d2net.py::TestD2NetDetDesc::test_coordinates_range - EOFError: Ran out of input
FAILED tests/frontend/detector_descriptor/test_d2net.py::TestD2NetDetDesc::test_detect_and_describe_shape - EOFError: Ran out of input
FAILED tests/frontend/detector_descriptor/test_d2net.py::TestD2NetDetDesc::test_number_of_detections - EOFError: Ran out of input
FAILED tests/frontend/detector_descriptor/test_d2net.py::TestD2NetDetDesc::test_pickleable - EOFError: Ran out of input
FAILED tests/frontend/detector_descriptor/test_d2net.py::TestD2NetDetDesc::test_scale - EOFError: Ran out of input
===== 10 failed, 425 passed, 7 skipped, 24 warnings in 1023.58s (0:17:03) ======

@johnwlambert
Copy link
Collaborator

could you pull in the latest from master?

@akshay-krishnan akshay-krishnan merged commit 7646969 into master Jan 23, 2024
23 of 24 checks passed
@akshay-krishnan akshay-krishnan deleted the ta-outlier-remove branch January 23, 2024 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants