The Future of 80+ Feature Modules, Technical Debt & Refactoring Load

[silvanheller]

I would like to open a discussion on either removing or moving to a separate repository or library some of our 80+ retrieval features classes. This is because there is significant workload associated with maintaining and updating the existing retrieval features with changes to architecture and new functionality.

A lot of features for audio & 3D retrieval rely on StagedFeatureModule (and some HOG-Features). While these are fine for their original purpose, we have better MLT-Feature extractors with InceptionResnetV2, improved 3D model retrieval coming in #369, and audio fingerprinting or retrieval could also be done in a single retrieval feature which does not carry as much technical debt.

The SURF/HOG features and existing 3D features could probably be (re-)moved from the repo altogether, and some of the audio retrieval features could probably be reduced to a sensible amount in the spirit of reducing technical debt and refactoring load.

A similar point could be made for the plethora of sketch features - it would be nice to pick out a few that are sensible for general-purpose retrieval, and decide on the future of the others.

[lucaro]

While I agree with the overall sentiment, there are some points I'd like to bring up for consideration. For the low-level visual features, there is indeed a lot of redundancy there, and many of the approaches offer too little difference between them to be of use outside of maybe some extremely specific use cases for which one probably would want to design custom features anyway. So I would suggest keeping 1-2 of them per type in the main repository and (re)move all the other ones. In general, I think it would be good to just have a secondary repository for 'additional features' where this stuff can be moved to, with little to no future compatibility guarantees, but as a reference for future features.

I'm not sure I understand the arguments in the second paragraph. The behavior of the InceptionResnetV2 is rather different from the HOG feature and not comparable to the 3D or audio ones. Are there any specific issues with the StagedFeatureModule that would need to be addressed?

While I'm not sure about the exact state of the new additions in 3D features, I'm sure there is some overall redundancy, and a selection of the ones to keep can be made. For SURF and HOG, I'd argue for keeping at least some interest-point features around, as this stuff is still relevant in some domains and operates quite differently from the other end-to-end learned features.

Maybe we can start a table somewhere with all the modules we currently have and then decide for each one whether to keep it, archive it, or delete it altogether.