Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

in-depth comparison of the image preprocessing pipeline #24

Open
alexshepard opened this issue Feb 29, 2024 · 6 comments
Open

in-depth comparison of the image preprocessing pipeline #24

alexshepard opened this issue Feb 29, 2024 · 6 comments
Assignees

Comments

@alexshepard
Copy link
Contributor

alexshepard commented Feb 29, 2024

this came up as followup todos during the bounding box experiments.

across the clades, evaluate the following different approaches:

  • source photo squashed into 299x299
  • source photo square center cropped to 299x299
  • source photo center 85% center cropped, squashed into 299x299
  • source photo center 85% cropped, then square cropped to 299x299

also, visualize the different approaches in a slide or presentation or notebook so that people know what we're talking about.

@alexshepard alexshepard moved this to Todo in iNat ML Mar 1, 2024
@alexshepard alexshepard self-assigned this Mar 6, 2024
@alexshepard
Copy link
Contributor Author

I made a comparison between three different preprocessing strategies:

  1. no crop, just squeeze
  2. central crop, then squeeze
  3. resize with crop, no squeezing

@alexshepard
Copy link
Contributor Author

here is how the "no crop, just squeeze" approach transforms an image

Image

@alexshepard
Copy link
Contributor Author

alexshepard commented Mar 19, 2024

here is how the "central crop, then squeeze" approach transforms an image

Image

note this is how we do things in production now

@alexshepard
Copy link
Contributor Author

here is how the "resize with crop, no squeezing" approach transforms an image

Image

@alexshepard
Copy link
Contributor Author

Image

here's a look at performance metrics across our normal categories for each approach.

based on this, I'm recommending we switch to using tf.image.resize_with_crop_or_pad to preprocess our images. I'll make a pull request for this in the inatVisionAPI repo.

@alexshepard
Copy link
Contributor Author

I made a PR for this here inaturalist/inatVisionAPI#17 and requested that Patrick take a look

@alexshepard alexshepard moved this from Todo to Done in iNat ML Mar 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

No branches or pull requests

1 participant