Adds efficientnet2 presets #1983

pkgoogle · 2024-11-11T22:29:05Z

Adds these variants:

efficientnet2_rw_m_agc_imagenet
efficientnet2_rw_s_ra2_imagenet
efficientnet2_rw_t_ra2_imagenet

Branched from Edge Presets: #1976

Can just merge this one and close that one or merge that one first for more modularity.

divyashreepathihalli

Thanks Piseth! Left a few comments

divyashreepathihalli · 2024-11-20T20:36:40Z

keras_hub/src/models/efficientnet/convbnact.py

+        batch_norm_epsilon=1e-3,
+        activation="swish",
+        dropout=0.2,
+        nores=False,


add missing args in docstring - example : nores

divyashreepathihalli · 2024-11-20T20:37:16Z

keras_hub/src/models/efficientnet/convbnact.py

@@ -0,0 +1,136 @@
+import keras


rename file to be more readable - convolution_batch_norm_activation.py

this conflicts w/ the below review, so I chose the later review.

divyashreepathihalli · 2024-11-20T20:37:42Z

keras_hub/src/models/efficientnet/convbnact_test.py

@@ -0,0 +1,22 @@
+import keras


rename file to be more readable

this conflicts w/ the below review, so I chose the later review.

divyashreepathihalli · 2024-11-20T20:39:25Z

keras_hub/src/models/efficientnet/fusedmbconv.py

@@ -77,6 +77,7 @@ def __init__(
        activation="swish",
        dropout=0.2,
        nores=False,


the docstring args are not matching the arg list here - please make sure they are consistent

mattdangerw

Having a partially "stackwise" arg that isn't name "stackwise_" is confusing. Left a suggestion to be more consistent in our arg names here. Less if branching.

keras_hub/src/models/efficientnet/efficientnet_presets.py

mattdangerw · 2024-11-20T22:53:29Z

keras_hub/src/models/efficientnet/convbnact.py

+}
+
+
+class ConvBNActBlock(keras.layers.Layer):


Maybe just call this CBABlock? And rename the file to match?

Kind of confusing the different forms of abbreviation we are using here.

mattdangerw · 2024-11-20T22:54:21Z

keras_hub/src/models/efficientnet/convbnact.py

+        return x
+
+    def get_config(self):
+        config = {


we don't do this style generally. follow this this https://github.com/keras-team/keras-hub/blob/master/keras_hub/src/models/bert/bert_backbone.py#L200-L213

mattdangerw · 2024-11-20T22:57:20Z

keras_hub/src/models/efficientnet/efficientnet_backbone.py

@@ -260,11 +271,26 @@ def __init__(
                        name=block_name,
                    )
                    x = block(x)
+                else:  # cba block


Why do this? you extend get_conv_constructor to handle "cba" then don't use it. I'm confused.

Changing my mind mid-change 😄. Refactored to make block_kwargs more dynamic now.

mattdangerw · 2024-11-20T23:07:39Z

keras_hub/src/models/efficientnet/efficientnet_backbone.py

-        for i in range(len(stackwise_kernel_sizes)):
+        num_stacks = len(stackwise_kernel_sizes)
+
+        if isinstance(depth_coefficient, tuple):


This is confusing, most that can be passed "stackwise" have stackwise_ as a prefix, except depth_coefficient now.

It's probably better UX to just rename these first arguments stackwise_depth_coefficient and stackwise_width_coefficient. At least then our argument names are consistent. Remember to update docstrings.

You could keep backwards compat by adding a few lines at the top of the constructor here. Something like this. You have to allow stackwise_depth_coefficient=None and stackwise_width_coefficient=None in the arg list.

num_stacks = len(stackwise_kernel_sizes) if "depth_coefficient" in kwargs: stackwise_depth_coefficient = kwargs.pop("depth_coefficient") * num_stacks if "width_coefficient" in kwargs: stackwise_width_coefficient = kwargs.pop("width_coefficient") * num_stacks

mattdangerw · 2024-11-20T23:14:28Z

keras_hub/src/utils/timm/convert_efficientnet.py

@@ -189,15 +236,22 @@ def port_batch_normalization(keras_layer, hf_weight_prefix):

    # Stages
    num_stacks = len(backbone.stackwise_kernel_sizes)
+
+    depth_coefficient = VARIANT_MAP[variant]["depth_coefficient"]


if we follow the suggestion about arg renaming above, we can get rid of this whole block. Just update above from

"width_coefficient": 0.8, "depth_coefficient": 0.9,

to

"stackwise_width_coefficient": [0.8] * 6, "stackwise_depth_coefficient": [0.9] * 6,

I made it so both ways should work.

pkgoogle · 2024-11-21T21:46:09Z

I should note I'm noticing a different prediction for one variant:

python tools/checkpoint_conversion/convert_efficientnet_checkpoints.py --preset efficientnet2_rw_s_ra2_imagenet
✅ Loaded TIMM model.
I1121 13:45:18.860764 8329934912 _builder.py:196] Loading pretrained weights from Hugging Face hub (timm/efficientnetv2_rw_s.ra2_in1k)
I1121 13:45:19.120539 8329934912 _hub.py:184] [timm/efficientnetv2_rw_s.ra2_in1k] Safe alternative available for 'pytorch_model.bin' (as 'model.safetensors'). Loading weights using safetensors.
✅ Loaded KerasHub model.
1/1 ━━━━━━━━━━━━━━━━━━━━ 2s 2s/step
🔶 Keras output: [-0.575618   -0.4144789  -0.73163635 -0.8395867  -0.5637497   0.87436163
 -1.3687949  -0.1337489  -0.0662252   0.01332632]
🔶 TIMM output: [-0.7259099   0.3929606  -0.05580516  0.11954936  0.02080455  0.996287
  0.3669875   0.5366461   0.01827425 -0.37024903]
🔶 Keras label: 349
🔶 TIMM label: 345
🔶 Modeling difference: 0.52596486
🔶 Preprocessing difference: 0.0044535464
🏁 Preset saved to ./efficientnet2_rw_s_ra2_imagenet

Which I'm looking into

mattdangerw · 2024-11-22T22:57:48Z

@pkgoogle thanks! Yeah a different label and that level of different output floats worth looking at!

…pancy

pkgoogle · 2024-11-25T22:48:08Z

Should be good now:

python tools/checkpoint_conversion/convert_efficientnet_checkpoints.py     --preset efficientnet2_rw_s_ra2_imagenet
✅ Loaded TIMM model.
I1125 14:46:30.043703 8329934912 _builder.py:196] Loading pretrained weights from Hugging Face hub (timm/efficientnetv2_rw_s.ra2_in1k)
I1125 14:46:30.252438 8329934912 _hub.py:184] [timm/efficientnetv2_rw_s.ra2_in1k] Safe alternative available for 'pytorch_model.bin' (as 'model.safetensors'). Loading weights using safetensors.
✅ Loaded KerasHub model.
1/1 ━━━━━━━━━━━━━━━━━━━━ 1s 1s/step
🔶 Keras output: [-0.37625     0.6832866   0.28495154  0.5878901   0.20605645  1.0627874
  0.40687525  0.40137056 -0.118939   -0.30322495]
🔶 TIMM output: [-0.7259099   0.3929606  -0.05580516  0.11954936  0.02080455  0.996287
  0.3669875   0.5366461   0.01827425 -0.37024903]
🔶 Keras label: 345
🔶 TIMM label: 345
🔶 Modeling difference: 0.15455398
🔶 Preprocessing difference: 0.0044535464
🏁 Preset saved to ./efficientnet2_rw_s_ra2_imagenet

mattdangerw

Looking good! Just little comments.

mattdangerw · 2024-12-03T21:51:19Z

keras_hub/src/models/efficientnet/cba.py

+
+BN_AXIS = 3
+
+CONV_KERNEL_INITIALIZER = {


Kinda funky to return this as a dict, any particular reason for it?

More in line with other models would be to just add a function here...

def conv_kernel_initializer(scale=2.): return keras.initializers.VarianceScaling( scale=scale, ... )

No particular reason, I was following the original style of the fusedmbconv/mbconv blocks. Done.

mattdangerw · 2024-12-03T21:52:23Z

keras_hub/src/models/efficientnet/efficientnet_backbone.py

@@ -99,8 +106,8 @@ class EfficientNetBackbone(FeaturePyramidBackbone):
    def __init__(
        self,
        *,
-        width_coefficient,
-        depth_coefficient,
+        stackwise_width_coefficient=None,


probably make these plural, in keeping with other args? stackwise_width_coefficients and stackwise_depth_coefficients

mattdangerw · 2024-12-03T21:54:20Z

keras_hub/src/utils/timm/convert_efficientnet.py

+        "num_features": 1792,
+    },
+    "rw_s": {
+        "width_coefficient": 1.0,


I'd pass these the new way instead of the legacy and now undocumented path. "stackwise...": [1.0] * 6

mattdangerw · 2024-12-03T21:54:35Z

keras_hub/src/utils/timm/convert_efficientnet.py

+    },
+    "rw_s": {
+        "width_coefficient": 1.0,
+        "depth_coefficient": 1.0,


same here, and elsewhere in this file

mattdangerw · 2024-12-03T21:56:14Z

keras_hub/src/models/efficientnet/efficientnet_backbone.py

-        activation: activation function to use between each convolutional layer.
-        input_shape: optional shape tuple, it should have exactly 3 input
-            channels.
+        stackwise_width_coefficient: list[float] or float, scaling coefficient


doesn't this need to be a list[float]?

correct, done

pkgoogle added 13 commits November 4, 2024 16:33

WIP initially adding edge presets

79110fb

WIP el variant working

9cb5c13

added all hf timm edge presets

64b6af8

removing irrelevant note

ccb508b

format pass

cebc921

remove irrelevant old commented code

11a86a2

fix unit test regression

5533922

add presets to preset file

a935e62

WIP starting changes needed for additional presets

b89638f

WIP 2 variants working

9e1a850

add ConvBNAct Block and tiny variant

e2adf47

adds preset data

753caaa

updated correct config

041de38

pkgoogle added the kokoro:force-run Runs Tests on GPU label Nov 12, 2024

pkgoogle requested a review from divyashreepathihalli November 12, 2024 21:32

kokoro-team removed the kokoro:force-run Runs Tests on GPU label Nov 12, 2024

pkgoogle and others added 2 commits November 18, 2024 20:01

Merge branch 'master' into add_efficientnet2_presets

02aeee5

resolve merge conflicts

9eacf21

pkgoogle added the kokoro:force-run Runs Tests on GPU label Nov 18, 2024

kokoro-team removed the kokoro:force-run Runs Tests on GPU label Nov 18, 2024

divyashreepathihalli requested changes Nov 20, 2024

View reviewed changes

mattdangerw reviewed Nov 20, 2024

View reviewed changes

Merge branch 'master' into add_efficientnet2_presets

c173e54

pkgoogle self-assigned this Nov 21, 2024

review updates

a84bddb

add projection_activation argument to fused blocks to fix timm discre…

e1a597d

…pancy

pkgoogle added the kokoro:force-run Runs Tests on GPU label Nov 25, 2024

kokoro-team removed the kokoro:force-run Runs Tests on GPU label Nov 25, 2024

mattdangerw reviewed Dec 3, 2024

View reviewed changes

additional review updates

5ed664f

pkgoogle added the kokoro:force-run Runs Tests on GPU label Dec 3, 2024

kokoro-team removed the kokoro:force-run Runs Tests on GPU label Dec 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds efficientnet2 presets #1983

Adds efficientnet2 presets #1983

pkgoogle commented Nov 11, 2024

divyashreepathihalli left a comment

divyashreepathihalli Nov 20, 2024

pkgoogle Nov 21, 2024

divyashreepathihalli Nov 20, 2024

pkgoogle Nov 21, 2024

divyashreepathihalli Nov 20, 2024

pkgoogle Nov 21, 2024

divyashreepathihalli Nov 20, 2024

pkgoogle Nov 21, 2024

mattdangerw left a comment

mattdangerw Nov 20, 2024

pkgoogle Nov 21, 2024

mattdangerw Nov 20, 2024

pkgoogle Nov 21, 2024

mattdangerw Nov 20, 2024

pkgoogle Nov 21, 2024

mattdangerw Nov 20, 2024

pkgoogle Nov 21, 2024

mattdangerw Nov 20, 2024

pkgoogle Nov 21, 2024

pkgoogle commented Nov 21, 2024

mattdangerw commented Nov 22, 2024

pkgoogle commented Nov 25, 2024

mattdangerw left a comment

mattdangerw Dec 3, 2024

pkgoogle Dec 3, 2024

mattdangerw Dec 3, 2024

pkgoogle Dec 3, 2024

mattdangerw Dec 3, 2024

pkgoogle Dec 3, 2024

mattdangerw Dec 3, 2024

pkgoogle Dec 3, 2024

mattdangerw Dec 3, 2024

pkgoogle Dec 3, 2024


		BN_AXIS = 3

		CONV_KERNEL_INITIALIZER = {

Adds efficientnet2 presets #1983

Are you sure you want to change the base?

Adds efficientnet2 presets #1983

Conversation

pkgoogle commented Nov 11, 2024

divyashreepathihalli left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattdangerw left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pkgoogle commented Nov 21, 2024

mattdangerw commented Nov 22, 2024

pkgoogle commented Nov 25, 2024

mattdangerw left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment