feat[next]: Optimisations for icon4py

[samkellerhals]

Description

Changes to reduce Python overhead when calling icon4py granules from ICON.

• Removing isinstance checks from extract_connectivity_args and convert_args.

[havogt]

Forwarding the review to @egparedes for improving Python patterns in performance critical code parts.

[havogt]

would functools.singledispatch work?

[samkellerhals]

I think it could work, however whilst nicer I have the feeling that it would be slower than a simple dictionary lookup.

[DropD]

I think it would be preferable to sacrifice readability only on the basis of hard evidence.

[havogt]

But in general, could make sense to discuss this in the context of going towards frozen programs.

[DropD]

If this pattern proves to be a significant optimization over singledispatch, it still needs to be made more readable.

I propose to encode the pattern in a class. That class then needs a docstring explaining when to use it over standard approaches and why, for example, subclasses of the relevant types won't work with it unless they are added explicitly to the dict.

Sketch:

class FastDispatch:
    """
    Optimized version of functools.singledispatch, does not take into account inheritance or protocol membership.
    
    This leads to a speed-up of XXX, as documented in ADR YYY.
    
    Usage:
    >>> @Fastdispatch.fastdispatch(Any)
    ... def extract_connectivity_args(connectivity, *args, **kwargs):
    ...     return None
    ...
    ... @extract_connectivity_args(NeighborTableOffsetProvider):
    ... def extract_connectivity_args_from_nbtable(connectivity, device, *args, **kwargs):
    ...     return (_ensure_is_on_device(connectivity.table, device), zero_tuple)
    """
    _registry: dict[type: str]
    
    def __call__(self, dispatch_arg, *args, **kwargs):
        return getattr(self, self._registry[type(dispatch_arg)])(dispatch_arg, *args, **kwargs)
        
    def register(self, type):
        def decorator(function):
            self._registry[type] = function
            return function
        return decorator
        
    @classmethod
    def fastdispatch(cls, default_type):
        return decorator(function):
            dispatcher = cls()
            dispatcher.register(default_type)(function)
            return dispatcher
        return decorator