Install errors on Mac

[neilheyes]

I've followed the instructions and every video just gives me the error:

"view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead."

Really stuck as to what to do.

Any help would be great.

[nagadomi]

Could you post all error messages including stack traces? It should be displayed in the terminal.
I do not have macOS which I can test with low cost.

[neilheyes]

2024-11-02 11:14:47.001 Python[57626:5314990] +[IMKClient subclass]: chose IMKClient_Modern
2024-11-02 11:14:55.630 Python[57626:5314990] +[IMKInputSession subclass]: chose IMKInputSession_Modern
/opt/homebrew/lib/python3.11/site-packages/torch/nn/functional.py:4565: UserWarning: The operator 'aten::_upsample_bilinear2d_aa.out' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/mps/MPSFallback.mm:13.)
return torch._C._nn._upsample_bilinear2d_aa(
File "/Users/neil/Downloads/depth_pro/nunif/iw3/gui.py", line 1203, in on_exit_worker
args = result.get()
^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/wx/lib/delayedresult.py", line 232, in get
raise self.__exception
File "/opt/homebrew/lib/python3.11/site-packages/wx/lib/delayedresult.py", line 267, in wrapper
result = workerFn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/utils.py", line 1685, in iw3_main
process_video(args.input, args.output, args, depth_model, side_model)
File "/Users/neil/Downloads/depth_pro/nunif/iw3/utils.py", line 769, in process_video
process_video_full(input_filename, output_path, args, depth_model, side_model)
File "/Users/neil/Downloads/depth_pro/nunif/iw3/utils.py", line 719, in process_video_full
VU.process_video(input_filename, output_filename,
File "/Users/neil/Downloads/depth_pro/nunif/nunif/utils/video.py", line 705, in process_video
for new_frame in get_new_frames(frame_callback(frame)):
^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/nunif/utils/video.py", line 1137, in call
future = self.thread_pool.submit(self.frame_callback, batch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/nunif/utils/video.py", line 1072, in submit
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/utils.py", line 709, in _batch_callback
return __batch_callback(x)
^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/utils.py", line 677, in __batch_callback
depths = args.state["depth_utils"].batch_infer(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/zoedepth_model.py", line 227, in batch_infer
out = _forward(model, x, enable_amp)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/zoedepth_model.py", line 134, in _forward
out = model(x)['metric_depth']
^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/pretrained_models/hub/nagadomi_Depth-Anything_iw3_main/metric_depth/zoedepth_depth_anything/models/zoedepth/zoedepth_v1.py", line 148, in forward
rel_depth, out = self.core(x, denorm=denorm, return_rel_depth=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/pretrained_models/hub/nagadomi_Depth-Anything_iw3_main/metric_depth/zoedepth_depth_anything/models/base_models/depth_anything.py", line 271, in forward
rel_depth = self.core(x)
^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/pretrained_models/hub/nagadomi_Depth-Anything_iw3_main/metric_depth/zoedepth_depth_anything/models/base_models/dpt_dinov2/dpt.py", line 156, in forward
depth = self.depth_head(features, patch_h, patch_w)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/pretrained_models/hub/nagadomi_Depth-Anything_iw3_main/metric_depth/zoedepth_depth_anything/models/base_models/dpt_dinov2/dpt.py", line 110, in forward
x = self.projectsi
^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 554, in forward
return self._conv_forward(input, self.weight, self.bias)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 549, in _conv_forward
return F.conv2d(
^^^^^^^^^
(.venv) ➜ nunif git:(depth_pro)

[neilheyes]

sorry for the long set of messages

[neilheyes]

this is one using DepthPro_HD - gives the same error in the ui

2024-11-02 11:21:56.895 Python[59056:5329354] +[IMKClient subclass]: chose IMKClient_Modern
2024-11-02 11:23:37.160 Python[59056:5329354] +[IMKInputSession subclass]: chose IMKInputSession_Modern
File "/Users/neil/Downloads/depth_pro/nunif/iw3/gui.py", line 1203, in on_exit_worker
args = result.get()
^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/wx/lib/delayedresult.py", line 232, in get
raise self.__exception
File "/opt/homebrew/lib/python3.11/site-packages/wx/lib/delayedresult.py", line 267, in wrapper
result = workerFn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/utils.py", line 1685, in iw3_main
process_video(args.input, args.output, args, depth_model, side_model)
File "/Users/neil/Downloads/depth_pro/nunif/iw3/utils.py", line 769, in process_video
process_video_full(input_filename, output_path, args, depth_model, side_model)
File "/Users/neil/Downloads/depth_pro/nunif/iw3/utils.py", line 719, in process_video_full
VU.process_video(input_filename, output_filename,
File "/Users/neil/Downloads/depth_pro/nunif/nunif/utils/video.py", line 705, in process_video
for new_frame in get_new_frames(frame_callback(frame)):
^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/nunif/utils/video.py", line 1137, in call
future = self.thread_pool.submit(self.frame_callback, batch)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/nunif/utils/video.py", line 1072, in submit
result = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/utils.py", line 709, in _batch_callback
return __batch_callback(x)
^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/utils.py", line 677, in __batch_callback
depths = args.state["depth_utils"].batch_infer(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/depth_pro_model.py", line 163, in batch_infer
out = _forward(model, x, enable_amp)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/depth_pro_model.py", line 124, in _forward
canonical_inverse_depth, _ = model(x)
^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/pretrained_models/hub/nagadomi_ml-depth-pro_iw3_main/depth_pro/depth_pro.py", line 246, in forward
encodings = self.encoder(x)
^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/neil/Downloads/depth_pro/nunif/iw3/pretrained_models/hub/nagadomi_ml-depth-pro_iw3_main/depth_pro/network/encoder.py", line 326, in forward
x2_features = self.upsample2(x2_features)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/container.py", line 250, in forward
input = module(input)
^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 554, in forward
return self._conv_forward(input, self.weight, self.bias)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/conv.py", line 549, in _conv_forward
return F.conv2d(
^^^^^^^^^

[nagadomi]

What version of pytorch do you have installed?
python -c "import torch; print(torch.__version__)"

Does Any_S and ZoeD_N work?

[neilheyes]

so have the nightly build of pytorch 2.6 install in an environment - the MPS doesn't work still:

/opt/homebrew/lib/python3.11/site-packages/torch/nn/functional.py:4673: UserWarning: The operator 'aten::_upsample_bilinear2d_aa.out' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications.

I get thats a different issue than I mentioned above, but thought I'd let you know too

[nagadomi]

That is a warning, not an error.
I previously tried to disable bilinear antialiasing to be able to use MPS backend, but did not see any performance improvement, so I did not change the code for compatibility with other OS.

[neilheyes]

cool, thanks - I think ZoeD_N is using the GPU/MPS Backend as its quite quick - 1.5fps

[nagadomi]

The warning means that only the bilinear upsampling layer is using CPU. other layers are using MPS.

[nagadomi]

If you are concerned about the warning, uncomment line 167 and use it instead of line 168.

nunif/iw3/zoedepth_model.py

Lines 165 to 169 in 31c0137

	# TODO: 'aten::_upsample_bilinear2d_aa.out' is not currently implemented for the MPS device
	# This did not cause any performance problems, so I decided not to handle it.
	# antialias = not device_is_mps(x.device)
	antialias = True

[nagadomi]

The problem of DepthPro probably fixed. nagadomi/ml-depth-pro_iw3@e796192
Try update with

git pull
python3 -m iw3.download_models

However, I have never tried DepthPro on macOS, so there may be another problem.

[nagadomi]

For DepthAnything, it is already using reshape, so it is a problem on PyTorch MPS side and cannot be fixed.

            x = x.permute(0, 2, 1).reshape((x.shape[0], x.shape[-1], patch_h, patch_w))
            
            x = self.projects[i](x)

https://github.com/nagadomi/Depth-Anything_iw3/blob/82a94d6fb5fd68add51d9a8be20e43249e728c81/metric_depth/zoedepth_depth_anything/models/base_models/dpt_dinov2/dpt.py#L108-L110

If you are using an older version of pytorch, try building an env with the latest pytorch version using venv.