DepthAnything on Browser

Description / Rationale

This repository demonstrates web-based implementation of Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data. Foundation Model for Monocular Depth Estimation and more capable Depth Anything V2. This project demonstrates the following:

1. Basic DepthAnything, which shows only generated depth ("basic" folder)
2. Interactive DepthAnything, which shows 3D model based on generated depth and Three.js ("interactive" folder).
3. Basic DepthAnythingV2, which shows only generated depth ("basic-v2" folder)
4. Interactive DepthAnything v2 which shows 3D model based on generated depth and Three.js ("interactive-v2" folder).

Instructions

To use a DepthAnything example, copy the corresponding html file contents. Onnx original models were taken and adapted from Fabio Milentiansen Sim's Depth Anything ONNX repository. Quantization was done using Onnx's quantization functionality.

To dowload and locally serve models download them from the link provided:

1. Depth Anything small model: https://cdn.glitch.me/0f5359e2-6022-421b-88f7-13e276d0fb33/depthanything_vits14.onnx (~97 mb)
2. Depth Anything quantized model: https://cdn.glitch.me/0f5359e2-6022-421b-88f7-13e276d0fb33/depthanything-quant.onnx (~26 mb)
3. Depth Anything v2 dynamic quantized small model: https://cdn.glitch.me/0f5359e2-6022-421b-88f7-13e276d0fb33/depthanythingv2-vits-dynamic-quant.onnx (~26 mb)
4. Depth Anything v2 dynamic small model: https://cdn.glitch.me/0f5359e2-6022-421b-88f7-13e276d0fb33/depthanythingv2-vits-dynamic.onnx (~97 mb)
5. Depth Anything v2 small model: https://cdn.glitch.me/0f5359e2-6022-421b-88f7-13e276d0fb33/depthanythingv2-vits.onnx (~97 mb)

Please note:

1. To increase quality, increase the input image size, for example make it 1024px.

  const size = 1024; // change input size as needed

2. To omit points in pointcloud generated, indicate how many:

 const skip = 2; // skip points here

3. To increase point cloud point size:

gl_PointSize = 5.0; // change size here

Webgpu version works on the browsers and devices supporting it (see: https://github.com/gpuweb/gpuweb/wiki/Implementation-Status).

Tech Stack

The project was made possible thanks to DepthAnything and DepthAnythingV2, Fabio Milentiansen Sim's Depth Anything ONNX repositories, ONNX runtime web and Three.js.

Demo

To see DepthAnything v1 and 2 at work, visit the following pages: DepthAnything v1 demo, DepthAnything v2 demo, DepthAnything v2 webgpu (fastest) demo, DepthAnything v2 webgpu with Sliders demo.