Awesome Mixed Reality

🌟 A curated list of awesome resources for Mixed Reality development.

Contents

• Awesome Mixed Reality
  • Contents
  • YouTube Tutorials
  • Websites
  • Tools
  • Debugging
  • Packages & Services
  • SDKs
  • Assets
  • GitHub Projects
  • Interesting Hardware
  • Books
  • Supporting Apple Vision Pro
    • Apple Vision Development on Xcode
  • Quest 3 Mixed Reality
    • Using Meta XR SDK
  • Use XR Interaction Toolkit
  • Industry Experts and Companies
  • Contribute

YouTube Tutorials

• Unity tutorials - Comprehensive Unity tutorials for mixed reality development.
• Unity Fireship - Advanced Unity topics, covering various aspects of Unity development.
• C# Fireship - In-depth guide to C# programming, essential for Unity developers.
• .NET framework explained - Overview of the .NET framework, useful for understanding Unity's backend.
• Valem - How to Make a XR Game in Unity - Valem's MR tutorial with the latest Meta XR All-In-One SDK.
• Valem - VR Dev for Beginners - Beginner-friendly tutorials on VR development using Unity.
• Valem's Let's make a VR game - Step-by-step guide to creating VR games from scratch.
• Dilmer Valecillos's YT Channel
• Justin P Barnett's YT Channel
• Samyam's YT Channel
• Brackey's YT Channel (Gamedev GOAT)
• Black Whale Studio YT Channel
• Procedural Animation
• GeoHot on AI Girlfriend - Exploration of AI integration in mixed reality applications.

Websites

• Awesome Unity GitHub - A curated list of Unity resources, tools, and tutorials.
• Quest Docs - Official Oculus Quest documentation and resources for developers.

Tools

• Meta XR Simulator - Simulate and test XR applications.
• Meta Quest Developer Hub - Comprehensive development tools for Oculus Quest.
• Wonderland Engine (WebXR) - Wonderland Engine for creating WebXR applications.
• Cognitive3D (Analytics) - Analytics platform tailored for XR applications.
• Meta Haptics Studio - Development tools for creating haptic feedback experiences.
• wit.ai (NLP) - Natural language processing tools for VR/AR applications.
• webxr.tools - A collection of tools for WebXR development.
• Mattercraft - Mixed reality development tools.
• Spark AR Studio - Tool for creating augmented reality experiences.
• Meta tools - A suite of tools provided by Meta for XR development.
• Lens Studio - Developing AR apps on Snap platform

Debugging

• Immersive Debugger - Immersive Debugger inside the headset by Meta SDK
• RenderDoc for Oculus - Frame capture and analysis tool for Oculus applications.
• Unity Profiler - Built-in performance profiling tool for Unity applications.
• OVR Metrics Tool - Oculus VR metrics analysis tool.
• ovrgpuprofiler - GPU profiling tool for Oculus applications.
• VrApi Logcat - Logging tool for VR applications, useful for debugging.
• XRGizmos - Gizmos for XR development in Unity.
• Graphy - Performance monitoring tool for Unity applications.

Packages & Services

• OpenAI - Integration of OpenAI services with Unity.
• Ezy Slice - Mesh slicing framework for Unity.
• Unity third person character controller - Pre-built character controller for Unity.
• Unity Gaming Services (Netcode) - Networking services provided by Unity for game development.
• ParrelSync (multiplayer) - Tool for synchronizing multiplayer sessions in Unity.
• Unity Relay - Networking relay service for Unity games.
• Unity Lobby - Lobby service for multiplayer games.
• Shadow Receiver URP - Universal Render Pipeline shadow receiver for Unity.

SDKs

• Meta XR all-in-one SDK - Comprehensive SDK for developing Meta XR applications.
• Meta Building Blocks - Pre-built components for XR development.
• Meta Spatial SDK - Native Android Framework for Meta XR
• MR Utility Kit - Utility kit for mixed reality development.
• SteamVR - SDK for developing VR applications for Steam.
• Unity XR Interaction Toolkit - Toolkit for creating XR interactions in Unity.
• MRTK - Mixed Reality Toolkit by Microsoft.
• VRTK - VR Toolkit for Unity, providing a range of tools and components.
• HurricaneVR - VR interaction framework.
• AutoHandVR - Tool for hand interactions in VR.
• VR Interaction Framework - Comprehensive framework for VR interactions.
• VR Physics Player Controller - Physics-based player controller for VR applications.
• Portals for VR - Portal system for VR applications.
• Final IK - Inverse kinematics system for VR.
• VR Hand Models - Hand models for VR applications.
• Amplify Impostor - Impostor rendering system for Unity.
• Easy Save - Data saving tool for Unity.
• Aura 2 - Volumetric lighting system for Unity.
• RayFire - Destruction and fragmentation tool for Unity.
• Poly Universal - Universal assets for VR/AR development.
• Synty Studios - Provider of stylized assets for Unity.
• Niantic Lightship - AR development platform by Niantic.
• LOVR - Lua Virtual Reality Framework
• AR Foundation - Unity framework for AR development.
• XR Hands - Hand tracking tools for XR applications.
• Meta XR interaction SDK OVR Samples - Sample projects for Meta XR.
• Meta XR Haptics SDK - SDK for developing haptic feedback in XR.
• Meta Avatars SDK - SDK for creating avatars in XR applications.
• Meta Voice SDK - SDK for integrating voice interactions in XR.

Assets

• Unity Asset Store - Marketplace for Unity assets.
• Sketchfab - Repository of 3D models.
• TurboSquid - Marketplace for 3D models.
• Free3D - Free 3D models for various applications.
• CGTrader - 3D model marketplace.
• Pro builder - 3D modeling tool for Unity.
• Mixamo - Tool for 3D character animations.
• Poliigon - 3D Assets and Textures
• Kenney.nl - Free game assets.
• Ready Player Me - Avatar creation tool.
• Character Creator - Tool for creating detailed 3D characters.
• Oculus Lipsync - Lip sync tool for Oculus.
• Meta 3DGen - Tool for generating 3D models.
• Luma AI - AI-based tool for creating 3D models.
• AR Foundation Remote - Remote testing tool for AR Foundation.
• DoTween Pro - Tweening engine for Unity.
• Hologram Shaders Pro - Hologram shaders for Unity.
• InWorld.ai - AI-based character behavior tool.
• Live Script Reload - Live scripting tool for Unity.
• Odin Inspector & Serializer - Advanced inspector and serialization tool for Unity.
• Vour - VR asset creation tool.
• SRDebugger - Debugging tool for Unity applications.
• Stable Fast 3D - Rapid 3D Asset Generation From Single Images
• Mesh Effects - Mesh effects tool for Unity.
• Pro Sound Collection - Sound effects collection for Unity.

GitHub Projects

• Oculus Samples - Sample projects provided by Oculus.
• Unity Showcase - Showcase of Unity projects.
• AR Foundation Samples - Sample projects for AR Foundation.
• Mixed Reality Example for Meta-OpenXR - Example project for Meta-OpenXR.
• First Hand - Hand interaction sample projects.
• The World Beyond - Sample project for world-building in VR.
• Phanto - VR interaction sample project.
• XR Interaction Toolkit Example - Example projects for XR Interaction Toolkit.
• VR Beginner The Escape Room - VR beginner tutorial project.
• Scape Scraper - VR game development series.
• Open Brush - Open-source VR painting application.
• VR Alchemy Lab - VR interaction sample projects.
• Move Fast - Mixed reality interaction sample project.
• Whisperer - Voice SDK sample projects.
• Ultimate Glove Ball - VR glove interaction sample project.

Interesting Hardware

• WEART Haptic Gloves - Advanced haptic feedback gloves.
• Magic Leap 2 - Mixed reality headset.
• Tilt Five AR Glasses - Augmented reality glasses.
• Logitech MX Ink pen - MR Stylus.
• Zapbox - iPhone as a MR headset device

Books

• Virtual Reality - Steven Lavalle - Comprehensive guide to virtual reality, providing foundational knowledge and advanced techniques.
• Interactive Computer Graphics - Edward Angel - Textbook on computer graphics with a focus on WebGL.

Supporting Apple Vision Pro

• Unity is building PolySpatial to use Unity assets in Apple's headset.
• URP and Standard Render Pipeline are supported, including shader graphs for custom shaders.
• Gaze interactor for eye tracking is available.
• Unity editor preview is supported.
• Signup for PolySpatial beta to get early access.
• Ensure Unity version 6 and above for compatibility.

Apple Vision Development on Xcode

• Use the latest version of Xcode with visionOS, and install Reality Composer Pro.
• Familiarize with concepts from SwiftUI and RealityKit.
• Use the built-in simulator to test applications.

Quest 3 Mixed Reality

Using Meta XR SDK

• Ensure the headset is in developer mode.
• Create a passthrough component:
  • Set underlay placement.
  • Remove the skybox in OVR rig.
  • Enable passthrough support, Quest 3 in OVR manager.
• Scene model:
  • Create a scene manager component on a game object.
  • Add a plane and volume prefab to it.
  • Enable scene support in OVRCameraRig.
• Enable depth:
  • Create a new mixed reality depth material.
  • Recommended: use Meta's depth API.
• Shadows:
  • Create a texture and set it to the shadow material.
  • Use a Raycast with the same layer mask as the scene's global mesh.
  • Recommended: Use the shadow receiver custom shader and add that to the scene's plane, and enable shadow in all the mesh renderers.

Use XR Interaction Toolkit

• Ensure to use Unity version 6 and above.
• Install the dependencies:
  • XR Plugin Management
  • XR Interaction Toolkit
  • OpenXR Plugin
  • AR Foundation
  • meta-openxr
• Configurations:
  • Enable Meta quest feature group.
  • Disable auto graphics API, and set color space to linear, target architecture to ARM64, Vulkan above OpenGLES3.
  • Disable terrain holes and HDR, post-processing.
• Edit the project hierarchy:
  • Add AR Session.
  • Add AR Plane Manager in XR Rig.
  • Add AR Camera Manager and AR Camera Background in Main Camera, remove the skybox.
• Customize the plane by adding a plane prefab to the AR Plane Manager.
• Anchor:
  • Add AR Anchor Manager to XR Rig.
  • Add AR Anchor to the prefab.

Industry Experts and Companies

Thanks to Robert Scoble for his valuable contribution. Here are the lists of VR/AR experts and companies curated by him:

• VR/AR Experts:

  • List 1 (550 people)
  • List 2

• Companies in XR:

  • Companies List

Contribute

Contributions welcome! Read the contribution guidelines first.

Author

Saurabh Chalke

Feel free to reach out to me on Twitter/X for any sort of help related to MR :)

---