[TOC]
Lullaby is a collection of C++ libraries designed to provide an efficient way to create and interact with objects in a virtual/augmented reality world.
It is the Engine that drives your VR/AR apps using the underlying SDKs.
- Your App: The Experience
- The virtual/augemented reality experiences only you can give to your users (eg. watching 360 videos, exploring the world, etc.)
- Lullaby: The Engine
- A collection of libraries designed for building VR experiences running on top of a high-performance framework.
- VR/AR SDKs: The Platform
- Provides access to low-level hardware (eg. HMD, Controller, etc.) and features (eg. OpenGL, Spatial Audio, Scanline Racing, etc.)
At a high level, Lullaby's architecture can be divided into 3 broad categories: Systems, Base Tech, and Utilities.
Lullaby's primary goal is to provide an efficient, data-driven way to create, manipulate, and interact with objects in a "virtual" world. For this, Lullaby uses an Entity-Component-System (ECS) architecture. An ECS consists of three parts: Entities, Components, and Systems.
An Entity represents a single, uniquely identifiable "thing" in the virtual reality world. However, an Entity is literally just a number; it has no other data on its own. Instead, specific pieces of data (like 3D position, velocity, shape, or interaction state) are associated with an Entity at runtime. These data objects are called Components. Lastly, Systems process and update one or more types of Components, effectively giving Entities their functions and behaviours. Each System is specialized to perform a specific job, such as playing animations, rendering graphics or detecting collisions.
In other words, the behaviour of an Entity is a result of the Components associated with it and the Systems which operate on those Components. For example, a Physics System could update the position of all Entities with both position and velocity Components, or a Render System would render Entities using the positon and shape Components.
Lullaby also has an offline data representation that enables fast data-driven iteration of its Entity-Component-System architecture. Applications use data files known as Blueprints to describe the state of an Entity that can then be instantiated in the runtime. Schema files are used to describe the structure of the data contained in a Blueprint.
The bulk of Lullaby's functionality is implemented as individual Systems, such as the TransformSystem, AudioSystem, RenderSystem, and AnimationSystem. A complete list of Systems can be found here.
Details about Lullaby's ECS implementation can be found here.
A list of available Systems can be found here.
Lullaby's Systems often require access to a common set of functionality in order to work with each other. This functionality, along with the ECS implementation, represents the "base" of the Lullaby Engine. It is important for users of Lullaby to have a good understanding these libraries.
More information about these base libraries can be found here.
Lastly, Lullaby provides several utility libraries. These range from simple functions (e.g. string hashes and bit manipulation), to classes (e.g. variant types, optional type, and thread-safe containers) to complete modules (like a JNI interface and integration with scripting languages like JavaScript and Lua).
Most of these Utilities are used internally by Systems and Core Tech, or are available as optional extensions to Lullaby's primary functionality. Unfortunately, the only way to learn about these Utilities is to explore the Lullaby codebase.
All C++ code is available under the src/ folder. It contains the following subfolders:
- base: The ECS implementation and the base tech libraries.
- systems: The "core" Lullaby Systems that provide most of the critical functionality like animation, audio, and rendering. These libraries are developed and maintained by the main Lullaby team.
- contrib: This contains useful Systems developed by various teams throughout Google. These tend to be more "mid-level" Systems, like dropdowns and tooltips.
- util: Various utility libraries as described above.
- schemas: Flatbuffer schema files.
Note: Some code that should be in util/ may actually be in base/ and vice-versa due to legacy reasons. (Mainly because it's somewhat arbitrary as to what is a "base" technology vs. what is a utility.)
Finally, the rest of the folders are:
- g3doc: This documentation you are reading.
Instructions on building the Lullaby open source libraries is available here.
Follow our step-by-step Integration Guide for how to integrate Lullaby into your application.