diff --git a/docs/Doxyfile b/docs/Doxyfile index b21aac83..e83474a6 100644 --- a/docs/Doxyfile +++ b/docs/Doxyfile @@ -911,9 +911,9 @@ INPUT = ../modules/sl/source \ ../modules/wai/source \ ../modules/utils/source \ ../modules/math/source \ - ../apps/app_demo_slproject/include \ - ../apps/app_demo_slproject/glfw \ - ../apps/app_demo_slproject/source \ + ../apps/app_demo_slproject \ + ../apps/app_demo_node \ + ../apps/source \ ../apps/app_demo_webgpu \ pages/Introduction.md \ pages/ExampleScenes.md \ @@ -930,6 +930,7 @@ INPUT = ../modules/sl/source \ pages/ExampleSkinnedAnimations.md \ pages/ExampleVolumeRendering.md \ pages/OnPaint.md \ + pages/AppFramework.md \ pages/Emscripten.md \ pages/BlenderToSL.md \ pages/SLProject.md @@ -963,6 +964,7 @@ INPUT_ENCODING = UTF-8 FILE_PATTERNS = *.cpp \ *.h \ + *.mm \ *.frag \ *.vert @@ -1101,7 +1103,7 @@ USE_MDFILE_AS_MAINPAGE = # also VERBATIM_HEADERS is set to NO. # The default value is: NO. -SOURCE_BROWSER = NO +SOURCE_BROWSER = YES # Setting the INLINE_SOURCES tag to YES will include the body of functions, # classes and enums directly into the documentation. diff --git a/docs/images/app_framework.svg b/docs/images/app_framework.svg new file mode 100644 index 00000000..355d1bbb --- /dev/null +++ b/docs/images/app_framework.svg @@ -0,0 +1,4 @@ + + + +
SL Layer
Platform Layer
App Layer
AppDemoMain
AppNodeMain
AppExample
App.h
AppGLFW.cpp
AppEmscripten.cpp
AppAndroid.cpp
AppIOS.mm
SLInterface.h
AppDemo
SLScene
SLSceneView
SLImGui
 \ No newline at end of file diff --git a/docs/pages/AppFramework.md b/docs/pages/AppFramework.md new file mode 100644 index 00000000..3b6f965a --- /dev/null +++ b/docs/pages/AppFramework.md @@ -0,0 +1,126 @@ +\page app-framework The %App Framework + +\section overview Overview + +SLProject provides a framework for quickly setting up platform-independent apps that run on Windows, macOS, Linux, Android, iOS, and the Web. +The app framework is divided into three layers: + +- The **%App Layer** contains the code of your app. It contains the main function, some configuration code, and callbacks for hooking into + the SLProject systems. The central interface every app uses is the `App.h` header, which is implemented by the platform layer. +- The **Platform Layer** contains all platform-specific code for creating the rendering surface, setting up event handlers, etc. Every + platform has its own implementation of the `App.h`. The implementation corresponding to the target platform is selected at compile time by CMake. + The platform layer uses `SLInterface.h` to access SLProject modules. +- The **SL Layer** contains all the SLProject libraries. This is where rendering, UI, math, asset loading, input handling, video capture, etc. + is done. + +
+ +
+ +\section usage Usage + +\subsection cpp C++ + +The main C++ file of an app looks something like this: + +```cpp +#include +// Other includes... + +static SLScene* createScene(SLSceneID sceneID) +{ + return new AppExampleScene(); +} + +static void buildGui(SLScene* s, SLSceneView* sv) +{ + // ... +} + +int SL_MAIN_FUNCTION(int argc, char* argv[]) +{ + App::Config config; + config.argc = argc, + config.argv = argv; + config.windowWidth = 640; + config.windowHeight = 480; + config.windowTitle = "SLProject Example App"; + config.onNewScene = createScene; + config.onGuiBuild = buildGui; + // More config... + + return App::run(config); +} +``` + +The main function is defined using a macro that expands to the name of the main function on +the current platform. This is usually `main`, but on Android it expands to `slAndroidMain`, +a fake main function that is called from the JNI to initialize the application because there is +no actual `main` on Android. + +Inside the main function, an `App::Config` instance is created. This configuration specifies +things like the window size, the ID of the start scene, or a callback for building the GUI. +`App::run` is then called with this config to start the app. Every platform implements its own +version of this function. The various implementations can be found in `apps/source/platforms`. +Here's a list of all of them: + +- `AppGLFW.cpp` (Windows, macOS, Linux) +- `AppEmscripten.cpp` (Web) +- `AppAndroid.cpp` (Android) +- `AppIOS.mm` (iOS) + +An implementation of `App::run` typically sets up the rendering surface, registers event handlers, +loads core assets, starts the event loop, and returns when the program terminates. +Only the Android version returns immediately because the event loop is handled by Java code and +interacts with the app via the JNI. + +For more information on creating a new app, see +[this wiki page](https://github.com/cpvrlab/SLProject4/wiki/Creating-a-New-App). + +\subsection cmake CMake + +New apps are added by calling the custom CMake function `sl_add_app`. This function is defined +in `apps/CMakeLists.txt` and takes care of creating a new target and setting source files, +headers, compile options, linked libraries, properties, etc. depending on the target platform. +The function has parameters for specifying things like the name of the target, the platforms +the app supports, additional source files (e.g. for scenes), the directory of the Android +project, or iOS resources. + +`sl_add_app` uses the CMake variable `SL_PLATFORM` to decide which platform is currently being +targeted. This variable is set in the top-level `CMakeLists.txt` based on the current target +system. The possible platforms are: + +- `GLFW` (Windows, macOS, Linux) +- `EMSCRIPTEN` (Web) +- `ANDROID` (Android) +- `IOS` (iOS) + +\section scene-loading Scene Loading + +A scene is loaded by setting the value of `AppDemo::sceneToLoad` to the ID of the new scene. +Inside the event loop of every platform, we check whether this variable has a value and switch +to the new scene if so: + +```cpp +if (AppDemo::sceneToLoad) +{ + slSwitchScene(sv, *AppDemo::sceneToLoad); + AppDemo::sceneToLoad = {}; +} +``` + +Scene switching is implemented inside `AppLoad::switchScene`. This function calls the callback +`AppConfig::onNewScene` to create a new scene instance. The callback is specified by the app and +returns an instance of a class derived from `SLScene` (e.g. `AppDemoSceneMinimal`). +After creating the new scene instance, we switch to it in two stages: + +1. Register assets to be loaded. We call the overloaded `SLScene::registerAssetsToLoad` so it can + register tasks on the `SLAssetLoader` to be executed asynchronously in a different thread. + Meanwhile, the event loop on the main thread continues running so we can for example render + a loading screen. Every frame, `SLAssetLoader::checkIfAsyncLoadingIsDone` is called to check whether + the worker thread is done. When that happens, the next stage is triggered. + Note: All I/O operations have to be registered here because they cannot be executed synchronously on + the main thread on the Emscripten platform. +2. Assemble the scene. When the asset loader is done, we call the overloaded `SLScene::assemble` + function that uses the loaded assets to assemble the scene. After assembly, `AppDemo::scene` + is set to the new scene and the scene switch is completed. diff --git a/docs/pages/BlenderToSL.md b/docs/pages/BlenderToSL.md index 22a295e2..ae336ea7 100644 --- a/docs/pages/BlenderToSL.md +++ b/docs/pages/BlenderToSL.md @@ -1,7 +1,4 @@ \page blender-to-sl Blender to SLProject Export -\htmlonly - -\endhtmlonly \section intro Introduction

This page gives an overview of things to consider when preparing models created in the 3D software Blender for use in SLProject. diff --git a/docs/pages/DataProtectionDeclaration.md b/docs/pages/DataProtectionDeclaration.md index 92ec0633..100e9ad1 100644 --- a/docs/pages/DataProtectionDeclaration.md +++ b/docs/pages/DataProtectionDeclaration.md @@ -1,7 +1,4 @@ \page data-protection-declaration SLProject Data Protection Declaration -\htmlonly - -\endhtmlonly The SLProject uses the following permissions: * **Internet access**: If available the app downloads predefined camera calibration parameters. diff --git a/docs/pages/Emscripten.md b/docs/pages/Emscripten.md index e44417d6..bf7a74bb 100644 --- a/docs/pages/Emscripten.md +++ b/docs/pages/Emscripten.md @@ -1,7 +1,4 @@ \page emscripten Emscripten -\htmlonly - -\endhtmlonly SLProject can be built to run in a web browser. We use the [Emscripten](https://emscripten.org/) toolchain, which compiles C/C++ code to a binary instruction format called [WebAssembly](https://webassembly.org/). The generated Wasm module can be loaded by ordinary JavaScript at runtime. The browser then compiles the instructions to native machine code and runs it, making it possible to run high-performance code in the browser. To access OpenGL functions, browsers expose the [WebGL API](https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API), which Emscripten wraps in standard OpenGL headers. SLProject uses WebGL 2, which is based on OpenGL 3.0 ES. Not all scenes from app-Demo-SLProject can run in the browser because OpenGL 4.0 functions are not available or because some OpenCV modules can't be compiled for WebAssembly. diff --git a/docs/pages/ExampleGlTF.md b/docs/pages/ExampleGlTF.md index c9c93755..7be22287 100644 --- a/docs/pages/ExampleGlTF.md +++ b/docs/pages/ExampleGlTF.md @@ -1,7 +1,4 @@ \page example-gltf glTF File Format -\htmlonly - -\endhtmlonly SLProject supports loading the [glTF model format](https://en.wikipedia.org/wiki/GlTF). This scene showcases loading and rendering the damaged helmet sample by Khronos. Notice how the environment is reflected onto the helmet. Note that it may take some time to load this scene since there is a lot of diff --git a/docs/pages/ExampleImageBasedLighting.md b/docs/pages/ExampleImageBasedLighting.md index ec398d49..c58b6d90 100644 --- a/docs/pages/ExampleImageBasedLighting.md +++ b/docs/pages/ExampleImageBasedLighting.md @@ -1,7 +1,4 @@ \page example-ibl Image Based Lighting -\htmlonly - -\endhtmlonly The following scene shows 7 x 7 spheres with different metallic (vertical) and roughness (horizontal) material parameters. These parameters can be passed either by float values or as textures per pixel. diff --git a/docs/pages/ExampleLevelOfDetail.md b/docs/pages/ExampleLevelOfDetail.md index 35f3c5bb..82629ae0 100644 --- a/docs/pages/ExampleLevelOfDetail.md +++ b/docs/pages/ExampleLevelOfDetail.md @@ -1,7 +1,4 @@ \page example-lod Level of Detail -\htmlonly - -\endhtmlonly The following scene shows 2500 corinthian columns with each 83k triangles in their highest resolution. With 3 levels of detail the amount of geometry is reduced depending on the size of the bounding rectangle in screen space. In addition, we do automatically a view frustum culling to optimize rendering performance. diff --git a/docs/pages/ExampleNodeAnimations.md b/docs/pages/ExampleNodeAnimations.md index 3cd4b055..e0a477eb 100644 --- a/docs/pages/ExampleNodeAnimations.md +++ b/docs/pages/ExampleNodeAnimations.md @@ -1,7 +1,4 @@ \page example-node-animations Node Animations -\htmlonly - -\endhtmlonly See the appDemoLoadScene function in AppDemoLoad.cpp for this scene (SID_AnimationNode) on how to create node animations and how to assign them to an SLNode instance. SLAnimation instances are created and controlled by the SLAnimManager. diff --git a/docs/pages/ExampleParticles.md b/docs/pages/ExampleParticles.md index fc956fab..ff39749d 100644 --- a/docs/pages/ExampleParticles.md +++ b/docs/pages/ExampleParticles.md @@ -1,7 +1,4 @@ \page example-particles Particle Systems -\htmlonly - -\endhtmlonly The SLParticleSystem is the most complex derivative of the SLMesh class and allows the creation of highly configurable particle systems (PS). All its parameters can be modified in the _Properties_ tab (see menu Info > Properties). The implementation uses for rendering geometry shaders on systems that have OpenGL >= 4.1 or OpenGLES >= 3.2. All other diff --git a/docs/pages/ExamplePathtracing.md b/docs/pages/ExamplePathtracing.md index c8317e80..29095803 100644 --- a/docs/pages/ExamplePathtracing.md +++ b/docs/pages/ExamplePathtracing.md @@ -1,7 +1,4 @@ \page example-pathtracing Path Tracing -\htmlonly - -\endhtmlonly Alternatively you can render a scene with path tracing using the menu *Renderer > Path Tracing*. The path tracer is optimized for the Blinn-Phong reflection model extended with reflection and refraction coefficient. By default, the resolution 0.5 times the window resolution. You can adjust the path tracing setting over the menu *PT*. diff --git a/docs/pages/ExampleRaytracing.md b/docs/pages/ExampleRaytracing.md index 6e59bb22..97ebd2e1 100644 --- a/docs/pages/ExampleRaytracing.md +++ b/docs/pages/ExampleRaytracing.md @@ -1,7 +1,4 @@ \page example-raytracing Ray Tracing -\htmlonly - -\endhtmlonly Alternatively you can render a scene with ray tracing using the menu *Renderer > Ray Tracing*. The ray tracer is optimized for the Blinn-Phong reflection model. By default, the resolution 0.5 times the window resolution. You can adjust the ray tracing setting over the menu *RT*. diff --git a/docs/pages/ExampleRevolver.md b/docs/pages/ExampleRevolver.md index 0037f31e..aa1c23d8 100644 --- a/docs/pages/ExampleRevolver.md +++ b/docs/pages/ExampleRevolver.md @@ -1,7 +1,4 @@ \page example-revolver Revolving Meshes -\htmlonly - -\endhtmlonly Examples of revolving mesh objects constructed by rotating a 2D curve. The classes SLArrow, SLCone, SLCylinder, SLDisk, SLLens and SLSpheric inherit the SLRevolver class that inherits from SLMesh. diff --git a/docs/pages/ExampleScenes.md b/docs/pages/ExampleScenes.md index ac9f244c..f89caebb 100644 --- a/docs/pages/ExampleScenes.md +++ b/docs/pages/ExampleScenes.md @@ -1,7 +1,4 @@ \page example-scenes Online Example Scenes -\htmlonly - -\endhtmlonly All examples presented here run live in browsers that support [WebAssembly](https://webassembly.org/) and the [Emscripten framework](https://emscripten.org/docs/introducing_emscripten/index.html). These are at the moment: Google Chrome, Firefox and Microsoft Edge. diff --git a/docs/pages/ExampleShadowMapping.md b/docs/pages/ExampleShadowMapping.md index 2cee5910..40a3746d 100644 --- a/docs/pages/ExampleShadowMapping.md +++ b/docs/pages/ExampleShadowMapping.md @@ -1,7 +1,4 @@ \page example-shadow-mapping Shadow Mapping -\htmlonly - -\endhtmlonly In this scene you can see all light types: Directional, rectangular, spot and point light. All light types (SLLightDirect, SLLightRect, SLLightSpot) are derived from SLLight and can have a SLShadowMap. diff --git a/docs/pages/ExampleSkinnedAnimations.md b/docs/pages/ExampleSkinnedAnimations.md index b20df5ad..064180a1 100644 --- a/docs/pages/ExampleSkinnedAnimations.md +++ b/docs/pages/ExampleSkinnedAnimations.md @@ -1,7 +1,4 @@ \page example-skinned-animations Skinned Animations -\htmlonly - -\endhtmlonly Skinned skeleton animation are imported at the moment with the Collada file format (DAE). Skinned skeleton animation are controlled by the SLAnimManager. Its interface is in the menu *Animation*. diff --git a/docs/pages/ExampleVolumeRendering.md b/docs/pages/ExampleVolumeRendering.md index 68e98b8d..3b29f937 100644 --- a/docs/pages/ExampleVolumeRendering.md +++ b/docs/pages/ExampleVolumeRendering.md @@ -1,8 +1,5 @@ \page example-volume-rendering Volume Rendering -\htmlonly - -\endhtmlonly Volume Rendering of an MRI angiography dataset. In the material properties you can adjust the transfer function for color and transparency points that transform the MRI intensities into color and transparency. Volume rendering is realized with a special fragment shader that performs a ray cast through the MRI volume. diff --git a/docs/pages/Introduction.md b/docs/pages/Introduction.md index aa092c09..ee73865f 100644 --- a/docs/pages/Introduction.md +++ b/docs/pages/Introduction.md @@ -1,143 +1,105 @@ \page introduction Introduction \htmlonly - \endhtmlonly \tableofcontents + \section overview Overview -

-There are 5 code sections in an SLProject application: -

+ +There are 5 code sections in SLProject: + +- **APP**: Application code + - OS-dependent OpenGL context and window creation + - UI definition using ImGUI + - Scene definition using SL + - Video processing using CV +- **SL**: Scene Library code + - Scene & scene view management + - Scenegraph classes + - 3D and 2D rendering + - Animation +- **CV**: Computer vision code: + - Video capturing + - Calibration + - Tracking +- **Utils**: Utilities used by SL and CV +- **Externals**: External dependencies + -

-

+ \section diagram Class Diagram -

+ The following class diagram gives you an overview of the major classes with its important attributes and methods: -

- -

-

+ +- The **gray boxes** on top contain the application code that depend on the OS and do the GUI, the scene assembly and the video processing. +- The **light blue classes** are the central classes with the top-level instances of SLAssetManager and SLInputManager. The core classes for the scene are SLScene and SLSceneView. +- The **dark blue classes** are the alternative renderers for ray tracing and path tracing. +- The **yellow classes** define the materials that are responsible for the visual appearances of the mesh objects. +- The **green classes** build the scene graph that defines the spacial structure of the visible objects. +- The **pink classes** define a single triangulated mesh object. +- The **violet classes** encapsulate all OpenGL vertex array object and buffer objects. +- The **red classes** build the animation framework. +- The **orange classes** encapsulate the video, image and AR tracking functionality using OpenCV. CV classes are independent from all SL classes. Only SLGLTexture uses CVImage for its texture images. +- The **red classes** build the animation framework. +- The **white classes** are low level classes for the math. Some of them are within the namespace Utils. +- The **gray boxes** at the bottom are the external libraries that are used within the frame work. + + + \section app Application Code -

-The applications code (grey boxes at the top of the diagram) contains the code for the operating system, the scene definition with SLProject library (SL), the video processing using CV-classes and the UI with ImGUI. In all cases we have the most outer shell of the application that handles the window and the OpenGL context creation and passes the events to a thin C-function interface before it is handled by the C++-framework in the library lib-SLProject. The following OS' are supported and applications are provided for demonstration: -

-

+ +The application's code (grey boxes at the top of the diagram) contains the code for the operating system, the scene definition with SLProject library (SL), the video processing using CV-classes and the UI with ImGUI. In all cases we have the most outer shell of the application that handles the window and the OpenGL context creation and passes the events to a thin C-function interface before it is handled by the C++-framework in the library lib-SLProject. The following systems are supported and applications are provided for demonstration: + +- **Windows, Linux and macOS** applications use the [GLFW](http://www.glfw.org/) + C-library for the platform independent window and context creation. + GLFW is included in the SLProject repository. See the app-Demo-GLFW for demonstration. +- **Android** applications start in Java and pass the events using the JNI (Java + Native Interface) to the C-interface. See the [wiki for build instructions](https://github.com/cpvrlab/SLProject4/wiki/Build-for-Android) + and `apps/source/platforms/android/example_project` for the project used in demos. +- **Apple iOS** applications start in Objective-C before it passes the events to the C-interface. + See the [wiki for build instructions](https://github.com/cpvrlab/SLProject4/wiki/Build-on-MacOS-with-XCode-for-iOS) + and `apps/source/platforms/ios/example_project` for the project used in demos. +- **Web** applications use [Emscripten](https://emscripten.org/) to compile C++ code to + [WebAssembly](https://webassembly.org/). This allows applications to be served by a web server + and run inside the browser. + +For all demo apps (desktop and mobile) we use the [Dear ImGui](https://github.com/ocornut/imgui) library for the UI. +The UI for `app-Demo-SLProject` is implemented in AppDemoGui. Dear ImGui is also included in the repository. +You could in fact use any GUI library on any OS that can render using OpenGL. +Other alternatives could be e.g. [Qt](https://www.qt.io/), [freeglut](http://freeglut.sourceforge.net/), +[FLTK](http://www.fltk.org/index.php), [wxWidgets](http://www.wxwidgets.org/), +[Nana](http://www.nanapro.org/en-us/), or [Juce](http://www.juce.com/). + +SLInterface.h and SLInterface.cpp define the C-Interface of the SLProject library. +We use a C-interface because this type can be called from any higher level language. +The SLInterface talks only to the SLInputManager, SLScene and SLSceneView classes. + +For more information about SLProject's app framework, see [this page](#app-framework). \section central Central Classes -

+ The light blue classes form the center of the SLProject framework: -

-

+ +- SLAssetManager holds the expensive resources such as meshes, materials, textures and shader programs in vectors. +- SLInputManager collects all user events from the mouse and keyboard as well as from additional input devices such as a LeapMotion or Kinect sensor. +- SLScene is the top-level class of the framework that represents the scene with + its properties. The scene content is created in SLScene::onLoad. + It also holds one or more pointers to SLSceneView instances. +- SLSceneView represents a dynamic real time 3D view onto the scene. + A scene can be shown in multiple sceneviews as demonstrated in the app-Viewer-Qt application. + A scene view receives all events (keyboard, mouse etc.) from the GUI via the SLInputManager. \section node Scenegraph Classes -

+ SLNode is the major building block for the the scenegraph structure (green classes) and can have 0-N children nodes and 0-N triangle meshes. A node can be transformed (translated, rotated and scaled) in 3D-space. -

-

+ +- SLLightDirect, SLLightSpot and SLLightRect are from SLNode derived and define lights that can be placed and directed in space. +- SLNodeLOD implements the level of detail functionality. +- SLCamera that defines the view to the scene. The scene can have multiple cameras but only one can be active for the scene view. \section mesh Mesh Classes -

+ SLMesh is the base class for triangulated or wire framed meshes (pink classes). A mesh is rendered with a material defined in SLMaterial. A mesh has all the vertex attributes such as position, normals, texture coordinates. @@ -146,140 +108,81 @@ A mesh has an instance of SLGLVertexArray that does all the OpenGL drawing. This vertex array object (VAO) stores all attributes in either a float or half float vertex attribute buffer (SLGLVertexBuffer) that is generated in the memory of the GPU. The attribute data on the client side is not deleted because it is used for ray tracing. -

-

+ +- SLRevolver, SLSphere, SLCylinder, SLCone, SLBox, SLPolygon and SLRectangle are all inheritants from SLMesh and represent the according platonic shapes. +- SLAABBox and SLUniformGrid implement the space partitioning. Every SLNode has an axis aligned AABB that is used for fast frustum culling and ray shooting. \section vao VertexArray Classes -

+ SLGLVertexArray and SLGLVertexBuffer encapsulate all OpenGL buffer stuff and provides the core drawing functionality with OpenGL. -

\section material Material Classes -

+ SLMaterial is the core of the yellow classes that define the appearance of a mesh. A material can have one or more texture images and is rendered with a specific shader program written in the OpenGL shading language (GLSL). -

-

+ +- SLGLTexture defines a texture image with filtering parameters. +- SLGLProgram defines a shader program with at least one vertex and one fragment shader program. +- SLGLProgramGenerated implements the automatic GLSL shader generation based on the material parameters, the lights and if shadow mapping is used. +- SLGLShader defines a vertex or fragment shader where the source code is read from a file. +- All OpenGL code is restricted to the classes beginning with SLGL. (SLGLState, SLGLTexture, SLGLShader, SLGLProgram, SLGLVertexArray and SLGLVertexBuffer.) +- The linear algebra math is implemented in the classes SLMat3, SLMat4, SLVec3, SLVec4 and SLQuat4. \section animation Animation Classes -

+ The red animation classes provide the functionality for simple node animations or skeletal animations. -

-

+ +- SLAnimManager: A single instance of this class is held by the SLScene instance and is + responsible for updating the enabled animations and to manage their life time. + It keeps a list of all skeletons and node animations and also holds a list of + all animation playback controllers. The update of all animations is done before + the rendering of all SLSceneView instances. +- SLAnimPlayback manages the playback state and the local time of an SLAnimation. + It manages the way the time advances and how the animation loops. It has all + functionality to play, pause, stop, enable, speed up and slow down a playback. + A list of all SLAnimPlayback instances is held by the SLAnimManager. +- SLAnimation is a container for multiple SLAnimTrack that build an animation. + E.g. a walk animation would consist of all the SLAnimTrack that make a + SLSkeleton walk. It also knows the length of the animation. +- SLAnimTrack and SLNodeAnimTrack: An animation track is a track that affects a + single SLNode or an SLJoint of an SLSkeleton by interpolating its transform. + It holds therefore a list of SLKeyframe. For a smooth motion it can interpolate + the transform at a given time between two neighbouring SLKeyframe. +- SLKeyframe and SLTransformKeyframe define a transform at a certain time on an + SLAnimTrack. +- SLSkeleton: A skeleton is used to animate a hierarchical object like a human figure. + An SLSkeleton keeps track of its bones (SLJoints) in a tree structure and + points with _root to the root node of the skeleton hierarchy. + An SLSkeleton is not actively transforming any SLNode in the scene graph. + It just keeps track of its transformed SLJoint. + A mesh that is associated with a skeleton transforms all its vertices every + frame by the joint weights (Jw). Every vertex of a mesh has weights for up to four joints + by which it can be influenced. +- SLJoint is a specialised SLNode that represents a single joint (or bone) in a skeleton + The main addition of SLJoint to the base SLNode is the offset matrix which is the + inverse transformation of the joint's binding pose in mesh space. + It is used to transform the vertices of a rigged SLMesh to the origin of the joint + to be able to manipulate them in the joint's local space. \section imageprocessing Image and Video Processing Classes -

+ The orange classes provide the functionality for video and image processing using the OpenCV framework. The SLProject framework can now process the images from attached live video cameras. This works via OpenCV on desktop OS as well as on iOS and Android. The live video image is constantly fed into an OpenGL texture that can be used as a texture on an objects material or as the scenes background. With the live video in the background you can create augmented reality (AR) applications. Examples can be found in the demo application under Load Scene > Using Video > Track Chessboard or Track AruCo. -

-

- -

-Authors: marcus.hudritsch@bfh.ch
-Date: January 2022
-Copyright (c): 2002-2022 Marcus Hudritsch, Kirchrain 18, 2572 Sutz, Switzerland -

+ +- CVImage: Replaces the deprecated SLImage class and provides all for loading, saving and converting images. Internally it stores the image in a cv::Mat instance. +- CVCapture: Holds static images from the OpenCV video capture or from an external (iOS and Android) video capture service. There is an CVCapture::lastFrame and an CVCapture::lastFrameGray with the gray level version of the last capture video frame. +- CVCalibration holds all functionality to calibrate the video camera. A classic chessboard pattern is used for calibration. In the demo application a special scene is provided for the calibration (Preferences > Video > Calibrate Camera). +- CVTracked is the base class for tracking classes. The scene can have multiple trackers. +- A tracker is associated with a Node. When the object to be tracked is found, it controls the nodes transform. If the associated node is the scenes active camera a classic augmented reality application can be generated. +- CVTrackedChessboard tracks the same chessboard that is used for the camera calibration. +- CVTrackedAruco tracks special markers called AruCo markers. These markers are optimal in tracking performance and stability. +- CVTrackedFeatures tracks any 2D features. Supported are any feature detectors and descriptors that are provided by OpenCV. + In addition we include the enhanced ORB feature detector developed by [Raul Mur](https://github.com/raulmur/ORB_SLAM2). + +Authors: marcus.hudritsch@bfh.ch \ +Date: Juli 2024 \ +Copyright (c): 2002-2024 Marcus Hudritsch, Kirchrain 18, 2572 Sutz, Switzerland diff --git a/docs/pages/OnPaint.md b/docs/pages/OnPaint.md index d3770d5c..2ce87fb7 100644 --- a/docs/pages/OnPaint.md +++ b/docs/pages/OnPaint.md @@ -1,7 +1,4 @@ \page on-paint How one Frame gets painted -\htmlonly - -\endhtmlonly As an example we start here at the outer shell of a desktop app using GLFW. The most outer onPaint routine gets called in the main event loop diff --git a/docs/pages/SLProject.md b/docs/pages/SLProject.md index b5af9047..018f887c 100644 --- a/docs/pages/SLProject.md +++ b/docs/pages/SLProject.md @@ -1,7 +1,4 @@ \mainpage SLProject -\htmlonly - -\endhtmlonly

Welcome to the SLProject. SL stands for Scene Library. It is developed at the @@ -48,6 +45,7 @@ The code is provided without any warranties whether expressed or implied.

Read the \subpage introduction for an overview of the framework.
Read \subpage on-paint for an overview of how on frame gets rendered.
+Read \subpage app-framework for an overview of the app framework.
Read \subpage emscripten for an overview of how SLProject runs in the browser.