diff --git a/extensions/2.0/Khronos/KHR_gaussian_splatting/README.md b/extensions/2.0/Khronos/KHR_gaussian_splatting/README.md index cccf932263..a6b7e656ce 100644 --- a/extensions/2.0/Khronos/KHR_gaussian_splatting/README.md +++ b/extensions/2.0/Khronos/KHR_gaussian_splatting/README.md @@ -32,159 +32,245 @@ Release Candidate Written against the glTF 2.0 spec. -## Table of Contents - -- [Contributors](#contributors) -- [Status](#status) -- [Dependencies](#dependencies) -- [Overview](#overview) -- [Adding 3D Gaussian Splats to Primitives](#adding-3d-gaussian-splats-to-primitives) -- [Geometry Type](#geometry-type) -- [Mathematics of rendering using the default Ellipse Kernel](#mathematics-of-rendering-using-the-default-ellipse-kernel) -- [Lighting](#lighting) -- [glTF JSON Example](#gltf-json-example) -- [Extension Properties](#extension-properties) -- [Attributes](#attributes) -- [Extending the Base Extension](#extending-the-base-extension) -- [Appendix A: Spherical Harmonics Reference](#appendix-a-spherical-harmonics-reference) -- [Known Implementations](#known-implementations) -- [Resources](#resources) - ## Overview -This extension defines basic support for storing 3D Gaussian splats in glTF, bringing structure and conformity to the 3D Gaussian splatting space. 3D Gaussian splatting uses fields of Gaussians that can be treated as a point cloud for the purposes of storage. 3D Gaussian splats are defined by their position, rotation, scale, and spherical harmonics which provide both diffuse and specular color. These values are stored as values on a point primitive. Since we treat the 3D Gaussian splats as points primitives, a graceful fallback to treating the data as a sparse point cloud is possible. +This extension defines basic support for storing 3D Gaussian splats in glTF assets, bringing structure and conformity to the 3D Gaussian splatting space. 3D Gaussian splatting uses fields of Gaussians that can be treated as a point cloud for the purposes of storage. This extension defines 3D Gaussian splats by their position, rotation, scale, opacity, and spherical harmonics, which provide both diffuse and specular color. These values are stored as attributes on a point primitive. Since the extension treats the 3D Gaussian splats as point primitives, a graceful fallback to treating the data as a sparse point cloud is possible. -A key objective of this extension is to establish a solid foundation for integrating 3D Gaussian splatting into glTF, while enabling future enhancements and innovation. To achieve this, the extension is intentionally designed to be extended itself, allowing extensions to introduce new kernel types, color spaces, projection methods, and sorting methods over time as 3D Gaussian splatting techniques evolve and become standards within the glTF ecosystem. +A key objective of this extension is to establish a solid foundation for integrating 3D Gaussian splatting into glTF assets, while enabling future enhancements and innovation. To achieve this, the extension is intentionally designed to be extended itself, allowing future extensions to introduce new kernel types, color spaces, projection methods, and sorting methods over time as 3D Gaussian splatting techniques evolve and become standards within the glTF ecosystem. -## Adding 3D Gaussian Splats to Primitives +## Extending Mesh Primitives -When a primitive contains an `extension` property defining `KHR_gaussian_splatting`, this indicates to the client that the primitive should be treated as a 3D Gaussian splatting field. +When a mesh primitive contains an `extension` property defining `KHR_gaussian_splatting`, this indicates to the client that the primitive SHOULD be treated as a 3D Gaussian splatting field. -The extension must be listed in `extensionsUsed`: +Other extensions that depend on this extension such as 3D Gaussian splatting compression extensions MAY require that this extension be included in `extensionsRequired`. -```json - "extensionsUsed" : [ - "KHR_gaussian_splatting" - ] -``` +The extension object contains the following properties: + +| | Type | Description | Required | +| ----------------- | -------- | -------------------------------------------------- | ------------------------------ | +| **kernel** | `string` | The kernel used to generate the Gaussians. | :white_check_mark: Yes | +| **colorSpace** | `string` | The color space of the reconstructed color values. | :white_check_mark: Yes | +| **projection** | `string` | The projection method for rendering the Gaussians. | No, default `"perspective"`. | +| **sortingMethod** | `string` | The sorting method for rendering the Gaussians. | No, default `"cameraDistance"` | + +### Kernel + +Gaussian splats can have a variety of shapes and this has the potential to change over time. The `kernel` property is a required property that defines an exact Gaussian type, i.e., its shape, attributes, intended rendering techniques, and interactions with the base glTF features. + +This extension defines only one kernel type called `"ellipse"`, that is a 2D ellipse kernel used to project an ellipsoid shape in 3D space. Additional kernel types can be added over time by supplying an extension that defines an alternative definition. See the section [Ellipse Kernel](#ellipse-kernel) below for the exact definitions. + +### Color Space + +The `colorSpace` property is a required property that specifies the color space of the reconstructed 3D Gaussian Splat color values. The color space is typically determined by the training process for the splats. This color space value only applies to the 3D Gaussian splatting data and does not affect any other color data in the glTF. + +Unless specified otherwise by additional extensions, this color space information refers to the reconstructed splat color values, therefore all intermediate rendering steps (including but not limited to alpha blending) MUST be performed before any color gamut or transfer function conversions. + +Additional values can be added over time by defining extensions to add new color spaces. See the section [Extending the Base Extension](#extending-the-base-extension) for more information. + +#### Available Color Spaces + +| Color Space | Description | +| --------------------- | --------------------------------------------- | +| `srgb_rec709_display` | BT.709 sRGB (display-referred) color space. | +| `lin_rec709_display` | BT.709 linear (display-referred) color space. | + +> [!NOTE] +> These string values follow the color space identification pattern recommended by the ASWF Color Interoperability Forum (CIF). See: [ASWF Color Space Encodings for Displays](https://github.com/AcademySoftwareFoundation/ColorInterop/blob/main/Recommendations/02_DisplayColorSpaces/DisplayColorSpaces.md). + +### Projection + +The `projection` property is an optional property that specifies how the Gaussians are projected onto the kernel shape. This is typically provided by the training process for the splats. This property is meant to be extended in the future as new projections become standardized within the community. + +This base extension defines a single projection method, `"perspective"`, which is the default value. + +Additional values can be added over time by defining extensions to add new projection methods. See the section [Extending the Base Extension](#extending-the-base-extension) for more information. + +### Sorting Method + +The `sortingMethod` property is an optional property that specifies how the Gaussian particles are sorted during the rendering process. This typically is provided by the training process for the splats. This property is meant to be extended in the future as new sorting methods become standardized within the community. + +This base extension defines a single sorting method, `cameraDistance`, which is the default value. This method sorts the splats based on the length of the vector from the splat to the camera origin (viewer's position). + +Additional values can be added over time by defining extensions to add new sorting methods. See the section, [Extending the Base Extension](#extending-the-base-extension), for more information. + +## Ellipse Kernel + +A 2D ellipse kernel type represents 3D Gaussian splats in an ellipsoid shape. + +### Dependencies on glTF + +The `mode` property of the mesh primitive MUST be `POINTS` (0). + +Unless specified otherwise by additional Gaussian splats extensions, the glTF material referenced from the mesh primitive (if any) MUST be ignored for splat rendering. + +> [!NOTE] +> If this extension is not supported, implementations are expected to follow the base glTF specification and thus render the splat primitive as a point cloud. In such a case, the referenced glTF material will be used. -Other extensions that depend on this extension such as 3D Gaussian splatting compression extensions may require that this extension be included in `extensionsRequired`. +The global transformation matrix of the node containing a mesh with splat primitives SHOULD conform to all of the following conditions: -## Geometry Type +- The last row is $(0, 0, 0, 1)$ (implicitly enforced by the base glTF specification). +- The lengths of the first three columns are finite positive numbers. +- The determinant of the upper-left 3x3 matrix formed by dividing each matrix element by the length of its column is close to positive one. -The `mode` of the `primitive` must be `POINTS`. +These rules ensure that the transformation matrix is decomposable into regular translation, rotation, and positive scale values. Splat rendering with non-decomposable transformation matrices or with negative scale values is undefined. -## Mathematics of rendering using the default Ellipse Kernel +The camera used for splat rendering SHOULD use perspective projection. Splat rendering with non-perspective projections is undefined. -To render a field of 3D Gaussian splats, the renderer must reconstruct each Gaussian splat using the same forward pass algorithm used during training. This involves using the position, scale, rotation, opacity, and spherical harmonics attributes to reconstruct the Gaussian splat in 3D space. +### Attributes -There are three key stages to reconstructing and rendering a 3D Gaussian splat: +This kernel defines the following attributes for the mesh primitive. - 1. [3D Gaussian Representation](#3d-gaussian-representation) - 2. [Projection of 3D Gaussian onto 2D Kernel](#projection-of-3d-gaussian-onto-2d-kernel) - 3. [Rendering: Sorting and Alpha Blending](#rendering-sorting-and-alpha-blending) +| Splat Data | Attribute Semantic | Accessor Type | Component Type | Required | +| ---------------------------- | --------------------------------- | - | - | - | +| Position | `POSITION` | VEC3 | Inherited from the glTF specification | :white_check_mark: Yes | +| Rotation | `KHR_gaussian_splatting:ROTATION` | VEC4 | _float_
_signed byte_ normalized
_signed short_ normalized | :white_check_mark: Yes | +| Scale | `KHR_gaussian_splatting:SCALE` | VEC3 | _float_
_unsigned byte_
_unsigned byte_ normalized
_unsigned short_
_unsigned short_ normalized | :white_check_mark: Yes | +| Opacity | `KHR_gaussian_splatting:OPACITY` | SCALAR | _float_
_unsigned byte_ normalized
_unsigned short_ normalized | :white_check_mark: Yes | +| Spherical Harmonics degree 0 | `KHR_gaussian_splatting:SH_DEGREE_0_COEF_0` | VEC3 | _float_ | :white_check_mark: Yes | +| Spherical Harmonics degree 1 | `KHR_gaussian_splatting:SH_DEGREE_1_COEF_[0-2]` | VEC3 | _float_ | no (yes if degree 2 or 3 are used) | +| Spherical Harmonics degree 2 | `KHR_gaussian_splatting:SH_DEGREE_2_COEF_[0-4]` | VEC3 | _float_ | no (yes if degree 3 is used) | +| Spherical Harmonics degree 3 | `KHR_gaussian_splatting:SH_DEGREE_3_COEF_[0-6]` | VEC3 | _float_ | no | -This extension defines a default `ellipse` kernel type that is based on the kernel defined in [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/). See the [Ellipse Kernel](#ellipse-kernel) section for more details on how this is defined. +The mean vector for the Gaussian splat is provided by the `POSITION` position of the mesh primitive. This defines the center of the Gaussian splat ellipsoid in local space. The effective global mean vector for the Gaussian splat is derived from the `POSITION` attribute value and the global transformation matrix of the glTF node that instantiates the mesh containing the splat primitive as defined in the glTF specification. -### 3D Gaussian Representation +The scale (`KHR_gaussian_splatting:SCALE`) attribute values correspond to the spread of the Gaussian along its local principal axes. Scale values are linear and MUST NOT be negative. -Each Gaussian splat using the default `ellipse` kernel represents a 3D Gaussian defined by the following equation: +The rotation (`KHR_gaussian_splatting:ROTATION`) attribute values correspond to the orientation of those axes in local space. Rotation values are stored as unit quaternions in the usual glTF order. + +> [!NOTE] +> This guarantees that renderers can use quaternion values directly without renormalization. + +The effective covariance matrix for the Gaussian splat is derived from the `KHR_gaussian_splatting:ROTATION` and `KHR_gaussian_splatting:SCALE` attributes and the effective transformation matrix of the glTF node that instantiates the mesh containing the splat primitive as described below. + +The opacity of a Gaussian splat is defined by the `KHR_gaussian_splatting:OPACITY` attribute. It stores a normalized linear value between _0.0_ (transparent) and _1.0_ (opaque). Out-of-range values are invalid. + +> [!NOTE] +> A sigmoid activation function applied during training ensures the value stays within the valid range. This guarantees that renderers can use the stored opacity directly for alpha blending without any extra processing. + +#### Spherical Harmonics Attributes + +The `KHR_gaussian_splatting:SH_DEGREE_0_COEF_0` attribute semantic provides the diffuse component coefficients for the spherical harmonics. The zeroth-order spherical harmonic coefficients are always required. The `KHR_gaussian_splatting:SH_DEGREE_l_COEF_n` attributes where `l` is greater than zero hold the higher degrees of spherical harmonics data. To use higher degrees of spherical harmonics the lower degrees MUST be defined. + +Each increasing degree of spherical harmonics requires more coefficients. At the first degree, three sets of coefficients are required, increasing to five sets for the second degree, and increasing to seven sets at the third degree. With all three degrees, this results in 45 spherical harmonic coefficients stored in the `KHR_gaussian_splatting:SH_DEGREE_l_COEF_n` attributes. + +Attributes are packed from the lowest order ($m$) to highest for each degree. For example, `KHR_gaussian_splatting:SH_DEGREE_1_COEF_0` provides $m = -1$ coefficients, `KHR_gaussian_splatting:SH_DEGREE_1_COEF_1` provides $m = 0$ coefficients, and `KHR_gaussian_splatting:SH_DEGREE_1_COEF_2` provides $m = 1$ coefficients. + +Each coefficient contains three values representing the red, green, and blue channels of the spherical harmonic. Spherical harmonic degrees MUST NOT be partially defined, that is, either all coefficients for a given degree and all lower degrees MUST be defined or none. + +## Ellipse Kernel Rendering + +To render a field of 3D Gaussian splats, the renderer reconstructs each Gaussian splat using the same forward pass algorithm used during training, projects it onto a 2D plane, computes its color, and composes it with other splats. + +This kernel assumes a _3σ_ cut-off (Mahalanobis distance of 3 units) for correct rendering. + +> [!NOTE] +> This is the shape used by the original [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/) paper. + +### Splat Reconstruction + +Each Gaussian splat represents a 3D Gaussian defined by the following equation: ```math G(x) = \exp\left(-\frac{1}{2}(x-\mu)^T \Sigma^{-1} (x-\mu)\right) ``` Where: + - $G(x)$ is the value of the Gaussian at position $x$. - $x$ is a 3D position vector. - $\mu$ is the mean vector, representing the center of the Gaussian. - $\Sigma$ is the 3x3 covariance matrix, defining the Gaussian's size, shape, and orientation. -For stable optimization and direct manipulation, the 3x3 covariance matrix $\Sigma$ is constructed from a rotation (quaternion) and a scaling vector. Using eigendecomposition of a covariance matrix, we can express $\Sigma$ as: +The first step is combining the scale and rotation attributes of the splat with the global transformation matrix of the glTF node containing the splat primitive to produce the 3x3 covariance matrix. + +The 3x3 3D covariance matrix $\Sigma$ is constructed as follows: ```math -\Sigma = \mathbf{R}\mathbf{S}\mathbf{S}^T\mathbf{R}^T +\Sigma = \mathbf{M}\mathbf{C}\mathbf{C}^T\mathbf{M}^T ``` Where: -- $\mathbf{R}$ is a 3x3 rotation matrix derived from the quaternion. -- $\mathbf{S}$ is a 3x3 scaling matrix constructed from the scale vector. -To derive the rotation matrix $\mathbf{R}$ from the quaternion $(q_x, q_y, q_z, q_w)$: +- $\mathbf{M}$ is a 3x3 upper-left submatrix of the node's global transformation matrix; +- $\mathbf{C}$ is a 3x3 matrix composed of the splat's scale and rotation attributes as defined below. ```math -\mathbf{R} = \begin{pmatrix} -1 - 2(q_y^2 + q_z^2) & 2(q_x q_y - q_w q_z) & 2(q_x q_z + q_w q_y) \\ -2(q_x q_y + q_w q_z) & 1 - 2(q_x^2 + q_z^2) & 2(q_y q_z - q_w q_x) \\ -2(q_x q_z - q_w q_y) & 2(q_y q_z + q_w q_x) & 1 - 2(q_x^2 + q_y^2) +\mathbf{C} = \begin{pmatrix} +s_x (1 - 2(q_y^2 + q_z^2)) & s_y (2(q_x q_y - q_w q_z)) & s_z (2(q_x q_z + q_w q_y)) \\ +s_x (2(q_x q_y + q_w q_z)) & s_y (1 - 2(q_x^2 + q_z^2)) & s_z (2(q_y q_z - q_w q_x)) \\ +s_x (2(q_x q_z - q_w q_y)) & s_y (2(q_y q_z + q_w q_x)) & s_z (1 - 2(q_x^2 + q_y^2)) \end{pmatrix} ``` -To derive the scale matrix $\mathbf{S}$ from the scale vector $(s_x, s_y, s_z)$: +### Splat Projection -```math -\mathbf{S} = \begin{pmatrix} -s_x & 0 & 0 \\ -0 & s_y & 0 \\ -0 & 0 & s_z -\end{pmatrix} -``` - -### Projection of 3D Gaussian onto 2D Kernel - -To render the scene, each 3D Gaussian splat must be projected onto a 2D kernel shape based on the camera's perspective. This involves transforming the 3D covariance matrix $\Sigma$ into a 2D covariance matrix $\Sigma'$ that represents the Gaussian's shape on the image plane: +The second step is projecting each splat onto a 2D kernel shape based on the camera's perspective transformation. This involves projecting the splat's mean vector (using the regular perspective projection math) and transforming the 3D covariance matrix $\Sigma$ into a 2D covariance matrix $\Sigma'$ that represents the Gaussian's shape on the image plane. The covariance matrix transformation uses the affine approximation: ```math \Sigma' = \mathbf{J}\mathbf{W}\Sigma\mathbf{W}^T\mathbf{J}^T ``` Where: -- $\mathbf{W}$ is the view transformation matrix and performs a rigid transformation (rotation and translation) from the world space to the camera space. This is your standard world-to-camera extrinsic matrix. -- $\mathbf{J}$ is the Jacobian matrix of the projection transformation. This is responsible for perspective. -The Jacobian matrix $\mathbf{J}$ for standard perspective projection is defined as: +- $\mathbf{W}$ is the 3x3 matrix that performs view-dependent transformation from the global space to the camera space; this is usually the upper-left 3x3 submatrix of camera's view transformation. +- $\mathbf{J}$ is the Jacobian matrix of the perspective projection transformation function including the perspective divide. This Jacobian matrix typically includes the camera's focal length and the splat's 3D mean vector transformed by the camera's view transformation matrix (rotation and translation). -```math -\mathbf{J} = \begin{pmatrix} -\frac{f_x}{z} & 0 & -\frac{f_x x}{z^2} \\ -0 & \frac{f_y}{z} & -\frac{f_y y}{z^2} -\end{pmatrix} -``` +> [!NOTE] +> Since the construction details of the view and perspective projection matrices are implementation-specific, the formal definitions of $\mathbf{W}$ and $\mathbf{J}$ are not provided in this specification. + +### Splat Lighting + +Each splat's color is computed from its spherical harmonics coefficients and the normalized viewing direction (vector from the camera position to the splat's mean in the global space) as described in the [Lighting](#lighting) section below. -### Rendering: Sorting and Alpha Blending +Implementations MAY ignore higher-degree coefficients for performance reasons. -Once the 2D projected Gaussian splats are computed, they must be sorted and alpha blended to produce the final image. The alpha-blending is order-dependent, so correct sorting is crucial for accurate rendering. +The spherical harmonics coefficients SHOULD be transformed by the rotation component of the global transformation matrix of the glTF node that instantiates the mesh containing the splat primitive. -The sorting method for the `ellipse` kernel is based on the depth value of each Gaussian, which is the z-coordinate in camera space. Sorting order is back-to-front based on this depth value and typically uses a radix sort for performance. See the [Sorting Method](#sorting-method) section for more details on sorting. +> [!NOTE] +> This transformation is usually performed using Wigner-D matrices. -Once sorted, the final color of each pixel is computed by alpha blending the splats in sorted order. The alpha blending equation is defined as: +### Splat Composition + +After the previous steps, splats are sorted and alpha blended to produce the final image. The alpha-blending is order-dependent, so correct sorting is crucial for accurate rendering. + +The sorting is based on the distance between the camera position and the splat's mean vector in the global space. Sorting order is back-to-front based on this depth value and typically uses a radix sort for performance. See the [Sorting Method](#sorting-method) section for more details on sorting. + +Once sorted, the final color of each pixel is computed by alpha blending the splats in the sorted order. The alpha blending equation is defined as: ```math C = \sum_{i \in \mathscr{N}} c_i \alpha_i \prod_{j=1}^{i-1} (1 - \alpha_j) ``` Where: -- $C$ is the final color of the pixel. + +- $C$ is the final color of the pixel. This value MUST be either clamped or tonemapped in an implementation-specific manner to fit into the splat's color space. - $\mathscr{N}$ is the set of splats that contribute to the pixel. -- $c_i$ is the color of the $i$-th Gaussian. See [Lighting](#lighting) for details on how to compute this color from the spherical harmonics. -- $\alpha_i = \alpha \cdot G(x)$ where $G(x)$ is the value of the projected 2D Gaussian's probability density function evaluated at the pixel center $x$. +- $c_i$ is the color of the $i$-th Gaussian. Unless specified otherwise by additional extensions, negative $c_i$ values MUST be clamped to zero. +- $\alpha_i = \alpha \cdot G(x)$ where $G(x)$ is the value of the projected 2D Gaussian's probability density function evaluated at the pixel center $x$ and $\alpha$ is the splat's opacity attribute value. - $\prod_{j=1}^{i-1} (1 - \alpha_j)$ is the accumulated transmittance. -*Non-normative Note: Within rasterizer implementations of 3DGS, transmittance can often be implicitly handled by the GPU's blending hardware. When using premultiplied alpha in WebGL2, using `glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA)` (or similar) will often work. Without premultiplied alpha, you can use `glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA)` (or similar) instead.* +> [!NOTE] +> Within rasterizer implementations, transmittance is usually handled by the GPU's blending hardware. +> +> Since intermediate results of splats composition are not explicitly limited to any range, blending should be performed using floating-point color buffer formats. This approach more closely follows the design of the original paper. +> +> In a case when a normalized color buffer format is used instead, e.g., for performance or memory reasons, blending should be performed with alpha-premultiplied values by passing the $c_i \alpha_i$ product to the blending unit and setting the source blend factor to one. While still lossy, this approach causes less range clamping than passing color and alpha values separately. +> +> If using unsigned normalized color buffers, clamping of $c_i$ values less than zero and $C$ values greater then one happens automatically. ## Lighting -At the time of writing, the most common method for lighting 3D Gaussian splats is via the real spherical harmonics. This extension defines attributes to store spherical harmonic coefficients for each splat. The zeroth-order spherical harmonic coefficients are always required. Higher order coefficients are optional. Each color channel has a separate coefficient, so for each degree $ℓ$, there are $(2ℓ + 1)$ coefficients, each containing RGB values. +This extension defines 3D Gaussian splat lighting via the real spherical harmonics. The corresponding attributes store spherical harmonic coefficients for each splat. The zeroth-order spherical harmonic coefficients are always required. Higher order coefficients are optional. Each color channel has a separate coefficient, so for each degree $l$, there are $(2l + 1)$ coefficients, each containing values contributing to the red, green, and blue color components. -These rules may be relaxed by future extensions that define alternative lighting methods or have specific requirements for handling compression, such as when a compression method stores the diffuse color components as linear color values instead of the zeroth-order coefficients. +These rules MAY be relaxed by future extensions that define alternative lighting methods or have specific requirements for handling compression, such as when a compression method stores the diffuse color components as linear color values instead of the zeroth-order coefficients. ### Image State & Relighting Image state is defined by ISO 22028-1:2016 and indicates the rendering state of the image data. **_Display-referred_** (also known as _output-referred_ in ISO 22028-1:2016) image state represents data that has gone through color-rendering appropriate for display. **_Scene-referred_** image state represents data that represents the actual radiance of the scene. -The default `ellipse` kernel based on the original 3D Gaussian splatting paper typically uses the _BT.709-sRGB_ color space with a _display-referred_ image state for training and rendering. This is different than the typical glTF PBR material model, where scene-referred linear color spaces are used. This extension defines two display-referred color spaces but scene-referred color spaces may be added by extensions. See: [Available Color Spaces](#available-color-spaces) +The ellipse kernel defined in this specification uses a _display-referred_ image state for training and rendering. This is similar to the material model described in the `KHR_materials_unlit` glTF extension, i.e., glTF scene lighting, exposure settings, and tonemapping generally do not affect rendered splats. -Implementations are allowed to relight the splats than the one they were trained in, but this may lead to visual differences compared to the original training results. +Still, implementations can relight the splats, e.g., to simulate shadows or additional light sources, or to adjust the rendered splats for an environment different than the one they were trained in. Naturally, this could lead to visual differences compared to the original results. ### Calculating color from Spherical Harmonics @@ -202,17 +288,17 @@ The diffuse color of the splat can be computed by multiplying the RGB coefficien Y_{0,0}(θ, φ) = \frac{1}{2} \sqrt{\frac{1}{π}} ≈ 0.282095 ``` -To keep the spherical harmonics within the [0, 1] range, the forward pass of the training process applies a _0.5_ bias to the DC component of the spherical harmonics. The rendering process must also apply this bias when reconstructing the color values from the spherical harmonics. This allows the training to occur around 0, ensuring numeric stability for the spherical harmonics, but also allows the coefficients to remain within a valid range for easy rendering. +To keep the spherical harmonics within the [0, 1] range, the forward pass of the training process applies a _0.5_ bias to the zeroth-order component of the spherical harmonics. The rendering process also applies this bias when reconstructing the color values from the spherical harmonics. This allows the training to occur around 0, ensuring numeric stability for the spherical harmonics, but also allows the coefficients to remain within a valid range for easy rendering. -Ergo, to calculate the diffuse RGB color from the only the DC component, the formula is: +Ergo, to calculate the diffuse RGB color from only the zeroth-order component, the formula is: ```math Color_{diffuse} = SH_{0,0} * 0.2820947917738781 + 0.5 ``` -Where $SH_{0,0}$ represents the a vector of the RGB coefficients for the zeroth-order real spherical harmonic. +Where $SH_{0,0}$ represents the vector of the RGB coefficients for the zeroth-order real spherical harmonic. -The [Condon–Shortley phase](https://mathworld.wolfram.com/Condon-ShortleyPhase.html) is the $(-1)^m$ sign factor included in some spherical‑harmonic definitions to keep their algebraic and normalization properties consistent. This convention ensures the spherical harmonics are orthogonal and behave cleanly with rotation. As defined, odd values for order ($m$) are negated. For Degree 2 and Degree 3, this gives some of the constants negative signs. +The Condon–Shortley phase is the $(-1)^m$ sign factor included in some spherical‑harmonic definitions to keep their algebraic and normalization properties consistent. This convention ensures the spherical harmonics are orthogonal and behave cleanly with rotation. As defined, odd values for order ($m$) are negated. For Degree 2 and Degree 3, this gives some of the constants negative signs. Subsequent degrees of spherical harmonics are used to compute more complex lighting effects, such as ambient occlusion and specular highlights, by evaluating the spherical harmonics at the appropriate angles based on the surface normal and light direction. As an example, the Degree 1 functions are: @@ -228,7 +314,7 @@ A full list of all spherical harmonic functions and constants including the Cond For all of these functions, $r$ represents the magnitude of the position vector, calculated as $r = \sqrt{x^2 + y^2 + z^2}$. Within 3D Gaussian splatting, normalization is used to ensure that the direction vectors are unit vectors. Therefore, $r$ is equal to $1$ when evaluating the spherical harmonics for lighting calculations. -We can use these functions combined with the DC component to calculate the full color of a Gaussian: +We can use these functions combined with the zero-order component to calculate the full color of a Gaussian: ```math \begin{aligned} @@ -252,190 +338,75 @@ Color_{final} =\,&Color_{SH_{0}} + Color_{SH_{1}} + Color_{SH_{2}} + Color_{SH_{ \end{aligned} ``` -Where $SH_{\ell,m}$ represents the RGB spherical harmonic coefficients for a particular degree ($\ell$) and order ($m$) and $x$, $y$, and $z$ represent the independent parts of the direction vector. +Where $SH_{l,m}$ represents the RGB spherical harmonic coefficients for a particular degree ($l$) and order ($m$) and $x$, $y$, and $z$ represent the independent parts of the direction vector. -The zeroth-order spherical harmonic is always required to ensure that the diffuse color can be accurately reconstructed, but higher order spherical harmonics are optional. If higher order spherical harmonics are used, lower order spherical harmonics must also be provided. Each order of spherical harmonics must be fully defined; partial definitions are not allowed. +The zeroth-order spherical harmonic is always required to ensure that the diffuse color can be accurately reconstructed, but higher order spherical harmonics are optional. If higher order spherical harmonics are used, lower order spherical harmonics MUST also be provided. Each order of spherical harmonics MUST be fully defined; partial definitions are not allowed. -Extensions extending this extension may define alternative lighting methods, have specific requirements for handling compression, or define different spherical harmonics handling. +Extensions extending this extension MAY define alternative lighting methods, have specific requirements for handling compression, or define different spherical harmonics handling. See [Appendix A: Spherical Harmonics Reference](#appendix-a-spherical-harmonics-reference) for an easy reference of the spherical harmonic basis functions and normalization constants. -*Non-normative Note: For training software and exporters, it is recommended that the Gaussians are trained within the [glTF coordinate system](https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#coordinate-system-and-units) when targeting glTF. Otherwise, when converting pretrained data from other coordinate systems into the [glTF coordinate system](https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#coordinate-system-and-units), the positions, quaternions and spherical harmonics must be properly rotated.* +> [!NOTE] +> For training software and exporters, it is recommended that the Gaussians are trained within the [glTF coordinate system](https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html#coordinate-system-and-units) when targeting glTF. Otherwise, when converting pre-trained data from other coordinate systems into the glTF coordinate system, the positions, quaternions and spherical harmonics will have to be rotated on load. ## glTF JSON Example -Partial glTF JSON example shown below including optional attributes and properties. This extension only affects any `primitive` nodes containing 3D Gaussian splat data. - -```json -"meshes": [{ - "primitives": [{ - "attributes": { - "POSITION": 0, - "COLOR_0": 1, - "KHR_gaussian_splatting:SCALE": 2, - "KHR_gaussian_splatting:ROTATION": 3, - "KHR_gaussian_splatting:OPACITY": 4, - "KHR_gaussian_splatting:SH_DEGREE_0_COEF_0": 5, - "KHR_gaussian_splatting:SH_DEGREE_1_COEF_0": 6, - "KHR_gaussian_splatting:SH_DEGREE_1_COEF_1": 7, - "KHR_gaussian_splatting:SH_DEGREE_1_COEF_2": 8 - }, - "mode": 0, - "extensions": { - "KHR_gaussian_splatting": { - "kernel": "ellipse", - "colorSpace": "srgb_rec709_display", - "sortingMethod": "cameraDistance", - "projection": "perspective" - } - } - }] -}] -``` - -## Extension Properties - -### Kernel - -Gaussian splats can have a variety of shapes and this has the potential to change over time. The `kernel` property is a required property that provides an indication to the renderer the properties of the kernel used. Renderers are free to ignore any values they do not recognize. - -Additional kernel types can be added over time by supplying an extension that defines an alternative shape definition and parameters. - -| Kernel Type | Description | -| --- | --- | -| ellipse | A 2D ellipse kernel used to project an ellipsoid shape in 3D space. | +Partial glTF JSON example shown below including optional attributes and properties. ```json -"meshes": [{ - "primitives": [{ - // snip... - "extensions": { - "KHR_gaussian_splatting": { - "kernel": "ellipse" +{ + "meshes": [{ + "primitives": [{ + "attributes": { + "POSITION": 0, + "COLOR_0": 1, + "KHR_gaussian_splatting:SCALE": 2, + "KHR_gaussian_splatting:ROTATION": 3, + "KHR_gaussian_splatting:OPACITY": 4, + "KHR_gaussian_splatting:SH_DEGREE_0_COEF_0": 5, + "KHR_gaussian_splatting:SH_DEGREE_1_COEF_0": 6, + "KHR_gaussian_splatting:SH_DEGREE_1_COEF_1": 7, + "KHR_gaussian_splatting:SH_DEGREE_1_COEF_2": 8 + }, + "mode": 0, + "extensions": { + "KHR_gaussian_splatting": { + "kernel": "ellipse", + "colorSpace": "srgb_rec709_display", + "sortingMethod": "cameraDistance", + "projection": "perspective" + } } - } + }] }] -}] +} ``` -#### Ellipse Kernel - -A 2D `ellipse` kernel type is often used to represent 3D Gaussian splats in an ellipsoid shape based on the kernel defined in [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/). This simple type contains no parameters. This is the shape used by the reference renderer implementations for 3D Gaussian splatting. Following the original reference implementation this kernel assumes a _3σ_ cut-off (Mahalanobis distance of 3 units) for correct rendering. - -The mean vector for the Gaussian splat is provided by the position of the mesh primitive. This defines the center of the Gaussian splat ellipsoid in global space. - -The opacity of a Gaussian splat is defined by the `KHR_gaussian_splatting:OPACITY` attribute for this kernel. It stores a normalized value between _0.0_ (transparent) and _1.0_ (opaque). A sigmoid activation function applied during training ensures the value stays within this range. Out-of-range values are invalid. This guarantees that renderers can use the stored opacity directly for alpha blending without any extra processing. - -The scale (`KHR_gaussian_splatting:SCALE`) and rotation (`KHR_gaussian_splatting:ROTATION`) attributes define the size and orientation of the ellipsoid in 3D space. These attributes represent the covariance matrix of the Gaussian in a factored form. The scale attribute values correspond to the spread of the Gaussian along its local principal axes and the rotation attribute values correspond to the orientation of those axes in global space. - -`KHR_gaussian_splatting:SCALE` is stored in log-space, so the actual scale along each principal axis is computed as `exp(scale)`. This allows for representing a wide range of scales while maintaining numerical stability. - -`KHR_gaussian_splatting:ROTATION` is stored as a unit quaternion in the order (x, y, z, w), where `w` is the scalar component. This quaternion represents the rotation from the local space of the Gaussian to global space. - -Together, the scale and rotation can be used to reconstruct the full covariance matrix of the Gaussian splat for rendering purposes. Combined with the position attribute, these values define the identity and shape of the ellipsoid in 3D space. - -More details on how to interpret these attributes for rendering can be found in the [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/) paper. - -See [Appendix A: Sample rendering with the base Ellipse Kernel and Spherical Harmonics](#appendix-a-sample-rendering-with-the-base-ellipse-kernel-and-spherical-harmonics) for more details on how to properly implement the ellipse kernel used by this extension. - -### Color Space - -The `colorSpace` property is a required property that specifies the color space of the 3D Gaussian Splat when spherical harmonics are being used for the lighting. The color space is typically determined by the training process for the splats. This color space value only applies to the 3D Gaussian splatting data and does not affect any other color data in the glTF. +## Fallback Behavior -Unless specified otherwise by additional extensions, color space information refers to the reconstructed splat color values, therefore splat reconstruction and alpha blending must be performed on the attribute values as-is, before any color gamut or transfer function conversions. +To support fallback functionality, the `COLOR_0` attribute semantic from the base glTF specification MAY be used to provide the diffuse color of the 3D Gaussian splat. This allows renderers to color the points in the sparse point cloud when 3D Gaussian splatting is not supported by a renderer. The value of `COLOR_0` is derived by multiplying the 3 diffuse color components of the 3D Gaussian splat with the constant zeroth-order spherical harmonic ($l$ = 0) for the RGB channels. The alpha channel SHOULD contain the opacity of the splat. -Additional values can be added over time by defining extensions to add new color spaces. See the section, [Extending the Base Extension](#extending-the-base-extension), for more information. - -#### Available Color Spaces - -| Color Space | Description | -| --- | --- | -| srgb_rec709_display | BT.709 sRGB (display-referred) color space. | -| lin_rec709_display | BT.709 linear (display-referred) color space. | - -*Non-normative Note: The string values for colorspace follow the color space identification pattern recommended by the ASWF Color Interoperability Forum (CIF). See: [ASWF Color Interoperability Forum](https://lf-aswf.atlassian.net/wiki/spaces/CIF/overview)* - -### Projection - -The `projection` property is an optional property that specifies how the Gaussians should be projected onto the kernel shape. This is typically provided by the training process for the splats. This property is meant to be extended in the future as new projections become standardized within the community. - -This base extension defines a single projection method, `perspective`, which is the default value. This keeps the behavior consistent with the original 3D Gaussian splatting paper. - -_Non-normative Note: See [the original 3D Gaussian Splatting paper](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/) appendix A, "Details of Gradiant Computation," for more details on how the perspective projection is computed._ - -Additional values can be added over time by defining extensions to add new projection methods. See the section, [Extending the Base Extension](#extending-the-base-extension), for more information. - -#### Known Projection Methods - -| Projection Method | Description | -| --- | --- | -| perspective | (Default) The typical 3D perspective projection based on scene depth. | - -### Sorting Method - -The `sortingMethod` property is an optional property that specifies how the Gaussian particles should be sorted during the rendering process. This typically is provided by the training process for the splats. This property is meant to be extended in the future as new sorting methods become standardized within the community. - -This base extension defines a single sorting method, `cameraDistance`, which is the default value. This keeps the behavior consistent with the original 3D Gaussian splatting paper. - -Additional values can be added over time by defining extensions to add new sorting methods. See the section, [Extending the Base Extension](#extending-the-base-extension), for more information. - -#### Known Sorting Methods - -| Sorting Method | Description | -| --- | --- | -| cameraDistance | (Default) Sort the splats based on the length of the vector from the splat to the camera origin. | - -## Attributes - -| Splat Data | glTF Attribute | Accessor Type | Component Type | Required | Notes | -| --- | --- | --- | --- | --- | --- | -| Rotation | KHR_gaussian_splatting:ROTATION | VEC4 | _float_
_signed byte_ normalized
_signed short_ normalized | yes | Rotation is a quaternion with `w` as the scalar. (xyzw) | -| Scale | KHR_gaussian_splatting:SCALE | VEC3 | _float_
_signed byte_
_signed byte_ normalized
_signed short_
_signed short_ normalized | yes | | -| Opacity | KHR_gaussian_splatting:OPACITY | SCALAR | _float_
_unsigned byte_ normalized
_unsigned short_ normalized | yes | | -| Spherical Harmonics degree 0 | KHR_gaussian_splatting:SH_DEGREE_0_COEF_0 | VEC3 | _float_ | yes (unless using a different method for lighting) | | -| Spherical Harmonics degree 1 | KHR_gaussian_splatting:SH_DEGREE_1_COEF_n (n = 0 to 2) | VEC3 | _float_ | no (yes if degree 2 or 3 are used) | Packed from lowest order $m$ (-1) to highest (1). | -| Spherical Harmonics degree 2 | KHR_gaussian_splatting:SH_DEGREE_2_COEF_n (n = 0 to 4) | VEC3 | _float_ | no (yes if degree 3 is used) | Packed from lowest order $m$ (-2) to highest (2). | -| Spherical Harmonics degree 3 | KHR_gaussian_splatting:SH_DEGREE_3_COEF_n (n = 0 to 6) | VEC3 | _float_ | no | Packed from lowest order $m$ (-3) to highest (3). | - -### Basic Attributes - -Each 3D Gaussian splat has the following attributes. At minimum the attributes must contain `POSITION`, `KHR_gaussian_splatting:ROTATION`, `KHR_gaussian_splatting:SCALE`, and `KHR_gaussian_splatting:OPACITY`. `POSITION` is defined by the base glTF specification. - -The `KHR_gaussian_splatting:ROTATION` and `KHR_gaussian_splatting:SCALE` attributes support quantized storage using normalized signed `byte` or `short` component types to reduce file size. If quantization is not needed, content creators should use the `float` component type for maximum precision. - -The content of `KHR_gaussian_splatting:OPACITY`, `KHR_gaussian_splatting:ROTATION` and `KHR_gaussian_splatting:SCALE` are defined by their kernel. See the [Ellipse Kernel](#ellipse-kernel) section for more information for information about how these are defined for the default `ellipse` kernel. - -### Spherical Harmonics Attributes - -When spherical harmonics are being used for lighting, the coefficients for the diffuse component must be provided using the `KHR_gaussian_splatting:SH_DEGREE_0_COEF_0` attribute semantic. The zero-order spherical harmonic coefficients are always required to allow for properly handling cases where the diffuse color is not in the _BT.709_ color gamut. The `KHR_gaussian_splatting:SH_DEGREE_ℓ_COEF_n` attributes where ℓ > 0 hold the higher degrees of spherical harmonics data and are not required. If higher degrees of spherical harmonics are used then lower degrees are required implicitly. - -Each increasing degree of spherical harmonics requires more coefficients. At the 1st degree, 3 sets of coefficients are required, increasing to 5 sets for the 2nd degree, and increasing to 7 sets at the 3rd degree. With all 3 degrees, this results in 45 spherical harmonic coefficients stored in the `KHR_gaussian_splatting:SH_DEGREE_ℓ_COEF_n` attributes. Attributes are packed from the lowest order ($m$) to highest for each degree. (i.e. Degree 2 spherical harmonics are packed order ($m$) -2 coefficients, rgb, then order -1 coefficients, rgb, and so forth.) - -Spherical harmonic data is packed in an (r, g, b) format within the VEC3 accessor type. Each coefficient contains 3 values representing the red, green, and blue channels of the spherical harmonic coefficient. Spherical harmonic degrees cannot be partially defined. For example, if any degree 2 spherical harmonics attribute semantics are used, then all degree 2 and degree 1 spherical harmonic coefficients must be provided. - -### Improving Fallback with COLOR_0 - -To support better fallback functionality, the `COLOR_0` attribute semantic from the base glTF specification may be used to provide the diffuse color of the 3D Gaussian splat. This allows renderers to color the points in the sparse point cloud when 3D Gaussian splatting is not supported by a renderer. The value of `COLOR_0` is derived by multiplying the 3 diffuse color components of the 3D Gaussian splat with the constant zeroth-order spherical harmonic (ℓ = 0) for the RGB channels. The alpha channel should contain the opacity of the splat. - -*_Non-normative Note:_* If the spherical harmonics are in the BT.709 gamut, the diffuse color can be computed from the `KHR_gaussian_splatting:SH_DEGREE_0_COEF_0` attribute by multiplying each of the RGB components by the constant spherical harmonic value of _0.282095_. +> [!NOTE] +> If the color space is `"srgb_rec709_display"` or `"lin_rec709_display"`, the diffuse color for the ellipse kernel suitable for `COLOR_0` fallback usage can be computed from the `KHR_gaussian_splatting:SH_DEGREE_0_COEF_0` attribute by multiplying each of the RGB components by the constant spherical harmonic value of _0.282095_, adding _0.5_, and clamping the sum to the $[0, 1]$ range; if the color space is `"srgb_rec709_display"`, the clamped values would have to be decoded from sRGB to linear because the `COLOR_0` attribute contains linear values as per the glTF specification. ## Extending the Base Extension 3D Gaussian splatting is an evolving technology with new techniques and methods being developed over time. To provide a solid foundation for 3D Gaussian splatting in glTF while allowing for future growth and innovation, this extension is designed to be extensible. New kernel types, color spaces, projection methods, and sorting methods can be added over time without requiring changes to the base extension. -Extensions may define additional attributes or custom properties as needed to support new features. Attribute semantics should be prefixed with their respective extension name to avoid naming collisions. Extensions may also define additional values for the `kernel`, `colorSpace`, `projection`, and `sortingMethod` properties. Custom properties should be included in the body of the new extension object. +Extensions MAY define additional attributes or custom properties as needed to support new features. Attribute semantics SHOULD be prefixed with their respective extension name to avoid naming collisions. Extensions MAY also define additional values for the `kernel`, `colorSpace`, `projection`, and `sortingMethod` properties. Custom properties SHOULD be included in the body of the new extension object. -*_Non-normative Note: It is possible to share data between two attributes by using the same accessor index for multiple attribute semantics. This can be useful to optimize the storage of data._* +> [!NOTE] +> It is possible to share data between two attributes by using the same accessor index for multiple attribute semantics. This can be useful to optimize the storage of data. -Compression extensions that operate on 3D Gaussian splatting data should extend this base extension to ensure compatibility. Compression extensions must define how the data can be decoded back into the base 3D Gaussian splatting format defined by this extension, but may also allow optimizations specific to their compression method. (e.g. passing textures or other data directly to the GPU for decoding.) +Compression extensions that operate on 3D Gaussian splatting data SHOULD extend this base extension to ensure compatibility. Compression extensions SHOULD define how the data can be decoded back into the base 3D Gaussian splatting format defined by this extension, but MAY also allow optimizations specific to their compression method, e.g., passing textures or other data directly to the GPU for decoding. -To use an extensions that extends `KHR_gaussian_splatting`, the extension must be included within the `extensions` property of the `KHR_gaussian_splatting` extension object. The extension must also be listed in `extensionsUsed` at the top level of the glTF. +To use an extension that extends `KHR_gaussian_splatting`, the extension MUST be included within the `extensions` property of the `KHR_gaussian_splatting` extension object. The new extension MUST also be listed in `extensionsUsed` at the top level of the glTF. -Extension authors are encouraged to define fallback behaviors for renderers that do not recognize the new extension, but this is not strictly required. If a fallback is not possible, the extension should be listed in `extensionsRequired` to ensure that renderers that do not support the extension do not attempt to render the data incorrectly. +Extension authors are encouraged to define fallback behaviors for renderers that do not recognize the new extension, but this is not strictly required. If a fallback is not possible, the new extension SHOULD also be listed in `extensionsRequired` to ensure that renderers that do not support the extension do not attempt to render the data incorrectly. -#### Example: Adding additional Kernel Types +### Example: Adding additional Kernel Types -*This section is non-normative.* +_This section is non-normative._ In order to add additional kernel types, a new extension should be defined that extends `KHR_gaussian_splatting`. This new extension would define the new kernel type and any parameters it may require. A renderer that recognizes the new kernel type can then use the parameters defined in the new extension to render the splats appropriately. Renderers that do not recognize the new kernel type should fall back to the default `ellipse` type. @@ -497,21 +468,21 @@ The extension must also be listed in `extensionsUsed` at the top level of the gl ## Appendix A: Spherical Harmonics Reference -*This appendix is non-normative and provided for informational purposes only.* +_This appendix is non-normative and provided for informational purposes only._ ### Real Spherical Harmonic Basis Functions -Degrees $0$ through $3$, in cartesian space. Including the [Condon–Shortley phase](https://mathworld.wolfram.com/Condon-ShortleyPhase.html) $(-1)^m$. +Degrees $0$ through $3$, in cartesian space. Including the Condon–Shortley phase $(-1)^m$. ```math -\textbf{Degree 0, ℓ = 0}\\ +\textbf{Degree 0, l = 0}\\ \begin{aligned} Y_{0,0}(θ, φ) &= \frac{1}{2} \sqrt{\frac{1}{\pi}}\\\\ \end{aligned} ``` ```math -\textbf{Degree 1, ℓ = 1}\\ +\textbf{Degree 1, l = 1}\\ \begin{aligned} Y_{1,-1}(θ, φ) &= -\sqrt{\frac{3}{4\pi}} \cdot \frac{y}{r}\\ Y_{1,0}(θ, φ) &= \sqrt{\frac{3}{4\pi}} \cdot \frac{z}{r}\\ @@ -520,7 +491,7 @@ Y_{1,1}(θ, φ) &= -\sqrt{\frac{3}{4\pi}} \cdot \frac{x}{r}\\\\ ``` ```math -\textbf{Degree 2, ℓ = 2}\\ +\textbf{Degree 2, l = 2}\\ \begin{aligned} Y_{2,-2}(θ, φ) &= \frac{1}{2} \sqrt{\frac{15}{\pi}} \cdot \frac{xy}{r^2}\\ Y_{2,-1}(θ, φ) &= -\frac{1}{2} \sqrt{\frac{15}{\pi}} \cdot -\frac{yz}{r^2}\\ @@ -531,7 +502,7 @@ Y_{2,2}(θ, φ) &= \frac{1}{4} \sqrt{\frac{15}{\pi}} \cdot \frac{x^2 - y^2}{r^2} ``` ```math -\textbf{Degree 3, ℓ = 3}\\ +\textbf{Degree 3, l = 3}\\ \begin{aligned} Y_{3,-3}(θ, φ) &= -\frac{1}{4} \sqrt{\frac{35}{2\pi}} \cdot \frac{y(3x^2 - y^2)}{r^3}\\ Y_{3,-2}(θ, φ) &= \frac{1}{2} \sqrt{\frac{105}{\pi}} \cdot \frac{xyz}{r^3}\\ @@ -562,14 +533,13 @@ Y_{3,3}(θ, φ) &= -\frac{1}{4} \sqrt{\frac{35}{2\pi}} \cdot \frac{x(x^2 - 3y^2) ## Known Implementations -*TODO: Add known implementations before final ratification.* - -*NOTE: If you are a developer of a glTF renderer that implements this extension, please open an issue in the glTF GitHub repository to have your implementation listed here.* +_TODO: Add known implementations before final ratification._ ## Resources - [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/) -- [Gaussian Splatting Algorithm Details](https://github.com/joeyan/gaussian_splatting/blob/main/MATH.md) +- [Condon–Shortley phase](https://mathworld.wolfram.com/Condon-ShortleyPhase.html) +- [ASWF Color Space Encodings for Displays](https://github.com/AcademySoftwareFoundation/ColorInterop/blob/main/Recommendations/02_DisplayColorSpaces/DisplayColorSpaces.md) ## Full Khronos Copyright Statement @@ -617,4 +587,3 @@ must be obtained separately and may involve royalty payments. Khronos® is a registered trademark, and glTF™ is a trademark of The Khronos Group Inc. All other product names, trademarks, and/or company names are used solely for identification and belong to their respective owners. - diff --git a/extensions/2.0/Khronos/KHR_gaussian_splatting/schema/mesh.primitive.KHR_gaussian_splatting.schema.json b/extensions/2.0/Khronos/KHR_gaussian_splatting/schema/mesh.primitive.KHR_gaussian_splatting.schema.json index 8cdc3c6759..f0c9711175 100644 --- a/extensions/2.0/Khronos/KHR_gaussian_splatting/schema/mesh.primitive.KHR_gaussian_splatting.schema.json +++ b/extensions/2.0/Khronos/KHR_gaussian_splatting/schema/mesh.primitive.KHR_gaussian_splatting.schema.json @@ -11,7 +11,7 @@ "properties": { "kernel": { "type": "string", - "description": "Property specifying parameters regarding the kernel used to generate the Gaussians.", + "description": "The kernel used to generate the Gaussians.", "anyOf": [ { "const": "ellipse" @@ -23,7 +23,7 @@ }, "colorSpace": { "type": "string", - "description": "Property specifying the color space of the spherical harmonics.", + "description": "The color space of the reconstructed color values.", "anyOf": [ { "const": "srgb_rec709_display" @@ -38,7 +38,7 @@ }, "projection": { "type": "string", - "description": "Optional property specifying how to project the Gaussians to achieve a perspective correct value. This property defaults to perspective.", + "description": "The projection method for rendering the Gaussians.", "default": "perspective", "anyOf": [ { @@ -51,7 +51,7 @@ }, "sortingMethod": { "type": "string", - "description": "Optional property specifying how to sort the Gaussians during rendering. This property defaults to cameraDistance.", + "description": "The sorting method for rendering the Gaussians.", "default": "cameraDistance", "anyOf": [ {