Specifying the data format #42
Description
The README.md contains some high-level information about what is stored in an SPZ file. But right now, this is not detailed enough for people to write an encoder or decoder based on that description. It is not a specification, in that manner. I'll try to summarize a few points of which I think that they require further clarification.
Coordinate systems
There is an issue about this at #14 . Some details about the coordinate system have been added in a recent PR, by inserting this section. I'm not sure whether the issue should be closed, or whether there are further details that should be added.
Beyond SPZ itself:
Apparently, there is a certain fragmentation when it comes to coordinate system conventions for splats. A recent comment in another issue lists a few conventions that can be found in the wild.
The KHR_spz_gaussian_splats_compression PR currently says
When compressing or decompressing the SPZ data to be stored within the glTF, you must specify a Left-Up-Front (LUF) coordinate system in the SPZ
PackOptionsorUnpackOptionswithin the SPZ library.
Assuming that SPZ data always is in 'RUB' convention, it is not clear whether any coordinate system conversion is required when SPZ is stored within glTF. Specifically: I have some SPZ with RUB convention. Can I use that SPZ directly as the bufferView/buffer data in this extension, or do I have to perform a coordinate system conversion? (Ideally, that should not be necessary to do such conversions at runtime, but maybe there are strong reasons to do such a conversion even in the context of a last-mile delivery format)
In any case, the spz-loader npm package seems to call the rotate180DegAboutX function by default. The rotate180DegAboutX function is just one (fixed) coordinate system conversion, of which it is not clear where and why it should be applied or not. And more importantly: It doesn't look like it's possible to define any other coordinate system conversion in that library! (One could bring this up as an issue in that repository)
Alpha/Opacity values
The "Alphas" section currently says
Alphas are represented as 8-bit unsigned integers.
This does not seem to be the case. From my understanding, the values that are stored in the "alphas" are values in [0, 255], but they are not classical alpha values where this range is mapped to [0,1]. Instead, they are undergoing the sigmoid/inverseSigmoid translation. This needs to be pointed out explicitly.
(Whether something should be called "alpha" when it's not really an "alpha" value is another question...)
Related: #22
Colors
Similar to the above: The "Colors" section currently says
Colors are stored as
(r, g, b)values, where each color component is represented as an unsigned 8-bit integer.
These are values in [0,255]. But they are not (r, g, b) color components, and not mapped to color values in [0,1]. They seem to be encodings of the spherical harmonics coefficients for dimension 0. If this was the case, then the values in [0,255] should be mapped to the range [-c,c], with c = 1.0 /(Math.sqrt(1.0 / Math.PI) * 0.5) (roughly 0.5/0.283 ~= 1.764). But apparently SPZ uses some colorScale, which (I think) causes these values to be mapped to a range [-3.3, 3.3]. Iff that "colorScale" has a mathematical justification, then this should be made more explicit. (If it does not have a mathematical justification, this may be more tricky...)