Integrate Vendor-specific SPZ Extensions, WASM Bindings, and CMake Improvements (Backward-Compatible) #67

raymondyfei · 2025-12-12T02:08:14Z

Summary

This PR merges the adobe-public branch into main and introduces several non-breaking, opt-in improvements for SPZ users:

Adobe’s extensions for SH quantization and safe-orbit camera metadata
A new Emscripten/WASM binding layer + TypeScript definitions
More modular CMake options for Python, Tools, and WASM builds
More robust Zlib handling (system-first, fallback to vendored)
Updated Python packaging metadata and wheel build flow

All changes are fully backward-compatible with existing pipelines and SPZ readers.

What’s New

1. CMake & Build System Improvements

Adds:
- SPZ_BUILD_PYTHON_BINDINGS
- SPZ_BUILD_TOOLS
- SPZ_BUILD_WASM
- SPZ_USE_EMSCRIPTEN_ZLIB
Emscripten builds:
- Automatically disable Python + CLI
- Automatically enable WASM
Zlib handling:
1. Prefer system ZLIB::ZLIB
2. Fall back to vendored zlib 1.3.1
When vendored zlib is used, install targets are disabled to avoid exporting non-exportable deps.

This makes builds predictable across Linux/macOS/Windows/Emscripten and prevents accidental install breakage.

2. SPZ Core Enhancements

We add an optional, vendor-specific extension system to SPZ, which can be identified through the flag in the header. Currently, it supports only two extensions. Nevertheless, more vendor-specific extensions can be similarly added in the future. The two new extensions are listed as following:

SH Quantization Extension

Optional metadata describing Adobe’s SH coefficient quantization:

sh1Bits
shRestBits
shMin
shMax

Safe Orbit Camera Extension

Optional metadata describing camera constraints for orbit controls:

safeOrbitElevationMin
safeOrbitElevationMax
safeOrbitRadiusMin

These extensions attach to the SPZ extension chain and remain invisible to legacy consumers. They are strictly additive and safe to ignore unless explicitly used.

3. WebAssembly / TypeScript Bindings

Introduces a WebAssembly runtime:

EmGaussianCloud with JS TypedArray views
loadSpzFromBuffer and saveSpzToBuffer
Adobe extension types exposed to JS
New TypeScript declarations: spz.d.ts

These modifications enable browser-based SPZ debugging/visualization with no impact on native runtimes.

4. Python Packaging Updates

Updated pyproject.toml with:
- SPZ_BUILD_PYTHON_BINDINGS
- Optional tests dependencies
- Cleaner macOS wheel settings

These modifications simplify Python wheel builds and align with modern scikit-build-core conventions.

Backward Compatibility

✔ Existing SPZ files load without changes
✔ Adobe extensions are optional and ignored by default
✔ No change to baseline SPZ serialization
✔ Build behavior unchanged unless new flags are enabled

Supplementary Info of this PR

Also, here are some details for the new compression options. We found sh_rest_bits=5 (up from 4 in the previous spec) with minmax scaling enabled to be the best setting, though it will slightly inflate spz size compared to the previous version's settings.

Configurable SH bits test

Input file: adobe_backpack_float32.ply (129.3 MB)
Points: 380,902
SH Degree: 4
SH Range: [-0.52, 0.96]

Without Min/Max Scaling

`sh_rest_bits`	SPZ Size	Compression Ratio	SH Rest MSE
3	7.1 MB	18.25x	0.001870
4 (default)	8.7 MB	14.94x	0.001026
5	10.6 MB	12.20x	0.000322
6	12.9 MB	10.06x	0.000081
7	15.5 MB	8.37x	0.000020
8	18.6 MB	6.97x	0.000021

With Min/Max Scaling

`sh_rest_bits`	SPZ Size	Compression Ratio	SH Rest MSE
3	8.2 MB	15.78x	0.002250
4	10.1 MB	12.79x	0.000763
5	11.7 MB	11.03x	0.000188
6	14.1 MB	9.15x	0.000053
7	16.9 MB	7.66x	0.000020
8	20.4 MB	6.35x	0.000003

Min/Max Scaled Qualitative Comparison

3 bits	4 bits	5 bits	6 bits

Recommendations:

Best compression: 3-4 bits without scaling (~15-18x ratio)
Best quality/size balance: 5-6 bits with scaling (~10-11x ratio, very low MSE)
Maximum quality: 8 bits with scaling (MSE essentially zero)

Caveats and notes:

Without minmax scaling, values outside of [-1, 1] will be clipped and lost completely
For splats with SH range between [-1, 1], we see no visual difference above 5 bits

skywolf829 · 2025-12-12T02:16:21Z

Seems we might need to update the github workflow to run the pytests? Or am i reading the CI wrong and they ran successfully already

raymondyfei · 2025-12-12T02:18:04Z

Seems we might need to update the github workflow to run the pytests? Or am i reading the CI wrong and they ran successfully already

I think they ran successfully already, which were still done through pytest.

spsDrop · 2026-01-14T07:24:37Z

The extensibility extensions stuff seems extremely useful for SPZ. How are they backwards compatible? Do you mean older SPZ loaders will be able to load files with these extensions?

raymondyfei · 2026-01-14T22:56:39Z

The extensibility extensions stuff seems extremely useful for SPZ. How are they backwards compatible? Do you mean older SPZ loaders will be able to load files with these extensions?

Older SPZ loaders won't load the files with these extensions used/enabled. But they can load the files generated with the new version of SPZ library without these extensions enabled. By default, these extensions are disabled and henceforth one may use this SPZ library as before, without caring about the changes here.

ProjitB

Thanks for adding more tests!
I think the idea of extending the sh bits is great but I do have more questions / concerns on how to make this a bit more modular. Perhaps we can setup a call to discuss a bit further on how the extensions are being added.

My concerns are broadly regarding:

keeping the core of the library simple, and thus pushing extensions logic further into the extensions code, if this is how we decide to support the behavior. I'd prefer more strict enables around the loading of spzs and a more explicit structure around the enablement of extensions.
simple, fixed size header format, that can be decoded without reading the rest of the gaussians (even though the header is part of the compression..)

If maintaining backwards compatibility adds a lot of code complexity, i think we should discuss with the larger community, the scope for incrementing the data version, and pushing some of the quantization related features into the core library rather than having it as an extension.

Sorry for the delay in reviewing this!

ProjitB · 2025-12-20T00:07:17Z

CMakeLists.txt

+    include(FetchContent)
+    FetchContent_Declare(
+      zlib
+      URL https://zlib.net/zlib-1.3.1.tar.gz


i don't think we've so far specified a version of zlib that we pin to. It does seem prudent to do so though..
Thanks for adding this. But perhaps we also make a note of it in our readme?

Sure! Will do!

ProjitB · 2026-01-05T21:59:45Z

README.md


 ### Header

+**Version 2 (current):**


This should be Version 3 right?

Yes! Nice catch.

ProjitB · 2026-01-05T22:21:37Z

src/cc/load-spz.h

+    float shMin = -1.0f, float shMax = 1.0f) const;
 };

 // Represents a full splat with lower precision. Each splat has at most 64 bytes, although splats


we probably have to update this comment?

ProjitB · 2026-01-27T07:04:30Z

README.md

 ## File Format

-The .spz format is a gzipped stream of data consisting of a 16-byte header followed by the
+The .spz format is a gzipped stream of data consisting of a variable-size header followed by the


We may want to more carefully consider the implications of variable header sizes..Is this actually backwards compatible? Any version of spz that changes the header size would break any renderer that tries to decode based on the header bytes right?

This is related to how we support the extensions. Currently we put extensions data immediately after the header. Another option is to use a fixed-size header and an offset to point to any extensions. We may discuss this further in a chat.

ProjitB · 2026-01-27T07:07:15Z

src/cc/load-spz.cc


-// Quantizes to 8 bits, the round to nearest bucket center. 0 always maps to a bucket center.
-uint8_t quantizeSH(float x, int32_t bucketSize) {
+// Quantizes to 8 bits using min/max scaling, then round to nearest bucket center. 0 always maps to a bucket center.


is this min/max scaling actually valuable in the average case? Larger splats have millions of gaussians, in which i guess the sh values would vary throughout the standard range. The benefits of this would only be in particular cases. I was wondering if it's worth the added complexity.

It depends on how the splat was created. In both optimization and feedforward models that generate splats, SH values are not always constrained, but are only learned via some render loss. Unless theres a regularizer on the SH values, we've seen model output (mostly from inference on transformer models) have SH ranges well outside of [-1, 1], prompting the minmax scaling feature here.

For what its worth, standard gsplat optimization pipelines tend to have SH in [-1,1], but with more generative 3D models popping up, it might be helpful for those cases?

I'm okay to go either way on this! We can keep it simple and remove minmax, and re-add it in the future if deemed necessary.

ProjitB · 2026-01-27T07:10:19Z

src/cc/load-spz.cc

  return success;
 }

+void packQuaternionFirstThree(uint8_t r[3], const float rotation[4], const CoordinateConverter& c) {


Is the reason for having first three that in the less compressed cases (ie more sh bits), we store the quaternions at the highest precision possible?

This is for recovering the compatibility with the older SPZ loaders that can process only V2 files. These loaders assume the quaternions are packed with only the first three components and we want our library to still be able to save such files.

ProjitB · 2026-01-27T07:12:20Z

src/cc/load-spz.cc

+  float minSH = -1.0f;
+  float maxSH = 1.0f;
+  if (o.sh1Bits != DEFAULT_SH1_BITS || o.shRestBits != DEFAULT_SH_REST_BITS || o.enableSHMinMaxScaling) {
+    auto ext = std::make_shared<SpzExtensionSHQuantizationAdobe>();


it seems like we're implicitly using this extension. Shouldn't we skip any/all computation if the extension isn't enabled?

Hmm...make sense. We may add an option to explicitly indicate whether to use extensions instead of guessing from the options.

ProjitB · 2026-01-27T07:17:59Z

src/cc/load-spz.cc

+  } else {
+    // Backward compatibility for older versions.
+    packed.usesQuaternionSmallestThree = false;
+    packed.rotations.resize(numPoints * 3);


the resize should be gated by the packing condition right? rather than the verison?

For V2, it doesn't seem there's thsupport for the smallest-three packing condition so I assume we'd turn off the usesQuaternionSmallestThree by default. But sure, we can put the force-off of usesQuaternionSmallestThree during initialization of this variable and gate the code here with usesQuaternionSmallestThree.

Raymond Fei and others added 6 commits August 22, 2025 10:44

Sync from corp/adobe-v2.0.0-public

5907cb2

Update pyproject.toml

1b4cf9c

Update pyproject.toml

72661b5

Merge branch 'main' into adobe-public

4414042

Merge main and clean up

faffa6e

final clean up

ec5faf9

raymondyfei requested review from ProjitB, azadbal and daniel-niantic-spatial as code owners December 12, 2025 02:08

raymondyfei requested review from nbutko and skywolf829 December 12, 2025 02:08

ProjitB reviewed Jan 27, 2026

View reviewed changes

Integrate Vendor-specific SPZ Extensions, WASM Bindings, and CMake Improvements (Backward-Compatible) #67

Are you sure you want to change the base?

Integrate Vendor-specific SPZ Extensions, WASM Bindings, and CMake Improvements (Backward-Compatible) #67

Uh oh!

Conversation

raymondyfei commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What’s New

1. CMake & Build System Improvements

2. SPZ Core Enhancements

SH Quantization Extension

Safe Orbit Camera Extension

3. WebAssembly / TypeScript Bindings

4. Python Packaging Updates

Backward Compatibility

Supplementary Info of this PR

Configurable SH bits test

Without Min/Max Scaling

With Min/Max Scaling

Min/Max Scaled Qualitative Comparison

Recommendations:

Caveats and notes:

Uh oh!

skywolf829 commented Dec 12, 2025

Uh oh!

raymondyfei commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

spsDrop commented Jan 14, 2026

Uh oh!

raymondyfei commented Jan 14, 2026

Uh oh!

ProjitB left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

raymondyfei Jan 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

raymondyfei commented Dec 12, 2025 •

edited

Loading

raymondyfei commented Dec 12, 2025 •

edited

Loading

raymondyfei Jan 27, 2026 •

edited

Loading