Skip to content

Attempt to reduce linking overhead (Scala Native) and improve pipeline performance#69

Merged
davesmith00000 merged 25 commits intomainfrom
speed-up-the-pipeline
Mar 17, 2026
Merged

Attempt to reduce linking overhead (Scala Native) and improve pipeline performance#69
davesmith00000 merged 25 commits intomainfrom
speed-up-the-pipeline

Conversation

@davesmith00000
Copy link
Member

This is going to be another stage in the work to make Scala Native viable. Currently the JVM and JS compilation of the pipeline works just fine, since the don't do full ahead-of-time linking. Unfortunately Scala Native does do that, and blows up.

The general plan here is quite simple:

  1. Refactor, reorganise, and tidy up the indigo-render-pipeline module.
  2. Make it procedural.
  • Switch the mutable structures to reduce allocations.
  • Remove as many lambdas as possible.
  • Reduce as far as possible, matching on the SceneNode types (get as concrete as possible as fast as possible).

@davesmith00000 davesmith00000 self-assigned this Mar 10, 2026
@davesmith00000 davesmith00000 force-pushed the speed-up-the-pipeline branch from d4fb36d to 4c45626 Compare March 14, 2026 10:17
@davesmith00000 davesmith00000 requested a review from Copilot March 17, 2026 21:53
@davesmith00000 davesmith00000 marked this pull request as ready for review March 17, 2026 21:53
@davesmith00000 davesmith00000 changed the title WIP: Speeding up the pipeline Attempt to reduce linking overhead (Scala Native) and improve performance Mar 17, 2026
@davesmith00000 davesmith00000 changed the title Attempt to reduce linking overhead (Scala Native) and improve performance Attempt to reduce linking overhead (Scala Native) and improve pipeline performance Mar 17, 2026
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the Indigo render pipeline to be more procedural and allocation-conscious (motivated by Scala Native’s AOT linking costs), while reorganizing pipeline types/utilities into clearer subpackages and introducing a visitor-based traversal for SceneNodes.

Changes:

  • Adds SceneNodeVisitor + SceneNode.accept and implements accept across core scenegraph node types.
  • Reorganizes render-pipeline packages (assets/, datatypes/, sceneprocessing/, displayprocessing/) and updates imports across renderer/platform code.
  • Introduces new procedural utilities/builders (e.g., light UBO builder, UBO packing, layer compaction) and adds targeted MUnit test coverage.

Reviewed changes

Copilot reviewed 65 out of 65 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
indigo-scenegraph/src/indigo/scenegraph/Text.scala Adds accept implementation for visitor traversal.
indigo-scenegraph/src/indigo/scenegraph/Sprite.scala Adds accept implementation for visitor traversal.
indigo-scenegraph/src/indigo/scenegraph/Shape.scala Adds accept implementations for concrete shape variants.
indigo-scenegraph/src/indigo/scenegraph/SceneNodeVisitor.scala Introduces visitor interface for scenegraph node processing.
indigo-scenegraph/src/indigo/scenegraph/SceneNode.scala Adds accept to SceneNode and default visitor hook for EntityNode.
indigo-scenegraph/src/indigo/scenegraph/Mutants.scala Adds accept implementation for visitor traversal.
indigo-scenegraph/src/indigo/scenegraph/Group.scala Adds accept implementation for visitor traversal.
indigo-scenegraph/src/indigo/scenegraph/Graphic.scala Adds accept implementation for visitor traversal.
indigo-scenegraph/src/indigo/scenegraph/CloneTiles.scala Adds accept implementation for visitor traversal.
indigo-scenegraph/src/indigo/scenegraph/CloneBatch.scala Adds accept implementation for visitor traversal.
indigo-scenegraph/src/indigo/scenegraph/Clip.scala Adds accept implementation for visitor traversal.
indigo-scenegraph/src/indigo/scenegraph/BlankEntity.scala Adds accept implementation for visitor traversal.
indigo-render/src/indigo/render/Renderer.scala Updates imports to new render-pipeline datatypes package.
indigo-render/src-js/indigo/render/webgl2/TextureLookupResult.scala Updates AtlasId import to new assets package.
indigo-render/src-js/indigo/render/webgl2/RendererWebGL2.scala Updates imports to new render-pipeline datatypes package.
indigo-render/src-js/indigo/render/webgl2/LoadedTextureAsset.scala Updates AtlasId import to new assets package.
indigo-render/src-js/indigo/render/webgl2/LayerRenderer.scala Updates display type imports to new datatypes/assets packages.
indigo-render/src-js/indigo/render/webgl2/LayerMergeRenderer.scala Updates uniform-data type import to new datatypes package.
indigo-render-pipeline/test/src/indigo/render/pipeline/sceneprocessing/utils/CompactLayersTests.scala Adds tests for new layer-compaction utility.
indigo-render-pipeline/test/src/indigo/render/pipeline/sceneprocessing/utils/BuildLightingDataTests.scala Adds tests for new light-UBO builder logic and edge cases.
indigo-render-pipeline/test/src/indigo/render/pipeline/sceneprocessing/SceneProcessorTests.scala Adds tests for new procedural SceneProcessor.makeDisplayLayers.
indigo-render-pipeline/test/src/indigo/render/pipeline/displayprocessing/utils/TextConversionTests.scala Adds tests for text character layout logic.
indigo-render-pipeline/test/src/indigo/render/pipeline/displayprocessing/PackUBOsTests.scala Adds detailed std140 UBO packing tests.
indigo-render-pipeline/test/src/indigo/render/pipeline/displayprocessing/DisplayObjectConversionsTests.scala Adds tests for new DisplayObjectConversions behavior.
indigo-render-pipeline/test/src/indigo/render/pipeline/datatypes/SpriteSheetFrameTests.scala Moves + expands SpriteSheetFrame tests for new datatypes location.
indigo-render-pipeline/test/src/indigo/render/pipeline/SceneProcessorTests.scala Removes legacy scene processor tests (split into new focused suites).
indigo-render-pipeline/test/src/indigo/render/pipeline/DisplayObjectConversionsTests.scala Removes legacy conversions tests (split into new focused suites).
indigo-render-pipeline/test/src-jvm/indigo/render/pipeline/displayprocessing/utils/ToCacheKeySyntaxTests.scala Adds JVM test for shader cache-key generation.
indigo-render-pipeline/test/src-js/indigo/render/pipeline/displayprocessing/utils/ToCacheKeySyntaxTests.scala Adds JS test for shader cache-key generation.
indigo-render-pipeline/src/indigo/render/pipeline/sceneprocessing/utils/MergeUniformData.scala Extracts shader→uniform-data merge into utility.
indigo-render-pipeline/src/indigo/render/pipeline/sceneprocessing/utils/CompactLayers.scala Extracts layer compaction into utility.
indigo-render-pipeline/src/indigo/render/pipeline/sceneprocessing/utils/BuildLightingData.scala Adds procedural light packing into std140-friendly float array.
indigo-render-pipeline/src/indigo/render/pipeline/sceneprocessing/SceneProcessor.scala Introduces new procedural scene processing pipeline entrypoint.
indigo-render-pipeline/src/indigo/render/pipeline/sceneprocessing/LightDataBuilder.scala Adds mutable-array builder for lighting UBO data.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/utils/ToCacheKeySyntax.scala Introduces shader-data cache-key builder.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/utils/TextureLookups.scala Extracts cached texture lookup/offset helpers.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/utils/TextConversion.scala Adds procedural text→display conversion + caching.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/utils/SpriteConversion.scala Adds procedural sprite→display conversion + caching.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/utils/ShapeConversion.scala Adds procedural shape→display conversion.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/utils/PackUBOs.scala Adds procedural std140 UBO packing implementation.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/utils/MutantConversion.scala Adds procedural mutants→display conversion.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/utils/GroupConversion.scala Adds procedural group→matrix conversion.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/utils/GraphicConversion.scala Adds procedural graphic→display conversion + caching.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/utils/EntityNodeConversion.scala Adds procedural entity-node→display conversion + caching.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/utils/ConversionHelpers.scala Centralizes shader uniform-block conversion to display uniform data.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/utils/CloneTilesConversion.scala Adds cached conversion for clone-tiles display entities.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/utils/CloneBatchConversion.scala Adds cached conversion for clone-batch display entities.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/DisplayObjectConversions.scala Introduces frame/asset scoped caches + visitor-based conversion pipeline.
indigo-render-pipeline/src/indigo/render/pipeline/displayprocessing/DisplayObjectConversionVisitor.scala Implements SceneNodeVisitor for display conversion.
indigo-render-pipeline/src/indigo/render/pipeline/display/SpriteSheetFrame.scala Removes legacy SpriteSheetFrame location.
indigo-render-pipeline/src/indigo/render/pipeline/datatypes/SpriteSheetFrame.scala Adds new datatypes SpriteSheetFrame implementation.
indigo-render-pipeline/src/indigo/render/pipeline/datatypes/ProcessedSceneData.scala Moves ProcessedSceneData to datatypes package.
indigo-render-pipeline/src/indigo/render/pipeline/datatypes/DisplayLayer.scala Moves DisplayLayer to datatypes package.
indigo-render-pipeline/src/indigo/render/pipeline/datatypes/DisplayEntity.scala Refactors display entity structures and constructors.
indigo-render-pipeline/src/indigo/render/pipeline/assets/TextureRefAndOffset.scala Moves TextureRefAndOffset to assets package.
indigo-render-pipeline/src/indigo/render/pipeline/assets/AtlasId.scala Moves AtlasId to assets package.
indigo-render-pipeline/src/indigo/render/pipeline/assets/AssetMapping.scala Moves AssetMapping to assets package.
indigo-render-pipeline/src/indigo/render/pipeline/SceneProcessor.scala Removes legacy SceneProcessor implementation.
indigo-render-pipeline/src/indigo/render/pipeline/DisplayObjectConversions.scala Removes legacy DisplayObjectConversions implementation.
indigo-platform/test/src-js/indigo/platform/assets/TextureAtlasTests.scala Updates AtlasId import to new assets package.
indigo-platform/src-js/indigo/platform/gameengine/GameLoop.scala Updates SceneProcessor import to new sceneprocessing package.
indigo-platform/src-js/indigo/platform/gameengine/GameEngine.scala Updates AssetMapping/SceneProcessor imports to new packages.
indigo-platform/src-js/indigo/platform/assets/TextureAtlas.scala Updates AtlasId import to new assets package.
indigo-platform/src-js/indigo/platform/JsPlatform.scala Updates pipeline asset type imports to new assets package.
indigo-core/src/indigo/core/utils/QuickCache.scala Updates Scala 3 using/given syntax for cache APIs.
Comments suppressed due to low confidence (2)

indigo-render-pipeline/src/indigo/render/pipeline/datatypes/DisplayEntity.scala:28

  • DisplayGroup and DisplayTextLetters were changed from case classes to plain classes, but the code still calls DisplayGroup(...) / DisplayTextLetters(...) (e.g. in the empty values and elsewhere). Plain classes don’t get an auto-generated apply, so this won’t compile. Either switch these back to final case class, or add companion apply methods (or update all call sites to use new ...).
    indigo-render-pipeline/src/indigo/render/pipeline/datatypes/DisplayEntity.scala:141
  • DisplayObjectUniformData is now a plain class, but the rest of the PR constructs it as DisplayObjectUniformData(...) (e.g. in ConversionHelpers, MergeUniformData, etc.). This won’t compile unless you either make it a case class, add a companion object DisplayObjectUniformData with an apply, or update call sites to use new DisplayObjectUniformData(...).

@davesmith00000 davesmith00000 merged commit 687aea6 into main Mar 17, 2026
7 checks passed
@davesmith00000 davesmith00000 deleted the speed-up-the-pipeline branch March 17, 2026 22:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants