How to Render from a Custom Camera into a Texture

[Nyubug]

Hi!

I'm following up on an older discussion – sorry for opening a new one 😅

The response I marked as the solution was helpful, but it didn’t explain how to render from a custom camera. In aVersionOfReality’s case, they needed to render a specific region (from 0 to 1 on the X and Y axes) using an orthographic camera, but the provided answer didn’t touch on how to actually define and use a different camera for the texture render.

Anyways, I’ve made some progress based on your advice.

So far, I’ve:

• Started with a minipipeline
• Created a custom render target
• Bound it to a TextureShaderResource
• Successfully used it in my shader

However, I’m stuck on how to render this texture using a custom camera of my choosing. As far as I can tell, Malt only provides one camera (either scene_eval.camera or context.region_data.view_matrix) through MaltRenderEngine.py, and the Scene class supports only a single camera as well.

I’m not sure how to add another camera without interfering with the main rendering. Would I need to extend the Scene class to support more cameras? Or modify MaltRenderEngine.py to pass in extra camera data? What changes to the do_render section would i need to make?

Any guidance or pointers would be hugely appreciated. I’m so close, I just need to expose a new camera to use for my texture without breaking the main rendering. I'm definitely no programmer by any means so I'm surprised I managed to get this far.

Thanks!

[Psi25]

bump

[pragma37]

Hey! Sorry for the late reply.
There's no built-in way to do this.

You could add an extra property to your custom MiniPipeline, like this:

self.parameters.world['Test'] = Parameter(
            [1.0,0.0,0.0,0.0,
             0.0,1.0,0.0,0.0,
             0.0,0.0,1.0,0.0,
             0.0,0.0,0.0,1.0
            ], Type.FLOAT, 16)

And you will get something like this:

Then you would need to add a driver to each property to connect it to the camera matrix, which is not exactly clean, but workable.

Another option would be to modify BlenderMalt itself and add something like this (not that I have not tested it, there may be errors):

diff --git a/BlenderMalt/MaltRenderEngine.py b/BlenderMalt/MaltRenderEngine.py
index 25fcdbb..6cb7a8b 100644
--- a/BlenderMalt/MaltRenderEngine.py
+++ b/BlenderMalt/MaltRenderEngine.py
@@ -101,6 +101,16 @@ class MaltRenderEngine(bpy.types.RenderEngine):
             ))
             scene.camera = Scene.Camera(camera_matrix, projection_matrix)

+        if "MyOtherCamera" in depsgraph.scene_eval.objects:
+            camera = depsgraph.scene_eval.objects["MyOtherCamera"]
+            camera_matrix = flatten_matrix(camera.matrix_world.inverted())
+            projection_matrix = flatten_matrix(
+                camera.calc_matrix_camera( depsgraph, 
+                    x=depsgraph.scene_eval.render.resolution_x, 
+                    y=depsgraph.scene_eval.render.resolution_y
+            ))
+            scene.my_other_camera = Scene.Camera(camera_matrix, projection_matrix)
+        
         if request_scene_update == False:
             return scene

Then, as long as you scene has a camera named MyOtherCamera (or whatever you choose), you will be able to read scene.my_other_camera from your pipeline and pass it to self.common_buffer.load()

[Nyubug]

I see you responded as I was typing this, and yep I took the approach of just modifying BlenderMalt.
I still have one more question for you but I figured I'd update you as I have made a lot of progress!

I ended up just adding a new cameras dictionary to the scene class in Scene.py.
Then in MaltRenderEngine.py I just copied the section that was responsible for the original camera variable in scene class and tweaked it to instead populate the cameras dictionary with all camera object types.

In the MiniPipeline It was just a matter of selecting a camera to render into the custom pass before the main pass renders using an object parameter.

But that wasn't enough! I needed more control over the resolution per texture in the TextureArray since each texture is same size.

I did this in a very ghetto way by just cloning the bind and clear functions in RenderTarget.py, and tweaked them to allow me to specify the width and height. But I noticed that draw_scene_pass function inside of Pipeline.py was calling the original bind and clear functions, so I also cloned the draw_scene_pass function to use the tweaked bind and clear functions.
Yeah, its very brute force but it works.

There was one more thing I needed access to, which was cropping the camera view to a specific screen space region. This was tough but i just ended up building my own projection matrix directly in the MiniPipeline allowing me to specify the top, bottom, left, and right corners. I ended up finding the math for this in matrix44.py

My final obstacle is trying to access the vertex positions of all object(s) within a collection inside the pipeline. As far as I can tell, I only have access vertex data inside shaders. I still haven't figured this out.

The idea is the camera will automatically crop the projection matrix to the bounding box and render only what is needed into the texture. Using some simple math I can calculate the width and height resolution as well as the UV.

So my new question is, can I access vertex positions/data inside the pipeline?

[pragma37]

Glad to see you're making progress. :)

So my new question is, can I access vertex positions/data inside the pipeline?

Kind of. They're discarded after the mesh is loaded, so the data is no longer available.
However, you can override the Pipeline.load_mesh function, and store anything you need.