Reading framebuffer data to pixel buffers

[micahscopes]

Wondering if anyone has tried reading data from gpu-io framebuffers to a pixel buffer, and if so how did it go?

I'm considering trying this. I think it could potentially simplify the process of using data computed with gpu-io as regular old uniform buffer/vertex buffer data, as an alternative to transform feedback, and to extra texture sampling. That said, I'm unsure about performance, so will need to experiment.

If it works it'd be cool to have a little API for it, especially if it could be used with THREE.js buffers in the same way as the textures can be shared.

(by the way, this library is awesome, I tried like 5 different WebGL GPGPU libraries before settling on this one and it's been working out great)

[amandaghassaei]

do you mean pulling the data back to the cpu using readPixels() and then putting it back into threejs that way?

you can call getValues() or getValuesAsync() on a DataLayer object, which both call readPixels() internally and return the values in the framebuffer as a TypedArray:
https://github.com/amandaghassaei/gpu-io/blob/6cbfb1287fa629f7e9bdc1a44670edb2652ddb45/src/GPULayer.ts. (starting at line 917)
(getValuesAsync uses a PBO)

Though it's obviously less efficient to copy the data from the gpu->cpu->gpu, in practice I've found that you can still run realtime sims of a decent size with no problems. And the interface with threejs is a lot simpler! I'm actually using a similar strategy in this project: https://origamisimulator.org/

maybe you have something else in mind though, let me know if I'm not understanding the question.

thanks for the nice comment, glad the library has been helpful!

[micahscopes]

@amandaghassaei I was thinking just copying pixels directly gpu -> gpu... I just went ahead and implemented it in #13!

[micahscopes]

Uh, by the way the origami simulator is 🤯

[amandaghassaei]

awesome, thanks for the PRs - so I guess the idea is to pass e.g. the position buffer of a threejs BufferGeometry into the copyToWebGLBuffer method to transfer the results from gpu-io to threejs? Do you have that interop working with the changes in #13?

I will have some time to take a closer look next week and hopefully merge in, thanks so much for the contribution!

[micahscopes]

That's the idea, yup, though I have to say I haven't tried that part yet. As you can see in the tests the GPU -> GPU transfer is working and hopefully the connection with THREE.js WebGLBuffer backed objects will go smoothly. Setting vertex attributes seems like the a good simple test case for that.