memory leak when updating reagent component

[jamiepratt]

See discussion here:

https://clojureverse.org/t/memory-efficient-vega-lite-reagent-components/9419/5

It seems that garbage collection is not working in underlying vega libraries possibly because they are expecting view.finalize to be called. So for complex reagent graphs the memory heap used can increase by MBs each time the graph is updated.

For example here:

https://jointprob.github.io/jointprob-shadow-cljs/memory-efficiency.html

Notice I did try to fix the issue in this example by calling finalize, see this commit:

jointprob/jointprob-shadow-cljs@e0d6621

But that hasn't fixed the issue.

[jamiepratt]

Neither does this work:

jointprob/jointprob-shadow-cljs@042c593

[jamiepratt]

I experimented with using the "componentWillUnmount" as suggested in the clojureverse thread hook but it is not called when vegaEmbed replaces one graph with another.

I did eventually find the react-vega npm library which can be used to create a react component out of a vega or vega-lite spec and data. Whatever this component is doing under the hood it doesn't appear to be leaking memory see:

https://jointprob.github.io/jointprob-shadow-cljs/memory-efficiency-2.html which uses react-vega

In comparison to:

https://jointprob.github.io/jointprob-shadow-cljs/memory-efficiency.html which uses the oz vega-lite react component.

[metasoarous]

@jamiepratt Thanks again for bringing this up and working on a fix!

The problem may be that the component is not technically unmounting, but just "updating". So perhaps we need componentWillUpdate to do this in addition to componentWillUnmount.

Thanks again!

[jamiepratt]

There is a react-vega component maintained as part of the vega project that recompiles a changed spec or uses view.change when only the data changes:

https://github.com/vega/react-vega/blob/master/packages/react-vega/src/utils/updateSingleDatasetInView.ts

[jamiepratt]

Discussing with jsa-aerial, Hanami maintainer, here: https://clojurians.zulipchat.com/#narrow/stream/210075-saite-dev/topic/animated.20vega-lite.20graphs/near/303463351

[metasoarous]

Hey @jamiepratt. Sorry for the delayed response.

I actually want to do something similar to regarding calling view.change when just the data changes (see #95). I suspect that this will be much more efficient for us in ClojureScript as well, due to equality checks being more efficient with persistent data structures.

In previous discussions, I didn't realize that part of your problem was that you were trying to create an animation. Presumably just the data is changing between frames then? If this is the case, you could take advantage of Oz's :view-callback option to grab the vega view object, and use the view.change method yourself to update just the data. Please let me know if this makes sense.

Don't supposed you've tried updating your code to use componentWillUpdate, did you?

Thanks again!

[metasoarous]

PS I've used the react-vega project in the past from vanilla JS, and had issues with it. It may have gotten better now that it's part of the official vega project, but I think there are still likely to be advantages in having our own reagent adapter; We've got enough of our own Clojurisms for managing state changes & flow that having direct access is useful. I also don't like depending on react libs when I can to avoid issues with versioning.