Removing omegaconf and dynamic config

[Luke2642]

Hi again! :-)

When vibe coding it's best to remove all dynamic config and hard code the yaml data into classes. Would you accept a pull request for this?

I've also combined dit_3b and dit_7b into one class. I've also vendored in lots of code from diffusers to remove the dependency. Would you accept a pull request for this too? It's quite a big refactor, before I start on using comfyui model memory management properly.

[naxci1]

Hi @Luke2642

Have you tested it? Can you share the chanlog? Send us a PR so we can review and test it. What are the advantages?

[Luke2642]

This is why I asked, because it's a massive refactor, almost every file changed. I think it's probably something you want to make antigravity/curso plan and execute yourself, it'll take half a day or so. The advantage is code readability, reduced dependencies, and all the code is static, so both humans and LLM agents will make far fewer errors. You can type check everything as you're writing it!

The overall goal is to make it Comfy native rather than a wrapper of a project that plays havoc with the comfyui memory management. I'm currently on my third attempt. In the second I manged to make a new VAE loader that is 100% comfy native, and so you just use the regular ComfyUI VAE encode/decode but the built in comfy tiled encode/decode isn't working yet. However, I decided to stop working it and the ksampler, and began this third refactor removing the awful omegaconf and other dynamic stuff before beginning a new clean rewrite. With a more comfy focused approach, you get all the memory management benefits, built in support for sage attention, of ComfyUI without nasty hacks!

[naxci1]

It's true that the code isn't fully compatible with new technologies because it was written a long time ago, and there's an excessive amount of code that needs simplification, but that would mean rewriting it from scratch. Actually, this would have little impact on performance; the real performance increase requires new DIT and VAE models, which are what affect speed. VAE decode is still a bottleneck; I've been working on it for a month, but I haven't achieved the desired result yet. Since VAE and DIT models are trained simultaneously, different VAE models don't work together. Therefore, the models need to be retrained, and VAE should be optional.

If you see a real performance increase through testing, please share it with us and send us a PR so we can try the code as well.