Feature Request: EMA

[JonathanSchmidt1]

Hi,
I noticed that training sevennet models seems quite a bit more difficult than nequip even when using the nequip architecture.
The main difference that I found that might explain this is the lack of ema so it would be nice to have it.
Or did you maybe try it and not find a difference in training performance?

[YutackPark]

Hi, actually we already have tried ema, but couldn't find notable difference, at least in MAE errors.

Instead, have you tried amsgrad? To the best of my knowledge, nequip have this as a True by default while 7net is not. In my experience, it stabilizes the training process a lot, but the last converged errors were a bit higher than vanilla Adam.

You can easily try this by setting amdgrad = True right below the lr, where you can set optimizer specific arguments.

However, I agree with that training MLIP itself is a hard problem.