How does optax.MultiSteps handle the last few steps?
#554
Unanswered
ayaka14732
asked this question in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I am using
optax.MultiStepsfor gradient accumulation.My question is: if my
n_accumulation_stepsis 8,total_stepis 807, then807 % 8 == 7, there would be 7 steps left not updated in the end the epoch.Is there anything like
optimizer.finalize()that can update the last few steps?Beta Was this translation helpful? Give feedback.
All reactions