Using restarts instead of continuous run leads to differences

[alxthor]

I am doing a (cold start) simulation with zstar on master branch. One time in one go from 1958 to 1960. The other time one year after another with in total two restarts. Subtracting both runs leads to identical results in 1958 but differences building up in the first half year of 1959 and persisting after then.
The differences of upto approx. +-2 Kelvin / +-1 psu in temp / salt are most prominent in top layers near the equator and travel westwards in waves.
@dsidoren asked me to post an issue here. I hope I am doing it correctly.

[hegish]

@alxthor, thank you for reporting this. Could you tell us which branch and commit hash you are using? I want to try to reproduce the problem.

[alxthor]

Thanks, sorry for the missing info. I am using master branch, commit e7562dc

[koldunovn]

I think it's the consequence of forcing treatment during the restart. In continuous run you interpolate forcing between timesteps, in the case of the restart, one just takes the forcing that is closest to the time step. So you end up with runs, that have a bit different forcings.

[alxthor]

I see. Can that explain, why the effects are so large in magnitude and mainly occur in the equatorial region (especially mid Pacific and Malaysia) and sometimes near Antarctica?

[patrickscholz]

Could you say which mixing scheme you using ?

[alxthor]

I'm using KPP

[patrickscholz]

"KPP" ... than it is OK i just discovered this week a small mistake in the restart of TKE+IDEMIX mixing.

[alxthor]

Thank you. So you recon that this mistake could be causing this effect and fixing this could remove the large descrepancy between the runs?

[patrickscholz]

No fixing this bug in TKE+IDEMIX will not affect KPP, So it will not fix your problem.

[dsidoren]

I think it's the consequence of forcing treatment during the restart. In continuous run you interpolate forcing between timesteps, in the case of the restart, one just takes the forcing that is closest to the time step. So you end up with runs, that have a bit different forcings.

I guess you are right. We need to setup a test for this.

[JanStreffing]

This requires a few weeks of work, rewriting the way how FESOM reads annual restarts. Deferred to 2.8