incorrect MODEL_ATMOS tracer number registration in atmos_null

[cd10kfsu]

Case

I'm running the test case ice_ocean_SIS2/OM_1deg with MOM6-SIS2 configuration.

• Version of MOM6-examples: commit 8121a9f
• Version of the compiler: gcc/14.3 + openmpi/5.0.9

Running error

I notice that my run gives the error:

At line 919 of file ../../src/coupler/atm_land_ice_flux_exchange.F90
Fortran runtime error: Index '2' of dimension 3 of array 'atm%tr_bot' outside of expected range (1:1)

Error termination. Backtrace:
At line 919 of file ../../src/coupler/atm_land_ice_flux_exchange.F90
Fortran runtime error: Index '2' of dimension 3 of array 'atm%tr_bot' outside of expected range (1:1)

Traceback

Tracing back shows the related lines causing the errors are https://github.com/NOAA-GFDL/FMScoupler/blob/14578f09a25c8e6101faba18342630af267cdba9/atm_land_ice_flux_exchange.F90#L913-L915

    do tr = 1,n_atm_tr
       call get_tracer_names( MODEL_ATMOS, tr, tr_name )
       call data_override('ATM', trim(tr_name)//'_bot', Atm%tr_bot(:,:,tr), Time, override=used)

I printed out shape of n_atm_tr, Atm%tr_bot(:,:,:) which shows

n_atm_tr, shape( Atm%tr_bot)=   4      180          80           1

where n_atm_tr=4 is correct, since I have 4 atmos_mode TRACERS in field_table, but the array initialization (1 here) of Atm%tr_bot is incorrect for its dimension

Tracing back the initialization of the array Atm%tr_bot leads me to the file
https://github.com/NOAA-GFDL/atmos_null/blob/aeac506b910c06ce949fea408c2968c0a928a62e/atmos_model.F90#L471-L473

call register_tracers(MODEL_LAND, ntracers, ntprog, ndiag)
allocate ( Atmos % t_bot(is:ie,js:je) )
allocate ( Atmos % tr_bot(is:ie,js:je,ntprog) )

which seems to me we have should have MODEL_ATMOS instead of MODEL_LAND here

Please let me know if this is a bug and I would like to submit a PR if so.

Thanks,
Cheng

[marshallward]

Are you running with any particular debug flags? We run this regularly in our regression testing, but it could be a "silent bug" that does not affect the more production-like builds.

I'm not very familiar with the atmos_null code but your explanation seems plausible. If so, then you may want to move this to their repo: https://github.com/NOAA-GFDL/atmos_null/

[cd10kfsu]

Thanks @marshallward !

Are you running with any particular debug flags?

Here are my full compiler flags (FFLAGS+FFLAGS_REPRO which I include REPRO=1 when compiling)

FFLAGS := -fcray-pointer -fdefault-double-8 -fdefault-real-8 -Waliasing -ffree-line-length-none -fno-range-check -fallow-invalid-boz -fallow-argument-mismatch

# Flags based on perforance target (production (OPT), reproduction (REPRO), or debug (DEBUG)
FFLAGS_OPT = -O3
FFLAGS_REPRO = -O2 -fbounds-check (Cheng Da: I used REPRO=1 when building)

The actual flags compiling source code are shown as, e.g.,

mpif90 -Duse_AM3_physics -D_USE_LEGACY_LAND_ -Duse_netCDF -DHAVE_SCHED_GETAFFINITY -DHAVE_GETTID  -I/glade/u/apps/derecho/25.10/spack/opt/spack/netcdf/4.9.3/packages/netcdf-fortran/4.6.2/gcc/14.3.0/qgi4/include -I/glade/u/apps/derecho/25.10/spack/opt/spack/netcdf/4.9.3/packages/netcdf-fortran/4.6.2/gcc/14.3.0/qgi4/include -fcray-pointer -fdefault-double-8 -fdefault-real-8 -Waliasing -ffree-line-length-none -fno-range-check -fallow-invalid-boz -fallow-argument-mismatch  -O2 -fbounds-check  -I../fms  -c	../../src/coupler/coupler_main.F90

---

We run this regularly in our regression testing, but it could be a "silent bug" that does not affect the more production-like builds.

I believe so. Only after I start to use gcc-14 on NCAR's derecho last week, then this OM_1deg no longer runs, and throws errors. No problem when I was using gcc-11+openmpi on another HPC.

---

I'm not very familiar with the atmos_null code but your explanation seems plausible. If so, then you may want to move this to their repo: https://github.com/NOAA-GFDL/atmos_null/

Thanks for the suggestion! I will file an issue there.

[cd10kfsu]

call register_github_repo_issues(atmos_null, NOAA-GFDL/atmos_null#14)

[cd10kfsu]

Hi @marshallward , may I ask what compiler (name + version) are you using for the routine regression tests? It seems I encounter this same problem even if I switch to intel 2025 compiler, plus my previous gcc-12 and gcc-14. Thanks!

[marshallward]

@cd10kfsu Here are our environment setup scripts.
intel.env (ifort, 2023.2.0):

export CFLAGS='-sox -traceback -O2 -debug minimal -march=core-avx-i -qno-opt-dynamic-align'
export FCFLAGS='-fpp -Wp,-w   -g -traceback -fno-alias -auto -safe-cray-ptr -ftz -assume byterecl -i4 -r8 -nowarn -sox -traceback -O2 -debug minimal -fp-model source -march=core-avx-i -qno-opt-dynamic-align'

gnu.env (12.3):

export FCFLAGS='-g -fbacktrace -fcray-pointer -fdefault-real-8 -fdefault-double-8 -Waliasing -ffree-line-length-none -fno-range-check -fallow-argument-mismatch -fallow-invalid-boz'

This isn't necessarily what is used in production, but we try to stay in sync (or at least enough to prevent answer changes).

[cd10kfsu]

Thanks @marshallward ! Can you also share which mpi (e.g., intel MPI, openMPI, MPICH) are you using? Thanks

[marshallward]

We use Cray's MPI ("Shasta"), which is based on MPICH.