testing errors in `make test_python` and `make_test_cmip6cv` using new `QUICKSTART` instructions

[ilaflott]

wondering about make test_python and make_test_cmip6cv:

the approach i took given the instructions

# conda env creation + checkout code --> good
git clone --recursive git@github.com:PCMDI/cmor && cd cmor
module load conda
conda create -n cmor_dev -c conda-forge \
  libuuid \
  json-c \
  udunits2 \
  hdf5 \
  libnetcdf \
  openblas \
  netcdf4 \
  numpy \
  openssl \
  python \
  pyfive \
  hdf5plugin \
  $CONDA_COMPILERS
conda activate cmor_dev

# configure --> good
export LDSHARED_FLAGS="-shared -pthread"
export PREFIX=$(python -c "import sys; print(sys.prefix)")

./configure \
  --prefix=$PREFIX \
  --with-python \
  --with-uuid=$PREFIX \
  --with-json-c=$PREFIX \
  --with-udunits2=$PREFIX \
  --with-netcdf=$PREFIX \
  --enable-verbose-test

# good
make install
export HDF5_PLUGIN_PATH="$(python -c 'import hdf5plugin; print(hdf5plugin.PLUGINS_PATH)')"

# largely good but did not work
make test

first failure is here:

make test_python

error output seems testing clean-up related. removing the directory and rerunning will not fix the problem, so it's a clash of output between tests under test_python is my guess:

python Test/test_cmor_parent_attrs.py
....EEE
======================================================================
ERROR: test_cmip7_warns_on_parent_attributes_without_parent_experiment (__main__.TestCMIP7WithParentAttributes.test_cmip7_warns_on_parent_attributes_without_parent_experiment)
----------------------------------------------------------------------

OSError: [Errno 39] Directory not empty: './CMIP/PCMDI-test-1-0/piControl/r009i000001dp1f30/tos/tavg-u-hxy-sea/gn'

======================================================================
ERROR: test_cmip7_with_parent_attributes (__main__.TestCMIP7WithParentAttributes.test_cmip7_with_parent_attributes)
----------------------------------------------------------------------

OSError: [Errno 39] Directory not empty: './CMIP/PCMDI-test-1-0/historical/r009i000001dp1f30/tos/tavg-u-hxy-sea/gn'

======================================================================
ERROR: test_cmip7_without_parent_attributes (__main__.TestCMIP7WithParentAttributes.test_cmip7_without_parent_attributes)
----------------------------------------------------------------------

OSError: [Errno 39] Directory not empty: './CMIP/PCMDI-test-1-0/piControl/r009i000001dp1f30/tos/tavg-u-hxy-sea/gn'

----------------------------------------------------------------------
Ran 7 tests in 0.579s

FAILED (errors=3)
make[1]: *** [Makefile:157: test_a_python] Error 1
make[1]: Leaving directory '/home/Ian.Laflotte/Working/cmor'
make: *** [Makefile:236: test_python] Error 2

perhaps i shouldn't expect this to pass because it's no longer cmip6, but, second failure is here:

make test_cmip6_cv

error output similarly seems testing cleanup related:

E
======================================================================
ERROR: testCMIP6 (__main__.TestCase.testCMIP6)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/Ian.Laflotte/Working/cmor/Test/base_CMIP6_CV.py", line 32, in tearDown
    self.remove_file_and_directories(filename)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/home/Ian.Laflotte/Working/cmor/Test/base_CMIP6_CV.py", line 14, in remove_file_and_directories
    os.rmdir(filename)
    ~~~~~~~~^^^^^^^^^^
OSError: [Errno 39] Directory not empty: 'CMIP6/CMIP6/DCPP/PCMDI/PCMDI-test-1-0/dcppA-hindcast/s2000-r3i1p1f1/Omon/masso/gn/v20260423'

----------------------------------------------------------------------

Originally posted by @ilaflott in #951 (comment)

[mauzey1]

@ilaflott Have you tried cleaning up the output directory by removing ./CMIP and then running python Test/test_cmor_parent_attrs.py by itself?

[ilaflott]

yes! that did not work. i also tried calling make test_python in isolation.

this is non urgent- this is doc'ing something someone else might see if they follow QUICKSTART.md

[ilaflott]

you probably want shutil.rmdir instead of os.rmdir

[ilaflott]

you probably want shutil.rmdir instead of os.rmdir

this is a guess, to clarify

[mauzey1]

Okay, I'm currently looking at how the base unit test class is deleting files. I'll investigate a way to make this simpler and less error-prone.

cmor/Test/base_CMIP6_CV.py

Lines 10 to 15 in 2883d46

	def remove_file_and_directories(self, filename):
	os.remove(filename)
	filename = os.path.dirname(filename)
	while glob.glob(os.path.join(filename, "*")) == []:
	os.rmdir(filename)
	filename = os.path.dirname(filename)

[mauzey1]

@ilaflott

The function remove_file_and_directories is doing the following.

1. Removes the file.
2. Gets the directory of the file.
3. If the directory is empty, then delete the directory.
4. Repeat step 3 with the parent directory.

This is done to clean up the test output directory by not only removing the test output files but also the directory structure that it created. For some reason, your test environment caused glob.glob(os.path.join(filename, "*")) to generate an empty list when the directory path in filename still had files in it.

I think the safest and simplest solution is to just replace self.remove_file_and_directories(filename) in the tearDown function of BaseCVsTest with os.remove(filename). It won't delete the directory structure left after running make test_cmip6_cv but it will at least remove the output files and not cause the error. Running make test will leave a bunch of files/directories anyway.