Problem with NumPy arrays and JSON

[meteorologist15]

Using esgpublish v5.2.4, on an attempted publishing action, I received the following traceback:

Traceback (most recent call last):
  File "/esg/publishing/miniconda3/envs/esgf-pub/lib/python3.8/site-packages/esgcet/generic_netcdf.py", line 75, in mk_dataset
    out_json_data = mkd.get_records(map_json_data, scan_arg, self.json_file, user_project=self.proj_config)
  File "/esg/publishing/miniconda3/envs/esgf-pub/lib/python3.8/site-packages/esgcet/mk_dataset.py", line 419, in get_records
    self.publog.debug("Record:\n" + json.dumps(self.dataset, indent=4))
  File "/esg/publishing/miniconda3/envs/esgf-pub/lib/python3.8/json/__init__.py", line 234, in dumps
    return cls(
  File "/esg/publishing/miniconda3/envs/esgf-pub/lib/python3.8/json/encoder.py", line 201, in encode
    chunks = list(chunks)
  File "/esg/publishing/miniconda3/envs/esgf-pub/lib/python3.8/json/encoder.py", line 431, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/esg/publishing/miniconda3/envs/esgf-pub/lib/python3.8/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/esg/publishing/miniconda3/envs/esgf-pub/lib/python3.8/json/encoder.py", line 438, in _iterencode
    o = _default(o)
  File "/esg/publishing/miniconda3/envs/esgf-pub/lib/python3.8/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type ndarray is not JSON serializable

I personally used this Pandas solution to get around the issue, but perhaps other strategies may be utilized.

[sashakames]

Hi @meteorologist15 thanks for catching this (do I know you from a different context, I see you are at GFDL?) Are you running with verbose: true or --verbose, I don't think we log the error otherwise. We may need to replicate your data to LLNL so I can reproduce the issue in order to test a bugfix.

[sashakames]

Still an issue @meteorologist15 ? If so, is the file available for me to try out (can you share a link to that on your data node)?