ASDF Failing Unit Tests

[hs-ye]

Hi Team,

Wondering if this is still being maintained/would there be any effort to update this to run on current versions of Julia?

Just downloaded this and getting fails on the tests when attempting to use this package, assuming this is due to changed behaviour of PyCall since this was last updated?

Cheers,

[eschnett]

I haven't used this package in a while. I was hoping it's still working, I am still a fan of the ASDF file format.

I don't know what is wrong. I don't know about changes to PyCall. (Do you have a pointer?) What I am seeing is that the test case is looking for a property yaml_tag that is supposed to be set by the Python asdf library, and this property isn't there. This seems to me more like a change in the Python asdf library.

[giordano]

This looks an argument for #4 🙂

[eschnett]

Things work with an older version of asdf, namely asdf 2.5.

[eschnett]

Apparently there are other version conflicts as well, concerning jsonschema as well as the version of Python itself. I think asdf 2.5 works with Python 3.7 (it does on my laptop), but not Python 3.8. I probably won't have time to dig deeper this week.

[eschnett]

This looks an argument for #4 🙂

Oh yes. Unfortunately, using asdf-cxx is difficult (it's written in C++), so we'd have to re-implement reading the binary data sections in Julia. Clearly the way to go, but not this week.

[hs-ye]

Hey team, yeah makes sense that ASDF on python side that's changed, I've managed to run the current version of python-ASDF (2.7.1 on py3.8) through the current version of pycall (v1.91.4), which is enough for my current purposes.

It would be nice to have a higher performance version of the ASDF package that's not bound by python speeds.

Let me know if there's any other info I can provide that might help.

[eschnett]

In the short term, it would be useful to know how ASDF's layout of Python objects has changed. There used to be a property (field?) called yaml_tag that was used to determine its type on the Julia side. Do you know how this changed, and where the equivalent information is available now?

[hs-ye]

Here's the ASDF header for a small test i wrote using PyCall for a single 100x100 integer array - is this what you're after?

#ASDF 1.0.0
#ASDF_STANDARD 1.5.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.1.0
asdf_library: !core/software-1.0.0 {author: Space Telescope Science Institute, homepage: 'http://github.com/spacetelescope/asdf',
  name: asdf, version: 2.7.1}
history:
  extensions:
  - !core/extension_metadata-1.0.0
    extension_class: asdf.extension.BuiltinExtension
    software: !core/software-1.0.0 {name: asdf, version: 2.7.1}
data: !core/ndarray-1.0.0
  source: 0
  datatype: float64
  byteorder: little
  shape: [100, 100]
  strides: [8, 800]
...
<binary data here>

[icweaver]

Update: this may be of interest #10

[icweaver]

Just marking this as something to deal with post v2 release if anyone wants to pick up the v1 development here