Usage: Finch backend returns Julia-native dtypes like 'Julia: Float64.' Is this intentional?

[amalia-k510]

Please provide a description of what you'd like to do.

Hello,

I’ve run into an issue when using the Finch backend via pydata/sparse, specifically with libraries that expect NumPy-compatible dtypes (e.g. when calling np.dtype(...) on sparse arrays).

When I set os.environ["SPARSE_BACKEND"] = "Finch"

and create a COO array from a NumPy array:

import numpy as np

os.environ["SPARSE_BACKEND"] = "Finch"
import sparse

sa = sparse.COO.from_numpy(np.eye(4))
print(sa.dtype)  # returns <class 'finch.tensor.Tensor'>

This causes downstream failures in libraries that assume they can call np.dtype(sa.dtype) with the following error TypeError: Cannot interpret 'Julia: Float64' as a data type.

So I have the following questions:

• Is returning 'Julia: Float64' as a dtype string intentional behavior?
• Are there plans to return NumPy-compatible dtype objects instead?
• Are there known workarounds for this kind of downstream compatibility issue?

I wasn’t able to find a discussion of this in the issue tracker, apologies if I missed it.

Thanks so much!

/cc @ilan-gold

Example Code

No response

[hameerabbasi]

Hi, we're working on a pure-Python implementation of Finch over at https://github.com/finch-tensor/finch-tensor-lite (not meant for user consumption in its current form).

The reproducer isn't working for me as-is, so I modified it, and added a workaround:

import os
import numpy as np

os.environ["SPARSE_BACKEND"] = "Finch"
import sparse

sa = sparse.asarray(np.eye(4), format="coo")
print(sa.dtype) # Float64
print(type(sa.dtype)) # <class 'juliacall.TypeValue'>


# Populate a mapping of dtypes, both ways
np_to_sparse = {}

for possible_dtype in np.__all__:
    np_dtype = getattr(np, possible_dtype)
    sparse_dtype = getattr(sparse, possible_dtype, None)
    if sparse_dtype is None:
        continue
    try:
        np_dtype = np.dtype(np_dtype)
    except (TypeError, ValueError):
        continue

    np_to_sparse[np_dtype] = sparse_dtype

sparse_to_np = {v: k for k, v in np_to_sparse.items()}

# Now use the mapping:
print(sparse_to_np[sparse.float64])  # float64

Let me know if this works for you.

[ilan-gold]

@hameerabbasi We were pushing this upstream into cubed so this is less about "can one in theory make this work?" than "is this intentional?" In other words, is the goal (long term, let's say) to maintain a "Julia-based" type system or implement some form of numpy compat? We are running into a situation (I understand it's not array-api compat in theory) where the caller is doing:

np.dtype(sa_dtype)

and we are not the caller.

For example, jax uses numpy's type system it appears:

import jax.numpy as jnp
import numpy as np

isinstance(jnp.array([0]).dtype, np.dtype)

We've opened a separate issue with the caller code about array-api compat but are just curious what the goal of pydata/sparse is given that with the numba backed, it uses numpy types:

import sparse

isinstance(sparse.COO.from_numpy(np.eye(4)).dtype, np.dtype)

[hameerabbasi]

Cc @willow-ahrens Would you accept a PR to finch-tensor (the Julia wrapper) to support NumPy dtypes as default?