pint-cf: A Lark-based parser and units registry for CF-compliance with pint

[pgamez]

(I’ve also shared this on MetPy’s discussions)

Hi all,

I’ve recently published pint-cf, a lightweight package designed to bring CF-compliant unit support to pint. While excellent tools like yours and MetPy already handle many use cases, they often rely on regex-based preprocessors or partial custom definitions to bridge the gap between UDUNITS-2 and Pint.

This package tries to provide a more formal approach:

• A specific CF unit parser, a Lark-based processor that translates CF units string into pint-compatible expressions.
• A preconfigured pint.UnitRegistry that loads UDUNITS2 definitions and handles pluralization by disabling pint's standard suffixes
• A dedicated CF formatter, supporting both long and short forms.

Code is available at https://github.com/pgamez/pint-cf

I would be very grateful if you could take a look and share any feedback.

I hope you find it interesting!

Thanks,

Pedro

[dcherian]

Beautiful!

I'd be happy to switch out our regex stuff for this.

cc @aulemahal

[pgamez]

That’s fantastic to hear! I’m really happy and grateful

[trexfeathers]

¹

Hi @pgamez, I'm reaching out on behalf of the SciTools organization, who maintain cf-units. We think pint-cf looks seriously exciting!

We had the same thought as you: there is a need for small project that provides CF-compliant Pint units, and we had almost started writing our own parser, so we're very glad you shared this project when you did ❤️. We are keen to use Pint for unit operations since it is more widely accepted by the community, and it would enable Pip-installation (UDUNITS2 prevents this).

In our own projects, we would love pint-cf as a dependency, as an alternative to cf-units. So, we have some questions:

1. How do you imagine the future of this project? You've presented it as a proof-of-concept, are you up for maintaining something in the long-term?
2. Would you welcome outside maintainers/developers? We want to see this succeed, and we know the realities of keeping an open source Python project going, so we would be happy to help out. As a start we could set up a conda-forge recipe.
3. Would you be open to adding support for time coordinates and climate calendars? We have ideas for how this could be done, and would be happy to help, but adding the support would slightly alter the identity/philosophy of pint-cf, so that might not be wanted.
4. Do you have any plans to implement the differences between CF and UDUNITS2? These are well summarised in the cfunits documentation: cfunits.Units. We would be happy to help with this.

Happy to talk in any form you like; here, on your repo, or on a call, whatever 👍

Thanks for your time!

Footnotes

1. (@dcherian sorry for piggybacking on this conversation; it seemed best to post where there was already activity. You can tell me to post elsewhere if you prefer!) ↩

[pgamez]

thank you @aulemahal !

I agree with you here. The way I see it, pint only handles units and so would pint-cf, that makes sense to me. cftime is good at what it does. Both these package are focused at their job and should be unaware of the whole dataset structure. This is what xarray does, with cf-xarray adding layers of cf comprehension that would be too niche/heavy for xarray. What I mean here, is that 100% CF compliance might be impossible outside of cf-xarray (in the xarray ecosystem at least). And its ok.

That's a great point! I think this is the definitive answer

An example for that last point is the units_metadata attribute of CF. Currently it is only used to differentiate between absolute and relative temperatures, degC vs delta_degC in pint-speak. For pint-cf to manage this, that would mean initializing a Unit with an extra argument ? From the attributes directly ? Not something possible in pint I believe, so kinda already out-of-scope. Maybe pint_cf.parser.cf_string_to_pint could have an extra argument. This would be the job of cf-xarray I think, may be adding an accessor ds.cf.units that reads in the units_metadata.

Ah yes! pint manages well the temperature conversions (if you substract 2 temperature quantities, the resulting unit will be delta_*) but not supported yet in pint-cf...

How do you see a solution like this? (it is a quick-and-dirty solution, it needs to be implemented properly):

from collections.abc import Callable, Iterator
from contextlib import contextmanager

import pint

from pint_cf import cf_unitregistry


@contextmanager
def units_metadata(
    ureg: pint.UnitRegistry, units_metadata: str | None = None
) -> Iterator[Callable[[str], pint.Unit]]:
    if units_metadata == "temperature: difference":
        yield lambda units: ureg.parse_units(f"delta_{units}")
    else:
        yield lambda units: ureg.parse_units(units)


def get_metadata(unit: pint.Unit) -> str | None:
    if unit._REGISTRY.get_dimensionality(unit) == "[temperature]":
        if unit._REGISTRY._is_multiplicative(str(unit)):
            return "temperature: difference"
        else:
            return "temperature: on_scale"


#
# EXAMPLE USAGE
#

if __name__ == "__main__":
    ureg = cf_unitregistry()

    # --------------------- Parsing a unit with units_metadata attribute

    with units_metadata(ureg, "temperature: difference") as metadata_parser:
        u = metadata_parser("degrees_Celsius")

    print(repr(u))  # <Unit('delta_degree_Celsius')>

    # ------------------------------------ Obtaining metadata from units

    metadata = get_metadata(ureg.parse_units("delta_degrees_Celsius"))
    print(metadata)  # temperature: difference

    metadata = get_metadata(ureg.parse_units("degrees_Celsius"))
    print(metadata)  # temperature: on_scale

    metadata = get_metadata(ureg.parse_units("meters"))
    print(metadata)  # None

I'm not comfortable relying on Pint's private attributes/methods, but it seems to be the only way the current API allows for this.

One final note: Currently the previous example does not work with degC. It seems that Pint generates the delta_ suffix only for definitions but not for aliases. I do not know if this behaviour is intended or not, or I have to fix the pint-cf's registry to prevent defining temperature aliases as @alias. At the moment degC is defined in pint-cf as:

@alias degree_Celsius = ℃ = celsius = celsiuses = degree_C = degrees_C = degreeC = degreesC = deg_C = degs_C = degC = degsC

I think I would report this to Pint.

[pgamez]

Hi @dcherian !

That sounds fantastic! I am really grateful for the support. I'd love to see how pint-cf could help improving our work field :) Thanks for the trust!

[pgamez]

Well, the context manager is not needed actually...

[pgamez]

hi @aulemahal

As an alternative, I think I can initialize a Unit with an extra argument, but I need to subclass the UnitRegistry, so cf_unitregistry() will return a pint_cf.UnitRegistry instead of a pint.UnitRegistry

from importlib import resources

import pint

from pint_cf.parser import cf_string_to_pint


class UnitRegistry(pint.UnitRegistry):
    def parse_units(
        self, input_string, as_delta=None, case_sensitive=None, units_metadata=None
    ):
        if units_metadata == "temperature: difference":
            input_string = f"delta_{input_string}"
        return super().parse_units(input_string, as_delta, case_sensitive)


def cf_unitregistry() -> pint.UnitRegistry:
    with resources.path("pint_cf.resources.registry", "udunits2.txt") as filename:
        ureg = UnitRegistry(
            filename=str(filename),
            autoconvert_offset_to_baseunit=True,
            preprocessors=[cf_string_to_pint],
        )

    ureg._suffixes = {"": ""}
    return ureg


ureg = cf_unitregistry()
unit = ureg.parse_units("degrees_Celsius", units_metadata="temperature: difference")

print(type(ureg))  # <class '__main__.UnitRegistry'>
print(unit)  # delta_degree_Celsius

[pgamez]

@aulemahal My apolologies for multiple messages. I hope this is the final update for now.

I've tried another option. I think this could be a better solution. Please look how to use and tell if it could fit well with cf-xarray.

ureg = cf_unitregistry()

units = "degrees_Celsius"
units_metadata = UnitsMetadata("temperature: difference")

# ---------------------- Option 1: Modifying input string before parsing

input_str = units_metadata.apply_to(units)
unit = ureg.Unit(input_str)

print(unit)  # delta_degree_Celsius

# -------- Option 2: Using a contextvar that the preprocessor can access

with units_metadata:
    unit = ureg.Unit(units)

print(unit)  # delta_degree_Celsius

# -- How to obtain metadata from an existing unit (serializing to netCDF)

q1 = 10 * ureg("degC")
q2 = 15 * ureg("degC")
value = q2 - q1  # It should be "5 delta_degree_Celsius"

metadata = UnitsMetadata.from_unit(value.units)
print(metadata.to_str())  # temperature: difference

metadata = UnitsMetadata.from_quantity(value)
print(metadata.to_str())  # temperature: difference

metadata = UnitsMetadata.from_quantity(ureg("degrees_Celsius"))
print(metadata.to_str())  # temperature: on_scale

metadata = UnitsMetadata.from_quantity(ureg("meters"))
print(metadata.to_str())  # None