Skip to content

bug: ibis-framework 11.0.0 incompatible with sqlglot 28.7.0 - binary_double_nan error on import #11890

@saryeHaddadi

Description

@saryeHaddadi

What happened?

After reinstalling the virtual environment, the my script immediately failed on startup with a decimal.InvalidOperation error during the import of ibis.backends.trino. The error occurred in the Oracle compiler when trying to create a NAN literal using sqlglot.

The script couldn't even start execution, it failed during module imports.

I solved it by adding a version constraint:

sqlglot<26.0.0

What version of ibis are you using?

ibis-framework version: 11.0.0
sqlglot version (broken): 28.7.0
sqlglot version (working): <26.0.0
Python version: 3.10.19
OS: Windows

What backend(s) are you using, if any?

Backend: Trino

Relevant log output

$ uv run python scripts/generate_dbt_models.py
Demarrage du script...
Traceback (most recent call last):
  File "***\.venv\lib\site-packages\sqlglot\expressions.py", line 2634, in to_py
    return int(self.this)
ValueError: invalid literal for int() with base 10: 'binary_double_nan'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "***\scripts\generate_dbt_models.py", line 672, in <module>
    main()
  File "***\scripts\generate_dbt_models.py", line 662, in main
    with ibis_connection(TRINO_CONFIG) as conn:
  File "***\lib\contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "***\.venv\lib\site-packages\azfr_skywalker_utils\ibis\trino.py", line 16, in ibis_connection
    conn = ibis.trino.connect(
  File "***\.venv\lib\site-packages\ibis\__init__.py", line 142, in __getattr__
    return load_backend(name)
  File "***\.venv\lib\site-packages\ibis\__init__.py", line 82, in load_backend
    module = entry_point.load()
  File "***\lib\importlib\metadata\__init__.py", line 171, in load      
    module = import_module(match.group('module'))
  File "***\lib\importlib\__init__.py", line 126, in import_module      
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "***\.venv\lib\site-packages\ibis\backends\trino\__init__.py", line 17, in <module> 
    import ibis.backends.sql.compilers as sc
  File "***\.venv\lib\site-packages\ibis\backends\sql\compilers\__init__.py", line 37, in <module>
    from ibis.backends.sql.compilers.oracle import OracleCompiler
  File "***\.venv\lib\site-packages\ibis\backends\sql\compilers\oracle.py", line 25, in <module>
    class OracleCompiler(SQLGlotCompiler):
  File "***\.venv\lib\site-packages\ibis\backends\sql\compilers\oracle.py", line 41, in OracleCompiler
    NAN = sge.Literal.number("binary_double_nan")
  File "***\.venv\lib\site-packages\sqlglot\expressions.py", line 2615, in number
    to_py = expr.to_py()
  File "***\.venv\lib\site-packages\sqlglot\expressions.py", line 2636, in to_py
    return Decimal(self.this)
decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>]

Root Cause

The newer version of sqlglot (28.7.0) changed how it handles special numeric literals like binary_double_nan. The Literal.number() method now tries to convert the literal to a Python number, which fails for the string "binary_double_nan".

This breaks the OracleCompiler class in ibis-framework even when using the Trino backend, because the Oracle compiler is imported unconditionally during the SQL compilers module initialization.

Code of Conduct

  • I agree to follow this project's Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugIncorrect behavior inside of ibis

    Type

    No type

    Projects

    Status

    backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions