Skip to content

Refactor duplicate function-evaluation paths in ExpressionTransformer and CustomFunctionEnricher #18216

@xiangfu0

Description

@xiangfu0

Motivation

pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/ExpressionTransformer.java and pinot-segment-local/src/main/java/org/apache/pinot/segment/local/recordtransformer/enricher/function/CustomFunctionEnricher.java still carry reciprocal TODOs to merge the two implementations.

Both code paths evaluate function-based transformations during ingestion, but they live behind different extension points (RecordTransformer vs RecordEnricher). That makes behavior drift and duplicate maintenance more likely.

Scope

  • Document the overlap and intended ownership between the two implementations.
  • Consolidate shared evaluation logic behind one path or a shared abstraction.
  • Preserve current ingestion behavior for transform configs and enricher configs.
  • Add or update focused tests for parity and backward compatibility.

Notes

Observed on upstream/master on April 15, 2026.

Metadata

Metadata

Assignees

No one assigned

    Labels

    cleanupCode cleanup or removal of dead codeingestionRelated to data ingestion pipelinejavaPull requests that update Java codepriority: lowNice to have, can waitrefactorCode restructuring without changing behavior

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions