-
Notifications
You must be signed in to change notification settings - Fork 203
[Feature][Metric] Custom SQL support export error data #560
Copy link
Copy link
Open
Description
Search before asking
- I had searched in the feature and found no similar feature requirement.
Description
The current custom_aggregate_sql metric requires users to write a complete aggregate SQL statement in the following format:
SELECT COUNT(1) AS actual_value FROM ${table} WHERE ...This design has the following limitations:
- Unable to output the actual error/invalid data records (isInvalidateItemsCanOutput()=false)
- Users cannot intuitively identify which specific rows violate the data quality rules
- Reduced efficiency in troubleshooting and data remediation
add a new metric type (e.g., custom_count_sql) that allows users to write only the SQL for querying error data:
SELECT * FROM ${table} WHERE <error_condition>
The system will automatically wrap it into an aggregate SQL during execution:
SELECT COUNT(*) AS actual_value FROM (
SELECT * FROM ${table} WHERE <error_condition>
) dv_tempUsage Scenario
No response
Related issues
No response
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels