Skip to content

Conversation

@DanielZhu58
Copy link
Contributor

What changes were proposed in this pull request?

We need a way to drop the stats associated with the table/partition and its columns.
This can help a lot in migration or replication where the stats data take huge time to copy.
Particularly when the table is partitioned, we have stats rows for each table, partition, column combination, which can get huge when the number of partitions is huge.
This is the HMS side changes.
This is the part2 with the parameter COL_STATS_ACCURATE related changes.

Why are the changes needed?

This can give the users potential options to clear the unnecessary stats.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manual tests and unit tests.

@sonarqubecloud
Copy link

@sonarqubecloud
Copy link

}

// Serialize the map into a new JSON string
String updatedValue = mapper.writeValueAsString(statsMap);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use StatsSetupConst.removeColumnStatsState?

Copy link
Contributor

@soumyakanti3578 soumyakanti3578 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a few comments here, but I also noticed that I had reviewed the earlier version of this PR: #5790
Please go through the review comments there too.

Also it would be nice to go through the sonar report and fix the ones that make sense. For example, there are many reports on code conventions that you should incorporate to improve maintainability and consistency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants