Fix dangling lock error with storage based lock provider #18438
Description
Bug Description
What happened:
Underlying storage (s3/GCS/Azure storage) may throw 429 for rate limiting purpose. Due to this behavior, the storage lock file may not be updated during unblock phase, which cause the dangling lock error.
What you expected:
We should retry when we see 429 error to reduce the dangling lock error.
Steps to reproduce:
1.
2.
3.
Environment
Hudi version:
Query engine: (Spark/Flink/Trino etc)
Relevant configs:
Logs and Stack Trace
No response