Commit 16818b4
[Enhancement](recyclebin) Optimize lock granularity in CatalogRecycleBin (#61366)
All methods in `CatalogRecycleBin.java` use `synchronized` (single
monitor lock), creating extremely coarse lock granularity. When
`erasePartition()` runs slowly with many partitions, other
`synchronized` methods block waiting for the lock. Callers like
`recyclePartition()` hold TABLE WRITE LOCK while waiting, causing
cascading blocking that can bring down the entire Doris metadata
service.
Two complementary optimizations:
- **Lock-free** (8 methods): Simple ConcurrentHashMap lookups
(`isRecyclePartition`, `getRecycleTimeById`, etc.)
- **Read lock** (4 methods): Read-only iterations
(`allTabletsInRecycledStatus`, `getInfo`, `write`, etc.)
- **Write lock** (11 methods): Map mutations
(`recycleDatabase/Table/Partition`, `recover*`, `clearAll`)
Refactored all 12 erase methods to process items **one at a time** with
lock release between items:
- **Inside write lock (per item)**: cleanup RPCs + map removal + edit
log write
- **Release lock between items**: other operations can proceed
This reduces lock hold time from **O(N × T)** (all items) to **O(T)**
(one item) per acquisition.
Changed 4 internal maps from `HashMap` to `ConcurrentHashMap` to enable
lock-free reads.
1. **NPE in `getIdListToEraseByRecycleTime`**: Used `getOrDefault` to
handle stale IDs that may be concurrently removed between snapshot and
processing
2. **DdlException in cascade erase**: Added try-catch in
`eraseDatabaseInstantly`/`eraseTableInstantly` for partitions/tables
concurrently erased by daemon
- All 24 existing unit tests pass
- Added 3 new concurrency tests:
- `testConcurrentReadsDoNotBlock` — 10 concurrent reader threads
- `testConcurrentRecycleAndRead` — writer + 5 readers simultaneously
- `testMicrobatchEraseReleasesLockBetweenItems` — verifies
recyclePartition succeeds during erase
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent 38ae4a9 commit 16818b4
File tree
2 files changed
+1151
-734
lines changed- fe/fe-core/src
- main/java/org/apache/doris/catalog
- test/java/org/apache/doris/catalog
2 files changed
+1151
-734
lines changed
0 commit comments