feat: resume interrupted dataset generation runs (sync + async engine)#526

Open

przemekboruta wants to merge 14 commits intoNVIDIA-NeMo:mainfrom

przemekboruta:main

Commits on Apr 13, 2026

docs: add implementation plan for resume mechanism
przemekboruta
committed
feat(storage): add resume flag and clear_partial_results()
przemekboruta
committed
feat(batch-manager): add start_batch param to start()
przemekboruta
committed
feat(builder): implement resume logic in DatasetBuilder
przemekboruta
committed
feat(interface): expose resume on DataDesigner.create()
przemekboruta
committed
test: add tests for resume mechanism
przemekboruta
committed
feat(builder): extend resume to async engine (DATA_DESIGNER_ASYNC_ENGINE=1)
przemekboruta
committed
fix(builder): skip after-generation processors when resume finds dataset already complete
przemekboruta
committed
fix(builder): use filesystem count for initial_total_num_batches on async resume
przemekboruta
committed
Merge branch 'main' into main
andreatgretel
authored