-
Notifications
You must be signed in to change notification settings - Fork 87
Open
Labels
Description
Motivations
the current structure's write model only supports serial writes(serial_parallelizer.rs), which can lead to performance bottlenecks when there are many tables.
[extractor]
extract_type=struct
[sinker]
sink_type=struct
[parallelizer]
parallel_type=serial
direct and rough concurrency may cause data problems, such as creating an index first but the table does not exist
Solution
- provides a new parallelizer implementation that includes data conflict management in concurrent scenarios.
- mark conflict information in advance in the data fetched in the extractor (xx-struct-extractor.rs)
'tables' depend on 'database';
'indexs' depend on 'tables';
'tables have FK' deplend on 'other tables' and so on