Improve data source importing #17
Description
Currently the NIST importing functionality is too slow, often taking many hours to import the dataset. Taking a look into the codebase it looks like where spawning multiple database transactions in order to import a single entry:
Since a lot of these entries are completely independent of each other we should batch insert them into the database in a single transaction (even packing 1000s of CVEs at a time).
INSERT INTO cves (columns)
VALUES
(cve_1),
(cve_2),
...
(cve_n)
RETURNING * Which will result in a single BEGIN/COMMIT per chunk rather than multiple per-CVE. The relational properties are still held within the transaction itself.