implement gather

[mahmudhera]

• implement sourmash gather
• edit order in which files are processed
• test if equivalent results are obtained
• test equivalency for multiple instances
• add test commands and required data (in CI)
• ensure tests are passing

[mahmudhera]

This is implemented, but is not fast enough. We may need to do some engineering here. CPU time suggests that we can make it fast enough, but needs to be implemented.

Command being timed: "sourmash scripts fastgather query_sketch.sig filelist_10k -t 0 -k 51 -o temp -c 128"
        User time (seconds): 40.64
        System time (seconds): 1.47
        Percent of CPU this job got: 1914%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.20
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 367328
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 5
        Minor (reclaiming a frame) page faults: 86043
        Voluntary context switches: 23767
        Involuntary context switches: 3504
        Swaps: 0
        File system inputs: 0
        File system outputs: 40
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

Command being timed: "gather query_sketch.sig filelist_10k index_10k temp -t 128 -b 0"
        User time (seconds): 29.46
        System time (seconds): 5.71
        Percent of CPU this job got: 272%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:12.92
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 1834012
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 509451
        Voluntary context switches: 171886
        Involuntary context switches: 2177
        Swaps: 0
        File system inputs: 0
        File system outputs: 16
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0