Traditional reconciliation approaches such as in this library are hard to scale because the more complex techniques require all series available for matrix algebra.
Options for scaling could be:
- Batch-wise processing, perform reconciliation iteratively. Unclear how to achieve coherence in general, and potentially compute intensive.
- Torch-based reconciliation (rewrite Numpy to Torch), enables accelerators and potential batching. Disadvantage: sparse methods would be not / poorly supported
- Dask's numpy backend
- ....
Traditional reconciliation approaches such as in this library are hard to scale because the more complex techniques require all series available for matrix algebra.
Options for scaling could be: