Is there a way to have the training completed on multiple computers and then summarize the results

[achinastone]

Is there a way to have the training completed on multiple computers and then summarize the results

[joaopauloschuler]

Not yet. At this moment, the max you can do is to run on a machine with 64 cores. If you chose this path, you will set batch sizes of multiples of 64. I would suggest batch sizes of 256 at minimum.

[joaopauloschuler]

I updated this ticket as an enhancement (feature) request.

[achinastone]

You are great.

[achinastone]

I recommend using NNG for cross-process and cross-host parallel training.
https://github.com/nanomsg/nng
https://github.com/zedalaye/Delphi-NNG