tree 56d8b3dad4dae3a65ce60bf0c0593e0c527d24a8
parent b8f1b30af72be8a6d9023f27c478559b4df53523
author George Karpenkov <cheshire@google.com> 1572460267 -0700
committer TensorFlower Gardener <gardener@tensorflow.org> 1572467051 -0700

[XLA] CPU AllReduce implementation: same process, all work done by "main" worker

Only "virtual" devices are supported which share the same thread pool.
Communication is performed via a global map.
This all-reduce implementation is not actually "collective": all the work is
performed by the main worker, while all the other threads are waiting.

PiperOrigin-RevId: 277548660
Change-Id: Iccd438dbe81329a02a157d9aae72d23c706b9f52