tree 913cf1f7d98c629a6a0d4efa0ef8e778977b4582
parent 4c4fe4b8e29044d681c423807bcf38dab56e2a75
author Ran Chen <crccw@google.com> 1605126175 -0800
committer TensorFlower Gardener <gardener@tensorflow.org> 1605126934 -0800

Switch to V2 collectives

There're still some performance issues, which don't seem to be blocking:
1) we don't have scoped allocator for v2 collective ops. The non scoped allocator concat/split adds about 4ms step time to bert pretrain
2) instance params is effectively leaked, but the effect is hidden by a far more serious leak from ScopedAllocatorContainer.
3) The overhead capture_call_time_value is a function of the number of collectives, and can cause serious performance issues if there're >100 collectives, which is uncommon.

PiperOrigin-RevId: 341885335
Change-Id: If8f773847b18fd5ae37a00e3a4c2563ec913b907