| commit | fc7f0269808581499571c5db8af87311c943cd4e | [log] [tgz] |
|---|---|---|
| author | Pieter Noordhuis <pietern@fb.com> | Tue Feb 28 10:37:03 2017 -0800 |
| committer | Pieter Noordhuis <pietern@devgpu244.prn2.facebook.com> | Thu Mar 02 10:16:38 2017 -0800 |
| tree | e0ce08ffc7d6c5947fb5a741d9ceb762759c3955 | |
| parent | 9f18f83375538fe8bbfb98f8c3bc965b931ed4a0 [diff] |
Refactor ibverbs transport to prepare for sync mode Summary: All pairs created by a device would use the same completion queue. Supporting sync mode that way is difficult, as there is no way to filter completions for a particular pair. This change refactors this to use a single completion queue per pair so that this is no longer an issue. This change is a preparation for supporting synchronous mode (where the calling thread itself will poll the ibv library for completions instead of the device thread). This change also includes a refactoring of the way transient memory regions are handled so that they are properly deregistered and deallocated when no longer needed. Reviewed By: andrewwdye Differential Revision: D4625146 fbshipit-source-id: 21bf5ab321534fbd5c03f12049c10fc67da68944
Gloo is a collective communications library. It comes with a number of collective algorithms useful for machine learning applications. These include a barrier, broadcast, and allreduce.
Transport of data between participating machines is abstracted so that IP can be used at all times, or InifiniBand (or RoCE) when available.
Where applicable, algorithms have an implementation that works with system memory buffers, and one that works with NVIDIA GPU memory buffers. In the latter case, if the InfiniBand transport is used, GPUDirect can be used to accelerate cross machine GPU-to-GPU memory transfers.
Gloo is built to run on Linux and has no hard dependencies other than libc.
Optional dependencies are:
You can build Gloo using CMake.
Since it is a library, it is most convenient to vendor it in your own project and include the project root in your own CMake configuration.
For standalone builds (e.g. to run tests or benchmarks), first populate the third-party directory with a few dependencies to compile both the tests and the benchmark tool:
cd third-party ./fetch.sh
Then, to build:
mkdir build cd build cmake ../ -DBUILD_TEST=1 -DBUILD_BENCHMARK=1 ls -l gloo/gloo_{test,benchmark}
Please refer to docs/ for detailed documentation.
Gloo is BSD-licensed. We also provide an additional patent grant.