Create only one instance of SigridTransform in DPerExample.

Summary:
DPer example have been creating multiple copies of the transform config in net
defition till this moment, that resulted in the fact that I've hit the limit of
ProtoBuf (64MB) for a certain Task requests (especially visible because of the
ValidationPipeline that I was adding).

After this diff we're going to store SigridTransforms in one instance per
machine for training (or 1 instance per reading).

Difference in sizes of the plans for some simple SparseNN model ~30 MB (even including the fact that second model have validation plan as well).

TODO: Do similar logic for NNPreProc as well (it's also pretty large).

Reviewed By: dzhulgakov

Differential Revision: D4441441

fbshipit-source-id: 4452dd86a4dc49b2c7f5b7642f443aed5720b047
2 files changed
tree: 6d8a7a2e9d92d3add14e1b18b3640c8891504c18
  1. .travis/
  2. caffe/
  3. caffe2/
  4. cmake/
  5. docs/
  6. third_party/
  7. .Doxyfile
  8. .gitignore
  9. .gitmodules
  10. .travis.yml
  11. CMakeLists.txt
  12. LICENSE
  13. Makefile
  14. README.md
README.md

Caffe2

Caffe2 is a deep learning framework made with expression, speed, and modularity in mind. It is an experimental refactoring of Caffe, and allows a more flexible way to organize computation.

License and Citation

Caffe2 is released under the BSD 2-Clause license.

Building Caffe2

Build Status

git clone --recursive https://github.com/bwasti/caffe2.git
cd caffe2

OS X

brew install automake protobuf
mkdir build && cd build
cmake ..
make

Ubuntu

sudo apt-get install libprotobuf-dev protobuf-compiler libatlas-base-dev libgoogle-glog-dev libgtest-dev liblmdb-dev libleveldb-dev libsnappy-dev python-dev python-pip libiomp-dev libopencv-dev libpthread-stubs0-dev cmake
sudo pip install numpy
wget http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1404/x86_64/cuda-repo-ubuntu1404_8.0.44-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu1404_8.0.44-1_amd64.deb
sudo apt-get update
sudo apt-get install cuda
sudo apt-get install git

CUDNN_URL="http://developer.download.nvidia.com/compute/redist/cudnn/v5.1/cudnn-8.0-linux-x64-v5.1.tgz" &&
curl -fsSL ${CUDNN_URL} -O &&
sudo tar -xzf cudnn-8.0-linux-x64-v5.1.tgz -C /usr/local &&
rm cudnn-8.0-linux-x64-v5.1.tgz &&
sudo ldconfig

mkdir build && cd build
cmake ..
make

Python support

To run the tutorials you'll need ipython-notebooks and matplotlib, which can be installed on OS X with:

brew install matplotlib --with-python3
pip install ipython notebook

Build status (known working)

Ubuntu 14.04 (GCC)

  • [x] Default CPU build
  • [x] Default GPU build

OS X (Clang)

  • [x] Default CPU build
  • [x] Default GPU build

Options (both Clang and GCC)

  • [ ] Nervana GPU
  • [ ] ZMQ
  • [ ] RocksDB
  • [ ] MPI
  • [ ] OpenMP
  • [x] No LMDB
  • [x] No LevelDB
  • [x] No OpenCV

BLAS

  • [x] OpenBLAS
  • [x] ATLAS
  • [ ] MKL

Other

  • [x] CMake 2.8 support
  • [x] List of dependencies for Ubuntu 14.04
  • [x] List of dependencies for OS X