Google Git
Sign in
android / platform / external / tensorflow / 2bebdba6d52
commit2bebdba6d527691580a88d05b3fae94e56ab0e7d[log] [tgz]
authorFaijul Amin <md.faijul.amin@intel.com>Thu Dec 16 19:41:09 2021 -0800
committerTensorFlower Gardener <gardener@tensorflow.org>Thu Dec 16 19:46:38 2021 -0800
tree6fba2033726597a3dfb2eb0624f947030ed958a4
parentf0147751fd5d2ff23251149ebad9af9f03010732 [diff]
PR #51903: [oneDNN] Keras LayerNormalization fusion with oneDNN CPU backend

Imported from GitHub PR https://github.com/tensorflow/tensorflow/pull/51903

Keras LayerNormalization API creates a set op smaller ops that can be realized by a single operation with oneDNN library on CPU. This PR fuses smaller ops into a single op using grappler remapper optimizer. Current fusion is restricted to the scenario when LayerNormalization uses FusedBatchNorm and the input tensor is 2D/3D.

This fusion improves performance for inference with Transformer based language models.
Copybara import of the project:

--
a366688dd39456c43925bd9a8180260a0b711947 by mdfaijul <md.faijul.amin@intel.com>:

Enabled LayerNorm with 2D/3D inputs

--
85c5dc9da9909c234ddb78fdb28b38b6b025ee26 by mdfaijul <md.faijul.amin@intel.com>:

Drop python keras layer norm test

--
65507d1e55c3baa21dcd513a594bdbd834d09ec4 by mdfaijul <md.faijul.amin@intel.com>:

Addressed comment.

COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/tensorflow/pull/51903 from Intel-tensorflow:amin/layer-norm 65507d1e55c3baa21dcd513a594bdbd834d09ec4
PiperOrigin-RevId: 416944331
Change-Id: Id9b56da3bdd14b61e66185acd03bf45ff9b8b7f1
5 files changed
tree: 6fba2033726597a3dfb2eb0624f947030ed958a4
  1. .github/
  2. tensorflow/
  3. third_party/
  4. tools/
  5. .pylintrc ⇨ tensorflow/tools/ci_build/pylintrc
  6. .bazelrc
  7. .bazelversion
  8. .clang-format
  9. .gitignore
  10. .zenodo.json
  11. arm_compiler.BUILD
  12. AUTHORS
  13. BUILD
  14. CITATION.cff
  15. CODE_OF_CONDUCT.md
  16. CODEOWNERS
  17. configure
  18. configure.cmd
  19. configure.py
  20. CONTRIBUTING.md
  21. ISSUE_TEMPLATE.md
  22. ISSUES.md
  23. LICENSE
  24. models.BUILD
  25. README.md
  26. RELEASE.md
  27. SECURITY.md
  28. WORKSPACE
README.md

Python PyPI DOI

Documentation
Documentation

TensorFlow is an end-to-end open source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries, and community resources that lets researchers push the state-of-the-art in ML and developers easily build and deploy ML-powered applications.

TensorFlow was originally developed by researchers and engineers working on the Google Brain team within Google's Machine Intelligence Research organization to conduct machine learning and deep neural networks research. The system is general enough to be applicable in a wide variety of other domains, as well.

TensorFlow provides stable Python and C++ APIs, as well as non-guaranteed backward compatible API for other languages.

Keep up-to-date with release announcements and security updates by subscribing to announce@tensorflow.org. See all the mailing lists.

Install

See the TensorFlow install guide for the pip package, to enable GPU support, use a Docker container, and build from source.

To install the current release, which includes support for CUDA-enabled GPU cards (Ubuntu and Windows):

$ pip install tensorflow

A smaller CPU-only package is also available:

$ pip install tensorflow-cpu

To update TensorFlow to the latest version, add --upgrade flag to the above commands.

Nightly binaries are available for testing using the tf-nightly and tf-nightly-cpu packages on PyPi.

Try your first TensorFlow program

$ python

>>> import tensorflow as tf
>>> tf.add(1, 2).numpy()
3
>>> hello = tf.constant('Hello, TensorFlow!')
>>> hello.numpy()
b'Hello, TensorFlow!'

For more examples, see the TensorFlow tutorials.

Contribution guidelines

If you want to contribute to TensorFlow, be sure to review the contribution guidelines. This project adheres to TensorFlow's code of conduct. By participating, you are expected to uphold this code.

We use GitHub issues for tracking requests and bugs, please see TensorFlow Discuss for general questions and discussion, and please direct specific questions to Stack Overflow.

The TensorFlow project strives to abide by generally accepted best practices in open-source software development:

Fuzzing Status CII Best Practices Contributor Covenant

Continuous build status

You can find more community-supported platforms and configurations in the TensorFlow SIG Build community builds table.

Official Builds

Build TypeStatusArtifacts
Linux CPUStatusPyPI
Linux GPUStatusPyPI
Linux XLAStatusTBA
macOSStatusPyPI
Windows CPUStatusPyPI
Windows GPUStatusPyPI
AndroidStatusDownload
Raspberry Pi 0 and 1StatusPy3
Raspberry Pi 2 and 3StatusPy3
Libtensorflow MacOS CPUStatus Temporarily UnavailableNightly Binary Official GCS
Libtensorflow Linux CPUStatus Temporarily UnavailableNightly Binary Official GCS
Libtensorflow Linux GPUStatus Temporarily UnavailableNightly Binary Official GCS
Libtensorflow Windows CPUStatus Temporarily UnavailableNightly Binary Official GCS
Libtensorflow Windows GPUStatus Temporarily UnavailableNightly Binary Official GCS

Resources

Learn more about the TensorFlow community and how to contribute.

License

Apache License 2.0