tree b88ed714bc51ee6997e2705e7fcc5085a7201905
parent ea0535f29636c6f0487e033915c84e96fad516b4
author Peng Sun <peng.sun@arm.com> 1587583686 -0700
committer TensorFlower Gardener <gardener@tensorflow.org> 1587584026 -0700

PR #34903: TransposeConv with Bias

Imported from GitHub PR https://github.com/tensorflow/tensorflow/pull/34903

### Description of issue:
For generating .tflite file with TFLiteConverter, when model contains
Conv2DTranspose layers, bias cannot fold into Operator TRANSPOSECONV.
It will result with extra Op ADD following Op TRANSPOSECONV.
But with other CONV-like layers (Conv2D, DepthwiseConv2D),
bias will be fold into CONV layer.
(check detailed TF issue: https://github.com/tensorflow/tensorflow/issues/34622)

### PR try to resolve it:
So we try to resolve this issue by enable TransposeConv with bias for TFLite:

- Update TFLite graph_transform features with:
      fill TransposeConv bias with zero if there is no bias
      fuse bias add into preceding TransposeConv(TEST added)

- Update TransposeConv with bias:
      add bias input to TransposeConv
      add optional bias to TransposeConv kernels

### example of the results:
  TRANSPOSE_CONV inputs:

1. output_shape
2. weights
3. activation
4. bias

![fused_transposeconv](https://user-images.githubusercontent.com/55463253/70334128-bc088200-183c-11ea-9f94-a803cc80df99.png)

### Need to discuss:
~~currently this PR only update reference kernel for transpose_conv, optimised kernal is commented out.~~
~~several TEST need to be added as well, but~~ further suggestions are needed for adding additional test.
Copybara import of the project:

--
1c6eb9c98229a9e8248dc1fe913a20cc6dd89332 by Peng Sun <peng.sun@arm.com>:

Fuse TransposeConv with Bias

For generating .tflite file with TFLiteConverter, when model contains
Conv2DTranspose layers, bias cannot fold into Operator TRANSPOSECONV.
It will result with extra Op ADD following Op TRANSPOSECONV.
But with other CONV-like layers (Conv2D, DepthwiseConv2D),
bias will be fold into CONV layer.
(check TF issue: https://github.com/tensorflow/tensorflow/issues/34622)

So we try to resolve this issue by enable TransposeConv with bias for TFLite:
  Update TFLite graph_transform features with:
    fill TransposeConv bias with zero if there is no bias
    fuse bias add into preceding TransposeConv(TEST added)
  Update TransposeConv with bias:
    add bias input to TransposeConv
    add optional bias to TransposeConv kernels(TEST added)

--
22611b880c94eb753c88a0a3e2977200e55ebd2c by Peng Sun <peng.sun@arm.com>:

clang-format with google style.

COPYBARA_INTEGRATE_REVIEW=https://github.com/tensorflow/tensorflow/pull/34903 from psunn:TransposeConvWithBias 22611b880c94eb753c88a0a3e2977200e55ebd2c
PiperOrigin-RevId: 307872447
Change-Id: I367fcd65f2662f4c7846d37bc69dc43670c83961