docs/source/nn.rst

.. role:: hidden
    :class: hidden-section

torch.nn
===================================

.. automodule:: torch.nn
.. currentmodule:: torch.nn

Parameters
----------

.. autoclass:: Parameter
    :members:

Containers
----------------------------------

:hidden:`Module`
~~~~~~~~~~~~~~~~

.. autoclass:: Module
    :members:

:hidden:`Sequential`
~~~~~~~~~~~~~~~~~~~~

.. autoclass:: Sequential
    :members:

:hidden:`ModuleList`
~~~~~~~~~~~~~~~~~~~~

.. autoclass:: ModuleList
    :members:

:hidden:`ModuleDict`
~~~~~~~~~~~~~~~~~~~~

.. autoclass:: ModuleDict
    :members:

:hidden:`ParameterList`
~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: ParameterList
    :members:

:hidden:`ParameterDict`
~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: ParameterDict
    :members:

Convolution layers
----------------------------------

:hidden:`Conv1d`
~~~~~~~~~~~~~~~~

.. autoclass:: Conv1d
    :members:

:hidden:`Conv2d`
~~~~~~~~~~~~~~~~

.. autoclass:: Conv2d
    :members:

:hidden:`Conv3d`
~~~~~~~~~~~~~~~~

.. autoclass:: Conv3d
    :members:

:hidden:`ConvTranspose1d`
~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: ConvTranspose1d
    :members:

:hidden:`ConvTranspose2d`
~~~~~~~~~~~~~~~~~~~~~~~~~


.. autoclass:: ConvTranspose2d
    :members:

:hidden:`ConvTranspose3d`
~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: ConvTranspose3d
    :members:

:hidden:`Unfold`
~~~~~~~~~~~~~~~~

.. autoclass:: Unfold
    :members:

:hidden:`Fold`
~~~~~~~~~~~~~~

.. autoclass:: Fold
    :members:


Pooling layers
----------------------------------

:hidden:`MaxPool1d`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: MaxPool1d
    :members:

:hidden:`MaxPool2d`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: MaxPool2d
    :members:

:hidden:`MaxPool3d`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: MaxPool3d
    :members:

:hidden:`MaxUnpool1d`
~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: MaxUnpool1d
    :members:

:hidden:`MaxUnpool2d`
~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: MaxUnpool2d
    :members:

:hidden:`MaxUnpool3d`
~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: MaxUnpool3d
    :members:

:hidden:`AvgPool1d`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: AvgPool1d
    :members:

:hidden:`AvgPool2d`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: AvgPool2d
    :members:

:hidden:`AvgPool3d`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: AvgPool3d
    :members:

:hidden:`FractionalMaxPool2d`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: FractionalMaxPool2d
    :members:

:hidden:`LPPool1d`
~~~~~~~~~~~~~~~~~~

.. autoclass:: LPPool1d
    :members:

:hidden:`LPPool2d`
~~~~~~~~~~~~~~~~~~

.. autoclass:: LPPool2d
    :members:

:hidden:`AdaptiveMaxPool1d`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: AdaptiveMaxPool1d
    :members:

:hidden:`AdaptiveMaxPool2d`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: AdaptiveMaxPool2d
    :members:

:hidden:`AdaptiveMaxPool3d`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: AdaptiveMaxPool3d
    :members:

:hidden:`AdaptiveAvgPool1d`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: AdaptiveAvgPool1d
    :members:

:hidden:`AdaptiveAvgPool2d`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: AdaptiveAvgPool2d
    :members:

:hidden:`AdaptiveAvgPool3d`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: AdaptiveAvgPool3d
    :members:


Padding layers
--------------

:hidden:`ReflectionPad1d`
~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: ReflectionPad1d
    :members:

:hidden:`ReflectionPad2d`
~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: ReflectionPad2d
    :members:

:hidden:`ReplicationPad1d`
~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: ReplicationPad1d
    :members:

:hidden:`ReplicationPad2d`
~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: ReplicationPad2d
    :members:

:hidden:`ReplicationPad3d`
~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: ReplicationPad3d
    :members:

:hidden:`ZeroPad2d`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: ZeroPad2d
    :members:

:hidden:`ConstantPad1d`
~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: ConstantPad1d
    :members:

:hidden:`ConstantPad2d`
~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: ConstantPad2d
    :members:

:hidden:`ConstantPad3d`
~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: ConstantPad3d
    :members:


Non-linear activations (weighted sum, nonlinearity)
---------------------------------------------------

:hidden:`ELU`
~~~~~~~~~~~~~

.. autoclass:: ELU
    :members:

:hidden:`Hardshrink`
~~~~~~~~~~~~~~~~~~~~

.. autoclass:: Hardshrink
    :members:

:hidden:`Hardtanh`
~~~~~~~~~~~~~~~~~~

.. autoclass:: Hardtanh
    :members:

:hidden:`LeakyReLU`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: LeakyReLU
    :members:

:hidden:`LogSigmoid`
~~~~~~~~~~~~~~~~~~~~

.. autoclass:: LogSigmoid
    :members:

:hidden:`MultiheadAttention`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. autoclass:: MultiheadAttention
    :members:

:hidden:`PReLU`
~~~~~~~~~~~~~~~

.. autoclass:: PReLU
    :members:

:hidden:`ReLU`
~~~~~~~~~~~~~~

.. autoclass:: ReLU
    :members:

:hidden:`ReLU6`
~~~~~~~~~~~~~~~

.. autoclass:: ReLU6
    :members:

:hidden:`RReLU`
~~~~~~~~~~~~~~~

.. autoclass:: RReLU
    :members:

:hidden:`SELU`
~~~~~~~~~~~~~~

.. autoclass:: SELU
    :members:

:hidden:`CELU`
~~~~~~~~~~~~~~

.. autoclass:: CELU
    :members:

:hidden:`Sigmoid`
~~~~~~~~~~~~~~~~~

.. autoclass:: Sigmoid
    :members:

:hidden:`Softplus`
~~~~~~~~~~~~~~~~~~

.. autoclass:: Softplus
    :members:

:hidden:`Softshrink`
~~~~~~~~~~~~~~~~~~~~

.. autoclass:: Softshrink
    :members:

:hidden:`Softsign`
~~~~~~~~~~~~~~~~~~

.. autoclass:: Softsign
    :members:

:hidden:`Tanh`
~~~~~~~~~~~~~~

.. autoclass:: Tanh
    :members:

:hidden:`Tanhshrink`
~~~~~~~~~~~~~~~~~~~~

.. autoclass:: Tanhshrink
    :members:

:hidden:`Threshold`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: Threshold
    :members:

Non-linear activations (other)
------------------------------

:hidden:`Softmin`
~~~~~~~~~~~~~~~~~

.. autoclass:: Softmin
    :members:

:hidden:`Softmax`
~~~~~~~~~~~~~~~~~

.. autoclass:: Softmax
    :members:

:hidden:`Softmax2d`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: Softmax2d
    :members:

:hidden:`LogSoftmax`
~~~~~~~~~~~~~~~~~~~~

.. autoclass:: LogSoftmax
    :members:

:hidden:`AdaptiveLogSoftmaxWithLoss`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: AdaptiveLogSoftmaxWithLoss
    :members:

Normalization layers
----------------------------------

:hidden:`BatchNorm1d`
~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: BatchNorm1d
    :members:

:hidden:`BatchNorm2d`
~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: BatchNorm2d
    :members:

:hidden:`BatchNorm3d`
~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: BatchNorm3d
    :members:

:hidden:`GroupNorm`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: GroupNorm
    :members:

:hidden:`SyncBatchNorm`
~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: SyncBatchNorm
    :members:

:hidden:`InstanceNorm1d`
~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: InstanceNorm1d
    :members:

:hidden:`InstanceNorm2d`
~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: InstanceNorm2d
    :members:

:hidden:`InstanceNorm3d`
~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: InstanceNorm3d
    :members:

:hidden:`LayerNorm`
~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: LayerNorm
    :members:

:hidden:`LocalResponseNorm`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: LocalResponseNorm
    :members:

Recurrent layers
----------------------------------

:hidden:`RNN`
~~~~~~~~~~~~~

.. autoclass:: RNN
    :members:

:hidden:`LSTM`
~~~~~~~~~~~~~~

.. autoclass:: LSTM
    :members:

:hidden:`GRU`
~~~~~~~~~~~~~

.. autoclass:: GRU
    :members:

:hidden:`RNNCell`
~~~~~~~~~~~~~~~~~

.. autoclass:: RNNCell
    :members:

:hidden:`LSTMCell`
~~~~~~~~~~~~~~~~~~

.. autoclass:: LSTMCell
    :members:

:hidden:`GRUCell`
~~~~~~~~~~~~~~~~~

.. autoclass:: GRUCell
    :members:

Transformer layers
----------------------------------

:hidden:`Transformer`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: Transformer
    :members:

:hidden:`TransformerEncoder`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: TransformerEncoder
    :members:

:hidden:`TransformerDecoder`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: TransformerDecoder
    :members:

:hidden:`TransformerEncoderLayer`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: TransformerEncoderLayer
    :members:

:hidden:`TransformerDecoderLayer`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: TransformerDecoderLayer
    :members:

Linear layers
----------------------------------

:hidden:`Identity`
~~~~~~~~~~~~~~~~~~

.. autoclass:: Identity
    :members:

:hidden:`Linear`
~~~~~~~~~~~~~~~~

.. autoclass:: Linear
    :members:

:hidden:`Bilinear`
~~~~~~~~~~~~~~~~~~

.. autoclass:: Bilinear
    :members:

Dropout layers
----------------------------------

:hidden:`Dropout`
~~~~~~~~~~~~~~~~~

.. autoclass:: Dropout
    :members:

:hidden:`Dropout2d`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: Dropout2d
    :members:

:hidden:`Dropout3d`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: Dropout3d
    :members:

:hidden:`AlphaDropout`
~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: AlphaDropout
    :members:


Sparse layers
----------------------------------

:hidden:`Embedding`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: Embedding
    :members:

:hidden:`EmbeddingBag`
~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: EmbeddingBag
    :members:

Distance functions
----------------------------------

:hidden:`CosineSimilarity`
~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: CosineSimilarity
    :members:

:hidden:`PairwiseDistance`
~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: PairwiseDistance
    :members:


Loss functions
----------------------------------

:hidden:`L1Loss`
~~~~~~~~~~~~~~~~

.. autoclass:: L1Loss
    :members:

:hidden:`MSELoss`
~~~~~~~~~~~~~~~~~

.. autoclass:: MSELoss
    :members:

:hidden:`CrossEntropyLoss`
~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: CrossEntropyLoss
    :members:

:hidden:`CTCLoss`
~~~~~~~~~~~~~~~~~

.. autoclass:: CTCLoss
    :members:

:hidden:`NLLLoss`
~~~~~~~~~~~~~~~~~

.. autoclass:: NLLLoss
    :members:

:hidden:`PoissonNLLLoss`
~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: PoissonNLLLoss
    :members:

:hidden:`KLDivLoss`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: KLDivLoss
    :members:

:hidden:`BCELoss`
~~~~~~~~~~~~~~~~~~~

.. autoclass:: BCELoss
    :members:

:hidden:`BCEWithLogitsLoss`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: BCEWithLogitsLoss
    :members:

:hidden:`MarginRankingLoss`
~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: MarginRankingLoss
    :members:

:hidden:`HingeEmbeddingLoss`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: HingeEmbeddingLoss
    :members:

:hidden:`MultiLabelMarginLoss`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: MultiLabelMarginLoss
    :members:

:hidden:`SmoothL1Loss`
~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: SmoothL1Loss
    :members:

:hidden:`SoftMarginLoss`
~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: SoftMarginLoss
    :members:

:hidden:`MultiLabelSoftMarginLoss`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: MultiLabelSoftMarginLoss
    :members:

:hidden:`CosineEmbeddingLoss`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: CosineEmbeddingLoss
    :members:

:hidden:`MultiMarginLoss`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: MultiMarginLoss
    :members:

:hidden:`TripletMarginLoss`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: TripletMarginLoss
    :members:


Vision layers
----------------

:hidden:`PixelShuffle`
~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: PixelShuffle
    :members:

:hidden:`Upsample`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: Upsample
    :members:

:hidden:`UpsamplingNearest2d`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: UpsamplingNearest2d
    :members:

:hidden:`UpsamplingBilinear2d`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: UpsamplingBilinear2d
    :members:


DataParallel layers (multi-GPU, distributed)
--------------------------------------------

:hidden:`DataParallel`
~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: DataParallel
    :members:

:hidden:`DistributedDataParallel`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autoclass:: torch.nn.parallel.DistributedDataParallel
    :members:


Utilities
---------

:hidden:`clip_grad_norm_`
~~~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: torch.nn.utils.clip_grad_norm_

:hidden:`clip_grad_value_`
~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: torch.nn.utils.clip_grad_value_

:hidden:`parameters_to_vector`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: torch.nn.utils.parameters_to_vector

:hidden:`vector_to_parameters`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: torch.nn.utils.vector_to_parameters

:hidden:`weight_norm`
~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: torch.nn.utils.weight_norm

:hidden:`remove_weight_norm`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: torch.nn.utils.remove_weight_norm

:hidden:`spectral_norm`
~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: torch.nn.utils.spectral_norm

:hidden:`remove_spectral_norm`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: torch.nn.utils.remove_spectral_norm


.. currentmodule:: torch.nn.utils.rnn

:hidden:`PackedSequence`
~~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: torch.nn.utils.rnn.PackedSequence


:hidden:`pack_padded_sequence`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: torch.nn.utils.rnn.pack_padded_sequence


:hidden:`pad_packed_sequence`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: torch.nn.utils.rnn.pad_packed_sequence


:hidden:`pad_sequence`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: torch.nn.utils.rnn.pad_sequence


:hidden:`pack_sequence`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: torch.nn.utils.rnn.pack_sequence

.. currentmodule:: torch.nn

:hidden:`Flatten`
~~~~~~~~~~~~~~~~~

.. autoclass:: Flatten
    :members: