swishLayer

Swish layer

Since R2021a

Description

A swish activation layer applies the swish function on the layer inputs.

The swish operation is given by $$f(x)=\frac{x}{1+e^{-x}}$$.

Creation

Syntax

layer = swishLayer

layer = swishLayer('Name',Name)

Description

layer = swishLayer creates a swish layer.

layer = swishLayer('Name',Name) creates a swish layer and sets the optional Name property using a name-value argument. For example, swishLayer('Name','swish1') creates a swish layer with the name 'swish1'.

Properties

Name — Layer name

"" (default) | character vector | string scalar

Layer name, specified as a character vector or a string scalar. For Layer array input, the trainnet, trainNetwork, assembleNetwork, layerGraph, and dlnetwork functions automatically assign names to layers with the name "".

The SwishLayer object stores this property as a character vector.

Data Types: char | string

NumInputs — Number of inputs

1 (default)

This property is read-only.

Number of inputs to the layer, returned as 1. This layer accepts a single input only.

Data Types: double

InputNames — Input names

{'in'} (default)

This property is read-only.

Input names, returned as {'in'}. This layer accepts a single input only.

Data Types: cell

NumOutputs — Number of outputs

1 (default)

This property is read-only.

Number of outputs from the layer, returned as 1. This layer has a single output only.

Data Types: double

OutputNames — Output names

{'out'} (default)

This property is read-only.

Output names, returned as {'out'}. This layer has a single output only.

Data Types: cell

Examples

Create Swish Layer

Create a swish layer with the name 'swish1'.

layer = swishLayer('Name','swish1')

layer = 
  SwishLayer with properties:

    Name: 'swish1'

   Learnable Parameters
    No properties.

   State Parameters
    No properties.

Use properties method to see a list of all properties.

Include a swish layer in a Layer array.

layers = [ ...
    imageInputLayer([28 28 1])
    convolution2dLayer(5,20)
    batchNormalizationLayer
    swishLayer
    maxPooling2dLayer(2,'Stride',2)
    fullyConnectedLayer(10)
    softmaxLayer
    classificationLayer]

layers = 
  8x1 Layer array with layers:

     1   ''   Image Input             28x28x1 images with 'zerocenter' normalization
     2   ''   2-D Convolution         20 5x5 convolutions with stride [1  1] and padding [0  0  0  0]
     3   ''   Batch Normalization     Batch normalization
     4   ''   Swish                   Swish
     5   ''   2-D Max Pooling         2x2 max pooling with stride [2  2] and padding [0  0  0  0]
     6   ''   Fully Connected         10 fully connected layer
     7   ''   Softmax                 softmax
     8   ''   Classification Output   crossentropyex

Algorithms

Swish Layer

A swish activation layer applies the swish function on the layer inputs. The swish operation is given by $$f(x)=\frac{x}{1+e^{-x}}$$. The swish layer does not change the size of its input.

Activation layers such as swish layers improve the training accuracy for some applications and usually follow convolution and normalization layers. Other nonlinear activation layers perform different operations. For a list of activation layers, see Activation Layers.

Layer Input and Output Formats

Layers in a layer array or layer graph pass data to subsequent layers as formatted dlarray objects. The format of a dlarray object is a string of characters, in which each character describes the corresponding dimension of the data. The formats consists of one or more of these characters:

• "S" — Spatial

• "C" — Channel

• "B" — Batch

• "T" — Time

• "U" — Unspecified

For example, 2-D image data represented as a 4-D array, where the first two dimensions correspond to the spatial dimensions of the images, the third dimension corresponds to the channels of the images, and the fourth dimension corresponds to the batch dimension, can be described as having the format "SSCB" (spatial, spatial, channel, batch).

SwishLayer objects apply an element-wise operation and supports input data of any format. The layer and does not add or remove any dimensions, so it outputs data with the same format as its inputs.

Extended Capabilities

C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.

GPU Code Generation
Generate CUDA® code for NVIDIA® GPUs using GPU Coder™.

Version History

Introduced in R2021a