Cannot assemble a convolutional network with output sequence length less than input sequence length

I have defined a simple CNN with one convolutional layer. When the stride is set to "1" and the "padding" is set to "same" the network can be assembled using assembleNetwork. However, when the stride is set to something greater than one (I have set it to "2" in the snippet below), or the padding is set to [0 0] (i.e. no padding), assembleNetwork complains that:

Network: Incompatible input and output sequence lengths. The network must return sequences with the same length as the input data or a sequence with length one.

This is correct; the output sequence is shorter than the input sequence by design. One can imagine a network in which a longer stride is used to reduce computational burden, and it is expected that the output will be shorter than the input... and the network can be built and trained in Keras/Tensorflow (in fact, the problem was initiallly encountered when using the importKerasNetwork function on a similar network).

Here's the code for the simple CNN:

%% a minimal cnn
clc;
clearvars;
% input layer
layers(1) = sequenceInputLayer(1, MinLength=8);
% convolutional layer spec
filter_length = 4;
num_channels = 1;
num_filters = 1;
% Layer weights for the transposed convolution operation, specified as a FilterSize-by-NumChannels-by-numFilters numeric array. (Matlab docs: convolution1dLayer)
w = reshape(1:filter_length * num_channels * num_filters, filter_length, num_channels, num_filters);
% Layer biases for the transposed convolutional operation, specified as a 1-by-NumFilters numeric array. (Matlab docs: convolution1dLayer)
b = reshape(1:num_filters(1), 1, num_filters(1));
% conv1-1
layers(2) = convolution1dLayer(filter_length(1), num_filters(1), Name='conv1', NumChannels=num_channels(1), Stride=2, Padding='same', Weights=w, Bias=b);
% activation
layers(3) = reluLayer;
% output
layers(4) = regressionLayer;
% net
net = assembleNetwork(layers);

I have tried using different input layer types (which are not subject to this sequence length check) to no avail. Neither the featureInputLayer or ImageInputLayer are accepted as valid inputs to a convolution1DLayer. I have also tacked on a fullyConnectedLayer at the end to reduce the sequence length to 1, and this does work, but is not the network I want.

Thanks in advance for any feedback. I'll be curious to see if others have run into the same issue.

Answer 1

[David Ho]

Hello bwaite56,

Unfortunately the "assembleNetwork" and "trainNetwork" functions don't currently support sequence-to-sequence regression where the output and input sequences have different lengths.

There are two possible workarounds that I can think of:

1) If all your sequences are the same length (or can be padded so they are), you can use an imageInputLayer where one of the input dimensions has size 1. You can then replace the convolution1dLayer with a convolution2dLayer, but as one dimension is singleton this should function as a 1-D convnet. Here's an adaptation of your code that represents the sequences as 1x8 images:

clearvars;
% input layer
numTimesteps = 8;
layers(1) = imageInputLayer([1 numTimesteps], "Normalization","none");
% convolutional layer spec
filter_size = [1 4];
num_channels = 1;
num_filters = 1;
% Layer weights for the transposed convolution operation, specified as a FilterSize-by-NumChannels-by-numFilters numeric array. (Matlab docs: convolution1dLayer)
w = reshape(1:filter_size(2) * num_channels * num_filters, filter_size(1), filter_size(2), num_channels, num_filters);
% Layer biases for the transposed convolutional operation, specified as a 1-by-NumFilters numeric array. (Matlab docs: convolution1dLayer)
b = reshape(1:num_filters(1), 1, num_filters(1));
% conv1-1
layers(2) = convolution2dLayer(filter_size, num_filters, Name='conv1', NumChannels=num_channels, Stride=2, Padding='same', Weights=w, Bias=b);
% activation
layers(3) = reluLayer;
% output
layers(4) = regressionLayer;
% net
net = assembleNetwork(layers);

2) If this is not flexible enough for your workflow, you could explore using a custom training loop. This is a more advanced, but more flexible workflow, which permits a greater range of network architectures than "trainNetwork". To get started, you can refer to this documentation page and the examples linked

https://mathworks.com/help/deeplearning/deep-learning-custom-training-loops.html

I hope that one of these workarounds is suitable for your workflow.

Best regards,

David