example of formatting data etc. for trainNetwork where I have a colleciton of images and I want to train to generate a single image

I'm tryinjg to build a network that will learn how to generate a single image from a collection of images. For example, I have sets of 200 x 100 pixel images (grey scale) each collected into sets of ten images and I want the network to learn how to generate a single image using those 10 images as input over multiple set.

set 1, 10 images, 200 x 100 pixels, desired image 1

set 2, 10 images, 200 x 100 pixels, desired image 2

set 3.......

I have the data sorted out, but I don't know how to format it to satisfy the trainNetwork funtion. I've been trying various permuation of image stacks, but haven't gotten this to work. My code is below. The data is a collection of images and the "truth" data is a single image for each collection of input images.

% format the training evaluation data

formatted_train_evaluation = reshape(train_evaluation, [[] [] [] number_of_train_observations 1] );

formatted_train_evaluation = categorical(formatted_train_evaluation);

% format the test data

% each row is an single observation of data points

[number_of_test_observations, images_in_stack, rows, cols] = size(test_data);

% format the test data to height, width, data depth, number of

% rearrange the data the training data to height, width, number of frames, number

% of data sets

formatted_test_data = permute(test_data, [3, 4, 2, 1]);

% format the test data to height, width, number of frames, color

% depth, number of data sets

formatted_test_data = reshape(test_data, [rows, cols, images_in_stack 1 number_of_test_observations]);

% format the test evaluation data

formatted_test_evaluation = reshape(test_evaluation, [[] [] [] number_of_test_observations 1] );

formatted_test_evaluation = categorical(formatted_test_evaluation);

layers = [

image3dInputLayer([rows cols images_in_stack 1])

convolution3dLayer([5 5 5], 25, 'Stride', [1 1 1], 'Padding','same')

batchNormalizationLayer

tanhLayer

convolution3dLayer([5 5 5], 25, Stride = [1 1 1], Padding = 'same')

batchNormalizationLayer

tanhLayer

fullyConnectedLayer(20, name= "fully connected layer 1")

tanhLayer(name="tanh layer 4")

% last layer is number of possible outcomes

fullyConnectedLayer(2)

softmaxLayer

classificationLayer

];

options = trainingOptions('sgdm', ...

'InitialLearnRate',0.001, ...

'MaxEpochs', 10, ...

'MiniBatchSize', 20, ...

'ValidationData',{formatted_test_data, formatted_test_evaluation}, ...

'ValidationFrequency',30, ...

'Shuffle','every-epoch', ...

'Verbose',false, ...

'Plots','training-progress');

[net, info] = trainNetwork(formatted_train_data, formatted_train_evaluation, layers, options);

Answer 1

[arushi]

Hi JG,

I understand that you would like to format the data in a way that each set of input images results in a single output image.

You can format the input and output data structure of the image as follows: -

For the Input Image:

1. Convert each set of 10 images into a 4D array.
2. Concatenate the 4D arrays of all sets

For the Output Image:

1. Convert each desired image into a 3D array.
2. Concatenate the 3D arrays of all desired images into a single array.

By organizing the input and output data in the above format, you will be able to train the model in such a way that it considers one image as the result for every 10 images.

For additional information please refer to the following documentation:

https://www.mathworks.com/help/deeplearning/ref/trainnetwork.html

I hope this helps!