classify time related images using deep learning

Hi

I'm about to classify plasma images in tokamak using deep learning CNN. But if I use each image file to train network, I would loss my time information of images that links to other images.

I want to classify authomatically by shot-numbers located at one subfolder : the plasma turns on and turns off during one shot, so that I wish which conditions of plasma(shot-number) is similar with others using classification. I already know some physicsal features in plasma but I expect more information that AI finds from training and classification.

Is it possible? and what type model I should use? Could you give me example code for that?

Thank you so much

[Image Analyst]

How many classes do you want to specify? What are they? What is your ground truth information? Can you put all images with the same ground information into a folder named with that info? Like if the ground truth is the shot number, have one folder called "shot1" for all the 1 shot images, a folder called "shot2" for all the 2 shot images, etc.?

[Inhyeok]

https://www.mathworks.com/matlabcentral/answers/1996738-classify-time-related-images-using-deep-learning#comment_2817753

yes, For example, I have one folder called,"39024" for all the 39024 shot images that fast camera images at 0.4ms intervals during a plasma operation, and I have over 10000 different shot numbers. And I want to classify these shots to 5 or 6 classes depends on plasma occuring shape or shape of plasma contour just before it is disappered. Or with or withnot light bolb inside of plasma like small feature in the image including how fast shape of that changes. These pictures are occuring of plasma and disappearing of plasma in the one shot. I have around 20 pictures of growth plasma in one shot folder.

Also each image file's name is 39024_308.4_ms like that

How can I do that?

Thank you so much!

Answer 1

[Shreeya]

Hello Inhyeok

It appears that the problem statement requires the classification of images, which are time related within the folder. Recurrent Neural Networks (RNNs) or Long Short-Term Memory networks (LSTMs) can be used for this purpose.

Since the data across various folders is independent of each other, you can train the Neural Network on each folder separately. Find the example of an LSTM Network here.

Hope this helps.

Answer 2

[Milan Bansal]

Hello,

It looks like there are a number of shot folders with each folder containing around 20 images captured in sequence, and these folders are needed to be classified into 5-6 classes. Time information(or the sequence information) is being lost when training the model treating all the images from all the folder as independent.

First the images in each folder needs to be converted into sequential data so that the time information is maintained. This can be achieved by following steps in matlab.

1. Select the folder.
2. Flatten all the images in the folder to form a vector. (50x50 image is converted to 2500x1 vector)
3. Concatenate these vectors to form a Matrix. (2500 x 20 sized matrix is formed)

This Matrix is the sequential data with respect to the selected folder.

Repeat the same process to generate the matrices for all the folders.

folder = 'path_to_folder'; 
seqData = [] % sequence data for the folder 
% Get a list of files in the folder 

file_list = dir(fullfile(folder, '*.jpg'));  % Replace '*.jpg' with the desired file extension 
% Iterate through each file 
for i = 1:numel(file_list) 
    file_name = file_list(i).name; 
    file_path = fullfile(folder, file_name); 
    
    image = imread(file_path); 
    [rows, cols, channels] = size(image); 
    image_vector = reshape(image, rows * cols * channels, 1); % flattening the image 
    seqData = [seqData image_vector] % concatenating the flattend image. 
end 

The folder data is converted into sequence and Deep learning methods such as RNN (Recurrent Neural Networks) and LSTM (Long Short Term Memory) and can be employed to classify the sequence data. Additionally, Attention Layers can be used to further improve the performance.

Refer to the below documentation link for example code:

https://in.mathworks.com/help/deeplearning/ug/classify-sequence-data-using-lstm-networks.html?searchHighlight=sequence%20classification&s_tid=srchtitle_support_results_2_sequence%2520classification