Regarding Multi-label transfer learning with googlenet

I have a dataset with pictures with presence of objects of different classes. I want to perform a multilabel classification, which means I need to classify the pictures into different classes with the picture belonging to more than one class at the same time. That is, for pictures with objects of type A and type B, the net should output both the labels A and B.

If I am designing a CNN for this from scratch, I will have a sigmoid activation at the last layer. The number of output neurons will be equal to the number of classes with the output of each neuron giving 1 if the picture belongs to the particular class or 0 if not. However, there seems to be no provision for adding a sigmoid function and the Image datastore cannot hold binary vectors as the label. How do I overcome this?

[SC P]

@Balakrishnan Rajan ,how you have resolved this problem? ( how you did this?:defining classes which are unique combination of the previous class occurences). Is there any code of it

Answer 1

[Shounak Mitra]

We do not support sigmoid activation. You can use the softmax activation function. You don't need to define the neurons in the softmaxLayer. Define the no of neurons (= no of classes) you want in the fullyConnectedLayer. So, your network structure would be like:

inputLayer -- -- fullyConnectedLayer softmaxLayer ClassificationLayer

HTH Shounak

[kira]

but softmaxLayer make the sum of all probabilities to be 1 (and therefere mutually exclusive classes), with a "logsigLayer", every class could take probability 1 (and therefore not mutually exclusive classes). Is there any workaround to this?

[o.cefet cefet]

Hello all?

As the images do not have a single class, how can I build the ImageDataStore, because the images cannot be separated by folders, that is, I cannot endow "Labels" with "Folders".

The images are in the same folder and a CSV file destines them. Like this:

Image, Class A, Class B, Class C, Class D
00000001_000.png, 1,1,0,0
00000001_001.png, 1,1,0,1
00000001_002.png, 0,0,0,1
00000002_000.png, 0,0,0,0
00000003_000.png, 0,0,1, -1
00000003_001.png, 0, -1,0,1
00000003_002.png, 1,0,0,0
00000003_003.png, 0,0,0,1
00000003_004.png, 0,1,0,0
00000003_005.png, 0,0,1,0
00000003_006.png, 0,1,1,0
00000003_007.png, 1,0,0,1
00000004_000.png, 0,0,1,0
00000005_000.png, 0,1,0,0
00000005_001.png, -1, -1,1,0
00000005_002.png, 0.1, -1.0
00000005_003.png, 0,0,0,1
00000005_004.png, 0,0,1,0
00000005_005.png, 0,1,0,0
00000005_006.png, 0,0, -1,1
00000005_007.png, 0,1,0, -1
00000006_000.png, 0,0,0,1
00000007_000.png, 0,1,0,0
00000008_000.png, 0,0,1,0
00000008_001.png, 0,0,0,1

......

......

......

......

[SC P]

@Kira, Have you found any solution of it? if yes please let me know. Thanks

Answer 2

[Antonio Quvera]

I'm also interested in this application (i.e. multi-label classification using CNN/LSTM). Any news? Does the latest deep learning toolbox resolve this issue?

Answer 3

[cui,xingxing]

Can I define multiple softmaxLayer at the end of the network? Each softmaxLayer is independent of each other, and each layer is used to classify a label so that there can be multiple loss functions, shared by the previous convolutional layer? But how do you enter the network goals?

[Tarily]

Do you have solven it? if yes please let me know. Thanks:)

Answer 4

[Greg Heath]

Decades old solution:

Divide each output by the sum to obtain the relative probability of each class

Hope this helps.

Thank you for formally accepting my answer

Greg

[kira]

but the sum is 1, so ...

[Balakrishnan Rajan]

I had resolved it by defining classes which are unique combination of the previous class occurences. That way now they are all mutually exclusive.

[Greg Heath]

To Kira:

My point was:

If you do not use softmax, the sum is not constrained to be 1 !

Greg

[o.cefet cefet]

Hello all?

As the images do not have a single class, how can I build the ImageDataStore, because the images cannot be separated by folders, that is, I cannot endow "Labels" with "Folders".

The images are in the same folder and a CSV file destines them. Like this:

Image, Class A, Class B, Class C, Class D
00000001_000.png, 1,1,0,0
00000001_001.png, 1,1,0,1
00000001_002.png, 0,0,0,1
00000002_000.png, 0,0,0,0
00000003_000.png, 0,0,1, -1
00000003_001.png, 0, -1,0,1
00000003_002.png, 1,0,0,0
00000003_003.png, 0,0,0,1
00000003_004.png, 0,1,0,0
00000003_005.png, 0,0,1,0
00000003_006.png, 0,1,1,0
00000003_007.png, 1,0,0,1
00000004_000.png, 0,0,1,0
00000005_000.png, 0,1,0,0
00000005_001.png, -1, -1,1,0
00000005_002.png, 0.1, -1.0
00000005_003.png, 0,0,0,1
00000005_004.png, 0,0,1,0
00000005_005.png, 0,1,0,0
00000005_006.png, 0,0, -1,1
00000005_007.png, 0,1,0, -1
00000006_000.png, 0,0,0,1
00000007_000.png, 0,1,0,0
00000008_000.png, 0,0,1,0
00000008_001.png, 0,0,0,1

......

......

......

......