How can I create a convolutional neural network that first classifies, and if a certain classification is met also performs regression.

Hi!

So I am a bit stuck.

I have images containing particles, and images containing no particles.

I want to create a neural network so that it can observe if there is a particle in an image, and if so also predicts its coordinates.

My questions are:

Can this be done in a single convolutional neural network? With perhaps a regression output layer áfter a classification output layer?

Also, do I give coordinates for the images that do not contain a particle, if not, won't the neural network ''crash'' if it predicts a particle to be in an empty image and calculates the coordinates and there is no ground truth available? How do I adjust for that?

Help would be appreciated!

Answer 1

[Avadhoot]

Hi Kevin,

I understand that you would like to use the same neural network for classification and regression tasks.

Regarding your first question, though it is convenient to create such a network which performs conditional regression step, it is not feasible to do so as the model would be trained to perform classification and not regression. Simply adding a regression layer after classification won’t work as the layer weights would also need to be different for classification and regression tasks.

It is advisable to first train your model for classification of images and then train a separate model for regression. After that you can use a conditional statement to decide when the regression model should be used.

Regarding your second question, you need to provide the coordinates for all the images even if some images do not contain particles. The reason for this is however well you train the model it is bound to have some error and thus can classify an image not containing particles as an image containing particles. To account for this, you could set the coordinates of the images not containing a particle to infinity. This would ensure that the model does not break. Also, it would generate a huge error term in the regression model from which you could flag the error. Just ensure that you do not train the regression model with such samples as it will affect the layer weights adversely.

I hope this helps in your task.

Regards,

Avadhoot.