Alternative to softmax function for Neural Network predicting fractions of a whole

2 views (last 30 days)
Hi, I created a feed forward Regression Neural Network to predict variables which are fractions of a whole (i.e. they sum up to 1). In order to have the network fullfil this criterion perfectly, I am using the softmax transfer function. Unfortunately, I realize that the network predicts smaller fractions very poorly, and I think this is due to the fact that the softmax transfer function normalizes my target fractions by dividing exponent of the fractions minus the largest fraction by its sum (exp(n-nmax)/sum(exp(n-nmax))), which results in much larger values for very small fractions. It wouldn't have to do that, since my fractions are already between 0 and 1. Can I change that somehow in the softmax transfer function, or is there an alternative to it that doesn't do this normailzation?
  6 Comments
F S
F S on 28 Aug 2019
It is not multinomial logistic regression in a classical sence, as there are no classes to be predicted, although it is mathematically equal to it as far as I can judge. It is fractions of constituents of a mass, of which I know the weight and weight distribution, and I now the density of the constituents. So now I want to predict how much of which constituent I have, knowing that their weights ALWAYS sum up to the total weight. It can't be physically solved unfortunately, because there is several combinations that can lead to the same total weight, but the weight distribution holds information that can help solve the problem.
If there is a neural network or other method that can solve such a problem other than softmax, I would be happy if somebody can point that out to me.
I am also happy to send you more information in a private email, containing some data that I cannot disclose publicly. My licence and support subscription are up to date.
Ajay Pattassery
Ajay Pattassery on 29 Aug 2019
Did you tried creating a custom layer which can force your output to one like the one I mentioned above instead of having a softmax layer.
Please refer the following link for creating custom layers.

Sign in to comment.

Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!