I have a question regarding batche gardient descent, stochastic gradient descent and mini batches gradient descent

I have a non-linear equation, with 8 variables. the goal obviously is to to find the right variable values. My data set is a vector sized (1,124).

To optimize, I apply the gradient descent (so far I do not know with one is suitable for my case).

To do so, I avoided the non-linearity by applying the taylor expansion to my cost function. Indeed it gave me a linear equation.

Still, I do not know with gradient descent to apply.

my code is devided as follows:

• The first .m (MiniBatches_SGD_Trial) file to calculate the cost function, its derivatives and sum up everything (in case of batch gradient descent).
• The second .m (MiniBatches_SGD_Trial2) file is to run the gradient descent algorithm.

The unusual things are:

• My cose function does not decrease
• I found a hard time to choose the initial guess

I update my two .m files.

Please could you assist me at what exactly is relatively wrong with my code.

Thank you very much in advance