Is there a way to extract partial derivatives of specific layers in deep learning toolbox?
1 view (last 30 days)
I asked this question last year, in which I would like to know if it is possible to extract partial derivatives involved in back propagation, for the parameters of layer so that I can use for other purpose. At that time, the latest MATLAB version is 2019b, and I was told in the above post that it is only possible when the final output y is a scalar, while my desired y can be a vector, matrix or even a tensor (e.g. reconstruction tasks).
Now, is it possible to extract the partial derivatives of layer in 2020b? Thanks.
Amanpreetsingh Arora on 12 Nov 2020
As mentioned in the answer to the question referred by you, the only way to find partial derivatives of a tensor is by looping over elements and calling "dlgradient" as "dlgradient" only supports scalar input for auto differentiation. However, I understand your concern that this will waste time recomputing overlapping traces. In order to improve time performance, you can call "dlgradient" with 'RetainData' parameter set to 'true' which will retain the intermediate computation of gradient and any subsequent calls to the function will reuse these intermediate computations. Refer to the following documentation for more information on this.