Deep learning layer with custom backward() function

I need to implement a complicated function (that computes a regularizing penalty of a deep learning model) of which I will then take the gradient with respect to the weights of the model to optimize them.

One operation within this "complicated function" is not currently supported for automatic differentiation (determinant of a matrix). I'm using the option (mentioned here) of implementing the operation as a custom layer, of which I can define the derivative myself with a custom backward() function. It works, but:

1. In order to evaluate this custom layer, do I need to encapsulate this layer in a dlgraph/dlnetwork (a 2-layer network, since it also needs an input layer) so that I can then calling .predict() on this network ? Is there anything simpler, without having to build a network ? It would be nice if one could call something like the following, and the underlying gradient trace would be built to go through my custom backward function: y = myLayer.predict(x);
2. I am using the automatic differentiation for second-order derivatives available in the R2021a prelease. Does this support layers with custom backward() function ?

Answer 1

[Jack Xiao]

dlnetwork is for customed training loop and dlfeval is for customed model gradients using defined gradient funtction.

you can refer the demo for gan：

https://www.mathworks.com/help/deeplearning/ug/train-generative-adversarial-network.html?s_tid=srchtitle

[Damien T]

I don't think this answers either of the original questions ?!