Gaussian Process Regression with input-dependent noise (Sigma)

I have sampled data that I want to fit using a Gaussian process regression model. The data has variable noise, that depends on the inputs (predictors), i.e. it is different at different sampling locations. The noise is known, i.e. it is fixed and not a fit parameter.

However, in the fitrgp function, it appears that I can only set constant noise (called 'Sigma'), as a single scalar value for all inputs. I suppose that I can incorporate the input-dependent noise in the covariance kernels, but that means that I have to provide a custom covariance function for which the gradients are not available in analytic form, thus impacting performance.

Is there a way to set input-dependent noise, while still using the analytic gradients?

Answer 1

[Gautam Pendse]

Hi Rick,

Incorporating input-dependent noise in a custom covariance kernel is a sensible approach. For a custom kernel, gradients of kernel parameters are calculated by finite differences. While this can be slower than analytic gradients, you could consider using 'FitMethod','sd' along with an 'ActiveSetSize' and 'ActiveSetMethod' to speed things up. Using 'lbfgs' as the 'Optimizer' may also help.

Hope this helps,

Gautam

Answer 2

[Gergo Bohner]

Dear Gautam,

as a follow-up question to that, I've been working with custom implementations of Gaussian Processes for quite a while, and I was happy to see that Matlab decided to include a reasonably rich implementation of them.

However, I would very much like to efficiently use various custom kernels (mainly to model additive processes) and I was wondering if there was any way to supply analytic gradients and store parameters conveniently by subclassing classreg.learning.gputils.Kernel ?

The problem seems to be with this approach that the set of available built-in kernel classes are stored in a read-only file and thus adding new Kernels (even in the appropriate format) never gets recognized. Is the only way of doing that by creating a local copy of the whole +gputils folder as well as GPImpl, and then putting it higher in path after the required modifications? Also, is that something a Matlab licence allows for legally? (Not for a commercial implementation, purely academic)

Thanks!