## MMSE Forecasting Regression Models with ARIMA Errors

### What Are MMSE Forecasts?

An objective of time series analysis is generating forecasts for responses over a future time horizon. That is, you can generate predictions for yT + 1,yT + 2,...,yT + h given the following:

• An observed series y1, y2,...,yT

• A forecast horizon h

• Nonstochastic predictors x1,x2,...,xT,...,xT + h, where xk is an r-vector containing the measurements of r predictors observed at time k

• A regression model with ARIMA errors

`$\begin{array}{l}{y}_{t}=c+{X}_{t}\beta +{u}_{t}\\ Η\left(L\right){u}_{t}=Ν\left(L\right){\epsilon }_{t},\end{array}$`

where H(L) and N(L) are compound autoregressive and moving average lag operator polynomials (possibly containing integration), respectively.

Let ${\stackrel{^}{y}}_{t+1}$ denote a forecast for the process at time t + 1, conditional on the history of the process up to time t (Ht), and assume that the predictors are fixed. The minimum mean square error (MMSE) forecast is the forecast ${\stackrel{^}{y}}_{t+1}$ that minimizes expected square loss,

`$E{\left({y}_{t+1}-{\stackrel{^}{y}}_{t+1}|{H}_{t}\right)}^{2}.$`

Minimizing this loss function yields the MMSE forecast,

`${\stackrel{^}{y}}_{t+1}=E\left({y}_{t+1}|{H}_{t}\right).$`

### How forecast Generates MMSE Forecasts

`forecast` generates MMSE forecasts recursively. When you call `forecast`, you must specify a `regARIMA` model (`Mdl`) and the forecast horizon. You can also specify presample observations (`Y0`), predictors (`X0`), innovations (`E0`), and conditional disturbances (`U0`) using name-value pair arguments.

To begin forecasting yt starting at time T + 1, use the last few observations of yt and Xt as presample responses and predictors to initialize the forecast. Alternatively, you can specify presample unconditional disturbances or innovations.

However, when you specify presample data:

• If you provide presample predictor data (`X0`), then you must also provide predictor forecasts (`XF`). It is best practice to set `X0` to the same predictor matrix that estimates the parameters. If you do not provide presample and future predictors, then `forecast` ignores the regression component in the model.

• If the error process in `Mdl` contains a seasonal or nonseasonal autoregressive component, or seasonal or nonseasonal integration, then `forecast` requires a minimum of P presample unconditional disturbances to initialize the forecast. The property `P` of `Mdl` stores P.

• If the error process in `Mdl` contains a seasonal or nonseasonal moving average component, then `forecast` requires a minimum of Q presample innovations to initialize the forecast. The property `Q` of `Mdl` stores Q.

• If you provide a sufficient amount of presample unconditional disturbances, then `forecast` ignores `Y0` and `X0`. If you also do not provide `E0`, but provide enough presample unconditional disturbances, then `forecast` infers the required amount of presample innovations from the ARIMA error model and `U0`.

• If you provide a sufficient amount of presample responses and predictors (and do not provide `U0`), then `forecast` uses the regression model to infer the presample unconditional disturbances.

• If you do not provide presample observations, then `forecast` sets the required amount of presample unconditional disturbances and innovations to 0.

• If you provide an insufficient amount of presample observations, then `forecast` returns an error.

Consider generating forecasts from a regression model with ARMA(3,2) errors:

`$\begin{array}{c}{y}_{t}=c+{X}_{t}\beta +{u}_{t}\\ \left(1-{a}_{1}L-{a}_{2}{L}^{2}-{a}_{3}{L}^{3}\right){u}_{t}=\left(1+{b}_{1}L+{b}_{2}{L}^{2}\right){\epsilon }_{t}\\ \text{or}\\ a\left(L\right){u}_{t}=b\left(L\right){\epsilon }_{t},\end{array}$`

where a(L) and B(L) are lag operator polynomials. The largest AR lag is 3, the largest MA lag is 2. This model does not contain any seasonal lags nor integration. Therefore, P = 3 and Q = 2. To forecast this model, you need three presample responses and predictors, or three presample unconditional disturbances, and two presample innovations.

Given presample unconditional disturbances $\left({u}_{T-2},{u}_{T-1},{u}_{T}\right),$ presample innovations $\left({\epsilon }_{T-1},{\epsilon }_{T}\right),$ and future predictors $\left({X}_{T+1},{X}_{T+2},...\right),$ you can forecast the model as follows:

• $\begin{array}{l}{\stackrel{^}{u}}_{T+1}={a}_{1}{u}_{T}+{a}_{2}{u}_{T-1}+{a}_{3}{u}_{T-2}+{b}_{1}{\epsilon }_{T}+{b}_{2}{\epsilon }_{T-1}\\ {\stackrel{^}{y}}_{T+1}=c+{X}_{T+1}\beta +{\stackrel{^}{u}}_{T+1}.\end{array}$

• $\begin{array}{l}{\stackrel{^}{u}}_{T+2}={a}_{1}{\stackrel{^}{u}}_{T+1}+{a}_{2}{u}_{T}+{a}_{3}{u}_{T-1}+{b}_{2}{\epsilon }_{T}\\ {\stackrel{^}{y}}_{T+2}=c+{X}_{T+2}\beta +{\stackrel{^}{u}}_{T+2}.\end{array}$

• $\begin{array}{l}{\stackrel{^}{u}}_{T+3}={a}_{1}{\stackrel{^}{u}}_{T+2}+{a}_{2}{\stackrel{^}{u}}_{T+1}+{a}_{3}{u}_{T}\\ {\stackrel{^}{y}}_{T+3}=c+{X}_{T+3}\beta +{\stackrel{^}{u}}_{T+3}.\end{array}$

...

Note that:

• Future innovations take on their unconditional mean, 0.

• For stationary error processes, such as this one:

• The forecasted unconditional disturbances converge to their unconditional mean,

`$E\left({u}_{t}\right)=\frac{b\left(L\right)}{a\left(L\right)}E\left({\epsilon }_{t}\right)=0.$`

• c + Xtβ governs the long-term behavior of the forecasted responses.

### Forecast Error

The forecast error for an s-step ahead forecast of a regression model with ARIMA errors is

`$\begin{array}{c}\text{MSE}=E{\left({y}_{T+s}-{\stackrel{^}{y}}_{T+s}|{H}_{T+s-1}\right)}^{2}\\ =E{\left(c+{X}_{T+s}\beta +{u}_{T+s}-c-{X}_{t+s}\beta -{\stackrel{^}{u}}_{T+s}|{H}_{T+s-1}\right)}^{2}\\ =E{\left({u}_{T+s}-{\stackrel{^}{u}}_{T+s}|{H}_{T+s-1}\right)}^{2}\\ =\frac{Ν\left(L\right)}{Η\left(L\right)}E\left({\epsilon }_{t}{}^{2}|{H}_{T+s-1}\right)\\ =\psi \left(L\right){\sigma }^{2},\end{array}$`

where the dividend ψ(L) is an infinite lag operator polynomial, and σ2 is the innovation variance.

If the error process is stationary, then the coefficients of ψ(L) are absolutely summable. Therefore, the MSE (mean square error) converges to the unconditional variance of the process [1].

If the error process is not stationary, then the MSE grows with increasing s.

## References

[1] Box, G. E. P., G. M. Jenkins, and G. C. Reinsel. Time Series Analysis: Forecasting and Control. 3rd ed. Englewood Cliffs, NJ: Prentice Hall, 1994.