# Verify Predictive Ability Robustness of a regARIMA Model

This example shows how to forecast a regression model with ARIMA errors, and how to check the model predictability robustness.

Load the Credit Defaults data set, assign the response (`IGD`) to `y` and the predictors `AGE`, `CPF`, and `SPR` to `X`. For illustration, specify that the response series is a regression model with AR(1) errors. To avoid distraction from the purpose of this example, assume that all predictor series are stationary.

```load Data_CreditDefaults y = Data(:,5); X = Data(:,[1 3:4]); T = size(X,1); % Sample size Mdl = regARIMA(1,0,0);```

Vary the validation sample size (`m`), and forecast responses from `Mdl` recursively. That is, for each validation sample size:

1. Fit the model to the data (`EstMdlY`).

2. Forecast responses from the estimated model (`yF`).

3. Compute the two performance statistics, root mean square error (RMSE) and root prediction mean square error (RPMSE).

```m = 4:10; % Validation sample lengths rPMSE = m; % Preallocate rPMSE rMSE = m; % Preallocate rMSE for k = 1:numel(m); yEst = y(1:(T-m(k))); % Response data for estimation yVal = y((T-m(k)+1):T); % Validation sample EstMdlY = estimate(Mdl,yEst,'X',X,'display','off'); yHat = EstMdlY.Intercept + X(1:(T-m(k)),:)*EstMdlY.Beta';... % Estimation sample predicted values [e0,u0] = infer(EstMdlY,yEst,'X',X); yF = forecast(EstMdlY,m(k),'Y0',yEst,... 'X0',X(1:T-m(k),:),'XF',X((T-m(k)+1):T,:));... % Validation sample predicted values rMSE(k) = sqrt(mean((yEst - yHat).^2)); rPMSE(k) = sqrt(mean((yF - yVal).^2)); end```

`rMSE` and `rPMSE` are vectors that contain the RMSE and RPMSE, respectively, for each validation sample.

Display the performance measures.

`fprintf('\n m | rMSE | rPMSE\n')`
``` m | rMSE | rPMSE ```
`fprintf('====================\n')`
```==================== ```
```for k = 1:length(m) fprintf('%2d | %0.4f | %0.4f\n',m(k),rMSE(k),rPMSE(k)) end```
``` 4 | 0.0947 | 0.2274 5 | 0.0808 | 0.1902 6 | 0.0810 | 0.2036 7 | 0.0714 | 0.1924 8 | 0.0809 | 0.1532 9 | 0.0720 | 0.1557 10 | 0.0899 | 0.1300 ```

The predictive ability of this model is fairly robust because `rPMSE` changes slightly for increasing `m`. However, `rMSE` is less than `rPMSE` for all `m`. This signifies poor predictive ability.

Search for a better model by specifying, e.g., more AR or MA lags in the error model, and compare the PMSEs over these models. Choose the model with the lowest PMSE for a given validation sample size.