unconditionalDE

Unconditional Du-Escanciano (DE) expected shortfall (ES) backtest

Description

example

TestResults = unconditionalDE(ebtde) runs the unconditional Du-Escanciano (DE) expected shortfall (ES) backtest [1] . The unconditional test supports critical values by large-scale approximation and by finite-sample simulation.

example

[TestResults,SimTestStatistic] = unconditionalDE(___,Name,Value) specifies options using one or more name-value pair arguments in addition to the input argument in the previous syntax.

Examples

collapse all

Create an esbacktestbyde object for a t model with 10 degrees of freedom, and then run an unconditionalDE test.

load ESBacktestDistributionData.mat
    rng('default'); % For reproducibility
    ebtde = esbacktestbyde(Returns,"t",...
       'DegreesOfFreedom',T10DoF,...
       'Location',T10Location,...
       'Scale',T10Scale,...
       'PortfolioID',"S&P",...
       'VaRID',["t(10) 95%","t(10) 97.5%","t(10) 99%"],...
       'VaRLevel',VaRLevel);
    unconditionalDE(ebtde)
ans=3×14 table
    PortfolioID        VaRID        VaRLevel    UnconditionalDE     PValue     TestStatistic     LowerCI      UpperCI     Observations    CriticalValueMethod    MeanLS      StdLS      Scenarios    TestLevel
    ___________    _____________    ________    _______________    ________    _____________    _________    _________    ____________    ___________________    ______    _________    _________    _________

       "S&P"       "t(10) 95%"        0.95          accept            0.181       0.028821       0.019401     0.030599        1966          "large-sample"        0.025    0.0028565       NaN         0.95   
       "S&P"       "t(10) 97.5%"     0.975          accept         0.086278       0.015998      0.0085028     0.016497        1966          "large-sample"       0.0125    0.0020394       NaN         0.95   
       "S&P"       "t(10) 99%"        0.99          reject         0.016871      0.0080997      0.0024575    0.0075425        1966          "large-sample"        0.005    0.0012972       NaN         0.95   

Input Arguments

collapse all

esbacktestbyde (ebtde) object, which contains a copy of the data (the PortfolioData, VarData, and ESData properties) and all combinations of portfolio ID, VaR ID, and VaR levels to be tested. For more information on creating an esbacktestbyde object, see esbacktestbyde.

Name-Value Pair Arguments

Specify optional comma-separated pairs of Name,Value arguments. Name is the argument name and Value is the corresponding value. Name must appear inside quotes. You can specify several name and value pair arguments in any order as Name1,Value1,...,NameN,ValueN.

Example: TestResults = unconditionalDE(ebtde,'CriticalValueMethod','large-sample','TestLevel',0.99)

Method to compute critical values, confidence intervals, and p-values, specified as the comma-separated pair consisting of 'CriticalValueMethod' and a character vector or string with a value of 'large-sample' or 'simulation'.

Data Types: char | string

Test confidence level, specified as the comma-separated pair consisting of 'TestLevel' and a numeric value between 0 and 1.

Data Types: double

Output Arguments

collapse all

Results, returned as a table where the rows correspond to all combinations of portfolio ID, VaR ID, and VaR levels to be tested. The columns correspond to the following:

  • 'PortfolioID' — Portfolio ID for the given data

  • 'VaRID' — VaR ID for each of the VaR levels

  • 'VaRLevel' — VaR level

  • 'UnconditionalDE'— Categorical array with the categories 'accept' and 'reject', which indicate the result of the unconditional DE test

  • 'PValue'P-value of the unconditional DE test

  • 'TestStatistic'— Unconditional DE test statistic

  • 'LowerCI'— Confidence-interval lower limit for the unconditional DE test statistic

  • 'UpperCI'— Confidence-interval upper limit for the unconditional DE test statistic

  • 'Observations'— Number of observations

  • 'CriticalValueMethod'— Method for computing confidence intervals and p-values

  • 'MeanLS'— Mean of the large-sample normal distribution; if CriticalValueMethod is 'simulation', 'MeanLS' is reported as NaN

  • 'StdLS'— Standard deviation of the large-sample normal distribution; if CriticalValueMethod is 'simulation', 'StdLS' is reported as NaN

  • 'Scenarios'— Number of scenarios simulated to get the p-values; if CriticalValueMethod is 'large-sample', the number of scenarios is reported as NaN

  • 'TestLevel'— Test confidence level

Note

For the test results, the terms accept and reject are used for convenience. Technically, a test does not accept a model; rather, a test fails to reject it.

Simulated values of the test statistics, returned as a NumVaRs-by-NumScenarios numeric array.

More About

collapse all

Unconditional DE Test

The unconditional DE test is a two-sided test to check if the test statistic is close to an expected value of ɑ/2, where ɑ = 1- VaRLevel.

The test statistic for the unconditional DE test is

UES=1Nt=1NHt

where

  • Ht is the cumulative failures or violations process; Ht = (α - Ut)I(Ut < α) / α, where I(x) is the indicator function.

  • Ut are the ranks or mapped returns Ut = Pt(Xt), where Pt(Xt) = P(Xt | θt) is the cumulative distribution of the portfolio outcomes or returns Xt over a given test window t = 1,...N and θt are the parameters of the distribution. For simplicity, the subindex t is both the return and the parameters, understanding that the parameters are those used on date t, even though those parameters are estimated on the previous date t-1, or even prior to that.

Significance of the Test

The test statistic UES is a random variable and a function of random return sequences:

UES=UES(X1,...,XN).

For returns observed in the test window 1,…,N, the test statistic attains a fixed value:

UESobs=UES(X1obs,...,XNobs).

In general, for unknown returns that follow a distribution of Pt, the value of UES is uncertain and follows a cumulative distribution function:

PU(x)=P[UESx].

This distribution function computes a confidence interval and a p-value. To determine the distribution PU, the esbacktestbyde class supports the large-sample approximation and simulation methods. You can specify one of these methods by using the optional name-value pair argument CriticalValueMethod.

For the large-sample approximation method, the distribution PU is derived from an asymptotic analysis. If the number of observations N is large, the test statistic UES is distributed as

UESdistN(α2,α(1/3α/4)N)=PU

where N(μ,σ2) is the normal distribution with mean μ and variance σ2.

Because the test statistic cannot be smaller than 0 or greater than 1, the analytical confidence interval limits are clipped to the interval [0,1]. Therefore, if the analytical value is negative, the test statistic is reset to 0, and if the analytical value is greater than 1, it is reset to 1.

The p-value is

pvalue=2min{PU(UESobs),1PU(UESobs)}.

The test rejects if pvalue < αtest.

For the simulation method, the distribution PUis estimated as follows

  1. Simulate M scenarios of returns as

    Xs=(X1s,...,XNs), s=1,...,M.

  2. Compute the corresponding test statistic as

    UESs=UESs(X1s,...,XNs), s=1,...,M.

  3. Define PU as the empirical distribution of the simulated test statistic values as

    PU=P[UESx]=1MI(UESsx),

    where I(.) is the indicator function.

In practice, simulating ranks is more efficient than simulating returns and then transforming the returns into ranks. For more information, see simulate.

For the empirical distribution, the value of 1-PU(x) can differ from the value of P[UESx] because the distribution may have nontrivial jumps (simulated tied values). Use the latter probability for the estimation of confidence levels and p-values.

If ɑtest = 1 - test confidence level, then the confidence intervals levels CIlower and CIupper are the values that satisfy equations:

PU(CIlower)=P[CIlowerUES]=αtest2,P[UESCIupper]=αtest2.

The reported confidence interval limits CIlower and CIupper are simulated test statistic values UsES that approximately solve the preceding equations.

The p-value is determined as

pvalue=2min{P[UESUESobs],P[UESUESobs]}.

The test rejects if pvalue < αtest.

References

[1] Du, Z., and J. C. Escanciano. "Backtesting Expected Shortfall: Accounting for Tail Risk." Management Science. Vol. 63, Issue 4, April 2017.

[2] Basel Committee on Banking Supervision. "Minimum Capital Requirements for Market Risk". January 2016 (https://www.bis.org/bcbs/publ/d352.pdf).

Introduced in R2019b