Part 4: Model validation and back-testing

Model validation

Eurex Clearing commits itself to a regular and thorough validation of all risk models that are in place at the CCP. Throughout the year, the independent model validation uses a number of validation instruments to regularly validate the conceptual soundness of the frameworks and adequacy of the risk models.

The three most important risk models operated by Eurex Clearing are the initial margin model, the collateral haircut model and the stresstesting model. Throughout the year, the independent model validation uses a number of validation instruments to regularly validate the conceptual soundness of the frameworks and adequacy of these risk models. At the end of the year an annual comprehensive validation is performed along the model landscape. In the annual validation report, all validation results obtained throughout the year are summarized and provided to the internal management as well as the regulators. This yields an overall review of model performance and appropriateness. The report also assesses the effectiveness of processes and procedures relevant for managing model risks.

Since Q4 2016, Eurex Clearing started to quarterly disclose a summary of margin and collateral validation results as part of the CPMI-IOSCO quantitative disclosure. One key aspect is to test those models performances with respect to the target level of confidence (see 6_4_5 for the initial margin model and 5_3_1 for collateral haircut model), which is primarily checked by backtesting.

Backtesting compares the model forecast with actual observed outcome and keeps track of occasions where the observed outcome exceeded the forecast. This procedure runs on a daily basis, meaning that each day one new observation of (forecast, observed outcome) is added to the history. At Eurex Clearing model validation uses a 3-year history for both, collateral haircut backtesting (see 5_3_4) and margin backtesting.

• For the initial margin model, the model forecast corresponds to the initial margin of an individual portfolio and the observed outcome would be the respective realized clean PnL for that portfolio. Note that the term portfolio respect the segregation logic and corresponds to ‘the collection of positions in products of the same liquidation group within one account’, rather than the whole account or all positions under the same Clearing Member.

• For the collateral haircut model, the model forecast corresponds to the collateral haircut of an individual security/ISIN and the observed outcome to the realised PnL for the respective security/ISIN. This is again opposed to a more aggregated view, which in that case would be collateral pool level.

“Model validation checks the confidence levels for individual portfolios and ISINs as well as aggregates of these results. The achieved confidence levels confirm the conceptual soundness and adequacy of our risk frameworks – which we continuously validate”.

The calculation of the aggregated achieved confidence level is explained below. Figure 1 shows the achieved confidence levels of the margin model (see 6_5_3) on an aggregated basis for all major asset classes. All are well above the target levels of confidence of 99% for listed equity and fixed income derivatives and 99.5% for OTC interest rate swaps respectively.

The calculation of the aggregated achieved confidence level follows the CPMI-IOSCO guidance, that is 

For instance, the achieved confidence level for equity derivatives of 99.68 results from dividing the number of outliers across all equity derivatives portfolios by the number of observations collected for all equity derivatives and subtracting it from 1.

Backtesting outlier materiality analysis
The backtesting analysis includes another aspect: the assessment of the size and therefore materiality of individual outliers. The largest observed outlier (see 6_5_4) for the initial margin model was observed before 2016 and therefore remained constant in the 2016 and 2017 reporting. All individual outliers and combined two largest outliers (see 4_4_7) would have been fully covered by Eurex Clearing’s own contribution to the lines of defence and the default fund. 

Key points to consider while interpreting the backtesting figures for margin and collateral haircut models: 

• Margin backtesting runs on portfolio level, which is the same granular level on which the initial margin is calculated. The term portfolio refers to all products belonging to the same liquidation group within one account. The reported figures in section 6_5 are aggregates of the numbers on portfolio level.  Consequently, one observation in backtesting corresponds to a combined pair of margin and PnL for one day and one portfolio. The backtesting figures from the recent demonstrate that the target confidence level was achieved. 

• Haircut backtesting runs on individual-asset level rather than collateral-pool level. This is in line with the CPMI-IOSCO guidance, which requires a reporting of figures based on asset level results. Therefore, the number of observations and respectively outliers is driven by the number of assets eligible for collateral. The magnitudes of the results are therefore higher than if reports would have been prepared on collateral pool level. The CPMI-IOSCO quantitative disclosure file does not include achieved confidence level for individual assets which are eligible for collateral, nor does it include aggregated achieved confidence levels for collateral types. As of 31/12/2017 the aggregated and individual asset haircut backtesting figures demonstrate that the target confidence level was achieved.

• The number of outliers on individual asset level (see 5_3_4) is explained by a large number of individual assets tested.